Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

      • Recent Topics

      • Manage Task on Mobile

        How do we manage our task on mobile devices? It seems that there should be a standalone mobile app to handle the new task features. The new features released in regards to Task management are great by the way! Now we need to bring that all together in

      • Set Default Payment Method & Default account

        Hi, I would like to know how to set the default payment method and default bank account when recording payments in zoho books. At present we have to change these fields everytime we record a payment, which leads to potential error and as we have a very

      • Customer Portal on Zoho Desk

        Hi, I'd like to know more about the items below I found when setting up the Customer Portal on Zoho Desk. Could someone help me explaining these in details? Especially the 2nd and 3rd point. Thanking you in advance! Permissions Customers can sign up for Customer Portal Customers can view tickets of their organization (contacts) Customers must register to access Customer Portal Display Community in Customer Self Service portal

      • Slow Performance on desk.zoho.com. 11/3/2025

        I’m not seeing any active service alerts for desk.zoho.com, but everyone on our account is currently experiencing very slow load times when opening or navigating tickets. We’ve already tried the standard troubleshooting steps — clearing cache and cookies,

      • Books API Receiving an Error that Doesn't Make Sense when Creating Credit Note - trying to use 'ignore_auto_number_generation' argument

        Hello, I'm working on a newly created routine and I'm getting an error that doesn't make sense when trying to create a new Credit Note. Here is my POST request. Endpoint: https://www.zohoapis.com/books/v3/creditnotes?organization_id=########## Body:     {

      • Computer Showing Offline in Unattended Access

        I have a computer that was connected to the internet but showing offline in Assist. I tried uninstalling the program and deleting it from Zoho Assist the reinstalling and it still does not show up. I have been a user for several months and am not pleased with the lack of connectivity with Assist. If this continues I will have to find another product. The computer I reinstalled it on is not even showing up in Assist now. The name is NYRVLI-PC. Thanks

      • Closing Accounting Periods - Invoice/Posting dates

        Hi, I have seen in another thread but I'm unsure on how the 'transaction locking' works with regards to new and old transactions. When producing monthly accounts if I close December 24 accounts on 8th Jan 25 will transaction locking prevent me from posting

      • Zoho CRM Portal Error

        Hi, We’re experiencing an issue with the Zoho CRM Portal. When we try to access it, we receive an HTTPS connection error: net::ERR_CERT_COMMON_NAME_INVALID. If we proceed past that, we then get a 400 Bad Request error. Could you please help us resolve

      • Can we do Image swatches for color variants?

        We want to do something like the attached screenshot on our new zoho store. We need image swatches instead of normal text selection. We want to user to select an image as color option. Is this doable? I don't see any option on zoho backend. Please h

      • Integrating Zoho CRM EmbeddedApp SDK with Next.js — Initialization and Data Fetching Issues

        You can get an idea from my code I have given in end: First, I worked on a React project and tried the same thing — it worked. My goal was to import the Zoho script and then load contacts using the Zoho Widget SDK, which was successful in React. Now,

      • Feature enhancement: Highlight rows based on a cell value

        Hello Sheet users, We're excited to announce a new feature enhacement, shaped directly by your valuable feedback! As you might know, conditional formatting is a great tool for anyone dealing with large data sets. Previously, if you’ve ever wanted to draw

      • Script Editor not an option

        I am trying to apply a script to a sheet and Script Editor is not an option. I don't want to go outside Sheets to do this (like Creator) if it can be done inside Sheets.

      • monetizing the courses

        Can I add a price for course enrollment ?

      • Can we add zoho books features like invoices estemates etc on our zohocommerce website. When our customer login with their account they can able to see all books features in one place on zohocommerce?

        Can we add zoho books features like invoices estemates etc on our zohocommerce website. When our customer login with their account they can able to see all books features in one place on zohocommerce?

      • Copy paste from word document deletes random spaces

        Hello Dear Zoho Team, When copying from a word document into Notebook, often I face a problem of the program deleting random spaces between words, the document become terribly faulty, eventhough it is perfect in its original source document (and without

      • Create custom rollup summary fields in Zoho CRM

        Hello everyone, In Zoho CRM, rollup summary fields have been essential tools for summarizing data across related records and enabling users to gain quick insights without having to jump across modules. Previously, only predefined summary functions were

      • Download a file from within a zoho creator widget

        I have a widget running in Zoho Creator , it displays uploaded documents in a table file, and I have added a download link in the view. ( The widget is created with html, css and javascript). I do not succeed in getting the download working. Do I have

      • Taxes for EU B2B Transactions

        Currently, ZC doesn't seem to have a procedure for validating VAT numbers of businesses purchasing in another EU state, and removing local VAT is valid. This is essential for all inter EU B2B trade.

      • Unable to Receive Emails on Zoho Mail After Office 365 Coexistence Setup – Error: 553 Relaying Disallowed

        Hello, My domain name is bigniter.com, and I’ve been using Zoho Mail as my email service provider without any issues. Recently, I followed the steps outlined in the Zoho documentation to enable Coexistence with Office 365: 🔗 https://www.zoho.com/mail/help/adminconsole/coexistence-with-office365.html#multi-server

      • Enhance Sign CRM integration

        Hello all, I'm working on a custom Deluge script to enhance the integration between Zoho CRM and Sign by using a writer merge template for additional flexibility. I want to replicate the post-sign document integration that exists between CRM and Sign

      • CRM Related list table in Zoho analytics

        In Zoho Analytics, where can I view the tables created from zoho crm related lists? For example, in my Zoho CRM setup, I have added the Product module as a related list in the Lead module, and also the Lead module as a related list in the Product module.

      • Candidate Registration/Invitation

        It would be great to include the 'invite' candidate functionality into some of the automation functions - ether through a custom function trigger or webhook or accessible through an email template.  Currently there is no way to add this functionality into any workflows or blueprint steps which, I find limits the ability to invite candidates to engage with us directly through our candidate portal. 

      • [Free Webinar] Learning Table Series - Creator for Infrastructure Management | A Partner-driven collaborative session

        Hello Everyone! We’re excited to invite you to another edition of Learning Table Series, where we showcase how Zoho Creator empowers industries with innovative and automated solutions. About the Learning Table Series The Learning Table Series is a free,

      • Where we can change the icon in social preview

        Hi, we changed our logo, and the image that appear in preview (ex : when we post a appointment link somewhere) is still our old logo. I did change our logo in the org setting. https://bookings.zoho.com/app/#/home/dashboard/settings/basic-info?clview=false

      • Cadences

        I have just started using Cadences for follow-up up email pipeline. Is it just me or do you find the functionality very basic? For example, it will tell me (if I go looking for it) if someone has replied to a follow-up and been unenrolled; but it won't

      • I have error AS101 when I try to add paypal@mydomain.com to Zoho

        Please help me with this. I tried to call the help line 4 times but don't get any response.

      • VAT on Multiple Expenses

        Firstly, might I say I am based in Ireland, and I'm not an accountant, and therefore my understanding of procedures could be somewhat limited. Scenario. An employee pays for items throughout a month, and therefore their claim for expenses includes multiple items - some of the items are VAT deductible (eg, an IT product) and some are not (eg, a client lunch). Discovery The odd thing in ZBooks, is that whilst it seems possible to add a VAT rate to a one-off item of expense, it is not possible to add

      • Detect and ignore bots in visitors

        The SalesIQ visitor numbers are basically useless to us because there is no bot detection. We get the same bots coming in from the same countries looking at the same pages every day. It can't be that difficult to tell the difference between an actual

      • Agent assignment filter?

        Godo day, We are starting to play with FSM to see if it's going to work for our needs. Now so far we have found that it's very restrcitve in the field department you you have layout rules or can't even hide fields depending on the users roles. We can't

      • Default tax type for mileage?

        Where we are, mileage includes a government tax. Is there any way to set a default tax for the Personal Car Mileage category of expense? (Or any other way?)

      • DKIM Now Mandatory - Changes to Zoho Forms Email Policies

        Hello Zoho Forms Users, This post is to inform you about an important update regarding the authentication of all email domains in your Zoho Forms account. This year, we are doubling down on our commitment to deliver a secure, seamless, and empowering

      • All new Address Field in Zoho CRM: maintain structured and accurate address inputs

        The address field will be available exclusively for IN DC users. We'll keep you updated on the DC-specific rollout soon. It's currently available for all new sign-ups and for existing Zoho CRM orgs which are in the Professional edition. Managing addresses

      • In arattai received message can't be deleted

        The issue has been noticed in following: arattai app (Android) arattai app (Window) arattai web While the message posted by me may be deleted, the ones received from others can't be. The item <Delete> change to <Report> when the message is a received

      • is it possible to add more than one Whatsapp Phone Number to be integrated to Zoho CRM?

        so I have successfully added one Whatsapp number like this from this User Interface it seems I can't add a new Whatsapp Number. I need to add a new Whatsapp Number so I can control the lead assignment if a chat sent to Whatsapp Phone Number 1 then assign

      • Kiosk Page Refresh

        We have a Kiosk running from a button in contacts to update values and also add related lists, which works great, but when the kiosk is finished the page does not refresh to show the changes. Is there a way to force the contact to refresh/update when

      • Tips & tricks: Make SalesIQ automations work for you

        Every day, thousands of visitors land on your website. Some browse, some buy, and some leave without a word. But, wouldn’t it be great if you could automatically know who’s interested, engage them at the right moment, and never miss a lead, and all this

      • Add Custom Reports To Dashboard or Home Tab

        Hi there, I think it would be great to be able to add our custom reports to the Home Tab or Dashboards. Thanks! Chad

      • Finding missing records

        I have a challenge and I am not really sure where to start with it. I can't find any similar threads on here, can anyone help: I have two forms, FormA and FormB. Both forms have records that contain a field called Job_Number. What I am trying to achieve

      • Digest Octobre - Un résumé de ce qui s'est passé le mois dernier sur Community

        Chers utilisateurs, Faisons le point sur les temps forts du mois d'octobre au sein de notre Community Zoho France. Partager des informations sensibles, comme des mots de passe, peut vite devenir compliqué et risqué. Les données circulent par email ou

      • How do I fully delete a user account

        Hi, I have two old accounts on my CRM which are inactive and show as 'Closed' in the list of users. BUT they are basically still there. How do I fully delete these accounts? They appear in some parts of CRM still such as the on the contact record emails

      • Next Page