Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Assign multiple departments to multiple helpcenters

      Hi there! I have a reseller company for a software and I'm using Zoho Desk as my helpcenter and ticket management system. The software is great and I would like to make a suggestion! With multi-branding activated, your departments that visible in help

    • Zoho Desk Training

      Hello, We've had Zoho desk for a while now, but we run into issues occasionally, and I was wondering if there was a customer who currently uses it and really enjoys the functionality, that would be wiling to chat with us?

    • Advanced Customization of the Help Center using JavaScript

      Hello everyone, The Help Center in Zoho Desk can be customized by using HTML and CSS to provide structure and enhance the page's appearance—but what if you want to add interactive and dynamic elements? You can add these effects with JavaScript, a programming

    • Embed CRM record images in email templates

      I have email templates that I want to embed dynamic images in their body - not as an attachment. For the context, the image is a QR code individual to each contact. So there are couple of challenges for which I think there is no solution in CRM: 1/ I

    • Exciting Updates to the Kiosk Studio Feature in Zoho CRM!

      Hello Everyone, We are here again with a series of new enhancements to Kiosk Studio, designed to elevate your experience and bring even greater efficiency to your business processes. These updates build upon our ongoing commitment to making Kiosk a powerful

    • Edit default "We are here to help you" text in chat SalesIQ widget

      Does anyone know how this text can be edited? I can't find it anywhere in settings. Thanks!

    • PO Based Advance payment to Vendor

      We recommend to introduce a provision at PO to make advance payment to vendors and auto apply that advance paid later at the time of Vendor Bill submission for that PO. This will help us track PO-wise Total Payments.

    • Converting Customer Invoice to Purchase Bill

      Hi, In my service-based business, I sometimes create the customer invoice first, and later I receive the purchase bill from the vendor for the same job. Is there any option in Zoho Books to: Convert a customer invoice into a purchase bill, or Link/associate

    • Getting Project Template List using the REST API

      I am trying to confirm that I can use the REST API to create a project using a project template. The API documentation indicates this is possible by providing the Template ID, but it is not clear at all how to get a list of available Project Templates

    • How to get Quickbooks Desktop Info into Zoho?

      Our team has used Quickbooks desktop for years and is looking at switching to Zoho books in 2026. I want to bring all old sales history over since we use Zoho CRM. I can export Item sales history and generic sales orders from Quickbooks desktop. How do

    • ZeptoMail API Request

      We tried to send mail using ZeptoMail using Django. Following is my payload {'from': {'address': 'abc@abc.com'}, 'to': [{'email_address': {'address': 'xyz@xyz.in', 'name': 'Bhavik'}}], 'subject': 'Report Name', 'htmlbody': '<p>Test</p>'} Following is

    • Zoho Inventory - Allow Update of Marketplace Generated Sales Orders via API

      Hi Inventory Team, I was recently asked by a client to create an automation which updated a Zoho Inventory Sales Order if a Shopify Order was updated. I have created the script but I found that the request is blocked as the Sales Order was generated by

    • Dropdown data depends on filters in another field.

      In my quote form I have a lookup field called Reseller that pulls from Accounts. I would like it to pull from Accounts, but only those accounts with an account field 'Type' where that is 'Reseller'. Does anyone know a way to do this? Similarly, I'd like

    • Automated log-out/session end

      I'm concerned about security of our data. Is it possible to set an automatic time-out for user sessions on Zoho CRM, after a certain period of inactivity or when the session reaches a certain duration (12 hours perhaps)? 

    • Admin asked me for Backend Details when I wanted to verify my ZeptoMail Account

      Please provide the backend details where you will be adding the SMTP/API information of ZeptoMail Who knows what this means?

    • Mass import of documents into Zoho Writer

      I'm using Google's word processor at the moment but feel that Zoho does a better job (on the online apps market). Iwant to move my documents (about 50-70) to Zoho but it seems to me that I have to import them seperately. Is it already possible to upload several documents at a time or is this a forthcoming feature? Cheers Rolli :?:

    • When will it be possible to edit Subform records via either views or tabular reports?

      Hey there, data maintenance often requires mass update of quite a lot of records. While this is a piece of cake via either List view or Zoho sheet view, the same cannot be carried out for subform records yet. When one of the two options will be made available?

    • Subform auto populate values

      Hi Team, I’m trying to retrieve values from Zoho People using API functions and dynamically populate them into a subform. For example, I’ve created a form with several fields that users will fill out. Based on their input, I need to fetch records from

    • Add "Fetch Composite Item" Action for Inventory

      I want to make a Flow that uses information returned in the GET call for Composite Items, and it's not currently available in Zoho Flow. Please consider adding this functionality.

    • How we cut CRM updates from ~20 minutes down to 2, our real workflow

      Updating the Zoho CRM after every call used to be one of the biggest time sucks for our team. By the time you write your notes, clean them up, fill in the fields, and log everything properly… you’ve easily lost 15–20 minutes per call. We started experimenting

    • Text/SMS With Zoho Desk

      Hi Guys- Considering using SMS to get faster responses from customers that we are helping.  Have a bunch of questions; 1) Which provider is better ClickaTell or Screen Magic.  Screen Magic seems easier to setup, but appears to be 2x as expensive for United States.  I cannot find the sender id for Clickatell to even complete the configuration. 2) Can customer's reply to text messages?  If so are responses linked back to the zoho ticket?  If not, how are you handling this, a simple "DO NOT REPLY" as

    • Add home page or dashboard in CRM customer portal

      is it possible to add home page or dashboard in CRM customer portal?

    • User Tips: How to change the the label display name of a system defined field

      Most users know how to change field label names via Settings > Modules & Fields but if you want to change the name of a system defined field you can’t as there is no “edit properties” option.  However with a simple hack you can edit any system defined

    • Search not working!

      I have items in my notebook tagged but when I search for a tag nothing comes up! Any fix for this?

    • Zoho Analytics Regex Support

      When can we expect full regex support in Zoho Analytics SQL such as REGEXP_REPLACE? Sometimes I need to clean the data and using regex functions is the easiest way to achieve this.

    • Zoho CRM Community Digest - October 2025 | Part 1

      Hello Everyone! Here's a quick recap of first two weeks of October! Product Updates: Zoho CRM Android App Update: Surveys, Blueprints, and Smarter Mobile Features! Zoho CRM’s Android app just got a useful upgrade. You can now share records, upload your

    • Add Custom Field Inside Parts Section

      How to Add Custom Field Inside Parts Section in Workorder like Category and Sub- Category

    • Automate onboarding emails with CRM Workflow and Accounts module

      We’re a B2B SaaS company selling to public-sector organisations. Each organisation is stored as an Account in Zoho CRM, and each organisation typically has multiple associated Contacts. Our backend syncs product-usage data (setup status, user activity,

    • Using a CRM Client Script Button to create a Books Invoice

      Hello, I need help handling error messages returned to my client script from a function. The scenario I have setup a client script button which is available from each Deal. This CS executes a crm function, which in turn creates an invoice based on the

    • Important update: Enhanced security measures for account operations in Zoho Cliq

      Greetings from the Zoho Cliq team! We’d like to share an important security update that has an influence on some admin actions such as password reset, MFA reset, and MFA backup code generation. What’s changing? With our latest security enhancements, these

    • Boosting efficiency through faster ticket rendering

      This enhancement will provide faster access for support teams and end-users, significantly boosting productivity for everyone. Get ready for a more efficient and satisfying experience! Immediate benefits Faster ticket rendering reduces wait times and

    • Read webpage - MSXML2.ServerXMLHTTP

      I have the following VBA script, put together from various sources (mainly zoho forum/help/support, so it once worked, I guess): private Sub GetListOfSheets() Dim url As String Dim xmlhttp As Object Dim parameters As String Dim html As String range("B1").value

    • Sales Receipts Duplicating when I run reports why and how do we rectify this and any other report if this happens

      find attached extract of my report

    • Can I execute two 'functions' when completing a mail merge from CRM?

      Hi, I have set up a mail merge from CRM Deals to a template. I want a copy of this to be saved in Workdrive, and then a copy also saved back into the deal record from which the merge occurred. I can do both independent of each other, and managed to get

    • No Functional Autosave or Manual Save Button

      Application : Zoho Notebook So I wanted to try Zoho Notebook(On Ubuntu) as an application, I installed the application and went solving my LeetCode problems visually(Drawing mode), at one point the app just stopped saving anything... Every time I tried

    • Enterprise subscription support

      My organization sells subscription services to enterprise customers, which is a different model from the consumer subscription model that Zoho Billing has been designed to support and I beleve this capability should be added. An enterprise subscription

    • Issue with Creator's IF logic

      Hi, I found the following code produces unexpected results: if(-1.0 < 0.0000000) {       info "True"; } else {       info "False"; } if(-1.0 < 0.000000) {       info "True"; } else {       info "False"; } The output returned is: False True However, the

    • Need option to send Package PDF in shipment email (Shipment PDF is missing Lot info)

      Is there any way to automatically attach the Package PDF instead of (or alongside) the Shipment PDF in the notification emails? We really need this feature because the default Shipment PDF creates a blind spot for our customers. It does not display Batch/Lot

    • zoho creator view is not present in the workspace and blank reports

      Hi Support,  Users who have "write" permissions keep getting this error for all of our embedded reports all of a sudden. See screen shot below: Meanwhile, my developer permissions account sees a blank screen in view and edit mode as shown in the screenshots

    • Customize portal email template

      Can i fetch only first name of the user in portal email template instead of the below code Hi ${User.FULL_NAME}

    • Next Page