Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Blueprint transitions on locked records

      We use the ability to automatically lock records (quotes, sales orders, etc.) based on criteria, such as stage. For instance, if a quote has been sent to a client, the quote is then locked for further edits. Our ideal quote stage process is: Draft>Sent>Won.

    • Whatsapp Limitation Questions

      Good day, I would like to find out about the functionality or possibility of all the below points within the Zoho/WhatsApp integration. Will WhatsApp buttons ever be possible in the future? Will WhatsApp Re-directs to different users be possible based

    • This campain is on hold

      Trying to start a Zoho Campaign for our company but the error message when we try to send it out it this: This campaign is on hold Our compliance team needs additional information about your sender domain address. They'll reach out to you shortly via

    • Sales Ordes removes TAX at random for some or all prodcuts, even after creating it

      Hi at Zoho CRM, in the Sales Orders section, when Creating, Cloning or editing any Sales Orders the TAX of one or all products gets "unchecked" at random. All of our products ALWAYS must have the same TAX selected automatically. At least it used to be

    • Turning off self view in Zoho Meeting

      Hi Is it possible to turn off/hide self view in meetings?  Cant see any options for it. Thanks

    • CRM to Desk Contact Sync Issues?

      Hello, I noticed none of the contact changes in CRM are being transferred over to Desk. Anyone else experiencing this? I checked the sync history in desk and it confirms it - it doesn't show "X Contacts Synced" like it normally does.

    • Please add Deluge code editing for Forms

      Assuming Forms is built on the same codebase as Creator, etc., it would be wonderful to be able to edit form code, cut and paste form elements across different forms, share code with other users, and so on.

    • Is there any workaround or method to restrict user login to a particular device

      Hi, I want to restrict my CRM user from accessing the Zoho CRM/Zoho One application to the company provided devices only. As far as I have explored there is no direct way to do this. Is there any workarounds that will allow me to do this. I want to ensure

    • Email from CRM being Blocked or Marked as Spam by Google (and maybe more)

      In the past 24 hours we’ve noticed that emails sent via Zoho CRM are being blocked or flagged as phishing, particularly by Google. The issue seems to occur specifically when emails contain links. URLs like www.domain.com or www.example.com are automatically

    • Introducing SlyteUI : From Idea to a Working Interface in Minutes

      Hello everyone! Are you spending hours building basic UIs? Does even the smallest customization feel like a major task? CRM customization should feel intuitive and straightforward, not time consuming or exhausting. SlyteUI makes this possible by simplifying

    • All new Address Field in Zoho CRM: maintain structured and accurate address inputs

      Availability Update: 29 September 2025: It's currently available for all new sign-ups and for existing Zoho CRM orgs which are in the Professional edition exclusively for IN DC users. 2 March 2026: Available to users in all DCs except US and EU DC. 24

    • Zoho Tables - Form default entry

      How so I set a form field to be populated with an entry from another table? I created a view on my source table limiting the results to just the one thing I want to see. When adding a Client via my Add Client form, I want the Client Type field to already

    • Payment Card or Identity form-fill from Vault?

      Hello! I'm working on replacing Bitwarden with Vault and one issue I've run into is that I can't find any option to fill address and payment forms from Payment Card or Identity info that has been saved in Vault. Is there a way to do this? Is it a planned

    • Introducing Bin Locations In Zoho Inventory

      Hello users, We are excited to let you know that your wait for the Bin Locations feature has now come to an end! Yes, you heard us right! We are here to introduce the much-awaited Bin Locations now in Zoho Inventory. But before we dive into the feature

    • Add Actionable “Smart Commits” for GitHub Integration

      Hi Zoho Projects Team, I am submitting a feature request to introduce Smart Commits support for the GitHub integration in Zoho Projects. The Current Limitation: Currently, Zoho Projects offers a "passive" integration. Including a Task ID in a commit message

    • Item Details Field - New Barcode / Document option?

      Is this a new feature??? its in both books and inventory.

    • Items attribute questions

      Many of my items have attributes, such as size and color. How can I add new fields to the "New Items" screen to capture that in my Purchase Orders, Items, and Sales Order pages? I only see these attribute fields when adding an Item Group. Also, on the

    • SEO recommendation of H1 tag for website tittle

      The exact words are “ It is good practice to place the page title inside the H1tag.” Now I already have one H1 tag on my website but it is not website tittle. In the SEO recommendation that is clear too that I have h1 tag on my page. Now I don’t know

    • Anyone Building AI-Based SEO Dashboards in Zoho Analytics?

      Hey everyone, I’m currently working on an SEO reporting dashboard in Zoho Analytics and looking to enhance it with AI-based insights—especially around AI visibility, keyword trends, and traffic sources. The goal is to track not just traditional metrics

    • Knowledgebase SEO

      We have a custom-domain mapped help center that is not restricted via login. I have some questions: a) will a robots.txt file still allow us to control indexing? b) do we have the ability to edit the sitemap? c) do category URLs get indexed by search

    • Marketing Tip #15: Rank better with keyword-rich URLs for product pages

      Your product page URL is a small detail that can make a surprisingly big difference. Clean, readable URLs help in two ways: They’re easier for customers to trust and remember (no one likes clicking a link that looks messy or random). They help search

    • Marketing Tip #26: Optimize product images for SEO

      Product images can do more than make your store look good. They can also help customers discover your products through search. Since search engines can’t "see" images, they rely on text signals to understand what an image is about. Two small actions make

    • Zia Agents in Zoho CRM: a better way to set up digital employees

      Hello everyone, If you've been using Zia Agents in Zoho CRM, so far using Connections was the only deployment method you're familiar with. You create an agent in Zia Agents (define its objective, write instructions, use tools, add knowledge base) and

    • Cross Module Filtering – Use Fields from Lookup modules in Custom Views criteria and Advanced Filters

      Hello everyone, Zoho CRM now enables you to achieve deeper filtering of records in a module, using fields of a lookup, thereby enhancing your data management experience manifold. This filtering based on lookup module fields is now available in advanced

    • How can we use Loop Structures in Deluge script?

      Hi, I am using GetRecords() function to fetch records from a module in Zoho CRM,  the number of records is 800+, but GetRecords() function returns 200 records at one time,  So, I want to use loops to achieve my task, but it gives me error when i use any loop instead of For each loop. How can I perform my task? Please help Zoho community. Thanks 

    • Stock Count

      The stock count is a nice new feature, but we cannot figure out how to: 1. Use it without assigning to a person, we have a team or one of multiple do stock counts as do most any company. 2. Add any extra fields to what the "counter" sees. The most important

    • SalesIQ Operator Activity Reports in Zoho Analytics

      I'm busy building a dashboard in Zoho Analytics and I want to include SalesIQ stats in the dashboard, but I'm unable to get the statistics mentioned in the attached image. Any idea where I can get the stats for Operator Activity?

    • Multi-Select Field Option Limit Inquiry

      Hello FSM Team, We would like to inquire about the limitation of the Multi-Select field in Zoho FSM. Currently, the system allows a maximum of 10 selectable options, and we encountered the message: “Option count more than allowed count of 10.” Issue:

    • MS Teams integration

      I followed all the steps to configure the bot chanel notifications, for all events. All configuration worked fine, without issues, but I don´t recevie any notification in the Teams chanel. Must I configurate aditional permission set in MS 365 or Zoho

    • Zoho Bookings - Feature Request - Services Which Include A Resource and Consultant

      Hi Bookings Team, My feature request is to have the ability to add Consultants and Resources to Services. Use case: Your business provides first aid training and there are certain equpment you require to provide the half day training. There are only specific

    • Six important date functions every professional should know

      You probably use plenty of formulas already, but there are some that are essential for people in business roles, such as date functions. These tools help with scheduling, predicting, and tracking, and they can be useful whether you're dealing with payrolls,

    • Inactive Items - Make Less Prominent by Default

      Currently, when one marks an Item as "Inactive", it really doesn't do much of anything to hide it or get it out of the way. Search and reporting within Finance should, by default, hide inactive Items from standard reports, searches, etc. If one specifically

    • Why Contracts Should Not Be Downloaded And Sent During Negotiation

      During contract negotiation, it may seem easy to download a document, send it to the counterparty, have the counterparty make changes, and send it back. But this approach creates serious risks around security, version control, and compliance. ISO 27001:2022

    • Add Native “Create GitHub Branch” Action for Tasks and Bugs

      Hi Zoho Projects Team, I would like to submit a feature request to enhance the GitHub integration in Zoho Projects by adding the ability to initiate a GitHub branch directly from a Task or Bug. The Current Limitation: Currently, Zoho Projects lacks a

    • Super Admin Logging in as another User

      How can a Super Admin login as another user. For example, I have a sales rep that is having issues with their Accounts and I want to view their Zoho Account with out having to do a GTM and sharing screens. Moderation Update (8th Aug 2025): We are working

    • Solving the bug in Zoho Writer API for styling

      So... the Zoho Writer APIs for programatically creating a document do not respect a template's style. The result is that any document you generate via API needs to be manually, paragraph by paragraph, reformatted. That bug alone is sufficient to render

    • Is anyone else experiencing downtime on Desk?

      Desk is slow/crashing and not stable the last couple of hours. Is anyone else experiencing the same issues?

    • Push Notification for New Bookings in Zoho Bookings App

      when a someone schedules an appointment through the booking page, is there any option to receive a push notification in the mobile app?

    • Data Import into Zoho Forms

      Hi everyone, I am migrating several Google Forms into Zoho Forms. I can recreate the forms, but I also need to bring over historical response data currently stored in Google Sheets. I want to keep using that data inside Zoho for analysis, reporting, and

    • Add Native GitHub Pull Request Creation & Management from Tasks/Bugs

      Hi Zoho Projects Team, I am submitting a feature request to enhance our development lifecycle by enabling native GitHub Pull Request (PR) creation and management directly within Zoho Projects. The Current Limitation: Currently, the GitHub integration

    • Next Page