Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    Nederlandse Hulpbronnen


      • Recent Topics

      • Unable to copy into a new document

        Whe I create a new Writer doc and attemp to copy and past I get this message. The only way to copy into a document is I duplicate an existing document, erase the text and save it under a different name and then paste the information. Not ideal. Can you
      • Kaizen #157: Flyouts in Client Script

        Hello everyone! Welcome back to another exciting edition of our Kaizen series, where we explore fresh insights and innovative ideas to help you discover more and expand your knowledge!In this post, we'll walk through how to display Flyouts in Client Script
      • Admin Control Over Profile Picture Visibility in Zoho One

        Hello Zoho Team, We hope you are doing well. Currently, as per Zoho’s design, each user can manage the visibility of their profile picture from their own Zoho Accounts page: accounts.zoho.com → Personal Information → Profile Picture → Profile Picture
      • Published Course Not Reflecting In Hub

        Hi! I am trying to create micro-learning courses for our team to be available for self-guided learning. I have published the courses with enrollment settings open to all users of the hub, but they don't appear to be available for enrollment. Am I missing
      • Unlock Locked Users via Zoho One Mobile App

        Hello Zoho One Team, We have noticed that in the Zoho One web admin panel, we can unlock a locked user when needed. However, when using the Zoho One mobile app, there is no indication that a user is locked, nor is there an unlock button similar to what
      • Tip #47- Stay Ahead with Automated Scheduled Reports in Zoho Assist- 'Insider Insights'

        We’ve made it easier for you to stay informed, even when you’re busy managing remote sessions! With Scheduled Reports in Zoho Assist, you can now automatically receive detailed insights about your remote support and unattended access activities directly
      • Colour Coded Flags in Tasks Module List View

        I really like the colour coded flags indicating the status of the tasks assigned to a Contact/Deal in the module list view. It would be a great addition to have this feature available in the list view of activities/tasks. I understand you have the Due
      • Uploading data to S3

        Has anyone successfully uploaded data from Creator to S3 using the InvokeURL command or any other method in Deluge? (keywords: S3, AWS, Amazon, bucket)
      • UPS Label size when generated via Zoho

        We've integrated UPS with Zoho inventory. When creating and downloading the shipping labels they are created in a larger paper size. I'd like them to be generated to print on a 4x6 printer. Zoho have told me I need to do this within our UPS portal. UPS
      • Credit Management: #4 Credits on Unused Period

        Recall a familiar situation. You sign up for a monthly gym membership. You pay the subscription fee upfront, get motivated, and show up consistently for the first week. Then, suddenly, you get caught up in work deadlines, travel plans, or a dip in motivation.
      • Zoho Analytics Automatically Deletes Queries and Reports When a Synced CRM Field Is Removed

        We’ve encountered a serious and recurring issue that poses a massive data integrity risk for any Zoho Analytics customer using Zoho CRM integration. When a field is deleted in Zoho CRM — even an unused one — Zoho Analytics automatically deletes every
      • Prevent new Record Association

        Hello all, We have a small sales organization therefore, it's helpful for everyone on the sales team to be able to view the full list of accounts to assist in preventing duplicate accounts from being created. However we want to prevent people from creating
      • Tip of the Week #73– Automate workflow with Make integration.

        Make is a no-code workflow automation platform designed to connect your favorite apps and automate repetitive tasks across services. By integrating Make with Zoho TeamInbox, you can streamline everyday inbox management and save valuable time. With this
      • Viewing attachments

        I'm using a Web Form integrated in my web site to collect Leads several info, including a image upload. In order to to approve each lead, we have to view the image uploaded. Unfortunately, in the Leads view, the Attachments can only be downloaded, not
      • Kaizen #89 - Color Coding using Client Script

        Hello everyone! Welcome back to another exciting Kaizen post. Today let us see how you can apply color codes to the List and Detail Pages of Zoho CRM using Client Script. Need for color code in Zoho CRM When you mark things with different colors as a
      • Instant Sync of Zoho CRM Data?

        With how valuable Zoho Analytics is to actually creating data driven dashboards/reports, we are surprised that there is no instant or near instant sync between Zoho CRM and Zoho Analytics. Waiting 3 hours is okay for most of our reports, but there are
      • Is it possibly to directly set the tax amount on order instead of indirectly via tax rates?

        We own an eCommerce application and want to funnel submitted orders from that system into Zoho. We're already calculating tax in our application and just need a way to set it in Zoho. We tried to use Zoho's tax objects for tax by setting the rates to
      • Zoho account sign in with passkey

        Hello, I am trying to sign in using passkey, but the option doesn't show up in the web and is disabled in Oneauth on mobile, saying the admin has restricted the use. On the Admin page in Security MFA I can find no option for passkey. Help would be greatly
      • Rescheduled US meetups: Zoho Desk user meetups are coming to seven U.S. cities in October and November, 2025

        Hello to our Zoho Desk users in the United States, We're excited to share the revised dates for the upcoming Zoho User Groups happening across the US this summer. Our product experts are heading to seven cities throughout the country, and for the first
      • Checklists as templates

        Is it possible to save checklists as templates to reuse them in other tasks? Example: I have a web project. I maintain individual web URLs as tasks. Within the tasks the same checklist should be used again and again: - Page created in CMS - Properties
      • Send mass email using my secondary email

        Hello, When I send an email to just one person from Zoho CRM, a complete email editor appears, where I can choose which of my email addresses I want to use in the From field. When I send a mass email, there is not such option. I can only select the email
      • ZOHO TEAM INBOX Calendar Integration

        The Problem: Clients send meeting invitations to our TeamInbox address. TeamInbox receives these invites, but we cannot accept them. We do not use individual inboxes for transparency purposes. Ideal Solution: A way to accept calendar invites sent to our
      • Unearned / Deferred Revenue Automatic Calculation for Subscriptions

        As a SaaS business, we have multiple active subscriptions with varying dates and amounts. Is there a way to have a monthly automatic calculation for all of them that debits or credits the unearned revenue and revenue accounts accordingly? Alternatively,
      • Zoho Desk Limit for Comma Separated Queries

        Hi, I have just discovered a limit that I believed is not mentioned in any of Zoho's documentations. My search query looks like so: "query: {"accountId":"786050000091629966,786050000163589273,786050000163589427,786050000162753705,786050000162112971,786050000161987643,786050000160752868,786050000167089504,786050000167089378,786050000167089024,786050000167070005,786050000166295256,786050000128153693,786050000110560021,786050000046594575,786050000039106461,786050000002225356,786050000076889093,786050000047895103,786050000043365354,786050000044765191,786050000041790249,786050000040359116,786050000037945198,786050000024605077,786050000000525015,786050000155333895,786050000157741437,786050000000718125,786050000011574353,","departmentId":"786050000042648070","status":"Finished","sortBy":"createdTime","customField2":"cf_completion_date:2025-01-28T03:00:00.000Z,2025-10-28T03:00:00.000Z","customField3":"cf_billed:false"}"
      • Module Name doesn't exist

        I am trying to create a module named Activity, with plural Activities, but I have an error that module name already exists. This module is doesn't exist, and I don't have a single field called Activity or Activities.
      • Zoho Desk iOS and Android app update: AI powered: Reply Assistance and Refine Messages on IM module.

        Hello everyone! We are excited to introduce new AI powered features on the IM module of the Zoho Desk app. Reply Assistance: Reply Assistance generates suggested responses for incoming chat messages, which you can directly insert into the conversation
      • Unify All Zoho Video Meeting Experiences into One Standardized Platform

        Hi Zoho Team, We would like to share an important user experience concern regarding the current state of video meeting functionality across the Zoho ecosystem. The Problem Within Zoho, there are multiple ways to initiate or schedule a video meeting: Zoho
      • Is it possible to embed Youtube shorts?

        Hi Zoho desk support, This is Ryan from Accuver America. While I'm trying to create a knowledge base article with embed video, I ran into this issue. "www.youtube.com refuse to connect" A little bit background is that because this video is recorded on
      • Split functionality - Admins need ability to do this

        Admins should be able to split an expense at any point of the process prior to approval. The split is very helpful for our account coding, but to have to go back to a user and ask them to split an invoice that they simply want paid is a bit of an in
      • Delegates - Access to approved reports

        We realized that delegates do not have access to reports after they are approved. Many users ask questions of their delegates about past expense reports and the delegates can't see this information. Please allow delegates see all expense report activity,
      • How to include total km for multiple trips in expense report.

        Whenever I create a mileage report it only shows the total dollar amount to be reimbursed. The mileage for each individual trip is included but I also need to see the total distance for all trips in a report? How do I do this?
      • Get logged in user ID in Deluge script

        Hello all, How do I get the id of the logged-in user in a deluge script? the "zoho.loginuserid" function actually returns the users email address or whatever the user id they use to login to zoho with and not the id of the user record, and given that
      • Item Details Field - New Barcode / Document option?

        Is this a new feature??? its in both books and inventory.
      • Shared Mailbox - Mark as read for all users

        Hi all, Maybe someone can help me out. At the moment we have a shared mailbox without streams. When a users reads an mail or marks it as read other users will not see this. How can we resolve this? We now archive the mails when read and followed up. However
      • How do I increase the email attachment size in Zoho CRM ?

        It looks like I'm limited to 10MB when sending an attachment using the email widget on a record in Zoho CRM. Is there a way to increase the size? Or can I use some other tool? From what I'm reading online, I'm maxed out at 10MB. Any insight would be greatly
      • Can I export to PDF in Zoho Learn

        I have seen help pages where export to pdf options are available but I do not see that option available from the application. I see that exprt is available in my free trial version but that is only to html pages. I need to be able to export my manuals
      • Zoho Sites "pages" management page

        I have 80 plus pages on zoho sites. When I go to the "pages" link to view and edit pages, They are not in any kind of order, so I spend lots of time searching for pages when I need to edit or create new. How can I change the view order of all my pages
      • Staff rules

        Hi! Do you people know what are the default staff rules when a new booking is created? We have two staff members in my team (me as the admin, and my employee). As we share the same services, I'm wondering how Zoho will pick the staff for new apointments.
      • Add Image Upload Field to Zoho Bookings Registration Form

        Hi, We would like to request the addition of an image upload field to the Zoho Bookings registration form. Currently, Zoho Bookings only supports text-based fields (e.g., Single Line, Multi-Line, Email, Checkbox, Dropdown, Radio Button, and Date), but
      • Merge Tags Output Incorrect Placeholder Text After CRM Sync

        Hi everyone, I’m experiencing an issue with merge tags in Zoho Campaigns after last sync of contacts and leads from Zoho CRM (days before everything worked perfectly). Here’s the situation (seems like a default configuration in Campaigns) : My leads have
      • Next Page