Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team




      • Sticky Posts

      • Register for Zoho Desk Beta Community

        With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

      • Share your Zoho Desk story with us!

        Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

      • Tip #1: Learn to pick the right channels

        Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

      • Welcome to Zoho Desk Community - Say hello here!

        Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

      • Webinar 1: Blueprint for Customer Service

        With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

        • Recent Topics

        • Zoho CRM Tracking Google Enhanced Conversions

          Can anyone @Zoho, consultants, or users help me understand if Zoho CRM is going to support Google's Enhanced Conversions? I included some information from Google below about it. We use Google Adwords for our pay per click advertising for lead generation,

        • zoho click, and nord VPN

          Unfortunately, we've been having problems with Zoho Click, where essentially the line cuts off after about a minute's worth of conversation every time we are on VPN. Is there a way we can change this within the settings so it does not cut the line off

        • Connecting Portals from different Zoho apps

          Hi, I note that Zoho has functionality for customer portals for several of the Zoho apps, like CRM, Projects, Desk etc. Is there any way to connect these portals?  It would be great if we could give our customers access to a portal in which they could

        • Billing Management: #5 Usage Billing

          After understanding the nuances of Advance Billing and Retainers, we will explore one of the booming billing models. Long ago, villagers drew water from a shared well in a small village. The well was a lifeline for the entire community. Ravi, the well

        • Function #10: Update item prices automatically based on the last transaction created

          In businesses, item prices are not always fixed and can fluctuate due to various factors. If you find yourself manually adjusting the item rates every time they change, we have the ideal time-saving solution for you. In today's post, we bring you custom

        • Inventory Adjustments

          Hi, How to transfer the material from one head to another ? Like materials purchased for manufacturing the laptop need to transfer from consumption inventory (Quantity of raw materials reduced) to destination inventory ( Quantity of Laptop increased)

        • Zoho CRM Community Digest - Aug 2025 | Part 1

          Hey everyone! The first half of August went by, and we have a few announcements and some good noteworthy discussions. So, let's take a look at them! Product Updates: Introducing Connected Records feature: Zoho CRM’s Next-Gen UI now includes Connected

        • Please make it easier to Pause syncing

          right now it takes 3 clicks to get there. sounds silly, but can you make it just 2 clicks to get it done instead? thats how dropbox does it, 2 clicks to pause instead of 3.

        • How to create a Zoho CRM report with 2 child modules

          Hi all, Is it possible to create a Zoho CRM report or chart with 2 child modules? After I add the first child module, the + button only adds another parent module. It won't let me add multiple child modules at once. We don't have Zoho Analytics and would

        • SalesIQとPageSenseの利用について

          初めての投稿で場違いだったらすいません。 弊社ではSalesIQを運用しているのですが、追加でPageSenseの導入もしたいと現場からの声があります。 両サービスともクッキー同意バナーが必要なサービスなのですが 弊社では同意無しに情報はとりませんという方針なので 2つ入れると2つバナーを出す必要がでてきます・・・ 両サービスを運用されてる方があれば運用状況とか教えてほしいです。 PageSenseについては詳細まで機能を理解してないなかでの質問です。

        • How to integrate Zoho Forms with Zoho CRM on Standard Plan

          Hello Zoho Support Team, I am using the Standard Zoho Forms plan (USD 30/user) and I would like to integrate Zoho Forms with Zoho CRM so that certain fields in my forms can be automatically prefilled using data from Deals in CRM. Specifically, I want

        • Export PDF File Name

          Is it possible to change the default Zoho .pdf naming scheme for inventory items like quotations? Would like to use the the Subject as the default quote name. Is this possible?

        • How to change the from address from 'no reply' for an email template in CRM

          Hi, We have our CRM set up with the from field as sales@XXX. I have just created a series of email templates and sent a test and they are sending from noreply@zoho I have tried searching for how to change the email template but don't have the options

        • Zoho CRM Client Script - SetCriteria in lookup Field

          Hello All One More Zoho CRM Client Script Tips & Trick. Now you can Set the Criteria on Your lookup in zoho CRM, It Comes With a Create Page, Edit Page, and Details Page (Standard). Example:- We have a Room Module that includes Room Name, Status, Campus,

        • Kaizen #71 - Client Script ZDKs for Detail (Canvas) Page

          Hello everyone! Welcome back to another interesting Kaizen post. In this post, we can discuss Client Script ZDKs support for Detail (Canvas) Page. What is Detail (Canvas) Page? A Detail(Canvas) Page allows you to customize the record detail page to your

        • Zoho Reports Duplicating Entries

          I have a custom costing tab with a table where we entre invoices. These are under a Heading (PO Subject) and notes added in the form with different line items. In the reports, I have organised the report to group per PO Subject, with the total of the

        • Validation Rule Not Working for Mandatory Field in Zoho Blueprint

          As a Zoho user, we created a validation rule for a specific field. However, we noticed that when we made the same field mandatory within a Blueprint, the validation rule we defined did not work. When we reported this issue to Zoho Support, they stated

        • Notes Issues

          Been having issues with Notes in the CRM. Yesterday it wasn't showing the notes, but it got resolved after a few minutes., Now I have been having a hard time saving notes the whole day. Notes can't be saved by the save button. it's grayed out or not grayed

        • Export from Contacts module to Products module in Zoho CRM

          Good afternoon, I would like to send a number of contact info from the Contacts module into the customized module (tickets to an event) in one operation. I have selected several contacts in the Contact module (people who I have labelled as people I want

        • Can’t receive emailI c

          I have generated a basic for but when I submit it I don’t get a email, I’ve been in the settings and tested me email, all appears correct, can you please help me

        • Data Capture for Historical Activity (Especially One Lead Downloading Variois reports without Overwriting the info)

          Is there a better way in Zoho CRM to capture and archive a lead’s historical activity—specifically whenever they download reports—so that the data is stored without being overwritten?”

        • Client Script - Updating Field Value in Detail Page of a Lead

          Hello, I'm trying to use Client Script To enrich some data of the Lead when one of my User fill the "City" field in the detail page of the Lead. This is my Script: log (value); var response = ZDK.Apps.CRM.Functions.execute("getInfoCitta", { "nomeCitta":

        • Auto shapes in Zoho sheet.

          Does Zoho sheet supports inserting auto shapes (rectangle, circle...). I did not see any option to do so.  If its not supported currently, is there any plans on bring in this features. Any timelines ?

        • How to get the call recording external ID via desk API

          I have enabled phonbridge integration with Zoom Call. I am trying to access the call recording in Zoom by calling Zoom API. I have built a Desk workflow to trigger on a new call, to call a custom function. when calling the API, the response doesn't contain

        • Can't View Project Names in Mobile App

          I can't view project names on PO's in the app, nor can I add that as a viewable PDF field in inventory on the computer. I've attached screenshots showing that in the mobile version whether you are on the PO, editing the PO, or viewing the PO line items,

        • Notebook

          I have purchased the monthly pro subscription of Notebook. But it does not support my XP-Pen to write something in it. So it is not useful to me. Hence I am requesting you to help me to discontinue this subscription.

        • Domain Mapping & Image Publishing Issues on Zoho Sites

          Hello, I am facing two issues with my Zoho Sites account: 1. Images not visible after publishing. 2. Domain mapping error: "Domain already exists". I am a paid customer. Please connect me with Live Chat Support or Zoho Assist so I can show my issue

        • Prevent duplicate with custom fields?

          I was wondering something about custom field/custom modules in Zoho Desk. For some reason you can make a custom field mandatory but not unique? For example, if I create a custom module to manage equipment and renewal and make a field serial number no

        • Generate leads from instagram

          hello i have question. If connect instagram using zoho social, it is possible to get lead from instagram? example if someone send me direct message or comment on my post and then they generate to lead

        • Where is the desktop app for Zoho Projects???

          As a project manager, I need a desktop app for the projects I manage. Yes, there's the web app, which is AWESOME for cross browser and platform compatibility... but I need a real desktop app for Projects that allow me to enter offline information where

        • How to Automate Monthly PDF Reports with Filters in Zoho Creator

          Hi everyone, I’m trying to build an automated monthly reporting process in Zoho Creator and would appreciate suggestions or best practices from anyone who has done something similar. What I’m trying to do: I have a form called New_Customer with fields

        • Feedback: Streamlining Note Management in Zoho Notebook

          Dear Team/Support, I would like to share some feedback regarding the note management system that could help improve usability and accessibility for users like myself. Notebook 1 (screenshot attached): Currently, the system does not allow selecting and

        • showing Limit exceeded

          Good afternoon...trust you're good. I've been having issues working with but it's not responding. it's showing Limit exceeded, sorry it seems like too many people are working on the sheet right now please try again later. meanwhile no one is working on

        • Upload API

          I'm trying to use the Upload API to upload some images and attach them to comments (https://desk.zoho.com/DeskAPIDocument#Uploads#Uploads_Uploadfile) - however I can only ever get a 401 or bad request back. I'm using an OAuth token with the Desk.tickets.ALL

        • Losing description after merging tickets

          Hello, We merge tickets when they are about the same topic from the same client. It happens sometimes. We recently noticed that after the merger only the description from the master ticket is left in a thread. And the slave-ticket description is erased.

        • update linked contacts when update happens in account

          Hi, I have a custom field called Licence in the Accounts module. When someone buys a licence, I’d like to update a custom field in the related Contacts. How can I achieve this? I noticed that workflows triggered on Accounts only allow me to update fields

        • Problem Management Module

          I am looking for a Problem Management module within Zoho Desk. I saw in some training videos that this is available, and some even provided an annual price for it. I want an official confirmation on whether this is indeed available. This is not a particularly

        • Unable to explore desk.zoho.com

          Greetings, I have an account with zoho which already has a survey subscription. I would like to explore desk.zoho.com, but when I visit it while logged in (https://desk.zoho.com/agent?action=CreatePortal) I just get a blank page. I have tried different

        • Offline support for mobile app

          Accessing your files and folders from your mobile devices is now quicker and simpler, thanks to the power of offline support. Whether on an Android or iOS device, you can use the Offline function to save files and folders, so you can review them even

        • Zoho Desk KB article embedded on another site.

          We embed KB articles from Zoho Desk on another site (our application). When opening the article in a new tab, there is no issue, but if we choose lightbox, we are getting an error "To protect your security, help.ourdomain.com will not allow Firefox to

        • Next Page