Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere

        Zoho FSM Video Tutorials


              All-in-one knowledge management and training platform for your employees and customers.






                                    Zoho Developer Community




                                                          • Desk Community Learning Series


                                                          • Digest


                                                          • Functions


                                                          • Meetups


                                                          • Kbase


                                                          • Resources


                                                          • Glossary


                                                          • Desk Marketplace


                                                          • MVP Corner


                                                          • Word of the Day


                                                          • Ask the Experts



                                                                    • Sticky Posts

                                                                    • Register for Zoho Desk Beta Community

                                                                      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                                    • Share your Zoho Desk story with us!

                                                                      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                                    • Tip #1: Learn to pick the right channels

                                                                      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                                    • Welcome to Zoho Desk Community - Say hello here!

                                                                      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                                    • Webinar 1: Blueprint for Customer Service

                                                                      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                                    Manage your brands on social media



                                                                          Zoho TeamInbox Resources



                                                                              Zoho CRM Plus Resources

                                                                                Zoho Books Resources


                                                                                  Zoho Billing Resources

                                                                                    Zoho Projects Resources


                                                                                      Zoho Sprints Resources


                                                                                        Qntrl Resources


                                                                                          Zoho Creator Resources



                                                                                              Zoho CRM Resources

                                                                                              • CRM Community Learning Series

                                                                                                CRM Community Learning Series


                                                                                              • Kaizen

                                                                                                Kaizen

                                                                                              • Functions

                                                                                                Functions

                                                                                              • Meetups

                                                                                                Meetups

                                                                                              • Kbase

                                                                                                Kbase

                                                                                              • Resources

                                                                                                Resources

                                                                                              • Digest

                                                                                                Digest

                                                                                              • CRM Marketplace

                                                                                                CRM Marketplace

                                                                                              • MVP Corner

                                                                                                MVP Corner









                                                                                                  Design. Discuss. Deliver.

                                                                                                  Create visually engaging stories with Zoho Show.

                                                                                                  Get Started Now


                                                                                                    Zoho Show Resources

                                                                                                      Zoho Writer

                                                                                                      Get Started. Write Away!

                                                                                                      Writer is a powerful online word processor, designed for collaborative work.

                                                                                                        Zoho CRM コンテンツ




                                                                                                          Nederlandse Hulpbronnen


                                                                                                              ご検討中の方






                                                                                                                        • Recent Topics

                                                                                                                        • Announcing the new SKILL.md for Zoho CRM and the updated OAS repository!

                                                                                                                          We are introducing a new zoho-crm skill to make working with Zoho CRM Developer tools (like APIs, functions, widgets, client scripts, queries etc) easier and faster, with the help of AI in your preferred AI harness like Claude Code, Codex, Cursor, VSCode
                                                                                                                        • Kaizen #258 - Getting Started with zoho-crm SKILL.md

                                                                                                                          Howdy tech wizards, Welcome to a fresh week of Kaizen. This week, we are taking a look at the zoho-crm SKILL.md, an Agent Skill designed to help AI coding agents work with Zoho CRM’s developer capabilities. What is zoho-crm SKILL.md? The zoho-crm skill
                                                                                                                        • Support Required for Integrating Zoho SalesIQ in Flutter Web

                                                                                                                          Hi Team, I have developed a Flutter application and I am currently using the salesiq_mobilisten: ^6.5.4 package. As per the package documentation ( https://pub.dev/packages/salesiq_mobilisten ), it supports only Android and iOS, and it is working fine
                                                                                                                        • Books is extremely slow again today !

                                                                                                                          Everything is running slowly even with 500mb connection speed
                                                                                                                        • MCP for Zoho Sign is not working

                                                                                                                          I'm using the mcp builder at mcp.zoho.com and selected Zoho Sign. I have created it 4 different times but my other mcp servers are working. However, mcp for zoho sign WILL NOT WORK. It throws a 400 error. I've checked everything. I tried with both Grok
                                                                                                                        • Automate Credit Card Surcharge

                                                                                                                          Is there a way to create an automation that will add a 3.0% credit card surcharge to a subscription whenever a customer pays via credit card?
                                                                                                                        • How to charge Convenience fee OR payment gateway charges to the end client who is paying the invoice?

                                                                                                                          Hello, I am creating this topic after having discussions with various sets of users and have understood that with people moving more and more to digital payments, it is important for the client to enable the "Convenience fee" kind of scenario. I have
                                                                                                                        • How to add a fee based on payment method

                                                                                                                          I currently accept both ACH payments and credit card. I want to add a 3% fee to a number of my subscription plans if the customer chooses to use a credit card (and not ACH/checking). How do I do this? Is there an in-built feature, or would i have to create
                                                                                                                        • Automating CRM backup storage?

                                                                                                                          Hi there, We've recently set up automatic backups for our Zoho CRM account. We were hoping that the backup functionality would not require any manual work on our end, but it seems that we are always required to download the backups ourselves, store them,
                                                                                                                        • Zoho Inventory - Profit Margin Feature - Allow Users to Set Default Cost Price Calculation Method

                                                                                                                          Hi Zoho Inventory Team, I'm working with an automotive parts distributor using Zoho Inventory. They have expressed frustration about having to change the Cost Price Method on every line item they add from the defult "Inventory Valuation Method" to "Item
                                                                                                                        • OpenAI Is Moving to the Responses API: Here's What It Means for SalesIQ

                                                                                                                          OpenAI has deprecated its Assistants API and is moving to the Responses API. If you're using OpenAI Assistants with SalesIQ, you may be wondering if you need to make any changes to your existing setup. You don't. SalesIQ has already taken care of the
                                                                                                                        • Deprecation Notice: OpenAI Assistants API will be shut down on August 26, 2026

                                                                                                                          I recieved this email from openAI what does it means for us that are using the integration and what should we do? Earlier this year, we shared our plan to deprecate the Assistants API once the Responses API reached feature parity. With the launch of Conversations,
                                                                                                                        • Payment for product renewal

                                                                                                                          Good morning, I am requesting your kind assistance to renew my subscription because when I try to make the payment I get this way
                                                                                                                        • All new Address Field in Zoho CRM: maintain structured and accurate address inputs

                                                                                                                          Availability Update: 29 September 2025: It's currently available for all new sign-ups and for existing Zoho CRM orgs which are in the Professional edition exclusively for IN DC users. 2 March 2026: Available to users in all DCs except US and EU DC. 24
                                                                                                                        • API - Available Stock Definitions

                                                                                                                          Okay, Zoho team... your copywriters fell down on the job for this one :) I think these warrant a bit more explanation as to what they include and what they don't.
                                                                                                                        • Feature Request: Add Sort by Cell Color and Text Color functionality

                                                                                                                          Hello Zoho Sheet Development Team, I am writing to suggest a feature enhancement for Zoho Sheet. Currently, the platform supports "Filter by Color," which is incredibly useful. However, there is no native option to "Sort by Color" (by cell fill or text
                                                                                                                        • Zoho Creator to Zoho Writer for prefilled documents...

                                                                                                                          In response to the question about connecting Zoho Creator to Zoho Writer for prefilled documents, I wanted to share a working implementation that demonstrates how to use the record_id parameter with the Zoho Writer Merge API. This allows Writer to automatically
                                                                                                                        • Calendar Year View?

                                                                                                                          Is there a way I can view the calendar in year view? Maybe create a page with a view like this?
                                                                                                                        • Razorpay + Zoho Billing + Zoho Books Integration

                                                                                                                          Please help us set up this integration.
                                                                                                                        • Proyecto SII

                                                                                                                          Buenas tardes amigos de zoho. en mi empresa vamos a implementar el proyecto de emisión de facturas SII pero para poder hacerlo hay que subir un xml, estoy intentando automatizar este proceso pero no se como convertir la factura que retorna books a xml
                                                                                                                        • Automate Workflows at the Subform Level

                                                                                                                          Hello, Enterprise Support Community! We are excited to introduce an enhancement to Workflow Rules that gives you more control over automation within your records. Previously, Workflow Rules could only be configured at the module level, which meant changes
                                                                                                                        • Custom module - change from autonumber to name

                                                                                                                          I fear I know the answer to this already, but thought I'd ask the question. I created a custom module and instead of having a name as being the primary field, I changed it to an auto-number. I didn't realise that all searches would only show this reference.
                                                                                                                        • What is wrong with this sendmail message?

                                                                                                                          I keep getting improper statement errors on the message, but I cannot see a problem. Also, if the message is much longer the whole message turns black instead of being coloured yellow, red and blue. client = Clients[ID == input.Client_Lookup]; sendmail
                                                                                                                        • sales IQ issue on website

                                                                                                                          i integrated the zoho sales IQ code on the website but it is comming in distroted form i am sharing the screenshot below the website is bulit in wix platform
                                                                                                                        • Zoho Creator - Install app from Gallery takes too long

                                                                                                                          I'm trying to install the 'Volunteer Portal' from the app Gallery of Zoho Creator. It has been busy installing the app for over 2 hours now. It might be okay, but how long is something like that supposed to take? I never installed anything from that Gallery
                                                                                                                        • Automate customer follow-ups with Sequences in Bigin

                                                                                                                          Greetings, We hope you're all doing well! We're happy to announce a new feature that we've added to Bigin. Following up with every lead at the right time can be challenging, especially when your team is managing multiple conversations across emails, calls,
                                                                                                                        • How can I load a network into the cliq desktop app?

                                                                                                                          I have both the standard cliq log in for my org and I am part of a cliq network. In the browser I can choose which I log in to. However, in teh desktop app if I log in it will alwasy load my org's cliq. Can I switch this to the network I have create
                                                                                                                        • 4.7.500 Server busy. Please try again later from [136.143.188.52]. (S77719) [SJ1PEPF00001CE2.namprd05.prod.outlook.com 2026-09-04T11:42:24.261Z 08DF0918255760D6]

                                                                                                                          4.7.500 Server busy. Please try again later from [136.143.188.52]. (S77719) [SJ1PEPF00001CE2.namprd05.prod.outlook.com 2026-09-04T11:42:24.261Z 08DF0918255760D6]
                                                                                                                        • Set Up Google Tag Manager for Your Zoho Recruit Career Site

                                                                                                                          Understanding how candidates interact with your career site can help you improve their experience and make better decisions about your recruitment process. With the new Google Tag Manager (GTM) integration in Zoho Recruit, you can connect your career
                                                                                                                        • Gmail users think they've replied to invite when they havent.

                                                                                                                          Hi, I think there's an issue when inviting gmail users to a zoho calendar event. Basically, when an invite from zoho calendar is received by a gmail user, gmail has this functionality where it will render "yes/no/maybe" buttons that, when clicked, they
                                                                                                                        • Translated country names in Address field not sorted alphabetically

                                                                                                                          Hello. I have a form with an address field that includes the country dropdown. The primary form language is English (as is the Zoho Forms account language). The form has been translated into German from the form's Settings > Translation functionality.
                                                                                                                        • Zoho Commerce Microstore to bypass customer portal on spesific items

                                                                                                                          Hi everyone, I’m currently testing the Microstore feature in Zoho Commerce and trying to understand how it behaves with access restrictions. My goal is the following: Even though the customer portal is enabled (and the main store requires users to sign
                                                                                                                        • Calendar Attachments in iOS Mail App

                                                                                                                          It appears that viewing or accessing attachments to a calendar event is not possible in the iOS mail app. Is this correct? The ability to view calendar attachments in the mobile app is vital for me. Is this just a setting I'm missing somewhere?
                                                                                                                        • Zoho Desk API modifiedTimeRange returns HTTP 500 around 2026-03-08T02:00:00.000Z

                                                                                                                          Hello Zoho Support Team, We are experiencing a reproducible HTTP 500 Internal Server Error when querying the Zoho Desk API search endpoint with a specific modifiedTimeRange boundary. ### API Endpoint GET /api/v1/tickets/search ### Reproduction Steps &
                                                                                                                        • Grey border on inserted Sheet on Write document

                                                                                                                          When inserting a Sheet on a Write document, it show an ugly gray border. How to get rid of it?
                                                                                                                        • Send workflow alerts as a single mass email to all recipients

                                                                                                                          Managing workflow notifications becomes easier when everyone needs visibility into who received an alert. With this enhancement, workflow email alerts can now be sent as a single email to all recipients, giving recipients visibility into the other users
                                                                                                                        • Kaizen #253 : Custom Validation for Blueprint to Enforce Mandatory File Uploads and File Extension Compliance Using Client Script

                                                                                                                          Welcome to another exciting Kaizen post on Client Script! Have you ever wanted more control over what gets uploaded to a File or Image field in Zoho CRM, like restricting file types, validating file names, or showing a custom error before an inappropriate
                                                                                                                        • WORK DRIVE CUSTOMIZATION

                                                                                                                          WORK DRIVE CUSTOMIZATION Create folder for every Case by “Contact Name “in Work Drive under Associate Deals Folder Directory: User Work Drive Accounts: Route Directory Associate Cases Case Name, Case Number Client Upload- Shared with contact (for collaboration)
                                                                                                                        • How do I migrate from Office 365 to Zoho Mail?

                                                                                                                          Check out Advik Email Migration Wizard, this software is specially developed to move mailboxes from Office 365 to Zoho Webmail. In addition you can migrate from Gmail, Yahoo, Rediffmail and 80+ webmail servers to ZOHO MAIL. Isn't it amazing? This is an all in one email migration solution. Steps to export emails from Office 365 to Zoho Webmail are as follows; Run Advik Email Migration Tool in your system. Select Office 365 as source and enter its login credentials. Select mailbox folders and choose
                                                                                                                        • 【続々公開】Zoholics Japan 2026 セッション情報を更新しました!

                                                                                                                          ユーザーの皆さま、こんにちは! Zoholics Japan 2026のセッション情報を続々と公開しています! 今年は3つのトラックで、Zohoの最新情報やAI活用、製品アップデート、 活用事例など、多彩なセッションをお届けします。 現在公開中のセッション(一部) ・Zoho CRM 製品アップデート ・Zoho CRM Plus 製品アップデート ・Zoho Community セッション ・Zoho Desk セッション ・中堅・成長企業の人事DX ~AI時代に何から始めるべきか?~ ・Zoho
                                                                                                                        • Next Page