Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.






                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Register for Zoho Desk Beta Community

                                                                With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                              • Share your Zoho Desk story with us!

                                                                Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                              • Tip #1: Learn to pick the right channels

                                                                Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                              • Welcome to Zoho Desk Community - Say hello here!

                                                                Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                              • Webinar 1: Blueprint for Customer Service

                                                                With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ






                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • Cloning Item With Images Or The Option With Images

                                                                                                                  Hello, when I clone an item, I expect the images to carry over to the cloned item, however this is not the case in Inventory. Please make it possible for the images to get cloned or at least can we get a pop up asking if we want to clone the images as
                                                                                                                • Cliq iOS can't see shared screen

                                                                                                                  Hello, I had this morning a video call with a colleague. She is using Cliq Desktop MacOS and wanted to share her screen with me. I'm on iPad. I noticed, while she shared her screen, I could only see her video, but not the shared screen... Does Cliq iOS is able to display shared screen, or is it somewhere else to be found ? Regards
                                                                                                                • Zoho Books | Product updates | May 2026

                                                                                                                  Hello users, We're back with the latest updates and enhancements we've rolled out in Zoho Books. From sales tax automation to scanning receipts for free, explore the updates designed to upgrade your bookkeeping experience. Sales Tax Automation [US & Canada
                                                                                                                • Blueprint Not Triggering When Lead Status Is Updated by Workflow (IndiaMART Integration)

                                                                                                                  I have set up a blueprint that triggers when a lead’s status is “New Lead.” Our CRM is integrated with IndiaMART, and when leads are created from IndiaMART, their Lead Status is initially set to None. To handle this, I created a workflow that automatically
                                                                                                                • Zoho Books | Product updates | June 2026

                                                                                                                  Hello users, Welcome to this month's roundup of what's new in Zoho Books! We have an exciting line-up this time. The highlight is the launch of the all-new France Edition with full ISCA compliance. We're also introducing features such as Layout Rules
                                                                                                                • Cloning a View

                                                                                                                  When I clone a View, it doesn't make a copy; it only creates a new copy with the same default fields as if I were creating a new view. What is the purpose of cloning if it doesn't bring in the same fields? Thanks Rudy
                                                                                                                • Does Zoho Learn integrate with Zoho Connect,People,Workdrive,Project,Desk?

                                                                                                                  Can we propose Zoho LEarn as a centralised Knowledge Portal tool that can get synched with the other Zoho products and serve as a central Knowledge repository?
                                                                                                                • Request to Update Billing Information and Payment Method

                                                                                                                  Hello, I’m using Zoho and I would like to update the billing information and change the payment card to our company card. Could you please let me know how I can do this? Thank you in advance for your help.
                                                                                                                • Zoho Analytics "Esc" key problem

                                                                                                                  I frequently use the Escape (Esc) key while building dashboards, reports, and writing SQL queries. Since the recent updates to Zoho Analytics, the Esc key no longer behaves as expected. When writing SQL queries, pressing Esc to dismiss a suggestion now
                                                                                                                • Zoho Analytics Filter Bug

                                                                                                                  I encountered a bug where typing the letter "A" in the drop-down filter of a table or query table causes the drop-down to close unexpectedly. For example, when typing "Today", the drop-down list closes as soon as "a" is entered. I tested this on another
                                                                                                                • Zoho CRM Kiosk question – Passing Screen Fields to a Function

                                                                                                                  I am building a Kiosk in Zoho CRM to create new Supplier (Vendors) records. Current setup: Screen 1 contains user input fields: Supplier Name (Vendor_Name) First Name (First_Name) I created a Deluge function: createSupplier(vendor_name, first_name) The
                                                                                                                • Using Email Triggers on Zoho Flow

                                                                                                                  Hello, I'm sending the email to create the variables as this article says: https://help.zoho.com/portal/en/kb/flow/user-guide/create-a-flow/articles/email-trigger#How_email_trigger_works But the collection of the variables only seems to work when the
                                                                                                                • Number of Reopn

                                                                                                                  Hi Zoho, Is there any appropriate API call for This URL "http://support.zoho.com/api/v1/dashboards/reopenedTickets?...." what I thought is the resulting output of this call has data for number of reopen... "https://desk.zoho.com/api/v1/tickets/" + Ticket_ID
                                                                                                                • How to customize the "Placeholder Text" separately from the "Field Label" on the Booking Form?

                                                                                                                  Hi, I am currently customizing the Booking Form for one of my Workspaces in Zoho Bookings, and I need some help adjusting a custom text field. Right now, when I create a custom text field, the gray "placeholder text" inside the text box automatically
                                                                                                                • What's New in Zoho Inventory | April & May 2026

                                                                                                                  Hello users, We're excited to roll out the latest Zoho Inventory updates for April and May 2026. These enhancements are designed to make your daily operations smoother and more efficient, from advanced inventory management and flexible pricing to automated
                                                                                                                • Introducing Spotlight Forms

                                                                                                                  Hey form builders! If someone opens your form, sees the wall of fields ahead, and quietly closes the tab. It may not be because the questions were hard. It could be because the experience felt like too much. Which is why we have now introduced a new form
                                                                                                                • Workflow Assistance in Zoho CRM

                                                                                                                  Our client's sales team visits customers on-site and currently fills a physical paper form to capture customer details, and then separately re-enters the same data into Zoho CRM via the mobile app — resulting in double data entry. We want the salesperson
                                                                                                                • Related products & AI product recommendations through commerce API.

                                                                                                                  Hello Zoho team I’m looking to add related products and AI product recommendations to my Zoho Commerce webshop with custom storefront. Is this supported through the API? And if not, is this on your roadmap? Thanks in advance David
                                                                                                                • Why don't Zia agents support file uploads?

                                                                                                                  I am trying to build a Zia Agent that allows uploading of a PDF file and uses the GLM5 model to process it and extract information. But agents.zoho.com has no way to enable file uploads on the agent. Additionally, GLM5 based agents keep outputting their
                                                                                                                • Pasting Images in Zoho Desk ignores cursor location

                                                                                                                  My team has reported an issue which started recently where when we paste an image into a new or existing reply or comment, the pasted image seems to ignore the current cursor location instead paste itself at the last character present in the reply/comment,
                                                                                                                • 'Pinned' notes feature of a pipeline record

                                                                                                                  Hi team, Could you please implement a feature which will allow users to pin different notes so that they will appear at the very top of the notes tab in a pipeline record. Sometimes we have a wide range of notes on a record which means more important
                                                                                                                • Canvas Detail View Related List Sorting

                                                                                                                  Hello, I am having an issue finding a way to sort a related list within a canvas detail view. I have sorted the related list on the page layout associated with the canvas view, but that does not transfer to the canvas view. What am I missing?
                                                                                                                • Announcing new features in Trident for Mac (1.37.0)

                                                                                                                  Hello everyone! We’re excited to introduce the latest updates to Trident, which are designed to take workplace communication to the next level. Let’s dive into the details. Import EML archives directly into Trident. You can now import EML archives into
                                                                                                                • Zia Agent activation in Zoho Desk forces new Organization creation instead of deploying to existing one

                                                                                                                  While attempting to complete the deployment and activation sequence of a new Zia Agent within our existing Zoho Desk environment, the activation process failed on the user interface, throwing a generic error (see print). However, despite the activation
                                                                                                                • #10 Bill While You Sleep

                                                                                                                  A consultant is reviewing last month's work. Client meetings? Done. Deliverables? Sent. Support requests? Resolved. Then they realize something. "I have completed the work... but I haven't billed the client yet." The work was completed. The client was
                                                                                                                • Team Module Issues?

                                                                                                                  We are testing Team Licenses for use by our Customer Service staff. I created a Teamspace called CSR and only assigned two users to this space: Administrator (me) and “Team License Test.” Team License Test is assigned to the Team User profile, with a
                                                                                                                • Access images from form submission in power automate

                                                                                                                  Images from form submission show up as links in power automate. How do I access the image data?
                                                                                                                • Forms cannot be accessed.

                                                                                                                  https://forms.zoho.com/ is not available, please help to fix
                                                                                                                • Associate records via the Multi-select lookup RELATED LIST via API

                                                                                                                  In the REST API, is there a way to associate records for a multi-select lookup related list other than via the linking module? There are two methods for the lookup: 1. via insert records API 2. via the linking module ...as described in https://help.zoho.com/portal/en/community/topic/kaizen-125-manipulating-multi-select-lookup-fields-mxn-using-zoho-crm-apis
                                                                                                                • Problem with CRM Connection not Refreshing Token

                                                                                                                  I've setup a connection with Zoom in the CRM. I'm using this connection to automate some registrations, so my team doesn't have to manually create them in both the CRM and Zoom. Connection works great in my function until the token expires. It does not refresh and I have to manually revoke the connection and connect it again. I've chatted with Zoho about this and after emailing me that it couldn't be done I asked for specifics on why and they responded. "The connection is CRM is not a feature to
                                                                                                                • How do I post a new question in Zoho Community forums?

                                                                                                                  Hi everyone, I’m new to the Zoho Community and I’m trying to figure out how to properly create and publish a new topic in the forum. When I visit the community page, I can’t clearly find the option like “Add Topic” or “Post Question.” Could someone guide
                                                                                                                • Kaizen #245 - Real Time Signal Alerts for High-Value Abandoned Checkouts

                                                                                                                  Howdy, Tech Wizards! Welcome back to another week of Kaizen. In this post, we will build a real-time abandoned checkout notification system using Stripe, Zoho CRM Functions, Sales Signals, and Widgets. When a customer abandons a high-value purchase, Zoho
                                                                                                                • Unable to attach Fillable File Upload field to Merge Template ever since UI update

                                                                                                                  Ever since the new UI update, the field for Attachments for sending document for Signing in Writer has had an issue where trying to add a Fillable item in the Attachment field ends up always becoming a "Choose a File From Drive" option instead. No matter
                                                                                                                • Latest updates in Zoho Meeting | An improved Analytics tab and user interface, an invite pop-up revamp, an enhanced Zoho Meeting iOS app, a recording feature in the Android app, and more

                                                                                                                  Hello everyone, We’re excited to share a few updates and enhancements in Zoho Meeting. Here's what we've been working on lately: Improved analytics for meetings, an invite pop-up revamp, a multi-video feed interface in the iOS app, a recording feature
                                                                                                                • Inquiry Regarding Automated Assignment of Zoho TeamInbox Messages using Zoho Flow and Deluge

                                                                                                                  Hello, Our company is currently using Zoho TeamInbox, and we are interested in automating the assignment of responsible parties using tools such as ZOHO Flow and Deluge. Is it possible to achieve this? Allow me to provide more details. Currently, when
                                                                                                                • Kaizen #125 Manipulating Multi-Select Lookup fields (MxN) using Zoho CRM APIs

                                                                                                                  Hello everyone! Welcome back to another week of Kaizen. In last week's post in the Kaizen series, we discussed how subforms work in Zoho CRM and how to manipulate subform data using Zoho CRM APIs. In this post, we will discuss how to manipulate a multi-select
                                                                                                                • [Bug] WebAuthn passkey registration blocked on rpIds with TLDs longer than 6 characters (.accountant, .technology, etc.) — isValidDomain regex too strict

                                                                                                                  Hi, Filing on behalf of an enterprise customer where Zoho Vault is deployed across the company. The Chrome extension blocks WebAuthn passkey registration on legitimate sites whose Relying Party ID (rpId) has a TLD longer than 6 letters. This affects every
                                                                                                                • Celebrating the businesses behind Bigin: Customer Awards 2026

                                                                                                                  Hello Biginners, We're excited to announce the very first Bigin Customer Awards! If Bigin has played a role in your organization's journey, we'd love to hear about it. Share your story for a chance to be recognized among the best Bigin users across industries.
                                                                                                                • Client Script Button in Related List become invalid

                                                                                                                  Hi, I am the admin of our organization. And I setup a client script button in related list to raise payment refund request While this button become non selectable recently. I believe there is something wrong from zoho as this button had run for a year.
                                                                                                                • Send Email Directly to Channel

                                                                                                                  Hi, We are coming from Slack. In Slack each channel has a unique Email address that you can send emails too. I currently forward a specific type of email from my Gmail InBox directly do this channel for Verification Codes so my team doesn't have to ask
                                                                                                                • Next Page