Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Register for Zoho Desk Beta Community

                                                                With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                              • Share your Zoho Desk story with us!

                                                                Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                              • Tip #1: Learn to pick the right channels

                                                                Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                              • Welcome to Zoho Desk Community - Say hello here!

                                                                Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                              • Webinar 1: Blueprint for Customer Service

                                                                With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • Canvas View bug

                                                                                                                  I would like to report a bug. When clone a canvas view from an existing canvas view, if the original canvas view have canvas button with client script. Then the new create canvas view will have canvas button, it is make sense. But when I try to delete
                                                                                                                • Integrating Zoho Suite and apps more with Linux

                                                                                                                  I just got introduced with Zoho just couple of months ago, and I've already planned to contribute to it, even though it's not an open-source software. Still I have found it's potential to beat the tech giants and still being respective towards data privacy
                                                                                                                • Store Submitted Forms as PDF

                                                                                                                  Hi there, Is there a method to store Form submissions as a PDF in workdrive for archival? I set the form to email the submission to the user and an internal inbox but I'd really love to store the final form as PDF in workdrive automatically. I can't seem
                                                                                                                • Separate Items & Services

                                                                                                                  Hi, please separate items and services into different categories. Thank you
                                                                                                                • CRM->INVENTORY, sync products as composite items

                                                                                                                  We have a product team working in the CRM, as it’s more convenient than using Books or Inventory—especially with features like Blueprints being available. Once a product reaches a certain stage, it needs to become visible in Inventory. To achieve this,
                                                                                                                • Add an option to disable ZIA suggestions

                                                                                                                  Currently, ZIA in Zoho Inventory automatically provides suggestions, such as sending order confirmation emails. However, there is no way to disable this feature. In our case, orders are automatically created by customers, and we’ve built a custom workflow
                                                                                                                • What's new in Zoho Sheet: Simplify data entry and collaboration

                                                                                                                  Hello, Zoho Sheet community! Last year, our team was focused on research and development so we could deliver updates that enhance your spreadsheet experience. This year, we’re excited to deliver those enhancements—but we'll be rolling them out incrementally
                                                                                                                • Zoho Creator Developer Console | Improved Distribution and Lifecycle Management for apps

                                                                                                                  Hello everyone, We're excited to introduce new enhancements now in the Zoho Creator Developer Console. These updates strengthen private app distribution through licensing controls and extend environment support across all installed apps, helping teams
                                                                                                                • [Webinar] Solving business challenges- Handling the challenge of transitioning to Zoho Writer from legacy tools

                                                                                                                  Moving to Zoho Writer is a great way to consolidate your business tools and become more agile. With multiple accessibility modes, no-code automation, and extensive integration with business apps and content platforms, Zoho Writer helps solve your organization's
                                                                                                                • Feature Request - Gift Cards or Gift Voucher Capability in Zoho Commerce

                                                                                                                  Hi Zoho Commerce team, I'm comming accross more and more retail businesses who sell gift cards. As there is currently no way to manage this in Zoho Commerce, it is a blocker to addoption. This is particularly popular in Europe and North America. I recently
                                                                                                                • Highlight a candidate who is "off limits"

                                                                                                                  Hello: Is there a way to highlight a candidate who is "off limits"?  I would like to have the ability to make certain candidate and / or Client records highlighted in RED or something like that.   This would be used for example when we may have placed a candidate somewhere and we want everyone in our company to quickly and easily see that they are off limits.  The same would apply when we want to put a client or former client off limits so no one recruits out of there. How can this be done? Cheers,
                                                                                                                • Closing off tasks automatically based on project status

                                                                                                                  Is there a way to close off all open tasks when a project is set as Cancelled?
                                                                                                                • [Free webinar] Understanding connections and datasources in Zoho Creator - Creator Tech Connect

                                                                                                                  Hello everyone, We’re excited to invite you to another edition of the Creator Tech Connect webinar. About Creator Tech Connect The Creator Tech Connect series is a free monthly webinar featuring in-depth technical sessions designed for developers, administrators,
                                                                                                                • Bridge the Gap Between your Service and Finance Teams

                                                                                                                  In field service, the most expensive distance isn’t the drive to a customer’s home—it’s the gap between a finished job and a paid invoice. Zoho FSM leverages the power of Zoho Invoice to handle and manage billing operations. It enables you to: Instantly
                                                                                                                • Direct URL to "View Record"

                                                                                                                  I would like to create a button (Deluge Script) to jump to the "view record" without passing by the record menu, but I can not aford to find the right url. I don't want the "view record" as a single page. I want the "view record" in the application like the picture below: Thanks in advance for your help
                                                                                                                • Live Chat for user

                                                                                                                  Hi everyone, I’m new to Zoho Creator and wanted to ask if it’s possible to add a live chat option for all logged-in portal users so they can chat internally. I’m trying to create a customer portal similar to a service desk, but for vehicle breakdowns,
                                                                                                                • SalesIQ Chat Owner to CRM Lead Owner Mapping

                                                                                                                  There is no proper mapping between the SalesIQ chat owner and the CRM lead owner. When a chat is assigned to an agent, the lead created in CRM is often assigned to a different user, forcing admins to manually change ownership every time. This creates
                                                                                                                • Product line search in quotes

                                                                                                                  Is there a way to change the search setting when you add a product line to a quote. We have created a field in products called 'Part Number' and I would like when I'm adding a product line to the quote that I can search on that field. At the moment it only searches on product description. In products I can search on the 'Part Number' field Any help welcome.
                                                                                                                • 🚀 WorkDrive 6.0 (Phase 1): Empowering Teams with Content Intelligence, Automation, Accessibility, and Control

                                                                                                                  Hello, everyone! WorkDrive continues to evolve from a robust file management solution into an intelligent, secure, and connected content collaboration platform for modern businesses. Our goal remains unchanged: to simplify teamwork, strengthen data security,
                                                                                                                • Storage Limits

                                                                                                                  Hello, What is the storage limitations for each user on professional edition?  Thanks, Mekki
                                                                                                                • Zoho Recruit Community Meetup - London 🇬🇧

                                                                                                                  Hello Recruiters! We’re excited to announce that the Zoho Recruit team is coming to the UK for an in-person Zoho User Group (ZUG) Meetup in London! This is your chance to connect with fellow Zoho users, learn from experts, and walk away with actionable
                                                                                                                • Power Pivot and Data Modeling functionality in Zoho Sheet

                                                                                                                  When will MS Excel functionalities like Power Pivot and Data Modeling functionalities be available in Zoho Sheet?
                                                                                                                • Global Search placement in the new UI

                                                                                                                  Having a hard time with the global search placement in the UI redesign. Surely I can't be the only one. Previously global search placement was perfect. A bar at the top/center of the page. Exactly where you would expect it to be. Since the new UI has
                                                                                                                • What's New - January 2026 | Zoho Backstage

                                                                                                                  Welcome to 2026! If you thought last year was big, buckle up. We’re stepping into this year with a singular focus: unimaginable growth, elite scalability, and performance benchmarks that leave the old ways of event management in the rearview mirror. As
                                                                                                                • Cliq on your mobile phone

                                                                                                                  Some of the employees in my company get their cliq on their mobile phones but it will not let them answer the call. The employee has to go to their laptop to access it. 
                                                                                                                • Bulk Delete Messages in Cliq

                                                                                                                  Isn't it possible to bulk delete messages in Cliq? I have them delivered to me as notifications from another service but most of them need to be deleted as not of value. I get dozens of them per day, and deleting them is already difficult is the "Delete"
                                                                                                                • stock movement report by each transaction for each item

                                                                                                                  Dear Sir we are using zoho inventory for foodstuff and we lack the transaction report which clearly shows movement of an item from Opening balance of the month to Goods received in, Supplier return out, Invoice out, Sales return In, Transfer out and Transfer
                                                                                                                • Select the task view on the board in the Zoho Connect iPhone app

                                                                                                                  Hello. How do I select the task view on the board in the Zoho Connect iPhone app? The Android app has this functionality.
                                                                                                                • Unable to change Company Logo

                                                                                                                  Hi Guys, I have an issue where I've uploaded a Company image in Gif format and the logo has now got a black background whereas the logo is supposed to be black not the background. I didn't think it would be an issue so I created a PNG version to upload
                                                                                                                • Metadata API Access to Functions

                                                                                                                  I think it would be incredibly helpful to have api access to every function's code. Our team primarily uses deluge functions to update fields across modules according to business logic. I would like to create a visual dependancy model for our CRM, but
                                                                                                                • Historical Sales Info - Blend with Finance Invoice Line Items, Access in CRM and Desk

                                                                                                                  My company has been using Zoho One since 2021, with sales data going back through 2020. However, we have been in business much longer, and we have historical sales information that we want to have at our fingertips when talking with customers (usually
                                                                                                                • Introducing Record Category in CRM: Group options to see record status at a glance.

                                                                                                                  Release update: Currently available for CN, JP, and AU DCs (all paid editions). It will be made available to other DCs by mid-March. Hello everyone, We are pleased to introduce Record Category in Zoho CRM - a new capability where the user can get an overview
                                                                                                                • Change Last Name to not required in Leads

                                                                                                                  I would like to upload 500 target companies as leads but I don't yet have contact people for them. Can you enable the option for me to turn this requirement off to need a Second Name? Moderation update (10-Jun-23): As we explore potential solutions for
                                                                                                                • Service disruption in Zoho Cliq (EU Region) – February 9, 2026

                                                                                                                  Hello everyone, We’d like to inform you about a brief service disruption that affected Zoho Cliq users in the EU region earlier today. The issue has been resolved, and all services are now operating normally. Summary On February 9, 2026, Zoho Cliq experienced
                                                                                                                • Spotlight series #26 - Build slides with visual clarity using Diagram

                                                                                                                  Hello all! The feature in the spotlight this month is Diagram. Let's say you are a part of a project kickoff meeting and your teammate starts explaining the project life cycle like this: "First we ideate, then we create an initial prototype and share
                                                                                                                • Client scripts for Zoho Books ?

                                                                                                                  Good day everyone, I am looking for a way to be able to interact with the Quotes and Invoices as they are being created. Think of it like Zoho client script in Zoho CRM. But for the life of me I dont see a way to do this. The issue with having function
                                                                                                                • Uplifted homepage experience

                                                                                                                  Editions: All editions. Release Note: Available in CA and SA DCs (Standard, Professional). Availability update (17th February 2026): The feature has been enabled for the following DCs: All editions in the CA and SA DC | Free, Standard and Professional
                                                                                                                • Unable to produce monthly P&L reports for previous years

                                                                                                                  My company just migrated to Books this year. We have 5+ years financial data and need to generate a monthly P&L for 2019 and a monthly P&L YTD for 2020. The latter is easy, but I'm VERY surprised to learn that default reports in Zoho Books cannot create
                                                                                                                • Crossbeam

                                                                                                                  Does anyone use Crossbeam with their Zoho CRM? I'm looking for a way to import Crossbeam partner leads into Zoho CRM. If so: - What's your experience been like? - Are you able to automatically import Crossbeam leads > Zoho CRM? How? - What doesn't work
                                                                                                                • Access forms within a form

                                                                                                                  I have multiple forms that I would like to create access to for users within one form. Example: I have forms A, B, C, D. I would like to create a form with the links to forms A, B, C, & D. That way we can send one link and then the user can select the
                                                                                                                • Next Page