Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team



      Zoho Campaigns Resources


        • Desk Community Learning Series


        • Digest


        • Functions


        • Meetups


        • Kbase


        • Resources


        • Glossary


        • Desk Marketplace


        • MVP Corner


        • Word of the Day


        • Ask the Experts


          • Sticky Posts

          • Register for Zoho Desk Beta Community

            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
          • Share your Zoho Desk story with us!

            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
          • Tip #1: Learn to pick the right channels

            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
          • Welcome to Zoho Desk Community - Say hello here!

            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
          • Webinar 1: Blueprint for Customer Service

            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

          Zoho CRM Plus Resources

            Zoho Books Resources


              Zoho Subscriptions Resources

                Zoho Projects Resources


                  Zoho Sprints Resources


                    Zoho Orchestly Resources


                      Zoho Creator Resources


                        Zoho WorkDrive Resources



                          Zoho CRM Resources

                          • CRM Community Learning Series

                            CRM Community Learning Series


                          • Tips

                            Tips

                          • Functions

                            Functions

                          • Meetups

                            Meetups

                          • Kbase

                            Kbase

                          • Resources

                            Resources

                          • Digest

                            Digest

                          • CRM Marketplace

                            CRM Marketplace

                          • MVP Corner

                            MVP Corner




                            Zoho Writer Writer

                            Get Started. Write Away!

                            Writer is a powerful online word processor, designed for collaborative work.

                              Zoho CRM コンテンツ



                                ご検討中の方

                                  • Recent Topics

                                  • Clone / Export Popup Design Across PageSense Projects

                                    Hello Zoho PageSense Team, We hope you’re doing well. We would like to request an enhancement that allows popup designs to be reused across different PageSense projects. Problem Statement: Currently, Zoho PageSense allows popups to be duplicated only
                                  • Are there settings for hyperlinks?

                                    Clicking a hyperlinked cell in Sheet creates this little pop-up with the actual hyperlink inside. Is it possible to have a 1-click link where if you click the cell it opens the link directly with no pop-up?
                                  • Automatically include all ticket attachments in the ticket resolution email

                                    Hello Zoho Community, We are implementing Zoho Desk in a real customer-facing production environment and have run into a limitation that is becoming a blocking requirement for our clients. The problem When a ticket is closed or resolved, Zoho Desk sends
                                  • Finding text within a ticket: Expand All or Search this Ticket

                                    The auto-collapse feature within a ticket is nice for screen scrolling, however it makes it difficult to find text within the ticket if the email is collapsed. In fact you cannot find text if it is collapsed. I would like to propose a feature that allows
                                  • Books & Desk. Client mapping

                                    Hi, I’ve been using Zoho Books for several years and am now looking to improve my customer service. I'm experimenting with Zoho Desk and want to sync and map my client data from Zoho Books. However, it seems that mapping requires both contacts to have
                                  • String handling

                                    If I cut a currency string from a quote and try and paste it into the Deal "Amount", it will fail unless I manually delete any commas. Dollar signs are no problem, but comma's seem to fail. Please correct this Input Validation error.
                                  • What's new in Zoho Sheet: Simplify data entry and collaboration

                                    Hello, Zoho Sheet community! Last year, our team was focused on research and development so we could deliver updates that enhance your spreadsheet experience. This year, we’re excited to deliver those enhancements—but we'll be rolling them out incrementally
                                  • WebDAV / FTP / SFTP protocols for syncing

                                    I believe the Zoho for Desktop app is built using a proprietary protocol. For the growing number of people using services such as odrive to sync multiple accounts from various providers (Google, Dropbox, Box, OneDrive, etc.) it would be really helpful
                                  • Feature Request - Allow Customers To Pick Meeting Duration

                                    Hi Bookings Team, It would be great if there was an option to allow customers to pick a duration based on a max and minimum amount of time defined by me and in increments defined by me. For example, I have some slots which are available for customers
                                  • Support for Custom Fonts in Zoho Recruit Career Site and Candidate Portal

                                    Dear Zoho Recruit Team, I hope you're doing well. We would like to request the ability to use custom fonts in the Zoho Recruit Career Site and Candidate Portal. Currently only the default fonts (Roboto, Lato, and Montserrat) are available. While these
                                  • YouTube Live streaming? how to? Zoom has this feature, built-in. Can't find it on zoho meetings.

                                    YouTube Live streaming? how to? Zoom has this feature, built-in. Can't find it on zoho meetings.
                                  • Feature Request - A Way To Search Item Groups

                                    Hi Inventory Team, I can't find any way to filter or search by fields of Item Groups. It would be great to see that functionality added. I have a use case where a single product might come from 5 or more suppliers and each supplier's item is an Item in
                                  • Feature Reqeust - Include MPN In Selectable FIelds

                                    I have noticed that the MPN is not available to show in the list view of Items. Please consider adding it as EAN, UPC and ISBN are all available, so it doesn't make much sense to exclude this similar option. Thanks for considering my feedback.
                                  • Feature Request - Option To Hide Default System Fields on Items

                                    Hi Zoho Inventory Team, As far as I know it is not possible to hid some of the defult system fields on Items, such as UPC, MPN, EAN, ISBN. A good use case is that in many cases ISBN is not relevant and it would be an improved user experience if we could
                                  • Making an email campaign into a Template

                                    I used a Zoho Campaign Template to create an email. Now I want to use this email and make it a new template, but this seems to be not possible. Am I missing something?
                                  • Campaigns does not work!

                                    I am running into so many problems trying to use Zoho Campaigns, that I am seriously considering dropping the app from my (shrinking) list of Zoho applications I actually use. Apart from having to fight the software trying to create a design and email,
                                  • Feature Request - Make Available "Alias Name" Field In Item List View

                                    Hi Zoho Inventory Team, I have noticed that the "Alias Name" field does not appear on the list of selectable columns in the Customise Columns feature in the Items module. This would be very useful to see for businesses who are using the Alias Name field
                                  • Marketing Automation

                                    L.S. Marketing Automation is and has always been part of the Zoho One bundle - according to the information provided on the Zoho Website. Why when I open Marketing Automation do I get the following message?: "Your trial has expired. We hope you enjoyed
                                  • Cliq iOS can't see shared screen

                                    Hello, I had this morning a video call with a colleague. She is using Cliq Desktop MacOS and wanted to share her screen with me. I'm on iPad. I noticed, while she shared her screen, I could only see her video, but not the shared screen... Does Cliq iOS is able to display shared screen, or is it somewhere else to be found ? Regards
                                  • Bigin, more powerful than ever on iOS 26, iPadOS 26, macOS Tahoe, and watchOS 26.

                                    Hot on the heels of Apple’s latest OS updates, we’ve rolled out several enhancements and features designed to help you get the most from your Apple devices. Enjoy a refined user experience with smoother navigation and a more content-focused Liquid Glass
                                  • project name field issue- n8n

                                    Hey guys, I have a question. I want to create a new product using the workflow. The problem is with the product name field; I don't know how to fill it in. The workflow starts with retrieving information from the leads table, retrieving links to scrape
                                  • Automatic Matching from Bank Statements / Feeds

                                    Is it possible to have transactions from a feed or bank statement automatically match when certain criteria are met? My use case, which is pretty broadly applicable, is e-commerce transactions for merchant services accounts (clearing accounts). In these
                                  • How to filter Packages in zoho inventory api

                                    Hi Team, I want to perform some tasks in a schedular on the packages which are in "Shipped" state. I tried to use filter_by in my api call but in return I get response as {"code":-1,"message":"Given filter is not configured"} My Api request is as follows
                                  • CRM

                                    Is anyone else experiencing this issue? Our company is not moving out of using Gmail's web app. It just has more features and is a better email program than Zoho Mail. Gmail has an extension (Zoho CRM for Gmail) that we're using but we've found some serious
                                  • ZOHO add-in issue

                                    I cannot connect ZOHO from my Outlook. I am getting this error.
                                  • Syncing with Google calendar, Tasks and Events

                                    Is it possible to sync Zoho CRM calendar, task and events with Google Calendar's tasks and events. With the increasing adoption by many major tool suppliers to sync seamlessly with Google's offerings (for instance I use the excellent Any.do task planning
                                  • How can i view "Child" Accounts?

                                    It can be very useful in our field of business to know the parent-child account relationship. However, there seems to be a shortcoming in the parent account view: no child account list. How can we view the child accounts per each account?
                                  • Ability to assign Invoice Ownership through Deluge in FSM

                                    Hi, As part of our process, when a service appointment is completed, we automated the creation of the invoice based on a specific business logic using Deluge. When we do that, the "Owner" of the invoice in Zoho FSM is defaulted to the SuperAdmin. This
                                  • Easily perform calculations using dates with the new DATEDIF function

                                    Hey Zoho Writer users! We've enhanced Zoho Writer's formula capabilities with the new DATEDIF function. This allows you to calculate the difference between dates in days, months, and years. Function syntax: =DATEDIF(start_date, end_date, unit) Inputs:
                                  • Adding Comments Using Workflows - How to Change User Attributed

                                    We have worklflows in Desk where a comment is added to a ticket based on certain criteria. It seems that the comment added is always attributed to the user who last edited the workflow. This does not make sense for us because: - It's misleading to other
                                  • Add Attachment Support to Zoho Flow Mailhook / Email Trigger Module

                                    Dear Zoho Support Team, We hope you are well. We would like to kindly request a feature enhancement for the Mailhook module in Zoho Flow. Currently, the email trigger in Zoho Flow provides access to the message body, subject, from address, and to address,
                                  • Conect chat of salesiq with zoho cliq

                                    Is there any way to answer from zoho cliq  the chat of salesiq initiated by customers?
                                  • Les dernières avancées en saisie de données et collaboration

                                    Après une année dédiée à la recherche et au développement, notre équipe est prête à dévoiler des améliorations majeures pour Zoho Sheet. Ces nouveautés seront lancées par étapes afin d’en assurer une prise en main optimale. Nous commençons avec des fonctionnalités
                                  • Deluge Learning Series – Client functions in Deluge | January 2026

                                    We’re excited to kick-start the first session of the 2026 Deluge Learning Series (DLS) with Client functions in Deluge. For those who are new to DLS, here’s a quick overview of what the series is all about: The Deluge Learning Series takes place on the
                                  • Rich Text For Notes in Zoho CRM

                                    Hello everyone, As you know, notes are essential for recording information and ensuring smooth communication across your records. With our latest update, you can now use Rich Text formatting to organize and structure your notes more efficiently. By using
                                  • Implement Meeting Polls in Zoho Bookings

                                    Dear Zoho Bookings Support Team, We'd like to propose a feature enhancement related to appointment scheduling within Zoho Bookings. Current Functionality: Zoho Bookings excels at streamlining individual appointment scheduling. Users can set availability
                                  • Zoho Bookings and Survey Integration through Flow

                                    I am trying to set up flows where once an appointment is marked as completed in Zoho Bookings, the applicable survey form would be sent to the customer. Problem is, I cannot customise flows wherein if Consultation A is completed, Survey Form A would be
                                  • Service Account Admin for API Calls and System Actions

                                    Hello, I would like to request the addition of a Service Account Admin option in Zoho product. This feature would allow API calls and system actions to be performed on behalf of the system, rather than an active user. Current Issue: At present, API calls
                                  • How to apply customized Zoho Crm Home Page to all users?

                                    I have tried to study manuals and play with Zoho CRM but haven't found a way how to apply customized Zoho CRM Home Page as a (default) home page for other CRM users.. How that can be done, if possible? - kipi Moderation Update: Currently, each user has
                                  • Please can the open tasks be shown in each customer account at the top.

                                    Hi there This has happened before, where the open tasks are no longer visible at the top of the page for each customer in the CRM. They have gone missing previously and were reinstated when I asked so I think it's just after an update that this feature
                                  • Next Page