Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team



      Zoho Campaigns Resources


        • Desk Community Learning Series


        • Digest


        • Functions


        • Meetups


        • Kbase


        • Resources


        • Glossary


        • Desk Marketplace


        • MVP Corner


        • Word of the Day


        • Ask the Experts


          • Sticky Posts

          • Register for Zoho Desk Beta Community

            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
          • Share your Zoho Desk story with us!

            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
          • Tip #1: Learn to pick the right channels

            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
          • Welcome to Zoho Desk Community - Say hello here!

            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
          • Webinar 1: Blueprint for Customer Service

            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

          Zoho CRM Plus Resources

            Zoho Books Resources


              Zoho Subscriptions Resources

                Zoho Projects Resources


                  Zoho Sprints Resources


                    Zoho Orchestly Resources


                      Zoho Creator Resources


                        Zoho WorkDrive Resources



                          Zoho CRM Resources

                          • CRM Community Learning Series

                            CRM Community Learning Series


                          • Tips

                            Tips

                          • Functions

                            Functions

                          • Meetups

                            Meetups

                          • Kbase

                            Kbase

                          • Resources

                            Resources

                          • Digest

                            Digest

                          • CRM Marketplace

                            CRM Marketplace

                          • MVP Corner

                            MVP Corner




                            Zoho Writer Writer

                            Get Started. Write Away!

                            Writer is a powerful online word processor, designed for collaborative work.

                              Zoho CRM コンテンツ



                                ご検討中の方

                                  • Recent Topics

                                  • Rich Text For Notes in Zoho CRM

                                    Hello everyone, As you know, notes are essential for recording information and ensuring smooth communication across your records. With our latest update, you can now use Rich Text formatting to organize and structure your notes more efficiently. By using
                                  • Implement Meeting Polls in Zoho Bookings

                                    Dear Zoho Bookings Support Team, We'd like to propose a feature enhancement related to appointment scheduling within Zoho Bookings. Current Functionality: Zoho Bookings excels at streamlining individual appointment scheduling. Users can set availability
                                  • Zoho Bookings and Survey Integration through Flow

                                    I am trying to set up flows where once an appointment is marked as completed in Zoho Bookings, the applicable survey form would be sent to the customer. Problem is, I cannot customise flows wherein if Consultation A is completed, Survey Form A would be
                                  • Service Account Admin for API Calls and System Actions

                                    Hello, I would like to request the addition of a Service Account Admin option in Zoho product. This feature would allow API calls and system actions to be performed on behalf of the system, rather than an active user. Current Issue: At present, API calls
                                  • How to apply customized Zoho Crm Home Page to all users?

                                    I have tried to study manuals and play with Zoho CRM but haven't found a way how to apply customized Zoho CRM Home Page as a (default) home page for other CRM users.. How that can be done, if possible? - kipi Moderation Update: Currently, each user has
                                  • Please can the open tasks be shown in each customer account at the top.

                                    Hi there This has happened before, where the open tasks are no longer visible at the top of the page for each customer in the CRM. They have gone missing previously and were reinstated when I asked so I think it's just after an update that this feature
                                  • How to Customize Task Creation to Send a Custom Alert Using JavaScript in Zoho CRM?

                                    Hello Zoho CRM Community, I’m looking to customize Zoho CRM to send a custom alert whenever a task is created. I understand that Zoho CRM supports client scripts using JavaScript, and I would like to leverage this feature to implement the alert functionality.
                                  • Send Whatsapp with API including custom placeholders

                                    Is is possible to initiate a session on whatsapp IM channel with a template that includes params (placeholders) that are passed on the API call? This is very usefull to send a Utility message for a transactional notification including an order number
                                  • Configurable Zoho Cliq Notifications for Zoho People Alerts

                                    Hello Zoho People Product Team, Greetings and hope you are doing well. We would like to request an enhancement to Zoho People notifications, enabling a native delivery via Zoho Cliq with admin-level control, similar to the notification settings available
                                  • Add Israel & Jewish Holidays to Zoho People Holidays Gallery

                                    Greetings, We hope you are doing well. We are writing to request an enhancement to the Holidays Gallery in Zoho People. Currently, there are several holidays available, but none for Israel and none for Jewish holidays (which are not necessarily the same
                                  • Keep Zoho People Feature Requests in the Zoho People Forum

                                    Hello Zoho People Product Team, Greetings. We would like to submit a feature request regarding the handling of feature requests themselves, specifically for Zoho People. Issue: Feature Requests Being Moved to Zoho One Zoho People feature requests are
                                  • ZO25: The refreshed, more unified, and intelligent OS for business

                                    Hello all, Greetings from Zoho One! 2025 has been a remarkable year, packed with new features that will take your Zoho One experience to the next level! From sleek, customizable dashboards to an all-new action panel for instant task management, we’ve
                                  • Introducing Multi-Asset Support in Work Orders, Estimates, and Service Appointments

                                    We’re excited to announce a highly requested enhancement in Zoho FSM — you can now associate multiple assets with Work Orders, Estimates, and Service Appointments. This update brings more clarity, flexibility, and control to your field service operations,
                                  • [Product Update] Locations module migration in Zoho Books integration with Zoho Analytics

                                    Dear Customers, As Zoho Books are starting to support an advance version of the Branches/Warehouses module called the Locations module, users who choose to migrate to the Locations module in Zoho Books will also be migrated in Zoho Analytics-Zoho Books
                                  • Introducing Schedules for smarter availability management

                                    Greetings from the Zoho Bookings team! We’re excited to introduce Schedules, a powerful enhancement to manage availability across your workspace. Schedules are reusable working-hour templates that help you define and maintain consistent availability across
                                  • Why Zoho Contracts Prefers Structured Approvals Over Ad-hoc Approvals

                                    Approvals are one of the most important stages in a contract’s lifecycle. They determine whether a contract moves forward, gets revised, or needs further discussion. The approval process also defines accountability within the organization. Zoho Contracts
                                  • Whatsapp Connection Status still "Pending" after migration

                                    Hello, I migrated my WhatsApp API to Zoho from another provider a day ago. So far the connection status is still “Pending”. There is a problem? How long does it usually take?
                                  • Kaizen #226: Using ZRC in Client Script

                                    Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this
                                  • How to Filter timewise question to check uploaded one month or two months before in these community question ?

                                    i want to find the question that is asked some month or before any particular year, so how can i filter it ?
                                  • Proposal for Creating a Unique "Address" Entity in Zoho FSM

                                    The "Address" entity is one of the most critical components for a service-oriented company. While homeowners may change and servicing companies may vary, the address itself remains constant. This constancy is essential for subsequent services, as it provides
                                  • Workflow Down/Bug

                                    We have a workflow that sends an email to one of our internal departments 10 minutes after a record is created in a custom module. The workflow actually works correctly. However, we have now noticed that on January 8, between 3:55 p.m. and 4:33 p.m.,
                                  • Service Locations: Designed for Shared Sites and Changing Customers

                                    Managing service addresses sounds simple—until it isn’t. Large facilities, shared sites, and frequently changing customers can quickly turn address management into an operational bottleneck. This is where Service Locations deliver clarity and control.
                                  • Can I re-send the Customer Satisfaction Survey after a ticket closure?

                                    Hello, Some customers does not answer the survey right after closure, is it possible to re-send after a few days or weeks? Best Regards!
                                  • Account blocked

                                    Yesterday I got my Zeptomail account blocked due to too many hard bounces. My account is used exclusively for sending transactional emails (eg. your order has been shipped, a form has been filled, etc) and the sudden blocking impacted hundreds of websites
                                  • Filter contacts based on selected category in Zoho Desk ticket

                                    Hello community, I’m setting up the Tickets module in Zoho Desk and I need help implementing the following: When a category is selected in a ticket, I want the Contact field to be filtered so that it only displays contacts that are related to that category.
                                  • Mapping a new Ticket in Zoho Desk to an Account or Deal in Zoho CRM manually

                                    Is there any way for me to map an existing ticket in Zoho desk to an account or Deal within Zoho CRM? Sometimes people use different email to put in a ticket than the one that we have in the CRM, but it's still the same person. We would like to be able
                                  • Assign Income to Project Without Invoice

                                    Hello, Fairly new user here so apologies if there is a really obvious solution here that I am just missing... I have hundreds of small deposits into a bank account that I want to assign to a project but do not want to have to create an invoice every time
                                  • Tracking Non-Inventory Items

                                    We have several business locations and currently use zoho inventory to track retail items (sales and purchase orders). We were hoping to use zoho inventory to track our non-inventory items as well (toilet paper, paper towels, etc). I understand that we
                                  • Profile Page View Customization

                                    I need to change the fields, sections from the profile view of an emplyoyee.
                                  • Zoho Desk Android app update: Filter, Sort and Saved filters Enhancements

                                    Hello everyone! We are excited to introduce the below features on the Android version Zoho Desk mobile app: 1. Filter & Sort support has been introduced for the Contacts and Accounts modules. 2. Sort options is now available in Custom Modules as well.
                                  • Accessing shared mailboxes through Trident (Windows)

                                    Hi, I have a created a couple of shared mailboxes. The mailboxes are showing up on the browser based Zoho workplace, but I cannot seem to figure out how to access my shared inboxes through Trident (Windows). Am I missing something or is this feature not
                                  • Feature Request: Ability to set Default Custom Filters and apply them via URL/Deluge

                                    I've discovered a significant gap in how Zoho Creator handles Custom Filters for reports, and I'm hoping the Zoho team can address this in a future update. This limitation has been raised before and continues to be requested, but remains unresolved. The
                                  • Closing the Loop: Why Lookup Asymmetry is Harming Data Integrity in Creator

                                    TL;DR: Lookup fields allow users to add new related records inline via the "+" icon, but there's no equivalent ability to edit an existing related record without navigating away and losing form context. Adding a native "Edit" icon—with automatic User
                                  • filtering lookup field options based on information in another module.

                                    In our CRM system. We have the standard Accounts and Deals modules. We would like to introduce the ability to classify Accounts by Sector. Our desired functionality is to have a global list of all sectors that an Account can select, with the ability to
                                  • Service op locatie organiseren met Zoho FSM: waar lopen organisaties tegenaan?

                                    Bij organisaties met service teams op locatie merken we vaak dat de complexiteit niet zozeer in de planning zelf zit, maar in wat er rond die planning gebeurt. Denk aan opvolging na interventies, consistente servicerapporten, en het bijhouden van installaties
                                  • Introducing Assemblies and Kits in Zoho Inventory

                                    Hello customers, We’re excited to share a major revamp to Zoho Inventory that brings both clarity and flexibility to your inventory management experience! Presenting Assemblies and Kits We’re thrilled to introduce Assemblies and Kits, which replaces the
                                  • Does the ability exist to make tax on the customer profile mandatory?

                                    I am reaching out to inquire about the possibility of making the "Customer Tax" field mandatory when creating a new customer in Zoho. We want to ensure that all customers have their tax information recorded to maintain compliance with our internal processes.
                                  • email association with CRM

                                    Why is it 2024 (almost 2025) and Zoho has not figured out how to integrate email with CRM? It is so inconsistent at associating emails within CRM. I am an attorney. I have clients and work with other attorneys. Attorney John Doe is associated with multiple
                                  • Fix the speed

                                    It takes ages to load on every step even though my dataset is quite small.
                                  • Credit Note for Shipped and Fatoora pushed invoices

                                    We have shipped a Sales Order and created an Invoice. The Invoice is also pushed to Fatoora Now we need to create a credit note for the invoice When we try it, it says we need to create a Sales Return in the Zoho Books, we have already created a Sales
                                  • Next Page