Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Register for Zoho Desk Beta Community

                                                                With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                              • Share your Zoho Desk story with us!

                                                                Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                              • Tip #1: Learn to pick the right channels

                                                                Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                              • Welcome to Zoho Desk Community - Say hello here!

                                                                Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                              • Webinar 1: Blueprint for Customer Service

                                                                With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • using the Client script I want to Hide Show the Fields

                                                                                                                  if Related to service means some of the field like service no want to shoe and hide Amc no , purchase no how i achive this let issu = ZDK.Page.getField('Issue_Related_To').getValue(); if (issu == 'Service') { var field_obj = ZDK.Page.getField('Warranty_Cases');
                                                                                                                • Ask the Expert – Zoho One Admin Track : une session dédiée aux administrateurs Zoho One

                                                                                                                  Vous administrez Zoho One et vous vous posez des questions sur la configuration, la gestion des utilisateurs, la sécurité ou encore l’optimisation de votre back-office ? Bonne nouvelle : une session Ask the Expert – Zoho One Admin Track arrive bientôt,
                                                                                                                • Write-Off multiple invoices and tax calculation

                                                                                                                  Good evening, I have many invoices which are long overdue and I do not expect them to be paid. I believe I should write them off. I did some tests and I have some questions:  - I cannot find a way to write off several invoices together. How can I do that,
                                                                                                                • Kaizen #210 - Answering your Questions | Event Management System using ZDK CLI

                                                                                                                  Hello Everyone, Welcome back to yet another post in the Kaizen Series! As you already may know, for the Kaizen #200 milestone, we asked for your feedback and many of you suggested topics for us to discuss. We have been writing on these topics over the
                                                                                                                • vendors / customers with 2 different address and gst no

                                                                                                                  Why can't we have option for more than one address and depending on the state option for more than 1 GST no. ? We have customers / vendors PAN india with different addresses and GST no. for different states.
                                                                                                                • Recurring Automated Reminders

                                                                                                                  Hi, The reminders feature in Zoho Books is a really helpful feature to automate reminders for invoices. However, currently we can set reminders based on number of days before/after the invoice date. It would be really helpful if a recurring reminder feature
                                                                                                                • Fail to send Email by deluge

                                                                                                                  Hi, today I gonna update some email include details in deluge, while this msg pops up and restrict me to save but my rules has run for one year. can you tell me how to use one of our admin account or super admin account to send the email? I tried to update
                                                                                                                • Transitions do not update fields until the record moves to next stage

                                                                                                                  We have a blueprint where a couple of stages have multiple transitions. If only some of the transitions are completed, but not all, Zoho does not update any of the fields impacted by the completed transitions. Is there any way Zoho can udate the fields
                                                                                                                • Zoho CRM - Kiosk Studio : Use action responses across your kiosks with sequential actions

                                                                                                                  Hello Everyone, Imagine building a kiosk that gives you full control over how actions are executed in later screens in that same kiosk. What if you could use data from a previous action later in that kiosk—with no interruptions or data gaps? This is exactly
                                                                                                                • Ability to CC on a mass email

                                                                                                                  Ability to CC someone on a mass email.
                                                                                                                • Get Cliq Meetings in my O365 calendar

                                                                                                                  Hi, we are currently evaluating to replace the Teams Messaging and Meetings with Cliq. We currently still have all our email and calendars in O365. What i want to achieve is, to create a (ZOHO) meeting from Cliq and have this meeting added to my Outlook/O365
                                                                                                                • Custom Button to convert a Deal to a Custom Module?

                                                                                                                  Hello Community I am in process of building out a custom CRM for my team and part of this is looking at building out a Custom Button or function of some sort where when a Deal is marked Closed Won the system will allow for a "Convert to Job" option to
                                                                                                                • Power up your Kiosk Studio with Real-Time Data Capture, Client Scripts & More!

                                                                                                                  Hello Everyone, We’re thrilled to announce a powerful set of enhancements to Kiosk Studio in Zoho CRM. These new updates give you more flexibility, faster record handling, and real-time data capture, making your Kiosk flows smarter and more efficient
                                                                                                                • Emails Failing with “Relaying Issues – Mail Sending Blocked” in ZeptoMail

                                                                                                                  Hello ZeptoMail Support Team, We are facing an email delivery issue in our ZeptoMail account where emails are failing with the status “Process failed” and the reason “Relaying issues – Mail sending blocked.” Issue Details Agent Name: mail_agent_iwwa From
                                                                                                                • Change eMail Template for Event-Invitations

                                                                                                                  Hello ZOHO-CRM Team How I can change the eMail Template for Event-Invitations? I work with the German Version of the Free Version. I know how I can modify eMail alerts or Signature Templates, but where I can other eMails modify you send out? Thank you
                                                                                                                • Workdrive Oauth2 Token Isn't Refreshing

                                                                                                                  I have set up oauth for a bunch of zoho apis and have never had a problem with oauth. With workdrive i am using the exact same template i usually use for the other zoho apps and it is not working. All requests will work for the first hour then stops so
                                                                                                                • Migrate Your Notes from OneNote to Zoho Notebook Today

                                                                                                                  Greetings Notebook Users, We’re excited to introduce a powerful new feature that lets you migrate your notes from Microsoft OneNote to Zoho Notebook—making your transition faster and more seamless than ever. ✨ What’s New One-click migration: Easily import
                                                                                                                • How can I import OLM to Yandex Mail easily?

                                                                                                                  For migrating Mac Outlook OLM data to Yandex Mail efficiently, the Aryson OLM Converter is a reliable professional tool that ensures complete data integrity throughout the process. Unlike manual methods, which can risk inconsistent formatting or missing
                                                                                                                • Introducing Radio Buttons and Numeric Range Sliders in Zoho CRM

                                                                                                                  Release update: Currently out for CN, JP, AU and CA DCs (Free and standard editions). For other DCs, this will be released by mid-March. Hello everyone, We are pleased to share with you that Zoho CRM's Layout Editor now includes two new field formats—
                                                                                                                • Is it possible to setup bin locations WITHOUT mandating batch tracking?

                                                                                                                  Hi fellow zoho users, I'm wondering if anyone else has a similar issue to me? I only have some products batch tracked (items with shelf life expiry dates) but I am trying to setup bin locations for my entire inventory so we can do stock counting easier.
                                                                                                                • Implementing Inventory Process

                                                                                                                  I am just starting to create an inventory system through Zoho for a nonprofit. We receive in-kind donations of items for kids, and utilize them in 2 or 3 different programs. Then families come in and take the items. I'm thinking of this structure: Our
                                                                                                                • Best way to start zoho inventory with bulk openning stock

                                                                                                                  We are already using zoho book since long time for cars trading company. Now to streamline more, would like to import the excel data of closing stock of inventory to zoho inventory and to start on. Since we need to track each VIN (unique vehicle id number)
                                                                                                                • Service Reports.

                                                                                                                  Hello Team, I have a requirement to create multiple service reports for a single AP. That means, in one AP I have 5 service line items, and all line items are linked to assets. Once I complete the AP, I want to generate 5 individual service reports, one
                                                                                                                • Blueprint enhancements - Parallel and multiple transitions, and more

                                                                                                                  Last modified on Sep 4, 2023: All Zoho CRM users can now access these enhancements. Initially, these features were available only on an early access basis and by request. However, as of August 2, 2023, they have been made available to all users in all
                                                                                                                • Item Bulk Edit - Allow for Reorder Level

                                                                                                                  We're implementing a process for using the Reorder Level field for Items, and I have to go through and add this value to a huge chunk of our Items. It's driving me bonkers that I have to do this individually through the UI rather than bulk updating. It
                                                                                                                • Zoho CRM || Unable to Bulk Assignment of Territories for Contacts

                                                                                                                  Dear Zoho CRM Support Team, I hope this email finds you well. We recently performed a bulk upload of Contacts into Zoho CRM using the official sample Excel template downloaded from the CRM. The upload itself was completed successfully; however, we encountered
                                                                                                                • What's New in Zoho Inventory | August – October 2025

                                                                                                                  Hello customers, The last quarter has been incredibly productive! We've released a powerful slate of new features and enhancements in Zoho Inventory designed to give you better control, greater efficiency, and expanded functionality across your inventory
                                                                                                                • Disable Zoho Inventory Tracking / Delink Zoho Books & Inventory

                                                                                                                  We have integrated zoho inventory with zoho books? Now after a long time, we want to disable inventory tracking and delink these 2 modules. Zoho says we cant do it. Anybody else going thru the same ? Any possibility at all? Why does zoho not allow to
                                                                                                                • Tracking Non-Inventory Items

                                                                                                                  We have several business locations and currently use zoho inventory to track retail items (sales and purchase orders). We were hoping to use zoho inventory to track our non-inventory items as well (toilet paper, paper towels, etc). I understand that we
                                                                                                                • Price Managment

                                                                                                                  I have been in discussions with Zoho for some time and not getting what I need. Maybe someone can help explain the logic behind this for me as I fail to understand. When creating an item, you input a sales rate and purchase rate. These rates are just
                                                                                                                • Set Warehouse based on Vendor

                                                                                                                  Greetings. I would like to set automaticaly the Warehouse based on the Vendor. Context: I am working on an adaptation of a Purchase Order to be used as a Quotation. I have defined that when a user has to raise a quote the Vendor will be "PROCUREMENT" I would like to set the Warehouse to a predefined value when "PROCUREMENT" is set as Vendor. I have tried to do with the Automation feature using the Field Update option, but Warehouse does not is listed as a field. Can you help? Thanks in advance.
                                                                                                                • Auto tagging

                                                                                                                  Some of the articles I enter into Notebook get there when I enter them in Raindrop.io and IFTTT copies the articles in Notebook. When this happens the notes are tagged but instead of useful one word tags with topic the tag pertains to the specific article
                                                                                                                • How do I save audio files to my PC that I record into Zoho Notebook from my phone?

                                                                                                                  I was thinking of using Zoho Notebook as a way to store composition ideas, as well as for other things if it can handle this. For this to be useful for me though, I need to be able to have an easy way to download those audio files to my PC, either individually
                                                                                                                • Search mails in shared mailbox

                                                                                                                  Hi everyone, is there a way to search mails in shared mailbox's? Search in streams or mail doesn't return anything from mails in shared mailboxes. Thanks! Rafal
                                                                                                                • Kaizen #186 : Client Script Support for Subforms

                                                                                                                  Hello everyone! Welcome back to another exciting Kaizen post on Client Script! In this edition, we’re taking a closer look at Client Script Support for Subforms with the help of the following scenario. " Zylker, a manufacturing company, uses the "Orders"
                                                                                                                • Writing SQL Queries - After Comma Auto Suggesting Column

                                                                                                                  When writing SQL Queries, does anyone else get super annoyed that after you type a comma and try to return to a new line it is automatically suggest a new column, so hitting return just inputs this suggested column instead of going to a new line? Anyone
                                                                                                                • Convert invoice from zoho to xml with all details

                                                                                                                  How to convert an Invoice to XML format with all details
                                                                                                                • Sync your Products Module for better context.

                                                                                                                  In customer support, context is everything. The integration between Zoho Desk and Zoho CRM helps your sales and support teams function as one, delivering better customer experiences. With the latest update to this integration, you can now sync the Product module in your Zoho CRM with your Zoho Desk portal. This feature enables products from Zoho CRM to reflect in the "product" field in Zoho Desk. This can save your support team valuable time and effort.    Some things to note when syncing the two:
                                                                                                                • Where is the desktop app for Zoho Projects???

                                                                                                                  As a project manager, I need a desktop app for the projects I manage. Yes, there's the web app, which is AWESOME for cross browser and platform compatibility... but I need a real desktop app for Projects that allow me to enter offline information where
                                                                                                                • CRM verify details pop-up

                                                                                                                  Was there a UI change recently that involves the Verify Details pop-up when changing the Stage of a Deal to certain things? I can't for the life of me find a workflow or function, blueprint, validation rule, layout rule ect that would randomly make it
                                                                                                                • Next Page