Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Register for Zoho Desk Beta Community

                                                                With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                              • Share your Zoho Desk story with us!

                                                                Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                              • Tip #1: Learn to pick the right channels

                                                                Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                              • Welcome to Zoho Desk Community - Say hello here!

                                                                Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                              • Webinar 1: Blueprint for Customer Service

                                                                With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • Client scripts for Zoho Books ?

                                                                                                                  Good day everyone, I am looking for a way to be able to interact with the Quotes and Invoices as they are being created. Think of it like Zoho client script in Zoho CRM. But for the life of me I dont see a way to do this. The issue with having function
                                                                                                                • Uplifted homepage experience

                                                                                                                  Editions: All editions. Release Note: Available in CA and SA DCs (Standard, Professional). Availability update (17th February 2026): The feature has been enabled for the following DCs: All editions in the CA and SA DC | Free, Standard and Professional
                                                                                                                • Unable to produce monthly P&L reports for previous years

                                                                                                                  My company just migrated to Books this year. We have 5+ years financial data and need to generate a monthly P&L for 2019 and a monthly P&L YTD for 2020. The latter is easy, but I'm VERY surprised to learn that default reports in Zoho Books cannot create
                                                                                                                • Crossbeam

                                                                                                                  Does anyone use Crossbeam with their Zoho CRM? I'm looking for a way to import Crossbeam partner leads into Zoho CRM. If so: - What's your experience been like? - Are you able to automatically import Crossbeam leads > Zoho CRM? How? - What doesn't work
                                                                                                                • Access forms within a form

                                                                                                                  I have multiple forms that I would like to create access to for users within one form. Example: I have forms A, B, C, D. I would like to create a form with the links to forms A, B, C, & D. That way we can send one link and then the user can select the
                                                                                                                • WorkDrive issues with Windows Explorer Not Responding

                                                                                                                  We are using WorkDrive to collaborate on editing video content. We have a lot of files and quite a few are a few gigs. Recently anytime I try and work with the files Explorer freezes for a couple minutes whether it's dragging the files into Premiere or
                                                                                                                • Issues with Actions By Zoho Flow

                                                                                                                  Hi, I have a workflow that fires when a deal reaches a stage. This then sends out a contract for the client to sign. I have connected this up through Actions by Zoho Flow. Unfortunately this fails to send out. I have tracked it down to the date fields.
                                                                                                                • Fill Mail Merge document up with subform fields of an Inventory module record being in the Related List

                                                                                                                  Hi, I try to insert subform fields from an inventory module record being on the Related List of another inventory module record into a Mail Merge template without success. For example: we use ratecards in licensing and this ratecard items are available
                                                                                                                • Enhancements for Currencies in Zoho CRM: Automatic exchange rate updates, options to update record exchange rates, and more

                                                                                                                  The multi-currency feature helps you track currencies region-wise. This can apply to Sales, CTC, or any other currency-related data. You can record amounts in a customer’s local currency, while the CRM automatically converts them to your home currency
                                                                                                                • Better use of contacts

                                                                                                                  Zoho inventory has the ability to add multiple contacts to customers. However Zoho inventory doesn't currently provide a way to link a contact to objects like sales orders. This means that while you can tell what company has placed a sales order you can't
                                                                                                                • How can we add products using a Wizard?

                                                                                                                  We want to create a Wizard to add products. Why is there no possibility to use the products module when creating a wizard?
                                                                                                                • Orphan email alias blocking user creation – backend cleanup required

                                                                                                                  Hello Zoho Mail Support, I´m unable to assign or create the address xx@iezzimatica.ar in my organization. Current situation: Alias cannot be assigned to any user (system says it is already in use) New user with this address cannot be created Address does
                                                                                                                • Ensure Consistent Service Delivery with Comprehensive Job Sheets

                                                                                                                  We are elated to announce that one of the most requested features is now live: Job Sheets. They are customizable, reusable forms that serve as a checklist for the services that technicians need to carry out and as a tool for data collection. While on
                                                                                                                • Something wrong with client script??

                                                                                                                  Someone have the same feeling? Client script behavior become very strange..
                                                                                                                • API 500 Error

                                                                                                                  Hello amazing ZOHO Projects Community, I get this message. How can we solve this? { "error": { "status_code": "500", "method": "GET", "instance": "/api/v3/portal/2010147XXXX/projects/2679160000003XXXX/timesheet", "title": "INTERNAL_SERVER_ERROR", "error_type":
                                                                                                                • Polish signer experience to compete with docusign

                                                                                                                  I would like to suggest that someone spend the little bit of time to polish the signer experience, and the email templates to more of a modern professional feel. They are currently very early 2000s and with some simple changes could vastly improve the
                                                                                                                • how to change the page signers see after signing a document in zoho sign

                                                                                                                  Hello, How can I please change the page a signer sees after signing a document in Zoho Sign? I cannot seem to find it. As it is now, it shows a default landing page "return to Zoho Sign Home". Thanks!
                                                                                                                • Digest Janvier - Un résumé de ce qui s'est passé le mois dernier sur Community

                                                                                                                  Bonjour chers utilisateurs, Le premier mois de l’année est déjà derrière nous ! Découvrons ensemble comment s'est passé janvier pour Zoho Community France. Nous avons démarré le mois avec une nouvelle intégration entre Zoho Desk et Zoho Contracts. Cette
                                                                                                                • contact support

                                                                                                                  I have 2 request for a phone call and not received one call as promise!
                                                                                                                • Can we fetch Deal owner and Deal owner email with COQL?

                                                                                                                  While fetching deal deatils with coql api and while fetching deal owner name and email it is just giving owner id not ginving name and email https://www.zohoapis.in/crm/v2/coql
                                                                                                                • Zoho Social - Cliq Integration / Bot

                                                                                                                  Dear community / zoho, I am looking for a way to create a bot within Zoho Cliq to update my colleagues about our Zoho Social activities. For example, if a new post is published, it would be great if this post automatically would be shared in our social
                                                                                                                • Comment to DM Automation

                                                                                                                  Comment to DM automation feature in Zoho Marketing Automation, similar to what tools like ManyChat offer. Use case: When a user comments on a social media post (Instagram / Facebook), the system should automatically: Send a private DM to the user Capture
                                                                                                                • Send / Send & Close keyboard shortcuts

                                                                                                                  Hello! My team is so close to using Zoho Desk with just the keyboard. Keyboard shortcuts really help us to be more efficient -- saving a second or two over thousands of tickets adds up quickly. It seems like the keyboard shortcuts in Desk are only for
                                                                                                                • Handling Agent Transfer from Marketing Automation Journey to SalesIQ WhatsApp

                                                                                                                  We are currently using Marketing Automation for WhatsApp marketing, and the features are great so far We have a scenario where, during a campaign or journey, we give customers an option to chat with our sales team. For this, we are using SalesIQ WhatsApp
                                                                                                                • Webinar - Getting Started with Zoho LandingPage

                                                                                                                  Want to launch landing pages that support real marketing goals? This webinar is designed to help you understand the role landing pages play in your overall campaigns and how they contribute to lead generation and conversions. During this session, you’ll
                                                                                                                • Increase Round Robin Scheduler Frequency in Zoho Desk

                                                                                                                  Dear Zoho Desk Team, We hope this message finds you well. We would like to request an enhancement to the Round Robin Scheduler in Zoho Desk to better address ticket assignment efficiency. Current Behavior At present, the Round Robin Scheduler operates
                                                                                                                • Zoho mail to contacts and leads, but not to accounts?

                                                                                                                  We use the accounts tab a lot for our business because they can be linked to the sales orders. Now we want to use the mail add on to link communication of our emails to our accounts. However this is only possible for contacts and leads? It would be convenient
                                                                                                                • Marketing Tip #19: Upsell on product page to increase order value

                                                                                                                  Upselling doesn’t have to feel pushy. It can actually feel helpful when done right. Think "people also bought", upgrades, add-ons, or complementary items. A smart upsell increases your average order value and improves the customer experience by making
                                                                                                                • CRM limit reached: only 2 subforms can be created

                                                                                                                  we recently stumbled upon a limit of 2 subforms per module. while we found a workaround on this occasion, only 2 subforms can be quite limiting in an enterprise setting. @Anonymous User I've read about imminent increase of other components (e.
                                                                                                                • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                                                                                  Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
                                                                                                                • オンライン勉強会のお知らせ Zoho ワークアウト (2/19 参加無料)

                                                                                                                  ユーザーの皆さま、こんにちは。コミュニティチームの中野です。 2月開催のZoho ワークアウトについてお知らせします。 今回はZoomにて、オンライン開催します。 ▶︎参加登録はこちら(無料) https://us02web.zoom.us/meeting/register/6AyVUxp6QDmMQiDGXGkxPA ━━━━━━━━━━━━━━━━━━━━━━━━ Zoho ワークアウトとは? Zoho ユーザー同士で交流しながら、サービスに関する疑問や不明点の解消を目指すイベント「Zoho
                                                                                                                • Removing Tables from HTML Inventory Templates - headers, footers and page number tags only?

                                                                                                                  I'm a bit confused by the update that is coming to HTML Inventory Templates https://help.zoho.com/portal/en/kb/crm-nextgen/customize-crm-account/customizing-templates/articles/nextgen-update-your-html-inventory-templates-for-pdf-generator-upgrade It says
                                                                                                                • Ability to Link Reported Issues from Zoho Desk to Specific Tasks or Subtasks in Zoho Projects

                                                                                                                  Hi Zoho Desk Team, Hope you're doing well. When reporting a bug from Zoho Desk to Zoho Projects, we’ve noticed that it’s currently not possible to select an existing task or subtask to associate the issue with. However, when working directly inside Zoho
                                                                                                                • How do i move multiple tickets to a different department?

                                                                                                                  Hello, i have several tickets that have been assigned to the wrong department.  I am talking about hundreds of automatically generated ones that come from a separate system. How can i select them all at once to move them to another department in one go? I can select them in "unsassigned open tickets view" but i can't find a "move to another department" option. I also can't seem to assign multiple tickets to the same agent in that same view. Could somebody advice?
                                                                                                                • Set Default Status of Assembly to "Assembled" When Entered in UI

                                                                                                                  I've just discovered the new "confirmed" status of Assemblies within Inventory. While I understand the intent of this (allowing for manufacturing planning and raw material stock allocation), it was initially confusing to me when manually entering some
                                                                                                                • How to apply customized Zoho Crm Home Page to all users?

                                                                                                                  I have tried to study manuals and play with Zoho CRM but haven't found a way how to apply customized Zoho CRM Home Page as a (default) home page for other CRM users.. How that can be done, if possible? - kipi Moderation Update: Currently, each user has
                                                                                                                • Plug Sample #14: Automate Invoice Queries with SalesIQ Chatbot

                                                                                                                  Hi everyone! We're back with a powerful plug to make your Zobot smarter and your support faster. This time, we're solving a common friction point for finance teams by giving customers quick access to their invoices. We are going to be automating invoice
                                                                                                                • Enrich your contact and company details automatically using the Data Enrichment topping

                                                                                                                  Greetings, I hope you're all doing well. We're happy to announce the latest topping we've added to Bigin: The Data Enrichment topping, powered by WebAmigo. This topping helps you automatically enhance your contact and company records in Bigin. By leveraging
                                                                                                                • Zoho Billing Partial Proration...

                                                                                                                  Hi, In Zoho billing, we offer upgrades, downgrades, suspends, and resumes. We use online payment for most subscriptions, but have only a few that are offline, but we treat them the same either way.. We prorate only parts of these changes.. Right now zoho
                                                                                                                • Any update on adding New Customer Payment Providers who support in store terminal devices?

                                                                                                                  Currently there is only one Customer payment provider listed for terminal devices in USA- Everyware. They charge a monthly fee of almost $149 minimum. Will you add other providers - like Zoho Payments or Stripe or Worldpay that would allow integrated
                                                                                                                • Next Page