Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Register for Zoho Desk Beta Community

                                                                With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                              • Share your Zoho Desk story with us!

                                                                Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                              • Tip #1: Learn to pick the right channels

                                                                Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                              • Welcome to Zoho Desk Community - Say hello here!

                                                                Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                              • Webinar 1: Blueprint for Customer Service

                                                                With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • bouncing emails

                                                                                                                  My recurring invoices have bounced back
                                                                                                                • Probable Scam / Phising attempt from email pretending to be Zoho

                                                                                                                  I think this is a scam email right? It says "zohCworkplace.com". I'm on the Mail Free plan The link in the CTA button seems to go to a redirect. Just wanted to bring it up to the security team.
                                                                                                                • Feature Request: Ability to set Default Custom Filters and apply them via URL/Deluge

                                                                                                                  I've discovered a significant gap in how Zoho Creator handles Custom Filters for reports, and I'm hoping the Zoho team can address this in a future update. This limitation has been raised before and continues to be requested, but remains unresolved. The
                                                                                                                • Email Address Search in "To" Field Search is broken, Zoho refuses to fix it

                                                                                                                  Typing a part of the email address string other than the beginning of the string does NOT work, I kindly urge Zoho to fix this. Let's say you remember writing an email so someone called "smith" from company "corp.com", but can't remember their first name,
                                                                                                                • Emails not being received by @hotmail.com, @outlook.com and a few others

                                                                                                                  When I try to send emails from zoho mail to people with email addresses ending @outlook.com and @hotmail.com (and a few others), I get a 'delayed' automatic email and then a few hours later, an 'undelivered' automatic email. This has started a few months
                                                                                                                • scroll bar for far left of screen

                                                                                                                  I am unable to even see the scroll bar to the right of "inbox" etc on the left; it is stuck at "streams" and I can't get to inbox or anything else. It would help if it could be made a lighter color as the black or dark grey can't bee seen.
                                                                                                                • OUt of office every friday

                                                                                                                  Hi, I tryed to configure my out of office, because i'm not working or emailing on fridays. But when i select only friday as unavailable day, the out of office still sets for the whole week. What am I doing wrong?
                                                                                                                • Signature line

                                                                                                                  How do I set signature line in email
                                                                                                                • Zoho email

                                                                                                                  I cannot send email to mail.ru
                                                                                                                • 【参加無料】東京 Zoho ユーザ交流会 NEXUS ー CRMで始めるマーケティング事例 / AI活用法(Zia Agents)

                                                                                                                  ユーザーの皆さま、こんにちは。コミュニティチームの藤澤です。 3月27日(金)に東京、新橋で東京 Zoho ユーザー交流会 NEXUS を開催します! 昨年度までより、さらにパワーアップして戻ってきました! ユーザー活用事例は、2人のユーザーさんからお話しいただきます。Zoho サービスの活用の幅を広げたい方や、他のユーザーの利用法を気軽に知りたい方など、多くの方にとって学びのあるセッションになること間違いなしです✨ また今年は、これまで以上に、AI機能にも焦点を当てて行く予定です。 初回として、Zoho社員からZoho
                                                                                                                • Issue Exporting Data – CSRF Token Invalid Error

                                                                                                                  Dear Zoho Team, We are experiencing an issue when exporting data from our Analytics workspace. Whenever we attempt to export data from our analytical pool, the system displays the following alert message: Alert Message: The CSRF token is invalid. It could
                                                                                                                • Deluge Learning Series – Client functions in Deluge | January 2026

                                                                                                                  We’re excited to kick-start the first session of the 2026 Deluge Learning Series (DLS) with Client functions in Deluge. For those who are new to DLS, here’s a quick overview of what the series is all about: The Deluge Learning Series takes place on the
                                                                                                                • Zoho Mail 505 error I can not send email

                                                                                                                  Hi, I’m having issues sending emails from my custom domain email address. When I send emails to Outlook addresses, I receive an “Undeliverable 505” error. However, emails send and receive correctly when I use Gmail. This is important for my business,
                                                                                                                • Multiple MFA Methods

                                                                                                                  With SMS-based MFA methods being discontinued, there is now no way to have mutliple MFA methods. I'd like to add my zoho account on two seperate phones using the Google Authenticator app. In the https://accounts.zoho.com/home#multiTFA/modes you can only
                                                                                                                • Reuse Standalone Function

                                                                                                                  I noticed that there's a missing information in documentation to reuse a standalone function and it is because the parameters require an argument. Here is my code and it is working. response = invokeurl [ url: "https://people.zoho.com/api/v3/function/sample/execute"
                                                                                                                • Domain renewals

                                                                                                                  Need to know how hoe to renew the domain
                                                                                                                • MX shopify problem

                                                                                                                  hello, i added all MX values in my shopify DNS - it shows those values on the shopify panel + your toolkit. I tried to send some email and it works, however on my gmail it says they cant verify this email. When i try to answer into my domain's email -
                                                                                                                • Best Way to Manage Email Notifications While Running a Strategy Website

                                                                                                                  I am currently managing a content-based website, and I use Zoho Mail for handling contact forms, user queries, and collaboration emails. One challenge I am facing is organizing incoming emails efficiently, especially when messages come from different
                                                                                                                • What is the maximum email domains ?

                                                                                                                  I help manage about 20 associations and I'm looking for a way to centralize them in one place. Does Zoho Mail pro or enterprise support 20-30 domains for 3-5 users each?
                                                                                                                • Add to Workdrive filter

                                                                                                                  I'm trying to create a filter that will upload attachments in emails and the e-mail body to a folder in workdrive. I am able to do one or the other (attachment, or e-mail content), but not both. I first tried it using the "Email (EML) + attachment" option.
                                                                                                                • Forgot my admin Panel Id and password

                                                                                                                  Sir, I have an account , where a domain mycityestate.in is added for Zoho email , now it is hard for me to manage email because i have forgotten the Email account and password registered with Admin Panel of Zoho. Just need email name which is registered
                                                                                                                • MCP no longer works with Claude

                                                                                                                  Anyone else notice Zoho MCP no longer works with Claude? I'm unable to turn this on in the claude chat. When I try to toggle it on, it just does nothing at all. I've tried in incognito, new browsers, etc. - nothing seems to work.
                                                                                                                • Knowledgebase SEO

                                                                                                                  We have a custom-domain mapped help center that is not restricted via login. I have some questions: a) will a robots.txt file still allow us to control indexing? b) do we have the ability to edit the sitemap? c) do category URLs get indexed by search
                                                                                                                • Problem with CRM Connection not Refreshing Token

                                                                                                                  I've setup a connection with Zoom in the CRM. I'm using this connection to automate some registrations, so my team doesn't have to manually create them in both the CRM and Zoom. Connection works great in my function until the token expires. It does not refresh and I have to manually revoke the connection and connect it again. I've chatted with Zoho about this and after emailing me that it couldn't be done I asked for specifics on why and they responded. "The connection is CRM is not a feature to
                                                                                                                • New Features: Repeat Last Action, Insert Cut/Copied Rows/Columns and Hyperlink

                                                                                                                  You might have noticed the constant updates to Zoho Sheet of late. Here are 3 more features that have been added to Zoho Sheet recently: F4 - Repeat Last Action Insert Cut/Copied Rows and Columns Insert Hyperlink Here is a screen cast demonstrating each of these features. Read further below to learn more about these new features. F4 - Repeat Last Action: You can now repeat the last action you made on your spreadsheet by using the keyboard shortcut, F4. It is quite handy and helps you get your work
                                                                                                                • Need help getting my mail on iPhone and Tablet

                                                                                                                  I need to access my Zoho mail via the iPhone Mail app. I have entered the login name, password and the incoming and outgoing servers, which my Mail Settings page says are imappro.zoho.com and smtppro.zoho.com. The iPhone keeps saying it cannot authenticate.
                                                                                                                • Add an background image to an email template in CRM

                                                                                                                  Hi all, We wants to put an background image behind all our email templates. Is there a way to import this thru html. If i put the option background image in <body style="background-image:...</body> and i look to the preview it shows our background, but
                                                                                                                • Is there a way to show contact emails in the Account?

                                                                                                                  I know I can see the emails I have sent and received on a Contact detail view, but I want to be able to see all the emails that have been sent and received between all an Accounts Contacts on the Account Detail view. That way when I see the Account detail
                                                                                                                • How do I bulk archive my projects in ZOHO projects

                                                                                                                  Hi, I want to archive 50 Projects in one go. Can you please help me out , How can I do this? Thanks kapil
                                                                                                                • Copy contents of File Upload Field into Workdrive

                                                                                                                  Hello, I have set up our CRM so that a Workdrive folder is automatically created for each Deal via workflow, this adds the id of the folder into a dedicated field. We also have a field on each Deal called 'Approved Layout', which is a file upload field.
                                                                                                                • Deleting a memorized email address

                                                                                                                  How can I delete a memorized email address? Even though the address has been deleted from Contacts, Zoho mail still auto suggests the address when typing it into the TO field. Thanks!
                                                                                                                • ZeptoMail API

                                                                                                                  Hello Since today, we experience issues with the ZeptoMail API. When trying to send e-mails using: https://api.zeptomail.eu/v1.1/email we receive the error: (503) Site unavailable due to a traffic surge. Please try again shortly. I kindly ask you to identify
                                                                                                                • 553 Relaying disallowed. Invalid Domain - gzkcompany.ro

                                                                                                                  Hi there, Can you please assist me in getting the right domain settings? I just renewed my domain subscription, after expired and i got error: 553 Relaying disallowed. Invalid Domain - gzkcompany.ro Zoho mail can receive emails, but its impossible to
                                                                                                                • Not able to receive emails for a while

                                                                                                                  I am not able to receive emails for a while now.
                                                                                                                • Confirmation requested: eligibility and process to downgrade to Forever Free — tenant bigbanghawking.com

                                                                                                                  Thank you for your reply. I am testing Zoho Mail from Brazil with the tenant bigbanghawking.com (endpoint: mail.zoho.com) and we are currently on the Premium trial that expires 21/01/2026. Before deciding whether to pay or cancel, I need written confirmation
                                                                                                                • Zoho API v2.0 - get ALL users from ALL projects

                                                                                                                  Hello,        I've been trying to work on an automatization project lately and I find it difficult to work with this strict structure. To be more explicit, if i would like to get all users participating in a project i would need to get all projects first.       Same thing with projects. If i want to get all projects, I would need to get all portals first.        The problem with this aproach is that it consumes a lot of time and resources.             I want to ask if there is another way of getting
                                                                                                                • الموقع لا يقوم بالسداد

                                                                                                                  السلام عليكم ورحمة الله وبركاته وبعد من أمس وانا احاول السداد للدومين YELLOWLIGHT ولا اتمكن من السداد اقوم بتعبئة جميع البيانات ولكن دون جدوى يطلع لى حدث خطأ ما
                                                                                                                • New in Office Integrator: Enhanced document navigation with captions and cross references

                                                                                                                  Hi users, We're pleased to introduce captions, table of tables and figures, and cross-references in the document editor in Zoho Office Integrator. This allows you to structure documents efficiently and simplify document navigation for your readers from
                                                                                                                • Where Do I set 24h time format in Cliq?

                                                                                                                  Where Do I set 24h time format? Thanks
                                                                                                                • 🎉 ¡Seguimos trayendo novedades a Español Zoho Community! 🎉 Confirmada la agenda y ubicación para los Workshops Certificados

                                                                                                                  Si todavía no te has hecho con tu entrada para nuestros Workshops Certificados del próximo 26 y 27 de marzo o, por el contrario, estabas esperando que confirmáramos dónde los celebraremos, ¡este post es para ti! 📍¿Dónde nos vemos?📍 Nuestros Workshops
                                                                                                                • Next Page