Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team



      Zoho Campaigns Resources


        • Desk Community Learning Series


        • Digest


        • Functions


        • Meetups


        • Kbase


        • Resources


        • Glossary


        • Desk Marketplace


        • MVP Corner


        • Word of the Day


        • Ask the Experts


          • Sticky Posts

          • Register for Zoho Desk Beta Community

            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
          • Share your Zoho Desk story with us!

            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
          • Tip #1: Learn to pick the right channels

            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
          • Welcome to Zoho Desk Community - Say hello here!

            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
          • Webinar 1: Blueprint for Customer Service

            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

          Zoho CRM Plus Resources

            Zoho Books Resources


              Zoho Subscriptions Resources

                Zoho Projects Resources


                  Zoho Sprints Resources


                    Zoho Orchestly Resources


                      Zoho Creator Resources


                        Zoho WorkDrive Resources



                          Zoho CRM Resources

                          • CRM Community Learning Series

                            CRM Community Learning Series


                          • Tips

                            Tips

                          • Functions

                            Functions

                          • Meetups

                            Meetups

                          • Kbase

                            Kbase

                          • Resources

                            Resources

                          • Digest

                            Digest

                          • CRM Marketplace

                            CRM Marketplace

                          • MVP Corner

                            MVP Corner




                            Zoho Writer Writer

                            Get Started. Write Away!

                            Writer is a powerful online word processor, designed for collaborative work.

                              Zoho CRM コンテンツ



                                ご検討中の方

                                  • Recent Topics

                                  • Zia Dashboard Insights : turn your dashboard into decisions

                                    When you look at a chart or KPI in a dashboard, you would possibly see something like: Revenue: $2.4M ↓ 18% vs last month. It can be a positive growth or a negative one, or a dip in revenue, a spike in deals, a slowdown in renewals—all you usually see
                                  • Tickets without registration

                                    Hi, would it be possible to give customers the opportunity to be able to read their tickets without registration?
                                  • Zoho Desk Answer Bot vs. Zia Agents – Knowledge Base & Ticket Access

                                    Hi everyone, I’m currently evaluating AI options in Zoho Desk and ran into some limitations with the Answer Bot: Answer Bot limitations Only uses Knowledge Base articles No access to tickets Limited control over sources: Either one Help Center or all
                                  • Como estruturar automações eficientes no Zoho Creator

                                    Como estruturar automações eficientes no Zoho Creator Introdução No contexto de aplicações empresariais, automação não é apenas uma conveniência, é um fator crítico para ganho de produtividade, redução de erros e escalabilidade operacional. O Zoho Creator
                                  • Changing the status of a work-order

                                    Is there a way to change the status of a work-order?
                                  • Online Payment Fees

                                    We don't take many online credit card payments so the merchant service provider (PayPal) charges us the 2.9% fee for processing the amount. I would like the ability for the fee to be automatically added to the total amount for "ease of payment". We'd
                                  • What is a realistic turnaround time for account review for ZeptoMail?

                                    On signing up it said 2-3 business days. I am on business-day 6 and have had zero contact of any kind. No follow-up questions, no approval or decline. Attempts to "leave a message" or use the "Contact Us" form have just vanished without a trace. It still
                                  • Zia Agents in Zoho CRM: a better way to set up digital employees

                                    Hello everyone, If you've been using Zia Agents in Zoho CRM, so far using Connections was the only deployment method you're familiar with. You create an agent in Zia Agents (define its objective, write instructions, use tools, add knowledge base) and
                                  • Bank Feeds

                                    Since Friday my bank feeds wont work. I have refreshed feeds, deactivate and reactivate and nothing is working
                                  • Logged out

                                    Hi, just been working on a sheet when a pop up box appeared telling me I'm going to be logged out in x number of seconds and if I reload I may lose any edits, or words to that effect. It did indeed log me out and I did indeed lose my last edits. Any idea
                                  • Zoho API

                                    I have little experience with API. I'm trying to get a Custom API working with Zoho creator. I have created a Custom API and created an Endpoint URL, but i get a 9400 error code "The provided HTTP method is not valid for this custom API". Based off the
                                  • #157127950

                                    Where did my initial question go?
                                  • Zoho writer unable to merge documents to PDF with basic fonts in Hebrew or fonts from my computer

                                    I created several forms that will be merged into PDF files through Zoho Writer and I am unable to receive the PDF in the basic fonts of the Hebrew language or in the fonts I have on my computer. The writer exports to PDF an exchange font that looks very
                                  • How I Implemented Subscription-Based Access Control and Expiry Handling in Zoho Creator

                                    I recently worked on a use case where users come into the application to request a service, but they should only be able to continue the process after completing a subscription. The challenge was not just controlling access, but also making sure that
                                  • Zoho Forms API

                                    Is there any way to get all form entry list using API? Looking forward to hear from you
                                  • Zoho Projects : Task should auto-update to 'In Progress' if timer started

                                    Namaskaram. Right now, if a Task's timer is started, the Task stays in 'Not Started' status. One has to manually update it to 'In Progress'. From a #uxdesign standpoint, it is an unnecessarily two step process to start working on a task. It would be better that, if I start the timer on a task, it should automatically change to 'In Progress' status. Crafted with ❤️ Zoho Gurus | Zoho One Practice Team @ CubeYogi Zoho Authorised Partner | 7+ Yrs | 200+ Projects | 100+ Customers
                                  • Laatste facturen en betalingen niet zichtbaar in mijn account

                                    Wij gebruiken ZOHO invoice al jaren, maar sinds afgelopen week is mijn laatst verzonden factuur niet zichtbaar in mijn account, en tevens de laatst betaalde facturen zie ik niet. Hoe kan dit? Ik heb de pagina al diverse keren gerefreshed.
                                  • Undelivered Mail uncategorized-bounce errors when sending invoices

                                    Recently we have been getting Undelivered Mail bounce notification when sending invoices. Reason: uncategorized-bounce Some go through no problem some bounce back. We recently sent 10 invoices, 6 received bounce notifications. After reaching out to the
                                  • Option to Delete Chats in IM

                                    Currently, there is no option to delete any chats in IM, regardless of their source.
                                  • Can I import MSG files into Microsoft 365 without Outlook?

                                    Yes, absolutely. You do not need Outlook installed to import MSG files into Microsoft 365. Aryson MSG file Converter is a dedicated tool that eliminates the Outlook dependency entirely, making the migration process simple and efficient for all users.
                                  • Introducing Radio Buttons and Numeric Range Sliders in Zoho CRM

                                    Release update: 1. 2 March 2026: Currently out for CN, JP, AU and CA DCs (Free and standard editions). For other DCs, this will be released by mid-March. 2. 26 March 2026: CA, SA, AU, JP, CN DCs - Free, Standard, Professional, Enterprise, Ultimate, CRM
                                  • Feature Request - A Way To Search Item Groups

                                    Hi Inventory Team, I can't find any way to filter or search by fields of Item Groups. It would be great to see that functionality added. I have a use case where a single product might come from 5 or more suppliers and each supplier's item is an Item in
                                  • Zoho Books/Inventory - Update Marketplace Sales Order via API

                                    Hi everyone, Does anyone know if there is a way to update Sales Orders created from a marketplace intigration (Shopify in this case) via API? I'm trying to cover a scenario where an order is changed on the Shopify end and the changes must be reflected
                                  • Ticket id issues

                                    When I reply a ticket from desktop, it doesn't have ticket id in the subject and it's great. When I reply a ticket from Zoho desk mobile, Zoho adds ticket id in the subject and I don't want that. Please help in this matter.
                                  • Advanced email configuration - agent's name vs. department name

                                    We currently have all four Advanced Configuration options turned ON at the Global-level (Channels > Email > Advanced Configuration) - including the "Show Agent name in Ticket replies and outgoing emails" option. We also had that same option turned ON
                                  • Add Bounced as an Email Action / Notification for Bounced Emails

                                    This is one of the hard requirements for the clients we're servicing. They want to get an internal email notification whenever the email they sent to their contacts have bounced, so that they can look into it and update the email address. Currently, the
                                  • Files Uploaded to Zoho WorkDrive Not Being Indexed by Search Engines

                                    Hello, I have noticed that the files I upload to Zoho WorkDrive are not being indexed by search engines, including Google. I’d like to understand why this might be happening and what steps I can take to resolve it. Here are the details of my issue: File
                                  • not able to convert pdf to jpg and other forms and vice versa.

                                    i want to change my pdf to jpg, word, etc and some times jpg to pdf. i don't know how to do in this.
                                  • What’s New in Zoho Analytics - March 2026

                                    Hello Users! In this month's update, we bring improvements across integrations, security, reporting, and analytics capabilities to help you work with your data more efficiently and with greater control. Explore what’s new and see how these enhancements
                                  • Zoho People > Performance Management > Appraisal cycle

                                    Hello All I am using this 2 users to test out how it work on Performance Management User 1 - Reportee User 2 - Reporting Manager : Li Ting Haley User 1 : Self Appraisal Error How do i fix this error?
                                  • SalesIQ Tip for Admins: Guide Operators in Real-time Without Interrupting the Chat

                                    Consider this. You're a supervisor and you're looking through the active conversations. An associate is mid-chat with a high-value prospect. The prospect asks something unexpected, maybe about a tailor-made subscription plan or a bulk discount that’s
                                  • Early Access: Check Printing in Zoho Books

                                    Hello Everyone,   Are you constantly writing checks to pay your vendors?   We've got a great news to share with you! You can now pay your vendors by writing and printing a check directly from Zoho Books. The feature is ready and we'll be rolling it out to our customers in phases.  It is available in the  US Edition of Zoho Books and also in the Global edition, where the country is selected as USA and the currency is USD.   Here’s a summary of what’s possible:   1. Write and print a check. 2. Make
                                  • Connecting Zoho Inventory to ShipStation

                                    we are looking for someone to help connect via API shipStation with Zoho inventory. Any ideas? Thanks. Uri
                                  • Syncing zoho books into zoho crm

                                    I was wondering how I can use zoho books in crm as I have been using them separately and would like to sync the two. Is this possible and if so, how? Thanks
                                  • Zoho no support response.

                                    Problem is Zoho support seems to be just a concept. Just completing my trial, am ready to purchae 3 user subscriptions pending answer to a question, submitted two suport request during my trial that weren't responded to. Gave up trying the 888 line. Hard to imagine my production data in hosting environment with no support response.
                                  • Updating transaction number series for fiscal year 2026-2027 in Zoho POS

                                    A fiscal year or financial year is a 12-month period that businesses follow to manage and track their financial activities such as expenses, revenue, and taxes. This doesn't need to match the calendar year (JAN-DEC) and varies based on region, and tax
                                  • Lack of Looping and Carry-Forward Functionality in Zoho Survey

                                    Zoho Survey currently does not support looping or carry-forward functionality, meaning it is not possible to dynamically generate follow-up questions based on each option selected in a previous question or to pipe selected responses (such as {Looping
                                  • Forecast in Zoho CRM Just Got Smarter with an upgraded Zia Intelligence

                                    Hello Everyone, We are here with an interesting enhancement to Forecasts in Zoho CRM — Enhanced Zia Insights for your sales Forecast. Imagine a regional sales manager reviewing their team’s performance using forecasts in Zoho CRM. Instead of switching
                                  • Update TDS and TCS rates for Income Tax Act, 2025 (effective April 1, 2026)

                                    Hello everyone, The Income-tax Rules, 2026 (G.S.R. 198(E), dated March 20, 2026) have been notified, marking a significant structural shift in India’s direct tax framework. From April 1, 2026, the Income Tax Act, 2025 replaces the Income Tax Act, 1961.
                                  • Service line items

                                    Hello Latha, Could you please let me know the maximum number of service line items that can be added to a single work order? Thanks, Chethiya.
                                  • Next Page