Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team



      Zoho Campaigns Resources


        • Desk Community Learning Series


        • Digest


        • Functions


        • Meetups


        • Kbase


        • Resources


        • Glossary


        • Desk Marketplace


        • MVP Corner


        • Word of the Day


        • Ask the Experts


          • Sticky Posts

          • Register for Zoho Desk Beta Community

            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
          • Share your Zoho Desk story with us!

            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
          • Tip #1: Learn to pick the right channels

            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
          • Welcome to Zoho Desk Community - Say hello here!

            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
          • Webinar 1: Blueprint for Customer Service

            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

          Zoho CRM Plus Resources

            Zoho Books Resources


              Zoho Subscriptions Resources

                Zoho Projects Resources


                  Zoho Sprints Resources


                    Zoho Orchestly Resources


                      Zoho Creator Resources


                        Zoho WorkDrive Resources



                          Zoho CRM Resources

                          • CRM Community Learning Series

                            CRM Community Learning Series


                          • Tips

                            Tips

                          • Functions

                            Functions

                          • Meetups

                            Meetups

                          • Kbase

                            Kbase

                          • Resources

                            Resources

                          • Digest

                            Digest

                          • CRM Marketplace

                            CRM Marketplace

                          • MVP Corner

                            MVP Corner




                            Zoho Writer Writer

                            Get Started. Write Away!

                            Writer is a powerful online word processor, designed for collaborative work.

                              Zoho CRM コンテンツ




                                ご検討中の方

                                  • Recent Topics

                                  • Project Notifcatiion Emails - Milestone

                                    Hello: I cannot get myself, or most importantly my portal client user to recieve an email upon completion of a milestone. I have set up our 1st project. I have set up a test client user. (accepted the invitation and is listed in the system as a client
                                  • Zoho Forms - Feature Request - Year Field

                                    Hi Zoho Forms Team, You currently have the following date and time fields: Date Time Date and Time Year and Month It would be useful if you could include a "Year" field For example a recent application I completed said "What year was your house built?"
                                  • Integrate Excel or Zoho Sheet functions / calculations to CRM product module

                                    Hello Community, I hope someone more experienced can help me with this question. Our price / payment plan calculations are in an Excel spreadsheet and I would like to use all those functions / calculations in my Products module. So when we send a quote
                                  • Sync CRM Contacts to USER'S contacts on Office 365

                                    I can see that the O365 sync is transferring contacts backwards and forwards between Zoho CRM and Office365. But it has created a separate address book in Office 365 called "Zoho CRM Contacts". This address book is not used by Office/Outlook's email function
                                  • How can I link Products in a Deal Subform to the Products Module

                                    Hello, I have a pricing subform on our Deals page and use a lookup field to associate a product with each line. I want to be able to look at a product page within the Products module and see a list of the deals connected to that product. I have this working
                                  • Manage Every Customer Conversation from Every Channel inside Zoho SalesIQ

                                    Your customers message you from everywhere. But are you really able to track, manage, and follow through on every conversation, without missing anything? With interactions coming in from websites, mobile apps, and messaging platforms like WhatsApp and
                                  • Zoho Books | Product updates | February 2026

                                    Hello users, We’ve rolled out new features and enhancements in Zoho Books. From Advanced Reporting Tags to the ability to mark projects as completed, explore the latest updates designed to improve your bookkeeping experience. Introducing Advanced Reporting
                                  • Update latitude & longitude address field API

                                    How do I update the coordinates of an address field from a widget? I can't modify the latitude and longitude of the address field. I think the problem is how I'm writing formdata variable. zoho_init.then(function (data) { var queryParams = ZOHO.CREATOR.UTIL.getQueryParams();
                                  • Zoho Forms - Feature Request - Past Into Scanning/OCR Field

                                    Hi Zoho Forms Team, You recently introduced the OCR/Scanning field which I have found great use for with one client who receives work orders as a screenshot from one customer. I want to raise a feature request here which would make that field even more
                                  • inability to use different primary address on invoice per location

                                    my company operates in two different locations with different email address. The problems then is the inability to edit the primary to suite the invoice for the second location.
                                  • Zoho Books - Breaking A Working App

                                    We've been using Zoho for many years now. Across all apps, entering phone numbers in standard formats was enabled in all apps. These formats are: xxx.yyy.zzzz xxx-yyy-zzzz (xxx) yyy-zzzz and we were able also to add extension numbers in these formats:
                                  • No Need To Fix Something That Is Working

                                    Zoho Books is a great financial tool which helps businesses to become more efficient and productive with day-to-day operations. As such, every change, upgrade, improvement needs to be carefully thought before implemented in the software and I'm sure Zoho
                                  • Use Zoho Creator as a source for merge templates in Zoho Writer

                                    Hello all! We're excited to share that we've enhanced Zoho Creator's integration with Zoho Writer to make this combination even more powerful. You can now use Zoho Creator as a data source for mail merge templates in Zoho Writer. Making more data from
                                  • Anyone in Australia using Zoho Books AND has their account with NAB?

                                    Hi I have an account with both NAB and Suncorp. Suncorp transaction come in the next day however NAB transactions take 4-5 business days to appear. eg: A deposit made today in my Suncorp will be imported into Zoho tomorrow. A deposit made today to the NAB account will be imported maybe Saturday (Friday overnight). I have contacted both Zoho and NAB but noone seems to know why. I was just wondering if anyone else in Australia uses NAB and has this issue (or doesn't) maybe we could compare notes and
                                  • Detailed Balance Sheet for tax preparer

                                    I'm using the free edition of Zoho Books. My tax preparer is asking for "detailed" Profit & Loss and Balance Sheet reports which include all the activity and transactions within the various categories. The default reports do not include these details.
                                  • On Edit Validation Blueprint

                                    Hello, I have a notes field and a signature field. When the Approve button is clicked, the Signature field will appear and must be filled in. When the Reject button is clicked, the Notes field will appear and must be filled in. Question: Blueprint will
                                  • Zoho Projects - Cloning a task does not trigger task workflow when created

                                    Hello! I have a Project where my team uses a set of tasks from a tasklist as templates, so we could simply clone it and drag it to another list in kanban view to avoid creating a new one from scratch. The process works well, but after cloning it the new
                                  • Host Group Appointments Online in Zoho Bookings

                                    Greetings from the Zoho Bookings team! We’re excited to announce a new enhancement to Group Booking that makes hosting online group events smoother and more professional than ever. You can now conduct online group events with auto-generated meeting links
                                  • Can't rename groups on Mac desktop app

                                    I'm working on an up-to-date Mac with a freshly downloaded Notebook app. I'm trying to rename a group within a notebook. Here I have, left to right, a note, a group, and a note. I select the group. On the top left, I select Action. On the dropdown, "Rename"
                                  • Workdrive Collaboration with an External User

                                    I would like to know if I can setup a collaboration space with an external user in workdrive or do I need to add them as a user on my system? If I need to add them, can I add them on Workdrive only and give limit access to our space only?
                                  • Delete button

                                    Hi, The delete button were hide into the three dot button. Can I display outside? why Zoho make this update?
                                  • FSM integration with Books

                                    Hi, I have spent a few months working with FSM and have come across a critical gap in the functionality, which I find almost shocking....either that, or I am an idiot. The lack of bi-directional sync between Books and FSM on Sales Orders/ Work Orders
                                  • Marketing Tip #23: Help customers with how-to guides and usage tips

                                    Customers don’t stop needing you after they place an order. Helping customers use your product correctly and confidently can improve satisfaction, reduce returns, and increase repeat purchases. Sharing simple how-to guides, usage tips, or care instructions
                                  • Powering Customer Support with our women

                                    In Zoho Desk support, women make up 50% of our team. We see this as one of our strengths, reflecting the spirit of this year’s theme, "Give to Gain". Our women find their balance Women carry many responsibilities — they represent frontline support, lead
                                  • Function #25: Automatically generate purchase orders from a sales order

                                    We kicked off the "Function Fridays" series with the goal of helping you automate your everyday accounting tasks. As we delve into today's post, I'm delighted to announce that we're here to present the 25th custom function in this series. While it is
                                  • Looking for Guidance on Building a Zoho Website

                                    I'm exploring the possibility of building a custom website with specific features using Zoho as an alternative platform. My goal is to create something similar to https://gtasandresapk.com , with the same kind of functionality and user experience. I'd
                                  • Auto-publish job openings on my Zoho Recruit Careers Website

                                    I have developed a script using the Zoho Recruit API that successfully inserts new jobOpening records to my Zoho Recruit website, but my goal is to auto-publish to the Careers Website. The jobOpening field data shows two possible candidates to make this
                                  • ZOHO Reports are taking longer time to get refresh

                                    Hi Team, Since last few days, I'm facing issues in getting updated reports. For eg: right after making an expense entry or even posting a journal, it is taking longer then expected for the updated reports. Refer below: "You are viewing the report that
                                  • Invalid scope choice: Workdrive integration in CRM

                                    Bug: There is an invalid option in the permission choices for Workdrive integration in CRM. If the entry "WorkDrive.teamfolder.CREATE" is selected, it will return a message indicating invalid OAuth scope scope does not exist.
                                  • Sales IQ chat is not working in signed android apk

                                    I have integrated ZOHO sales IQ support chat and i have followed each step and its working fine in my development build but when i create signed APK for it. Chat does not work in it and showing awaiting for detail. I previously asked the same query but
                                  • How to add line breaks in zoho.cliq.postToUser(...) message?

                                    In a CRM function using Deluge I'm sending this message and attempting to add some line breaks but they are ignored. Is there another way to add these breaks? My message: message: New urgent task\nDescription \nThis is a fake description.\n A new line?
                                  • Enable Free External Collaboration on Notecards in Zoho Notebook

                                    Hi Zoho Notebook Team, I would like to suggest a feature enhancement regarding external collaboration in Zoho Notebook. Currently, we can share notes with external users, and they are able to view the content without any issue. However, when these external
                                  • Problem with CRM Connection not Refreshing Token

                                    I've setup a connection with Zoom in the CRM. I'm using this connection to automate some registrations, so my team doesn't have to manually create them in both the CRM and Zoom. Connection works great in my function until the token expires. It does not refresh and I have to manually revoke the connection and connect it again. I've chatted with Zoho about this and after emailing me that it couldn't be done I asked for specifics on why and they responded. "The connection is CRM is not a feature to
                                  • Update Existing Records greyed out in Free Version

                                    Trying to update records from an Excel sheet, and not getting the option to update. Only option is to add as new accounts. All documentation I can see says update should be an option! Accounts, Leads, Contacts, all the same.
                                  • In Lesson Video

                                    Can anyone help me with this? I'm not sure what happened. It suddenly became like that. I tried to reupload. I tried create a new lesson. Still the same. Please help!
                                  • emailing estimates

                                    Shows up in the customer mail logs as sent but nobody is receiving them, even when I send them to myself I don't get them ??? Something wrong with the mail server or my end ?
                                  • Help with deluge script

                                    Hi Community, this is my first Deluge script. I've pieced it together from reading various articles I want to use it in a workflow to 1 Convert a lead to a contact 2. Create a record in a custom module Below is what I have got so far but it does not fire
                                  • How to use OR when filtering using two fields

                                    I want to create return a list of Account Names by filtering on Field1 = "yes" OR Field 2 = "no" I can't see how to use the OR in the filter.
                                  • Mobile phone version not working well

                                    I am working on the Zoho Site Builder. In the preview the desktop version looks okay, but in the mobile phone preview many words are cut off in the weirdest (wrong) way. How can I fix that?
                                  • Zoho - Please explain difference between Thread view and Conversation view on Ticket

                                    I have reviewed the help document here but am still not clear on the difference between the two views. As an example, I just had a back and forth on a ticket: - Customer emails support email. - I email back from Desk. - Customer responds back. - I email back from Desk. On the upper left drop down box on the ticket Zoho Desk now says this is "4 Threads" and "4 Conversations" . How is that 4 threads?? By my count it is 1 thread and 4 conversations (assuming by "conversation" Zoho means number of total
                                  • Next Page