Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere









                          Zoho Developer Community




                                                • Desk Community Learning Series


                                                • Digest


                                                • Functions


                                                • Meetups


                                                • Kbase


                                                • Resources


                                                • Glossary


                                                • Desk Marketplace


                                                • MVP Corner


                                                • Word of the Day


                                                • Ask the Experts



                                                          • Sticky Posts

                                                          • Register for Zoho Desk Beta Community

                                                            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                          • Share your Zoho Desk story with us!

                                                            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                          • Tip #1: Learn to pick the right channels

                                                            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                          • Welcome to Zoho Desk Community - Say hello here!

                                                            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                          • Webinar 1: Blueprint for Customer Service

                                                            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                          Manage your brands on social media



                                                                Zoho TeamInbox Resources



                                                                    Zoho CRM Plus Resources

                                                                      Zoho Books Resources


                                                                        Zoho Subscriptions Resources

                                                                          Zoho Projects Resources


                                                                            Zoho Sprints Resources


                                                                              Qntrl Resources


                                                                                Zoho Creator Resources



                                                                                    Zoho CRM Resources

                                                                                    • CRM Community Learning Series

                                                                                      CRM Community Learning Series


                                                                                    • Kaizen

                                                                                      Kaizen

                                                                                    • Functions

                                                                                      Functions

                                                                                    • Meetups

                                                                                      Meetups

                                                                                    • Kbase

                                                                                      Kbase

                                                                                    • Resources

                                                                                      Resources

                                                                                    • Digest

                                                                                      Digest

                                                                                    • CRM Marketplace

                                                                                      CRM Marketplace

                                                                                    • MVP Corner

                                                                                      MVP Corner







                                                                                        Design. Discuss. Deliver.

                                                                                        Create visually engaging stories with Zoho Show.

                                                                                        Get Started Now


                                                                                          Zoho Show Resources

                                                                                            Zoho Writer

                                                                                            Get Started. Write Away!

                                                                                            Writer is a powerful online word processor, designed for collaborative work.

                                                                                              Zoho CRM コンテンツ






                                                                                                Nederlandse Hulpbronnen


                                                                                                    ご検討中の方




                                                                                                          • Recent Topics

                                                                                                          • Create View to See Tickets Closed within the last 3 days

                                                                                                            I'm trying to create a view in Zoho Desk that shows me "recently closed ticket," which I will define as tickets closed in the last 3 days. I want this view to update so that whenever I click to view it is recalculates and shows me tickets closed within
                                                                                                          • Slow Performance on desk.zoho.com. 11/3/2025

                                                                                                            I’m not seeing any active service alerts for desk.zoho.com, but everyone on our account is currently experiencing very slow load times when opening or navigating tickets. We’ve already tried the standard troubleshooting steps — clearing cache and cookies,
                                                                                                          • Canvas View in Zoho Recruit

                                                                                                            Is it possible or would it be possible to have the new 'Canvas View' in Zoho Recruit?
                                                                                                          • Allow Multiple usage units to items while adding them to sales/purchase transactions

                                                                                                            The usage unit of items added in zoho books are static right now and can not be changed. But certain items are received or sold in multiple usage units. One example is fabric. It can be bought in Meters, inches, kgs or other units. Another example would
                                                                                                          • Will I Get a Refund If I Downgrade Zoho Mail?

                                                                                                            Hello, We upgraded an email account for our new employee. However, the employee left after one month, and now I've reduced the number of Zoho Mail users from 7 to 6. Can we get a refund for the remaining portion of our annual payment?
                                                                                                          • Introducing Profile Summary: Faster Candidate Insights with Zia

                                                                                                            We’re excited to launch Profile Summary, a powerful new feature in Zoho Recruit that transforms how you review candidate profiles. What used to take minutes of resume scanning can now be assessed in seconds—thanks to Zia. A Quick Example Say you’re hiring
                                                                                                          • How to copy value from a single line field into a picklist field within a module's subform?

                                                                                                            Hello there, I have a single line field in a module's subform. I would like the value in the field to automatically update a picklist field within the same subform (both have items with the same names). Is this possible via function? Unfortunately, workflows
                                                                                                          • Send Whatsapp message from Whatsapp template with custom variables

                                                                                                            Hi, I'm trying to do some basic integration for sending WhatsApp messages from Zoho CRM using Zoho Desk whatsapp templates. When creating new whatsapp template in Zoho Desk we can choose ticket related fields as variables but it's not clear how to use
                                                                                                          • Recurring Events Not Appearing in "My Events" and therefore not syncing with Google Apps

                                                                                                            We use the Google Sync functionality for our events, and it appears to have been working fine except: I've created a set of recurring events that I noticed were missing from my Google Apps calendar. Upon further research, it appears this is occurring
                                                                                                          • Dissociate account from contact

                                                                                                            Hello, When I make the call through the API to "Dissociate account from contact" using, /api/v1/contacts/**integer**/accounts/**integer**/dissociate I get this response back, 422Unprocessable Entity (WebDAV) (RFC 4918) { "errorCode": "INVALID_DATA", "message":
                                                                                                          • Zoho Desk and Zoho People Integration - is it possible in order to alert whether an agent is available

                                                                                                            Hi, We use Zoho People for our employees to log absence (Paid Time Off etc). I was wondering if it was possible that this information can be shared in to Zoho Desk, so that when allocating a ticket to an agent, if they are on leave there is an onscreen
                                                                                                          • Add Save button to Expense form

                                                                                                            A save button would be very helpful on the expense form. Currently there is a Save and Close button. When we want to itemize an expense, this option would be very helpful. For example, if we have a hotel expense that also has room service, which is a
                                                                                                          • Change Currency symbol

                                                                                                            I would like to change the way our currency displays when printed on quotes, invoices and purchase orders. Currently, we have Australian Dollars AUD as our Home Currency. The only two symbol choices available for this currency are "AU $" or "AUD". I would
                                                                                                          • Call transcrition working for ringcentral?

                                                                                                            I don't see anything about what telephony providers can be used. The Zoho support person A said that RingCentral isn't supported. Zoho support person B said that it works, just make sure the call recording link works. Excellent instructions here: Call
                                                                                                          • How to Send Email from within a custom module (with or without an email template)

                                                                                                            It is possible to send an email from the Deals module. However, I can't find a way to send an email from any of our custom modules. I have tried adding an email field to the modules (even though we don't really want one or need it there). That doesn't
                                                                                                          • All new Address Field in Zoho CRM: maintain structured and accurate address inputs

                                                                                                            The address field will be available exclusively for IN DC users. We'll keep you updated on the DC-specific rollout soon. It's currently available for all new sign-ups and for existing Zoho CRM orgs which are in the Professional edition. Managing addresses
                                                                                                          • Zoho Inventory - Managing Items With Multiple Suppliers

                                                                                                            Hi community, I'm working on a project at the moment for a car parts wholesale business. Each Item (part) has its own original manufacturer part number (sometimes more than one part number for the same item). When purchasing parts from 3rd party suppliers,
                                                                                                          • Empowered Custom Views: Cross-Module Criteria Now Supported in Zoho CRM

                                                                                                            Hello everyone, We’re excited to introduce cross-module criteria support in custom views! Custom views provide personalized perspectives on your data and that you can save for future use. You can share these views with all users or specific individuals
                                                                                                          • Zoho Creator customer portal limitation | Zoho One

                                                                                                            I'm asking you all for any feedback as to the logic or reasoning behind drastically limiting portal users when Zoho already meters based on number of records. I'm a single-seat, Zoho One Enterprise license holder. If my portal users are going to add records, wouldn't that increase revenue for Zoho as that is how Creator is monetized? Why limit my customer portal to only THREE external users when more users would equate to more records being entered into the database?!? (See help ticket reply below.)
                                                                                                          • Super Admin Logging in as another User

                                                                                                            How can a Super Admin login as another user. For example, I have a sales rep that is having issues with their Accounts and I want to view their Zoho Account with out having to do a GTM and sharing screens. Moderation Update (8th Aug 2025): We are working
                                                                                                          • Bug tracking

                                                                                                            Hi, does anyone know how to track errors during picking or packing? This way I can keep track and see how to improve and prevent errors in this area.
                                                                                                          • Retainer invoice in Zoho Finance modlue

                                                                                                            Hello, Is there a way of creating retainer invoices in the Zoho Finance module? If not can I request this is considered for future updates please.
                                                                                                          • Is it really true that I can't set the default 'deposit to' account in 2025?

                                                                                                            I've been using Books for 7 years and the default account has never been a problem. I usually manually reconcile invoices and have never had a thought about which account. It has always been my account. However, I recently noticed that for the past 4
                                                                                                          • How do I see the total leads during a certain period?

                                                                                                            I understand I can get the count of leads and potentials but the total number of leads in a certain period should be equal to Leads+potentials because when we convert a lead it gets moved to potentials and no longer exists there. is there a way i could
                                                                                                          • Unable to Add Asset to Work Order Due to Different Contact and Company

                                                                                                            Hello Zoho Team, I’m facing an issue in our environment related to asset linking in Work Orders. Here’s the situation: When our engineers go for field activities, they initially create the Work Order using the salesperson’s contact details. After completing
                                                                                                          • Where is the Global Search field?

                                                                                                            I am looking for an alternative to SF.com. Zoho CRM seems to be work fine, and be customizable in terms of the fields and reports. But there's one big thing missing and it's going to prevent us from using it: there's no global search box at the top of
                                                                                                          • API 500 Error

                                                                                                            Hello amazing ZOHO Projects Community, I get this message. How can we solve this? { "error": { "status_code": "500", "method": "GET", "instance": "/api/v3/portal/2010147XXXX/projects/2679160000003XXXX/timesheet", "title": "INTERNAL_SERVER_ERROR", "error_type":
                                                                                                          • In place field editing for candidates

                                                                                                            Wondering about any insight/best practices for efficiently updating candidate records while reviewing them in a Job Opening pipeline. We can do in-field editing (e.g. update job title or City) only when we have the full candidate record open, however
                                                                                                          • Audio/video quality issues with Zoho Meeting – Any roadmap for improvement?

                                                                                                            Hi Zoho Team, We’ve been using Zoho Meeting for both internal and external meetings, and unfortunately, the experience has been consistently poor. The video and audio quality are so unreliable that it often renders meetings ineffective—especially with
                                                                                                          • Default tax type for mileage?

                                                                                                            Where we are, mileage includes a government tax. Is there any way to set a default tax for the Personal Car Mileage category of expense? (Or any other way?)
                                                                                                          • Analytics Portal

                                                                                                            I have the "standard plan" and want to explore the portal option; I activated the 15-day trial but do not see the pricing for the add-on. How can I get the price under "Upgrade add-ons." Thanks Rudy
                                                                                                          • The Social Wall: October 2025

                                                                                                            Hello everyone, As we head toward the end of the year, we’re bringing you a few updates to help give your social media efforts a strong finish. This month, we’re rolling out new enhancements across both the web and mobile app. Post Preview Have you ever
                                                                                                          • Show price book list price

                                                                                                            When using price books, once you add products to the price book in the Products related list you can display the Unit price which is the default list price; however, there is no option to show the price book list price. To see the price book list price
                                                                                                          • Zoho CRM - Scheduled Reports Which Contain Chart

                                                                                                            Hi Zoho CRM Team, I'm requesting that the Report Export and Scheduling feature be enhanced to include a chart, if one has been created on a report. At the moment I have a report which shows Sales This Week by Deal Owner and a pie chart at the top of the
                                                                                                          • Cliq File Upload

                                                                                                            While uploading large file like 500MB, it takes time, that fines. But if you resize window or move window in other screen, that uploading disappears. After upload complete & sent it will be visible
                                                                                                          • Resizing a Record Template Background Inage

                                                                                                            Hi everyone, I have an issue which I can't seem to resolve: Basically, I'm designing a record template in certificate form. I've specified A5 landscape. I've set my background image the same dimensions with total pixels at 443,520. Whatever I try, when
                                                                                                          • Analytics : How to share to an external client ?

                                                                                                            We have a use case where a client wants a portal so that several of his users can view dashboards that we have created for them in Zoho Analytics. They are not part of our company or Zoho One account. The clients want the ability to have user specific,
                                                                                                          • Zia Actions: AI-powered Workflow Automation for Faster and Smarter Execution

                                                                                                            Hello everyone, Workflows got a notch better with AI-based actions. Actions such as field extraction, prediction, auto reply, and content generation facilitate quick execution with improved speed and accuracy. Zia can intercept useful details in newly
                                                                                                          • Constant color of a legend value

                                                                                                            It would be nice if we can set a constant color/pattern to a value when creating a chart. We would often use the same value in different graph options and I always have to copy the color that we've set to a certain value from a previous graph to make
                                                                                                          • What's New in Zoho POS - October 2025

                                                                                                            Hello everyone, Welcome to Zoho POS's monthly updates, where we share our latest feature updates, releases, changes, and more. Let’s take a look at how October went. Process returns for refunds, exchanges, or offer store credit Returns and exchanges can
                                                                                                          • Next Page