Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Changing Account in Quote form does not update address information.

      I am trying to update the address information in a quote I've created. I corrected the address in the "Account" but that did not change in the quote. If I re-enter the Account Name in the Quote form, nothing updates. How do I fix this?

    • Zoho One Backup of entire account

      Hello, When using Zoho one is there a way to backup your entire account of all apps that you are using \ activively using in a single step or do you have to backup each applications data individually? Thanks,

    • ZIA in Zoho Cliq

      Is It possible to use the ZIA feature from Zoho Analytics in the Zoho Cliq?

    • Issue with “CC” and “Subject Details” of the initial mail when reply / replied all / forward using Zoho Mail Client (Desktop / Web Mail / Mobile App)

      It is observed that when I reply / reply all / forward a mail using Zoho Mail Client (Desktop / Web Mail / Mobile App), the “CC” and “Subject Details” are omitted from the mail which was replied/forwarded. However this is not the case with outlook mail

    • Unable to send Emails - 452 4.3.1 Temporary System Error

      Whene ever i request smtp server to send the email (without attachment). i recieve error "452 4.3.1 Temporary System Error"

    • I can't receive email

      I cannot receive any email sent to my Zoho email after the free upgrade plan trial is finished.

    • Help for the alisa adding

      Sorry, I would like to add a paypal alias on my domain email address. However, the system blocked it. How can I do it?

    • Problem with Writer and Workdrive

      Hi team, I’m the super admin for our Zoho One org. WorkDrive is active, and Zoho Docs is deprecated for our org. However, Zoho Writer cannot connect to WorkDrive at all — we’ve cleared cache, tried incognito, and restarted several times. I was able to

    • Lite plan attachment said 250mb but actually 25mb ?

      Lite plan attachment said 250mb but actually 25mb ? I can't attach over 25mb files, and can't receive mails has attached files over 25mb too

    • Zoho Books | Product updates | August 2025

      Hello users, We’ve rolled out new features and enhancements in Zoho Books. From the right sidebar where you can manage all your widgets, to integrating Zoho Payments feeds in Zoho Books, explore the updates designed to enhance your bookkeeping experience.

    • Can no longer export as pdf

      In NOTEBOOK I am no longer able to export notes to pdfs as I used to be able to. When I hit that command it asks where I want to send it, but no matter what I chose there is no pdf attachment that shows up in order to send. Nothing happens.

    • Option to Customize Career Site URL Without “/jobs/Careers”

      Dear Zoho Recruit Team, I hope you are doing well. We would like to request an enhancement to the Career Site URL structure in Zoho Recruit. In the old version of the career site, our URL was simply: 👉 https://jobs.domain.com However, after moving to

    • Old vs New Value for Deleted Lookup Values

      Suppose the following scenario, where a value in a lookup is deleted: 1. User has countries form 2. Form A has a lookup to countries form 3. User selects Italy in Form A and saves it with the Italy ID 4. Form A report shows Italy 5. Italy is inadvertently

    • Desktop app doesn't support notecards created on Android

      Hi, Does anybody have same problem? Some of last notecards created on Android app (v. 6.6) doesn't show in desktop app (v. 3.5.5). I see these note cards but whith they appear with exclamation mark in yellow triangle (see screenshot) and when I try to

    • customize payment page

      Is there a way to customize, other than the theme colour, the payment page that customers are taken to from invoices? I can't seem to find a way. I just don't like the formatting of the current page and would like to make it look better. I've looked at

    • Are Environments Worth It?

      In concept, Environments in ZC is a great idea. I think the flow is pretty smart when you compare it to GitHub, especially for a low code audience. However, in practice, I've found it to be unpredictable, and I've only used it a few times. Aside from

    • Enhanced duplicate check for Leads in CRM

      Hello Everyone, We are excited to announce that you can now check for duplicate entries in leads by comparing them with similar records in the Contacts Module. Previously, when you added a lead, only the converted leads were checked for duplicates. This

    • Pause(1);

      I'm using scheduler to invoke an interaction via http post with an external service. The schedule code uses a for-each loop that runs so fast my external application's log files get messed-up (they are named by date-time stamp). What I'm suggesting is

    • Integration Request: Elementor

      Integrating Zoho CRM forms with Elementor, the most popular page builder on Wordpress, would be great. I use it for our site, goenergylink.com, and I have had to use Zapier webhooks to be able to connect it with Elementor. The one issue I have run into

    • Vertical images displayed as horizontal

      Some pictures that originally are in vertical position are displayed in horizontal after the upload. It seems that system rotates them by 90 degrees. How can I fix this issue?

    • Ability to Change Visibility of Published YouTube Videos

      Hi Zoho Social Team, How are you? We would like to request an enhancement in Zoho Social regarding the management of already published youtube videos. Currently, after publishing a youtube video through Zoho Social, there is no option to change its visibility

    • Adding anchor links in Zoho CRM email templates

      I know you can add anchor link in Campaigns, but I dont see the option to that in the CRM email template. Am I missing something?

    • Support “Never End” Option for Recurring Meetings in Zoho Meeting

      Hello Zoho Meeting Team, Hope you are doing well. We would like to request support for creating recurring meetings with no end date in Zoho Meeting. Currently, when scheduling a recurring meeting, Zoho Meeting requires us to select a specific end date.

    • openUrl in blueprints

      My customer wants to open a URL at the end of a blueprint transition. Seems this isn't possible right now but it would be very useful. In this thread, https://help.zoho.com/portal/en/community/topic/openurl-not-working the Zoho agent said that it's logically

    • Is it possible to pull the Zoho desk data into Zoho analytics in real time

      Hi - I am looking to add more dashboards and reports of Zoho Desk in analytics. I see there is a schedule to pull the data into analytics, but I'm wondering if there is an option to pull the data in real time instead of a specific interval?

    • CRM x WorkDrive: File storage for new CRM signups is now powered by WorkDrive

      Availability Editions: All DCs: All Release plan: Released for new signups in all DCs. It will be enabled for existing users in a phased manner in the upcoming months. Help documentation: Documents in Zoho CRM Manage folders in Documents tab Manage files

    • Introducing the revamped What's New page

      Hello everyone! We're happy to announce that Zoho Campaigns' What's New page has undergone a complete revamp. We've bid the old page adieu after a long time and have introduced a new, sleeker-looking page. Without further ado, let's dive into the main

    • Ability to Add YouTube Video to Playlist During Publishing

      Hi Zoho Social Team, How are you? While publishing YouTube videos through Zoho Social, we noticed that the platform currently does not allow selecting a playlist at the time of publishing. Instead, we can only add the video to a playlist after it has

    • Introducing Zoho Creator's 2025 Release Projection 2

      Hello Creators! I'm Prakash, from the Creator product management team, and today I'm delighted to unveil our next set of features as part of Release Projection 2 for 2025. With thoughtful analysis and planning, we've curated powerful new capabilities

    • Sharing Form Ownership Among Multiple Users

      I would really like the ability to share form ownership among multiple users. It's frustrating to me that if a co-worker wants to make an edit to a form, I have to transfer ownership to them. It would be great if forms could act like google forms, where multiple people can edit a form and view responses. 

    • Marketer’s Space - Ace Your Spooky-Season Marketing with Pre-designed Templates in Zoho Campaigns

      Hello marketers, Welcome back to another post in Marketer’s Space! We’re in Q4, which means that you have endless opportunities to connect with your audience, starting with Halloween campaigns! In this post, we’ll show you how to design the perfect Halloween

    • Zia expands to China with native features and DeepSeek-powered generative AI features

      Hello everyone, We are glad to support Zia native features and Zia generative AI features for our customers in China. From hereon, all AI-features in Desk will be accessible in China data center with the integration of DeepSeek generative AI model. DeepSeek

    • Email in each module

      We have a contact ,module which then has a link to customer assets which in turn the asset has a multiple link to service visits. When we link assets to customers we choose by name and it brings over the associate email via the lookup. Great feature.

    • Global Search placement in the new UI

      Having a hard time with the global search placement in the UI redesign. Surely I can't be the only one. Previously global search placement was perfect. A bar at the top/center of the page. Exactly where you would expect it to be. Since the new UI has

    • Introducing Skill-Based Ticket Assignment

      The goal of every support team is to provide great support, and to do so as fast as they can. To make this possible, it is important that agents spend their time judiciously, especially when they're dealing with a large number of tickets of varying urgency

    • Kaizen #213 - Workflow APIs - Part 1

      Welcome to another week of Kaizen! If you have ever managed complex business processes, you know that Workflows are the quiet backbone of any well-run business process. They keep things moving; assigning owners, sending alerts, keeping deals on track,

    • Browser and address bar hide

      Hi, How i can do hide the address bar with browser headline when i am working on the sheet, because i am using (freeze panes) which i want visible for full work. For your reference here i am attached the screen shot and marked yellow lines which really

    • Cells Border

      Hi I am using Zoho Sheet on S Tab , is there any option to make all border of any cell at once. I think this is very basic which we are missing. This is available in mobile but not in tab or suggest if i am missing this function. And for Tab can you give

    • Credit Management: #2 Configuring Right Payment Terms for Credit Control

      Think about the last time you ordered something online and saw that little note at the checkout, "Pay on Delivery" or "Pay later". It's simple, but it actually sets the tone. As a business owner, you know exactly when payment is expected. Now, imagine

    • Zobot and Sales IQ

      What will happen to the Zoho Sales IQ being integrated to the website after creating the Zobot on the website too

    • Next Page