Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team




      • Sticky Posts

      • Register for Zoho Desk Beta Community

        With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

      • Share your Zoho Desk story with us!

        Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

      • Tip #1: Learn to pick the right channels

        Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

      • Welcome to Zoho Desk Community - Say hello here!

        Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

      • Webinar 1: Blueprint for Customer Service

        With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

        • Recent Topics

        • New features in Zoho Vault

          We’re thrilled to introduce a wave of powerful updates in Zoho Vault, designed to enhance security, streamline workflows, and improve your overall experience. Let’s dive into what’s new! Folder creation restrictions Limit who can create folders in your

        • Join our World Password and Passkey Day expert Q&A 2025

          Hey everyone! World Password and Passkey Day is almost here, and there's no better time to talk about something we all rely on daily—secure authentication. Did you know that a staggering 60% of hacking-related breaches are tied to weak or stolen passwords?

        • Dashlane discontinued its free plan: Here's why Zoho Vault's free plan is worth the switch

          Hey everyone, Dashlane password manager has officially announced that its free plan will be discontinued starting September 16, 2025. This change means that current free users will need to either upgrade to a paid subscription or export their data and

        • Introducing SecureForms in Zoho Vault

          Hey everyone, Let’s face it—asking someone to send over a password or other sensitive data is rarely straightforward. You wait. You nudge. You follow up once, twice—maybe more. And when the information finally arrives, it shows up in the worst possible

        • Clickjacking: Zoho Vault's Response

          Issue: Password manager browser extensions are found to be vulnerable to clickjacking security vulnerabilities that could allow attackers to steal account credentials, TFA codes, and card details under certain conditions. Reported by: Marek Toth, Independent

        • Free webinar: Security that works: Building resilience for the AI-powered workforce

          Hello there, Did you know that more than 51% of organizations worldwide have experienced one or more security breaches, each costing over $1 million in losses or incident response? In today’s threat landscape, simply playing defense is no longer enough.

        • Free webinar—Redefining workforce security with Zoho Vault: Passwords, passkeys, and multi-factor authentication

          Hi everyone! Did you know that in Q2 alone, 94 million data records were leaked globally? Behind every breach is a combination of poor password habits, phishing attacks, privilege misuse, and simple human error. The fallout—including reputational damage,

        • Audio/video quality issues with Zoho Meeting – Any roadmap for improvement?

          Hi Zoho Team, We’ve been using Zoho Meeting for both internal and external meetings, and unfortunately, the experience has been consistently poor. The video and audio quality are so unreliable that it often renders meetings ineffective—especially with

        • In arattai received message can't be deleted

          The issue has been noticed in following: arattai app (Android) arattai app (Window) arattai web While the message posted by me may be deleted, the ones received from others can't be. The item <Delete> change to <Report> when the message is a received

        • Fill Colors

          Hello, Just curious, is it possible to manage the Used and standard colors in the fill drop down menu ? I would like to change/edit them is possible to colors I frequently use. Thanks for you reply, Bruce

        • Is there a way to generate a virtual meeting for a group service in Zoho Bookings?

          Are virtual meetings not supported for group services/meetings? I have integrated Zoom with one-on-one services, but I need a way to create an online group meeting. Thanks

        • Attachment reminder?

          My team and I often need to attach files to our messages, e.g. an explanatory screenshot or a shipping label. More often that I want to admit I mention the attachment but forget to actually attach it. Some email clients have a check-for-missing-attachments

        • Items Below Reorder Point Report?

          Is there a way to run a report of Items that are below the Reorder Point? I don't see this as a specific report, nor can I figure out how to customize any of the other stock reports to give me this information. Please tell me I'm missing something s

        • Markdown for Desk?

          Hi, my company wants to use markdown for formatting text in Desk (in all modules there, especially Tickets and Helpcenter). Zoho already offers use of markdown in several products (see https://help.zoho.com/portal/en/kb/backstage/microsite-guide/formatting-with-markdown/articles/formatting-with-markdo)

        • Multiple upload field CRM

          I desperately need the functionality to add more than one upload field to web to contacts form. How can I do this?

        • Client scripts for Zoho Books ?

          Good day everyone, I am looking for a way to be able to interact with the Quotes and Invoices as they are being created. Think of it like Zoho client script in Zoho CRM. But for the life of me I dont see a way to do this. The issue with having function

        • Fillable template with dynamic tables?

          Is there a way to build a fillable template so that users can add rows to a table? To describe what I'm trying to accomplish the table has 3 sections; a header row, some number of rows with custom information, and a summary row with totals. I can't figure

        • Invoicing multiple expense items as a single line item

                My client would like me to invoice them for multiple expense items on a single line item on an invoice.  Can this be done?  I know I can import billable expenses to an invoice, but I don't know how to show them as a single line item or have them

        • Zoho Books | Product updates | November 2025

          Hello users, We’ve rolled out new features and enhancements in Zoho Books. From translating email notification templates to the new transaction locking restrictions, explore the updates designed to enhance your bookkeeping experience. Making Tax Digital

        • Tables improvement ideas / features

          Heya, I've been using Zoho Tables for a few months now and wanted to post some features that I think will be greatly beneficial for the tool: 1. Ability to delete a record in automation or move a record in automation. - Usecase: I move a record from one

        • Announcing offline payment modes for event tickets

          Hello everyone,   Collecting ticket payments from prospective attendees can be a tricky part of event planning. While Zoho Backstage has always allowed you to collect payments online as people buy tickets from your event microsite, we have now enabled

        • What is going on with GMail extension.

          Suddenly, it is popping up and saying "Please wait while we work our magic... 10%" and it sits there. Yesterday I waited for an entire hour call and it never got further. Today, it's not resolved. Is there a fix I can do? I've tried all the typical

        • Gmail is ramping up its email sender policy from November 2025

          Hello marketers, As you may be aware, Gmail introduced a guideline for bulk senders starting February 2024. If not, here's a quick refresher straight from Google: After this policy was announced first in 2023 and soft-implemented in February 2024, we

        • Collapsible Sections & Section Navigation Needed

          The flexibility of Zoho CRM has expanded greatly in the last few years, to the point that a leads module is now permissible to contain up to 350 fields. We don't use that many, but we are using 168 fields which are broken apart into 18 different sections.

        • Gmail is ramping up its email sender policy as of November 2025

          Hello marketers, As you may be aware, Gmail introduced a guideline for bulk senders starting February 2024. If not, here's a quick refresher straight from Google: After this policy was announced first in 2023 and soft-implemented in February 2024, we

        • Good news! Calendar in Zoho CRM gets a face lift

          Dear Customers, We are delighted to unveil the revamped calendar UI in Zoho CRM. With a complete visual overhaul aligned with CRM for Everyone, the calendar now offers a more intuitive and flexible scheduling experience. What’s new? Distinguish activities

        • Deluge Error Code 1002 - "Resource does not exist."

          I am using the following script in a Custom Button on a Sales Return. Basically, the function takes the information in the sales return (plus the arguments that are entered by the user when the button is pushed) and creates a return shipping label via

        • Zoho Analytics Export API

          Hi Team, I’m working on some integration tasks and wanted to confirm if it’s possible to retrieve a Zoho Analytics table as JSON data using a Deluge script. I’ve already stored my custom data from multiple sources and combined it into a single source.

        • Item images

          Can we get an "On hover" expanded image for items please ?

        • Ability to Create and Manage YouTube Video Chapters in Zoho Social

          Hi Zoho Social Team, How are you? We would like to request an enhancement to Zoho Social that would allow users to create and manage YouTube video chapters directly within the platform. Currently, Zoho Social does not support manually adding chapters

        • What happens to my current site SEO if i opt for zoho creator?

          I have an existing website and I need to use Zoho creators for the rapid creation of my webpage creation. Currently, my IT team is creating the web pages, but I am concerned about the SEO of my current website if I shift to zoho will i loose it all?

        • Automatic Portal invite

          We have numerous customers we move through a blueprint in deals, when they get to a certain point we need to give them portal access, how can this be done through deluge or a workflow?

        • Preview Emails with Merge Fields before sending

          Hello, Are there plans to preview an email with merge fields before sending out? Currently, all you see in the preview are the merge field values, but before I send out the email I want to make sure the merge is working. Also, the saved emails only show

        • User Filter not selecting All Items

          We are encountering 2 issues when using the user filter. When users are trying to search using the filter option, the OK button is grayed out. Users have to unselect or make a change before it filters properly. 2. When filtering and the OK button works,

        • Support Nested JavaScript Variables in PageSense Pop-up Targeting

          Hello Zoho PageSense Team, I hope you're doing well. I’d like to request a feature enhancement for the PageSense pop-up targeting functionality. Current Limitation: Currently, PageSense pop-ups can be triggered using simple JavaScript variables. However,

        • Cannot Invert Axis for Rankings

          Hi there I want to be able to create a ranking graph in Analytics/BI, with 1 at the top of the Y axis, but I am unable to invert the axis. Super simple example in Excel below. Higher rankings need to be higher up on the graph to give the correct visual

        • How to copy value from a single line field into a picklist field within a module's subform?

          Hello there, I have a single line field in a module's subform. I would like the value in the field to automatically update a picklist field within the same subform (both have items with the same names). Is this possible via function? Unfortunately, workflows

        • Committed Stock and To Be Received Stock via API?

          Is it possible to retrieve Committed Stock and/or To Be Received Stock for an Item via the API? I want to use this information for calculating the amount of inventory needed to be purchased.

        • Creating Email template that attaches file uploaded in specific field.

          If there's a way to do this using Zoho CRM's built-in features, then this has eluded me! I'm looking to create a workflow that automatically sends an email upon execution, and that email includes an attachment uploaded in a specific field. Email templates

        • Marketer’s Space - Automate a Personalized Holiday Workflow with Zoho Campaigns

          Hello marketers, Welcome back to another post in Marketer’s Space! From Thanksgiving through Christmas, you have a flurry of opportunities to connect with your audience. In this post, we’ll see how you can plan an entire month-long automated workflow

        • Next Page