Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Will I see emails sent via campaigns in CRM?

      It would be useful for people to be able to see emails sent via campagins in Zoho CRM is that possible?

    • Zoho ZUGs Meetups 2026 en España: Descubriendo nuevas posibilidades con Zoho

      ¡Hola, Español Zoho Community! Hemos celebrado una nueva edición de los Zoho User Groups (ZUGs) en España (Madrid, Barcelona & Valencia), una excelente oportunidad para reunirnos, formarnos y descubrir nuevas formas de sacar más partido a nuestras herramientas

    • What is MCP and How Does It Connect to Zoho Invoice?

      If you've ever wished you could just tell your invoicing software what to do, without clicking through menus, pulling up reports manually, or switching tabs every five minutes, that's exactly what the Zoho MCP server is built for. MCP stands for Model

    • DYK 7 - Set dependency from Gantt and List view

      Did you know task dependencies in Zoho Projects can be managed across different task views? In a project, multiple teams work together towards the same goal but team members prefer accessing data with different perspectives depending on their responsibilities.

    • Goods In Transit

      Hi I was wondering if anyone had any tips or work-around to show goods in transit as a liability? We raise a PO with our supplier in China and when the goods are shipped and are invoiced, hence they are a liability of the business. However I can't convert

    • Accounts Payable and receivable

      I'm currently creating my accounts and noticed that I cannot create any more accounts payable or receivable. Is there any way to create more of this accounts and associate them to the default one available?

    • How to display Pick List values with vertical separation in Canvas

      I have a Multi-Select Pick List whose values I would like displayed vertically. So far, I have only been able to figure out how to adjust the width and other sizing of the field itself or even how to wrap the values to the next line below. I am trying

    • Introducing a knowledge management and training software in Zoho Projects Plus

      Keeping your workforce updated on emerging technology trends is essential to stay on top of the game. While project management software focuses on planning, execution, and project tracking, incorporating a learning management solution within project management

    • Problem with UTM Parameters: Zoho Forms - Zoho Desk Integration

      Hi Zoho Support Team, I want to automatically capture UTM Parameters from my website URLs and pass it from Zoho Forms into Zoho Desk. I have activated the UTM tracking feature. I've integrated the UTM Tracking code in my website footer on all pages. I've

    • #13 Fix Mistakes Without Breaking Your Records

      A customer purchases 100 custom-printed invitations for an upcoming event. A few days after delivery, they call to say that some invitations were damaged and others need to be reprinted. They request a return. The customer isn't asking for a completely

    • Zoho Directory : transformez vos données d’identité en décisions de sécurité

      Que sont les rapports d’identité dans Zoho Directory et pourquoi sont-ils importants? À mesure que les organisations se développent, la gestion des identités devient plus complexe. Les informations existent, mais elles sont souvent réparties entre plusieurs

    • How to use Zoho Billing info in a workflow

      We have Zoho Billing and CRM sync'd and the Billing info appears as expected for each Account in Zoho CRM, including subscription status, plan type etc. But how can we use this in a workflow? Thanks Dylan

    • Why can't we choose Fixed Asset account for Purchased Items? (eTims issue?)

      Hello, When the company purchase items not for sale and not supposed to be in the inventory stock, like equipment for operational use, there is no way to access the Fixed Asset accounts in the drop down list. Is that an eTims limitation again? Or something

    • Auto-approval for shared key access not working for new users

      Hello, I'm an admin of a Zoho Vault organization (34 active users, ZOHO identity provider). I have the "Auto-approval for shared key access" setting enabled in Settings → Fine-Grained Controls → Global, but it does not work as expected. The problem: When

    • Zoho Analytics UI Bug

      Hello, all, The Aggregate Formulas in Zoho Analytics' list of fields have a 3-dot menu: If the formula has a long name, the name is not fully visible in the list, and a 3-dot menu is not accessible from it: This is not convenient, especially when the

    • Canvases Auto-Skewing/Adding Scroll Bars When They Were Not There Prior

      Is anyone else noticing rendering issues in their canvases today? It seems to be mainly icons which now have scroll bars added which makes them all look off, though some fields seemed to revert to squished length as well. Were the icons replaced with

    • Marketing Tip #37: Reduce purchase doubts with better product images

      When customers shop online, they cannot pick up, turn around, or inspect a product in person. That is why your product images need to do more of the selling. Two simple upgrades can make a big difference: Use zoom-friendly images so customers can look

    • Zia Agent built in ChatKit UI does not render markdown

      Hi, You have a major shortcoming in the Zia Agent UI. The test UI that is embedded in agents.zoho.com allows you to test the agent has full support for rendering markdown, but your ChatKit UI does not have support for rendering markdown. If I embed it

    • GLM 5 not available

      Hello, I am trying to setup a Zia Agent using agents.zoho.com. The settings says that GLM5 is among the list of free zoho hosted models available. However, when I try to setup an agent and pick a model from the list only GLM 4.7 Flash is available. How

    • Supervisor Rules --> Custom Function

      Hello, currently I can't add a custom function to a supervise rule. Is there a reason for this? Background: We have BluePrint managed tickets and actually we have a Supervise rule which should set the ticket to "closed" after 168 hours since the last

    • Vault Chrome Extension Asks for Master Password every single time

      Is this supposed to be a feature, or is this a bug?  My assumption is that it's a bug, because every time I click on the Extension, there is a red exclamation mark/notification on the Zoho Vault extension.  Is the idea with the Extension that I have to

    • Permission Denied for Worklogs URL

      We're attempting to pull worklogs data from service desk plus and when using the below URL we are met with a message stating we do not have permission. https://sdpondemand.manageengine.com/api/v3/requests/xxxxxx/worklogs/ We used SDPOnDemand.requests.ALL

    • Anyone in Australia using Zoho Books AND has their account with NAB?

      Hi I have an account with both NAB and Suncorp. Suncorp transaction come in the next day however NAB transactions take 4-5 business days to appear. eg: A deposit made today in my Suncorp will be imported into Zoho tomorrow. A deposit made today to the NAB account will be imported maybe Saturday (Friday overnight). I have contacted both Zoho and NAB but noone seems to know why. I was just wondering if anyone else in Australia uses NAB and has this issue (or doesn't) maybe we could compare notes and

    • Partner with HDFC And Sbi Bank.

      Hdfc and sbi both are very popular bank if zoho books become partner with this banks then many of the zoho books users will benefit premium features of partnered banks.

    • #12 Never Leave a Billable Hour Behind

      A client approves a website redesign project. The estimated effort? 40 hours. The project is going well. A few additional review meetings are scheduled. Several rounds of content changes are requested. A few "quick fixes" get added along the way. The

    • Close task on Completion Date entry

      This discussion is similar to my issue: "backdated-task-completion-date" discussion Scenario: A bunch of tasks have been completed. But, they have not been closed. Time goes by You finally get around to closing those tasks Projects assigns the date the

    • Can I add Conditional merge tags on my Templates?

      Hi I was wondering if I can use Conditional Mail Merge tags inside my Email templates/Quotes etc within the CRM? In spanish and in our business we use gender and academic degree salutations , ie: Dr., Dra., Sr., Srta., so the beginning of an email / letter

    • Records from ATE 29: Knowledge Base, Community, and AI for smarter user education

      Hi Everyone, "Ask the Experts 29" was an engaging session, where we explored how to utilize the Knowledge Base for customers and internal teams, as well as emphasizing the importance of our Community. This post highlights the questions and use cases discussed,

    • Kaizen #246 - Incoming Lead Email Intent Detection using Zia Assistant API in Zoho CRM

      Hello all! Welcome back to a fresh Kaizen week. In this post, we will explore how Zia detects positive intent from incoming emails in the Leads module using the Zia Assistant API along with Workflow Rules and Custom Functions in Zoho CRM. Use Case Problem

    • Smart URL Determination

      I would like to see Vault implement some sort of "smart" URL determination. When one starts to add a new username-password combination from a new site, Vault brings in the exact URL of the page from which this is happening. All too often, it looks something

    • Removing To or CC Addresses from Desk Ticket

      I was hoping i could find a way to remove unnecessary email addresses from tickets submitted via email. For example, a customer may email the support address AND others who are in the helpdesk notification group, in either the TO or CC address. This results

    • Tip #76- Exploring Technician Console: Quick Launch- 'Insider Insights'

      Hello Zoho Assist Community! Welcome back to our Technician Console series. Last time we explored Power Options, and this time we are turning the spotlight on a feature that quietly saves you dozens of clicks in every session by getting you exactly where

    • Multi-Book Accounting Support in Zoho Books

      Currently, businesses that operate multiple entities, regions, or divisions need to maintain separate Zoho Books instances or resort to manual consolidation processes. This creates significant operational friction and increases the risk of errors. PROBLEM:

    • Automated Multi-Subsidiary Consolidation Engine in Zoho Books

      For organizations managing multiple subsidiaries across different geographies or business units, consolidation is a quarterly/annual nightmare. Zoho Books lacks native consolidation tools, forcing companies to export data, manipulate it in Excel, and

    • Unable to charge GST on shipping/packing & Forwarding charges in INDIA

      Currently, tax rates only apply to items. It does not apply tax to any shipping or packing & forwarding charges that may be on the order as well. However, these charges are taxable under GST in India. Please add the ability to apply tax to these charges.

    • How to add packing & forwarding charge in purchase order & quotation???

      Hello Zoho Team I have just started using Zoho for my company and I wanted to make purchase order. My supplier charges fix 2% as packing & forwarding on Total amount of material and then they charge me tax. For example, Material 1 = 100 Rs Material 2

    • How to book GST paid in zoho books

      hi, i am a new user to Zoho books and not able to book GST paid in books, kindly suggest how i can book it in books. thanks, siddharth

    • [Bug] WebAuthn passkey registration blocked on rpIds with TLDs longer than 6 characters (.accountant, .technology, etc.) — isValidDomain regex too strict

      Hi, Filing on behalf of an enterprise customer where Zoho Vault is deployed across the company. The Chrome extension blocks WebAuthn passkey registration on legitimate sites whose Relying Party ID (rpId) has a TLD longer than 6 letters. This affects every

    • Zoho Payroll: Product Updates - H1, 2026

      Over the last few months, Zoho Payroll has added updates that make payroll easier to process, review, explain, and manage for businesses. The most important improvements focus on payroll flexibility, gratuity tracking, employee self-service, reporting,

    • Pushover Notification Module

      Hello, it would be good if there would be a "Pushover" (https://pushover.net/) module besides the standard SMS module. Pushover is now very well known, especially in IT, and is becoming more and more popular. The biggest advantage are the customizable

    • Next Page