Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Need Help: Updating Related Records via Subform Entries in Zoho Creator

      Hi everyone, I’m trying to set up a workflow in Zoho Creator where each row in a subform updates related records in another form automatically. Here’s the situation: My main form (e.g., “Receipts”) contains a subform where each row selects a related record

    • Zoho Desk - Community - Customer Portal - Description Field UX Improvement

      Hi Zoho Desk Team, As a prolific user of Zoho Cares Community, I find it very frustrating that I cannot increase the size of the Description box (this one which I am typing this message). Many apps with multi line text fields have a small handle in the

    • Add the same FROM email to multiple department

      Hi, We have several agents who work with multiple departments and we'd like to be able to select their names on the FROM field (sender), but apparently it's not possible to add a FROM address to multiple departments. Is there any way around this? Thanks.

    • Can I change the format of the buttons in the email templates?

      Hi all! We have been working hard trying to brand our email templates, and have some way to go yet. One of the things we can't seem to edit is the green ${Cases.CUSTOMER_PORTAL_BUTTON} button and the font of the View Ticket text. Is there any way of doing

    • Introducing parent-child ticketing in Zoho Desk [Early access]

      Hello Zoho Desk users! We have introduced the parent-child ticketing system to help customer service teams ensure efficient resolution of issues involving multiple, related tickets. You can now combine repetitive and interconnected tickets into parent-child

    • Zero Personalization of the File Sharing Experience

      By now (2025) this is the maximum level of personalization available for a Zoho sharing link. We gently asked Zoho if we could modify at least the background, and they replied that it cannot be customized. We're truly disappointed – and surprised every

    • Lead to Contact Conversion with multiple email address fields

      We are a B2C business with a strong repeat cycle, and as such it's not uncommon for customers to use multiple email addresses with us. We have both our Contacts & Leads modules set up with 3 email fields. (Primary Email / Secondary Email / Historic Email)

    • Introducing Dark Mode / Light Mode : A New Look For Your CRM

      Hello Users, We are excited to announce a highly anticipated feature - the launch of Day, Night and Auto Mode implementation in Zoho CRM's NextGen user interface! This feature is designed to provide a visually appealing and comfortable experience for

    • Does Thrive work with Zoho Billing (Subscriptions)?

      I would like to use Thrive with Zoho Billing Subscriptions but don't see a way to do so. Can someone point me in the right direction? Thank you

    • How to display two measures (sales and price) divided by categories on one line chart

      Hi everyone, I’m having trouble figuring out how to display two columns on a line chart with category breakdowns. What I need is a line chart where one line represents Sales and the other represents Price. However, the Price data is divided into around

    • Exporting Charts from ZohoCRM

      Hi...I'm relatively new to ZohoCRM, but very happy with it so far. I have all my leads and potentials accurately entered, and like the reports that I can view, with charts at the top of the data. But when I export the data, I'm receiving only the data, whether I export as excel, csv or pdf. How can I export both the chart and the data? In case it makes a difference, I'm using the free version right now. I tried researching the other editions to see if a paid version of the software offered the ability

    • Automatically Update Ticket Status in Zoho Desk Based on Actions in Zoho Projects

      Hi Zoho Desk Team, Hope you’re doing well. We’re using the Zoho Desk–Zoho Projects integration to manage tasks related to customer tickets, and it works well for linking and tracking progress. However, there are a few important automation capabilities

    • sms long credits

      I’m trying to purchase Long Code credits so I can send SMS campaigns to my contacts. However, when I click the “Buy Now” button, the page appears blank and doesn’t load any purchase options. Could you please assist me in purchasing the Long Code credits

    • Tip #48- Power Your AI Workflows with Zoho Assist on Zapier’s MCP- 'Insider Insights'

      We’re thrilled to announce that Zoho Assist is now part of Zapier’s Model Context Protocol (MCP), bringing remote support automation right into your AI ecosystem. What is MCP? The Model Context Protocol (MCP) is Zapier’s new framework designed to connect

    • Insert Cookie Policy in Zoho Sites

      Hello, i need to insert a banner on my site because i'm in Italy so i have to respect EU laws for Cookie Policy and Privacy Policy. I see that i need to insert a code in <head> section of my site to show a banner/popup with cookie info. How i can do this? Thank you Luca

    • Marketing Tip #1: Optimize item titles for SEO

      Your item title is the first thing both Google and shoppers notice. Instead of a generic “Leather Bag,” go for something detailed like “Handcrafted Leather Laptop Bag – Durable & Stylish.” This helps your items rank better in search results and instantly

    • Customer Parent Account or Sub-Customer Account

      Some of clients as they have 50 to 300 branches, they required separate account statement with outlet name and number; which means we have to open new account for each branch individually. However, the main issue is that, when they make a payment, they

    • Forced Logouts - Daily and More Frequent

      In the last month or so, I've been getting "power logged out" of all of my Zoho apps at least daily, sometimes more frequently. This happens in the same browser session on the same computer, and I need to re-login to each app separately after this happens.

    • Paste issues in ZOHO crm notes

      Hi, since a week or so I have issues with the paste function in ZOHO CRM. I use "notes" to copy paste texts from Outlook emails and since a week or so, the pasting doesnt function as it should: some text just disappears and it gives a lot of empty lines/enters.....

    • ENTER key triggering Submit

      Is it possible to stopped the ENTER key from the mandatory triggering of the Submit button on Creator form? I want forms submitted "ONLY" when the Submit button is pressed. 

    • Is it possible to assign Client user to external task ON PROJECTS' TEMPLATES?

      Is it possible to assign Client user to external task ON TEMPLATES PROJECTS?

    • Draft & Schedule Emails Directly in Bigin

      Greetings, I hope all of you are doing well. We're happy to announce a few recent enhancements we've made to email in Bigin. We'll go over each one in detail, but here's a quick overview: Previously, you couldn't draft or schedule emails in Bigin, but

    • Zoho CRM Workflow and Function Backup Options

      Hi everyone! I have been able to make several backups of my CRM data and noticed that the Workflows and Functions are not included in these backups. To my knowledge, there is no backup feature for workflows and functions, which is problematic in of itself.

    • Cliq does not sync messages after Sleep on Mac

      I'm using the mac app of Cliq. When I open my mac after it was in sleep mode, Cliq does not sync the messages that I received. I always have to reload using cmd + R, which is not what I want when using a chat application.

    • Link to images

      I have added images in pages. I would like to link those images with linked in URL so that they open in new window. There is an option of image -> link but I am not able to use the same to open URL in new window. Please check the attached image. Can you

    • Canvas View - Print

      What is the best way to accomplish a print to PDF of the canvas view?

    • What's New in Zoho Analytics - October 2025

      Hello Users! We're are back with a fresh set of updates and enhancements to make data analysis faster and more insightful. Take a quick look at what’s new and see how these updates can power up your reports and dashboards. Explore What's New! Extreme

    • Respond faster and smarter with Zia in your IM Inbox

      You’re in the middle of a busy chat queue. New messages keep popping up. One customer sounds upset. Another is asking a long list of questions. You need context. You need speed. You need help. That’s exactly when Zia Insghts jumps into action. It shows

    • Meeting impossible to use when sharing screen

      he Meeting tool in Brazil is practically unusable when sharing anything, whether it’s a presentation or simple navigation. When accessed via Cliq, the situation gets even worse: even basic calls fail to work properly, constantly freezing. And as you are

    • Let's Talk Recruit: Meet Zia, your all-in-one AI assistant (Part-2)

      Welcome back to Let’s Talk Recruit series. In Part 1, we introduced Zia and how AI is reshaping the way recruiters work. This time, we’re taking a closer look at how far Zia has come and how each update continues to simplify your everyday tasks. When

    • WARNING : Smart list automatically unsubscribes your contacts.

      I created a smart list of my team members based on the criterion that their email domain was @_____. The list refreshed as expected — but it ended up unsubscribing almost all members of my team. I contacted support, but it took two months to get a reply,

    • Export Purchase orders as Excel

      Is it possible to export purchase orders as excel rather than PDF? Our suppliers don't want orders made in PDF, they need it to be excel

    • 5名限定 課題解決型ワークショップイベント Zoho ワークアウト開催のお知らせ (10/31)

      ユーザーの皆さま、こんにちは。Zoho ユーザーコミュニティチームの中野です。 10月開催のZoho ワークアウトについてお知らせします。 今回はZoomにて、オンライン開催します。 参加登録はこちら（無料）：https://us02web.zoom.us/meeting/register/BGYTysOnSqa9LA9eY2IKww ━━━━━━━━━━━━━━━━━━━━━━━━ Zoho ワークアウトとは？ Zoho ユーザー同士で交流しながら、サービスに関する疑問や不明点の解消を目的とした

    • Subforms in Stateless Forms

      Stateless Forms are very useful, but we can't build complex data entry pages without Subforms. Can this be implemented? This would make Stateless Forms truly powerful.

    • Custom Function not getting package details when triggered from Workflow Rules.

      I have a custom function for Packages that submits a form in our Creator app that we use to generate custom shipping labels (internal staff complete deliveries so we cannot generate shipping labels straight from Inventory). When the function is executed

    • Billing Preferences per Account

      Hello, We are trying to setup Billing Preferences in Zoho Desk to set up a different pricing per account. We charge different pricing per hour per customer/account. Account A = 100 per hour Account B = 125 per hour In the Billing Preferences in Time Entry

    • Time entry preview for custom time entry templates.

      Our company needed time entries in a specific format to document our client interactions. Since we are using a custom time entry layout, we have lost the "preview" on the time entry tab. Using the default time entry layout, you get a small preview of

    • Set to Review for all

      We are testing the use of Writer as part of an internal review process for statement of work documents and have found that when the document is changed from Compose to Review by one person, that is not reflected for all others who view the document. Is

    • Implementing a Self-Invoicing/Vendor Portal in Zoho Creator

      Hello Zoho Community / Creator Experts, We would like to build a Self-Invoicing Portal (Vendor Portal) in Zoho Creator for our external contractors. Our goal is to allow approved contractors to log in, submit their invoice details (hours worked, project

    • Blocked Email

      We are a Zoho One subscriber and use Yahoo as our MX provider. A few times each year, for the past four years, CRM blocks one or more of my Zoho One users from receiving internal email from CRM. This includes "@mentions" in all modules, and emails from

    • Next Page