Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • can I link a contacts to multiple accounts

      can I link a contacts to multiple accounts

    • Free webinar! Digitize recruitment and onboarding with Zoho Sign and Zoho Recruit

      Hello, Tired of being buried in onboarding paperwork? With the integration between Zoho Sign and Zoho Recruit, a powerful applicant tracking system, you can digitize and streamline the entire recruitment and onboarding process, all from one platform.

    • Open Activities view.

      I really like the new views for the open and closed activities inside the deals. But when you are in the tab view instead of the column view you can only complete and edit the open activity there isn't the 3 dot option to be able to delete the activ

    • Potentially Outdated and Vulnerable Chromium Engine Installed by Ulaa Browser Installer

      I just installed Ulaa Browser a few minutes ago. Whats My Browser page shows I am using an outdated Chromium engine meaning I might be vulnerable for security exploits that might have got fixed in the new version.

    • Potentially hardcoded list of Browsers to import from (after Ulaa Setup)

      I have just installed Ulaa Browser and found that the list of browser to import data is potentially hardcoded ones rather than looking at the system. I do not have FF, IE and Edge is not my default itself. I would appreciated if Ulaa detected my browsers

    • From Layout to Code: Finding Custom Field IDs in Zoho Projects.

      Hello everyone! Ever found yourself wondering how to get the API names and IDs of custom fields in Zoho Projects while working on custom functions? Here’s a simple and effective way to do it! This method makes it super easy to locate the right field details

    • Employee type and source translation

      In Zoho People, when I fill in the employee’s information, there is the option to determine the type of employment (employee type) and the hiring source. Both options ALWAYS appear in English. It is extremely inconvenient to deal with poorly translated

    • Sync Issue Between Zoho Notebook Web App on Firefox (PC) and Android App

      Hi Zoho Notebook Community, I'm facing a sync problem with Zoho Notebook. When I use the web version on Mozilla Firefox browser on my PC, I create and save new notes, and I've synced them successfully. However, these new notes aren't showing up in my

    • Request for Clarity on Timeline for True GPT/Zia Auto-Response Capabilities

      I appreciate Zoho’s steady innovation, but I’m concerned that Desk and Zia remain well behind modern AI capabilities. For years, GPT-based tools have been able to generate and send contextual responses, yet Zoho Desk only supports summarization or suggested

    • Notebook audio recordings disappearing

      I have recently been experiencing issues where some of my attached audio recordings are disappearing. I am referring specifically to ones made within a Note card in Notebook on mobile, made by pressing the "+" button and choosing "Record audio" (or similar),

    • Has anyone built a ticket export that allows Help Center users to export the tickets shown in the My Area list they are looking at?

      Hi, We are moving to Zoho Desk soon. Our current support system displays an option in our help center allowing customers to export their Open, Closed, or all tickets based on which list they are looking at. We need to offer the same in Zoho Desk help

    • Two factor authentication for helpdesk users

      The company i work for wants use the helpdesk site in Zoho desk, as a place for their distribution partners to ask question and look for information about our product. The things there is suppose to go up there is somewhat confidential between my company

    • Zoho Desk: Q2 2025 | What's New

      Hello everyone, We are excited to announce Zoho Desk's 2025 Autumn updates. This release brings new features and enhancements that improve work management and enable businesses to provide a better overall support experience. Spanning from Zia Agents to

    • Checkboxes not adhering to any policy in mail merge - data from CRM

      I want checkboxes to appear depending on whether the checkbox in the CRM module is ticked or not. However, the tickboxes that appear are either ticked or not, but don't correlate to the actual selections in the CRM module. This is is despite updating

    • Items Landed Cost and Profit?

      Hello, we recently went live with Zoho Inventory, and I have a question about the Landed Cost feature. The FAQ reads: "Tracking the landed cost helps determine the overall cost incurred in procuring the product. This, in turn, helps you to decide the

    • CC and/or BCC users in email templates

      I would like the ability to automatically assign a CC and BCC "User (company employee)" into email templates. Specifically, I would like to be able to add the "User who owns the client" as a CC automatically on any interview scheduled or candidate submitted

    • Create Contract API Endpoint Unclear "inputfields" Requirements

      Hello, I'm trying to create a Deluge function that accepts inputs from a form in Zoho Creator and creates a barebones contract of a given type. See below for the current code, cleaned of authentication information. // Fetch form data // Hidden field client_name

    • Kaizen #46 - Handling Notes through Zoho CRM API (Part 1/2)

      Hello everyone! Welcome back to another week of Kaizen! This week, we will discuss Handling Notes through Zoho CRM API. What will you learn from this post? Notes in Zoho CRM Working with Notes through Notes APIs 1. Notes in Zoho CRM 1a. Why add Notes to records? Notes are a great way to summarize your observations on customer and prospect interactions and outcomes. By saving notes as CRM data, a sales rep will always be able to keep track of how a sale is progressing. To know more about notes in

    • Marketer's Space - Why email marketing matters in ecommerce (and how to get started with Zoho Campaigns)

      Hello Marketers, Welcome to this week's Marketer's space post. Today, we'll discus why email marketing matters in ecommerce businesses. Running an online store is exciting but challenging. If you're running an online store, you've probably experienced

    • Zoho Campaigns Event timestamps do not propagate to Zoho CRM

      We have integrated Zoho CRM and Zoho Campaigns. But when looking at Contact records, the Campaign event data is missing the actual timestamps: especially when a particular email was sent. They're not in the Campaigns related list, and the cannot be found

    • Kaizen #121 : Customize List Views using Client Script

      Hello everyone! Welcome back to another interesting Kaizen post. In this post, we can discuss how to customize List Views using Client Script. This post will answer the questions Ability to remove public views by the super admin in the Zoho CRM and Is

    • Setting default From address when replying to request

      At the moment, if I want to reply to a request, the From field has three options, company@zohosupport.com, support@company.zohosupport.com, and support@company.com.  The first two are really internal address that should never be seen by the customer and

    • Tip #45 - Explore Your Support Reach with Zoho Assist’s Geo Insights - 'Insider Insights'

      Understanding where your remote support sessions are happening can help you make smarter decisions, allocate resources effectively, and improve overall customer satisfaction. In this week's Zoho Assist's community post we will be exploring Geo Insights

    • Formatting of text pasted into Zoho documents

      Howdy, I'm a newbie and finding Zoho an improvement to MS Word. Consider yourself hugged. High on my wish list would be plain text cut-and-paste. When pasting text from the web to Zoho, presently Zoho imports the formatting along with the text. This means that every cut-and-paste operation brings in text in a different font, size, or style. Can we have at least the option of importing plain text without formatting (or better yet, is this option already out there?) ... Thanks Helen

    • Add additional features to Zoho Tables

      Zoho Tables is a really great tool, why not add features like diagramming capability into the tool from applications like Draw.io which I believe is open source, you should be able to do wireframes, process flow diagrams, network design, etc. Please note

    • The Social Wall: August 2025

      Hello everyone, As summer ends, Zoho Social is gearing up for some exciting, bigger updates lined up for the months ahead. While those are in the works, we rolled out a few handy feature updates in August to keep your social media management running smoothly.

    • The Social Wall: July 2025

      Hello everyone! July has brought some exciting new updates to Zoho Social. From powerful enhancements in the Social Toolkit to new capabilities in the mobile app, we’ve packed this month with features designed to help you level up your social media presence.

    • Use Zoho Creator as a source for merge templates in Zoho Writer

      Hello all! We're excited to share that we've enhanced Zoho Creator's integration with Zoho Writer to make this combination even more powerful. You can now use Zoho Creator as a data source for mail merge templates in Zoho Writer. Making more data from

    • Tagged problem !!!

      Damn it, we're one of dozens of construction companies in Africa, but we can't link purchasing invoices to projects. Why isn't this feature available?

    • Syntax for URLs in HTML Snippets

      What are some best practices for inserting a URL in an HTML snippet? I've looked at Zoho Help articles on navigation-based and functional-based URLs, but I'm still unclear on how to incorporate them in an HTML snippet. For example, 1. How do I link to

    • The Social Wall: June 2025

      Hello everyone, We’re back with June Zoho Social highlights. This month brought some exciting feature updates—especially within the Social Toolkit—to enhance your social media presence. We engaged with several MSME companies through community meet-ups

    • Make panel configuration interface wider

      Hi there, The same way you changed the custom function editor's interface wider, it would be nice to be able to edit panels in pages using the full width of the screen rather than the currently max-width: 1368px. Is there a reason for having the configuration panel not taking the full width? Its impossible at this width to edit panels that have a lot of elements. Please change it to 100% so we can better edit the layouts. Thanks! B.

    • Tip 7: How to fetch data from another application?

      Hi everyone, Following our Zoho Creator - Tips and Tricks series every fortnight, we are back today with a tip based on one of the most popular questions asked in our forum. This tip would help you fetch data from another application(App B) and use it

    • The Social Wall: May 2025

      Hey everyone, We're excited to share some powerful updates for the month of May! Let's take a look! Reply to your Instagram and Facebook comments privately through direct messages Are you tired of cluttered comment threads or exposing customer queries

    • Sub-Form Fields as Filters for Reports

      Hi, I would like to use the Sub-Form Fields as Filters in Reports just like we do for Main Page Fields. Thanks Dan

    • Zoho CRM Formula - Current Time minus Date/Time field

      Hello, I am trying to prevent duplicate emails going to clients when more than 1 deal is being updated. To do this, I would like to create a formula to identify if a date/time field is >= 2 hours ago. Can someone please help me write this formula? Example:

    • Billing Management: #7 Usage Billing in Telecom & Internet Service Provider

      Telecom and Internet Service Providers operate in markets where usage varies drastically from one customer to another. While flexible, usage-based models align revenue directly with consumption, they also introduce operational challenges like real-time

    • Zoho Sprints - Q3 updates for 2025

      The updates for the third quarter of 2025 are out. A few significant features and enhancements have been rolled out to improve user experience and product capabilities. The following are the updates: Manage tags and cluster tags Record and maintain project

    • Kaizen #208 - Answering your Questions | Functions, AI and Extensions

      Hello Developers! Welcome back to a fresh week of Kaizen! We are grateful for your active participation in sharing feedback and queries for our 200th milestone. This week, we will answer the queries related to Functions and Extensions in Zoho CRM. 1.

    • Zoho CRM still doesn't let you manage timezones (yearly reminder)

      This is something I have asked repeatedly. I'll ask once again. Suppose that you work in France. Next month you have a trip to Guatemala. You call a contact there, close a meeting, record that meeting in CRM. On the phone, your contact said: "meet me

    • Next Page