Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Zoho Bookings No Sync with Outlook

      Zoho Bookings appointments are showing on my Outlook Calendar but Outlook events are not showing on Zoho Bookings. How do I fix this?

    • End Date in Zoho Bookings

      When I give my appointments a 30 minutes time I would expect the software not to even show the End Time.  But it actually makes the user pick an End Time.  Did I just miss a setting?  

    • Custom confirmation message

      How can I change the message that users see after they submit the booking form? I have to confirm some details before their appointment is officially "confirmed", so I want to change it where it doesn't say their appointment is "confirmed" but rather

    • Issue showing too many consultations in my workspace link.

      Hi Team, I’ve set up two Workspaces to track meetings from different sources. So far, this has been working well, and the two Workspaces are differentiated without any issues. However, when I navigate to Consultations and share the link to my personal

    • Deluge sendmail in Zoho Desk schedule can't send email from a verified email address

      I am trying to add a scheduled action with ZDesk using a Deluge function that sends a weekly email to specific ticket client contacts I've already verified the email address for use in ZDesk, but sendmail won't allow it in its "from:" clause. I've attached

    • Zoho Learn & Zoho Connect

      Hi, Is there a way to sync the knowledge base we have in Zoho Learn with the manuals section is Zoho Connect? Thanks,

    • Addin Support in Zoho Sheet

      Is there any addin support available in zoho sheet as like google marketplace to enhance productivity by connecting with other apps, providing AI data analysis, streamlining business processes, and more?

    • Changing Corporate Structure - How Best to Adapt Current and Future Zoho Instances

      My current company is Company A LLC with a dba ("doing business as" - essentially an alias) Product Name B. Basically, Company A is the legal entity and Product Name B is what customers see, but it's all one business right now. We currently have a Zoho

    • how to add subform over sigma in the CRM

      my new module don't have any subform available any way to add this from sigma or from the crm

    • How to Install Zoho Workdrive Desktop Sync for Ubuntu?

      Hi. I am newbie to Linux / Ubuntu. I downloaded a tar.gz file from Workdrive for installing the Workdrive Desktop Sync tool. Can someone give me step by step guide on how to install this on Ubuntu? I am using Ubuntu 19.04. Regards Senthil

    • How to upload own video?

      How can you upload your own video on your zoho website? I do not want to use another host, but i want to insert my own files. how can i do this?

    • Support new line in CRM Multiline text field display in Zoho Deluge

      Hi brainstrust, We have a Zoho CRM field which is a Muti Line (Small) field. It has data in it that has a carriage return after each line: When I pull that data in via Deluge, it displays as: I'm hoping a way I can change it from: Freehand : ENABLED Chenille

    • Announcing new features in Trident for Windows (v.1.32.5.0)

      Hello Community! Trident for Windows just got better! This update includes new features designed to improve and simplify email and calendar management—and it includes a feature you’ve been waiting for. Let’s dive into what’s new! Save emails in EML or

    • How to render either thumbnail_url or preview_url or preview_data_url

      I get 401 Unauthorised when using these urls in the <img> tag src attribute. Guide me on how to use them!

    • Zoho CRM Calendar | Custom Buttons

      I'm working with my sales team to make our scheduling process easier for our team. We primary rely on Zoho CRM calendar to organize our events for our sales team. I was wondering if there is a way to add custom button in the Calendar view on events/meeting

    • Create Lead Button in Zoho CRM Dashboard

      Right now to create Leads in the CRM our team is going into the Lead module, selecting the "Create Lead" button, then building out the lead. Is there anyway to add the "Create Lead" button or some sort of short cut to the Zoho CRM Dashboard to cut out

    • Searching customer field

      Hello, When entering a receipt, we select customer information. The customer information is synced with Zoho CRM. However, we can't find the customer information because it searches for words that begin with the entered value. It needs to search for words

    • Introducing Version-3 APIs - Explore New APIs & Enhancements

      Happy to announce the release of Version 3 (V3) APIs with an easy to use interface, new APIs, and more examples to help you understand and access the APIs better. V3 APIs can be accessed through our new link, where you can explore our complete documentation,

    • Zoho CRM Tracking Google Enhanced Conversions

      Can anyone @Zoho, consultants, or users help me understand if Zoho CRM is going to support Google's Enhanced Conversions? I included some information from Google below about it. We use Google Adwords for our pay per click advertising for lead generation,

    • zoho click, and nord VPN

      Unfortunately, we've been having problems with Zoho Click, where essentially the line cuts off after about a minute's worth of conversation every time we are on VPN. Is there a way we can change this within the settings so it does not cut the line off

    • Connecting Portals from different Zoho apps

      Hi, I note that Zoho has functionality for customer portals for several of the Zoho apps, like CRM, Projects, Desk etc. Is there any way to connect these portals?  It would be great if we could give our customers access to a portal in which they could

    • Billing Management: #5 Usage Billing

      After understanding the nuances of Advance Billing and Retainers, we will explore one of the booming billing models. Long ago, villagers drew water from a shared well in a small village. The well was a lifeline for the entire community. Ravi, the well

    • Function #10: Update item prices automatically based on the last transaction created

      In businesses, item prices are not always fixed and can fluctuate due to various factors. If you find yourself manually adjusting the item rates every time they change, we have the ideal time-saving solution for you. In today's post, we bring you custom

    • Inventory Adjustments

      Hi, How to transfer the material from one head to another ? Like materials purchased for manufacturing the laptop need to transfer from consumption inventory (Quantity of raw materials reduced) to destination inventory ( Quantity of Laptop increased)

    • Zoho CRM Community Digest - Aug 2025 | Part 1

      Hey everyone! The first half of August went by, and we have a few announcements and some good noteworthy discussions. So, let's take a look at them! Product Updates: Introducing Connected Records feature: Zoho CRM’s Next-Gen UI now includes Connected

    • Please make it easier to Pause syncing

      right now it takes 3 clicks to get there. sounds silly, but can you make it just 2 clicks to get it done instead? thats how dropbox does it, 2 clicks to pause instead of 3.

    • How to create a Zoho CRM report with 2 child modules

      Hi all, Is it possible to create a Zoho CRM report or chart with 2 child modules? After I add the first child module, the + button only adds another parent module. It won't let me add multiple child modules at once. We don't have Zoho Analytics and would

    • SalesIQとPageSenseの利用について

      初めての投稿で場違いだったらすいません。 弊社ではSalesIQを運用しているのですが、追加でPageSenseの導入もしたいと現場からの声があります。 両サービスともクッキー同意バナーが必要なサービスなのですが 弊社では同意無しに情報はとりませんという方針なので 2つ入れると2つバナーを出す必要がでてきます・・・ 両サービスを運用されてる方があれば運用状況とか教えてほしいです。 PageSenseについては詳細まで機能を理解してないなかでの質問です。

    • How to integrate Zoho Forms with Zoho CRM on Standard Plan

      Hello Zoho Support Team, I am using the Standard Zoho Forms plan (USD 30/user) and I would like to integrate Zoho Forms with Zoho CRM so that certain fields in my forms can be automatically prefilled using data from Deals in CRM. Specifically, I want

    • Export PDF File Name

      Is it possible to change the default Zoho .pdf naming scheme for inventory items like quotations? Would like to use the the Subject as the default quote name. Is this possible?

    • How to change the from address from 'no reply' for an email template in CRM

      Hi, We have our CRM set up with the from field as sales@XXX. I have just created a series of email templates and sent a test and they are sending from noreply@zoho I have tried searching for how to change the email template but don't have the options

    • Zoho CRM Client Script - SetCriteria in lookup Field

      Hello All One More Zoho CRM Client Script Tips & Trick. Now you can Set the Criteria on Your lookup in zoho CRM, It Comes With a Create Page, Edit Page, and Details Page (Standard). Example:- We have a Room Module that includes Room Name, Status, Campus,

    • Kaizen #71 - Client Script ZDKs for Detail (Canvas) Page

      Hello everyone! Welcome back to another interesting Kaizen post. In this post, we can discuss Client Script ZDKs support for Detail (Canvas) Page. What is Detail (Canvas) Page? A Detail(Canvas) Page allows you to customize the record detail page to your

    • Zoho Reports Duplicating Entries

      I have a custom costing tab with a table where we entre invoices. These are under a Heading (PO Subject) and notes added in the form with different line items. In the reports, I have organised the report to group per PO Subject, with the total of the

    • Validation Rule Not Working for Mandatory Field in Zoho Blueprint

      As a Zoho user, we created a validation rule for a specific field. However, we noticed that when we made the same field mandatory within a Blueprint, the validation rule we defined did not work. When we reported this issue to Zoho Support, they stated

    • Notes Issues

      Been having issues with Notes in the CRM. Yesterday it wasn't showing the notes, but it got resolved after a few minutes., Now I have been having a hard time saving notes the whole day. Notes can't be saved by the save button. it's grayed out or not grayed

    • Export from Contacts module to Products module in Zoho CRM

      Good afternoon, I would like to send a number of contact info from the Contacts module into the customized module (tickets to an event) in one operation. I have selected several contacts in the Contact module (people who I have labelled as people I want

    • Can’t receive emailI c

      I have generated a basic for but when I submit it I don’t get a email, I’ve been in the settings and tested me email, all appears correct, can you please help me

    • Data Capture for Historical Activity (Especially One Lead Downloading Variois reports without Overwriting the info)

      Is there a better way in Zoho CRM to capture and archive a lead’s historical activity—specifically whenever they download reports—so that the data is stored without being overwritten?”

    • Client Script - Updating Field Value in Detail Page of a Lead

      Hello, I'm trying to use Client Script To enrich some data of the Lead when one of my User fill the "City" field in the detail page of the Lead. This is my Script: log (value); var response = ZDK.Apps.CRM.Functions.execute("getInfoCitta", { "nomeCitta":

    • Next Page