Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team




      • Sticky Posts

      • Register for Zoho Desk Beta Community

        With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

      • Share your Zoho Desk story with us!

        Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

      • Tip #1: Learn to pick the right channels

        Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

      • Welcome to Zoho Desk Community - Say hello here!

        Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

      • Webinar 1: Blueprint for Customer Service

        With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

        • Recent Topics

        • Marketer’s Space - Automate a Personalized Holiday Workflow with Zoho Campaigns

          Hello marketers, Welcome back to another post in Marketer’s Space! From Thanksgiving through Christmas, you have a flurry of opportunities to connect with your audience. In this post, we’ll see how you can plan an entire month-long automated workflow

        • Zoho Inventory Custom Field Update

          Hello All, In this post I am describing how can we Update the Custom Field Value in Zoho Inventory. // Get Org ID orgid = organization.get("organization_id"); // Field Value resvp = ifnull(item.get("purchase_rate"),null); // Record ID iid = item.get("item_id");

        • Sorting columns in Zoho Projects

          Hi, In project management best practice, sorting columns (ascending, descending) is an important tool. Sorting dates to see the order of tasks starting, sorting on priority or even on planned hours is a must for an efficient project control. Currently,

        • Zoho CRM - Calendar Cards View - Let Users Decide What Is Displayed On Calendar Entries

          Imagine planning your week of face-to-face meetings across three counties. You’re trying to group appointments by location to make the best use of your time, but Zoho CRM’s calendar doesn’t show where each meeting is happening. You’re left trying to remember

        • Explore the Redesigned Quotes Module in Zoho Billing

          Dear users, We’re excited to introduce a refreshed look for the Quotes details page in Zoho Billing! This update brings you a more efficient user interface experience without changing your existing workflow. We've enhanced layouts with organized tabs,

        • Updating custom fields in Zoho Projects

          Hi I am wondering if anyone has experience with custom fields in Zoho Projects. I am struggling to update the field using either deluge or the api endpoint. My code is: //custom_Map = map(); custom_Map = {"UDF_DOUBLE_1":"0.27"}; update_Map = map(); update_Map.put("custom_fields",custom_Map.toList());

        • Zoho Inventory - How to pay a supplier up front then receive multiple deliveries

          How do we manage situations where we pay a supplier up front, then the receive the products in increments? Example Workflow: Create Purchase Order > Receive Bill for full amount > Receive Items 2 or more deliveries. Currently, once a Bill is created against

        • Pin multiple columns and adjust column widths in CRM subforms

          Hello all, Subforms act as secondary forms or tables in which you can associate multiple line items to a primary record and thereby ensure more structured and comprehensive data organization. We've made some recent enhancements to subforms. Here's what's

        • How can I add or change the active customer in Zoho Projects?

          I'm trying to change the customer in an active Zoho project. Is it possible to add or change the customer?

        • Almacenamiento

          Hola, Quisiera saber como podría hacer para bajar el almacenamiento de 5gb a mis usuarios, en otras palabras los quiero ir limitando de la cuota real, y luego ir agregando poco a poco la cantidad hasta llegar a los 5gb que me dan en el plan free. 

        • What happens when someone clicks 'unsubscribe' in a zoho CRM email

          Hello, As per, i am going round in circles trying to find an answer to these 2 question. I have an email template that has an unsubscribe link in the footer in zoho CRM. First, what happens when someone clicks it - where does the contact get unsubscribed

        • Nimble enhancements to WhatsApp for Business integration in Zoho CRM: Enjoy context and clarity in business messaging

          Dear Customers, We hope you're well! WhatsApp for business is a renowned business messaging platform that takes your business closer to your customers; it gives your business the power of personalized outreach. Using the WhatsApp for Business integration

        • Zoho CRM - Scheduled Reports Which Contain Chart

          Hi Zoho CRM Team, I'm requesting that the Report Export and Scheduling feature be enhanced to include a chart, if one has been created on a report. At the moment I have a report which shows Sales This Week by Deal Owner and a pie chart at the top of the

        • Can I add Conditional merge tags on my Templates?

          Hi I was wondering if I can use Conditional Mail Merge tags inside my Email templates/Quotes etc within the CRM? In spanish and in our business we use gender and academic degree salutations , ie: Dr., Dra., Sr., Srta., so the beginning of an email / letter

        • Cadences

          I have just started using Cadences for follow-up up email pipeline. Is it just me or do you find the functionality very basic? For example, it will tell me (if I go looking for it) if someone has replied to a follow-up and been unenrolled; but it won't

        • Canvas View in Zoho Recruit

          Is it possible or would it be possible to have the new 'Canvas View' in Zoho Recruit?

        • Zoho Inventory - Move Orders

          Quick question about Move Orders... Why is there no status to say something like "Draft", "In Progress" and "Completed", similar to Transfer Orders? I'm assuming that when something needs to be moved it should be planned in Inventory, executed and then

        • Kaizen #215 - Workflow APIs - Part 3 - Create and Update Workflow Rules

          Welcome back to another week of Kaizen! Over the last couple of weeks, we’ve joined Zylker Cloud Services as they review and improve their workflows. In Part 1, we discovered and audited their sprawling workflow landscape. In Part 2, we learned how to

        • DataPrep Bigquery Connection failed

          Hello everybody, I want to create a connnection beetwen Bigquery and Dataprep but when I try to connect my project I got this error Loading tables has failed. Table list fetched from the data source expired.

        • Issue in Zoho People Regularization – Incorrect Hour Calculation

          I have noticed that when applying attendance regularization in Zoho People for previous dates, the total working hours are not calculated correctly. For example, even if the check-in is 10:00 AM and check-out is 6:00 PM, the system shows an incorrect

        • Free Webinar : Unlock AI driven business insights with Zoho Inventory + Zoho Analytics

          Are you tired of switching between apps and exporting data to build customized reports? Say hello to smarter & streamlined insights! Join us for this exclusive webinar where we explore the power of the Zoho Inventory–Zoho Analytics integration. Learn

        • Allow Multiple usage units to items while adding them to sales/purchase transactions

          The usage unit of items added in zoho books are static right now and can not be changed. But certain items are received or sold in multiple usage units. One example is fabric. It can be bought in Meters, inches, kgs or other units. Another example would

        • PO receive quantities

          At last, Zoho has finally got around to allowing us to receive a larger qty than recorded in the PO :-) Saves us all from editing the PO's before receiving larger quantities ( usual for us ) ! It's still in "beta" but available upon request, I've tested

        • Invalid URL error when embedded sending url into iframe for my website when using in another region

          Hi team, My site is currently working on integrating your signature feature as part of the system functionality, it's working great but recently there's been a problem like this: After successfully creating the document, i will embed a sending url into

        • Analytics : How to share to an external client ?

          We have a use case where a client wants a portal so that several of his users can view dashboards that we have created for them in Zoho Analytics. They are not part of our company or Zoho One account. The clients want the ability to have user specific,

        • Reference Deal Categories in Deluge

          Hello, Is there a way to reference Deal Category in deluge functions? So for our Deals, we have several different WON stages in a pipeline.  Rather than type each stage into our functions, we'd like to be able to reference the Deal Category.  Similar

        • Payroll In Canada

          Hi, When can we expect to have payroll in Canada with books 

        • Users Name & Email in Reports

          Hi, I would like to show the Users Name from their Zoho Acount in All Entries/Reports as well as the current Account Email. Thanks Dan

        • Cliq iOS can't see shared screen

          Hello, I had this morning a video call with a colleague. She is using Cliq Desktop MacOS and wanted to share her screen with me. I'm on iPad. I noticed, while she shared her screen, I could only see her video, but not the shared screen... Does Cliq iOS is able to display shared screen, or is it somewhere else to be found ? Regards

        • Presenting ABM for Zoho CRM: Expand and retain your customers with precision

          Picture this scenario: You're a growing SaaS company ready to launch a powerful business suite, and are looking to gain traction and momentum. But as a business with a tight budget, you know acquiring new customers is slow, expensive, and often delivers

        • Whatsapp Limitation Questions

          Good day, I would like to find out about the functionality or possibility of all the below points within the Zoho/WhatsApp integration. Will WhatsApp buttons ever be possible in the future? Will WhatsApp Re-directs to different users be possible based

        • Empowered Custom Views: Cross-Module Criteria Now Supported in Zoho CRM

          Hello everyone, We’re excited to introduce cross-module criteria support in custom views! Custom views provide personalized perspectives on your data and that you can save for future use. You can share these views with all users or specific individuals

        • Workflow Creation with Zia gets stuck

          It gets stuck here:

        • Super Admin Logging in as another User

          How can a Super Admin login as another user. For example, I have a sales rep that is having issues with their Accounts and I want to view their Zoho Account with out having to do a GTM and sharing screens. Moderation Update (8th Aug 2025): We are working

        • Unable to Add Asset to Work Order Due to Different Contact and Company

          Hello Zoho Team, I’m facing an issue in our environment related to asset linking in Work Orders. Here’s the situation: When our engineers go for field activities, they initially create the Work Order using the salesperson’s contact details. After completing

        • I NEED MORE CUSTOM FIELDS!!!

          Why can I only have 60 custom fields! It's not enough. I want another 100 extra custom fields to do what's necessary for my business! I'm sure I can't be the only one with this problem!!! How hard would it be to fix this? Just fix it for me please at

        • Zoho Webinar et Zapier : de la prise de rendez-vous aux relances — une automatisation performante

          Organiser un webinaire réussi et gérer efficacement les participants peut être un véritable défi. Avec Zoho Webinar et Zapier, vous pouvez rendre vos processus plus simples. Vous gagnez du temps en automatisant les tâches répétitives. Résultat : une expérience

        • Ticket Stages Report

          From data to decisions: A deep dive into ticketing system reports Ticket stages refer to the series of transitions from the moment a support ticket is created until it is closed. It moves through various stages based on the actions taken at each preceding

        • 【参加無料】今年最後のZoho ユーザー交流会｜東京・大阪・名古屋で開催！ 活用事例＆ユーザー同士の情報交換

          ユーザーの皆さま、こんにちは。コミュニティチームの中野です。 11月に東京、大阪、名古屋の3都市でZoho ユーザー交流会を開催します！ 毎回ご好評いただいている本イベントでは、実際の Zohoユーザーによるリアルな活用事例の共有や 参加者同士でノウハウを交換し合うグループワークを予定しています。 「他社の活用を参考にしたい」「Zoho をもっと使いこなしたい」方にぴったりの場です。 初参加の方もぜひお気軽にご参加ください！ ーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーー

        • Slow Performance on desk.zoho.com. 11/3/2025

          I’m not seeing any active service alerts for desk.zoho.com, but everyone on our account is currently experiencing very slow load times when opening or navigating tickets. We’ve already tried the standard troubleshooting steps — clearing cache and cookies,

        • Next Page