Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Zoho CAMPAIGNS working hours

      Hi I use Campaigns Automation workflows to automate follow-ups to my Leads. I discovered this weekend that emails are being sent out on Sundays. How do I limit my Campaigns outgoing emails to business working hours? This is very important! Thanks, D

    • Depositing funds to account

      Hello, I have been using Quickbooks for many years but am considering moving to Zoho Books so I am currently running through various workflows and am working on the Invoicing aspect. In QB, the process is to create an invoice, receive payment and then

    • Create Receipt of a Donation (not a sale)

      We are a non-profit organization that receives general donations. How do I create a receipt of payment for the donor and categorize the payment as a Gift? I tried the method of creating an invoice; however that automatically created a "Sales" transaction

    • Function #2: Create a Deal in Zoho CRM when an Estimate is created in Zoho Books

      For those who use Zoho CRM integrated with Zoho Books, here's a nifty function that helps you optimize your sales process by adding a Deal in Zoho CRM whenever an estimate is created in Zoho Books. Custom Function: To create the custom function, go to

    • New Custom View -Sorting the Custom fields

      While creating a New Custom View in invoice , Customers, Bills ,expense etc , the sorting of custom fields are not available , a query function "order by / sort by  " may also be included in  Define new criteria module  which will be more beneficial to

    • Grouping Undeposited Funds to Move to other accounts

      In the bank option it would be nice to check what transactions in undeposited funds I want to move to other accounts. Then while checking this it can accumulate totals and created whats essentially a deposit slip. Once the transaction is moved it should

    • Finding rhythm through poetry

      Poetry has long been a powerful form of expression, discovery, and reflection. For many, it is a way to pen down their thoughts and experiences. The "poetic license" allows writers to shape their words with rhythm and flow. This year, on World Poetry

    • Sync your CRM Tasks with Zoho Projects

      Zoho Projects integration with Zoho CRM helps you manage your tasks more efficiently. You can create all project related activities right inside your CRM using this integration. Create new portal or associate an existing portal, add projects to the portal,

    • Zoho CRM strips whitespace in text fields

      When editing field text with multiple spaces: CRM - both UI and API trim / compress the whitespace to a single space when saving: Is this known / expected / documented behaviour?

    • WorkDrive API returning empty response even after placing file in Team Folder

      Hi everyone, I am trying to fetch a file from Zoho WorkDrive using a Deluge standalone function in Zoho People. The API call executes successfully using a configured connection, but the response is coming back empty. I have verified the following: The

    • Custom Button makes scroll bar go down in report

      I have a report with a Custom button called Completed. A colleague mentionned to me that when he pressed this custom button it scrolled down the page which is annoying since he want to stay at the same space on the repoort. There is no reload linked to

    • Quickbooks Integrations Stopped Working

      All of our Quickbooks integrations have stopped working. I am checking in to see if: a) this is a known issue b) if anyone else is having this issue. As usual, Zoho support is unavailable.

    • sync two zoho crm

      Hello everyone. Is it possible to sync 2 zoho crm? what would be the easiest way? I am thinking of Flow. I have a Custom Module that I would like to share with my client. We both use zoho crm. Regards.

    • CRM x WorkDrive: We're rolling out the WorkDrive-powered file storage experience for existing users

      Release plan: Gradual rollout to customers without file storage add-ons, in this order: 1. Standalone CRM 2. CRM Plus and Zoho One DCs: All | Editions: All Available now for: - Standalone CRM accounts in Free and Standard editions without file storage

    • Incorrect Functioning of Time Logs API (Version 3)

      We need to fetch the list of time logs for each task for our company internal usage. We are trying to achieve it by using the next endpoint: https://projects.zoho.com/api-docs#bulk-time-logs#get-all-project-time-logs Firstly, in the documentation the

    • Zoho CRM Queries Now Support Databases and Cloud Data Sources

      Hello everyone! We're thrilled to announce a major enhancement to the Queries feature in Zoho CRM! Queries now support a broader range of external data sources, allowing you to fetch live data and combine it with CRM records, all using a unified query

    • Salesforceに添付ファイルを格納したい

      お世話になっております。 Salesforceに添付ファイルを格納したく、カスタムオブジェクトに連携し、 「ファイルのアップロード」項目を設けました。 実際、エラーもなく送信出来たのですが、実際生成されたカスタムオブジェクトのレコードを見ると、どこにも添付ファイルがありません。仕様として、この添付ファイルはSalesforceのどこに格納されるのでしょうか？ 今回作りたいフォームは、複数の書類を添付するため、Zohoformのファイルアップロード項目「本人確認書類」「源泉徴収票」などの項目を、Salesforce側にも設けた「本人確認書類」「源泉徴収票」という各項目にURLリンクとして紐づけたいと思っておりました。

    • Dynamic image in form works in the app but not on the customer portal.

      img = frm_Fichas[ID == input.Nombre].Foto; imgno = Nophoto[ID2 = 1].Image; if(len(img) > 1) { img = img.replaceAll("/sharedBy/appLinkName/",zoho.appuri); img = img.replaceAll("viewLinkName","Fichas_de_personal_public"); img = img.replaceAll("fieldName","Foto");

    • Is it possible to retrieve function (Deluge) code from Zoho CRM externally?

      Hi Everyone, Is it possible to fetch or retrieve the Deluge function code from Zoho CRM using an external method (API or any other approach)? I would like to know if there is any way to access or extract the function script outside of Zoho CRM, or if

    • Uplifted homepage experience

      Editions: All editions. Availability update: 17th February 2026: All editions in the CA and SA DC | JP DC (Free, Standard and Professional editions) 23 February 2026: JP (All Editions) | AU, CN (Free, Standard, Professional editions) 27 February 2026:

    • Darshan Hiranandani : How many participants can join a Zoho Meeting at once?

      Hi everyone, I'm Darshan Hiranandani, trying to find out the maximum number of participants that can join a Zoho Meeting at once. Has anyone here used Zoho Meeting for larger groups and can share their experience or knowledge about the participant limit?

    • Where is the scheduled report in New UI?

      Hi Team, Seems there is not such a field in New UI, hence I have to switch to old UI to handle request..

    • Approval Workflow for Purchase Orders Abrir

      The requirement is , that all purchase orders greater than or equal to 5000 go through an approval process from certain people, but within books I only see that the approvers can be by levels or any approver but we cannot enter a rule like these. Can

    • WeTravel + Zoho CRM Integration - Has Anyone Built a Connector or Extension?

      Hi all, I'm exploring options for integrating Zoho CRM with WeTravel (booking & payment platform for tour operators). Zapier seems to be the common method but seems limited. I'm wondering if anyone in the community has developed a more comprehensive solution,

    • Option in pipeline deal to select which hotel or branch or store if client has more than one local store

      Hi, I would like to know if there is an option in the deal pipeline to select which hotel, branch, or store a deal is related to—if the company has more than one location. For example, I have a client that owns several hotels under the same company, and

    • CRM x WorkDrive: File storage for new CRM signups is now powered by WorkDrive

      Availability Editions: All DCs: All Release plan: Released for new signups in all DCs. It will be enabled for existing users in a phased manner in the upcoming months. Help documentation: Documents in Zoho CRM Manage folders in Documents tab Manage files

    • Undo article like/dislike

      It seems to be not possible to undo your like/dislike for an article. Would be great if you can. Kind regards, Helen

    • Nested notebooks

      Dear Sir/Madam, I would like to know if it is possible to nest notebooks. It would be very helpful when there are too many, as it would improve organization. Thank you for your response. Best regards.

    • Tax in Quote

      Each row item in a quote has a tax value. At the total numbers at the bottom, there is also a Tax entry. If you select tax in both of the (line item, and the total), the tax doubles. My assumption is that the Tax total should be totalling the tax from

    • Issue with "Send Email" from Quotes not loading Email Template data

      Hi everyone, I'm currently experiencing an issue when using the "Send Email" option from a Quote record in Zoho CRM. What’s happening: When I go to the Quotes module and select a record, then click Send Email, the attached file (Quote) correctly pulls

    • Dynamically Fetching Lookup Field Display Value

      I have an audit trail form, Audit_Changes, that tracks old vs new values across different forms. For lookup fields, the old/new value is the ID, but I also need the display value. What's a best practice for dynamically fetching the display value of the

    • Stop Scrolling, Start Asking: Meet Zia for Your Files

      Hey everyone 👋 The era of 'scrolling and searching' is officially over. Whether it's a dense legal contract or a long meeting recording, searching for specific details is a massive time-sink. We think you should be able to interact with your files, not

    • Introducing a smarter, faster, and more flexible charting experience

      Hello Zoho Sheet users, We're delighted to share the latest news about a major update to charts in Zoho Sheet! The new version supports dynamic data ranges, granular styling options, faster loading, and other interesting enhancements that allow you to

    • How to create a new Batch and update Stock via Inventory?

      Hi everyone, We are building an automation where a user enters batch details (Batch Number, Mfg Date, Expiry, and Quantity) into a Custom Module. I need this to trigger an API call to Zoho Inventory to: Create the new batch for the item. Increase the

    • Can't change form's original name in URL

      Hi all, I have been duplicating + editing forms for jobs regarding the same department to maintain formatting + styling. The issue I've not run into is because I've duplicated it from an existing form, the URL doesn't seem to want to update with the new

    • How do I open MSG files in Microsoft Word?

      If you want to open MSG files in Microsoft Word is not natively supported, as MSG is an email file format created by Microsoft Outlook. However, there are professional approaches to access MSG content in Word. First, open the MSG file in Outlook and copy

    • Unable to charge GST on shipping/packing & Forwarding charges in INDIA

      Currently, tax rates only apply to items. It does not apply tax to any shipping or packing & forwarding charges that may be on the order as well. However, these charges are taxable under GST in India. Please add the ability to apply tax to these charges.

    • How to add packing & forwarding charge in purchase order & quotation???

      Hello Zoho Team I have just started using Zoho for my company and I wanted to make purchase order. My supplier charges fix 2% as packing & forwarding on Total amount of material and then they charge me tax. For example, Material 1 = 100 Rs Material 2

    • How to create a boxplot chart in Zoho Analytics?

      Hi, I'm looking forward to making a boxplot in Zoho Analytics, either with all my data or with a time segmentation. No documentation or YouTube video explaining that was found. I guess this is a feature gap. How feasible would it be to add this to Analytics?

    • New UI for Writer - Disappointed

      I've been enjoying Zoho Writer as a new user for about 6 months, and I really like it. One of my favorite things about it is the menu bar, which you can hide or leave out while still seeing most of your page because it is off to the left. I think this

    • Next Page