Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team




      • Sticky Posts

      • Register for Zoho Desk Beta Community

        With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

      • Share your Zoho Desk story with us!

        Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

      • Tip #1: Learn to pick the right channels

        Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

      • Welcome to Zoho Desk Community - Say hello here!

        Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

      • Webinar 1: Blueprint for Customer Service

        With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

        • Recent Topics

        • Link to images

          I have added images in pages. I would like to link those images with linked in URL so that they open in new window. There is an option of image -> link but I am not able to use the same to open URL in new window. Please check the attached image. Can you

        • Canvas View - Print

          What is the best way to accomplish a print to PDF of the canvas view?

        • What's New in Zoho Analytics - October 2025

          Hello Users! We're are back with a fresh set of updates and enhancements to make data analysis faster and more insightful. Take a quick look at what’s new and see how these updates can power up your reports and dashboards. Explore What's New! Extreme

        • Respond faster and smarter with Zia in your IM Inbox

          You’re in the middle of a busy chat queue. New messages keep popping up. One customer sounds upset. Another is asking a long list of questions. You need context. You need speed. You need help. That’s exactly when Zia Insghts jumps into action. It shows

        • Meeting impossible to use when sharing screen

          he Meeting tool in Brazil is practically unusable when sharing anything, whether it’s a presentation or simple navigation. When accessed via Cliq, the situation gets even worse: even basic calls fail to work properly, constantly freezing. And as you are

        • Let's Talk Recruit: Meet Zia, your all-in-one AI assistant (Part-2)

          Welcome back to Let’s Talk Recruit series. In Part 1, we introduced Zia and how AI is reshaping the way recruiters work. This time, we’re taking a closer look at how far Zia has come and how each update continues to simplify your everyday tasks. When

        • WARNING : Smart list automatically unsubscribes your contacts.

          I created a smart list of my team members based on the criterion that their email domain was @_____. The list refreshed as expected — but it ended up unsubscribing almost all members of my team. I contacted support, but it took two months to get a reply,

        • Export Purchase orders as Excel

          Is it possible to export purchase orders as excel rather than PDF? Our suppliers don't want orders made in PDF, they need it to be excel

        • 5名限定 課題解決型ワークショップイベント Zoho ワークアウト開催のお知らせ (10/31)

          ユーザーの皆さま、こんにちは。Zoho ユーザーコミュニティチームの中野です。 10月開催のZoho ワークアウトについてお知らせします。 今回はZoomにて、オンライン開催します。 参加登録はこちら（無料）：https://us02web.zoom.us/meeting/register/BGYTysOnSqa9LA9eY2IKww ━━━━━━━━━━━━━━━━━━━━━━━━ Zoho ワークアウトとは？ Zoho ユーザー同士で交流しながら、サービスに関する疑問や不明点の解消を目的とした

        • Subforms in Stateless Forms

          Stateless Forms are very useful, but we can't build complex data entry pages without Subforms. Can this be implemented? This would make Stateless Forms truly powerful.

        • Custom Function not getting package details when triggered from Workflow Rules.

          I have a custom function for Packages that submits a form in our Creator app that we use to generate custom shipping labels (internal staff complete deliveries so we cannot generate shipping labels straight from Inventory). When the function is executed

        • Billing Preferences per Account

          Hello, We are trying to setup Billing Preferences in Zoho Desk to set up a different pricing per account. We charge different pricing per hour per customer/account. Account A = 100 per hour Account B = 125 per hour In the Billing Preferences in Time Entry

        • Time entry preview for custom time entry templates.

          Our company needed time entries in a specific format to document our client interactions. Since we are using a custom time entry layout, we have lost the "preview" on the time entry tab. Using the default time entry layout, you get a small preview of

        • Implementing a Self-Invoicing/Vendor Portal in Zoho Creator

          Hello Zoho Community / Creator Experts, We would like to build a Self-Invoicing Portal (Vendor Portal) in Zoho Creator for our external contractors. Our goal is to allow approved contractors to log in, submit their invoice details (hours worked, project

        • Blocked Email

          We are a Zoho One subscriber and use Yahoo as our MX provider. A few times each year, for the past four years, CRM blocks one or more of my Zoho One users from receiving internal email from CRM. This includes "@mentions" in all modules, and emails from

        • Able to change project on timelog entries

          Ability to move the timesheet entry from one project to another. When a user adds a wrong entry a manager can change/update the timesheet entry to the correct project.

        • i cannot use <b></b> to bold the message in Creator C6!!!!???What?

          Dear experts and friends, Now Creator 6 blocking us from using <b></b> Anyone facing this issue? I faced it on Creator C6 Previously, it used to work. Now failed to work. Faint~ Seek guidance from everyone on how to bold the message. The super simple

        • Knowledge base: The nitty-gritty of SEO tags

          A well-optimized knowledge base with great SEO can benefit your company by allowing customers to find help articles and support resources using search engines. This enables customers to quickly and efficiently find the information they need without direct

        • Introducing swipe actions in the Zoho CRM Android app

          Hello everyone, How much time do you spend each day on your list view? Probably quite a lot. Since it's such an important part of your work, wouldn't it be easier if you could take action directly from your list view instead of opening each record one

        • Content Security Policy

          Is there a place in ZOHO CRM to add a Content Security Policy to allow for a call to a google.com map, from inside our current app? Or, how do I resolve the issue below?? jquery.js:1 Refused to load the script 'https://maps.google.com/maps/api/js?v=3.41&libraries=places&sensor=true&key=AIzaSyAyQzKeKSbLci4LwZhn9oXvtCkbUo1Ae4g&callback=map_loader'

        • Option to select location?

          As a business coach, I meet with clients at various public locations. I have two or three pre-determined locations that I meet at. I would like the client to choose the location when booking an appointment. Is there a way to do that with a single service, or is the best way to accomplish this by creating one service for each location offered?

        • Unified WhatsApp Number Management in Zoho Desk and SalesIQ

          Dear Zoho Desk Support Team, We are currently utilizing both Zoho Desk and Zoho SalesIQ for our customer support operations. While both platforms offer WhatsApp integration, we are facing challenges due to the requirement of separate WhatsApp numbers

        • Provide a standard structure to your content using article templates

          Hello everyone, When multiple writers work on different documents, maintaining a standard structure can be challenging as each of the writer follows a different writing style. However, when the structure, tone, and format of every document is different,

        • How to update custom multi-user field in Zoho Projects?

          I'm trying to update custom multi-user fields in Zoho Projects via a Deluge function in CRM. The code I have so far is below. It works for updating standard project fields and single-line custom fields, but it does not work to update multi-user fields.

        • Accessibility Spotlight Series - 1

          Every user interacts with products differently, what feels intuitive to one may be challenging for another. Addressing this, accessibility is built into Zoho Project's design philosophy. This helps users navigate and perform actions with ease irrespective

        • Projects Tasks Not Showing Dependencies

          I'm clicking on tasks and the popup to add dependencies isn't showing. I can't disconnect the nodes either. For some reason when I slide a task backwards it says it cannot go before a predecessor, even though there is not predecessor. Double clicking

        • Deprecation of C4 endpoint URLs

          Note: This post is only for users who are still using the C4 endpoints. Hello everyone, At Zoho Creator, we're committed to continuously enhancing the security, performance, and capabilities of our platform. As part of this ongoing effort, we'll be deprecating

        • Introducing AWS authentication for connections in Deluge

          Hello everyone, We're incredibly excited to announce the all-new AWS authentication for connections in Deluge! This highly anticipated feature simplifies connecting to Amazon Web Services, opening up a world of possibilities and allowing you to seamlessly

        • Timeline Tracking Support for records updates via module import and bulk write api

          Note: This update is currently available in Early Access and will soon be rolled out across all data centers (DCs) and for all editions of Zoho CRM. The update will be available to all users within your organization, regardless of their profiles or roles.

        • Customize User Invites with Invitation Templates

          Invitation Templates help streamline the invitation process by allowing users to create customized email formats instead of sending a one-size-fits-all email. Different invitation templates can be created for portal users and client users to align with

        • ListObjects is recognized by VBA

          Sub addNewRow() Dim ws As Worksheet ' Set your worksheet name Set ws = ThisWorkbook.Sheets("Invoice") ' Set your table name (change "Table1" to your actual table name) ws.ListObjects("InvItems").ListRows.Add End Sub I am getting Unknown function: Li

        • KPI Widget dashboard select periods

          I have a problem with selecting periods as a user filter. In the beste scenario I would like to have to have a period filter like Google Analytics has of Datastudio (see attachment). In the KPI widget I "Group by "inquiry_date" on week&Year". It selects

        • Need a way to secure Prefill URLs in Zoho Forms (hide or encrypt prefilled values)

          Hi everyone, I often use Zoho Forms with prefilled URLs to simplify the user experience — for example: https://forms.zohopublic.com/.../form?Name=David&Amount=300 However, the problem is that all prefilled values are visible and editable in the link.

        • Can’t send emails from Zoho CRM after adding a new user — verification codes not received

          Hi everyone, We recently added a new user to our Zoho CRM account and purchased an additional license. Since then, we haven’t been able to send any emails from Zoho CRM. Our Zoho Mail accounts are working perfectly, we can send and receive emails directly

        • CRM for email in Outlook: how to ignore addresses?

          We’re using the "Zoho CRM for email" add-in for Outlook. When opening an email, the add-in displays all email addresses from the message and allows me to add them to the CRM or shows if they’re already contacts. However, sometimes people listed in To

        • Scheduling Calls in CommandCenter / Blueprints

          I would love it if you could add a function to schedule a call in the lead's record for a future date. I know you can add a Task by going to Instant Actions > Task and completing the form: These tasks go into the lead's record under Open Actions. But

        • Quickly send emails and sync conversations with custom email addresses in CRM

          Editions: All editions DCs: All DCs Release plan: This enhancement has been released for customers in all DCs except IN and US. We will be enabling it for IN and US DC customers soon. [Update on 22 May 2024] This enhancement has been released for all

        • Zoho製品と生成AIツールの活用について

          いつもありがとうございます。 弊社では、Zoho Oneを契約し、CRMを軸として、見込み客の管理から商談、その後の受注や請求の管理、サポート業務（Desk）、業務データのレポーティング（Analytics）などを行っております。 Zohoサービス自体には、Ziaというツールが搭載されているかと存じますが、それ以外の外部の生成AIツールと連携した活用などもできるのでしょうか？具体的には、CopilotなどがZohoに登録されているデータや情報を見て、対話型で必要なデータを提示してくれたり、商談や蓄積されたメモなどを分析してユーザが知見を得られるような活用ができないか、と考えております。

        • Work with Contacts who use multiple Emails / Manage obsolete Email addresses without loosing Emails in Context

          Hello List Work with Contacts who use multiple Emails Only after 1 week in using Zoho CRM productively we have contacts which randomly use 2 different email addresses. From the concept I've understood that Zoho CRM offers two email fields which are recognized

        • New Field in CRM Product Module Not Visible in Zoho Creator for Mapping

          I created a new single-line field in the Products module in Zoho CRM. Zoho CRM and Zoho Creator are integrated, but the newly created field in CRM is not visible in Zoho Creator when I try to map fields.

        • Next Page