Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team




      • Sticky Posts

      • Register for Zoho Desk Beta Community

        With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

      • Share your Zoho Desk story with us!

        Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

      • Tip #1: Learn to pick the right channels

        Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

      • Welcome to Zoho Desk Community - Say hello here!

        Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

      • Webinar 1: Blueprint for Customer Service

        With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

        • Recent Topics

        • Zoho CRM Analytics - Allow To Reorder Dashboards

          I would like to suggest that you add the ability to reorder dashboards in the Analytics Module. I can see that this has been requested some time ago, the latest 9 years ago. I am not sure if this is a big or small endeavor, but such a small fix can go

        • Zoho Form URL displays incorrect name

          Hi, I have a form I created called "Design Request form". It displays this way everywhere I look. However, in the URL, it shows up as "DesignJobRequestFormFINAL011325PROOFV1B" and I'm not sure why. I can't find where to fix this. Does anyone have any

        • Consumers are talking about your business. Are you listening?👂

          A loyal customer might be praising your product in a forum. A frustrated user could be posting a harsh review on a public site. An excited partner may have left a comment on your campaign. A domain expert might be deconstructing your product. A prospect

        • What counts as a Temp for Billing Purposes in Workerly

          I'm considering trying this product but am not sure how the temp count is used for billing purposes. For example, if we keep a large data base of 500 potential workers.....are we billed for that or only if they are assigned to a client at a given point

        • Form name incorrectly displayed in URL

          Hi, I have a form I created called "Design Request form". It displays this way everywhere I look. However, in the URL, it shows up as "DesignJobRequestFormFINAL011325PROOFV1B" and I'm not sure why. I can't find where to fix this. Does anyone have any

        • I can't receive mail

          Hello, I can't receive e-mail. I no longer receive e-mails to the e-mail I received for ​my site. I also edited the DNS settings, but it doesn't work at all.

        • 1‑to‑1 invite missing post-setup (needs re-invite) vs channel invite auto-joins without business prompt

          1. Zoho Cliq 1‑to‑1 external invite The inviter sent a 1‑to‑1 invite to an invitee who didn’t have a Cliq account. After the invitee completed account setup and created a business/organization, the website redirected them to Cliq, where they opened Cliq

        • 【開催報告】東京 ユーザー交流会 Vol.3 2025/10/17 Zoho サービスの活用促進を外部ツールとの連携で実現！

          ユーザーの皆さま、こんにちは。コミュニティチームの藤澤です。 10月17日(金)に新橋で「東京 ユーザー交流会 Vol.3」を開催しました。ご参加くださったユーザーの皆さま、ありがとうございました！ この投稿では、当日のセッションの様子や使用した資料を紹介しています。残念ながら当日お越しいただけなかった方も、ぜひチェックしてみてください😊 ユーザー活用事例セッション：Zoho Flowと決済システムの連携 あみろくの岡島さんに、Zoho サービスの活用事例として、Zoho Flow を活用した外部サービスとの連携事例をご共有いただきました。

        • received email opens in a new tab every time I log in

          as per the title: since about when I first made my email account, every single time Ive logged in to view my inbox, a new tab opens for an email I viewed once as if restoring a closed session. I thought I just didnt understand the "starting up" settings

        • Engage with your customers at scale using WhatsApp Marketing Template messages

          Hi everyone, To make it easier for organizations to communicate with customers, Desk now allows you to send individual, mass, and bulk WhatsApp template messages from both the Ticket and Contact modules. How is this going to benefit your business? WhatsApp

        • Importation Tickets error

          Hi, I'm newbie here 🤓 So, i'm importing data from csv, but when I try advance to mapping fields the importer tool show this message: Previously I try import, other data, and not show errors in this step. Some ideas? Best Regards,

        • Showing description in timesheet and timelogs.

          I am wondering if it’s possible in version 5 of Zoho People to have the description show by default or with a manipulation on the user’s part. Let me show you what I mean. As you can see this is the view for the users. Now if they want to see the full

        • Workaround: openURL in Blueprints - An alternate approach

          There is a roundabout way to open a URL in blueprints after a save event. By using the 'onBeforeMandatoryFormSave' in Client Script, you can open an external URL. Now, the problem is, this is designed to be run BEFORE the blueprint is saved, not after,

        • MTD SA in the UK

          Hello ID 20106048857 The Inland Revenue have confirmed that this tax account is registered as Cash Basis In Settings>Profile I have set ‘Report Basis’ as “Cash" However, I see on Zoho on Settings>Taxes>Income Tax that the ‘Tax Basis’ is marked ‘Accrual'

        • Migrate file from Single File Upload to Multi File Upload

          Dears, I have created a new field Multi File Upload to replace the old Single File Upload field. I'd like to ask you guys what is the best way to migrate the files to the new field?

        • Open "Live Chat" from a hyperlink?

          Hi, I often write paragraphs and text on our company website, and usually say you can get in touch with us via live chat. Can the chat window be triggered to pop open without clicking the chat graphic in the bottom window, and use it in a hyperlink? ie:

        • Zoho Sites search box

          Is there a Search box that can be added to a Zoho site? It would be for searching within the site only.

        • What stops me from packaging and shipping an order when the inventory is negative?

          It seems if the inventory value is negative, that Zoho Inventory should not allow me to create a Package and Ship it.   But, there seems to be nothing to stop me from doing that other than when I go to physically package the item and realize that there is no stock. There also seems to be nothing on the screen that even indicates to me that I should not package and ship.  To me this is the fundamental point of an inventory system.  Am I doing something wrong?   

        • Conditional formatting based on another field

          Hi I have two fields on my form stage 1 complete and stage 1 deadline.  I am trying to setup conditional formatting so that if stage 1 complete is after stage 1 deadline the record is highlighted in red. I need both stage 1 complete and stage 1 deadline

        • Is there API Doc for Zoho Survey?

          Hi everyone, Is there API doc for Zoho Survey? Currently evaluating a solution - use case to automate survey administration especially for internal use. But after a brief search, I couldn't find API doc for this. So I thought I should ask here. Than

        • Using IMAP configuration for shared email inboxes

          Our customer service team utilizes shared email boxes to allow multiple people to view and handle incoming customer requests. For example, the customer sends an email to info@xxxx.com and multiple people can view it and handle the request. How can I configure

        • The sending IP (136.143.188.15) is listed on spamrl.com as a source of spam.

          Hi, it just two day when i am using zoho mail for my business domain, today i was sending email and found that message "The sending IP (136.143.188.15) is listed on https://spamrl.com as a source of spam" I hope to know how this will affect the delivery

        • Changing a Single-Line Text field into a Multi-line Field without losing data

          Is it possible to change a Single-Line Text field into a Multi-line Field without losing data. I have a module with data for which I would like to change a single-line field into a multi-line field but I'm worried it might delete the pre-existing da

        • Webhook - Google Sheets

          I have 2 forms that are both integrated with Google Sheets. I've set up a webhook to pull form 1 data from Google Sheets to prefill data in Form 2. The issue I have is that the forms name fields are First Name & Last Name but the Google sheets integration

        • Verified Mark Certificate

          Hello Dears, Can anyone help and check my mail or direct me to the desired person who can add the verification tag to my mail https://www.zoho.com/blog/mail/email-authentication-with-bimi.html

        • Appointment booking is temporarily unavailable

          Embeded Zoho booking page in my WordPress website. When someone starts a booking, after choosing time and date, an error appears before payment - "Appointment booking is temporarily unavailable due to restricted settings." Used the embeded code given

        • Cannot connect to 365 business calendar and Teams, says personal but it is not.

          hi I have a number of users connected to their 365 business accounts. Adding a new user and it thinks hes got 365 personal edition. He does not.... Anyone know what's going on. Trying for days now. Bookings go into his MS calendar but as its thinks its

        • Limited review (/questions) for Bookings 2.0

          Hi all, I'm writing this review of Bookings 2.0 for two reasons: 1) it may be of interest to others, and 2) I'd like to be corrected if I'm wrong on any points. It's a very limited review, i.e. the things that have stood out as relevant, and particularly

        • Zoho Bookings - Reserve with Google

          Does Zoho Bookings plan to to integrate with Reserve with Google?

        • Tip #3: How to change your booking page language

          Displaying your booking page in your target audience's language can greatly increase customer satisfaction. By speaking their language, you will help customers feel more comfortable scheduling with you and create a stronger connection with them. Let's

        • Bigin Android app update: Alerts while creating tasks outside of working days, conflicting events and calls.

          Hello everyone! In the most recent version of the Bigin Android app, we have brought in support to display an alert if task is being scheduled outside of the working days. Also, when scheduling an event or call in the Activities module, a conflict alert

        • Bigin update: Link email messages to pipeline records.

          Hello everyone! In the latest version of the Bigin iOS(v1.11.9) and macOS(1.8.9) app, we have brought in support for an option to link email to pipeline records. This helps you to view emails specific to a deal, especially when a contact is associated

        • Bigin Android app update: Custom buttons and widgets

          Hello everyone! We are excited to introduce custom buttons and widgets on the Bigin Android app. Widgets: A widget is a customizable UI component in Bigin that improves efficiency and user experience. It lets businesses embed components, streamline interactions,

        • Zoho Sheet for Desktop

          Does Zoho plans to develop a Desktop version of Sheet that installs on the computer like was done with Writer?

        • Zoho Sheet - Desktop App or Offline

          Since Zoho Docs is now available as a desktop app and offline, when is a realistic ETA for Sheet to have the same functionality?I am surprised this was not laucned at the same time as Docs.

        • Collective-booking event not added to all staff calendars

          We assign two staff to certain events. When the client books this event, it adds it to one staff calendar (the 'organiser') but not the other. How can I ensure all staff assigned to a collective booking get the event in their calendar? (A side note: it

        • Project Management Bulletin: October, 2025

          Every need leads to innovation. Project Management tools were developed out of sheer necessity to drive large projects. With wide usage, the specifications grew precise, and so did the refinement. Over years we’ve crafted one of the best project management

        • Zoho Sheets saying locked when not locked

          Zoho Sheets won't let me add more rows to sheet because it's saying the sheet is locked, but the sheet is not locked. I tried using a different browser but I still have the same issue.

        • Identifying and Merging Accounts with Similar (Non-Exact) Names

          Hello everyone, I’m aware of the built-in deduplication feature in Zoho CRM that allows merging records with exactly matching values for selected fields. However, I’m running into a situation where our migrated data contains multiple variations of the

        • Option for super admin to recover the deleted chats in Zoho Cliq

          Currently, in Zoho Cliq, if a user accidentally deletes their chats, there is no option for the Super Admin to recover or view those deleted messages. I believe it would be helpful if Super Admins had a recovery option, perhaps within a certain time frame,

        • Next Page