Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Cannot connect to 365 business calendar and Teams, says personal but it is not.

      hi I have a number of users connected to their 365 business accounts. Adding a new user and it thinks hes got 365 personal edition. He does not.... Anyone know what's going on. Trying for days now. Bookings go into his MS calendar but as its thinks its

    • Limited review (/questions) for Bookings 2.0

      Hi all, I'm writing this review of Bookings 2.0 for two reasons: 1) it may be of interest to others, and 2) I'd like to be corrected if I'm wrong on any points. It's a very limited review, i.e. the things that have stood out as relevant, and particularly

    • Zoho Bookings - Reserve with Google

      Does Zoho Bookings plan to to integrate with Reserve with Google?

    • Tip #3: How to change your booking page language

      Displaying your booking page in your target audience's language can greatly increase customer satisfaction. By speaking their language, you will help customers feel more comfortable scheduling with you and create a stronger connection with them. Let's

    • Bigin Android app update: Alerts while creating tasks outside of working days, conflicting events and calls.

      Hello everyone! In the most recent version of the Bigin Android app, we have brought in support to display an alert if task is being scheduled outside of the working days. Also, when scheduling an event or call in the Activities module, a conflict alert

    • Bigin update: Link email messages to pipeline records.

      Hello everyone! In the latest version of the Bigin iOS(v1.11.9) and macOS(1.8.9) app, we have brought in support for an option to link email to pipeline records. This helps you to view emails specific to a deal, especially when a contact is associated

    • Bigin Android app update: Custom buttons and widgets

      Hello everyone! We are excited to introduce custom buttons and widgets on the Bigin Android app. Widgets: A widget is a customizable UI component in Bigin that improves efficiency and user experience. It lets businesses embed components, streamline interactions,

    • Biometric Access Support on Zoho Vault Desktop App

      Is there any plans to add biometric authentication (fingerprint, face recognition) for Vault desktop apps (Windows/macOS) to enhance security and ease of access. I would love to hear other members view on this

    • Zoho Sheet for Desktop

      Does Zoho plans to develop a Desktop version of Sheet that installs on the computer like was done with Writer?

    • Zoho Sheet - Desktop App or Offline

      Since Zoho Docs is now available as a desktop app and offline, when is a realistic ETA for Sheet to have the same functionality?I am surprised this was not laucned at the same time as Docs.

    • Collective-booking event not added to all staff calendars

      We assign two staff to certain events. When the client books this event, it adds it to one staff calendar (the 'organiser') but not the other. How can I ensure all staff assigned to a collective booking get the event in their calendar? (A side note: it

    • Project Management Bulletin: October, 2025

      Every need leads to innovation. Project Management tools were developed out of sheer necessity to drive large projects. With wide usage, the specifications grew precise, and so did the refinement. Over years we’ve crafted one of the best project management

    • Zoho Sheets saying locked when not locked

      Zoho Sheets won't let me add more rows to sheet because it's saying the sheet is locked, but the sheet is not locked. I tried using a different browser but I still have the same issue.

    • Identifying and Merging Accounts with Similar (Non-Exact) Names

      Hello everyone, I’m aware of the built-in deduplication feature in Zoho CRM that allows merging records with exactly matching values for selected fields. However, I’m running into a situation where our migrated data contains multiple variations of the

    • Option for super admin to recover the deleted chats in Zoho Cliq

      Currently, in Zoho Cliq, if a user accidentally deletes their chats, there is no option for the Super Admin to recover or view those deleted messages. I believe it would be helpful if Super Admins had a recovery option, perhaps within a certain time frame,

    • Zoho CRM iOS app updates: the Homepage, Zoho Survey integration, and support for multiple file uploads in subform

      Hello everyone! The Zoho CRM mobile app continues to evolve to bring you a smoother and more powerful CRM-on-the-go experience. We're excited to share some important updates now available in the iOS app. Here's what's new: Homepage support Zoho Survey

    • I am trying to integrate my module in sandbox environment to add data via API

      I went through the documents and tried to setup the API for adding the module via the api but the problem is the token setup is done on "https://accounts.zoho.in/oauth/v2/token" domain and the bulk write is done on "https://content.zohoapis.com/crm/v8/upload"

    • Setting up primary mobile number.

      Hi Sir Earlier I joined IIFL team and got an organizational email of zoho services. Later i quit and now my account is deactivated with IIFL but i am unable to link my mobile number to my own email id with zoho.

    • Zoho sheet for desktop

      Hi is zoho sheets available for desktop version for windows

    • Tables for Europe Datacenter customers?

      It's been over a year now for the launch of Zoho Tables - and still not available für EU DC customers. When will it be available?

    • Guided Conversations - Ticket Creation

      Hi there, Using Guided Conversations to Take Customer Data and apply it into a Support Ticket for internal use, Is there a way to take multiple Textual Variables Inputs (A series of questions), and have the answers all appear in the Description of the

    • How to send certain invoices for Approval based on condition?

      To streamline operations, I need the invoices to go for approval when certain conditions are met. For example, if my staff gives a discount of x amount, it must go for approval. If there is no discount, then the invoice will be approved automatically.

    • Unable to continue payment

      Hi, I want to make a payment for my email subscription, but I cannot proceed with the payment because my NITKU is invalid. In Indonesia, the NITKU consists of 22 digits, but Zoho only provides 20 digits, causing my NITKU to be invalid. Please help me

    • Is there a way to show contact emails in the Account?

      I know I can see the emails I have sent and received on a Contact detail view, but I want to be able to see all the emails that have been sent and received between all an Accounts Contacts on the Account Detail view. That way when I see the Account detail

    • Massive spam pretending to come from our address – is this a Zoho security issue?

      Hi, We’ve been receiving more and more suspicious emails lately — many of them clearly look like scams. But yesterday, we got an email that appeared to be sent from our own address, which was very concerning. We're starting to wonder if this might be

    • Cant recieve mails

      I’m having an issue with my external IMAP account not fetching any emails into Zoho Mail, even though the connection test says “Success”. I can send, if someone sends an email to me. my original directory (home.pl host) recieves it in their portal but

    • Zoho Sheet Autofit Data

      While using Autofit Data function in Zoho Sheets with Devnagri Maratji or Hindi Fonts, a word or a number, it keeps couple of characters outside the right side border.

    • Enhancements to finance suite integrations

      Update on the migration (October 28, 2025): Starting this week (October 29 tentatively), we’ll begin migrating Zoho CRM users to Zoho Books as integration users. This means actions will now run under each user’s own Zoho Books access, instead of the admin’s.

    • Edit 'my' Notes only

      The permissions around Notes should be more granular, and allow to user to be able to edit the notes he created only. The edit Notes permission is useful as it allows the user to correct any mistakes or add information as needed. However, with this same

    • How do I filter contacts by account parameters?

      Need to filter a contact view according to account parameter, eg account type. Without this filter users are overwhelmed with irrelevant contacts. Workaround is to create a custom 'Contact Type' field but this unbearable duplicity as the information already

    • Job Alerts for Candidates

      hi All, I have 2 questions relating to sharing job details with candidates. 1. is there a way to notify candidates that meet certain criteria of current jobs available? eg. I run a candidate search, and identify 50 candidates that might be suitable. Can

    • Add multiple users to a task

      When I´m assigning a task it is almost always related to more than one person. Practical situation: When a client request some improvement the related department opens the task with the situation and people related to it as the client itself, the salesman

    • Enhancing Task Accuracy: Automatically notify the owners upon Task duplication.

      Hello Everyone, A Custom function is a user-written set of code to achieve a specific requirement. Set the required conditions needed as when to trigger using the Workflow rules (be it Tasks / Project) and associate the custom function to it. The requirement

    • Show/ hide specific field based on user

      Can someone please help me with a client script to achieve the following? I've already tried a couple of different scripts I've found on here (updating to match my details etc...) but none of them seem to work. No errors flagged in the codes, it just

    • Where are Kanban swimlanes

      So i've been playing with Zoho Projects Kanban view a bit more. It appears that task lists are being used as the Kanban columns, which makes sense from the implementation point of view but not the logical one.  Kanban columns are statuses that a task can flow through, while a task list has been a logical way to organize related tasks and relate them to a mislestone. In other words a task in a particular task can go through several stages while remaining in the same task list. After doing some research

    • Tip 19: How to display images in Pages using ZML snippets

      Hi folks, We're back with another tip on how to use Zoho Markup Language (ZML) to create interactive pages in Zoho Creator. You can use ZML snippets to add various kinds of components to your pages and structure them in meaningful ways. That's not all, though—using ZML you can ensure your application is automatically compatible with all kinds of devices without any inconsistencies. We often get questions from users about how to invoke Deluge scripts within ZML code. Here's a simple use case for adding

    • Can zoho swign format block text spacing automatically when prefilled from zoho crm?

      I'm sending zoho sign template from zoho crm, so that zoho crm pre-populates most fields. I have existing pdf documents that i am having signed. I have no ability to change the pdf documents, they are standardized government forms. The problem I am facing

    • Zoho Inventory / Finance Suite - Add feature to prevent duplicate values in Item Unit field

      I've noticed that a client has 2 values the same in the Unit field on edit/create Items. This surprised me as why would you have 2 units with the same name. Please consider adding a feature which prevents this as it seems to serve no purpose.

    • Reference lookup field values in Client script

      hello all, I'm using a "ZDK.Apps.CRM.Products.searchByCriteria" function call, which is returning the matching records correctly; however, one of the fields is a lookup field which I need the associated data. I believe there is a way to get this data

    • In App Auto Refresh/Update Features

      Hi,    I am trying to use Zoho Creator for Restaurant management. While using the android apps, I reliased the apps would not auto refresh if there is new entries i.e new kitchen order ticket (KOT) from other users.   The apps does received notification but would not auto refresh, users required to refresh the apps manually in order to see the new KOT in the apps.    I am wondering why this features is not implemented? Or is this feature being considered to be implemented in the future? With the

    • Next Page