Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team




      • Sticky Posts

      • Register for Zoho Desk Beta Community

        With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

      • Share your Zoho Desk story with us!

        Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

      • Tip #1: Learn to pick the right channels

        Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

      • Welcome to Zoho Desk Community - Say hello here!

        Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

      • Webinar 1: Blueprint for Customer Service

        With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

        • Recent Topics

        • Slow Performance on desk.zoho.com. 11/3/2025

          I’m not seeing any active service alerts for desk.zoho.com, but everyone on our account is currently experiencing very slow load times when opening or navigating tickets. We’ve already tried the standard troubleshooting steps — clearing cache and cookies,

        • Create View to See Tickets Closed within the last 3 days

          I'm trying to create a view in Zoho Desk that shows me "recently closed ticket," which I will define as tickets closed in the last 3 days. I want this view to update so that whenever I click to view it is recalculates and shows me tickets closed within

        • Introducing Profile Summary: Faster Candidate Insights with Zia

          We’re excited to launch Profile Summary, a powerful new feature in Zoho Recruit that transforms how you review candidate profiles. What used to take minutes of resume scanning can now be assessed in seconds—thanks to Zia. A Quick Example Say you’re hiring

        • Send Whatsapp message from Whatsapp template with custom variables

          Hi, I'm trying to do some basic integration for sending WhatsApp messages from Zoho CRM using Zoho Desk whatsapp templates. When creating new whatsapp template in Zoho Desk we can choose ticket related fields as variables but it's not clear how to use

        • Recurring Events Not Appearing in "My Events" and therefore not syncing with Google Apps

          We use the Google Sync functionality for our events, and it appears to have been working fine except: I've created a set of recurring events that I noticed were missing from my Google Apps calendar. Upon further research, it appears this is occurring

        • Dissociate account from contact

          Hello, When I make the call through the API to "Dissociate account from contact" using, /api/v1/contacts/**integer**/accounts/**integer**/dissociate I get this response back, 422Unprocessable Entity (WebDAV) (RFC 4918) { "errorCode": "INVALID_DATA", "message":

        • Zoho Desk and Zoho People Integration - is it possible in order to alert whether an agent is available

          Hi, We use Zoho People for our employees to log absence (Paid Time Off etc). I was wondering if it was possible that this information can be shared in to Zoho Desk, so that when allocating a ticket to an agent, if they are on leave there is an onscreen

        • Add Save button to Expense form

          A save button would be very helpful on the expense form. Currently there is a Save and Close button. When we want to itemize an expense, this option would be very helpful. For example, if we have a hotel expense that also has room service, which is a

        • Change Currency symbol

          I would like to change the way our currency displays when printed on quotes, invoices and purchase orders. Currently, we have Australian Dollars AUD as our Home Currency. The only two symbol choices available for this currency are "AU $" or "AUD". I would

        • Call transcrition working for ringcentral?

          I don't see anything about what telephony providers can be used. The Zoho support person A said that RingCentral isn't supported. Zoho support person B said that it works, just make sure the call recording link works. Excellent instructions here: Call

        • How to Send Email from within a custom module (with or without an email template)

          It is possible to send an email from the Deals module. However, I can't find a way to send an email from any of our custom modules. I have tried adding an email field to the modules (even though we don't really want one or need it there). That doesn't

        • All new Address Field in Zoho CRM: maintain structured and accurate address inputs

          The address field will be available exclusively for IN DC users. We'll keep you updated on the DC-specific rollout soon. It's currently available for all new sign-ups and for existing Zoho CRM orgs which are in the Professional edition. Managing addresses

        • Zoho Inventory - Managing Items With Multiple Suppliers

          Hi community, I'm working on a project at the moment for a car parts wholesale business. Each Item (part) has its own original manufacturer part number (sometimes more than one part number for the same item). When purchasing parts from 3rd party suppliers,

        • Kaizen #59 - Creating alerts and custom messages using Client Script

          Hello everyone! We are happy to resume our Zoho CRM Developer Community series - The Kaizen series! Welcome back to the new start of Kaizen! This post is about Client Script and its simple use cases involving ZDK Client functions. What is Client Script?

        • Retainer invoice in Zoho Finance modlue

          Hello, Is there a way of creating retainer invoices in the Zoho Finance module? If not can I request this is considered for future updates please.

        • How do I see the total leads during a certain period?

          I understand I can get the count of leads and potentials but the total number of leads in a certain period should be equal to Leads+potentials because when we convert a lead it gets moved to potentials and no longer exists there. is there a way i could

        • Function #46: Auto-Calculate Sales Margin on a Quote

          Welcome back everyone! Last week's function was about displaying the discount amount in words. This week, it's going to be about automatically calculating the sales margin for a particular quote, sales order or an invoice. Business scenario Where there is sales, there's also evaluation and competition between sales reps. A healthy rivalry helps to better motivate your employees to do smart work and close deals faster and more efficiently. But how does a sales rep get evaluated? 90% of the time, it's

        • Where is the Global Search field?

          I am looking for an alternative to SF.com. Zoho CRM seems to be work fine, and be customizable in terms of the fields and reports. But there's one big thing missing and it's going to prevent us from using it: there's no global search box at the top of

        • API 500 Error

          Hello amazing ZOHO Projects Community, I get this message. How can we solve this? { "error": { "status_code": "500", "method": "GET", "instance": "/api/v3/portal/2010147XXXX/projects/2679160000003XXXX/timesheet", "title": "INTERNAL_SERVER_ERROR", "error_type":

        • In place field editing for candidates

          Wondering about any insight/best practices for efficiently updating candidate records while reviewing them in a Job Opening pipeline. We can do in-field editing (e.g. update job title or City) only when we have the full candidate record open, however

        • Default tax type for mileage?

          Where we are, mileage includes a government tax. Is there any way to set a default tax for the Personal Car Mileage category of expense? (Or any other way?)

        • Analytics Portal

          I have the "standard plan" and want to explore the portal option; I activated the 15-day trial but do not see the pricing for the add-on. How can I get the price under "Upgrade add-ons." Thanks Rudy

        • The Social Wall: October 2025

          Hello everyone, As we head toward the end of the year, we’re bringing you a few updates to help give your social media efforts a strong finish. This month, we’re rolling out new enhancements across both the web and mobile app. Post Preview Have you ever

        • Show price book list price

          When using price books, once you add products to the price book in the Products related list you can display the Unit price which is the default list price; however, there is no option to show the price book list price. To see the price book list price

        • Cliq File Upload

          While uploading large file like 500MB, it takes time, that fines. But if you resize window or move window in other screen, that uploading disappears. After upload complete & sent it will be visible

        • Resizing a Record Template Background Inage

          Hi everyone, I have an issue which I can't seem to resolve: Basically, I'm designing a record template in certificate form. I've specified A5 landscape. I've set my background image the same dimensions with total pixels at 443,520. Whatever I try, when

        • Zia Actions: AI-powered Workflow Automation for Faster and Smarter Execution

          Hello everyone, Workflows got a notch better with AI-based actions. Actions such as field extraction, prediction, auto reply, and content generation facilitate quick execution with improved speed and accuracy. Zia can intercept useful details in newly

        • Constant color of a legend value

          It would be nice if we can set a constant color/pattern to a value when creating a chart. We would often use the same value in different graph options and I always have to copy the color that we've set to a certain value from a previous graph to make

        • What's New in Zoho POS - October 2025

          Hello everyone, Welcome to Zoho POS's monthly updates, where we share our latest feature updates, releases, changes, and more. Let’s take a look at how October went. Process returns for refunds, exchanges, or offer store credit Returns and exchanges can

        • Loan and purchase

          My husband is lending me mobey to buy a vehicle intersst free ... I need to know how to record the cash receipt and how I pay it back... the money is for a vehicle do I just post the invoice for that as I normally would usung the loan money to pay for

        • Zoho Connect Module in Zoho Trident

          Hi I really like where Zoho Trident is going. Having Mail and Cliq in one place is especially powerful. However, Zoho Connect really needs to be included to make this a true communication and collaboration hub. I would like to request that Zoho Connect

        • Zoho FSM API Delete Record

          Hi FSM Team, It would be great if you could delete a record via API. Thank you,

        • File Field Validation

          Hello all, We are tracking our customer NDA agreements in our CRM and have created 2 fields to do so, an execution date field and a file upload field. I want to create a validation rule to ensure that when the execution date field is populated that the

        • 100 record view limitation

          I have just migrated from another CRM and am starting in ZOHOcrm with over 5000 contacts. It seems that my searches and sorts are limited to 100 live records....or am I missing something. This seems to be very limiting...in a lot of scenarios (mass email,

        • ZUG Meet-ups are back - Across India (December 2025)

          The Zoho User Group (ZUG) meet-ups are back, and this time, we’re travelling across India to reconnect with our amazing community! From Chennai to Delhi, Bengaluru to Mumbai, we can’t wait to meet you all in person and talk everything Zoho SalesIQ, automation,

        • Categorise Attachments

          We take ID, proof of address, right to work documentation and more.  I can upload a single file in to field, but we often receive multiple files for each category e.g. someone may send a separate file for the front and back of their national ID card.  My team don't have time to manipulate the files in order to upload them as a single file. The options, as far as I can tell, would be to create additional fields on attachments in order to categorise what the file is, or to be able to upload single

        • Scheduling a meeting for just a 1:1 phone call

          My business is B2C and many of my customer's don't want to engage in an online meeting for what can be handled in a regular phone call. I am trying to create a new meeting invitation, but there is no venue optoin for "phone call". How are other's handling

        • Need more details on API Usage Dashboard

          Hi Team, We have implemented Zoho Expense for a client and has done some integrations with well known third party ERP via api. Recently we have noticed a huge spike in the API consumption. But we couldn't get the root cause for the same. I accept there

        • Power of Automation:: Automating SLA Timelines for First Response & Resolution for Issues module.

          Hello Everyone, Ever wished SLAs could update automatically based on issue severity i.e no manual tracking, no missed timelines? That is exactly what one of our customers, Alex, wanted to achieve in the Issues module. So, we have setup a simple automation

        • Finding missing records

          I have a challenge and I am not really sure where to start with it. I can't find any similar threads on here, can anyone help: I have two forms, FormA and FormB. Both forms have records that contain a field called Job_Number. What I am trying to achieve

        • Next Page