Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Partner with HDFC And Sbi Bank.

      Hdfc and sbi both are very popular bank if zoho books become partner with this banks then many of the zoho books users will benefit premium features of partnered banks.

    • Customer Satisfaction (CSAT) Report

      From data to decisions: A deep dive into ticketing system reports The customer satisfaction (CSAT) report helps teams understand how customers feel about their support experience, identify service gaps, and continuously improve the help desk. It turns

    • Highlight a candidate who is "off limits"

      Hello: Is there a way to highlight a candidate who is "off limits"?  I would like to have the ability to make certain candidate and / or Client records highlighted in RED or something like that.   This would be used for example when we may have placed a candidate somewhere and we want everyone in our company to quickly and easily see that they are off limits.  The same would apply when we want to put a client or former client off limits so no one recruits out of there. How can this be done? Cheers,

    • Timeline Tracking Support for records updates via module import and bulk write api

      Note: This update is currently available in Early Access and will soon be rolled out across all data centers (DCs) and for all editions of Zoho CRM. The update will be available to all users within your organization, regardless of their profiles or roles.

    • Shifts in Zoho People vs Zoho Shifts?

      Hello Zoho People Team, We hope you are doing well. We are evaluating the Shifts functionality within Zoho People and comparing it to the standalone Zoho Shifts product. We’ve encountered comments and discussions suggesting that the Shifts feature inside

    • Disable fields in During action in Blueprint?

      Hi there. I've tried field disable (setReadOnly(true)) using client script and the event is onMandatoryFormLoad on detail page, assuming it'll work on blueprint fields, but it bears no result. Is this the expected behaviour? That we can't do this yet?

    • Allow Attaching Quartz Recordings to Existing Zoho Support Tickets

      Hi Zoho Team, We would like to request an enhancement to how Zoho Quartz recordings integrate with Zoho Support tickets. Current Behavior: At the moment, each Quartz recording automatically creates a new support ticket. However, in many real-world scenarios:

    • From Zoho CRM to Paper : Design & Print Data Directly using Canvas Print View

      Hello Everyone, We are excited to announce a new addition to your Canvas in Zoho CRM - Print View. Canvas print view helps you transform your custom CRM layouts into print-ready documents, so you can bring your digital data to the physical world with

    • FSM - Timesheet entires for Internal Work

      Hi FSM Team, Several of my clients have asked how they can manage internal timesheets within Zoho FSM. Since their technicians already spend most of their day working in FSM, it would be ideal if they could log all working hours directly in the FSM app.

    • Develop and publish a Zoho Recruit extension on the marketplace

      Hi, I'd like to develop a new extension for Zoho Recruit. I've started to use Zoho Developers creating a Zoho CRM extension. But when I try to create a new extension here https://sigma.zoho.com/workspace/testtesttestest/apps/new I d'ont see the option of Zoho Recruit (only CRM, Desk, Projects...). I do see extensions for Zoho Recruit in the marketplace. How would I go about to create one if the option is not available in sigma ? Cheers, Rémi.

    • Best Email Backup Wizard in 2026

      While searching for an email backup solution, my main hesitation was reliability. As a user, I had already seen many tools that looked promising but failed when handling large mailboxes, skipped folders, or caused authentication issues during the backup

    • Adding a threshold to a line chart based on date range

      I have a line chart that is tracking a percentage over time. It also has a filter for 50 different clients. I would like to create a threshold that is based on a portion of the date range. As I understand it, this would be done by adding a column to the

    • CRM gets location smart with the all new Map View: visualize records, locate records within any radius, and more

      Hello all, We've introduced a new way to work with location data in Zoho CRM: the Map View. Instead of scrolling through endless lists, your records now appear as pins on a map. Built on top of the all-new address field and powered by Mappls (MapMyIndia),

    • Workflow Down/Bug

      We have a workflow that sends an email to one of our internal departments 10 minutes after a record is created in a custom module. The workflow actually works correctly. However, we have now noticed that on January 8, between 3:55 p.m. and 4:33 p.m.,

    • Subforms and automation

      If a user updates a field how do we create an automation etc. We have a field for returned parts and i want to get an email when that field is ticked. How please as Zoho tells me no automation on subforms. The Reason- Why having waited for ever for FSM

    • Allow Managers to Create Shifts for Their Departments in Zoho People

      Hello Zoho People Product Team, Greetings and hope you are doing well. This feature request is related to Zoho People - please don't move it to zoho one! We would like to submit a feature request regarding shift management permissions in Zoho People.

    • Zoho Learn and Zoho CRM integration

      I would like to see an integration between Zoho Learn and Zoho CRM. 1. To be able to add articles in a related list in all modules 2. Zia to suggest related articles in a Deal or Case or Lead 3. Ability to read / search articles during a call / follow

    • Maintain steady traffic to your domain: How Domain Aliasing helps

      Consider this scenario: An organization has its primary domain as administrator.com. Now it wants to shorten its domain to admin.com because it's simpler and easier to remember. However, changing the domain completely can cause the following problems:

    • Why Sharing Rules do Not support relative date comparison???

      I am creating a Sharing Rule and simply want to share where "Last Day of Coverage" (Date field) is Greater than TODAY (Starting Tomorrow). However, sharing rules don't have the option to compare a date field to a relative date (like today), only to Static

    • How do I migrate OLM file to Gmail?

      Migrating emails from Outlook for Mac to Gmail can be challenging because Gmail does not support OLM files directly. This limitation often causes confusion and delays, especially when users need quick access to important emails and mailbox data on a web-based

    • CRM x WorkDrive: File storage for new CRM signups is now powered by WorkDrive

      Availability Editions: All DCs: All Release plan: Released for new signups in all DCs. It will be enabled for existing users in a phased manner in the upcoming months. Help documentation: Documents in Zoho CRM Manage folders in Documents tab Manage files

    • IF Statement in Zoho CRM Formula Field

      Hi, I am attempting to write a formula field that will give me one result if one statement AND another statement are true, then a different value if the first statement AND a different statement are true, else 0. Stated differently: if account = destination

    • CRM Percent custom fields: When will it show the % symbol and behave like %?

      1. Actually Percent custom fields fail to show the % symbol. 2. When in formulas Percent fields work like number: 100 x 5% = 5 ideal world 100 x 5% = 500 what happens actually 3. When importing Percent fields the % symbol has to be removed and the data

    • Free Webinar: Zoho Sign for Zoho Projects: Automate tasks and approvals with e-signatures

      Hi there! Handling multiple projects at once? Zoho Projects is your solution for automated and streamlined project management, and with the Zoho Sign extension, you can sign, send, and manage digital paperwork directly from your project workspace. Join

    • New in CRM: Dynamic filters for lookup fields

      Last modified on Oct 28, 2024: This feature was initially available only through Early Access upon request. It is now available to all users across all data centers, except for the IN DC. Users in the IN DC can temporarily request access using this form

    • Zoho Sheet - Desktop App or Offline

      Since Zoho Docs is now available as a desktop app and offline, when is a realistic ETA for Sheet to have the same functionality?I am surprised this was not laucned at the same time as Docs.

    • Automatically CC an address using Zoho CRM Email Templates

      Hi all - have searched but can't see a definitive answer. We have built multiple email templates in CRM. Every time we send this we want it to CC a particular address (the same address for every email sent) so that it populates the reply back into our

    • Editing the Ticket Properties column

      This is going to sound like a dumb question, but I cannot figure out how to configure/edit the sections (and their fields) in this column: For example, we have a custom "Resolution" field, which parked itself in the "Ticket Information" section of this

    • Zoho Bookings and Survey Integration through Flow

      I am trying to set up flows where once an appointment is marked as completed in Zoho Bookings, the applicable survey form would be sent to the customer. Problem is, I cannot customise flows wherein if Consultation A is completed, Survey Form A would be

    • "Total Hours" on Employee Attendance Report

      I'm learning that in Zoho jargon, "total hours" does not include paid breaks. Or at least not the way that my setup is working. That seems a little weird to me, since most jurisdictions in the US don't differentiate between time spent on paid break and

    • Bulk upload image option in Zoho Commerce

      I dont know if I am not looking into it properly but is there no option to bulk upload images along with the products? Like after you upload the products, I will have to upload images one by one again? Can someone help me out here? And what should I enter

    • Integration with...

      Dear Zoho Commerce team, Please could you consider the integration within Zoho Commerce / Inventory and Qapla'? (https://www.qapla.it/en/) This app is better than Aftership in many ways: - Aftership integration require PRO plan and price start from more

    • Image field in custom module

      Hi guy, Is there any hope of adding a custom image field in the custom module? We created a custom module to keep track of assets, and it would be helpful if we could attach an image to the record. Thanks Rudy

    • Repeat Column merge in ZOHO writer columns doesn't allow to set max columns per row

      I'm using ZOHO writer to merge data from a ZOHO CRM subform and I want it to make a table. We're using Insert Table for Column Repeat, because this is what we need. (Name of column (Teamname) and underneath that a list of names of teammembers). It works

    • Generate leads from instagram

      hello i have question. If connect instagram using zoho social, it is possible to get lead from instagram? example if someone send me direct message or comment on my post and then they generate to lead

    • Adding Markdown text using Zoho Desk API into the Knowledge Base

      Hi Zoho Community members, We currently maintain the documentation of out company in its website. This documentation is written in markdown text format and we would like to add it in Zoho Knowledge Base. Do you know if there is REST API functionality

    • Create case via email

      Good Afternoon, I have just registered and am taking a look around the system. Is it possible to create a case via email.  I.e. an employee/client/supplier emails a certain address and that auto generates the case which then prompts a member of staff

    • Need a Universal Search Option in Zohobooks

      Hello Zoho, Need a Universal Search Option in Zohobooks to search across all transactions in our books of accounts. Please do the needful Thanks

    • Custom Sorting based on other columns in table

      I need the ability to apply custom sorting to a text-based dimension in the X axis where the sorting is based on another column in the table. For example, I have a chart report where the X axis is a text label. I would like to be able to sort those text

    • Locked Notebook

      Hi, I hadn't used my Notebook in some time and was refamiliarizing myself with it. I clicked a lock icon and now I can't unlock. When I hit the information or unlock icons I'm taken to a page with the notebook icon and a keyboard. When I type, nothing

    • Next Page