Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Default Tagging on API-generated Transactions

      If one assigns tags to an Item or Customer, those tags get auto-populated in each line item of an Invoice or Sales Order when one creates those documents. However, if one creates the Sales Order or Invoice via the API (either directly coding or using

    • DKIM cannot be enabled for the domain as no verified default selector present

      Hi Support Team, For Domain DKIM record trying to enable status. but showing error "DKIM cannot be enabled for the domain as no verified default selector present" So, please resolve the issue. Thank you.

    • unable to send message reason 554 5.1.8 Email outgoing blocked

      unable to send message reason 554 5.1.8 Email outgoing blocked

    • I can't log in to my account on Thunderbird

      I've just had to rebuild my PC (calamitous mess from Microsoft with Win10/Win 11 'upgrade' - they confirmed I had to start with a new build). I have used Zoho mail for years via Mozilla Thunderbird, but now I've had to download the latest version of TBird,

    • Should I Use DMARC?

      When I configure Zoho Mail's DMARC settings, it's mandatory to fill in the RUA and RUF (Aggregate notification email address*, Forensic notification email address*) addresses. When we enter an email address in these fields, we receive reports from the

    • Introducing Profile Summary: Faster Candidate Insights with Zia

      We’re excited to launch Profile Summary, a powerful new feature in Zoho Recruit that transforms how you review candidate profiles. What used to take minutes of resume scanning can now be assessed in seconds—thanks to Zia. A Quick Example Say you’re hiring

    • Books API Receiving an Error that Doesn't Make Sense when Creating Credit Note - trying to use 'ignore_auto_number_generation' argument

      Hello, I'm working on a newly created routine and I'm getting an error that doesn't make sense when trying to create a new Credit Note. Here is my POST request. Endpoint: https://www.zohoapis.com/books/v3/creditnotes?organization_id=########## Body:     {

    • How to move emails to Shared Mailbox?

      Hello, I created a Shred Mailbox instead of using a distribution group. But I cannot move previous emails to certain shared mailbox. Is it possible move some emails from inbox to shared mailbox?

    • Collaboration with free plan user.

      Hello. Do both users need to be on Pro plan to collaborate? Or can a Pro user share a note with a free plan user with the ability to edit?

    • Zoho Inventory Now Supports VeriFactu for Businesses in Spain

      Starting from January 1, 2026, Spain requires real-time invoice reporting for all B2B transactions. From July 2026, this requirement will extend to B2C transactions as well. All reporting must be carried out through the VeriFactu to AEAT (Agencia Estatal

    • Connection to other user

      Zoho Cliq handles sharing of Custom OAuth Connections that require individual user logins.

    • In arattai received message can't be deleted

      The issue has been noticed in following: arattai app (Android) arattai app (Window) arattai web While the message posted by me may be deleted, the ones received from others can't be. The item <Delete> change to <Report> when the message is a received

    • Analytics Portal

      I have the "standard plan" and want to explore the portal option; I activated the 15-day trial but do not see the pricing for the add-on. How can I get the price under "Upgrade add-ons." Thanks Rudy

    • Can I filter a Lookup field to only show related records from another Lookup in Zoho CRM?

      In our Leads module, we have two Lookup fields: Recruitment Agency – the agency that the lead came from Recruiter – the individual contact from that agency who sent us the lead Both fields link correctly to their respective records in CRM. However, our

    • Zoho Books API invoice email bouncing with 'relaying-issues' error

      I have waited over 30 days for zoho books uk to assist with the following and i have had no replies or tickets erronously closed. The service has been terrible - very unlike zoho! So i am raising this here hoping that a community member can assist: Hello,

    • How do I filter contacts by account parameters?

      Need to filter a contact view according to account parameter, eg account type. Without this filter users are overwhelmed with irrelevant contacts. Workaround is to create a custom 'Contact Type' field but this unbearable duplicity as the information already

    • Zoho Writer to Zoho Sign Template Misalignment – Request for Community

      We are currently facing a critical issue impacting our live business operations due to a misalignment problem between Zoho Writer templates and Zoho Sign output. Issue Summary: We have an automated workflow in Zoho CRM using Deluge scripts that generates

    • Account blocked

      Yesterday I got my Zeptomail account blocked due to too many hard bounces. My account is used exclusively for sending transactional emails (eg. your order has been shipped, a form has been filled, etc) and the sudden blocking impacted hundreds of websites

    • Migration Tool Download

      Since apparently you have to ask for the migration tool to download (why?), I need the download for the migration tool for exchange servers. Please forward link, thanks.

    • IF Formula

      Hi, I'm having trouble setting up a custom formula in a field. Can you help me with it? The formula should do the following: If the created date is less or equal to 30 days the field should display a string New in pieline or Exisitng pipeline if above

    • Autorespond to one specific sender

      Is there any way to create a rule or filter that sends an automatic reply to messages from one specific email address?  I really need to set up something like a "vacation response" but only for one "received from" address.

    • View comprehensive logs of all actions in Custom Module History

      Hello everyone, We are happy to announce our new feature, Detail View in Custom Module History, which enhances visibility, control and accountability over your records. Users can now access a detailed timeline of every action that has been performed on

    • Track ZohoForm Conversion using Postmessage event

      Hi, I’ve been using a third-party lead tracking tool to capture leads from my website along with their source. Earlier, with the HubSpot form, the third-party script was able to detect the postMessage event that iframe forms typically send back to the

    • Email task creator when task is updated/marked complete

      I am looking for a way to notify the creator of a task in zoho todo when - Task is updated Task is closed Comments entered 1 and 2 are critical, and I cannot find a zoho flow to do this. There is no way that as a manager I will know when someone has completed

    • Email Recall Feature In Zoho Mail Which Should Also Work For Outside Organisation Members

      Add a feature to recall or undo sending an email within a configurable short time window (e.g., 30 seconds to 2 minutes) after hitting send, similar to Gmail’s undo send. Currently the sent email can not be recall If the recipient is not from within your

    • How to implement calculating average value of averaged values?

      Hi, I'm trying to implement a formula in my layout for a module that calculates the average of 2 averaged values, but I can't do so using the formula custom field. To calculate the 2 averaged values, I could use the formula custom field, but in the custom

    • Zoho Recruit update published job boards

      If we have a job thats already posted on several different job boards and we edit this job in zoho recruit (e.g. the description of the job), does this then automatically update this new information to all the published job boards?  If not, how does the

    • Narrative 14: The magic bazaar of Desk

      Behind the scenes of a successful ticketing system: BTS Series Narrative 14: The magic bazaar of Desk Marketplace is a centralized platform where users can discover Zoho and third-party applications that can be integrated with their Desk account to facilitate

    • Collections Management: #2 Late Payment Automation

      "Sir, I'll process the payment by evening, please don't apply any late payment charges", the customer said while arranging papers on his desk. Sanjay stood there with a tired smile. He had already visited twice that week. Payment collection was one thing.

    • Create a table in the layout section of modules

      Dear Sirs, I would like to know whether is possible to create a table under new fields in layouts section of modules. In fact in this section there are single line, multi line, pick list, multi-select, date, formula etc. but I am unable to add a table

    • Nimble enhancements to WhatsApp for Business integration in Zoho CRM: Enjoy context and clarity in business messaging

      Dear Customers, We hope you're well! WhatsApp for business is a renowned business messaging platform that takes your business closer to your customers; it gives your business the power of personalized outreach. Using the WhatsApp for Business integration

    • Power of Automation :: Automatic Task Status Handling for Users

      Hello Everyone, A custom function is a software code that can be used to automate a process and this allows you to automate a notification, call a webhook, or perform logic immediately after a workflow rule is triggered. This feature helps to automate

    • Question - why no way to input a 'tool description' and 'tree of 'tools'

      Every business is different, with different business processes. To be truly useful Zoho MCP needs to have user editable tool descriptions (or ruleset) and a 'tool tree' so that the LLM is context aware when being used. For example, the tool description

    • Develop Zoho Meeting as a Full Native Application (Not a Browser Wrapper)

      Hello Zoho Meeting Team, Hope you are doing well. We would like to suggest an important improvement regarding the Zoho Meeting desktop application. At the moment, the Zoho Meeting app feels more like a mini browser window or an iframe that loads the web

    • Can I create a custom function for chart of accounts?

      I'm trying to add a custom function that should run whenever a record in the Chart of Accounts module is edited, but on the form for creating a new function, the module dropdown does not include the chart of accounts as an option.

    • billing

      hi, I am being billed $12/year, and I can't remember why. My User ID is 691273115 Thanks for your help, --Kitty Pearl

    • Automatic Matching from Bank Statements / Feeds

      Is it possible to have transactions from a feed or bank statement automatically match when certain criteria are met? My use case, which is pretty broadly applicable, is e-commerce transactions for merchant services accounts (clearing accounts). In these

    • Unable to confirm Super Admin assignment — confirmation button not working

      I’m trying to change the roles within my organization. I am currently a super admin and would like to add another user as a super admin. When I attempt to confirm the action, a screen appears asking for my password to verify my identity. However, when

    • Automation#36: Auto-create time-entry after performing the Blueprint transition

      Hello Everyone, This week’s edition focuses on configuring a custom function within Zoho Desk to streamline time tracking within the Blueprint. In this case, we create a custom field, and request the agent to enter the spending time within the single

    • Feature Request: Reviews

      Any chance we could get a "Reviews" element to put on sites? It should be extremely simple to do. Basically just take the comments box and add an option for a star rating. At the very least, it would be nice to have more customization options for the comment box so the text could be changed from "Comments" to "Reviews". I've looked at a bunch of embeddable review widgets but nothing meets my needs so this would be a great feature to have added to the site builder.

    • Next Page