Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Search by contain letter in a column

      Hello, everyone I need a filter function that searches by letter in a cell, and it should be a macro. To clarify further, if I have a column with several names and I chose a search cell and what I want is search by a single letter, for example, "a" then

    • Enrich your contact and company details automatically using the Data Enrichment topping

      Greetings, I hope you're all doing well. We're happy to announce the latest topping we've added to Bigin: The Data Enrichment topping, powered by WebAmigo. This topping helps you automatically enhance your contact and company records in Bigin. By leveraging

    • Easier onboarding for new users with stage descriptions

      Greetings, I hope all of you are doing well. We're happy to announce a recent enhancement we've made to Bigin. You can now add descriptions to the stages in your pipeline. Previously, when creating a pipeline, you could only add stages. With this update,

    • Zoho Books Invoices Templates

      It would be really helpful to have more advanced features to customise the invoice templates in Zoho Books. Especially I´m thinking of the spacing of the different parts of the invoice (Address line etc.). If you have a sender and receiver address in

    • Can add a colum to the left of the item in Zoho Books?

      I would need to add a column to the left of the item column in Books. When i create custom fields, i can only display them to the right of the item.

    • Verifying Zoho Mail Functionality After Switching DNS from Cloudflare to Hosting Provider

      I initially configured my domain's (https://roblaxmod.com/) email with Zoho Mail while using Cloudflare to manage my DNS records (MX, SPF, etc.). All services were working correctly. Recently, I have removed my site from Cloudflare and switched my domain's

    • AI Bot and Advanced Automation for WhatsApp

      Most small businesses "live" on WhatsApp, and while Bigin’s current integration is helpful, users need more automation to keep up with volume. We are requesting features based on our customer Feedbacks AI Bot: For auto-replying to FAQs. Keyword Triggers:

    • Improved Contact Sync flow in Google Integration with Zoho CRM

      Hello Everyone, Your contact sync in Google integration just got revamped! We have redesigned the sync process to give users more control over what data flows into Google and ensure that this data flows effortlessly between Zoho CRM and Google. With this

    • 2025 Ask the Experts sessions wrap-up : Key highlights from the experts

      Here is a rewind journey of our Ask the Experts (ATE) Sessions, where we brought you expert insights and practical best practices together in one place. This recap highlights the key takeaways, learnings, and best practices from all these sessions so

    • How to disable the edit option in subform

      How to disable the edit option in subform

    • Adding non-Indian billing address for my Zoho subscription

      Hey Need help with adding a non-Indian billing address for my Zoho subscription, trying to edit the address to my Singapore registered company. Won't let me change the country. Would appreciate the help. Regards, Rishabh

    • Move record from one custom module to another custom module

      Is it possible to create a button or custom field that will transfer a record from one custom module to another? I already have the 'Leads' module used for the Sr. Sales department, once the deal is closed they convert it to the 'Accounts' module. I would like to create a 'Convert' button for a custom module ('Locations') for the department that finds locations for each account. Once the location is secured, I want to move the record to another custom module called 'Secured Locations'. It's basically

    • Convert Lead Automation Trigger

      Currently, there is only a convert lead action available in workflow rules and blueprints. Also, there is a Convert Lead button available but it doesn't trigger any automations. Once the lead is converted to a Contact/Account the dataset that can be fetched

    • Notes Not Saving

      Hello,  My notes are continuously not saving.  I make sure to save them, I know the process to save them.  It is not operator error.  I go back into a Leads profile a while later and do not see the previous notes that I have made.  I then have to go back and do unnecessary research that would have been in the notes in the first place.  Not a good experience and it is frustrating.  Slows me down and makes me do unnecessary work.  Please resolve.   As a quick heads up, deleting cookies is not a fix

    • Integration between "Zoho Sprints Stories" and "Zoho Projects Tasks/Subtasks"

      We have two separate teams in our organization using Zoho for project management: The Development team uses Zoho Sprints and follows Agile/Scrum methodology. The Infrastructure team uses Zoho Projects for traditional task-based project management. In

    • Prefill form with CRM/Campaigns

      I created a form in zForms and created prefill fields. I added this to the CRM and selected the fields so when sending from the CRM, the form works great. However, I want to use the same form in Campaigns and I want it to pull the data from CRM (which

    • Triggering a campaign automation from a Form

      I used Forms to create a lead form that is accessed by a button on my website. The field information flows into the CRM. However, I am trying to figure out how to use Campaign automations to start a workflow (series of campaign emails) that is triggered

    • Name changed in settings for mailbox but still not changed when typed in To field

      In the email account secretary@ i have updaetd the new staff members details but the old members name still appears when I type secretary@ in the To field. I cant work out where Zoho is finding the old name from. I have deleted the browser cache. If I

    • Employee Appraisal Applicability - Why is Date of Joining Hard-Coded?

      In the new (to me, at least) Performance Appraisal Cycle wizard, it's possible to set criteria to determine for whom the appraisal process should apply. This makes sense on its face. However, one MUST use the Date of Joining criterion as a filter. Why

    • Formula fields

      Zoho People now supports formula fields. This post illustrates it. Formula fields are fields whose value is calculated instead of being entered by the user. Using this, number, decimal and date manipulations can be done. The value of this field could be numeric or date depending on the output of the formula. In date manipulations, the result will be given in milliseconds, which you can format as per you need. The operators we support are +, - , *, /. Formula fields get recalculated automatically

    • Copy paste from word document deletes random spaces

      Hello Dear Zoho Team, When copying from a word document into Notebook, often I face a problem of the program deleting random spaces between words, the document become terribly faulty, eventhough it is perfect in its original source document (and without

    • Is it possible to use module field filters via URL parameters?

      It would be really convenient if I could quickly link to a filter. For reference, this is the filter functionality I'm referring to: https://help.zoho.com/portal/en/kb/crm/customize-crm-account/advanced-filters/articles/advanced-filters For example: My

    • Transitioning FESCO Bill Project to Zoho Sheets and Integration Options

      Hello Zoho Support, I'm considering transitioning my FESCO bill project from Google Sheets to Zoho Sheets and wanted to know if there are integration options to seamlessly migrate our existing work. You can view our platform here, any guidance would be

    • Support for Custom Fonts in Zoho Recruit Career Site and Candidate Portal

      Dear Zoho Recruit Team, I hope you're doing well. We would like to request the ability to use custom fonts in the Zoho Recruit Career Site and Candidate Portal. Currently only the default fonts (Roboto, Lato, and Montserrat) are available. While these

    • Lightbox Pop-up form

      I would like to embed my form using the lightbox pop up. I don't want it to load automatically. I want it to load when some clicks the button. I can see this option, however when I use the "show pop-up launch button" on the website, the button automatically

    • Data Processing Basis

      Hi, Is there a way to automate the data processing for a candidate every time an application arrives from job boards, without requiring manual intervention? That is, to automatically acquire consent for data processing. I've seen a workflow that allows

    • Lightbox Pop-up form

      I would like to embed my form using the lightbox pop up. I don't want it to load automatically. I want it to load when some clicks the button. I can see this option, however when I use the "show pop-up launch button" on the website, the button automatically

    • Zoho CRM for Everyone's NextGen UI Gets an Upgrade

      Hello Everyone We've made improvements to Zoho CRM for Everyone's Nextgen UI. These changes are the result of valuable feedback from you where we’ve focused on improving usability, providing wider screen space, and making navigation smoother so everything

    • Customer Management: #5 Never Let the Customer Slip

      When Rahul started Knight's Watch Consulting, his focus was simple: deliver good work and keep clients happy. He offered one-time consulting projects, monthly advisory retainers and usage-based support for growing clients. Business was steady, and customers

    • 10GB Email Storage Limits in Zoho CRM

      We’ve had Zoho One for almost 5 years and have always synced our emails from Gmail via IMAP… As of late, we’ve run into issues with our emails not syncing, due to being over the 10GB storage cap… What’s very odd is that we haven’t changed a thing? I know

    • Zoho Projects Android and iOS app update: Mobile device permission based on user profiles

      Hello everyone! We have brought in support for mobile device permissions based on the user profiles which are configured in organization level. Administrators can now configure the permissions on the web app(projects.zoho.com) by following the steps mentioned

    • Unlocking New Levels: Zoho Payroll's Journey in 2025

      Every year brings its own set of challenges and opportunities to rethink how payroll works across regulations and teams. In 2025, Zoho Payroll continued to evolve with one clear focus: giving businesses more flexibility, clarity, and control as they grow.

    • Zoho Projects Android and iOS app update: Timesheet module is now renamed as 'Time Logs', delete option has been renamed to 'Trash'.

      Hello everyone! We have now renamed the Timesheet module as Time Logs and the delete option as 'Trash' on the Zoho Projects Android and iOS app. Time Logs Android: Time Logs iOS: Trash option Android: Trash option iOS: Please update the app to the latest

    • Zoho Mail app update: Manage profile picture, Chinese (Traditional) language support

      Hello everyone! In the latest version (v3.1.9) of the Zoho Mail app update, we have brought in support to manage profile picture. You can now set/ modify the profile picture within the app. To add a new profile picture, please follow the below steps:

    • Reminders for Article Approval

      Is there a way to send reminders for approvers to review articles and approve/deny them? I'm not seeing that option anywhere.

    • To print Multiple delivery notes in batches

      In Zoho Books, we can print a Delivery Note from an Invoice using the Print Delivery Note option, but it is non-editable and always prints all line items from the invoice. Our requirement is to deliver invoiced items in batches and print delivery notes

    • Add Full-Screen Viewing for Quartz Recordings in the Client Interface

      Hi Zoho Team, We would like to request an enhancement to the Zoho Quartz client interface when viewing submitted recordings. Current Limitation: When viewing a Quartz recording from the client (user) interface, there is currently no option to switch the

    • 2025 Recap: A Year to Remember | Zoho Inventory

    • Important Update : Pipedrive deprecated fields no longer supported in Zoho Analytics

      Dear Pipedrive users, We would like to inform you about a recent update related to your Pipedrive integration with Zoho Analytics. The Pipedrive team has deprecated certain fields from their application. You can find more details in the official Pipedrive

    • Product Updates in Zoho Workplace applications | November 2025

      Hello Workplace Community, Let’s take a look at the new features and enhancements that went live across all Workplace applications this November. Zoho Mail Format comments easily using Slash Commands With Slash commands, you can easily format text, insert

    • Next Page