Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    Nederlandse Hulpbronnen


      • Recent Topics

      • This domain is not allowed to add in Zoho. Please contact support-as@zohocorp.com for further details

        This is the error i keep getting when trying to use my Zoho Domain Mail. This domain is not allowed to add in Zoho. Please contact support-as@zohocorp.com for further details Find attached. I hope this can be resolved very quickly so i can go on and make

      • Auto CC - Moving Departments

        We have Auto CC e-mail replies to your support mailbox enabled. We have two departments: Helpdesk (helpdesk@domain.com) Delivery (delivery@domain.com) If we create a Helpdesk ticket, and reply, replies are CC'd to helpdesk@domain.com (OK) We then move

      • unable to send message reason 554 5.1.8 Email outgoing blocked

        unable to send message reason 554 5.1.8 Email outgoing blocked

      • Mailk got blocked / Inquiry About Email Sending Limits and Upgrade Options

        Dear Zoho Support Team, My name is Kamr Elsayed I created this account to use for applying for vocational training in Germany. As part of this process, I send multiple emails to different companies. However, after sending only 8 emails today, I received

      • SMTP email sending problem

        Hello, I've sent emails before, but you haven't responded. Please respond. My work is being disrupted. I can't send emails via SMTP. Initially, there were no problems, but now I'm constantly receiving 550 bounce errors. I can't use the service I paid

      • unblock my zoho mail account. outlines@zoho.com

        please unblock my zoho mail account, outlines@zoho.com

      • Unable to see free plan option

        Hello Zoho Support Team, I hope you are doing well. I am trying to sign up for Zoho Mail, but I am unable to see the option for the free plan. Could you please guide me on how I can access or activate the free plan? Thank you for your assistance.

      • Outgoing blocked: Unusual activity detected.

        I just made payment for my Zohomail Today and have been debited so i will like to be Unblocked because this is what it says (Outgoing blocked: Unusual activity detected) Thank you i await your swift responses

      • How to remove some users in zoho accounts

        How to remove some users in Zoho accounts.

      • Forwarder

        Hi, I tried to add a forwarder from which emails are sent to my main zoho account email . However, it asks me for a code that should be received at the forwarder email, which is still not activated to send to my zoho emial account. So how can I get the

      • Forwarder

        Hi, I tried to add a forwarder from which emails are sent to my main zoho account email . However, it asks me for a code that should be received at the forwarder email, which is still not activated to send to my zoho emial account. So how can I get the

      • DKIM cannot be enabled for the domain as no verified default selector present

        Hi Support Team, For Domain DKIM record trying to enable status. but showing error "DKIM cannot be enabled for the domain as no verified default selector present" So, please resolve the issue. Thank you.

      • I can't log in to my account on Thunderbird

        I've just had to rebuild my PC (calamitous mess from Microsoft with Win10/Win 11 'upgrade' - they confirmed I had to start with a new build). I have used Zoho mail for years via Mozilla Thunderbird, but now I've had to download the latest version of TBird,

      • Sorten the domain of zoho mail ids

        I recently created zoho mail id and am quite excited with the features. But one issue ig nobody wanna type this big zohoaccounts.com I mean silly bold Suggestion zmail.com (sound gmail) (attraction) or some genz words looks cool

      • Weekly Tips : Customize your Zoho Mail Notifications

        In a professional context, email communication remains one of the most crucial channels for staying connected and managing workflows. Keeping track of your emails, task updates, or important announcements can be overwhelming—especially if you are juggling

      • How to move emails to Shared Mailbox?

        Hello, I created a Shred Mailbox instead of using a distribution group. But I cannot move previous emails to certain shared mailbox. Is it possible move some emails from inbox to shared mailbox?

      • Email task creator when task is updated/marked complete

        I am looking for a way to notify the creator of a task in zoho todo when - Task is updated Task is closed Comments entered 1 and 2 are critical, and I cannot find a zoho flow to do this. There is no way that as a manager I will know when someone has completed

      • The attendees will now be redirected to the landing webpage when they exit the webinar, set by the organizer.

        Hello All, In the latest version of the Zoho Meeting iOS mobile app (v1.3.16), we have brought in the below enhancements:   Post Webinar Re-direction Revamped exit UI for Webinar Post Webinar Re-direction:  The attendees will now be redirected to the

      • Download API file contents from browser

        Hi Team - is there something being planned to be able to trigger file downloads from the browser via a deluge script? i.e. retrieve a file via API, trigger the file download directly from the browser. Or... using the convertToPdf function (https://www.zoho.com/deluge/help/functions/file/convert-to-pdf.html)

      • March 15, 2023: Zoho Docs is discontinued

        As of today (March 15, 2023) Zoho Docs is discontinued for all users. We would like to thank our customers for trusting us for so many years! Going forward, we're confident you'll enjoy using Zoho WorkDrive for all your advanced file management and collaboration

      • When using "locations" in zoho books, can you keep the two locations totally separate from each other?

        I am looking to add a location but I don't want to intermingle the banking or other accounts. I want that to be like two separate independent branches that use different banking accounts, accounts payable, and accounts receivable. The people who are in

      • Adding a Channel to SalesIQ

        I have a client that currently uses Zoho CRM and Zoho Desk. They would like a live chat to place on their website that has a mobile app and chat bots (something like SalesIQ). However, they would also like to have all this work over SMS as well. Does the SalesIQ API allow this? Can I add visitors without them being a visitor on a website? Thanks! Bryan Redeagle

      • DataPrep Bigquery Connection failed

        Hello everybody, I want to create a connnection beetwen Bigquery and Dataprep but when I try to connect my project I got this error Loading tables has failed. Table list fetched from the data source expired.

      • How Do I Refund a Customer Directly to Their Credit Card?

        Hi, I use books to auto-charge my customers credit card. But when I create a credit note there doesn't seem to be a way to directly refund the amount back to their credit card. Is the only way to refund a credit note by doing it "offline" - or manually-

      • Zoho Flow Needs to Embrace AI Agent Protocols to Stay Competitive

        Zoho Flow has long been a reliable platform for automating workflows and integrating various applications. However, in the rapidly evolving landscape of AI-driven automation, it risks falling behind competitors like n8n, which are pioneering advancements

      • Important Features Missing

        Hey all I love linkthread, but i am missing some important features. I want to be able to include my Google Tag Manager. I have all the important stuff in my GTM Container: Facebook Pixel, LinkIn Pixel, Zoho Pagesense and so on. So i am able to do retargeting

      • Zoho SignForms: Prefill parameters with spaces render as “+” in the document (even when using %20)

        Hello Zoho Sign team, we are using SignForms with prefill parameters passed via URL. Spaces in parameter values are being rendered as “+” in the finalized document—even when we URL-encode spaces as %20. This is critical for us because we prefill addresses

      • Currency selector (based on variable) usage in comparsion

        Hi, I've developed a currency selector based on the following topic, and it's working well in pivot tables and charts: https://help.zoho.com/portal/en/community/topic/how-can-i-allow-my-users-to-choose-a-currency-for-the-dashboard I also have a comparison

      • Customize Column in Projects - Default View for Template?

        Is there a way to have the columns you'd like to see added to a template? I want my customized columns to show up every single time I create a project from that template. It appears I can customize them in the template but when I create the project they do not transfer over.  Template: See below how the columns in order are : Task, Owner, Status, Assigned, Website URL, Blog H1..etc. When I create a project from that same template it comes out as: Task, Owner, Status, Tags, Start Date, End Date, etc

      • Ask the Experts 24: Analytics, data administration, and mobile experience with Zoho Desk

        Hello Everyone! Welcome back to the Ask the Experts(ATE) series! We were so focused on our Autumn 2025 release that we didn't host an ATE session last month. In this month's ATE, we'd like to expand our areas for discussion: we'd like to listen to your

      • Multi-Page Forms in Zoho Creator!

        Let’s make long applications easier to handle by dividing them into pages, adding a progress bar, and guiding users step by step through complex data entry. This would be a total game-changer for the user experience and could significantly boost completion

      • Email login error

        Login successfully but email page error

      • Shared Mailboxes Disappeared

        Zoho Mail users in our company haven't been able to see their Shared Mailboxes for the past few hours. I've checked with App and Web Access, but they can't access them. When I send emails to their addresses, I get no error messages. They're still visible

      • Remote Server is misconfigured

        Dear Team, I am unable to use email id as remote server is misconfigured. It would be really great if you could help on this and get this resolved. Thanks & Regard Rohit Gupta

      • why cant i access my email account. it keeps asking me for reverifiying my account by entering my password.

        I cant access my account. it keeps asking me for reverifcaton by entering my password. once its entered it asks for it over and over.

      • Free webinar alert on October 16 - Less Complexity, More Security: Workplace + Directory

        Hello Zoho Workplace Community! Security and productivity shouldn't be at odds—and with Zoho, they're not. Discover how Zoho Workplace + Directory delivers seamless collaboration with enterprise-grade security, all in one integrated ecosystem. Join our

      • Email Recall Feature In Zoho Mail Which Should Also Work For Outside Organisation Members

        Add a feature to recall or undo sending an email within a configurable short time window (e.g., 30 seconds to 2 minutes) after hitting send, similar to Gmail’s undo send. Currently the sent email can not be recall If the recipient is not from within your

      • Workdrive and ChatGPT Team Synced Connectors

        Hi, we want to be able to integrate Zoho Workdrive with OpenAI’s ChatGPT Team plan. Google Drive and OneDrive both offer this, zoho please catch up asap. We dont want to have to put our company files in google drive, we want to allow chatgpt Team edition

      • Alias Name (on items) use case in Zoho inventory

        Hey, Hope everyone is well. Wondering if anyone can shed some light on the use case of Alias Names on Products in Zoho Inventory? Cheers, Chris

      • Updating an Invoice Line Item's Discount Account via API Call / Deluge Custom Function

        I need help updating an invoice line item's discount account via API. Below is a screenshot of the line item field I am referring to. Now the field to the left of the highlighted field (discount account) is the sales income account. I am able to modify

      • Next Page