Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    Nederlandse Hulpbronnen


      • Recent Topics

      • Workdrive and ChatGPT Team Synced Connectors

        Hi, we want to be able to integrate Zoho Workdrive with OpenAI’s ChatGPT Team plan. Google Drive and OneDrive both offer this, zoho please catch up asap. We dont want to have to put our company files in google drive, we want to allow chatgpt Team edition

      • Alias Name (on items) use case in Zoho inventory

        Hey, Hope everyone is well. Wondering if anyone can shed some light on the use case of Alias Names on Products in Zoho Inventory? Cheers, Chris

      • Updating an Invoice Line Item's Discount Account via API Call / Deluge Custom Function

        I need help updating an invoice line item's discount account via API. Below is a screenshot of the line item field I am referring to. Now the field to the left of the highlighted field (discount account) is the sales income account. I am able to modify

      • Send e-mail with attachments

        Dear Zoho, How is that possible in Zoho Flow to send an e-mail with attachment?  Just a simple example: Zoho Flow checks my Zoho mails and if the conditions starts the trigger then I would like to send an email with the original email's attachment. Any idea? BR, Adam

      • How to I get checkboxes on a subform to update via deluge

        Hello, would someone be able to tell me what I'm doing wrong here? I am trying to take the contents of a Deals subform and add them to an invoice then update the checkbox on each row so that 'add to invoice' is unticked and 'invoiced' is ticked. The output

      • Your bot just got smarter: AI-Powered routing that reads between the lines

        What if your bot could tell the difference? Between a visitor who just needs a quick answer, someone actively comparing options, and a frustrated customer one click away from leaving? Most bots can't. They deliver the same response to everyone, missing

      • Urgent Security Feature Request – Add MFA to Zoho Projects Client Portal Hello Zoho Projects Team,

        Hello Zoho Projects Team, We hope you are doing well. We would like to submit an urgent security enhancement request regarding the Zoho Projects Client Portal. At this time, as far as we are aware, there is no Multi-Factor Authentication (MFA) available

      • Unified customer portal login

        As I'm a Zoho One subscriber I can provide my customers with portal access to many of the Zoho apps. However, the customer must have a separate login for each app, which may be difficult for them to manage and frustrating as all they understand is that

      • Zoho Projects - Task Owner filter at Project level

        Hi Projects Team, The feature requests I would like to raise is the ability to create a custom view at the project level for projects with tasks owned by a user or users. For example "Ashley's Projects" custom view might contain a list of project in which

      • Cadences

        I have just started using Cadences for follow-up up email pipeline. Is it just me or do you find the functionality very basic? For example, it will tell me (if I go looking for it) if someone has replied to a follow-up and been unenrolled; but it won't

      • Zoho Books-Accounting on the Go Series!

        Dear users, Continuing in the spirit of our 'Function Fridays' series, where we've been sharing custom function scripts to automate your back office operations, we're thrilled to introduce our latest initiative – the 'Zoho Books-Accounting on the Go Series'.

      • Custom Fonts in Zoho CRM Template Builder

        Hi, I am currently creating a new template for our quotes using the Zoho CRM template builder. However, I noticed that there is no option to add custom fonts to the template builder. It would greatly enhance the flexibility and branding capabilities if

      • Would be really awesome to have Created Time and Modified Time showing for custom functions list

        It would be SO HELPFUL to be able to sort custom functions by created time/ modified time. Also seeing a created by/ modified by with the little profile picture would be supremely helpful as well. Just really hard sometimes to find a function you were

      • Ticket Export Does Not Include Ticket Threads

        Dear Zoho Desk Support Team, I hope you’re doing well. I would like to report an issue regarding the ticket export functionality in Zoho Desk. Currently, when exporting tickets, the ticket threads or conversation history are not included — only the ticket

      • Payments made notification

        This is a really wonderful feature but we can only use it for about 50% of payments made & have to revert to sending statements which is a real chore. Credits applied to the bills paid in the notification aren't included & this causes great confusion in the accounts receivable departments. Please, please add this required feature asap ! .....

      • Ability to add VAT to Retainer Invoices

        Hello, I've had a telephone conversation a month ago with Dinesh on this topic and my request to allow for the addition of VAT on Retainer Invoices.  It's currently not possible to add VAT to Retainer Invoices and it was mutually agreed that there is absolutely no reason why there shouldn't be, especially as TAX LAW makes VAT mandatory on each invoice in Europe!   So basically, what i'm saying is that if you don't allow us to add VAT to Retainer Invoices, than the whole Retainer Invoices becomes

      • ZOHO DESK link with Power BI

        HI, I am using ZOHO Desk for last two years and now my client is asking to integrate ZOHO desk data to Power BI so that they can use Data for reporting. Kindly guide in details so that i can give access to ZOHO desk export data for more visibility.

      • URLs being masked despite disabling tracking

        Hey, We had disabled click tracking from an email update we are sharing with our users. Despite this, the URL the end user is receiving is masked, and looks like "https://qksyl-cmpzourl.maillist-manage.net/click/1d8e72714515cda6/1d8e72714515ca70" instead

      • Zoho CRM - Calendar Cards View - Let Users Decide What Is Displayed On Calendar Entries

        Imagine planning your week of face-to-face meetings across three counties. You’re trying to group appointments by location to make the best use of your time, but Zoho CRM’s calendar doesn’t show where each meeting is happening. You’re left trying to remember

      • Dashboard Autorefesh

        Good day, I am a dashboard that displays the number of tickets based on "Product Name". This dashboard is displayed on a big TV for the team to monitor. Can the dashboard auto-refresh every few minutes to display the new values? Currently, we have closed

      • Deferred/ Unearned revenue

        Dear Zoho Team, Just in case you have missed out my query posted few days ago: We issue invoices relating to 12-month web hosting service. When we issue the invoice, we should record the entire amount of the invoice as DEFERRED/UNEARNED REVENUE (ie. $10

      • Report Hover Setting

        Would be great if we will able to show information to the user while hovering a record in a report.

      • Bigin Android app update: Zoho Books integration

        Hello everyone! We’re excited to introduce Zoho Books integration on the latest version(v1.8.5) Bigin Android app. Once the integration is completed in the web(bigin.zoho.com), the Zoho Books tab will be visible in detail View of Contacts, Companies,

      • Audit Log enhancements: Increased retention period, better user visibility, and more

        Hello everyone, The Audit Log brings in the following enhancements which improve the overall ability to manage data. Why did we make these updates? Extended Data Retention: Audit data can now be filtered and exported for a 60-day period, doubling the

      • Question Regarding Managing Sale Items in Zoho Books

        Good day, I was wondering about something. Right now, Zoho Books doesn’t seem to have a way to flag certain items as being on sale. For example, if I want a list of specific items to be on sale from October 1 to October 12, the user would have to export

      • [WEBINAR] Smooth year-end closure with Zoho Books (KENYA)

        Hello there, This webinar is for all Kenyan businesses looking to wrap up their financial year smoothly! Join our free session to learn how Zoho Books can simplify your year-end process. What to expect from this webinar: - All the latest updates in Zoho

      • System flaws and lack of response from Zoho

        I have had to go on here as no-one is replying to my emails regarding my problem (been 10 days and no email reply) and your chat facility online through your Zoho Books software opens and closes immediately, so not functioning properly. I actually called

      • Customer Grouping

        Hi, how can I group multiple customers into single group. So that I can have idea of accounts receivables of all the customers in single group. Like if there are multiple subsidiaries of same company we have having a business with, and want to view the

      • Two currencies

        More and more I am finding that internattional payments' fees are unpredictable. I would like, on my invoices that are in a foreign currency (eg. USD$ or EUR€) for there to be a GBP£ TOTAL display alongside the invoice's currency total. This would make

      • Zoho Books | Product updates | September 2025

        Hello users, We’ve rolled out new features and enhancements in Zoho Books. From PayNow payment method to applying journal credits to invoices and bills in other locations, explore the updates designed to enhance your bookkeeping experience. Integrate

      • GST Slabs Redefined: Stay Compliant Using Zoho Books!

        Hello Everyone! The Government of India is rolling out new GST rates, a major reform aimed at simplifying the current tax structure starting 22 September 2025. GST will move from four slabs (5%, 12%, 18%, 28%) to two main slabs (5% and 18%), plus a special

      • Zoho Books | Product updates | October 2025

        Hello users, We’ve rolled out new features and enhancements in Zoho Books. From iOS 26 updates to viewing reports as charts, explore the updates designed to enhance your bookkeeping experience. Zoho Books Updates for Apple Devices At WWDC 2025, Apple

      • Need Inactive accounts to be visible in Reports in Zoho Books

        I N=need Inactive accounts to be visible in Reports in Zoho Books to do recons of the accounts but when i see the same they are not visible in the Accountant - Account Transactions report

      • Edit item custom fields

        Getting this error : Transactions have been created with the custom field. Hence it cannot be deleted. Not trying to delete it, just trying to change which modules to show in or to not show at all in transactions !

      • Zoho Books - How to Invoke a Custom Function in Schedulers

        We have multiple schedulers that send emails to customers in batches. Currently, we are maintaining the same code across several schedulers. Is it possible to use a custom function inside a scheduler script? If yes, how can we invoke the custom function

      • Use Zoho Books to bill for work done in Zoho Desk??

        I'm trying to see if something is possible (and if yes, how). We use Zoho One to manage our business. We have a lot of clients that will put in a ticket (via portal) to have work done. Out techs will pick up the ticket, do the work, and then log the time

      • Zoho Finance Suite - Customer Custom Tabs - Dynamic Link

        Hi Finance Suite team, When creating a Custom Tab for a Client Portal, there is no option to add dynamic parameters. This would be very helpful for adding Zoho Analytics dashboards which can be dynamically filtered through the URL to only show information

      • Modular Permission Levels

        We need more modular Permissions per module in Books we have 2 use cases that are creating problems We need per module export permission we have a use case where users should be able to view the sales orders but not export it, but they can export other

      • Blueprint or Validation Rules for Invoices in Zoho Books

        Can I implement Blueprint or Validation Rules for Invoices in Zoho Books? Example, use case could be, Agent confirms from client that payment is done, but bank only syncs transactions tomorrow. in this case, Agent can update invoice status to done, and

      • Feature Enhancement Request – Text Formatting Options in Item Description (Zoho Books/Quotes Module)

        Dear Zoho Development Team, Greetings from Radiant360 Integrated Technical Services LLC. We would like to bring to your attention a functional limitation we've encountered within the Item Table / Quote Description section of Zoho Books (and Zoho CRM Quotes).

      • Next Page