Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Notes Attachments

      Two things it would be nice to have the attachment size the same as the attachments sections and it would be nice to be able to attach links like you can in the attachments section. Thank you

    • Canvas: empty images

      Hello. If I add an image field like contact photo in a Canvas design, and the field is empty, there is an ugly placeholder in its place. This doesn't happen in the standard view. In the standard view, if the contact photo is empty, nothing appears in

    • Google enhanced conversions not working

      Hi guys, I've connected Zoho CRM through Google Ads interface with the goal to setup the enhanced conversion tracking in Google Ads. I have to Zoho related conversion goals which you can see in the images below: For the conversion goal above I've setup

    • MS Teams Meeting to Zoho CRM

      Has anyone figured out a good way to push MS Teams meeting info on a trigger of "meeting end" to Zoho CRM? We're looking for a way to take attendees of a meeting and meeting duration and push it into Zoho CRM after the meeting has ended. If I can just

    • Font Size 11 - Zoho CRM Email Templates

      Our company communicates with our vendors exclusively using Calibri Font Size 11, as this is the standard formatting for professional emails. Since the CRM only allows for the selection of font sizes 10 & 12, we have been unable to utilize the CRM email

    • Subforms in stateless forms

      I think the title says it all. We need to be able to add subforms to stateless forms. Currently the only workaround is to create a Form and delete each record upon submission of the form. I need to build an interface to update our inventory. Basically

    • Calling Function via REST API with API Key gives 401 using Zoho Developer

      Hi, I created a couple of functions using the one month trial of Enterprise edition, which I was able to call using the API Key method from Postman and from an external site. Now that my trial has expired, I have created the same functions in the Developer

    • How do I move a section or element from one page to another in the NEW Zoho Sites UI

      I have a section on my home page with numerous elements within it and I'd like to move the entire section to a different page on my site so I don't have to recreate it from scratch.  Is there a way for me to do that easily? I could use a quick answer on this please.

    • Zoho Projects app update: Global Web Tabs support

      Hello everyone! In the latest version(v3.10.10) of the Zoho Projects app update, we have brought in support for Global Web Tabs. You can now access the web tabs across all the projects from the Home module of the app. Please update the app to the latest

    • Export as MP4 or GIF

      Hi, Just wondering if there's a way to export/convert a presentation to an MP4 video file or even a GIF. One use case would be to use the animation functionality to create social media graphics/charts/gifs/videos. Thanks for a great tool... Rgds Jon

    • Page Layout- Horizontal Rule

      When editing the layout of, for instance, the Potentials page, is there a way to insert a horizontal rule or white space in between fields?  I'd like to keep a group of fields in the same Section, but would like to create some seperation in order to further group together certain fields within the Section.  If this is not possible, does anybody have any other suggestions on how to create this same effect?   Thank you!

    • Zoho Books will discontinue support for older browser versions soon

      Hello users, Starting from May 15, 2024, Zoho Books will no longer support the following browser versions: Browsers Version Restrictions Firefox Browser Versions older than 100 Google Chrome Versions older than 100 Microsoft Edge Versions older than 100

    • Zoho Projects - Q3 Updates | 2025

      Hello Users, The final quarter of the year 2025 has begun, and we at Zoho Projects are all set with a plan. New targets to achieve and new milestones to reach, influenced by the lasting imprint of the past quarter. 2025's Q3 saw some new features and

    • Zoho Sheet - Printing - Page Breaks and Printing Customization

      I think the title is descriptive enough in that I cannot find help documentation on a simple task of adding in page brakes for separating pages on print. Thanks

    • Issue with Trident exe file

      Hello Team, Exe Setup file It's showing harmful for user pc please check and do needful. this message for developer team. Thanks Bhargav Purohit

    • Different languages for users

      Hello, Do you plan to enable individual users to select their languages for interface? Currently language can be changed for everyone - it looks like a settings for a whole portal, which is not good when you are working internationally. Best regards,

    • Transaction Locking with the dynamic date

      Is it possible to dynamically update dates on transaction locking. We want to lock transaction x days from today

    • Unable to change sales_order status form "not_invoiced" to "invoiced"

      I am automating process of creating of invoice from sales_orders by consolidated sales_orders of each customer and creating a single invoice per customer every month. I am doing this in workflow schedule custom function where i create invoice by getting

    • Apply Vendor Credits Automatically

      We are bulk importing Vendor credits in Zoho Books!!! Is there a way to apply vendor credits automatically to the first UNPAID bill of the Vendor?

    • Apply Vendor Credit Automatically

      Hello!!! Is there a way where in we can apply vendor credits automatically on the FIRST OUTSTANDING BILL of the vendor?? We have lots of VENDOR CREDITS ISSUES mostly!!! Applying it manually is a pain for us. Would be great if we have a way to apply the

    • Need Easy Way to Update Item Prices in Bulk

      Hello Everyone, In Zoho Books, updating selling prices is taking too much time. Right now we have to either edit items one by one or do Excel export/import. It will be very useful if Zoho gives a simple option to: Select multiple items and update prices

    • Creator problem: Edit form View not displaying whole form

      I'm having problems with the form in my database. The edit view is not showing the whole form: This is how it looks in the normal view: As you can see, there is a whole section in the bottom right of the form which is not displaying in the Edit View. This means that I can't change or delete any of these fields... Anybody had a similar problem or know a solution? Any help would be appreciated.   Cheers,

    • Vendor Master Enhancements for Faster Purchase Entry

      I’d like to suggest a few features that will improve accuracy and speed during purchase voucher entry: Automated Item Tax Preference in Vendor Master Add an option to define item tax preference in the vendor master. Once set, this preference should automatically

    • Quick Item Search & Auto-suggestion for Invoices

      Hi Team, I am facing an issue while creating invoices in Zoho Books. Currently, I have to type the full item name in the correct sequence and spelling for it to appear. For example, my item name is: "Distemper Acri Silk Special White 10kg" If I type something

    • Function #53: Transaction Level Profitability for Invoices

      Hello everyone, and welcome back to our series! We have previously provided custom functions for calculating the profitability of a quote and a sales order. There may be instances where the invoice may differ from its corresponding quote or sales order.

    • Integrating Chatbot with Zoho Creator Application

      Is it possible to integrate a chatbot with a Zoho Creator application?

    • Average Costing / Weighted Average Costing

      Hello fellow maadirs. I understand Zoho Books uses FIFO method of dealing with inventory costing, but do you guys have any plans to introduce average costing? We indians need average costing. It's part of our culture. Please. I beg thee. Thanks.

    • 'Add Tax To Amount' not reflected in Invoice

      Hi Zoho Support, I'm experiencing an issue with tax calculation display in my invoice template. Despite having "Add tax to amount" box checked in the template settings, the Amount column is not showing the tax-inclusive total for line items. Current behaviour:

    • "Subject" or "Narration"in Customer Statement

      Dear Sir, While creating invoice, we are giving in "Subject" the purpose of invoice. For Example - "GST for the month of Aug 23", IT return FY 22-23", "Consultancy", Internal Audit for May 23". But this subject is not coming in Customer Statement. Only

    • A real WYSIWYG field instead of the limited rich text

      Hi to everyone A "real" WYSIWYG or HTML field that outputs good HTML code when accessed through the API would be excellent. I tried to use the rich text field, but the styling options are limited. For example, there are no heading tags (h1 to h6), and

    • Delete my store of Zoho commerce

      Hi Team, I want to delete my stores of commerce. Please help me asap. Looking for the positive response soon. Thanks Shubham Chauhan Mob: +91-9761872650

    • Delete Inactive Zoho Accounts - Access Cleanup_User Id: 60001640923

      As part of our Zoho access hygiene, we’ve reviewed and deactivated several inactive user accounts. These accounts have not been used in the past year and are no longer tied to active operations. All access rights have been revoked, and records retained

    • No feedback from Zoho Books regarding Yodlee feeds for Investec

      I reported on 6 Feb 2025 that the Bank Feeds for Investec bank via Yodlee are not working. To date there has been no resolution. You are charging us for a suite of products where the functionality does not work. Please advise how you will refund me for

    • Inappropriate Removal of Features

      It's the first with any software where I am experiencing that an existing customer has some of his exiting features removed. Moreover, new Zoho One users are paying less than us. Is this going to be Zoho's "normal behaviour". If so, it is going to be

    • Filtering report embeded in page

      Right now we are using a page variable to filter a report on a page. However, it being a page variable makes it reset if you go to another report/form, we want the filter to be kept the same. I tried using an application variable according to zoho you

    • Export Invoices to XML file

      Namaste! ZOHO suite of Apps is awesome and we as Partner, would like to use and implement the app´s from the Financial suite like ZOHO Invoice, but, in Portugal, we can only use certified Invoice Software and for this reason, we need to develop/customize on top of ZOHO Invoice to create an XML file with specific information and after this, go to the government and certified the software. As soon as we have for example, ZOHO CRM integrated with ZOHO Invoice up and running, our business opportunities

    • Zia summarizes your IM bot chats in real time

      Hello everyone! Let's say you’re an agent taking over a chat where a customer was booking a flight through a bot on WhatsApp. They've already shared key details, like their destination, travel date, and number of passengers. You need to understand the

    • Megamenu

      Finally! Megamenu's are now available in Zoho-Sites, after waiting for it and requesting it for years! BUT ... why am I asked to upgrade in order to use a megamenu? First: Zoho promised to always provide premium versions and options for all included Zoho-applications

    • Cancel and refund

      Hi, Yesterday I paid for the Zoho email subscription. Within a few minutes, I realized that the subscription counts one email ID as one user. To make another email ID for the same domain name, I'd have to pay for another user. I emailed right away to

    • One Contact with Multiple Accounts with Portal enabled

      I have a contact that manages different accounts, so he needs to see the invoices of all the companies he manage in Portal but I found it not possible.. any idea? I tried to set different customers with the same email contact with the portal enabled and

    • Next Page