Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Delete button

      Hi, The delete button were hide into the three dot button. Can I display outside? why Zoho make this update?

    • Is there a way to show contact emails in the Account?

      I know I can see the emails I have sent and received on a Contact detail view, but I want to be able to see all the emails that have been sent and received between all an Accounts Contacts on the Account Detail view. That way when I see the Account detail

    • Invalid scope choice: Workdrive integration in CRM

      Bug: There is an invalid option in the permission choices for Workdrive integration in CRM. If the entry "WorkDrive.teamfolder.CREATE" is selected, it will return a message indicating invalid OAuth scope scope does not exist.

    • Dark Mode - Font Colors Don't Work

      When editing a document in Dark Mode and selecting font colors, they don't show up on screen.  Viewing/editing the same document in Light Mode shows them just fine.

    • Good news! Calendar in Zoho CRM gets a face lift

      Dear Customers, We are delighted to unveil the revamped calendar UI in Zoho CRM. With a complete visual overhaul aligned with CRM for Everyone, the calendar now offers a more intuitive and flexible scheduling experience. What’s new? Distinguish activities

    • Use URL field to populate Image in a Email Template

      We have two records Record 1: Appointments (with a Lookup to Doctors module) Related Record 2: Doctors (with a URL field that is an image of the doctor) We would like to populate a CRM email template with the image of the URL field for the related doctor

    • Can Zoho Marketing Automation send OTP codes via WhatsApp during user registration?

      The flow I would like to implement is: A user signs up in the mobile app and enters their phone number. The system generates a one-time verification code (OTP). The OTP is sent to the user via WhatsApp message. The user enters the code in the app to confirm

    • Zoho CRM WooCommerce Integrtation

      Looking for a solution for my 50 client to show detailed analytics on source of traffic that led to events, leads, conversions. Hi all, I have not been able to find a workable guide or solution to how exactly step by step integrate the Zoho CRM with WooCommerce

    • Attaching files to emails within CRM Deals.

      Hello, We have recently started using the extension "Workdrive for CRM" (Related List) to view/store our documents for each Deal, instead of using Attachments. Overall it feels like a better way to go but the user experience is not so great when it comes

    • Service line items

      Hello Latha, Could you please let me know the maximum number of service line items that can be added to a single work order? Thanks, Chethiya.

    • Marking a meeting 'done'.

      I would like to somehow mark a meeting 'done' and placed under the contact's page rather than deleting it and having no record of it. Am I missing a button that does this?

    • 500 internal server error on opening an iframe through a deluge script

      Hi Team I am trying to open an external url in an iframe through my deluge script which is associated to a custom button . By doing this i am getting an "Internal Server Error" . Please can you help me with this.

    • Kaizen #225 - Making Query-based Custom Related Lists Actionable with Lookups and Links

      Hello everyone! Welcome back to another post in the Kaizen series! This week, we will discuss an exciting enhancement in Queries in Zoho CRM. In Kaizen #190, we discussed how Queries bridge gaps where native related lists fall short and power custom related

    • Help Center and SEO: Any Benefit to My Domain-Mapped Website Ranking?

      First of, I love the Help Center which I've just decided to integrate into my website to replace its old-fashioned FAQs. So much more to achieve there now! Lots of new benefits to the site visitors and to me in terms of organizing and delivering all the

    • No Need To Fix Something That Is Working

      Zoho Books is a great financial tool which helps businesses to become more efficient and productive with day-to-day operations. As such, every change, upgrade, improvement needs to be carefully thought before implemented in the software and I'm sure Zoho

    • Filter button in the Zoho Sheet Android App doesn't toggle on

      I am a new Zoho Sheets user and experiencing a specific issue with the filter functionality in the Android mobile application. Detailed Issue Description: The filter icon appears correctly in the toolbar. Upon tapping the filter icon/button, the toggle

    • Is Zoho Sites still actively being developed?

      Hello, Is Zoho Sites still actively being developed as part of the Zoho ecosystem? I noticed that the What's New page (https://www.zoho.com/sites/whats-new.html) does not show any updates since Q1 2025. We were considering migrating our website from Squarespace

    • Project Notifcatiion Emails - Milestone

      Hello: I cannot get myself, or most importantly my portal client user to recieve an email upon completion of a milestone. I have set up our 1st project. I have set up a test client user. (accepted the invitation and is listed in the system as a client

    • Zoho Forms - Feature Request - Year Field

      Hi Zoho Forms Team, You currently have the following date and time fields: Date Time Date and Time Year and Month It would be useful if you could include a "Year" field For example a recent application I completed said "What year was your house built?"

    • Integrate Excel or Zoho Sheet functions / calculations to CRM product module

      Hello Community, I hope someone more experienced can help me with this question. Our price / payment plan calculations are in an Excel spreadsheet and I would like to use all those functions / calculations in my Products module. So when we send a quote

    • Sync CRM Contacts to USER'S contacts on Office 365

      I can see that the O365 sync is transferring contacts backwards and forwards between Zoho CRM and Office365. But it has created a separate address book in Office 365 called "Zoho CRM Contacts". This address book is not used by Office/Outlook's email function

    • How can I link Products in a Deal Subform to the Products Module

      Hello, I have a pricing subform on our Deals page and use a lookup field to associate a product with each line. I want to be able to look at a product page within the Products module and see a list of the deals connected to that product. I have this working

    • Manage Every Customer Conversation from Every Channel inside Zoho SalesIQ

      Your customers message you from everywhere. But are you really able to track, manage, and follow through on every conversation, without missing anything? With interactions coming in from websites, mobile apps, and messaging platforms like WhatsApp and

    • Zoho Books | Product updates | February 2026

      Hello users, We’ve rolled out new features and enhancements in Zoho Books. From Advanced Reporting Tags to the ability to mark projects as completed, explore the latest updates designed to improve your bookkeeping experience. Introducing Advanced Reporting

    • Update latitude & longitude address field API

      How do I update the coordinates of an address field from a widget? I can't modify the latitude and longitude of the address field. I think the problem is how I'm writing formdata variable. zoho_init.then(function (data) { var queryParams = ZOHO.CREATOR.UTIL.getQueryParams();

    • Zoho Forms - Feature Request - Past Into Scanning/OCR Field

      Hi Zoho Forms Team, You recently introduced the OCR/Scanning field which I have found great use for with one client who receives work orders as a screenshot from one customer. I want to raise a feature request here which would make that field even more

    • inability to use different primary address on invoice per location

      my company operates in two different locations with different email address. The problems then is the inability to edit the primary to suite the invoice for the second location.

    • Use Zoho Creator as a source for merge templates in Zoho Writer

      Hello all! We're excited to share that we've enhanced Zoho Creator's integration with Zoho Writer to make this combination even more powerful. You can now use Zoho Creator as a data source for mail merge templates in Zoho Writer. Making more data from

    • Anyone in Australia using Zoho Books AND has their account with NAB?

      Hi I have an account with both NAB and Suncorp. Suncorp transaction come in the next day however NAB transactions take 4-5 business days to appear. eg: A deposit made today in my Suncorp will be imported into Zoho tomorrow. A deposit made today to the NAB account will be imported maybe Saturday (Friday overnight). I have contacted both Zoho and NAB but noone seems to know why. I was just wondering if anyone else in Australia uses NAB and has this issue (or doesn't) maybe we could compare notes and

    • Detailed Balance Sheet for tax preparer

      I'm using the free edition of Zoho Books. My tax preparer is asking for "detailed" Profit & Loss and Balance Sheet reports which include all the activity and transactions within the various categories. The default reports do not include these details.

    • On Edit Validation Blueprint

      Hello, I have a notes field and a signature field. When the Approve button is clicked, the Signature field will appear and must be filled in. When the Reject button is clicked, the Notes field will appear and must be filled in. Question: Blueprint will

    • Zoho Projects - Cloning a task does not trigger task workflow when created

      Hello! I have a Project where my team uses a set of tasks from a tasklist as templates, so we could simply clone it and drag it to another list in kanban view to avoid creating a new one from scratch. The process works well, but after cloning it the new

    • Host Group Appointments Online in Zoho Bookings

      Greetings from the Zoho Bookings team! We’re excited to announce a new enhancement to Group Booking that makes hosting online group events smoother and more professional than ever. You can now conduct online group events with auto-generated meeting links

    • Can't rename groups on Mac desktop app

      I'm working on an up-to-date Mac with a freshly downloaded Notebook app. I'm trying to rename a group within a notebook. Here I have, left to right, a note, a group, and a note. I select the group. On the top left, I select Action. On the dropdown, "Rename"

    • Workdrive Collaboration with an External User

      I would like to know if I can setup a collaboration space with an external user in workdrive or do I need to add them as a user on my system? If I need to add them, can I add them on Workdrive only and give limit access to our space only?

    • FSM integration with Books

      Hi, I have spent a few months working with FSM and have come across a critical gap in the functionality, which I find almost shocking....either that, or I am an idiot. The lack of bi-directional sync between Books and FSM on Sales Orders/ Work Orders

    • Marketing Tip #23: Help customers with how-to guides and usage tips

      Customers don’t stop needing you after they place an order. Helping customers use your product correctly and confidently can improve satisfaction, reduce returns, and increase repeat purchases. Sharing simple how-to guides, usage tips, or care instructions

    • Powering Customer Support with our women

      In Zoho Desk support, women make up 50% of our team. We see this as one of our strengths, reflecting the spirit of this year’s theme, "Give to Gain". Our women find their balance Women carry many responsibilities — they represent frontline support, lead

    • Function #25: Automatically generate purchase orders from a sales order

      We kicked off the "Function Fridays" series with the goal of helping you automate your everyday accounting tasks. As we delve into today's post, I'm delighted to announce that we're here to present the 25th custom function in this series. While it is

    • Looking for Guidance on Building a Zoho Website

      I'm exploring the possibility of building a custom website with specific features using Zoho as an alternative platform. My goal is to create something similar to https://gtasandresapk.com , with the same kind of functionality and user experience. I'd

    • Next Page