Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Getting Permission denied to access this portal.

      We have one user that can't login to projects even though access has been granted. This user can login to accounts.zoho.com but when login to https://projects.zoho.com/portals.do we get this error: Unauthorized login to this portal Permission denied to access this portal. Check your portal URL again. Sometimes we also get "server too busy". We have tried killing sessions (in accounts.zoho.com) and we have deleted cookies; and tried different computers and still the same problem. All others use can

    • Does Zoho Docs have a Line Number function ?

      Hi, when collaborating with coding tasks, I need an online real time share document that shows line numbers. Does Zoho's docs offer this feature ? If yes, how can I show them ? Regards, Frank

    • Setting Default Views for Custom, List and Detail Views

      Hey, Is it possible to set a default custom view, list view and detail view for a module for every user? We are onboarding a lot of non technical people that struggle with these things. Setting the views as default would really help. Btw: also setting

    • Custom function return type

      Hi, How do I create a custom deluge function in Zoho CRM that returns a string? e.g. Setup->Workflow->Custom Functions->Configure->Write own During create or edit of the function I don't see a way to change the default 'void' to anything else. Adding

    • Filter Based API request in Zoho Books using POSTMAN

      How do I GET only specified CONTACTS based on created time or modified time in Zoho Books using POSTMAN. In the api documentation, it is written we can apply filters but I need a sample request.

    • URL validation

      We use an internal intranet site which has a short DNS name which Zoho CRM will not accept.   When attempting to update the field it says "Please enter a valid URL". The URL I am trying to set is http://intranet/pm/ Our intranet is not currently setup with a full DNS name and given the amount of links using the shortname probably isn't a feasible change for us.

    • Has anyone been experiencing slow issues?

      Dear all, I just want to ask if anyone has been experiencing slow issues with Zoho Creator in the past two weeks? I worked with the ISP to improve network quality by changing routes and upgrading bandwidth, but nothing changed. I am in Vietnam.

    • Zoho Projects Roadshows 2025 - USA

      Dear Users, After an amazing response to our roadshows in 2024, we are excited to be back for the second year in a row! Join our team of experts as they walk you through the most-used features in Zoho Projects, explore powerful automation capabilities,

    • Billing Management: #6 Usage Billing in SaaS

      Imagine a customer shuffling across multiple subscriptions, a streaming service, a music app, cloud storage, and a design tool. Each one charges a flat monthly fee, regardless of how much or how little they use. Some months, the customer barely opens

    • Is there anyone who has been experiencing issues regarding the Zoho Creator Certification Website in the past 2 weeks?

      Dear all , I just wanted to ask is there anyone who was planning on taking the Zoho Creator Developer Certification Test in the past 2 weeks and have been facing errors stating that the website is under maintennance and also not allowed to access the

    • Directly Edit, Filter, and Sort Subforms on the Details Page

      Hello everyone, As you know, subforms allow you to associate multiple line items with a single record, greatly enhancing your data organization. For example, a sales order subform neatly lists all products, their quantities, amounts, and other relevant

    • Allow syncing Activities from other applications

      Marketing Automation could be a much more powerful platform if you were able to sync activities into the platform (e.g. purchase, donation, etc) outside of a user doing something on your website. I'd love it if you could sync Custom CRM Modules as activities,

    • Create static subforms in Zoho CRM: streamline data entry with pre-defined values

      Last modified on (9 July, 2025): This feature was available in early access and is currently being rolled out to customers in phases. Currently available for users in the the AU, CA, and SA DCs. It will be enabled for the remaining DCs in the next couple

    • Global Sets for Multi-Select pick lists

      When is this feature coming to Zoho CRM? It would be very useful now we have got used to having it for the normal pick lists.

    • Integración Books para cumplir la ley Crea y Crece y Ley Antifraude (VeriFactu)

      Hola: En principio, en julio de 2025, entra en vigor la ley Crea y Crece y Ley Antifraude (VeriFactu). ¿Sabéis si Zoho va a cumplir con la ley para cumplir con la facturación electrónica conectada a Hacienda? Gracias

    • Ask the Experts #1

      Hello everyone! It’s time to transform how you manage projects. Define the processes. Automate the tasks. Streamline the workflows. Let us dive into automation in Zoho Projects — from configuring workflows and custom functions to building triggers, using

    • How to overcome Zoho Deluge's time limit?

      I have built a function according to the following scheme: pages = {1,2,3,4,5,6,7,8,9,10}; for each page in pages { entriesPerPage = zoho.crm.getRecords("Accounts",page,200); for each entry in entriesPerPage { … } } Unfortunately, we have too many entries

    • Checking if Creator has Change History

      Like zForms - whenever an entry was updated there's an option to attached change history to email notif. Trigger -> Successful form submission

    • how to use validation rules in subform

      Is it possible to use validation rules for subforms? I tried the following code: entityMap = crmAPIRequest.toMap().get("record"); sum = 0; direct_billing = entityMap.get("direct_billing_details"); response = Map(); for each i in direct_billing { if(i.get("type")

    • Adding contact role to a specific deal js sdk malfunctioning

      i was trying to add the contact role to a specific deal contact but repeatedly i am getting this error: { "code": "SUCCESS", "details": { "statusMessage": { "code": "INVALID_DATA", "details": { "expected_data_type": "jsonobject" }, "message": "body",

    • Q3 Updates from Bigin!

      Hey Biginners, Hope you’re doing great! As we approach the end of 2025, we truly hope Bigin has been a part of helping you build your dream business this year! We've been busy working behind the scenes to bring you features that make running your business

    • New Series Announcement - Ecommerce Marketing Tips

      Running an online business is more than just having a website. It’s about getting the right customers to discover you, trust you, and keep coming back. To support your growth journey, we’re launching a weekly Marketing Tips series right here on Zoho Commerce

    • Client Script | Update - Introducing Subform Events and Actions

      Are you making the most of your subforms in Zoho CRM? Do you wish you could automate subform interactions and enhance user experience effortlessly? What if you had Client APIs and events specifically designed for subforms? We are thrilled to introduce

    • {"errors":[{"id":"500","title":"Servlet execution threw an exception"}]}

      Here's the call to move a file to trash. The resource_id is accurate and the file is present. header = Map(); header.put("Accept","application/vnd.api+json"); data = Map(); data_param1 = Map(); att_param1 = Map(); att_param1.put("status",51); data_param1.put("attributes",att_param1);

    • Converting Sales Order to Invoice via API; Problem with decimal places tax

      We are having problems converting a Sales Order to an Invoice via API Call. The cause of the issue is, that the Tax value in a Sales Order is sometimes calculated with up to 16 decimal places (e.g. 0.8730000000000001). The max decimal places allowed in

    • Zoho Canvas - Custom templates for related lists

      Hi, I see that the example pages load always one of our related lists in a custom template, but I dont know how to work with that:  1) How can i make my own custom templates for related lists?  2) Where and how can i check out existing custom templates?

    • Kaizen #147 - Frequently Asked Questions on Zoho CRM Widgets

      Heya! It's Kaizen time again, folks! This week, we aim to address common queries about Zoho CRM Widgets through frequently asked questions from our developer forum. Take a quick glance at these FAQs and learn from your peers' inquiries. 1. Where can I

    • open word file in zoho writer desktop version

      "How can I open a Microsoft Word (.doc or .docx) file in Zoho Writer if I only have the file saved on my computer and Zoho Writer doesn't appear as an option when I try 'Open with'? Is there a way to directly open the .doc file in Zoho Writer?"

    • Zoho PDF editor has a lot of issues.

      Zoho PDF editor needs a lot of work. It hangs and glitches a lot. Deletes annotations and clearings randomly.

    • Zohom mail

      Plz resolve the problem . I hope u understand .

    • Zoho sheet desktop version

      Hi Zoho team Where can I access desktop version of zoho sheets? It is important as web version is slow and requires one to be online all the time to do even basic work. If it is available, please guide me to the same.

    • ZOHO SHEETS

      Where can I access desktop version of zoho sheets? It is important to do basic work If it is available, please guide me to the same

    • Zoho Books - France

      L’équipe de Zoho France reçoit régulièrement des questions sur la conformité de ses applications de finances (Zoho Books/ Zoho Invoice) pour le marché français. Voici quelques points pour clarifier la question : Zoho Books est un logiciel de comptabilité

    • Using Zoho Flow to create sales orders from won deal in Zoho CRM

      Hi there, We are using Zoho Flow to create sales orders automatically when a deal is won in Zoho CRM. However, the sales order requires "Product Details" to be passed in "jsonobject", and is resulting in this error: Zoho CRM says "Invalid input for invalid

    • Is Zoho Sheet available for Linux ?

      Is Zoho Sheet available for Linux ?

    • Zoho pdf suit

      Pl. design products with following feature: 1. Please add all features given in Ilovepdf website to work on pdf files. It is mandatory to use pdf in court work. 2. Courts have prescribed New Times Roman, pl. add this font as well 3. Indexing, signature

    • Bharat

      a

    • how to disable staff selection Zoho Booking integrated to SalesIQ?

      currently there is only one Consultant in my Zoho Bookings like this I integrate Zoho Bookings into Zoho SalesIQ to create a chatbot. Unfortunately, even though I only have one consultant for a consultation, the user have to pick the consultant. It will

    • Zoho Bookings No Sync with Outlook

      Zoho Bookings appointments are showing on my Outlook Calendar but Outlook events are not showing on Zoho Bookings. How do I fix this?

    • End Date in Zoho Bookings

      When I give my appointments a 30 minutes time I would expect the software not to even show the End Time.  But it actually makes the user pick an End Time.  Did I just miss a setting?  

    • Next Page