What are article crawlability and indexability?

What are article crawlability and indexability?

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Issue showing too many consultations in my workspace link.

      Hi Team, I’ve set up two Workspaces to track meetings from different sources. So far, this has been working well, and the two Workspaces are differentiated without any issues. However, when I navigate to Consultations and share the link to my personal

    • 👋 Welcome to the Zoho MCP Community

      Hello all, glad to have you here! This is your space for everything AI agents, MCP tools, and intelligent business apps. This community is for you — developers, partners, creators, and businesses exploring how agents can transform work. Whether you’re

    • CRM Validation Rules Support Only Single Condition

      Simply put, CRM validation rules support only a single condition for each field on "All Records". You also cannot specify additional validation rules on the same field because it has already been used in an existing validation rule. The ONLY solution

    • Unapproved Leaves are hard to distinguish in Attendance View

      This is a an unapproved leave request It appears in the Attendance view without any visual indicator if its approved or not For a whole day request this might be manageable but for hourly requests it gets very hard to know which are approved, which are

    • Performance Appraisal Probation Period

      Hello All,  Is there any possible way to create an appraisal cycle for new staff members, at the end of probation period? Many thanks!

    • Zoho Creatorの一括操作における処理の同期/非同期について

      現在、Creatorのレポート機能を利用して、複数のレコードに対して一括で処理を実行しようとしていますが、処理の実行順序について確認したいことがあります。 レポート内の複数レコードに一括で処理を実行した際、処理は同期的に行われるのでしょうか？それとも非同期的に行われるのでしょうか？ 【同期処理の場合】 レコード①に対する処理が開始され、終了後にレコード②に対する処理が開始され、最後にレコード③に対する処理が実行されるように、処理が順番に行われる場合。 【非同期処理の場合】 レコード①、レコード②、レコード③の処理が一斉に開始され、それぞれ並行して処理が行われ、全処理が終了する場合。

    • Adding Chargebee as a Data Connector

      Is it possible to get Chargebee added as a Zoho Analytics data connector?

    • Delete commerce website

      I need to delete a commerce website, but the only option is to click on settings, REQUEST DELETE, choose an urgency notice, add a message....AND THEN nothing, no way to send the request. Why is nothing simple!?!?!  I just want to delete the store.  The

    • Adding external users to Zoho Social under Zoho ONE licence - how to best achieve this

      My client has a small business, and we are looking to implementing Zoho ONE with a single flexible user licence as that is all they really need and offers the best pricing for the range of modules we eventually wish to set them up with, one of which will

    • Has anyone built a custom AI support agent inside Zoho (SalesIQ/Zobot)?

      Hi all, I’ve been experimenting with building my own AI support assistant and wanted to see if anyone here has tackled something similar within Zoho. Right now, I’ve set up a Retrieval-Augmented Generation (RAG) pipeline outside of Zoho using FAISS. It

    • This mobile number has been marked spam. Please contact support.

      Problem Description: One of our sales agents in our organization is unable to sign in to Zoho Mail. When attempting to log in, the following message appears: This mobile number has been marked as spam. Please contact support at as@zohocorp.com @zohocorp

    • What’s New in Zoho Inventory | April 2025

      Hello users, April has been a big month in Zoho Inventory! We’ve rolled out powerful new features to help you streamline production, optimise stock management, and tailor your workflows. While several updates bring helpful enhancements, three major additions

    • When Zoho Tables Beta will be open to EU data center

      Hello all, We in EU are looking at you all using and testing and are getting jealous :) When we will be able to get into the beta also? We don't mind testing and playing with beta software. Thank you!

    • Pass current date to a field using Zoho Flow

      I am trying to generate an invoice automatically once somebody submits a record in Zoho CRM. I get an error in the invoice date. I have entered {{zoho.currentdate}} in the Date field. When I test the flow, I get "Zoho Books says "Invalid value passed

    • API: Mark Sales Order as Open + Custom Status

      Hi, it's possible to create Custom Status (sub-status actually) states for the Sales Order. So you have Open, Void. Then under Open you can have Open, and create one called Order Paid, Order Shipped, etc etc...which is grouped under Open. I can use the

    • Multi-Unit Inventory with Flexible Unit Selection (Purchase in One Unit, Sell in Another)

      We need multi-unit inventory management in Zoho Books with the flexibility to choose units (e.g., Box or Piece) at the time of purchase or sale. For example, if 1 Box = 10 Pieces, we should be able to record purchases in Boxes but sell either in Boxes

    • Zoho Quartz Screen Recording

      Hello, can we get access to Quartz, please, as a standalone solution? It would be great for creating training videos for current and future staff on how to use Zoho software according to our company requirements. Thank you

    • Tip 26: How to hide the "Submit" button from a form

      Hi everyone, Hope you're staying safe and working from home. We are, too. By now, we at Zoho are all very much accustomed to the new normal—working remotely. Today, we're back with yet another simple but interesting tip--how to hide the Submit button from your forms. In certain scenarios, you may want to hide the submit button from a form until all the fields are filled in.  Use case In this tip, we'll show you how to hide the Submit button while the user is entering data into the form, and then

    • filter broke my data

      I uploaded a file recently from Sheets and it has top 2 rows frozen, with table headers in second row and each one is filterable. somehow my first 2 columns became unfiltered and no matter what I do I cannot reapply the filter?? also didn't realize they

    • How do I move Notes around within a Group?

      It says here: " You can now sort notes by title (alphabetically), or by date modified and date created. You can even organize your notes by dragging and dropping them into a particular order. To sort your notes, simply go to Settings and tap “Sort By.” Please note: all sort settings will be saved and synced across devices, except for custom sorting. Custom sorting will be device specific."However, I am unable to 'custom sort' in either Notebook for Mac or on the Web. In addition, I can't find the

    • Zoho Sign: need to leave document pending for up to a year, or maybe there's a better way?

      I have zoho one, maybe there's a better way to do this with another service than sending a zoho sign template from zoho crm. At the end of the day this requirement is due to regulations, no matter how dumb it may seem. I'm just looking for a way of getting

    • 'Add Tax To Amount' not reflected in Invoice

      Hi Zoho Support, I'm experiencing an issue with tax calculation display in my invoice template. Despite having "Add tax to amount" box checked in the template settings, the Amount column is not showing the tax-inclusive total for line items. Current behaviour:

    • Separate Items & Services

      Hi, please separate items and services into different categories. Thank you

    • What's New in Zoho Inventory | Q2 2025

      Hello Customers, The second quarter have been exciting months for Zoho Inventory! We’ve introduced impactful new features and enhancements to help you manage inventory operations with even greater precision and control. While we have many more exciting

    • Mastering Zia Match Scores | Let's Talk Recruit

      Feeling overwhelmed by hundreds of resumes for every job? You’re not alone! Welcome back to Let’s Talk Recruit, where we break down Zoho Recruit’s features and hiring best practices into simple, actionable insights for recruiters. Imagine having an assistant

    • We are unable to process your request now. Please try again after sometime or contact support@zohoaccounts.com

      I cannot sign up and return the error of we are unable to process your request now. Please try again after sometime or contact support@zohoaccounts.com

    • Multi-currency - What's cooking ?

      Hi,       We have been doing this feature for sometime and we would like to give you some glimpses of it.  Working with Multi Currency :        Multicurrency support gives you the ability to handle business transactions in multiple currencies. You can define a base currency for your organization and add more currencies with exchange rates based on the base currency.  Setup :        From the setup page, you can manage all the currencies supported by your organization.       Currencies page        

    • Integrating Chatbot with Zoho Creator Application

      Is it possible to integrate a chatbot with a Zoho Creator application?

    • How to reduce programmatically the image uploaded by user?

      I need a function that will automatically reduce the pixel dimension to 800 x 600 pixels / 180 resolution or (approx. 1.37MB) of image uploaded by user from digital camera, for example, 2271 x 1704 pixels /180 resolution or approx. 11.1MB. After the user selected the image, the function will able to detect if pixels is above 800x600, process the photo (crop/ reduce) and resume upload. Need help...  

    • Dark mode for Zoho Creator / Zoho CRM Code editor

      Hi Team, Is there any plans for Dark mode in Zoho creator / Zoho Crm code editor and development pages in pipeline?

    • Is there a way to make a button scroll down?

      Looking to have a button on a landing page scroll down to another section on the page. Any recomendations outside of coding?

    • Collective-booking event not added to all staff calendars

      We assign two staff to certain events. When the client books this event, it adds it to one staff calendar (the 'organiser') but not the other. How can I ensure all staff assigned to a collective booking get the event in their calendar? (A side note: it

    • ZOHO Android Client

      Hi, I installed the Android app, but it had an issue, so I reinstalled it. I was able to add multiple accounts, but now when I add the next account, it just duplicates the one I already have and will not even allow me to enter the info for another account.

    • I'd like to suggest a feature enhancement for SalesIQ that would greatly improve the user experience across different channels.

      Hello Zoho Team, Current Limitation: When I enable the pre-chat form under Brands > Flow Controls to collect the visitor’s name and email, it gets applied globally across all channels, including WhatsApp, Messenger, and Instagram. This doesn't quite align

    • Enhance Barcode/QR Code scanner with bulk scanning or continuously scanning

      Dear Zoho Creator, As we all know, after each scan, the scanning frame closes. Imagine having 100 items; we would need to tap 100 times and wait roughly 1 second each time for the scanning frame to reopen on mobile web. It's not just about wasting time;

    • Non-depreciating fixed asset

      Hi! There are non-depreciable fixed assets (e.g. land). It would be very useful to be able to create a new type of fixed asset (within the fixed assets module) with a ‘No depreciation’ depreciation method. There is always the option of recording land

    • Managing Rental Sales in Zoho Inventory

      I am aware that Zoho Inventory is not yet set up to handle rental sales and invoicing. Is anyone using it for rentals anyway? I'd like to hear about how others have found work arounds to manage inventory of rental equipment, rental payments, etc. Th

    • Megamenu

      Finally! Megamenu's are now available in Zoho-Sites, after waiting for it and requesting it for years! BUT ... why am I asked to upgrade in order to use a megamenu? First: Zoho promised to always provide premium versions and options for all included Zoho-applications

    • Cannot access KB within Help Center

      Im working with my boss to customize our knowledge base, but for some reason I can see the KB tab, and see the KB categories, but I cannot access the articles within the KB. We have been troubleshooting for weeks, and we have all permissions set up, customers

    • Zoho Flow to Creator 3001 Respoonse

      I have updated my Flows with the new V2 connection to Zoho Creator, but now some Flows do not work. They take in data from a Webhook and are supposed to create a record in Creator, however creator returns a 3001 message along with a failure, but I cannot

    • Next Page