What are article crawlability and indexability?

What are article crawlability and indexability?

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • How to retrieve my following requests on this forum?

      Sorry, but I did not find the proper subforum for this question.

    • How to list emails in a folder, e.g. Inbox, on multiple pages when using Zoho mail webpage?

      Something as shown in the figure. There are totally 50 emails in Sent folder. If "Mail per page" equals 20, then the Sent folder is split into 3 pages. When I wander through Sent folder, I can just select a specific page to jump to. BTW, it seems that

    • Unable to Create Zoho Booking via the Book Appointment API

      Its giving the below error {     "response": {         "errormessage": "Error setting value for the variable:customer_details\n null",         "status": "Error"     } } Request: POST Url: https://www.zohoapis.in/bookings/v1/json/appointment attached Zoho-oauthtoken

    • SHEET - Send email when a cell changes

      I would like to create a custom function for Zoho Sheet that triggers when a paticular cell changes to a specific value. This would result in sending an email to a recipient (this would be an address that remains the same and included in the script). Example: = IF(N4= "Drafted", <>EmailFunction) 1)     Cell N4 changes to "Drafted" 2)    Email is sent to recipient            or alternatively 3)    Post to chat channel I have found the Custom function editor in Sheet. I am not bad at scripting, but

    • 【開催報告】 福岡 ユーザー交流会 2025/8/8（金）

      皆さま、こんにちは。コミュニティチームの中野です。 8/8（金）に、福岡 ユーザー交流会を開催しました。 本投稿では、その様子をお届けします。当日の登壇資料などもこちらに共有しますので、参加できなかった皆さまもご参照ください。 今年初の開催となる福岡 ユーザー交流会では、CreativeStudio樂合同会社 前田さんによるZoho CRM / Sign / Survey の事例セッションのほか、 Zoho社員セッションでは、Zoho Forms の活用法を解説。 さらに、「見込み客・顧客データの管理/活用方法」をテーマに参加者同士でZoho

    • no me llegan los correos a Zoho mail

      No puedo recibir correos pero sí enviarlos, ya hice la modificación de MX y la verificación de teléfonos, qué es lo que ocurre? gracias

    • Error: Invalid login: 535 Authentication Failed

      I have used zoho with nodemailer. const transporter = nodemailer.createTransport({ host: 'smtp.zoho.com', port: 465, secure: true, auth: { user: 'example@example.com', pass: 'password' } }); While sending the mail, it shows the following error: Error:

    • Zoho Renewal

      Hello, If I am not going for zoho email renewal. will i get back my free zoho account? and if yes then is it possible to get back my all free user. how many user get back 10 or 25?

    • javax.mail.authenticationfailedexception 535 authentication failed

      Hi, I am facing 535 authentication failed error when trying to send email from zoho desktop as well as in webmail. Can you suggest to fix this issue,. Regards, Rekha

    • Not reciving emails

      Apparently i cannot recive emails on my adress contact@sportperformance.ro I can send, but do not recive. The mail i'm trying to send from mybother adress gets sent and doesn't bounce back... but still doesn't get in my inbox. Please advise

    • Not receiving MailChimp verification e-mail

      It seems that their verification e-mails are blocked. I can receive their other e-mails, but not their verification of domain ownership e-mail. I've checked and double checked how I typed the e-mail, using different e-mails (my personal e-mail can receive it), white listing the domain and all that is left is for the IP's to be white listed, but I don't have that power.  If a staff member could take a look at this -> http://mailchimp.com/about/ips/ And perhaps white list them for me, that would be

    • Creating my 2nd email account

      After creating my first email address, I decided to get another email address. I would like to use this new address as the primary address too. I don't know how to set it up there doesn't seem to be an option for that

    • Cannot - create more email account - Unusual activity detected from this IP. Please try again after some time

      Hello, I come across the error message in Control Panel. Unusual activity detected from this IP. Please try again after some time and i cannot create any more users We are an IT company and we provide service for another company Please unlock us.

    • "Unable to send message;Reason:553 Relaying disallowed. Invalid Domain"

      Good day. When I try to send mail through ZOHO mail I get the following error : "Unable to send message;Reason:553 Relaying disallowed. Invalid Domain" I need help with this. My zohomail is : @eclipseweb.site Thank you,

    • Transfert de domaine pour création des comptes emails

      Bonjour , je ne parviens point à créer des mails avec le domaine 'raeses.org' suite à la souscription du domaine auprès d'un autre hébergeur, dont j'ai fait la demande du code de transfert qui est le suivant : J2[U8-l0]p8[ En somme, attente de l'activation

    • Help! Unable to send message;Reason:554 5.1.8 Email Outgoing Blocked.

      Kindly help me resolved this issue that i am facing here.

    • How are people handling estimates with Zoho inventory?

      We are often using Zoho Books for estimates that then get converted to invoices within Books. We would like the sales team to migrate entirely to Zoho Inventory and no longer need to use Zoho Books so that they are only on one system. How are people managing

    • Relative Date Searches

      Currently in the search options, it has "date", "from date" and "to date". I think it would be great if there were options like "date greater than x days ago" and "date less than x days ago". I realise that as a once off you can just use the existing

    • Performance is degrading

      We have used Mail and Cliq for about three years now. I used to use both on the browser. Both have, over the past 6 months, had a severe degradation in performance. I switched to desktop email, which appeared to improve things somewhat, although initial

    • Ask the Experts 23: Customize, utilize, and personalize with Zoho Desk

      Hello everyone! It's time for the next round of Ask the Experts (ATE). This month is all about giving Zoho Desk a complete makeover and making it truly yours. Rebrand Zoho Desk with your organization’s details, customize ticket settings based on your

    • Dear Zoho CEO: Business Growth is about how you prioritise!

      All of us in business know that when you get your priorities right, your business grows. Zoho CRM and Zoho Books are excellent products, but sadly, Zoho Inventory continues to lag behind. Just this morning, I received yet another one-sided email about

    • Payroll In Canada

      Hi, When can we expect to have payroll in Canada with books 

    • Please review and re-enable outgoing emails for my domain

      Hello Zoho Support, I have recently purchased a new domain and set up email hosting with Zoho. However, my account shows "Outgoing Email Blocked". I am a genuine user and not sending bulk/spam emails. Please review and re-enable outgoing emails for my

    • Payroll without tax integrations (i.e. payroll for international)

      It seems as though Zoho waits to develop integrations with local tax authorities before offering Zoho Payroll to Zoho customers in a country. Please reconsider this approach. We are happy Zoho Books customers, but unhappy that we have to run payroll in

    • goingout e mail block

      info@ozanrade.com.tr

    • Incoming mails blocked

      Zoho User ID : 60005368884 My mail Id is marketing#axisformingtechnology.com .I am getting following message "Your Incoming has been blocked and the emails will not be fetched in your Zoho account and POP Accounts. Click here to get unblocked." Please

    • Assistance Needed: Ticket Status Not Updating and Sorting by Last Customer Reply in Zoho Desk

      Hello, I’m facing two issues in Zoho Desk that I’d like your guidance on: Ticket Status Not Updating: When a customer replies to a ticket, the status does not change to Reopened. Instead, it remains in Waiting on Customer, even after the customer’s response

    • Configuring Email Notifications with Tautulli for Plex

      Hi I'm new to Zoho. I am from Canada and I have a I use a web based application called Tautulli for Plex that monitors my Plex media server. It also sends a newsletter to my followers. To set this up they require a "From" email address., a smtp server

    • Is there a way to automatically add Secondary Contacts (CCs) when creating a new ticket for specific customers?

      Some of our customers want multiple contacts to receive all notifications from our support team. Is there a way to automatically add secondary contacts to a ticket when our support team opens a new ticket and associates it with an account? This would

    • How to Set Up Zoho Mail Without Cloudflare on My Website

      I'm having some trouble with Cloudflare here in Pakistan. I want to configure Zoho Mail for my domain, but I'm not sure how to set it up without going through Cloudflare. My website, https://getcrunchyrollapk.com/ , is currently using CF, but I'd like

    • Spam is Being Forwarded

      I am filtering a certain sender directly to the Trash folder. Those messages are still being forwarded. Is this supposed to happen?

    • IMAP Block

      My two accounts have been blocked and I am not able to unblocked them myself. Please respond to email, I am traveling and this is urgent.

    • "DKIM not configured"

      Hello. I have been attempting get the DKIM verified but Toolkit keeps sending the message that it is not configured, but both Namecheap and Zoho show it as configured properly. What am I missing?

    • Zoho mail with custom domain not receiving email

      i registered zoho mail with my own domain. I can login and access the mail app. I tried to send email from an outlook email account and an icloud email account. Both emails were not received. My plan is free. I also tried to send email from this zoho

    • Set connection link name from variable in invokeurl

      Hi, guys. How to set in parameter "connection" a variable, instead of a string. connectionLinkName = manager.get('connectionLinkName').toString(); response = invokeurl [ url :"https://www.googleapis.com/calendar/v3/freeBusy" type :POST parameters:requestParams.toString()

    • Waterfall Chart

      Hello, I would like to create a waterfall chart on Zoho analytics which shows the movement in changes of budget throughout a few months, on a weekly basis. Should look something like the picture below. Does anyone know how to?

    • Help Center and SEO: Any Benefit to My Domain-Mapped Website Ranking?

      First of, I love the Help Center which I've just decided to integrate into my website to replace its old-fashioned FAQs. So much more to achieve there now! Lots of new benefits to the site visitors and to me in terms of organizing and delivering all the

    • Issue with Importing Notes against Leads

      Hi, I am attempting to import some Notes into Zoho CRM which need to be assigned to various Leads. I have created a csv file which I am testing that contains just one record. This csv file contains the following columns: Note Id Parent Id       (I have

    • Trigger a Workflow Function if an Attachment (Related List) has been added

      Hello, I have a Case Module with a related list which is Attachment. I want to trigger a workflow if I added an attachment. I've seen some topics about this in zoho community that was posted few months ago and based on the answers, there is no trigger

    • Option to Hide Project Overview for Client Profiles

      In Zoho Projects, the Project Overview section is currently visible to client profiles by default. While user profiles already have the option to restrict or hide access to the project overview, the same flexibility isn’t available for client profiles.

    • Next Page