What are article crawlability and indexability?

What are article crawlability and indexability?

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • 5名限定 課題解決型ワークショップイベント Zoho ワークアウト開催のお知らせ (9/25)

      ユーザーの皆さま、こんにちは。Zoho ユーザーコミュニティチームの藤澤です。 9月開催のZoho ワークアウトについてお知らせします。 今回はZoomにて、オンライン開催します。 ▷▷参加登録はこちら：https://us02web.zoom.us/meeting/register/6OSF2Bh6TumsMIlDwaY_PQ ━━━━━━━━━━━━━━━━━━━━━━━━ Zoho ワークアウトとは？ Zoho ユーザー同士で交流しながら、サービスに関する疑問や不明点の解消を目的とした「Zoho

    • hiding a topic from all but one segment (or list)

      My organization sends out a number of newsletters using Zoho Campaigns. One of those newsletters is for volunteers. In order to become a volunteer, a person has to first go through our volunteer orientation (training). After that, they can receive newsletters

    • How do I set up this automation correctly?

      When contacts enter my Subscribers list, I want it to reference a custom field to see if it is empty. Then I want it to do two things: If empty: Assign a tag based on a different custom field. If that custom field is empty, assign a different tag. If

    • Function #62: Display associated Quote on Invoice details screen

      Hello everyone! Today, we are sharing a Related List script that makes it easy to view and access the quote from which an invoice was created right from the invoice details screen. This Related List displays the Quote number and Amount, allowing you to

    • Custom confirmation message

      How can I change the message that users see after they submit the booking form? I have to confirm some details before their appointment is officially "confirmed", so I want to change it where it doesn't say their appointment is "confirmed" but rather

    • Has anyone integrated SMS well for Zoho Desk?

      Our company does property management and needs to be able to handle inbound sms messages which create a ticket for Zoho Desk. We then need to be able to reply back from Zoho desk which sends the user an sms message. This seems like a fairly common thing

    • Desk x CRM Integration

      Howdy! We currently use SalesIQ but we are considering moving across to Desk as it seems to have more functionality that we want. One of the pulls is the ability for our customers to self serve. But, I might be getting over excited and not actually need

    • Function #53: Transaction Level Profitability for Invoices

      Hello everyone, and welcome back to our series! We have previously provided custom functions for calculating the profitability of a quote and a sales order. There may be instances where the invoice may differ from its corresponding quote or sales order.

    • Issue with Save & Share Link – Works for Others but Not Creator

      I’ve enabled the "Save and Share" feature in Zoho Forms, and it works correctly for all users accessing the public link—except for me (the form creator). Issue Details: When I save and share the link, recipients see a “no permission” error. The form is

    • Save the pdf report from a registration form in a Folder

      Through Zoho Forms I created a registration form with quite a lot of rules to handle different variables (TEAM types, payment in installments, etc.), and the result is pretty good. The automatic PDF that is sent to the respondent upon submission is already

    • zoho forms integration to zoho work drive

      Scenario: A user fills out a Zoho Form, entering details such as email, mobile number, and other required information, and uploads supporting documents like PAN, Aadhaar, etc. Upon submission, the data is available in Zoho Forms Reports. Requirement:

    • Zoho Forms to Zoho CRM Integration failed - can I restart it?

      The integration of Zoho Form to Zoho CRM has failed. Is there a way for me to restart it for the impacted entry?

    • insert an equation, but in document, it shows an image not available.

      I'm new to Zoho writer and used Zoho writer in desktop app version . When I inserted an equation, I got an image not available sign. Any help is appreciated. Thanks, Charles

    • Zoho Desk Android app update - Swipe action customization

      Hello everyone! We have brought in support for the swipe action customization in latest version(v2.9.13) of the Zoho Desk Android app update, enabling you to configure left and right swipe gestures on tickets to carry out the actions swiftly without opening

    • Import KB template OR Export template for zoho desk?

      Greetings. Can you tell me if there is a way to get an EXPORT of my KB articles? OR is there a template you supply for importing KB articles into my zoho desk? I am looking for a method of understanding what fields can be imported, and what their possible

    • XML format to import knowledgebase into Zoho Desk

      Hi, We just started to use Zoho Desk and want to import our knowledgebase from our old support system (Freshdesk) to Zoho Desk. Can anyone give us information about the format of xml file to import? There is no explanation on the related page.

    • Zoho Desk integration with Power BI

      Hi, I want to be able to create a Power BI report which has live updates of ticket data from zoho desk, is this possile at all? Thanks Jack

    • Arranging Ticket Templates

      Is there any way to arrange our ticket templates? It doesn't look very organised when it's just arranged according to when they were created. We want the list to look more organised, by arranging/grouping them by topics, or even just by alphabetical

    • What they mean with "Portal"?

      I just downloaded this app to take my personal notes on digital art studio because it seemed to me the most comfortable app to do it, I use it very often, what do they mean by “no activity on your portal”? I have about 1 year of work here and I don't

    • Inactive User Auto Response

      We use Zoho One, and we have a couple employees that are no longer with us, but people are still attempting to email them. I'd like an autoresponder to let them no the person is no longer here, and how they can reach us going forward. I saw a similar

    • favorites no longer listed

      Hello, I can no longer navigate to my Favorites from the pulldown at the top of my grid view (see attached screenshot). I wasn't able to find it anywhere! Please help me display my Favorites. Thank you!

    • Notecards Disappeared

      Got a new phone. Downloaded and signed in Zoho Notebook. Not finding the Note cards. Just Empty Note books. Frustrating.

    • Table Errors

      Hello, I'm new to Zoho Notebook and using it on several Mac/Apple devices. My question is regarding tables: Why do they become broken, and how to fix the error without retyping all my data? (Please see the screenshot attached.)

    • Aggregating the First Value in the Group By of a dataset

      Hi I am trying to get the following Aggregate Formula to work in my chart, but cannot seem to get the right format. I have a series of data that I am running an include_groupby and want to SUM only a column in the first row of each group. So for example.

    • Online Calculator with Zoho Forms?

      I'd like to build a custom calculator on my website. Zoho Forms seems to offer all the fields for basic calculations performed live without pressing any "Calculate" buttons. Unfortunately, there is still that "Submit" button at the bottom. There is nothing

    • Zoho Tables instead of Zoho Creator Spreadsheet reports <3

      That would make my day for sure. Zoho Creator is create, but miss data entering as AirTable/Spreadsheet can. Seeing Zoho putting effort in this makes me think maybe one day we could see a similar interface for Zoho Creator spreadsheet reports. B.

    • Zoho Sheet - Desktop App or Offline

      Since Zoho Docs is now available as a desktop app and offline, when is a realistic ETA for Sheet to have the same functionality?I am surprised this was not laucned at the same time as Docs.

    • Notebook Stack

      Hi Everyone I Hope you´re fine, I´m sorry if this idea was posted before (I made a search but I haven´t find anything related). So, Having the possibility of stacking notebooks would be an interesting way to order notebooks that correspond to the same

    • Writing on sketch cards is bugged when zoomed in

      When zoomed in, it writes a noticeable distance above or to the side of where you're actually trying to write. The further you're zoomed in, the more noticeable it is. Zooming is also entirely absent on the desktop version.

    • Swipe between notes on iPhone

      It'd be convenient if I could move from one note to the next in a notebook simply by swiping left to right.

    • Sales IQ chat is not working in signed android apk

      I have integrated ZOHO sales IQ support chat and i have followed each step and its working fine in my development build but when i create signed APK for it. Chat does not work in it and showing awaiting for detail. I previously asked the same query but

    • COQL order by COUNT not working

      Dear community, I am trying to get a list of deal amounts per planner working on it and sorted to get see who has the least amount of deals. For some reason, I am unable to use sort by in combination with a COUNT. My original code was: query = "select

    • I want to duplicate a report and name it something else

      Hi, I have created a report, and now want to reproduce it and call it something else. so that I will end up with TWO separate reports with different titles. Please tell me how do I copy / reproduce a report please

    • Zoho CRM: Sales Rep Professional Certification Program on Coursera

      We are happy to share that we have published the Zoho Sales Representative Professional Certificate in partnership with Coursera, a leading platform for online learning and career development that offers access to courses and degrees from leading universities

    • OS X Notebook quits immediately upon launch

      NoteBook for OS X (Sequoia, but also under Sonoma) always quits immediately upon launch (so I cannot use the "Attach user log" option). I've restarted my MacBook but the problem persists. If it helps, attached is a diagnostic report from Library>Log

    • Can External users upload files or images to WorkDrive?

      I want to know if it is possible for someone externally through a link and PW be able to upload files and images onto WorkDrive?

    • Multi-line fields character limits

      Is there a way to set the character limit higher on multi-line fields so that we are not losing information pasted into the field? When the text is entered or pasted, there is no error to say that the text is too large. After saving and going back to view most of the text is gone.   Also, when viewing the resume, the text is not wrapped in the multi line fields and can t be read without scrolling across the page.

    • Taz bot not working — What should I do to resolve this issue?

      I am experiencing issues with the Taz bot in Zoho Cliq—not receiving responses or it does not seem to work as expected. Could you please explain why the Taz bot might not be functioning and what steps I should take to resolve this issue? Thank you!

    • Zoho Calendar soft bounce on @hotmail.com and @yahoo.com email addresses

      Hello, our Zoho calendar recently does not send the calendar invites to emails with hotmail and yahoo domains and comes back with a "soft bounce". other domains like Gmail works fine. Also sending "email" to the same emails to the above domains work well

    • OneNote Migration

      I am trying to migrate two notebooks from OneNote. For five days now I have had no notification that migration has completed and the migration page show 50% complete - one notebook completed one not finished. It just stays like this. I am unable to cancel

    • Next Page