Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Adding Taxes to paid consultations in Zoho Bookings

      I created a 'paid' consultation under Zoho Booking and integrated it with payment gateways for online/instant payment before a booking is done. How can I add 'taxes' to the price of consultation? I can add taxes to other Zoho apps (liks Books, Checkout,

    • Zoho CRM Reports Module on Mobil App

      I have the mobile app and the reports module doesn't appear in the sidebar for some reason. I saw a Youtube video where the user had the Reports module on mobile. Is there a setting to show it on mobile? Thanks.

    • Inquiry on Help Centre Tab Customisation

      Hi Zoho team, I’m wondering if it’s possible to further customise the Help Center tabs, specifically the descriptions under Knowledge Base, Community, and Tickets. While the current setup allows customising tab names, being able to tailor the descriptions

    • Passing a form object to a function

      Suppose I have a sort_order field in multiple tables and I want to increment it by +1 onCreate of a new record. Is there a way to pass the form object as an argument into the function to keep things DRY? The following function from Zia works, but I'd

    • Can you copy and paste a page within a form?

      I have been looking at trying to copy/duplicate a whole page within a form. I can't see how I can do this without having to add all the data in again. This is very time consuming. Any help would be great.

    • SMS to customers from within Bigin

      Hi All, Is there anyone else crying out for Bigin SMS capability to send an SMS to customers directly from the Bigin interface? We have inbuilt telephony already with call recordings which works well. What's lacking is the ability to send and receive

    • Unable to sort as Descending order

      Trying to change the sort order for a lookup field (checkboxes) from Ascending to Descending and keep getting an error in Deluge that the order must be Ascending Did anyone ran into this? Thanks Eyal

    • How do I copy an email message to one or more other folders?

      I can move a message to another folder using the "move to" option but I can't figure out how to copy a message. 

    • New feature request: Allow copy of email message to another folder

      Hello Zoho team, This is a suggested new feature to allow copy an email message to either another folder or the same folder. Within the same Zoho account. This is not a support request about "label". This is a suggested new feature to allow copies of

    • Hotmail is blocking the zoho mail IP

      Greetings, Since last Tuesday (5 days ago today) I wrote to Zoho support and I still haven't received a single response (Ticket ID: 2056917). Is this how you treat people who pay for your email service? I am making this public so that those who want to

    • Desk DMARC forwarding failure for some senders

      I am not receiving important emails into Desk, because of DMARC errors. Here's what's happening: 1. email is sent from customer e.g. john@doe.com, to my email address, e.g info@acme.com 2. email is delivered successfully to info@acme.com (a shared inbox

    • Creator Simplified #11: Create a custom button to download files

      Hey there! This time, let's see how to create custom buttons to download the files uploaded in the file upload field. Sample use-case A training app has a module that provides the materials for all the available courses. Students can click on the corresponding

    • Hide horizontal lines at bottom of form

      Hi I have a page with 3 forms embedded in html snippets. On each of the forms I can hide the submit button easily but I am left with the horizontal lines that bound that section and I cannot figure out to remove them (see below for the two sets of horizontal

    • Autosaving of Form Field

      HI can anyone suggest a way of autosaving or simulating an autosave function for a specific field in a form. The story is as follows. I have a bunch of technicians who write notes on how they do a repair. Sometimes they write for a couple of hours the troubleshooting process. If they click incorrectly, press ESC, Cancel or the browser crashes the information is lost which can be heartbreaking so my guys use notepad and then copy the information in. The only way around this is to click submit on the

    • Zoho Tracking Image location

      So we've been having an issue with tracking email opens. Specifically in Gmail. Our emails are not that long either, maybe 4 sections of image/250 characters of text/button per section.  But all my test accounts I used via Gmail we're showing opens. But then come to find out the tracking image is at the very bottom of the email. So If the message is clipped (It always just clips our social icons on the bottom) and the user doesn't click the show more button it never tracks the open.  Looking at other

    • AI Interview Insights: Turn Recorded Interviews into Quick Transcripts & Summaries

      Evaluating interviews shouldn’t require replaying long recordings or taking manual notes. With AI Interview Insights, you can now review complete transcripts and AI-generated summaries of your One-way (Recorded) interviews right inside Zoho Recruit. This

    • Employee Appraisal Applicability - Why is Date of Joining Hard-Coded?

      In the new (to me, at least) Performance Appraisal Cycle wizard, it's possible to set criteria to determine for whom the appraisal process should apply. This makes sense on its face. However, one MUST use the Date of Joining criterion as a filter. Why

    • Spotlight series #6: The Show app for Android TV has a new look!

      Hello everyone! We are delighted to introduce our revamped and redesigned Show app for Android TV.  Smart TVs are exploding in popularity. Android TV alone has over 110 million active monthly devices. Zoho Show, as part of a constant effort to improve

    • Ask the Experts 26: Brighten every customer interaction with Zoho Desk all year long

      Hello everyone, Greetings and welcome to Ask the Experts 26. As we wrap up 2025, we are excited to invite you to the 26th episode of our Ask the Expert series. 🎄The Merry Metrics Edition = Best of Zoho Desk [Best Practices + Holiday Automation + Year-End

    • Pricing calculator

      Alright, so I'm trying to create a pricing calculator/pricing report generator to use in a cabinet shop. I did all the logic and such, and the calculator works. I have a few other things that I would need it to be able to do though, I'll do my best to

    • Bug Details

      In the Bug detail the milestoneid is missing

    • Stop completed task lists from disappearing?

      Is there any way to stop projects from making tasks lists disappear when all the tasks in the list are completed? That's one of those little things where we're constantly fighting the product. For instance we have some projects which are ongoing - no start and end date. We use Kanban view to show the various task lists. We don't want the kanban list to disappear every time the items on it happen to be closed out. Thanks

    • Change of Blog Author

      Hi, I am creating the blog post on behalf of my colleague. When I publish the post, it is showing my name as author of the post which is not intended and needs to be changed to my colleague's name. How can I change the name of the author in the blogs?? Thanks, Ramanan

    • Allow customers to choose meeting venue and meeting duration on booking page

      My business primarily involves one-to-one meetings with my clients. Given the hybrid-work world we now find ourselves in, these meetings can take several forms (which I think of as the meeting "venue"): In-person Zoom Phone call I currently handle these

    • Booking outside of scheduled availability

      Is there a way for staff (such as the secretary) to book appointments outside of the scheduled availability? Right now to do this special hours must be set each time. There should be a quicker way. Am I missing something?

    • Operation Questions.

      Hello, I hope you are well. To explain a little, we are a company that sells services and products with a technical team responsible for installation and maintenance. Zoho FSM can be useful for the technical team, namely for the technical coordinator

    • Direct Access and Better Search for Zoho Quartz Recordings

      Hi Zoho Team, We would like to request a few enhancements to improve how Zoho Quartz recordings are accessed and managed after being submitted to Zoho Support. Current Limitation: After submitting a Quartz recording, the related Zoho Support ticket displays

    • Update date & time when a cell is edited

      Hi All, I am desiring to have a cell update with the current date and time when another cell is edited. Any ideas?  Thank you

    • Add "Groups" to "Share With" on Reports & All Entries

      Hi, On Forms we can share Publicly, with Specific Users And/Or Specific Groups or All Users. With Reports and All Entries we lack the "Groups" option, please add this as with many users this saves a lot of work. Thanks Dan

    • Add Pause / Resume Option to Zoho Quartz Recordings

      Hi Zoho Team, We would like to request an enhancement to Zoho Quartz recordings: the ability to pause and resume an ongoing recording. Current Limitation: At the moment, when recording an issue with Zoho Quartz, the recording continues even when we are

    • Custom Fields Not Showing Up in Invoice PDF Template Document Information

      I have added 2 custom fields under Sales > Invoices > Manage Custom Fields. They are set to show in all PDFs. However when I am editing my Invoice PDF template, i do not see the custom fields under Document Information.

    • Tracking Non-Inventory Items

      We have several business locations and currently use zoho inventory to track retail items (sales and purchase orders). We were hoping to use zoho inventory to track our non-inventory items as well (toilet paper, paper towels, etc). I understand that we

    • Can I use a Standalone CRM Function as the Callback URL For Async Export Data API?

      I am creating an export job using this API https://www.zoho.com/analytics/api/v2/bulk-api/export-data-async/create-export/view-id.html There is a "callbackUrl" key in the CONFIG object. I tried copying the URL for a standalone function in CRM which can

    • ZOHO Books

      Hi there, Why after I upgrade my Zoho invoice > Books then i wanted to add plugin which i cannot do. Please advise.

    • How is Your eCommerce Experience w/Zoho Inventory?

      First off, I'm SUPER grateful for the advent of Zoho Inventory and now the Zoho Commerce Suite. Overall, Inventory is a great product, especially for customers without an eCommerce presence. For eCommerce companies (especially those shipping more than ~10 packages/day), however, there are certain drawbacks that keep my clients from moving over to Zoho Inventory: Cons: 1. Invoice + Package Creation from Shopify/Other eCommerce Integrations: Zoho Inventory makes the somewhat perplexing decision to

    • Error AS101 when adding new email alias

      Hi, I am trying to add apple@(mydomain).com The error AS101 is shown while I try to add the alias.

    • Simplify Mass Replies with Predefined Templates and Snippets

      Hello everyone, We are happy to introduce a new enhancement to Mass Reply that helps agents respond to customers quickly and consistently. With the addition of Email Templates and Snippets in the reply window, agents can use predefined messages while

    • Mapping custom fields from one module to another

      I have a custom field, "Subscription Period" that appears as a required field in every Opportunity (Potential). I want that field to appear on any Quotes derived from that Opportunity (and have created a custom field of the same name in Quotes for that

    • Passing the CRM

      Hi, I am hoping someone can help. I have a zoho form that has a CRM lookup field. I was hoping to send this to my publicly to clients via a text message and the form then attaches the signed form back to the custom module. This work absolutely fine when

    • The Social Wall: November 2025

      We’re nearing the end of the year, and the holiday season is officially kicking in! It’s that time when sales peak and your social media game needs to be stronger than ever. We’re back with exciting new updates across AI, analytics, and the mobile app

    • Next Page