Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere







                          Zoho Developer Community






                                                • Desk Community Learning Series


                                                • Digest


                                                • Functions


                                                • Meetups


                                                • Kbase


                                                • Resources


                                                • Glossary


                                                • Desk Marketplace


                                                • MVP Corner


                                                • Word of the Day


                                                • Ask the Experts



                                                          • Sticky Posts

                                                          • Register for Zoho Desk Beta Community

                                                            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                          • Share your Zoho Desk story with us!

                                                            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                          • Tip #1: Learn to pick the right channels

                                                            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                          • Welcome to Zoho Desk Community - Say hello here!

                                                            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                          • Webinar 1: Blueprint for Customer Service

                                                            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                          Manage your brands on social media



                                                                Zoho TeamInbox Resources



                                                                    Zoho CRM Plus Resources

                                                                      Zoho Books Resources


                                                                        Zoho Subscriptions Resources

                                                                          Zoho Projects Resources


                                                                            Zoho Sprints Resources


                                                                              Qntrl Resources


                                                                                Zoho Creator Resources



                                                                                    Zoho CRM Resources

                                                                                    • CRM Community Learning Series

                                                                                      CRM Community Learning Series


                                                                                    • Kaizen

                                                                                      Kaizen

                                                                                    • Functions

                                                                                      Functions

                                                                                    • Meetups

                                                                                      Meetups

                                                                                    • Kbase

                                                                                      Kbase

                                                                                    • Resources

                                                                                      Resources

                                                                                    • Digest

                                                                                      Digest

                                                                                    • CRM Marketplace

                                                                                      CRM Marketplace

                                                                                    • MVP Corner

                                                                                      MVP Corner







                                                                                        Design. Discuss. Deliver.

                                                                                        Create visually engaging stories with Zoho Show.

                                                                                        Get Started Now


                                                                                          Zoho Show Resources


                                                                                            Zoho Writer Writer

                                                                                            Get Started. Write Away!

                                                                                            Writer is a powerful online word processor, designed for collaborative work.

                                                                                              Zoho CRM コンテンツ





                                                                                                Nederlandse Hulpbronnen


                                                                                                    ご検討中の方




                                                                                                          • Recent Topics

                                                                                                          • Getting Attachments in Zoho Desk via API

                                                                                                            Is there a way to get attachments into Zoho Desk via an API?      We have a process by which a zoho survey gets sent to the user as a link in a notification.    The survey has several upload fields where they can upload pdf documents.    I've created
                                                                                                          • Introducing Zoho's own SMS gateway

                                                                                                            We're thrilled to announce the launch of our own SMS gateway feature within Zoho Marketing Automation! This new feature enables seamless SMS campaign management alongside your email marketing initiatives, providing a more integrated and efficient way
                                                                                                          • Embedding in Desk articles

                                                                                                            We would like to embed documents in our Desk articles. When we use an iframe for the embed, we get scrollbars and a frame border. Neither of those is acceptable. I've spoken with the Desk Support team about what we want and they tell me that it cannot
                                                                                                          • Zoho Desk KB article embedded on another site.

                                                                                                            We embed KB articles from Zoho Desk on another site (our application). When opening the article in a new tab, there is no issue, but if we choose lightbox, we are getting an error "To protect your security, help.ourdomain.com will not allow Firefox to
                                                                                                          • Zoho CRM button to download images from image upload field

                                                                                                            Hello, I am trying to create a button in Zoho CRM that I can place in my record details view for each record and use it to download all images in the image upload fields. I tried deluge, client scripts and even with a widget, but feel lost, could not
                                                                                                          • Mass Update Contacts In Zoho Campaigns

                                                                                                            Is there a way to mass update contacts in zoho campaigns? I want to be able to change the content of a field for a few hundred contacts, and can't go through all of them individually.
                                                                                                          • report showing assignment type

                                                                                                            Hi, We've created a number of workflows to allow us to auto assign tickets to agents based on keywords and other criteria. I'm struggling to create a report that would show me what is the percentage of tickets that are assigned automatically via workflows
                                                                                                          • Option to Disable Knowledge Base Section in Feedback Widget Popup Hello Zoho Desk Team

                                                                                                            Hello Zoho Desk Team, How are you? We are actively using Zoho Desk and would like to make more use of the Feedback Widget. One of the ways we implement it is through the popup option. At the moment, the popup always displays the Knowledge Base section,
                                                                                                          • Notes Issues

                                                                                                            Been having issues with Notes in the CRM. Yesterday it wasn't showing the notes, but it got resolved after a few minutes., Now I have been having a hard time saving notes the whole day. Notes can't be saved by the save button. it's grayed out or not grayed
                                                                                                          • Placeholders in Ticket Templates

                                                                                                            We should be able to use placeholders in ticket templates. When we create a new ticket, our description field is shown to the client in the email they receive.  It would be very handy to be able to personalize that description field in our ticket templates to pull in the name of the client that the ticket is for. Using them in the subject field as well, so we can auto populate Account Names, etc. 
                                                                                                          • when the record is created the tag want to Show as Opportunity how i achive this using Deluge Script

                                                                                                            In the quotation i have the work flow schedule for create opportunity record in the module , on that time the quotation tag select as opportunity created. How i achive this using Deluge Script . this like i want to Do tag1 = Map(); tag1.put("name","Nurturing
                                                                                                          • Delete a channel

                                                                                                            I need delete a channel in tickets.
                                                                                                          • Copy / Duplicate Workflow

                                                                                                            I have workflows setup that are very similar to each other. We have a monitoring system watching servers, and all notifications - no matter what client it is about - will come from a  noreply@ address which is not very helpful in having it auto assigned to the right account. I have setup a workflow that will change the contact name of the ticket (currently it would say noreply@) to the correct customer which is based on the subject line, as that mentions which server the alert it is about. I need
                                                                                                          • Subtasks don't update parent task's times

                                                                                                            Hi there: I've recently upgraded to premium and check that subtasks completion % don't update the proportional completion of the parent tasks related to it.  We've been challenging with the problem of having to update twice or sometimes 3 times the completion of the related tasks. I've seen posts similar to this, of 3 years old.  Is there any roadmap for making this happen in a future release? Thanx César Ratto Lima, Perú.
                                                                                                          • Apply transaction rules to multiple banks

                                                                                                            Is there any way to make transaction rules for one bank apply to other banks? It seems cumbersome to have to re-enter the same date for every account.
                                                                                                          • Should I Use DMARC?

                                                                                                            When I configure Zoho Mail's DMARC settings, it's mandatory to fill in the RUA and RUF (Aggregate notification email address*, Forensic notification email address*) addresses. When we enter an email address in these fields, we receive reports from the
                                                                                                          • Mail ToDo & Tasks Webhooks

                                                                                                            Our company uses Zoho ToDo inside Mail to manage our tasks. When I create a task and assign it to a team member it does not notify them unless I add a reminder via mail. I'm trying to create a webhook for when a task is created to send a cliq message
                                                                                                          • What is Resolution Time in Business Hours

                                                                                                            HI, What is the formula used to find the total time spent by an agent on a particular ticket? How is Resolution Time in Business Hours calculated in Zohodesk? As we need to find out the time spent on the ticket's solution by an agent we seek your assistance
                                                                                                          • Create static subforms in Zoho CRM: streamline data entry with pre-defined values

                                                                                                            Last modified on (9 July, 2025): This feature was available in early access and is currently being rolled out to customers in phases. Currently available for users in the the AU, CA, and SA DCs. It will be enabled for the remaining DCs in the next couple
                                                                                                          • Can't upload attachments.

                                                                                                            I can't upload attachment in Zoho Mail.
                                                                                                          • Tip #44 – Get Deeper Insights with Zoho Assist’s Custom Reports – ‘Insider Insights’

                                                                                                            In today’s fast-paced IT environment, having a clear view of your remote support activities is more important than ever. Zoho Assist’s Custom Reports feature gives IT teams the ability to generate tailored reports that provide actionable insights and
                                                                                                          • Message "...does not support more than 100 distinct values..." WHY????

                                                                                                            I get this message on one of my reports: Sorry, Zoho Reports currently does not support more than 100 distinct values in columns. 'Account Name * Sum(Amount),Count(Amount Tier)' contains more than 100 distinct values.  Possibly, you can apply filter to reduce the number of distinct values in 'Account Name' or drop the 'Account Name' field in Rows. I want to list all ACCOUNT NAMES (about 500) with SALES BY ACCOUNT.  What is blocking this?
                                                                                                          • Need Easy Way to Update Item Prices in Bulk

                                                                                                            Hello Everyone, In Zoho Books, updating selling prices is taking too much time. Right now we have to either edit items one by one or do Excel export/import. It will be very useful if Zoho gives a simple option to: Select multiple items and update prices
                                                                                                          • Let's Talk Recruit: Super-charge hiring with Zoho Recruit add-ons

                                                                                                            Welcome back to our Let’s Talk Recruit series! This time, we’re diving into something that might seem like a small upgrade but has a huge impact on recruiter productivity: Zoho Recruit add-ons. Think about how much of your day is spent in your inbox or
                                                                                                          • Vendor Signatures Needed for Purchase Orders

                                                                                                            Hello everyone, We have a unique requirement that necessitates that Vendors & Suppliers formally acknowledge our Purchase Orders upon receipt. I was hoping that there would be an option to do so in Zoho Books, but that does not appear to be the case.
                                                                                                          • in zoho creator Sales Returns form has sub form Line Items return quantity when i upate the or enter any values in the sub form that want to reflect in the Sales Order form item deail sub form field Q

                                                                                                            in zoho creator Sales Returns form has sub form Line Items return quantity when i upate the or enter any values in the sub form that want to reflect in the Sales Order form item deail sub form field Quantity Returned\ pls check the recording fetch_salesorder
                                                                                                          • Is there an API to "File a Ticket" in Desk

                                                                                                            Hi, Is there an API to "File a Ticket" in Desk to zoho projects?
                                                                                                          • Store "Sign in with Google/Microsoft/GitHub etc." details

                                                                                                            Quite often now, users are using a sign-in provider like Google or Microsoft to sign into various apps and services. It would be great if Vault could remember which providers you use for each website and sign you in with that provider instead of a username
                                                                                                          • Critical Issue: Tickets Opened for Zoho Support via the Zoho Help Portal Were Not Processed

                                                                                                            Hi everyone, We want to bring to your attention a serious issue we’ve experienced with the Zoho support Help Portal. For more than a week, tickets submitted directly via the Help Portal were not being handled at all. At the same time no alert was posted
                                                                                                          • Tip of the Week #72– Assign thread ownership to avoid confusion.

                                                                                                            When teams handle a large volume of emails, managing threads becomes important to stay organized. Without a clear system, duplicate replies, missed follow-ups, or confusion over responsibilities can happen. Thread assignment solves this by designating
                                                                                                          • Migrate Your Notes from OneNote to Zoho Notebook Today

                                                                                                            Greetings Notebook Users, We’re excited to introduce a powerful new feature that lets you migrate your notes from Microsoft OneNote to Zoho Notebook—making your transition faster and more seamless than ever. ✨ What’s New One-click migration: Easily import
                                                                                                          • Unarchive tickets

                                                                                                            How can I manually unarchive tickets?
                                                                                                          • Optimize your Knowledge Base for enhanced accessibility by adding alt tags for images

                                                                                                            Let's learn why alt tags are crucial for your articles. You can add alternative tags (alt tags) and alternative text (alt text) to the images you share on your community forums or when embedding them in articles. Alt tags refer to the HTML attribute,
                                                                                                          • FSM trying again

                                                                                                            have not linked FSM yet to the rest of out Zoho suit. It certainly looks like the apointment and service part is more manageable for our staff. The question is that our engineers multi task examples 1. deliver products to customers not fitted 2. Service
                                                                                                          • Feature Request: Conditional Field Mandatoriness Based on Display Status

                                                                                                            Hello Zoho Creator Team, I would like to suggest an enhancement to improve the flexibility of form validations. Currently, when we need a field to be mandatory only if it's displayed on the form, the only option is to: Set the field as not mandatory in
                                                                                                          • Data Migration Strategies for Moving to a Cloud Solution

                                                                                                            Hi everyone, I’ve been working on moving some of our critical systems, including CRM and project data, to a Zoho cloud solution, and one of the biggest challenges I’ve encountered is data migration. Transferring large volumes of data while keeping it
                                                                                                          • Commerce Order as Invoice instead of Sales Order?

                                                                                                            I need a purchase made on my Commerce Site to result in an Invoice for services instead of a Sales Order that will be pushed to Books. My customers don't pay until I after I add some details to their transaction. Can I change the settings to make this
                                                                                                          • How to set different item selling prices for Zoho Commerce and Zoho Books

                                                                                                            Item selling prices for Zoho Commerce and Zoho Books are in sync. If we update the Item selling price in Books, the same will happen in commerce and vice versa. I need a separate commerce selling price for online users and a separate books selling price
                                                                                                          • Time Entry Notifications

                                                                                                            Hi All - I have support staff who place notes of their work in the time entry section of Zoho Desk. Is there a specific workflow or setting I need to enable to have the ticket holder updated via email when an entry is saved?
                                                                                                          • Zoho CRMの流入元について

                                                                                                            Zoho CRMとZoho formsを連携し、 formsで作成したフォームをサイトに埋め込み運用中です。 UTMパラメータの取得をformsを行い、Zoho CRMの見込み客タブにカスタム項目で反映される状況になっています。 広告に関してはUTMパラメータで取得できているため問題ないのですが、オーガニック流入でフォーム送信の場合も計測したいです。メールやGoogle、Yahoo、directなどの流入元のチャネルが反映されるようにしたいのですが、どのように設定したら良いでしょうか。 また、
                                                                                                          • Next Page