Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere







                          Zoho Developer Community






                                                • Desk Community Learning Series


                                                • Digest


                                                • Functions


                                                • Meetups


                                                • Kbase


                                                • Resources


                                                • Glossary


                                                • Desk Marketplace


                                                • MVP Corner


                                                • Word of the Day


                                                • Ask the Experts



                                                          • Sticky Posts

                                                          • Register for Zoho Desk Beta Community

                                                            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                          • Share your Zoho Desk story with us!

                                                            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                          • Tip #1: Learn to pick the right channels

                                                            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                          • Welcome to Zoho Desk Community - Say hello here!

                                                            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                          • Webinar 1: Blueprint for Customer Service

                                                            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                          Manage your brands on social media



                                                                Zoho TeamInbox Resources



                                                                    Zoho CRM Plus Resources

                                                                      Zoho Books Resources


                                                                        Zoho Subscriptions Resources

                                                                          Zoho Projects Resources


                                                                            Zoho Sprints Resources


                                                                              Qntrl Resources


                                                                                Zoho Creator Resources



                                                                                    Zoho CRM Resources

                                                                                    • CRM Community Learning Series

                                                                                      CRM Community Learning Series


                                                                                    • Kaizen

                                                                                      Kaizen

                                                                                    • Functions

                                                                                      Functions

                                                                                    • Meetups

                                                                                      Meetups

                                                                                    • Kbase

                                                                                      Kbase

                                                                                    • Resources

                                                                                      Resources

                                                                                    • Digest

                                                                                      Digest

                                                                                    • CRM Marketplace

                                                                                      CRM Marketplace

                                                                                    • MVP Corner

                                                                                      MVP Corner







                                                                                        Design. Discuss. Deliver.

                                                                                        Create visually engaging stories with Zoho Show.

                                                                                        Get Started Now


                                                                                          Zoho Show Resources


                                                                                            Zoho Writer Writer

                                                                                            Get Started. Write Away!

                                                                                            Writer is a powerful online word processor, designed for collaborative work.

                                                                                              Zoho CRM コンテンツ





                                                                                                Nederlandse Hulpbronnen


                                                                                                    ご検討中の方




                                                                                                          • Recent Topics

                                                                                                          • Store "Sign in with Google/Microsoft/GitHub etc." details

                                                                                                            Quite often now, users are using a sign-in provider like Google or Microsoft to sign into various apps and services. It would be great if Vault could remember which providers you use for each website and sign you in with that provider instead of a username
                                                                                                          • Critical Issue: Tickets Opened for Zoho Support via the Zoho Help Portal Were Not Processed

                                                                                                            Hi everyone, We want to bring to your attention a serious issue we’ve experienced with the Zoho support Help Portal. For more than a week, tickets submitted directly via the Help Portal were not being handled at all. At the same time no alert was posted
                                                                                                          • Tip of the Week #72– Assign thread ownership to avoid confusion.

                                                                                                            When teams handle a large volume of emails, managing threads becomes important to stay organized. Without a clear system, duplicate replies, missed follow-ups, or confusion over responsibilities can happen. Thread assignment solves this by designating
                                                                                                          • Migrate Your Notes from OneNote to Zoho Notebook Today

                                                                                                            Greetings Notebook Users, We’re excited to introduce a powerful new feature that lets you migrate your notes from Microsoft OneNote to Zoho Notebook—making your transition faster and more seamless than ever. ✨ What’s New One-click migration: Easily import
                                                                                                          • Unarchive tickets

                                                                                                            How can I manually unarchive tickets?
                                                                                                          • Optimize your Knowledge Base for enhanced accessibility by adding alt tags for images

                                                                                                            Let's learn why alt tags are crucial for your articles. You can add alternative tags (alt tags) and alternative text (alt text) to the images you share on your community forums or when embedding them in articles. Alt tags refer to the HTML attribute,
                                                                                                          • FSM trying again

                                                                                                            have not linked FSM yet to the rest of out Zoho suit. It certainly looks like the apointment and service part is more manageable for our staff. The question is that our engineers multi task examples 1. deliver products to customers not fitted 2. Service
                                                                                                          • Feature Request: Conditional Field Mandatoriness Based on Display Status

                                                                                                            Hello Zoho Creator Team, I would like to suggest an enhancement to improve the flexibility of form validations. Currently, when we need a field to be mandatory only if it's displayed on the form, the only option is to: Set the field as not mandatory in
                                                                                                          • Data Migration Strategies for Moving to a Cloud Solution

                                                                                                            Hi everyone, I’ve been working on moving some of our critical systems, including CRM and project data, to a Zoho cloud solution, and one of the biggest challenges I’ve encountered is data migration. Transferring large volumes of data while keeping it
                                                                                                          • Commerce Order as Invoice instead of Sales Order?

                                                                                                            I need a purchase made on my Commerce Site to result in an Invoice for services instead of a Sales Order that will be pushed to Books. My customers don't pay until I after I add some details to their transaction. Can I change the settings to make this
                                                                                                          • How to set different item selling prices for Zoho Commerce and Zoho Books

                                                                                                            Item selling prices for Zoho Commerce and Zoho Books are in sync. If we update the Item selling price in Books, the same will happen in commerce and vice versa. I need a separate commerce selling price for online users and a separate books selling price
                                                                                                          • Time Entry Notifications

                                                                                                            Hi All - I have support staff who place notes of their work in the time entry section of Zoho Desk. Is there a specific workflow or setting I need to enable to have the ticket holder updated via email when an entry is saved?
                                                                                                          • Zoho CRMの流入元について

                                                                                                            Zoho CRMとZoho formsを連携し、 formsで作成したフォームをサイトに埋め込み運用中です。 UTMパラメータの取得をformsを行い、Zoho CRMの見込み客タブにカスタム項目で反映される状況になっています。 広告に関してはUTMパラメータで取得できているため問題ないのですが、オーガニック流入でフォーム送信の場合も計測したいです。メールやGoogle、Yahoo、directなどの流入元のチャネルが反映されるようにしたいのですが、どのように設定したら良いでしょうか。 また、
                                                                                                          • How to report 'Response violation' OR 'Resolution violation'

                                                                                                            Hi, I want to report on SLA Violation Type. I grouped my tickets on this column. It seems I only get 'Response and Resolution Violation' or 'Not Violated'. The former seems to be given to a ticket if only the Response Time was violated. I would expect
                                                                                                          • [Webinar] Automate sales and presales workflows with Writer

                                                                                                            Sales involves sharing a wide range of documents with customers across the presales, sales, and post-sales stages: NDAs, quotes, invoices, sales orders, and delivery paperwork. Generating and managing these documents manually slows down the overall sales
                                                                                                          • Power of Automation :: Quick way to associate your Projects with Zoho CRM

                                                                                                            A custom function is a software code that can be used to automate a process and this allows you to automate a notification, call a webhook, or perform logic immediately after a workflow rule is triggered. This feature helps to automate complex tasks and
                                                                                                          • Date triggering Workflow rule

                                                                                                            I have a function triggered by a workflow rule. The function takes a date and creates a task for that date and fills in a field with the name of the day for that date. It also updates the status field of the record. The workflow rule is set to run whenever
                                                                                                          • Restricting contact creation

                                                                                                            Hi all! I am looking to use Zoho Desk in a part of the business that takes end user enquiries. These are generally single interactions, and not linked to an account name. As Desk is Account centric, has anyone designed a way to manage these incoming emails
                                                                                                          • Import Holiday Calendars

                                                                                                            HI Zoho Is there anyway of importing an online calendar like https://www.calendarlabs.com into the business hours calendars, to speed up setup of holiday calendars. Also could we also request a feature where you can specify a Holiday as hours, i.e it could be that the company is on a 1/2 day due to a holiday or when it is Eid in the UAE and they are only allowed to work restricted hours so we need the calendar to be flexible to allow for this. Regards Jamie
                                                                                                          • Filtering Tickets based on Email headers

                                                                                                            We're starting to get a lot more junk coming into our Zoho Desk, which is then triggering unnecessary email alerts to agents. Once thing we could do to cut this junk in half, is to filter tickets based on email headers. Any email containing the `List-Unsubscribe`
                                                                                                          • Error 550 5.4.1

                                                                                                            I’ve tried sending an email to someone but keep receiving this back. Any help would be greatly appreciated 
                                                                                                          • Billing Management: #2 Fair way of Billing- Prorated Billing

                                                                                                            Hello, From speaking about the traditional ways of billing in the previous post, we are moving into the deep sea of billing. We are now in a zone to break out the most complex yet, I would call it the fairest way of billing, the Prorated Billing. Prorated
                                                                                                          • Automatic Matching from Bank Statements / Feeds

                                                                                                            Is it possible to have transactions from a feed or bank statement automatically match when certain criteria are met? My use case, which is pretty broadly applicable, is e-commerce transactions for merchant services accounts (clearing accounts). In these
                                                                                                          • Has Anyone successfully integrated Zoho and Sage Intact?

                                                                                                            Hey all, We’re evaluating Zoho One + Sage Intacct and I’m trying to connect with anyone who has actually implemented the two together.Specifically, I’d love to know: -- Which functions you kept in Zoho vs. Intacct (e.g., Product Catalog, AR/AP, invoicing,
                                                                                                          • How can I filter a field integration?

                                                                                                            Hi,  I have a field integration from CRM "Products" in a form, and I have three product Categories in CRM. I only need to see Products of a category. Thanks for you answers.
                                                                                                          • Adding image in HTML report page

                                                                                                            Hi, I want to know two things: 1. Can anyone advise how to add an image in HTML report. The tagged used is <img> but what path do I mention for the image to be added in the HTML report. 2. Also, I want to know if I am creating an application for the market
                                                                                                          • How to change view of HTML report based on device but always print in A4

                                                                                                            Hello everyone, I am aware that HTML report view can be configured to adjust according to the screen size like Laptop, Tablet and mobile using media queries. But my concern is no matter on which device the reports is opened when printed should always
                                                                                                          • Age Calculation

                                                                                                            I've attempted to calculate the age of someone based on their birthday input by using the formula field. It works but I don't want all those decimals on there. I then tried to use "set variable" after birthday input but I get a field type mismatch, long vs. floating. Any ideas would be wonderful.
                                                                                                          • Search on Custom Field

                                                                                                            We're working on an integration with the Zoho FSM API and are trying to retrieve companies based on a custom field we added to the Companies module. However, we can't find a way to filter or query records using custom fields through the API. We have a
                                                                                                          • Sendmail function / custom action?

                                                                                                            I've setup a function hoping to email various business departments the details of a record once all work in that record is complete so gone about setting up a custom action in such way that each record line on the report has a button to click. Question is how do I actually include data from that record in the email that is sent when the button is clicked? I had thought that since this were being sent per record the email would include the data which had been entered
                                                                                                          • API to post drafts for social media

                                                                                                            I we want to post draft posts to our zoho social account and then approve and schedule them within Zoho social. is this possible with for example: https://apis.zoho.com/social/v2/post TIA Jon
                                                                                                          • Integración Books para cumplir la ley Crea y Crece y Ley Antifraude (VeriFactu)

                                                                                                            Hola: En principio, en julio de 2025, entra en vigor la ley Crea y Crece y Ley Antifraude (VeriFactu). ¿Sabéis si Zoho va a cumplir con la ley para cumplir con la facturación electrónica conectada a Hacienda? Gracias
                                                                                                          • Canvas View in Zoho Recruit

                                                                                                            Is it possible or would it be possible to have the new 'Canvas View' in Zoho Recruit?
                                                                                                          • What impactful sales coaching techniques have you used to boost your team's performance?

                                                                                                            I'm curious about the real-world impact of sales coaching on team performance. What specific techniques or strategies have you found most effective in driving consistent improvement and growth in your sales team? Any success stories or lessons learned
                                                                                                          • Adding Taxes to paid consultations in Zoho Bookings

                                                                                                            I created a 'paid' consultation under Zoho Booking and integrated it with payment gateways for online/instant payment before a booking is done. How can I add 'taxes' to the price of consultation? I can add taxes to other Zoho apps (liks Books, Checkout,
                                                                                                          • Zoho Finance Suite - Customer Custom Tabs - Dynamic Link

                                                                                                            Hi Finance Suite team, When creating a Custom Tab for a Client Portal, there is no option to add dynamic parameters. This would be very helpful for adding Zoho Analytics dashboards which can be dynamically filtered through the URL to only show information
                                                                                                          • Feature Request - Insert URL Links in Folders

                                                                                                            I would love to see the ability to create simple URL links with titles in WorkDrive. or perhaps a WorkDrive extension to allow it. Example use case: A team is working on a project and there is project folder in WordDrive. The team uses LucidChart to create
                                                                                                          • How to Calculate MTTR (Mean Time to Resolve)

                                                                                                            We want to calculate MTTR (Mean Time to Resolve) in our Zoho Analytics report under Tickets. Currently, we are using the following fields: Ticket ID Ticket Created Time Ticket Closed Time Ticket On Hold Time We are planning to calculate MTTR (in days)
                                                                                                          • Possible to bold or indent text in the description field?

                                                                                                            As part of one item, I often have a detailed description that would be much easier to read if there was the ability to have a bulleted list or bold text and the like. Is this possible? My last invoicing software allowed markup in the field so, for example, an asterisk meant a bullet. I haven't been able to find any documentation related to this.  Any information would be appreciated. Thank you.
                                                                                                          • How can I setup Zoho MCP with Chat GPT

                                                                                                            I can set up custom connections with Chat GPT but I cat an error when I try to set it up. The error is: "This MCP server can't be used by ChatGPT to search information because it doesn't implement our specification: search action not found" Thoughts?
                                                                                                          • Next Page