Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere






                            Zoho Developer Community




                                                  • Desk Community Learning Series


                                                  • Digest


                                                  • Functions


                                                  • Meetups


                                                  • Kbase


                                                  • Resources


                                                  • Glossary


                                                  • Desk Marketplace


                                                  • MVP Corner


                                                  • Word of the Day


                                                  • Ask the Experts



                                                            • Sticky Posts

                                                            • Register for Zoho Desk Beta Community

                                                              With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                            • Share your Zoho Desk story with us!

                                                              Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                            • Tip #1: Learn to pick the right channels

                                                              Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                            • Welcome to Zoho Desk Community - Say hello here!

                                                              Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                            • Webinar 1: Blueprint for Customer Service

                                                              With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                            Manage your brands on social media



                                                                  Zoho TeamInbox Resources



                                                                      Zoho CRM Plus Resources

                                                                        Zoho Books Resources


                                                                          Zoho Subscriptions Resources

                                                                            Zoho Projects Resources


                                                                              Zoho Sprints Resources


                                                                                Qntrl Resources


                                                                                  Zoho Creator Resources



                                                                                      Zoho CRM Resources

                                                                                      • CRM Community Learning Series

                                                                                        CRM Community Learning Series


                                                                                      • Kaizen

                                                                                        Kaizen

                                                                                      • Functions

                                                                                        Functions

                                                                                      • Meetups

                                                                                        Meetups

                                                                                      • Kbase

                                                                                        Kbase

                                                                                      • Resources

                                                                                        Resources

                                                                                      • Digest

                                                                                        Digest

                                                                                      • CRM Marketplace

                                                                                        CRM Marketplace

                                                                                      • MVP Corner

                                                                                        MVP Corner







                                                                                          Design. Discuss. Deliver.

                                                                                          Create visually engaging stories with Zoho Show.

                                                                                          Get Started Now


                                                                                            Zoho Show Resources

                                                                                              Zoho Writer

                                                                                              Get Started. Write Away!

                                                                                              Writer is a powerful online word processor, designed for collaborative work.

                                                                                                Zoho CRM コンテンツ




                                                                                                  Nederlandse Hulpbronnen


                                                                                                      ご検討中の方







                                                                                                              • Recent Topics

                                                                                                              • Unable to send message;Reason:554 5.1.8 Email Outgoing Blocked.

                                                                                                                My email account is unable to send emails, and I urgently need to use it. How can I resolve this?If there is anything we have done wrong, please let us know in advance so we can actively cooperate to improve. User ID: 850482493
                                                                                                              • URGENT: Email stopped workin - can't access admin panel

                                                                                                                For some reason email sending stopped working. When I try to send an email it fails with "Unable to send message;Reason:451 4.7.1 Temporary system error" I can receive email just fine I see in my notifications some errors about the MX records, however
                                                                                                              • Can I associate an invoice to a Project after the fact?

                                                                                                                We have generated an invoice but would like to assoicate it to a Zoho Project after the fact.  Is there any way to accomplish that? Thanks, Scott
                                                                                                              • Emails I send as a cc or bcc NEVER GO THROUGH TO THA RECEIVER !!

                                                                                                                every time i send a cc or bcc copy of an email to anyone when I’m using my zohomail.com email - no one ever gets the cc of bcc copy if the email: why???? And i triple check that the email addresses are correct; then i get back an email message (EVERYTIME)
                                                                                                              • Cannot see correct DNS config for mail after moving domain to another provider

                                                                                                                I have moved my domain from one provider to another and after that zoho mail stopped working (expected). Problem is, zoho mail admin panel still shows (10 hours after move) that all records are correct while I haven't changed anything in my domain DNS
                                                                                                              • Add Support for Authenticator App MFA in Zoho Desk Help Center

                                                                                                                Hello Zoho Desk Team, We hope you are doing well. We would like to request an enhancement related to security for the Zoho Desk Help Center (customer portal). Currently, the Help Center supports MFA for portal users via SAML, JWT, SMS authentication,
                                                                                                              • How to unlink a SAML user from the existing Zoho Desk user (domain change case)

                                                                                                                Hi everyone, I’m trying to understand how to handle a situation where a customer changes their company domain. In our setup, users authenticate via SAML, so when the domain changes, the SAML system treats them as a new user. However, in Zoho Desk, I’d
                                                                                                              • Price Managment

                                                                                                                I have been in discussions with Zoho for some time and not getting what I need. Maybe someone can help explain the logic behind this for me as I fail to understand. When creating an item, you input a sales rate and purchase rate. These rates are just
                                                                                                              • Related to zoho survey

                                                                                                                Hi team. I want to know something regarding zoho survey question builder. I have two questions each of dropdown (One answer) - question type. In the first question, there are 16 answer choices and in the second question, there are 3 answer choices. For
                                                                                                              • 60 Days Into Zoho - Tiktok Branding Startup -7 Questions?!

                                                                                                                Wsp Everybody I co-own a TikTok Branding / Consulting Startup & have been using Zoho for the past 60 days - Am now looking to make our overall operations & processes more Efficient & Effective! Curious to know how others are using the platform & what's
                                                                                                              • Text Message

                                                                                                                When trying to sent a text message, it says its an error i should contact a zoho agent
                                                                                                              • Zoho POS is now available for Canadian retailers

                                                                                                                Hey everyone, We're excited to introduce the all-new Canadian edition of Zoho POS, which helps retail businesses simplify and manage their end-to-end business operations. Start by signing up and exploring the 15-day free trial. Sign up now How does Zoho
                                                                                                              • Payroll In Canada

                                                                                                                Hi, When can we expect to have payroll in Canada with books 
                                                                                                              • Mass update to change or shift all project dates and keep project structure + shift all sub-tasks dates with main tasks

                                                                                                                Most users would expect that if they change the start date of their project then that will be reflected inside the actual project and any project structure would be retained.  Additionally if a task list with associated sub-tasks is moved, most users would expect that those sub-tasks would also be moved along with their parent task.  This is not the case. For a total Project shift of dates: * the start and end date from > "Edit Project" page can have a radio button added to "shift all project tasks
                                                                                                              • Closing Accounting Periods - Invoice/Posting dates

                                                                                                                Hi, I have seen in another thread but I'm unsure on how the 'transaction locking' works with regards to new and old transactions. When producing monthly accounts if I close December 24 accounts on 8th Jan 25 will transaction locking prevent me from posting
                                                                                                              • In-person ZUG Meetups for Real Estate Professionals - US Q1 2026

                                                                                                                The Real Estate Zoho User Group is going on a multi-city, in-person meetup tour across the US, and we’d love to see you there! These meetups are a great opportunity to: Connect with fellow real estate professionals using Zoho Share challenges and discover
                                                                                                              • Zoho CRM custom fields not showing in zoho creator

                                                                                                                Hi Team, I have created a Products form with Zoho CRM integration and connected it to Products module of CRM. But when I see the reports of Products in Zoho creator then I am not able to see custom fields of Products module. Only standard fields of Products
                                                                                                              • Sending email notifications based on language

                                                                                                                Hello. I would like to know how we can bypass the default notifications (which are just in English) for when a ticket is created/replied to/closed, to be in other languages, based on the language field in the ticket? I can create other email templates,
                                                                                                              • Zoho Error: This Operation has been restricted. Please contact support-as@zohocorp.com for further details

                                                                                                                Hello There, l tried to verify my domain (florindagoreti.com.br) and its shows this error: This Operation has been restricted. Please contact support-as@zohocorp.com for further details. Screenshot Given Below -  please check what went wrong. Thanks
                                                                                                              • Download a file from within a zoho creator widget

                                                                                                                I have a widget running in Zoho Creator , it displays uploaded documents in a table file, and I have added a download link in the view. ( The widget is created with html, css and javascript). I do not succeed in getting the download working. Do I have
                                                                                                              • Correlated subqueries not supported in Zoho Analytics. This creates huge limitations

                                                                                                                Running into a major limitation in Zoho Analytics: correlated subqueries simply don’t work, even in completely standard SQL patterns inside a JOIN. Example: LEFT JOIN "Bills" b ON d."Id" = b."Deal ID" AND EXISTS ( SELECT 1 FROM "Bill
                                                                                                              • Zoho / Outlook Calendar sync

                                                                                                                The current Marketplace -> Microsoft -> Meetings integration needs 2 changes. 1. The current language for the Two-Way sync option should be changed. It currently states, "Sync both your Zoho CRM Calendar and Office 365 Calendar meetings with each other."
                                                                                                              • Email content just contain 'OK' ,not what we expect

                                                                                                                create campaign API URL: https://campaigns.zoho.com/api/v1.1/createCampaign req params: {'campaignname': 'General_Outreach_d0cfc415-43aa-4b96-bb09-558e76a3dda3_50_20251117_214806_660', 'from_email': 'admin@allinmedia.ai', 'subject': 'ALL IN MEDIA', 'list_details':
                                                                                                              • Possible to connect Zoho CRM's Sandbox with Zoho Creator's Sandbox?

                                                                                                                We are making some big changes on our CRM so we are testing it out in CRM's Sandbox. We also have a Zoho Creator app that we need to test. Is it possible to connect Zoho CRM's Sandbox to Zoho Creator's Sandbox so that I can perform those tests?
                                                                                                              • Reopen ticket

                                                                                                                Hello! Can I reopen a ticket just using the API ticket/sendReply ? What's the rules to do it? I'm trying but it doesn't reopen the ticket, it just send the reply
                                                                                                              • Allow Admins to Transfer Ownership of Their Own Files & Folders

                                                                                                                Hi Zoho WorkDrive Team, Hope you are doing well. We would like to request an important enhancement to the ownership-transfer functionality in Zoho WorkDrive, specifically regarding administrator capabilities. As administrators, we have the ability to
                                                                                                              • Tip#46: Capture accurate log hours

                                                                                                                Hello everyone, Use the newly introduced timer settings that will streamline the usage of timers and help admins or workspace owners to manage the time entries of the workspace users better. Check out the below mentioned timer settings added to the Timesheet
                                                                                                              • Moving to app-specific authentication for Google integrations

                                                                                                                Hello everyone, We’re making an important change to how Google integrations work in our platform. Until now, we used a common Google project across Zoho to enable integrations like Google Drive, Calendar, and more. Going forward, we’ll be moving to an
                                                                                                              • Lost the ability to sort by ticket owner

                                                                                                                Hi all, in the last week or so, we have lost the ability to sort tickets by Ticket Owner. Unlike the other columns which we can hover over and click on to sort, Ticket Owner is no longer clickable. Is it just us, or are other customers seeing this too?
                                                                                                              • How Can i put a form in Zobot

                                                                                                                Hi,how can i integrate a form which has a multiple options to choose from.the form should be opened or displayed by zobot after it meets a requirement in the conversation. Thanks in advance !
                                                                                                              • Playback and Management Enhancements for Zoho Quartz Recordings

                                                                                                                Hello Zoho Team, We hope you're all doing well. We would like to submit a feature request related to Zoho Quartz, the tool used to record and share browser sessions with Zoho Support. 🎯 Current Functionality As of now, Zoho Quartz allows users to record
                                                                                                              • Zoho Analytics - Feature Request For Time Based Data Source Fetch

                                                                                                                Hi Analytics Team, I have a client using Zoho CRM and they want a weekly report at 4:30pm every Friday, emailed to the sales team showing a pie chart of Closed Won Deals for that week. This is easy to achieve in Analytics but not so easy to ensure the
                                                                                                              • Which user's capacity is used for Shared Mailbox storage?

                                                                                                                We use shared mailboxes at our company, and their size is increasing daily. Which user(s)'s total mailbox limit is being used up by this space?
                                                                                                              • Inserting the current date / time

                                                                                                                I'd like to use Zoho Notebook as a log - so it would be great to be able to insert the current date and time at the beginning (or end) of each my log/journal entries - or wherever I want. Something simple, like pressing a button/icon to insert it wherever my text cursor is. It could be formatted like "Mon Dec 11, 2017 - 11:43p". 
                                                                                                              • Migrate data from old to new account

                                                                                                                Hy, Have one Old Zoho Notebook Account with Data , want to migrate that whole Data to New Zoho Notebook Account which is in Zoho One . Is that possible ? If Yes then how?
                                                                                                              • Campaign editor overrides href value

                                                                                                                I've been trying to insert a deep link in an email campaign so recipients can directly open a native app on their device. My deep link looks something like "myapp://". The options to insert links in the campaign editor are limited to strict urls, emails,
                                                                                                              • how to add subform over sigma in the CRM

                                                                                                                my new module don't have any subform available any way to add this from sigma or from the crm
                                                                                                              • Bug Report and Suggestions for Improvement in Zoho Applications

                                                                                                                Hi Zoho Team, I’d like to report a few bugs and improvement suggestions I’ve noticed while using Zoho products: Zoho Cliq Video Call: The camera sometimes turns off automatically during video calls. This seems to be a bug — please check and fix it. Zoho
                                                                                                              • Canvas Flex Box containers should not require a width/height.

                                                                                                                Flexbox containers are often used as organizational concepts, to ease re-flow on mobile etc. - I cannot use % for flexbox W or H - I cannot omit W or H This means that the content cannot dictate, and a Flexbox container cannot be used merely as an organizational
                                                                                                              • Zoho Desk Android app update: Filter enhancement, Save Filters

                                                                                                                Hello everyone! We have now enhanced the filter section on the ticket listing screen of Zoho Desk Android app as it is on the web(desk.zoho.com). Also, we have introduced an option to save filters for tickets on the mobile app. You can easily apply, rename,
                                                                                                              • Next Page