Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere









                          Zoho Developer Community




                                                • Desk Community Learning Series


                                                • Digest


                                                • Functions


                                                • Meetups


                                                • Kbase


                                                • Resources


                                                • Glossary


                                                • Desk Marketplace


                                                • MVP Corner


                                                • Word of the Day


                                                • Ask the Experts



                                                          • Sticky Posts

                                                          • Register for Zoho Desk Beta Community

                                                            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                          • Share your Zoho Desk story with us!

                                                            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                          • Tip #1: Learn to pick the right channels

                                                            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                          • Welcome to Zoho Desk Community - Say hello here!

                                                            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                          • Webinar 1: Blueprint for Customer Service

                                                            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                          Manage your brands on social media



                                                                Zoho TeamInbox Resources



                                                                    Zoho CRM Plus Resources

                                                                      Zoho Books Resources


                                                                        Zoho Subscriptions Resources

                                                                          Zoho Projects Resources


                                                                            Zoho Sprints Resources


                                                                              Qntrl Resources


                                                                                Zoho Creator Resources



                                                                                    Zoho CRM Resources

                                                                                    • CRM Community Learning Series

                                                                                      CRM Community Learning Series


                                                                                    • Kaizen

                                                                                      Kaizen

                                                                                    • Functions

                                                                                      Functions

                                                                                    • Meetups

                                                                                      Meetups

                                                                                    • Kbase

                                                                                      Kbase

                                                                                    • Resources

                                                                                      Resources

                                                                                    • Digest

                                                                                      Digest

                                                                                    • CRM Marketplace

                                                                                      CRM Marketplace

                                                                                    • MVP Corner

                                                                                      MVP Corner







                                                                                        Design. Discuss. Deliver.

                                                                                        Create visually engaging stories with Zoho Show.

                                                                                        Get Started Now


                                                                                          Zoho Show Resources

                                                                                            Zoho Writer

                                                                                            Get Started. Write Away!

                                                                                            Writer is a powerful online word processor, designed for collaborative work.

                                                                                              Zoho CRM コンテンツ






                                                                                                Nederlandse Hulpbronnen


                                                                                                    ご検討中の方




                                                                                                          • Recent Topics

                                                                                                          • Deleting Fields in CRM Deletes Views in Analytics

                                                                                                            Hey friends! I'm having some issues when we modify some fields within ZohoCRM. There are times where we need to sunset a field and eventually completely remove it. In these instances, it seems like a lot of views are removed in Analytics. This ranges
                                                                                                          • Updating custom fields in Zoho Projects

                                                                                                            Hi I am wondering if anyone has experience with custom fields in Zoho Projects. I am struggling to update the field using either deluge or the api endpoint. My code is: //custom_Map = map(); custom_Map = {"UDF_DOUBLE_1":"0.27"}; update_Map = map(); update_Map.put("custom_fields",custom_Map.toList());
                                                                                                          • Audio/video quality issues with Zoho Meeting – Any roadmap for improvement?

                                                                                                            Hi Zoho Team, We’ve been using Zoho Meeting for both internal and external meetings, and unfortunately, the experience has been consistently poor. The video and audio quality are so unreliable that it often renders meetings ineffective—especially with
                                                                                                          • Pin multiple columns and adjust column widths in CRM subforms

                                                                                                            Hello all, Subforms act as secondary forms or tables in which you can associate multiple line items to a primary record and thereby ensure more structured and comprehensive data organization. We've made some recent enhancements to subforms. Here's what's
                                                                                                          • Paste issues in ZOHO crm notes

                                                                                                            Hi, since a week or so I have issues with the paste function in ZOHO CRM. I use "notes" to copy paste texts from Outlook emails and since a week or so, the pasting doesnt function as it should: some text just disappears and it gives a lot of empty lines/enters.....
                                                                                                          • Cash based businesses cannot use the new fixed asset module

                                                                                                            Hello all, If your bookkeeping is reporting in cash, you cannot use the new fixed acid module, as it does all the depreciation bookings accrual and not cash. This is definitive and you can't turn them into a cash booking. They will never appear in your
                                                                                                          • Zoho Learn & Zoho Connect

                                                                                                            Hi, Is there a way to sync the knowledge base we have in Zoho Learn with the manuals section is Zoho Connect? Thanks,
                                                                                                          • Calendar not working

                                                                                                            Are we the only ones. On any browser we cannot click on any of our calendar appointments and get them to open. They just make the browser loop. WE have reached out and have been told they are working on it. The office staff are really stuck. The point
                                                                                                          • Modifying iframe data of Zoho booking iframe

                                                                                                            Hello, I have integrated a Zoho Bookings embedded iframe into my website. Currently, I am pre-filling the booking form with default values as part of our process flow. However, I want to ensure that if an input field is already populated with a default
                                                                                                          • New Customization options in the module builder: Quick Create and Detail view

                                                                                                            Hello everyone, We have introduced two new components to the module builder: Quick create and Detail view. The Quick Create Component It is a mini form used to create a record and associate it to the parent record from a lookup field. For example, if you have a Deals lookup in the Contacts module, then you can associate existing deals or create a deal and associate it with the contact. You can customize this Quick Create form by adding standard as well as custom fields. There is no limit to the number
                                                                                                          • Apply Payment Received Amount Zoho Books Invoice

                                                                                                            Hello team here is the sample code How can apply the payment received record over a unpaid zoho books invoice. //......................... paymentID = customer_payment.get("payment_id"); organizationID = organization.get("organization_id"); paymentmaplist
                                                                                                          • Enhance Sign CRM integration

                                                                                                            Hello all, I'm working on a custom Deluge script to enhance the integration between Zoho CRM and Sign by using a writer merge template for additional flexibility. I want to replicate the post-sign document integration that exists between CRM and Sign
                                                                                                          • Zoho Books Finance Modules Not Accessible in Zoho CRM Mobile App

                                                                                                            We have integrated Zoho CRM with Zoho Books using the Zoho Finance Suite integration. In the CRM web version, we can see the Finance modules (Estimates/Quotes, Invoices, Sales Orders, Items, Payments) and are able to create invoices and quotes directly
                                                                                                          • Does Zoho offer a full WhatsApp Marketing Platform (like WATI / GallaBox)?

                                                                                                            lm exploring WhatsApp marketing and I’d like to understand what options are available within the Zoho ecosystem. Currently, I see: Zoho SalesIQ has a broadcast option, but it seems very limited — for example, it doesn’t support uploading contacts via
                                                                                                          • Is it really true that I can't set the default 'deposit to' account in 2025?

                                                                                                            I've been using Books for 7 years and the default account has never been a problem. I usually manually reconcile invoices and have never had a thought about which account. It has always been my account. However, I recently noticed that for the past 4
                                                                                                          • Cliq iOS can't see shared screen

                                                                                                            Hello, I had this morning a video call with a colleague. She is using Cliq Desktop MacOS and wanted to share her screen with me. I'm on iPad. I noticed, while she shared her screen, I could only see her video, but not the shared screen... Does Cliq iOS is able to display shared screen, or is it somewhere else to be found ? Regards
                                                                                                          • Automating CRM backup storage?

                                                                                                            Hi there, We've recently set up automatic backups for our Zoho CRM account. We were hoping that the backup functionality would not require any manual work on our end, but it seems that we are always required to download the backups ourselves, store them,
                                                                                                          • [Live Webinar] New in Zoho WorkDrive: AI enhancements, Data Loss Prevention, Version Controls, and more

                                                                                                            Hello everyone, We're excited to bring you another round of powerful updates in Zoho WorkDrive! Join us on May 15 for an exclusive live webinar where we’ll unveil the latest features designed to enhance your team’s productivity, collaboration, and data
                                                                                                          • Live webinar: Streamlining legal operations: Leveraging Zoho WorkDrive for law firm success

                                                                                                            Hello everyone, Managing legal documents across departments and jurisdictions can be complex, but it doesn’t have to be. Join us on March 6 for an exclusive webinar where we’ll show you how Zoho WorkDrive empowers legal teams to stay compliant, organized,
                                                                                                          • Live Webinar: Optimizing back-office operations in the manufacturing industry to maximize profitability

                                                                                                            Hello everyone, We’re excited to invite you to our upcoming live webinar on February 6! Discover how Zoho WorkDrive can help manufacturing businesses optimize back-office operations, improve efficiency, and boost profitability. Our product experts will
                                                                                                          • Live webinar: 2024 recap of Zoho WorkDrive

                                                                                                            Hello everyone, We’re excited to invite you to our year-end live webinar! This session will take you through the transformative features and updates we’ve introduced in Zoho WorkDrive this year, helping you streamline document management like never before.
                                                                                                          • Live webinar: Explore WorkDrive's seamless integrations with key Zoho apps

                                                                                                            Hello everyone, We’re excited to invite you to our upcoming live webinar, where we'll delve into the seamless integration of WorkDrive with other key Zoho applications! This is a fantastic opportunity to enhance your productivity and streamline your workflows
                                                                                                          • Live webinar: Getting the most out of WorkDrive in Zoho Workplace

                                                                                                            Hello everyone, We’re excited to invite you to our upcoming live webinar, where we’ll explore how to maximize your use of WorkDrive as part of the Zoho Workplace bundle. This is a fantastic opportunity to elevate your productivity and streamline your
                                                                                                          • Live webinar: Mastering data migration, organization, and team collaboration

                                                                                                            Hello everyone, We’re excited to invite you to our upcoming live webinar! Discover how to seamlessly migrate your data, optimize file organization, and boost team collaboration using Zoho WorkDrive’s powerful features. This is a fantastic opportunity
                                                                                                          • BARCODE PICKLIST

                                                                                                            Hello! Does anyone know how the Picklist module works? I tried scanning the barcode using the UPC and EAN codes I added to the item, but it doesn’t work. Which barcode format does this module use for scanning?
                                                                                                          • Join our live webinar: Explore the WorkDrive TrueSync application!

                                                                                                            Hello everyone, We are thrilled to invite you to a live webinar focused on mastering the WorkDrive TrueSync application. Discover how to seamlessly sync your content between the cloud and your computer, ensuring smooth and efficient file management. Our
                                                                                                          • Bug tracking

                                                                                                            Hi, does anyone know how to track errors during picking or packing? This way I can keep track and see how to improve and prevent errors in this area.
                                                                                                          • Can the Product Image on the Quote Template be enlarged

                                                                                                            Hello, I am editing the Quote Template and added ${Products.Product Image} to the line item and the image comes up but it is very tiny. Is there anyway that you can resize this to be larger? Any help would be great! Thanks
                                                                                                          • Zoho Creator customer portal limitation | Zoho One

                                                                                                            I'm asking you all for any feedback as to the logic or reasoning behind drastically limiting portal users when Zoho already meters based on number of records. I'm a single-seat, Zoho One Enterprise license holder. If my portal users are going to add records, wouldn't that increase revenue for Zoho as that is how Creator is monetized? Why limit my customer portal to only THREE external users when more users would equate to more records being entered into the database?!? (See help ticket reply below.)
                                                                                                          • Script Editor not an option

                                                                                                            I am trying to apply a script to a sheet and Script Editor is not an option. I don't want to go outside Sheets to do this (like Creator) if it can be done inside Sheets.
                                                                                                          • Send Automated WhatsApp Messages and Leverage the Improved WhatsApp Templates

                                                                                                            Greetings, I hope all of you are doing well. We're excited to announce a major upgrade to Bigin's WhatsApp integration that brings more flexibility, interactivity, and automation to your customer messaging. WhatsApp message automation You can now use
                                                                                                          • Envio de mails

                                                                                                            Hola! No puedo enviar mails pero si recibirlos. No se como solucionarlo! Mi dominio es chidobebes.com.ar
                                                                                                          • ERROR CODE :554 - Your access to this mail system has been rejected due to poor reputation of a domain used in message transfer

                                                                                                            In my email configuration: The domain's MX Records are pointed to Zoho The domain's SPF Records have been pointed out successfully DKIM is enabled. DMARC Record is pointed for the domain. The domain name is digioorja.in. Still facing the issue of Error:
                                                                                                          • This Operation has been restricted. Please contact support-as@zohocorp.com for further details

                                                                                                            l tried to verify my domain (casalimpaeperfumada.com.br) and its shows this error: This Operation has been restricted. Please contact support-as@zohocorp.com for further details.
                                                                                                          • SLOW EMAILS

                                                                                                            Is there an issue with the Zoho server? For two days now I've been having issues with very long buffering. Please advise. Thank you.
                                                                                                          • POP3 authentication error - SOLVED

                                                                                                            Just in case others are as forgetful as me ... As Zoho has changed the POP server for personal and free organisational users, I needed to change the POP server on my email client. This failed persistently but eventually I remembered that I had chosen
                                                                                                          • Solution to Import PST File into Office 365.

                                                                                                            MailsDaddy OST to Office 365 Migration Tool is an outstanding solution to recover OST files and migrate them into Office 365 without any hassle. Using this software users can multiple OST files into Office 365 with complete data security. It offers users
                                                                                                          • Add Zoho One Groups/Departments to Shared Mailbox Access

                                                                                                            Hi, I hope you're doing well. Currently, in Zoho Mail, I can manually add specific users or the entire organization to a shared mailbox. However, there is no option to add Zoho One groups or departments. Feature Request: We would like the ability to assign
                                                                                                          • Allow Filters with Only Special Characters in Zoho Mail

                                                                                                            Hi Zoho Mail Team, I hope you're doing well. We have noticed that currently, Zoho Mail does not allow creating filter criteria using only special characters, such as = or #. However, there are scenarios where such a filter is necessary. For example: Filtering
                                                                                                          • How to save email as PDF?

                                                                                                            I saw 2 previous threads about this. One is from 14 years ago. The other was closed as "answered" a year ago but the feature was never implemented: https://help.zoho.com/portal/en/community/topic/how-to-download-save-emails-as-pdf Is the "save as PDF"
                                                                                                          • Next Page