Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere







                          Zoho Developer Community






                                                • Desk Community Learning Series


                                                • Digest


                                                • Functions


                                                • Meetups


                                                • Kbase


                                                • Resources


                                                • Glossary


                                                • Desk Marketplace


                                                • MVP Corner


                                                • Word of the Day


                                                • Ask the Experts



                                                          • Sticky Posts

                                                          • Register for Zoho Desk Beta Community

                                                            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                          • Share your Zoho Desk story with us!

                                                            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                          • Tip #1: Learn to pick the right channels

                                                            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                          • Welcome to Zoho Desk Community - Say hello here!

                                                            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                          • Webinar 1: Blueprint for Customer Service

                                                            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                          Manage your brands on social media



                                                                Zoho TeamInbox Resources



                                                                    Zoho CRM Plus Resources

                                                                      Zoho Books Resources


                                                                        Zoho Subscriptions Resources

                                                                          Zoho Projects Resources


                                                                            Zoho Sprints Resources


                                                                              Qntrl Resources


                                                                                Zoho Creator Resources



                                                                                    Zoho CRM Resources

                                                                                    • CRM Community Learning Series

                                                                                      CRM Community Learning Series


                                                                                    • Kaizen

                                                                                      Kaizen

                                                                                    • Functions

                                                                                      Functions

                                                                                    • Meetups

                                                                                      Meetups

                                                                                    • Kbase

                                                                                      Kbase

                                                                                    • Resources

                                                                                      Resources

                                                                                    • Digest

                                                                                      Digest

                                                                                    • CRM Marketplace

                                                                                      CRM Marketplace

                                                                                    • MVP Corner

                                                                                      MVP Corner







                                                                                        Design. Discuss. Deliver.

                                                                                        Create visually engaging stories with Zoho Show.

                                                                                        Get Started Now


                                                                                          Zoho Show Resources


                                                                                            Zoho Writer Writer

                                                                                            Get Started. Write Away!

                                                                                            Writer is a powerful online word processor, designed for collaborative work.

                                                                                              Zoho CRM コンテンツ





                                                                                                Nederlandse Hulpbronnen


                                                                                                    ご検討中の方




                                                                                                          • Recent Topics

                                                                                                          • Kaizen #205 - Answering Your Questions | Managing Picklists and Enabling History Tracking via Zoho CRM APIs

                                                                                                            Hello everyone! Welcome back to another post in our Kaizen series. In this post, we will look at how you can manage picklist fields in Zoho CRM using APIs. This topic was raised as feedback to Kaizen #200, so we are taking it up here with more details.
                                                                                                          • Multiple Vendor SKUs

                                                                                                            One of the big concerns we have with ZOHO Inventory is lack of Vendor Skus like many other inventory software packages offer. Being able to have multiple vendor skus for the same product would be HUGE! It would populate the appropriate vendor Sku for
                                                                                                          • Internally created tickets

                                                                                                            Hi there When tickets are created internally on-behalf of customers - there is nothing to show that the ticket was created by an internal agent. This means, that it's easy for our agents to confuse tickets which were created by internal team members and
                                                                                                          • Automatically change website passwords

                                                                                                            Hi everyone, We just switched to a Professional package to also use the "Automatically change website passwords" function. But I cannot find anything about it, how to use it, anywhere. Does anyone know how I can use this function? Best, Caspar
                                                                                                          • Change Invoice Prices for an Effective Date

                                                                                                            Hi, It would be a really good feature to be able to change the prices on invoices/recurring invoices from an effective date in the event of price increases. For instance, I am in the process of increasing prices that will be effective from a specific
                                                                                                          • "Other Current Asset" accounts as "Paid Through" accounts in Expense

                                                                                                            It would be incredibly useful to be able to assign accounts of type Other Current Asset as Paid Through accounts in Expense. Currently, Other Current Liability are permitted as Paid Through Accounts. This makes sense, as Credit Cards are current liabilities.
                                                                                                          • Multi column open text questions that allows respondents to add rows for additional information

                                                                                                            I need to create a question that has 2 columns with open text, but I also need to allow respondents to click a "+" button, or something similar, so that they can add additional information if they choose to. I've tried using the Multiple Textboxes type
                                                                                                          • Bot Filtering & Apple Mail Privacy Protection Compliance in Zoho Campaigns

                                                                                                            Dear Campaigns Users, The wait is over! We’re excited to announce that the enhanced bot filtering feature is now live in Zoho Campaigns. This update brings greater accuracy to your email campaign reports by distinguishing real user engagement from automated
                                                                                                          • Découvrons les détails qui simplifient vos journées de travail avec Trident

                                                                                                            Nous nous installons dans des routines efficaces et rodées avec le temps. Chaque matin, nous ouvrons nos e-mails, passons aux messages, consultons notre agenda, puis attaquons nos tâches. Ce processus nous semble maîtrisé, mais est-il réellement optimisé
                                                                                                          • Issue with Purchase Rate Showing as “0” After Importing Items List

                                                                                                            Dear Zoho Books Support Team, Good day. I’m reaching out regarding an issue I’m facing while importing my items list into Zoho Books. Despite mapping all fields correctly and including the purchase price for each product in my Excel file, the Purchase
                                                                                                          • API for Task Entity in Zoho Books

                                                                                                            I’m working on automating task creation in Zoho Books via a custom button in the Bills Module. The goal is to create a task in the Tasks Module and assign it to the Finance Team, so they can track progress efficiently. While reviewing Zoho Books documentation,
                                                                                                          • create invoice in zoho books from the zoho forms

                                                                                                            Is there a native way to have create invoice in zoho books, when zoho form is completed?
                                                                                                          • Email undelivered

                                                                                                            GOod Day I am always receiving an uncategorized-bounce to my email. I am not sure why this is happening.
                                                                                                          • Add inventory_valuation_method to items endpooints

                                                                                                            To ensure consistent item creation it would be helpful to have the inventory_valuation_method (FIFO vs WAC) be able to be set at item creation or as an update (consistent with current behavior where it is not allowed for items with existing transactions)
                                                                                                          • Use Zoho to send sales receipts for Gocardless transactions

                                                                                                            I've been using gocardless for years and have d/d mandates set up on there. Each week we get bulk payments from customer d/d's. However, we need to send sales receipts to these customers. So I know I can sync mandates into Zoho, and then I can set up
                                                                                                          • Zoho - Gocardless sales receipts

                                                                                                            I've been using gocardless for years and have d/d mandates set up on there. Each week we get bulk payments from customer d/d's. However, we need to send sales receipts to these customers. So I know I can sync mandates into Zoho, and then I can set up
                                                                                                          • Introducing Rollup summary in Zoho CRM

                                                                                                            ------------------------------------------Moderated on 5th July'23---------------------------------------------- Rollup summary is now available for all organizations in all the DCs. Hello All, We hope you're well! We're here with an exciting update that
                                                                                                          • Introducing Connected Workflows in Zoho CRM for Everyone : Free Your Teams to Focus on What Matters

                                                                                                            Hello Everyone, We’re thrilled to introduce the next big evolution in Zoho CRM for Everyone -- Connected Workflows. This new feature builds on our commitment to deliver a CRM that’s truly inclusive, adaptable, and designed for consistent collaboration
                                                                                                          • Cooling-off Period Just Got Better: More Coverage, More Control

                                                                                                            We’ve enhanced the Cooling-off Period feature in Zoho Recruit to give you more control over repeat applications and referrals. This helps you maintain a cleaner, more efficient recruitment pipeline. With this enhancement, you can: Prevent duplicate candidate
                                                                                                          • Revenue Management: #7 Revenue Recognition in Construction & Real Estate Industry

                                                                                                            If you are in the construction or real estate business, you are used to long project timelines and progressive invoicing to keep up with your billing. But when does revenue get recognized? Will it happen when the contract gets signed? At different milestones
                                                                                                          • TikTok (and other social platform) Messages and comments of the past

                                                                                                            When I link a social channel, Zoho will show in "Inbox", "Messages" and "Contact" sections the interaction done in the past? (comment, messages...)
                                                                                                          • Restrict Employee mail deletion

                                                                                                            Dear Zoho, Is there a way where i can restrict my employees to delete any mails from their account
                                                                                                          • 554 5.1.8 Email Outgoing Blocked.

                                                                                                            Hi guys, I just singed up for mateusz.nowicki@zoho.com mail and I can't send any mails.. Why? Everytime I try to send something I got error like the one in the screenshot. Please, help me.
                                                                                                          • Zoho IP blocked by SpamHaus

                                                                                                            ERROR CODE :550 - 5.7.0 Your server IP address is in the SpamHaus SBL-XBL database, bye
                                                                                                          • File Upload in Creator's Subfrom

                                                                                                            Hello Sir/Madam, Here is a Problem......... Scenario: In CRM One Custom Module (Payments) have one File Upload Field now we have to Upload that File into Creator's Custom Form (Documents) have one Subform (Documents) in Document Upload Field using Deluge
                                                                                                          • Trigger workflow base on email clic

                                                                                                            Searching the help and forum, I see that there were workflow trigger rules based on email. But now, I can't find this type of trigger when I create a custom workflow. What I'm looking for would be to automate the sending of an email for a new prospect,
                                                                                                          • Bigin Form Acknowledgement

                                                                                                            How to troubleshoot and find out why form acknowledgement is not sending emails after form submission?
                                                                                                          • Option to Customize Career Site URL Without “/jobs/Careers”

                                                                                                            Dear Zoho Recruit Team, I hope you are doing well. We would like to request an enhancement to the Career Site URL structure in Zoho Recruit. In the old version of the career site, our URL was simply: 👉 https://jobs.domain.com However, after moving to
                                                                                                          • Zoho Mail POP & IMAP Server Details

                                                                                                            Hello all! We have been receiving a number of requests regarding the errors while configuring or using Zoho Mail account in POP/ IMAP clients. The server details vary based on your account type and the Datacenter in which your account is setup. Ensure
                                                                                                          • Ever since the new Android App udpates notifications are not working

                                                                                                            notifications are not working for the app is its closed I followed the tutuorial to the notificaction fixed and everythig seems to be right but notifications are not workig
                                                                                                          • Zoho Analytics & Zoho Desk - but not all desks

                                                                                                            I have several desks in our company and one of those is used by our HR department. I want to bring through the data to the shared Zoho Analytics workspace - except for the HR desk. Can this be excluded at data import stage ?
                                                                                                          • Incoming Emails Not Showing Up in Zoho Inbox

                                                                                                            Hi - I have my Zoho email account set up to forward a copy of all incoming emails to a secondary Gmail address, whilst retaining the original email in the Zoho inbox. However, all my incoming emails are currently not showing up in my Zoho inbox, so I'm
                                                                                                          • Unable to Create Zoho Booking via the Book Appointment API

                                                                                                            Its giving the below error {     "response": {         "errormessage": "Error setting value for the variable:customer_details\n null",         "status": "Error"     } } Request: POST Url: https://www.zohoapis.in/bookings/v1/json/appointment attached Zoho-oauthtoken
                                                                                                          • SHEET - Send email when a cell changes

                                                                                                            I would like to create a custom function for Zoho Sheet that triggers when a paticular cell changes to a specific value. This would result in sending an email to a recipient (this would be an address that remains the same and included in the script). Example: = IF(N4= "Drafted", <>EmailFunction) 1)     Cell N4 changes to "Drafted" 2)    Email is sent to recipient            or alternatively 3)    Post to chat channel I have found the Custom function editor in Sheet. I am not bad at scripting, but
                                                                                                          • 【開催報告】 福岡 ユーザー交流会 2025/8/8(金)

                                                                                                            皆さま、こんにちは。コミュニティチームの中野です。 8/8(金)に、福岡 ユーザー交流会を開催しました。 本投稿では、その様子をお届けします。当日の登壇資料などもこちらに共有しますので、参加できなかった皆さまもご参照ください。 今年初の開催となる福岡 ユーザー交流会では、CreativeStudio樂合同会社 前田さんによるZoho CRM / Sign / Survey の事例セッションのほか、 Zoho社員セッションでは、Zoho Forms の活用法を解説。 さらに、「見込み客・顧客データの管理/活用方法」をテーマに参加者同士でZoho
                                                                                                          • no me llegan los correos a Zoho mail

                                                                                                            No puedo recibir correos pero sí enviarlos, ya hice la modificación de MX y la verificación de teléfonos, qué es lo que ocurre? gracias
                                                                                                          • Error: Invalid login: 535 Authentication Failed

                                                                                                            I have used zoho with nodemailer. const transporter = nodemailer.createTransport({ host: 'smtp.zoho.com', port: 465, secure: true, auth: { user: 'example@example.com', pass: 'password' } }); While sending the mail, it shows the following error: Error:
                                                                                                          • Zoho Renewal

                                                                                                            Hello, If I am not going for zoho email renewal. will i get back my free zoho account? and if yes then is it possible to get back my all free user. how many user get back 10 or 25?
                                                                                                          • Not reciving emails

                                                                                                            Apparently i cannot recive emails on my adress contact@sportperformance.ro I can send, but do not recive. The mail i'm trying to send from mybother adress gets sent and doesn't bounce back... but still doesn't get in my inbox. Please advise
                                                                                                          • Not receiving MailChimp verification e-mail

                                                                                                            It seems that their verification e-mails are blocked. I can receive their other e-mails, but not their verification of domain ownership e-mail. I've checked and double checked how I typed the e-mail, using different e-mails (my personal e-mail can receive it), white listing the domain and all that is left is for the IP's to be white listed, but I don't have that power.  If a staff member could take a look at this -> http://mailchimp.com/about/ips/ And perhaps white list them for me, that would be
                                                                                                          • Next Page