Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere








                    Zoho Campaigns Resources


                          Zoho Developer Community



                              Bootcamp
                              Community
                              Pricing
                              Webinars
                              What's New
                              Integrations
                              Contact Support
                              Contact Sales

                                Zoho Survey Resources
                                  Features
                                  Survey Templates
                                  Integrations
                                  Community
                                  iOS Mobile Surveys
                                  Pricing
                                  Android Mobile Surveys
                                  Insurvey Blogs
                                  Insight Blogs
                                  Tips & Tricks






                                              Latest Feature Updates

                                                • Desk Community Learning Series


                                                • Digest


                                                • Functions


                                                • Meetups


                                                • Kbase


                                                • Resources


                                                • Glossary


                                                • Desk Marketplace


                                                • MVP Corner


                                                • Word of the Day


                                                • Ask the Experts




                                                    Zoho SalesIQ Resources

                                                          • Sticky Posts

                                                          • Register for Zoho Desk Beta Community

                                                            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

                                                          • Share your Zoho Desk story with us!

                                                            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

                                                          • Tip #1: Learn to pick the right channels

                                                            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

                                                          • Welcome to Zoho Desk Community - Say hello here!

                                                            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

                                                          • Webinar 1: Blueprint for Customer Service

                                                            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                          Manage your brands on social media
                                                          Access Zoho Social

                                                            Zoho Pagesense Resources


                                                                Zoho TeamInbox Resources

                                                                  Zoho DataPrep Resources


                                                                    Zoho CRM Plus Resources

                                                                      Zoho Books Resources


                                                                        Zoho Subscriptions Resources

                                                                          Zoho Projects Resources


                                                                            Zoho Sprints Resources


                                                                              Qntrl Resources


                                                                                Zoho Creator Resources



                                                                                    Zoho CRM Resources

                                                                                    • CRM Community Learning Series

                                                                                      CRM Community Learning Series


                                                                                    • Kaizen

                                                                                      Kaizen

                                                                                    • Functions

                                                                                      Functions

                                                                                    • Meetups

                                                                                      Meetups

                                                                                    • Kbase

                                                                                      Kbase

                                                                                    • Resources

                                                                                      Resources

                                                                                    • Digest

                                                                                      Digest

                                                                                    • CRM Marketplace

                                                                                      CRM Marketplace

                                                                                    • MVP Corner

                                                                                      MVP Corner







                                                                                        Design. Discuss. Deliver.

                                                                                        Create visually engaging stories with Zoho Show.

                                                                                        Get Started Now


                                                                                          Zoho Show Resources


                                                                                            Zoho Writer Writer

                                                                                            Get Started. Write Away!

                                                                                            Writer is a powerful online word processor, designed for collaborative work.

                                                                                              Zoho CRM コンテンツ





                                                                                                Nederlandse Hulpbronnen


                                                                                                    ご検討中の方




                                                                                                          • Recent Topics

                                                                                                          • Zoho CRM Analytics - Allow To Reorder Dashboards

                                                                                                            I would like to suggest that you add the ability to reorder dashboards in the Analytics Module. I can see that this has been requested some time ago, the latest 9 years ago. I am not sure if this is a big or small endeavor, but such a small fix can go

                                                                                                          • Territory view for custom modules?

                                                                                                            I have recently activated territories however I can't seem to find how to use territories for custom modules? These modules have territories:  Contacts /  Accounts / Opportunities These modules don't have territories:   Buildings (custom module) and 

                                                                                                          • Zoho Books - How to Invoke a Custom Function in Schedulers

                                                                                                            We have multiple schedulers that send emails to customers in batches. Currently, we are maintaining the same code across several schedulers. Is it possible to use a custom function inside a scheduler script? If yes, how can we invoke the custom function

                                                                                                          • Approval Process Comments

                                                                                                            Is it possible to view the comments entered during the approval or rejection of a record in the approval process? If not, is there a way to require a field to be completed upon approval or rejection?

                                                                                                          • Login for test case

                                                                                                            Had a few questions regarding authentication test cases and couldn't find an answer in the the docs. 1. If an app like Zoho Creator requires authentication before providing access, do I need to create a login function and add it to each test case? 2.

                                                                                                          • Zoho / Outlook Calendar sync

                                                                                                            The current Marketplace -> Microsoft -> Meetings integration needs 2 changes. 1. The current language for the Two-Way sync option should be changed. It currently states, "Sync both your Zoho CRM Calendar and Office 365 Calendar meetings with each other."

                                                                                                          • Please make it easier to Pause syncing

                                                                                                            right now it takes 3 clicks to get there. sounds silly, but can you make it just 2 clicks to get it done instead? thats how dropbox does it, 2 clicks to pause instead of 3.

                                                                                                          • Customer members area

                                                                                                            Does FSM support a customer members area? If not what do you propose we use if we want the data used in FSM for customers to give them an area / login to see past orders, create new orders and general announcements.

                                                                                                          • Zoho Books-Accounting on the Go Series!

                                                                                                            Dear users, Continuing in the spirit of our 'Function Fridays' series, where we've been sharing custom function scripts to automate your back office operations, we're thrilled to introduce our latest initiative – the 'Zoho Books-Accounting on the Go Series'.

                                                                                                          • Desktop app doesn't support notecards created on Android

                                                                                                            Hi, Does anybody have same problem? Some of last notecards created on Android app (v. 6.6) doesn't show in desktop app (v. 3.5.5). I see these note cards but whith they appear with exclamation mark in yellow triangle (see screenshot) and when I try to

                                                                                                          • Notes created in mobile can no longer be accessed in desktop

                                                                                                            Working with a 2013 Mac running OS 10.14.6; Desktop Notebook version 4.5.3. Using Motorola Moto G Power 5G - 2024; Android app version 6.7 I have been using Notebook for some years. Starting several weeks ago, the notes newly created ion the phone can

                                                                                                          • Function #49: Manage varying installment payments using Zoho Books

                                                                                                            . Hello everyone, and welcome back to our series! Last week, we discussed automating the collection of fixed installment payments in Zoho Books. But what if your payment structure involves charging varying percentages of the invoice total as installments?

                                                                                                          • Zoho Writer - Option to Export as .zdoc format

                                                                                                            I've noticed that it's not possible to export a Zoho Writer Document in the .zdoc format. Isn't zdoc, Zoho Writer's own format? My use case is that I sometimes need to create quite complex documents with floating elements, which sometimes need to become

                                                                                                          • Is it possible for contacts to "Re-enter" a workflow in Zoho Campaign?

                                                                                                            We are currently working on a way to automatically add users to from one list to other lists based on specific criteria, but can't seem to find a native way of doing this so we are trying to use Workflows to do this. So, for example, if a user's status is set to "Active," then they should be added to the list "Active Users." If the same user's status is then set to "Paused," they should be added to the list "Paused Users" and removed from the list "Active Users." This works fine for the first go

                                                                                                          • Bulk upload images and specifications to products

                                                                                                            Hi, Many users have asked this over the years and I am also asking the same. Is there any way in which we can bulk upload product (variant) images and product specifications. The current way to upload/select image for every variant is too cumbersome.

                                                                                                          • Out of Office for Just One of My Alias Email

                                                                                                            Can I set up the Out of Office Reply for Just One of my Alias Email Addresses?

                                                                                                          • Can I map multiple Surveys into the CRM using the same fields?

                                                                                                            Hello, We are a healthcare practice that offers two distinct services (Nutrition and Primary Care). We use Zoho Survey for our lead generation form (Get Started Survey), which allows people to express interest in one of the two services and even allows

                                                                                                          • Dealing with API responses where integers have more than 16 digits

                                                                                                            Hi there How do I deal with an api response contaning an int or float with more than 16 digits (before any decimal places for a float). I constantly receive the response "Unable to cast the 'BigInteger' value into a 'BIGINT' value because the input is

                                                                                                          • Can't change form's original name in URL

                                                                                                            Hi all, I have been duplicating + editing forms for jobs regarding the same department to maintain formatting + styling. The issue I've not run into is because I've duplicated it from an existing form, the URL doesn't seem to want to update with the new

                                                                                                          • Need Inactive accounts to be visible in Reports in Zoho Books

                                                                                                            I N=need Inactive accounts to be visible in Reports in Zoho Books to do recons of the accounts but when i see the same they are not visible in the Accountant - Account Transactions report

                                                                                                          • unblock e-mail

                                                                                                            please unblock my e-mails info@meatnews.gr and myrtokaterini@meatnews.gr

                                                                                                          • Add Zoho Mail for users who do not need Zoho One

                                                                                                            We have licenses for ZOho One for teams that need to use the suite of products that Zoho One offers.  We have 8 more people who only need email access and we would like to add just a Zoho Mail.  They do not need the Zoho One license.  We are currently

                                                                                                          • ZML vs HTML Snippet - which is better?

                                                                                                            Are there certain use cases where one is better than the other?

                                                                                                          • Auto CheckOut Based On Shift.

                                                                                                            This Deluge script runs on a scheduled basis to automatically set the 'Actual_Check_Out' time for employees who haven't manually checked out. If the current time is past their scheduled 'Shift_End_Time', the script updates the check-out time to match

                                                                                                          • How to remove some users in zoho accounts

                                                                                                            How to remove some users in Zoho accounts.

                                                                                                          • Infinite loop of account verification

                                                                                                            Hi I can't do anything on my zoho account.  I always get this message Hi Sheriffo Ceesay As a security measure, you need to link your phone number with this account and verify it to proceed further. When ever I supply the details, it displays that the number is associated with another account. I don't have any other account on zoho so this is really annoying. 

                                                                                                          • Load PO_Date field (Purchase Order) with current date in Deluge

                                                                                                            Hi, I'm not a full time developer, just helping to customize our CRM, in the small company I work for. There must be something wrong with me, because I can't do something so simple as complete a field with the current date in a function using Deluge.

                                                                                                          • Zoho CRM in Microsoft Power Automate Custom Connector

                                                                                                            Hi everyone, I’m building a Power Automate flow that integrates Microsoft Bookings with Zoho CRM. The goal is to automatically create a meeting (event) in Zoho CRM whenever a new appointment is booked via Microsoft Bookings. To achieve this, I created

                                                                                                          • Spell check sucks

                                                                                                            Come on guys, it's 2024 and your spell check is completely retarded. You gotta fix it.

                                                                                                          • How to include total km for multiple trips in expense report.

                                                                                                            Whenever I create a mileage report it only shows the total dollar amount to be reimbursed. The mileage for each individual trip is included but I also need to see the total distance for all trips in a report? How do I do this?

                                                                                                          • Outgoing blocked: Unusual activity detected.

                                                                                                            I just made payment for my Zohomail Today and have been debited so i will like to be Unblocked because this is what it says (Outgoing blocked: Unusual activity detected) Thank you i await your swift responses

                                                                                                          • Zoho One Login Issue - Unable to receive OTP

                                                                                                            Hi Support Team, I am experiencing a unique login issue with Zoho One. I am attempting to log in from India using Zoho Login credentials provided by a USA-based client. Their Zoho account is hosted on a US data center. After entering the username and

                                                                                                          • Question Regarding Managing Sale Items in Zoho Books

                                                                                                            Good day, I was wondering about something. Right now, Zoho Books doesn’t seem to have a way to flag certain items as being on sale. For example, if I want a list of specific items to be on sale from October 1 to October 12, the user would have to export

                                                                                                          • In the Zoho Creator Customer Payment form i Have customer field on select of the field Data want to fetch from the invoice from based on the customer name In the Customer Payment form i Have subf

                                                                                                            In the Zoho Creator Customer Payment form i Have customer field on select of the field Data want to fetch from the invoice from based on the customer name In the Customer Payment form i Have subform update Invoice , there i have date field,Invoice number

                                                                                                          • Problem of Import Client Users From CRM and or Expense

                                                                                                            I am premium plan user on Projects. I have about 500 customers on Expense and CRM that integrated with each other. According to at below link, I am trying to import clients from CRM, system not allowed to select any customer. If I import from Expense,

                                                                                                          • Unable to see free plan option

                                                                                                            Hello Zoho Support Team, I hope you are doing well. I am trying to sign up for Zoho Mail, but I am unable to see the option for the free plan. Could you please guide me on how I can access or activate the free plan? Thank you for your assistance.

                                                                                                          • unblock my zoho mail account. outlines@zoho.com

                                                                                                            please unblock my zoho mail account, outlines@zoho.com

                                                                                                          • domain not verified error

                                                                                                            Hi when i try to upload a video from zoho creator widget to zoho work drive iam getting domain not verified error.I don't know what to do .In zoho api console this is my home page url https://creatorapp.zoho.com/ and this is my redirect url:www.google.com.Iam

                                                                                                          • equest to Disassociate Bigin from Zoho One and Migrate to Standalone (Upgrade to Bigin Premier – 3 Seats, Annual)

                                                                                                            Dear Zoho One Support Team, I’m writing to request your assistance to disassociate (remove) the Bigin application from our Zoho One organization while preserving all existing Bigin data. After the disconnection is successfully completed, we intend to

                                                                                                          • SMTP email sending problem

                                                                                                            Hello, I've sent emails before, but you haven't responded. Please respond. My work is being disrupted. I can't send emails via SMTP. Initially, there were no problems, but now I'm constantly receiving 550 bounce errors. I can't use the service I paid

                                                                                                          • Next Page