Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    Access your files securely from anywhere







                          Zoho Developer Community






                                                • Desk Community Learning Series


                                                • Digest


                                                • Functions


                                                • Meetups


                                                • Kbase


                                                • Resources


                                                • Glossary


                                                • Desk Marketplace


                                                • MVP Corner


                                                • Word of the Day


                                                • Ask the Experts



                                                          • Sticky Posts

                                                          • Register for Zoho Desk Beta Community

                                                            With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features
                                                          • Share your Zoho Desk story with us!

                                                            Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 
                                                          • Tip #1: Learn to pick the right channels

                                                            Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 
                                                          • Welcome to Zoho Desk Community - Say hello here!

                                                            Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here
                                                          • Webinar 1: Blueprint for Customer Service

                                                            With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing


                                                          Manage your brands on social media



                                                                Zoho TeamInbox Resources



                                                                    Zoho CRM Plus Resources

                                                                      Zoho Books Resources


                                                                        Zoho Subscriptions Resources

                                                                          Zoho Projects Resources


                                                                            Zoho Sprints Resources


                                                                              Qntrl Resources


                                                                                Zoho Creator Resources



                                                                                    Zoho CRM Resources

                                                                                    • CRM Community Learning Series

                                                                                      CRM Community Learning Series


                                                                                    • Kaizen

                                                                                      Kaizen

                                                                                    • Functions

                                                                                      Functions

                                                                                    • Meetups

                                                                                      Meetups

                                                                                    • Kbase

                                                                                      Kbase

                                                                                    • Resources

                                                                                      Resources

                                                                                    • Digest

                                                                                      Digest

                                                                                    • CRM Marketplace

                                                                                      CRM Marketplace

                                                                                    • MVP Corner

                                                                                      MVP Corner







                                                                                        Design. Discuss. Deliver.

                                                                                        Create visually engaging stories with Zoho Show.

                                                                                        Get Started Now


                                                                                          Zoho Show Resources


                                                                                            Zoho Writer Writer

                                                                                            Get Started. Write Away!

                                                                                            Writer is a powerful online word processor, designed for collaborative work.

                                                                                              Zoho CRM コンテンツ





                                                                                                Nederlandse Hulpbronnen


                                                                                                    ご検討中の方




                                                                                                          • Recent Topics

                                                                                                          • Try FSM again for our business

                                                                                                            We already have our customers individual equipment in CRM with serial numbers, install dates, warranty length and importantly next service which is generally 2 years. a month before the service date is due we get get a report and send out service reminders.
                                                                                                          • Use Zoho Books to bill for work done in Zoho Desk??

                                                                                                            I'm trying to see if something is possible (and if yes, how). We use Zoho One to manage our business. We have a lot of clients that will put in a ticket (via portal) to have work done. Out techs will pick up the ticket, do the work, and then log the time
                                                                                                          • Get Cliq Meetings in my O365 calendar

                                                                                                            Hi, we are currently evaluating to replace the Teams Messaging and Meetings with Cliq. We currently still have all our email and calendars in O365. What i want to achieve is, to create a (ZOHO) meeting from Cliq and have this meeting added to my Outlook/O365
                                                                                                          • Issue with Zoho Help Portal – Tickets Missing or Not Answered

                                                                                                            Hi, How are you? I think there may be an issue with the Zoho Help Portal. I opened a few tickets directly in the help portal a some time ago but never received any response I also opened ticket 148356451 by email. I did receive a reply to it, but the
                                                                                                          • No bank feeds from First National Bank South Africa since 12 September

                                                                                                            I do not know how Zoho Books expects its customers to run a business like this. I have contacted Zoho books numerous times about this and the say it is solved - on email NO ONE ANSWERS THE SOUTH AFRICAN HELP LINE Come on Zoho Books, you cannot expect
                                                                                                          • DUNS & Bradstreet and Credit risk monitoring integration with Zoho books

                                                                                                            Small businesses not being paid by bigger clients and clients of all sizes is a huge problem. It will be nice if Zoho develops integration with DUNS & Bradstreet(D&B) and Credit risk monitoring integration with Zoho books. That have small businesses can
                                                                                                          • Zoho Forms - Form Rules based on attachment fields

                                                                                                            Many businesses use forms to collect documents and images from customers. In many cases, you may want to trigger a notification or other automation based on whether or not an attachment was added. I've noticed that attachment fields do not appear in Rules
                                                                                                          • How to sync from Zoho Projects into an existing Sprint in Zoho Sprints?

                                                                                                            Hi I have managed to integrate Zoho Projects with Zoho Sprints and I can see that the integration works as a project was created in Zoho Sprints. But, what I would like to do is to sync into an existing Zoho Sprints project. Is there a way to make that
                                                                                                          • Add multiple users to a task

                                                                                                            When I´m assigning a task it is almost always related to more than one person. Practical situation: When a client request some improvement the related department opens the task with the situation and people related to it as the client itself, the salesman
                                                                                                          • Can't open draft email for editing

                                                                                                            Last night I started composing an email and I let it save in drafts. This morning I want to continue working on the email. It is in my Drafts folder but it will not open. The only option there is to delete it. This is not the first time it has happened. On previous occasions I have just deleted the draft and started afresh, but I really want this one back. Windows 10 with Pale Moon 28.10.0 browser.
                                                                                                          • Failing to generate Access and Refresh Token

                                                                                                            Hello.  I have two problems: First one when generating Access and Refresh Token I get this response:  As per the guide here : https://www.zoho.com/books/api/v3/#oauth (using server based application) I'm following all the steps. I have managed to get
                                                                                                          • Open filtered deals from campaign

                                                                                                            Do you think a feature like this would be feasible? Say you are seeing campaign "XYZ" in CRM. The campaign has a related list of deals. If you want to see the related deals in a deal view, you should navigate to the Deals module, open the campaign filter,
                                                                                                          • Change scheduling emails time

                                                                                                            When sending an individual email there is a great feature to schedule them to send later. I could only use the one time that is suggested. Is there a way to select another time? Regards, Glenn
                                                                                                          • Zoho CRM: how can I control which contacts to sync with Outlook?

                                                                                                            I was just playing around syncing contacts from Zoho to MS Outlook (MS365 account.) The problem is our firm has hundreds of thousands of contacts and I don't want to bury my contacts list in outlook. Any help with this is greatly appreciated.
                                                                                                          • How to overcome limitations in meetings

                                                                                                            As a company, one of our deliverables is a meeting between two other companies, where we act as facilitators. So, if we recorded this meeting  in Zoho CRM, it should be connected to 2 accounts, 2 contacts, and 1 campaign (a campaign, in our use, is the
                                                                                                          • Different MRP / Pricing for same product but different batches

                                                                                                            We often face the following situations where MRP of a particular product changes on every purchase and hence we have to charge the customer accordingly. This can't be solved by Batch tracking as of now so far as I understand Zoho. How do you manage it as of now? 
                                                                                                          • Add a 'Log a Call' link to three dot icon in Canvas

                                                                                                            Hi, There's a three dot element when creating a canvas called 'More'. I would like to modify this to add a link that says 'Log a Call' in order to quickly record the details of a cellphone call. I'd also like this to be a simple 'contact' selection and
                                                                                                          • Syncing Zoho Forms with Bigin - Embedding issue?

                                                                                                            Hello everyone, I created a Zoho Form for a page on my GoDaddy website to collect leads, which then transfers the data to Bigin. However, I'm facing an issue where it doesn't seem to work properly. I've integrated Zoho Forms with Bigin and tried embedding
                                                                                                          • Can not add fields to a Section

                                                                                                            I feel like I'm missing something obvious: I can add new Sections to my form but I can not add fields to the Sections. I've tried fields already on the form as well as dragging and dropping new fields into the Section but nothing will go into it. What
                                                                                                          • Record Logged in User while using CRM lookup field

                                                                                                            Is it possible, while using the Zoho CRM lookup field, to automatically use the user account logged into Zoho CRM in a hidden field? I was hoping to add employee accounts to my current plan. But would like a record on the Form submission of who submitted
                                                                                                          • Form Rules for Suburb Categories to alternate landing pages or Making a Fields Contents ALL CAPS

                                                                                                            I need to send differentform submissions to two to three different thank-you URLs (for Meta/Google pixels) depending on which suburb a user selects in a form. I have ~400 suburbs split into two categories (A and B, based on business value). Current challenges:
                                                                                                          • Collaps Notes

                                                                                                            There are times when long/large notes are added to a record i.e. Accounts or Deals etc. Currently, the full note is displayed in the notes related list section. It would be great if by default only 5 to 10 rows of the note are displayed when the note
                                                                                                          • Zoho Down

                                                                                                            I have a drop in my Zoho One services.
                                                                                                          • Runing RPA Agents on Headless Windows 11 Machines

                                                                                                            Has anyone tried this? Anything to be aware of regarding screen resolution?
                                                                                                          • Problem for EU users connecting Zoho CRM through Google Ads for Enhanced conversions

                                                                                                            Has anyone else experienced this problem when trying to connect Zoho CRM through Google Ads interface to setup enhanced conversions? Did you guys get it fixed somehow? The Problem: The current Google Ads integration is hardcoded to use Zoho's US authentication
                                                                                                          • Why am I getting event Pop-up Notification for events that have been cancelled?

                                                                                                            Why is Calendar Notification still popping up for events that have been cancelled or changed? Each time events are cancelled or changed, I have observed that I am still getting notifications for them. Below is a sample pop-up notification for one of the
                                                                                                          • Whatsapp Limitation Questions

                                                                                                            Good day, I would like to find out about the functionality or possibility of all the below points within the Zoho/WhatsApp integration. Will WhatsApp buttons ever be possible in the future? Will WhatsApp Re-directs to different users be possible based
                                                                                                          • Create a draft in reply to an email via Emails API

                                                                                                            Hi, I’d like to use the outgoing webhook to automatically create a draft reply to incoming mail. How can I use the Emails API to create a draft reply that is linked to an existing email thread? I couldn’t find the relevant method in the documentation.
                                                                                                          • India Tech Support

                                                                                                            Is there no phone tech support number for India? And no chat facility either?
                                                                                                          • Billing Management: #1 Billing a Universal Business Routine

                                                                                                            Hello, As the saying goes, "Do the hardest job first." We started with the complex subject of finance and revenue management, which is considered the backbone of any business. Now, let's shift our focus and take a deep dive into this Billing Management,
                                                                                                          • Show/ hide specific field based on user

                                                                                                            Can someone please help me with a client script to achieve the following? I've already tried a couple of different scripts I've found on here (updating to match my details etc...) but none of them seem to work. No errors flagged in the codes, it just
                                                                                                          • What is a a valid JavaScript Domain URI when creating a client-based application using the Zoho API console?

                                                                                                            No idea what this is. Can't see what it is explained anywhere.
                                                                                                          • 5名限定 課題解決型ワークショップイベント Zoho ワークアウト開催のお知らせ (9/25)

                                                                                                            ユーザーの皆さま、こんにちは。Zoho ユーザーコミュニティチームの藤澤です。 9月開催のZoho ワークアウトについてお知らせします。 今回はZoomにて、オンライン開催します。 諸事情につき、今月の開催回は中止となりました。 次回は10/31(金)14時からの開催を予定しています。 ━━━━━━━━━━━━━━━━━━━━━━━━ Zoho ワークアウトとは? Zoho ユーザー同士で交流しながら、サービスに関する疑問や不明点の解消を目的とした「Zoho ワークアウト」を開催します。 Zoho
                                                                                                          • Zoho Calendar not syncing correctly with personal Google Calendar

                                                                                                            Coming to this forum as Zoho Calendar support team is not responding, any more. For the past 8 weeks, I have been having an issue with Zoho Calendar not syncing with my personal Google Calendar correctly. I subscribed to Zoho Calendar iCal in my personal
                                                                                                          • Introducing Assemblies and Kits in Zoho Inventory

                                                                                                            Hello customers, We’re excited to share a major revamp to Zoho Inventory that brings both clarity and flexibility to your inventory management experience! Presenting Assemblies and Kits We’re thrilled to introduce Assemblies and Kits, which replaces the
                                                                                                          • Customer Parent Account or Sub-Customer Account

                                                                                                            Some of clients as they have 50 to 300 branches, they required separate account statement with outlet name and number; which means we have to open new account for each branch individually. However, the main issue is that, when they make a payment, they
                                                                                                          • need a packing list feature

                                                                                                            In our business, goods listed on an invoice are packed in separate boxes and shipped off. for e.g. an invoice may have 10 items. each item could then be packed in different boxes depending on qty of each item. this packing list is as important as the invoice for purposes of shipping documents.  Request you to add this feature asap.
                                                                                                          • Prevent stripping of custom CSS when creating an email template?

                                                                                                            Anyone have a workaround for this? Zoho really needs to hire new designers - templates are terrible. A custom template has been created, but every time we try to use it, it strips out all the CSS from the head.  IE, we'll define the styles right in the <head> (simple example below) and everything gets stripped (initially, it saves fine, but when you browse away and come back to the template, all the custom css is removed). <style type="text/css"> .footerContent a{display:block !important;} </style>
                                                                                                          • Workdrive 5.0 / API Documentation Workflows

                                                                                                            Hi Zoho, When will the API documentation of the workflows be published? We are interested in using it to trigger manual workflows from an external application. Greetings, Justin
                                                                                                          • How to keep track of bags, cans, drums of inventory?

                                                                                                            We buy and sell products that are packaged in bags 🛍️, cans🥫, drums🛢️, etc. with batch numbers. When we get a shipment of one of the products, how do we track we received (say) 10 cans each of 5L of a product and maybe we received 10 cans of another
                                                                                                          • Next Page