Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Spotlight series #6: The Show app for Android TV has a new look!

      Hello everyone! We are delighted to introduce our revamped and redesigned Show app for Android TV.  Smart TVs are exploding in popularity. Android TV alone has over 110 million active monthly devices. Zoho Show, as part of a constant effort to improve

    • Employee Appraisal Applicability - Why is Date of Joining Hard-Coded?

      In the new (to me, at least) Performance Appraisal Cycle wizard, it's possible to set criteria to determine for whom the appraisal process should apply. This makes sense on its face. However, one MUST use the Date of Joining criterion as a filter. Why

    • WorkDrive for Excel Add on

      Dear Sir/Madam Have installed Workdrive for Microsoft add on But unable to view the same added in Excel

    • CRM x WorkDrive: We're rolling out the WorkDrive-powered file storage experience for existing users

      Release plan: Gradual rollout to customers without file storage add-ons, in this order: 1. Standalone CRM 2. CRM Plus and Zoho One DCs: All | Editions: All Available now for: - Standalone CRM accounts in Free and Standard editions without file storage

    • Zoho Books integration sync from Zoho CRM does not work

      Hi Zoho Community & Zoho Support We just tried to get a sync some products into Zoho Books from CRM using the native sync and we're getting an error: "It looks like some mandatory fields you're trying to map are empty. Please provide valid field names

    • Appraisals - Order of Previous Reviews Should Be Newest to Oldest

      The new Zoho People layout generally does a decent job at providing the necessary information for performing appraisals of employee performance. One example of this is the Previous Reviews section: This information is helpful when conducting a review.

    • Is it possible to remove filtering options?

      My CRM has a lot of custom fields that should not be used in filters or views. Fields that are automated and exist only to store temporary values that get used in functions. These create a lot of noise in the list of fields to filter. Isn't there any

    • auto close automated alert tickets which are similar

      Hello ZOHO Community, we are using ZOHO Desk to process automated monitoring alerts. Scenario: Our monitoring system creates a ticket when a threshold is exceeded, e.g. Subject: Computer 1 – CPU usage 100% – Error A few minutes later, once the issue resolves

    • Paging through API results. a major gap in your documentation.

      There is no way for me. to get all of my data through a single API call. Typically REST APis have mechanisms for paging through API results. But the documentation for the API I am using: https://desk.zoho.com/DeskAPIDocument#Introduction Has no mention

    • RouteIQ for Zoho FSM

      Beste, Zou wel top zijn dat we een RouteIQ hebben voor FSM aangezien we constant moeten zien wat de beste route is voor onze monteurs. Nu moeten we een speciale aparte programma hebben om de beste route te berrekenen voor onze monteurs aangezien de planning

    • How do people handle using Outlook and Zoho Project calendar at the same time?

      We have an ongoing problem in our organisation where we use Zoho Projects to plan all of our projects tasks and that also allows us to look forward using the workload report to see which of our consultants are overstretched etc and which are available.

    • Knowledgebase SEO

      We have a custom-domain mapped help center that is not restricted via login. I have some questions: a) will a robots.txt file still allow us to control indexing? b) do we have the ability to edit the sitemap? c) do category URLs get indexed by search

    • Please, make writer into a content creation tool

      I'm tired of relying on Google Docs. I'm actually considering moving to ClickUp, but if Writer were a good content creation tool instead of just a word processor, I would finally be able to move all my development within the Zoho ecosystem, rather than

    • Changing field types

      Question im a Zoho CRM user and curious, if I change a flied type from single or multi line text to a URL field type will i lose the data in the field or will it be converted to website link automatically. Thanks

    • How do I associate pricebooks to a customer?

      I setup a few pricebooks, that worked fine. But now the only thing I can do with it, when I enter a quote or sales order, I can select which pricebook to use, but I have to do this product by product every time I add one. Is there a way to connect a pricebook

    • Write-Off multiple invoices and tax calculation

      Good evening, I have many invoices which are long overdue and I do not expect them to be paid. I believe I should write them off. I did some tests and I have some questions:  - I cannot find a way to write off several invoices together. How can I do that,

    • Leave Report Emailed Weekly

      I am wondering if someone knows how to have a report generated either weekly or monthly or both for department heads and ownership of upcoming employee leave. For instance, it would be nice to get an emailed report on Friday for the upcoming week of who

    • Zoho desk desktop application

      does zoho desk has a destop applicaion?

    • Tip #54- Exploring the Files Icon in Zoho Assist- 'Insider Insights'

      As we’re already in mid-December, it’s a good time to take a closer look at one of the most useful options in the Zoho Assist remote support dashboard—the Files icon. To get started, log in to Zoho Assist using your credentials. Once you’re in, navigate

    • Tip#47: Estimation planner

      Sprint planning becomes easier, smoother, more collaborative and more accurate with the Estimation Planner extension. Most work items involve multiple users, and each user's role and effort vary. To provide an unbiased and fair estimation point to the

    • Painfully Slow Zoho mail

      Since yesterday Zoho Mail seems to have starting functioning very slowly and having a few bugs. It's slow to open mails, slow to send, slow to change between email accounts. Sometimes clicking on a particular folder (eg Sent folder) stops working and

    • Upcoming update to Google Drive integration in Zoho Creator

      Hello everyone, We're writing to inform you about an upcoming update to how Zoho Creator integrates with Google Drive. In Zoho Creator, during actions like importing files to create an app or attaching files to fields, you can choose files directly from

    • Changing Default PDF Name

      Is it possible to change the default name of a PDF? As of right now, all of my quotes are named 'QT_$QuoteNumber' (i,e: 'QT_19803471298374) - would it be possible to change this to: '$CompanyName - $AccountName - $QuoteNumber' for instance?

    • Time Tracking on Iphone doesn't stop

      When I start a time tracking session the timer starts as expected. However when I hit stop, the timer remains on that project. The only buttons available are discard and start. Start runs the timer more and discard says it will throw away the data from

    • Zoho Sign - Zoho CRM extension upgrade

      Hi everyone, We've updated Zoho Sign extension for Zoho CRM with significant internal changes. Impact on existing Zoho Sign extension users Users using the extension without customization If you are using the integration without implementing Zoho Sign's

    • Zoho Books blocks invoicing without VeriFactu even though it is not mandatory until 2027

      I would like to highlight a very serious issue in Zoho Books for Spain. 1. The Spanish government has postponed the mandatory start of VeriFactu to January 1st, 2027. This means that during all of 2026 businesses are NOT required to transmit invoices

    • Group by Owner/ Owner Kanban /Group by Custom Field

      Hi, We are missing the option of viewing Tasks grouped by their owner, as it is an essential function for us to manage our employees' working tasks. We find it hard to inspect what are the tasks that our employees are working on.  We are seeking an option

    • Flow using way too many tasks

      I built a flow that is using way too many tasks for what it is doing. I reached out to support a week ago but never heard back so hoping the community can help. Does a filter count as a task? I am seeing tons of records filtered with < 1 Sec Time Taken

    • Regarding the integration of Apollo.io with Zoho crm.

      I have been seeing for the last 3 months that your Apollo.io beta version is available in Zoho Flow, and this application has not gone live yet. We requested this 2 months ago, but you guys said that 'we are working on it,' and when we search on Google

    • Start Workflow from Deluge Script

      I have developed a customized process from our CRM that leverages a deluge script to create a statement of work document. Once the document has been created via the merge and store function, I would like the ability to start a workdrive review & approve

    • Saved Sections?

      In sites editor, the + button reveals options to add a section, element, etc. It includes Saved Sections, but I can find no way to save a section I've already created. Otherwise, is there a way to copy a section from one page to another?

    • Marketer's Space: Unable to select a sender address? Here's the fix

      Hello Marketers, Welcome back to yet another post! So you've added a sender address in Zoho Campaigns and later found that you're not able to select it while creating an email campaign. Has this ever happened to you? If so, don't worry—you're not alone.

    • Loading Project Balances in ZOHO Books for each project

      Hello, What is the best method for loading project balances actual and budget into ZOHO books to provide tracking to our project managers. We have projects and federal awards (also treated as projects) which span multiple years. We are converting from

    • Auto-Invite Users to Portals in Zoho CRM based on Conditions

      Hello Everyone, You can now automate portal invitations in Zoho CRM with the new Auto-Invite users feature in Portal management. No more manually enabling portal access one by one. With this enhancement, you can automatically send invites for users to

    • Scale up your writing style with content analysis tool in Knowledge Base

      Hello everyone, Sometimes writers may have strong knowledge about the subject but may find it challenging to articulate clearly or produce comprehensible, jargon free content. To make the process easier and to ensure consistent, high-quality articles,

    • Zoho Books "Update" Trigger for Zapier?

      Hi Zoho Team, I've been in talks with Zapier about using their services to connect my Zoho Books account to various apps, but I require an additional trigger beyond the "New" trigger. I require an "Update" trigger, which will allow me to filter and create Zaps only when certain conditions are fulfilled on the Zoho Books end before I automate a separate app. Specifically this will help me create invoices in QuickBooks Online only when payments are recorded in Zoho Books, rather than when a new invoice

    • How to prevent editing of closed tickets in Zoho Desk?

      I would like to confirm whether it is possible to completely prevent agents from editing tickets once their status is set to Closed. Is there any configuration, permission, or best practice in Zoho Desk to ensure that closed tickets remain read-only and

    • Zoho Desk Limitations

      Good day, all, I would like to know whether others share my frustration with some of Zoho's limitations. Don't get me wrong, I like Desk (and I also have a subscription for Analytics), I have been with them for close to 10 years, and unfortunately, I

    • Cliq iOS can't see shared screen

      Hello, I had this morning a video call with a colleague. She is using Cliq Desktop MacOS and wanted to share her screen with me. I'm on iPad. I noticed, while she shared her screen, I could only see her video, but not the shared screen... Does Cliq iOS is able to display shared screen, or is it somewhere else to be found ? Regards

    • New Mandatory One-Click Unsubscribe Link Overshadowing Custom Unsubscribe Link

      I was recently informed by Zoho CRM Support that they are now mandated by the large email service providers like Google and Yahoo to provide a one-click unsubscribe option in the header (not the body) of all mass emails. I have a custom unsubscribe link

    • Next Page