Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team




      • Sticky Posts

      • Register for Zoho Desk Beta Community

        With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

      • Share your Zoho Desk story with us!

        Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

      • Tip #1: Learn to pick the right channels

        Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

      • Welcome to Zoho Desk Community - Say hello here!

        Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

      • Webinar 1: Blueprint for Customer Service

        With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

        • Recent Topics

        • Contact's title in "Contact Role Mapping"

          When I'm creating a deal, I'd like to see the contacts title in the listing. Right now, I only see this: How can I get the contact's title in there?

        • Zoho CRM - Client Portal - Hide Notes Related List

          Hi Community, I'm building a customer portal and I can't find a way to hide the notes related list. I don't want the client to see the notes I have about them. Is there a way to do this as it is no bin/trash icon when I hover over.

        • "Pivot Table" Conditional Formatting

          Team, I there a way to use conditional formatting a "Pivot Table"  report? Thanks, Arron Blue Pumpkin Hosting | Solutions Made Simple

        • How many clients can be added to Zoho Practice?

          How many clients can be added to Zoho Practice without having their zoho app?

        • Stage History

          when viewing a ticket , and you look at stage history tab (kanban view) and look at the stage duration column in days, it shows the current stage of the ticket as " current stage ". Should it not rather show the amount of days it has been in that current

        • Automating Ticket Responses Using Zoho Desk's AI Features

          We’re looking to set up an automation within Zoho Desk that can analyze incoming emails or tickets and automatically respond with relevant knowledge base articles based on the content of the request. Could you please guide us on how to configure this

        • Optimising CRM-Projects workflows to manage requests, using Forms as an intermediary

          Is it possible to create a workflow between three apps with traceability between them all? We send information from Zoho CRM Deals over to Zoho Projects for project management and execution. We have used a lookup of sorts to create tasks in the past,

        • Service locations are tied to contacts?

          Trying the system out. And what I discovered is that it seems that the whole logic of the app is, I'd say, backwards. There is a Customer - a company. The company has contact persons and service locations can be associated with different contact persons.

        • Enhancements to Zoho Map integration tasks

          Hello everyone, We're excited to announce enhancements to the Zoho Map integration tasks in Deluge, which will boost its performance. This post will walk you through the upcoming changes, explain why we're making them, and detail the steps you need to

        • Bug in Total Hour Calculation in Regularization for past dates

          There is a bug in Zoho People Regularization For example today is the date is 10 if I choose a previous Date like 9 and add the Check in and Check out time The total hours aren't calculated properly, in the example the check in time is 10:40 AM check

        • Narrative 12: Sandbox - Testing without the risk

          Behind the scenes of a successful ticketing system: BTS Series Narrative 12: Sandbox - Testing without the risk What is a sandbox environment? A sandbox environment is a virtual playground that allows you to test freely and experiment with various elements

        • Announcing new features in Trident for Mac (1.27.0)

          Hello everyone! Trident for macOS (v1.27.0) is here with new features and enhancements to improve scheduling and managing your calendar events. Let's take a quick look at them. Stay aligned across time zones. Both the scheduled and original time zones

        • Branding of native system emails

          Make system emails adjustable in terms of branding. We want our system to be completely white label, because it is not a CRM anymore, it's way more than that. We are following the strategy of "CRM for everyone" to use the CRM in all departments, not only

        • Slow uploads of large files

          I'm wanting to use Workdrive for transferring large images and video (we're talking things from 100MB-5GB). I'm running solo on a 500MBit/sec fiber connection. I'm getting upload speeds to Workdrive of no more than about 1-3Mbytes/sec when going through

        • Migrate Your Notes from OneNote to Zoho Notebook Today

          Greetings Notebook Users, We’re excited to introduce a powerful new feature that lets you migrate your notes from Microsoft OneNote to Zoho Notebook—making your transition faster and more seamless than ever. ✨ What’s New One-click migration: Easily import

        • need to upload from airtable to google drive

          I have a zapier zap that automates between airtable and google drive. When a customer uploads a new file into airtable via a client portal interface, zapier uploads that file into a folder linked to that customer's project record. I need to replicate

        • Can't delete functions that are associated with deleted workflow rules

          We have a handful of functions that were once associated with a workflow rule, but the rule has been deleted. The function still thinks it is associated so I can't assign it to a new rule. It is starting to get really messy because we have a list of functions

        • Default Sorting on Related Lists

          Is it possible to set the default sorting options on the related lists. For example on the Contact Details view I have related lists for activities, emails, products cases, notes etc... currently: Activities 'created date' newest first Emails - 'created

        • Credit Management: #1 Credit You Owe vs Credits Owed to the Business

          Think about the last time you ordered food online. You might have paid in advance through your card, but you received a $20 refund because your order got delayed or cancelled. In most apps, refunds don't go into the bank account directly; instead, they're

        • Tip #46- Turn Every Session into an Insight with Zoho Assist survey report- 'Insider Insights'

          Delivering exceptional remote support isn’t just about resolving issues, it’s about understanding how both customers and technicians experience each session. That’s where Survey Report in Zoho Assist come in. You can configure and customize survey questions

        • Enhancing Zia's service with better contextual responses and article generation

          Hello everyone, We are enhancing Zia's Generative AI service to make your support experience smarter. Here's how: Increased accuracy with Qwen One of the key challenges in AI is delivering responses that are both contextually accurate and empathetic while

        • Allow the usage of Multi-Select fields as the primary field on "Layout Rules"

          We want to force our users to enter some explanation strings when a multi-select field has been utilized. I can't understand the reason for the usage restriction of Multi-Select fields as a primary field. This is a simple "Display hidden mandatory field

        • CRM/Bookings integration edits Contact names

          Hi there, I've installed the extension that connects Zoho CRM and Zoho Bookings. When we get a new appointment from Bookings from an existing Contact, that Contact's record shows this: First Name was updated from asd to blank value Last Name was updated

        • Domain Change

          “Please update my Email-in domain from @biginmail.biginmail.in to @biginmail.zoho.com. Messages to the .in domain are bouncing.”

        • Webhooks Limit Exceeded

          Today, I received an error message saying, 'Total number of Webhook call exceeded', but when I look at Manage > Billing, it doesn't look like any of my invokeURL calls are being logged. Following the advice from this thread: https://help.zoho.com/portal/en/community/topic/webhooks-daily-limits-in-zoho-creator

        • Auto select option in CRM after Zoho Form merge

          Hi, I have a dropdown field in Zoho CRM that is filled with a Zoho Form. The data is filled but not automatically shown. After selecting the right value in the dropdown the information a second field is shown. So the question is; how can I make the dropdown

        • Bring your CRM data straight into your presentations in Zoho Show

          Let's say you are working on a presentation about your team's sales pipeline for an upcoming strategy meeting. All the information you need about clients and leads is in Zoho CRM, but you end up copying details from the CRM into your slides, adjusting

        • Improved RingCentral Integration

          We’d like to request an enhancement to the current RingCentral integration with Zoho. RingCentral now automatically generates call transcripts and AI-based call summaries (AI Notes) for each call, which are extremely helpful for support and sales teams.

        • Introducing New APIs in Zoho Contracts

          We are excited to announce the release of new APIs in Zoho Contracts to help you automate and manage every stage of your contract lifecycle more efficiently. Here’s a quick overview of what’s new: 1. Complete Contract Draft You can use this API to complete

        • Vimeo

          For me Vimeo is the most important video social channel for media and filmmakers. Would others agree and like it added to Zoho Social.

        • Delete a department or category

          How do I delete a Department? Also, how do I delete a Category? This is pretty basic stuff here and it's impossible to find.

        • Organization Emails in Email History

          How can I make received Org Emails to show up here?

        • How to setup pricing in Zoho

          Hi everyone, I am relatively new here and have just moved from my old inventory system to the Zoho one. I am trying to get my head around how it all works. I am mostly setup connected to a shopify store, but I do manual sales also For manual invoicing,

        • Prefilled Date fields auto-changed and then locked when using “Edit as new”

          If a document out for signature has date fields (not SignedDate fields) that were pre-filled before sending, and then you use “Edit as new” to create a new version of the same document, the value of those date fields gets automatically changed to today

        • Is there a way to update all the start and end dates of tasks of a project after a calendar change?

          Hi! Here's my situation. I've built a complete project planning. All its tasks have start dates and due dates. After completing the planning, I've realized that the project calendar was not the right one. So I changed the project calendar. I now have

        • Access Phone Field Components (Country Code) Directly

          Hello everyone, I'd like to propose an enhancement for the Phone field in Zoho Creator. The Problem: The Phone field captures the country code and local number separately, but stores them as a single string (e.g., +1234567890). To get the country code,

        • Send mass messages through WhatsApp from the Tickets module

          Hi Everyone! Effective communication is key to delivering prompt and reliable customer support. Because WhatsApp is one of the most widely used and familiar messaging platforms, it's an effective channel for agents to reach customers who have submitted

        • Lead Owner Signature Merge Field

          I want to automatically insert a signature (i.e. contact info usually found at the bottom of an email) into an email template, depending on who the lead owner is. What is the merge code for the Signature from a Users profile? CRM > Settings > Customization > Templates There is a popup near the bottom of the edit screen which says: "You can insert a Signature, which is available as a merge field in the users section." It is also referenced on this page: https://help.zoho.com/portal/en/kb/crm/customize-crm-account/customizing-templates/articles/template-builder#Merge_Fields

        • How do I create a time field?

          I want a field that only records time. I can only see how to create a date-time field. If I do that and enter a time, without a date, nothing is recorded. If I create a number or decimal field, I cannot use it in time calculations. All I want is a field

        • Alternating columns - How to reverse order on mobile - Responsive template

          Can the order of alternating columns be reversed on mobile so that image comes before the text? Example: Desktop Row 1 column Left (image) , column right (text) Row 2 column Left (text) , column right (image) Mobile Currently Row 1 Image over text Row

        • Next Page