Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

Optimize your Knowledge Base for better visibility by allowing search engine crawling and indexing

All you need to know about no-follow and no-index in KB.


What are article crawlability and indexability?  

Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, it reviews the site's content to assess its topic and relevance to specific search queries. If a site lacks crawlability, meaning search engines struggle to scan it effectively, the content will not be indexed and will consequently not appear in search results.
 
Additionally, a site’s content must be appropriately optimized for search engine crawlers, allowing it to be indexed accurately and appear in relevant search outcomes. Ensuring every article is crawlable and indexable can significantly enhance its visibility and improve its search engine rankings.

How do you block search indexing?

There are two methods to apply the no-index directive: using a <meta> tag or via an HTTP response header. Both methods achieve the same result, so select the one that suits your website and content type best. Additionally, you can pair the no-index directive with other indexing rules. For instance, you can combine a no-follow directive with an index like this: <meta name="robots" content=" index, nofollow" />.
Quote<meta> tag 

To prevent all search engines that support the no-index rule from indexing a page on your site, place the following <meta> tag into the<head> section of your page:

Quote<meta name="robots" content="noindex"> 
Notes
Certain search engines may interpret the no-index directive in various ways. Therefore, your page could still appear in other search engines' results.

Usage of the no-index tag 

The no-index rule can be implemented using a <meta> tag or an HTTP response header to restrict search engines that recognize the no-index directive, like Google, from indexing certain content. When Googlebot comes across a page with this tag or header, it will remove that page entirely from Google Search results, regardless of whether there are links from other websites.

Notes
To ensure the index rule works, the page or resource must be accessible and not blocked by a robots.txt file. If blocked, the crawler won't see the index rule, though the page might still show in search results if linked from other pages. An index is beneficial if you lack root server access, as it lets you manage page access individually.

What are no-follow links? 

Typically, it's best to allow robots to follow all links on a webpage. Being overly strict in indicating which links should be followed or marked as no-follow may give the impression that the site is trying to influence how a robot views it.

 

Page sculpting is the practice of using no-follow commands to influence the transfer of signals between pages. In the best-case scenario, these strategies to manipulate bots are ineffective. In the worst-case scenario, trying to control bots with nofollow may result in penalties.

When should you use meta no-follow?

 Consequently, there are limited scenarios in which using meta robots no-follow on websites is appropriate. Typically, encountering meta robots no-follow during SEO audits may indicate that a website has been over-optimized.

What are dofollow links?

 Dofollow links, often called "follow" links, are typical online hyperlinks. Unlike links with specific attributes such as “nofollow” or “sponsored,” these links do not carry unique identifiers. As a result, the following links contribute to the ranking control of the pages they connect to.

Experimenting with crawlability and indexability

Before a website's pages can be displayed in search results, search engine bots must first discover all the pages on the site and then analyze them to determine their rankings. The initial phase of finding pages is called crawling, while the subsequent analysis is called indexing.

 

Crawling starts with bots identifying all the URLs of a website's pages. They primarily find these URLs through internal links within the site or through backlinks from external sites. Once a bot identifies a page’s URL, it retrieves content, including the title, text, images, and additional data, such as the date it was last updated. Restrictions can dictate which files and pages a bot can crawl.

 

Indexing occurs after the crawling process. During this stage, bots assess all the information collected from the crawl. They evaluate whether the content is valuable and authoritative and identify the topics associated with the page and how it compares to other relevant pages. Additionally, search engine bots will determine which search results a page might appear in, if any, and its position within those results.

No-follow vs no-index

Let me give you a typical example of an ad campaign landing page designed solely for traffic from typical ads. It's not ideal to prevent organic search traffic from reaching these pages. Implementing meta robots and a no-index tag will keep search engines from ranking them. 

Summing up

  • No-index tells a robot not to index the page. It should be used to keep pages out of search results or to help with low-quality content issues.
  • Meta nofollow tells a robot not to follow a specific link or all of the links on a page. This should typically not be used.
  • Robots should be allowed to determine what gets indexed and shown in search results. If a page is not intended to be visible in search results, it often shouldn't be part of the website.

 

Please watch this space for more detailed use cases of SEO tags that you can apply to your organisation's knowledge base.

 

Cheers,

 

Kavya Rao,

The Zoho Desk Team


    • Sticky Posts

    • Register for Zoho Desk Beta Community

      With the start of the year, we have decided to take a small step in making the life of our customers a little easier. We now have easy access to all our upcoming features and a faster way to request for beta access. We open betas for some of our features

    • Share your Zoho Desk story with us!

      Tell us how you use Zoho Desk for your business and inspire others with your story. Be it a simple workflow rule that helps you navigate complex processes or a macro that saves your team a lot of time; share it here and help the community learn and grow with shared knowledge. 

    • Tip #1: Learn to pick the right channels

      Mail, live chat, telephony, social media, web forms—there are so many support channels out there. Trying to pick the right channels to offer your customers can get pretty confusing. Emails are most useful when the customer wants to put things on record. However, escalated or complicated issues should not be resolved over email because it's slow and impersonal.  When you need immediate responses, live chat is more suitable. It's also quick and convenient, so it's the go-to channel for small issues. 

    • Welcome to Zoho Desk Community - Say hello here!

      Hello everyone! Though we have been here for a while, it’s time to formally establish the Zoho Desk Community; we’re really happy to have you all here! This can be the place where you take a moment to introduce yourself to the rest of the community. We’d love to hear all about you, what you do, what company or industry you work for, how you use Zoho Desk and anything else that you will like to share! Here’s a little about me. I am Chinmayee. I have been associated with Zoho since 2014. I joined here

    • Webinar 1: Blueprint for Customer Service

      With the launch of a host of new features in Zoho Desk, we thought it’ll be great to have a few webinars to help our customers make the most of them. We’re starting off with our most talked about feature, Blueprint in Zoho Desk. You can register for the Blueprint webinar here: The webinar will be delivered by our in-house product experts. This is a good opportunity to ask questions to our experts and understand how Blueprint can help you automate your service processes. We look forward to seeing

    • Recent Topics

    • Why is the ability Customize Calls module so limited?

      Why can't I add additional sections? why can't I add other field types than the very limited subset that zoho allows? Why can I only add fields to the outbound/inbound call sections and not to the Call Information section?

    • Zoho Desk - Upsert Ticket

      Hi Desk Team, It is common to request more information from end-users. Using forms is a great way to ensure all the required information is collected. It would be great if there were an "upsert" option on the Zoho Form -> Zoho Desk integration which would

    • CRM gets location smart with the all new Map View: visualize records, locate records within any radius, and more

      Hello all, We've introduced a new way to work with location data in Zoho CRM: the Map View. Instead of scrolling through endless lists, your records now appear as pins on a map. Built on top of the all-new address field and powered by Mappls (MapMyIndia),

    • Enhance Appointment Buffers in Zoho Bookings

      There was previously a long-standing feature request related to enhancing the way appointment buffers work in Zoho Bookings, but it looks like the original post has been deleted. I am therefore adding a new request that Zoho Bookings adjust how appointment

    • Subscriptions for service call

      So we install products and we want to offer a service contract for the customers yearly service calls to be billed monthly. So ideally at some point we want to email them a quote for their needs. WE will choice it our end based on the equipment. It would

    • Connection to other user

      Zoho Cliq handles sharing of Custom OAuth Connections that require individual user logins.

    • How to invite friends on other social media platforms to one of my group chats in arattai?

      Hello, I have formed chat groups in arattai. I want to invite my friends on other social media platforms like WhatsApp/ FB to one of my groups. Different friends would be invited to different groups. How to share an invite link of one of my groups to

    • Cliq does not sync messages after Sleep on Mac

      I'm using the mac app of Cliq. When I open my mac after it was in sleep mode, Cliq does not sync the messages that I received. I always have to reload using cmd + R, which is not what I want when using a chat application.

    • Creator Offline

      We had online access setup and working on our iphones. We have just set it up on an 'Android Tablet' and it is not downloading all the images? We use it to show customers our catalogue. Any ideas. Offline components all setup on both devices

    • Optimum CRM setup for new B2B business

      Can some advise the most common way to setup Zoho CRM to handle sales for a B2B company? Specifically in how to handle inbound/outbound emails. I have spent hours researching online and can't seem to find an accepted approach, or even a tutorial. I have

    • Facing Issues with Sites Mobile font sizes

      my page renediaz.com is facing issues mobile view, when i try to lower font sizes in home page, instead of changing the size, it changes the line space

    • Does the Customer “Company Name” field appear anywhere in the Zoho Books UI outside of PDFs?

      Hi everyone, I’m trying to understand how the Company Name field is actually used in Zoho Books. There is a Company Name field on the customer record, but when viewing transactions like a Sales Order in the normal UI (non-PDF view), that field doesn’t

    • Set expiration date on document and send reminder

      We have many company documents( for example business registration), work VISA documents. It will be nice if we can set a expiry date and set reminders ( for example 90 days, 60 days, 30 days etc.,) Does Zoho workdrive provide that option?

    • Analytics : How to share to an external client ?

      We have a use case where a client wants a portal so that several of his users can view dashboards that we have created for them in Zoho Analytics. They are not part of our company or Zoho One account. The clients want the ability to have user specific,

    • Automatically Update Form Attachment Service with Newly added Fields

      Hi, When I have a Form Setup and connected to a 3rd Party Service such as OneDrive for Form Attachments, when I later add a new Upload Field I have to remove and redo the entire 3rd Party Setup from scratch. This needs to be improved, such as when new

    • Zoho Sheet for Desktop

      Does Zoho plans to develop a Desktop version of Sheet that installs on the computer like was done with Writer?

    • Payroll and BAS ( Australian tax report format )

      Hello , I am evaluating Zoho Books and I find the interface very intuitive and straight forward. My company is currently using Quickbooks Premier the Australian version. Before we can consider moving the service we would need to have the following addressed : 1.Payroll 2.BAS ( business activity statement ) for tax purposes 3.Some form of local backup and possible export of data to a widely accepted format. Regards Codrin Mitin

    • Zoho Desk API - Send Reply to CUSTOMERPORTAL

      Hello! I'll try to send a reply to Customer Portal, But the response is 500 (INTERNAL_SERVER_ERROR in service response). {"Error":"{\"errorCode\":\"INTERNAL_SERVER_ERROR\",\"message\":\"An internal server error occurred while performing this operation.\"}"}

    • Python - code studio

      Hi, I see the code studio is "coming soon". We have some files that will require some more complex transformation, is this feature far off? It appears to have been released in Zoho Analytics already

    • Dheeraj Sudan and Meenu Hinduja-How do I customize Zoho apps to suit my needs?

      Hi Everyone, I'm Meenu Hinduja and my husband Dheeraj Sudan, run a business. I’m looking to tweak a few things to fit my needs, and I’d love to hear what customizations others have done. Any tips or examples would be super helpful! Regards Dheeraj Sudan

    • From Zoho CRM to Paper : Design & Print Data Directly using Canvas Print View

      Hello Everyone, We are excited to announce a new addition to your Canvas in Zoho CRM - Print View. Canvas print view helps you transform your custom CRM layouts into print-ready documents, so you can bring your digital data to the physical world with

    • Zoho Books Payroll

      How am I supposed to do payroll and pay my employees with Zoho Books? I think it's pretty strange that an accounting software doesn't have the ability to perform one of the most common functions in business; paying your employees. Am I missing something,

    • Layout Rules Don't Apply To Blueprints

      Hi Zoho the conditional layout rules for fields and making fields required don't work well with with Blueprints if those same fields are called DURING a Blueprint. Example. I have field A that is used in layout rule. If value of field A is "1" it is supposed to show and make required field B. If the value to field A is "2" it is supposed to show and make required field C. Now I have a Blueprint that says when last stage moves to "Closed," during the transition, the agent must fill out field A. Now

    • validation rules doesn't work in Blueprint when it is validated using function?

      I have tried to create a validation rule in the deal module. it works if I try to create a deal manually or if I try to update the empty field inside a deal. but when I try to update the field via the blueprint mandatory field, it seems the validation

    • Sort Legend & stacked bar chart by value

      I'd love to see an option added to sort the legend of graphs by the value that is being represented. This way the items with the largest value in the graph are displayed top down in the legend. For example, let's say I have a large sales team and I create

    • Is It Possible to Convert a Custom Module to a Quote?

      I recently created a custom module in our CRM environment for RFQs so that our sales team can submit quote requests directly in Zoho rather than by email/phone. This allows for a cleaner overall process, minimizing potential errors and potentially encouraging

    • All new Address Field in Zoho CRM: maintain structured and accurate address inputs

      The address field will be available exclusively for IN DC users. We'll keep you updated on the DC-specific rollout soon. It's currently available for all new sign-ups and for existing Zoho CRM orgs which are in the Professional edition. Latest update

    • How can Data Enrichment be automatically triggered when a new Lead is created in Zoho CRM?

      Hi, I have a pipeline where a Lead is created automatically through the Zoho API and I've been trying to look for a way to automatically apply Data Enrichment on this created lead. 1) I did not find any way to do this through the Zoho API; it seems like

    • Client Side Scripts for Meetings Module

      Will zoho please add client side scripting support to the meetings module? Our workflow requires most meeting details have a specific format to work with other software we have. So we rely on a custom function to auto fill certain things. We currently

    • Proposal: Actionable API Error Messages to Reduce Support Ticket Volume

      I've encountered a long-standing frustration with how Zoho Creator's API communicates errors to developers, and I'm hoping the Zoho team can address this in a future update. This issue has persisted for over 9 years based on community discussions, and

    • Zoho CRM - Option to create Follow-Up Task

      When completing a Zoho CRM Task, it would be very helpful if there was an option to "Complete and Create Follow-Up Task" in the pop-up which appears. It could clone the task you are closing and then show it on the screen in edit mode, all the user would

    • Dynamic Field Folders in OneDrive

      Hi, With the 2 options today we have either a Dynamic Parent Folder and lots of attachments all in that one folder with only the ability to set the file name (Which is also not incremented so if I upload 5 photos to one field they are all named the same

    • Drag 'n' Drop Fields to a Sub-Form and "Move Field To" Option

      Hi, I would like to be able to move fields from the Main Page to a Sub-Form or from a Sub-Form to either the Main Page or another Sub-Form. Today if you change the design you have to delete and recreate every field, not just move them. Would be nice to

    • File Conversion from PDF to JPG/PNG

      Hi, I have a question did  anyone every tried using custom function to convert a PDF file to JPG/PNG format? Any possibility by using the custom function to achieve this within zoho apps.  I do know there are many third parties API provide this with

    • Allocating inventory to specific SO's

      Is there a way that allocate inventory to a specific sales order? For example, let's say we have 90 items in stock. Customer 1 orders 100 items. This allocates all 90 items to their order, and they have a back order for the remaining 10 items which could

    • Cliq iOS can't see shared screen

      Hello, I had this morning a video call with a colleague. She is using Cliq Desktop MacOS and wanted to share her screen with me. I'm on iPad. I noticed, while she shared her screen, I could only see her video, but not the shared screen... Does Cliq iOS is able to display shared screen, or is it somewhere else to be found ? Regards

    • Pull cells from one sheet onto another

      Hello all! I have created an ingredients database where i have pricing and information and i want to pull from that database into a recipe calculator. I want it to pull based on what ingredient I choose. The ingredients database has an idea and i want

    • Closing the Loop: Why Lookup Asymmetry is Harming Data Integrity in Creator

      TL;DR: Lookup fields allow users to add new related records inline via the "+" icon, but there's no equivalent ability to edit an existing related record without navigating away and losing form context. Adding a native "Edit" icon—with automatic User

    • Add the ability to Hide Pages in Page Rules

      Hi, We have Field Rules to show and hide fields and we have page Rules, but we can't hide a page in Page Rules so it isn't completed before the previous page (And then have the Deny Rules to prevent submitting without both pages completed), we can only

    • Enable Free External Collaboration on Notecards in Zoho Notebook

      Hi Zoho Notebook Team, I would like to suggest a feature enhancement regarding external collaboration in Zoho Notebook. Currently, we can share notes with external users, and they are able to view the content without any issue. However, when these external

    • Next Page