Kaizen #140 - Integrating Blog feed scraping service into Zoho CRM Dashboard

Kaizen #140 - Integrating Blog feed scraping service into Zoho CRM Dashboard



Howdy Tech Wizards! Welcome to a fresh week of kaizen.

This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of  Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget. 

Overview

For the use case mentioned above, 
  • Build and deploy a web scraping service on a cloud. 
  • Create a Zoho CRM variable to store the response from this web scraping service.
  • Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
  • Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.

1. Create a Web Scraping Service 

Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
  • Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
  • Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post. 

Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function. 

Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.

2. Create a Zoho CRM Variable

Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.

Step 2: Click on the Create New Variable button and provide the necessary details.



Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.

For a step-by-step guide on creating a variable, check here. 

3. Write a Custom Schedule Function 

Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.

Step 2: Click on the Create New Schedule button and provide a name to the schedule.



Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.



Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements. 

First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API. 

You can also copy and paste the code from here. 

response = invokeurl
[
url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/"
type :GET
];
info response;
scrape_response = response.toString();
variable = Map();
variable.put("id","5545974000007587001");
variable.put("value",scrape_response);
variables_list = List();
variables_list.add(variable);
param = Map();
param.put("variables",variables_list);
response = invokeurl
[
url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001"
type :PUT
parameters:param.toString()
connection:"crm_oauth_connection"
];
info response;

For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.

Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.



For more information about Schedules, check this help page. 

Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable. 

4. Code your Widget

Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.

Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.

Copy and paste the following code in your widget.html file. 

<html>
<head>
</head>
<body>
<div id="blogs-table-container"></div>
<script type="text/javascript">
ZOHO.embeddedApp.on("PageLoad", function () {
var conn_name = "test_api_connection";
var req_data = {
"method": "GET",
"param_type": 1
};
ZOHO.CRM.CONNECTION.invoke(conn_name, req_data)
.then(function (data) {
console.log(data)
const div = document.getElementById("blogs-table-container");
div.innerHTML = data.details.statusMessage.variables[0].value;
})
})
ZOHO.embeddedApp.init();
</script>
</body>
</html>

5. Upload the Widget and Check Your Results

Step 1: Execute the zet validate and zet pack commands to validate and pack the widget. 

Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button. 

Fill in the necessary details and upload the package. 



Step 3: Let us now check the dashboard widget from the Zoho CRM Home page. 



We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites. 

Hope this post was insightful and help your interest. 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!

Stay tuned until we circle back to you next Friday!

Cheers!

-------------------------------------------------------------------------------------------------------------------

Recommended Reads

-------------------------------------------------------------------------------------------------------------------
Alert
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!  


    • Sticky Posts

    • Kaizen #197: Frequently Asked Questions on GraphQL APIs

      🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Kaizen #198: Using Client Script for Custom Validation in Blueprint

      Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Celebrating 200 posts of Kaizen! Share your ideas for the milestone post

      Hello Developers, We launched the Kaizen series in 2019 to share helpful content to support your Zoho CRM development journey. Staying true to its spirit—Kaizen Series: Continuous Improvement for Developer Experience—we've shared everything from FAQs

    • Kaizen #193: Creating different fields in Zoho CRM through API

      🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Client Script | Update - Introducing Commands in Client Script!

      Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! Commands

    • Recent Topics

    • Introducing Version-3 APIs - Explore New APIs & Enhancements

      Happy to announce the release of Version 3 (V3) APIs with an easy to use interface, new APIs, and more examples to help you understand and access the APIs better. V3 APIs can be accessed through our new link, where you can explore our complete documentation,

    • Round robin

      Hi, I'm trying to set up a round robin to automatically distribute tickets between agents in my team but only those tickets that are not otherwise distributed by other workflows or direct assignments. Is that possible and if so which criteria should I

    • Does Zoho Sheet Supports https://n8n.io ?

      Does Zoho Sheet Supports https://n8n.io ? If not, can we take this as an idea and deploy in future please? Thanks

    • Bigin Android app update: User management

      Hello everyone! In the most recent Bigin Android app update, we have brought in support for the 'Users and Controls' section. You can now manage the users in your organization within the mobile app. There are three tabs in the 'Users and Controls' section:

    • Share records with your customers and let them track their statuses in real time.

      Greetings, I hope everyone is doing well! We're excited to introduce the external sharing feature for pipeline records. This new enhancement enables you to share pipeline records with your customers via a shareable link and thereby track the status of

    • Live webinar: Discover Zoho Show: A complete walkthrough

      Hello everyone, We’re excited to invite you to our upcoming live webinar, Discover Zoho Show: A Complete Walkthrough. Whether you’re just getting started with Show or eager to explore advanced capabilities, this session will show you useful tips and features

    • Deal Stage component/widget/whatever it is... event

      Deal Stages I am trying to access the event and value of this component. I can do it by changing the Stage field but users can also change a Deal Stage via this component and I need to be able to capture both values. Clicking on 'Verbal' for instance,

    • Create advanced slideshows with hybrid reports using Zoho Projects Plus

      Are your quarterly meetings coming up? It’s time to pull up metrics, generate reports, and juggle between slides yet again. While this may be easier for smaller projects, large organizations that run multiple projects may experience the pressure when

    • Add an option to disable ZIA suggestions

      Currently, ZIA in Zoho Inventory automatically provides suggestions, such as sending order confirmation emails. However, there is no way to disable this feature. In our case, orders are automatically created by customers, and we’ve built a custom workflow

    • Email Integration - Zoho CRM - OAuth and IMAP

      Hello, We are attempting to integrate our Microsoft 365 email with Zoho CRM. We are using the documentation at Email Configuration for IMAP and POP3 (zoho.com) We use Microsoft 365 and per their recommendations (and requirements) for secure email we have

    • Formula field with IF statement based on picklist field and string output to copy/paste in multi-line field via function

      Hello there, I am working on a formula field based on a 3-item picklist field (i.e. *empty value*, 'Progress payment', 'Letter of credit'). Depending on the picked item, the formula field shall give a specific multi-line string (say 'XXX' in case of 'Progress

    • CRM x WorkDrive: File storage for new CRM signups is now powered by WorkDrive

      Availability Editions: All DCs: All Release plan: Released for new signups in all DCs. It will be enabled for existing users in a phased manner in the upcoming months. Help documentation: Documents in Zoho CRM Manage folders in Documents tab Manage files

    • Zoho CRMの流入元について

      Zoho CRMとZoho formsを連携し、 formsで作成したフォームをサイトに埋め込み運用中です。 UTMパラメータの取得をformsを行い、Zoho CRMの見込み客タブにカスタム項目で反映される状況になっています。 広告に関してはUTMパラメータで取得できているため問題ないのですが、オーガニック流入でフォーム送信の場合も計測したいです。メールやGoogle、Yahoo、directなどの流入元のチャネルが反映されるようにしたいのですが、どのように設定したら良いでしょうか。 また、

    • Error While Sign in on Zoho Work Drive

      Dear Team, I hope this email finds you well. I have recently created a Zoho account and started using it. But while I am trying to log in to Zoho work drive it won't log me in its crashing every time I try it. I have tried it on android app, phone browser

    • Choosing a portal option and the "Unified customer portal"?

      I am trialling Zoho to replace various existing systems, one of which is a customer portal. Our portal allows clients to add and edit bookings, complete forms, manage their subscriptions and edit some CRM info. I am trying to understand how I might best

    • Elevate your CX delivery using CommandCenter 2.0: Simplified builder; seamless orchestration

      Most businesses want to create memorable customer experiences—but they often find it hard to keep them smooth, especially as they grow. To achieve a state of flow across their processes, teams often stitch together a series of automations using Workflow

    • Unified Directory : How to Access ?

      I signed in to Zoho One this morning and was met with the pop up about the upgraded directory (yay!) I watched the video and pressed "Get Started" ... and it took me back to the standard interface. How do I actually access the new portal/directory ?

    • Translation support expanded for Modules, Subforms and Related Lists

      Hello Everyone!   The translation feature enables organizations to translate certain values in their CRM interface into different languages. Previously, the only values that could be translated were picklist values and field names. However, we have extended

    • Unified task view

      Possible to enable the unified task view in Trident, that is currently available in Mail?

    • Bigin, more powerful than ever on iOS 26, iPadOS 26, macOS Tahoe, and watchOS 26.

      Hot on the heels of Apple’s latest OS updates, we’ve rolled out several enhancements and features designed to help you get the most from your Apple devices. Enjoy a refined user experience with smoother navigation and a more content-focused Liquid Glass

    • Importing data into Assets

      So we have a module in Zoho CRM called customers equipments. It links to customers modules, accounts (if needed) and products. I made a sample export and created extra fields in zoho fsm assets module. The import fails. Could not find a matching parent

    • Allow instruction field in Job Sheets

      Hello, I would like to know if it is possible to have an instruction field (multi line text) in a job sheet or if there is a workaround to be able to do it. Currently we are pretty limited in terms of fields in job sheets which makes it a bit of a struggle

    • Streamlining Work Order Automation with Zoho Projects, Writer & WorkDrive

      Hello Community, Here is the first post in 'Integration & Automation' Series. Use Case :: Create, Merge, Sign & Store Documents in Zoho WorkDrive. Scenario :: You have a standard Work Order template created in Zoho Writer. When a task status is chosen

    • The dimensions of multilingual power

      Hola, saludos de Zoho Desk. Bonjour, salutations de Zoho Desk. Hallo, Grüße von Zoho Desk. Ciao, saluti da Zoho Desk. Olá, saudações da Zoho Desk. வணக்கம், Zoho Desk இலிருந்து வாழ்த்துகள். 你好，来自 Zoho Desk 的问候。 مرحباً، تحيات من Zoho Desk. नमस्ते, Zoho

    • Multi-line address lines

      How can I enter and migrate the following 123 state street Suite 2 Into a contact address. For Salesforce imports, a CR between the information works. The ZOHO migration tool just ignores it. Plus, I can't seem to even enter it on the standard entry screen.

    • Accessing Zoho Forms

      Hi all, We're having trouble giving me access to our company's Zoho Forms account. I can log in to a Forms account that I can see was set up a year ago, but can't see any shared forms. I can log into Zoho CRM and see our company information there without

    • Archiving Contacts

      How do I archive a list of contacts, or individual contacts?

    • Cost of good field

      Is there a way we can have cost of good sold as a field added to the back end of the invoicing procedure and available in reports?

    • How to add image to items list in Invoice or Estimate?

      Hello! I have just started using Zoho Invoice to create estimates and, possibly to switch from our current CRM/ERP Vendor to Zoho. I have a small company that is installing CCTV systems and Alarm systems. My question is, can I add images of my "items" to item list in Zoho Invoice and Estimates and their description? I would like to show my clients the image of items in our estimates so they can decide if they like these items. And I tell you, often they choose more expensive products just because

    • Issue with the Permission to Zoho Form

      I am getting an error by signing in to zoho form as it is stated that i don't have permission to access this is admin account

    • CRM templates

      Hello everyone, In my company we use Zoho campaigns where we set up all newsletters and we use Zoho CRM for transactional emails. I have created some templates in Zoho campaigns but from my understanding i cannot use those in Zoho CRM, right?

    • Meet Canvas' Grid component: Your easiest way to build responsive record templates

      Visual design can be exciting—until you're knee-deep in the details. Whether it's aligning text boxes to prevent overlaps, fixing negative space, or simply making sure the right data stands out, just ironing out inconsistencies takes a lot of moving parts.

    • Addin Support in Zoho Sheet

      Is there any addin support available in zoho sheet as like google marketplace to enhance productivity by connecting with other apps, providing AI data analysis, streamlining business processes, and more?

    • Where to integrate Price Book and Product List Price

      Hello, We sync zoho crm all modules with all data to zoho analytics. In zoho crm, we have "Price Books" and "Products" modules, where each product is assigned to a few price books with different list prices. From zoho crm, I am able to export a dataset

    • Pending Sales Order Reports

      Pending sale order report is available for any single customer, Individual report is available after 3-4 clicks but consolidated list is needed to know the status each item. please help me.

    • Zoho Mail SMTP IP addresses

      We are using Zoho Mail and needs to whitelist IP for some redirections from your service to another e-mails. You can provide IP address list for Zohomail SMTP servers?

    • Migrate Your Notes from OneNote to Zoho Notebook Today

      Greetings Notebook Users, We’re excited to introduce a powerful new feature that lets you migrate your notes from Microsoft OneNote to Zoho Notebook—making your transition faster and more seamless than ever. ✨ What’s New One-click migration: Easily import

    • Zoho Campaigns - Why do contacts have owners?

      When searching for contacts in Zoho Campaigns I am sometimes caught out when I don't select the filter option "Inactive users". So it appears that I have some contacts missing, until I realise that I need to select that option. Campaigns Support have

    • One Contact with Multiple Accounts with Portal enabled

      I have a contact that manages different accounts, so he needs to see the invoices of all the companies he manage in Portal but I found it not possible.. any idea? I tried to set different customers with the same email contact with the portal enabled and

    • End Date in Zoho Bookings

      When I give my appointments a 30 minutes time I would expect the software not to even show the End Time.  But it actually makes the user pick an End Time.  Did I just miss a setting?  

    • Next Page