Kaizen #140 - Integrating Blog feed scraping service into Zoho CRM Dashboard

Kaizen #140 - Integrating Blog feed scraping service into Zoho CRM Dashboard



Howdy Tech Wizards! Welcome to a fresh week of kaizen.

This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of  Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget. 

Overview

For the use case mentioned above, 
  • Build and deploy a web scraping service on a cloud. 
  • Create a Zoho CRM variable to store the response from this web scraping service.
  • Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
  • Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.

1. Create a Web Scraping Service 

Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
  • Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
  • Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post. 

Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function. 

Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.

2. Create a Zoho CRM Variable

Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.

Step 2: Click on the Create New Variable button and provide the necessary details.



Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.

For a step-by-step guide on creating a variable, check here. 

3. Write a Custom Schedule Function 

Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.

Step 2: Click on the Create New Schedule button and provide a name to the schedule.



Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.



Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements. 

First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API. 

You can also copy and paste the code from here. 

response = invokeurl
[
url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/"
type :GET
];
info response;
scrape_response = response.toString();
variable = Map();
variable.put("id","5545974000007587001");
variable.put("value",scrape_response);
variables_list = List();
variables_list.add(variable);
param = Map();
param.put("variables",variables_list);
response = invokeurl
[
url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001"
type :PUT
parameters:param.toString()
connection:"crm_oauth_connection"
];
info response;

For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.

Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.



For more information about Schedules, check this help page. 

Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable. 

4. Code your Widget

Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.

Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.

Copy and paste the following code in your widget.html file. 

<html>
<head>
</head>
<body>
<div id="blogs-table-container"></div>
<script type="text/javascript">
ZOHO.embeddedApp.on("PageLoad", function () {
var conn_name = "test_api_connection";
var req_data = {
"method": "GET",
"param_type": 1
};
ZOHO.CRM.CONNECTION.invoke(conn_name, req_data)
.then(function (data) {
console.log(data)
const div = document.getElementById("blogs-table-container");
div.innerHTML = data.details.statusMessage.variables[0].value;
})
})
ZOHO.embeddedApp.init();
</script>
</body>
</html>

5. Upload the Widget and Check Your Results

Step 1: Execute the zet validate and zet pack commands to validate and pack the widget. 

Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button. 

Fill in the necessary details and upload the package. 



Step 3: Let us now check the dashboard widget from the Zoho CRM Home page. 



We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites. 

Hope this post was insightful and help your interest. 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!

Stay tuned until we circle back to you next Friday!

Cheers!

-------------------------------------------------------------------------------------------------------------------

Recommended Reads

-------------------------------------------------------------------------------------------------------------------
Alert
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!  


    • Sticky Posts

    • Kaizen #198: Using Client Script for Custom Validation in Blueprint

      Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Kaizen #226: Using ZRC in Client Script

      Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this

    • Kaizen #222 - Client Script Support for Notes Related List

      Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how

    • Kaizen #217 - Actions APIs : Tasks

      Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.

    • Kaizen #216 - Actions APIs : Email Notifications

      Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are

    Nederlandse Hulpbronnen


      • Recent Topics

      • Search Bar positioning

        Why is the Search bar on the far right when everything is oriented towards the left?

      • Import Error: Empty values for mandatory fields - Closing Date

        Hello, I've tried multiple times to import a CVS Potential list from another Zoho account. But the error message I get is: Empty values for mandatory fields - Closing Date There are valid dates in this field, so I don't understand why this error messages

      • Feature Requests - Contact Coloured Picklist Visibility & Field Visibility During Ticket Creation

        Hi Desk Team, I have 2 feature requests for you. Since Coloured Picklists are now available in Desk, It would be great if the colours were visible on the Related Details (Contact Information) when creating a ticket. In the screenshot below, I have 2 fields

      • Accounts Receivable Balances Differs in Balance Sheet, Customer Balance Report, AR Ageing Summary, and AR Ageing Details.

        Hello Zoho Accounts Receivable Balances Differs in Balance Sheet, Customer Balance Report, AR Ageing Summary, and AR Ageing Details. Please clarify and fix the issues here. Thanks

      • How to integrate XML with Zoho CRM

        Hi, I have an eCom service provider that gives me a dynamic XML that contains order information, clients, shipments... The XML link is the only thing I have. No Oath or key, No API get... I want to integrate it into Zoho CRM. I am not a developer nor

      • Feature Request - Ability to Customise Contact Info Card on Ticket Details View

        Hi Desk Team, I've added a "Contact Priority" and "Account Prioirty" field and it would be very useful to agents if they could see that information in the Contact Info card on the Ticket Details view. It would be great if we could choose some fields to

      • Tax in Quote

        Each row item in a quote has a tax value. At the total numbers at the bottom, there is also a Tax entry. If you select tax in both of the (line item, and the total), the tax doubles. My assumption is that the Tax total should be totalling the tax from

      • Zoho Flow integration with Facebook Messenger and Whatsapp

        Hi there,  any plans of adding integrations with Facebook Messenger and Whatsapp into Zoho Flow? Seems that more and more business are delivering automated updates such as "your order is received",  "your order has been shipped" and so on via these two platforms. Not sure if Whatsapp has the API access needed i am pretty sure that Facebook Messenger has... Kind regards Bo Thygesen 

      • Campaigns unsubscribe/manage preferences links

        Hi, Where can I edit the unscubscribe and manage preferences link in the footer of the email. I would like it so that when you click 'manage preferences' an form opens up that allows the person to choose what type of emails they do and don't wish to

      • email address somehow still not verified (?!)

        L.S. After creating a new email template in CRM I was about to send a group email to my clients, then Zoho CRM announced that they would change the sender address to some kind of Zoho-e-ddress because my email address "has not been verified". Not only

      • Marketing Tip #17: Add credibility to your online store with Review Widgets

        One of the fastest ways to build trust in an online store is to show real customer feedback right where people are deciding to buy. Third-party widgets let you embed things like Google Reviews, Instagram feeds, or even a WhatsApp chat button. These add

      • adding several team members to an Opportunity

        How can we add several team members to one opportunity for collaboration? I have researched and only found something called Deal Team which I cannot find in my CRM to configure.

      • Bulk update Profile Permissions

        Dears, What should we do if we add new forms or reports and need to update more than 20 permissions? Updating them one by one feels pretty harsh, doesn’t it?

      • Filter in fields from Jira extension

        We have installed the Jira extension so we can maken Jira issues from Zoho desk. In Zoho desk I can also see the Jira issue status for example but I can not filter on this field. I would like to setup an filter showing me the closed Jira issues. How can

      • Zoho Creator customer portal limitation | Zoho One

        I'm asking you all for any feedback as to the logic or reasoning behind drastically limiting portal users when Zoho already meters based on number of records. I'm a single-seat, Zoho One Enterprise license holder. If my portal users are going to add records, wouldn't that increase revenue for Zoho as that is how Creator is monetized? Why limit my customer portal to only THREE external users when more users would equate to more records being entered into the database?!? (See help ticket reply below.)

      • Add Reporting feature to display variance/change columns when comparing periods

        When running reports to compare periods (for example, Profit and Loss comparing current year to previous), I would like to be able to display variance columns in both (a) amount or (b) percentage.

      • Link Contacts to Billed Accounts

        Hello, I want to do a survey on all my customers of 2025. For that I want to select all contacts linked to accounts who where billed in 2025. How to I create this link to I can then use Zoho Survey with this database of contacts?

      • Pre-Zoho Sales Info - Best Way to Add to Desk / CRM

        My company has been using Zoho One since 2021, with sales data going back through 2020. However, we have been in business much longer, and we have historical sales information that we want to have at our fingertips when talking with customers (usually

      • CRM function REST API response format

        Is there a way to control the JSON response returned by the CRM function REST API? If I call a function using either OAuth or an API key it returns a 200 OK response with a string in the format shown below. I am using a particular feature of an external

      • Using MPN across multiple SKUs and inventory tracking

        I have several different SKU's that share a common MPN and would like to track inventory by MPN. SKU1 has MPN1 assigned SKU2 has MPN1 assigned Here is an example If I start with 5 of MPN 1 in stock I want each SKU1 and SKU2 to show as 5 in stock, If I

      • Unable to Access Application:

        Whenever I try to access my application from the desktop, say I am editing it and want to test something in the desktop environment I get: An error has occurred. An internal error has occurred. Please check the URL , or try refreshing the page I can edit

      • Evernote (ENEX) import limitations

        I have been with Evernote since 2010, but the latest price increase is ridiculous. I am currently testing Zoho Notebook as a replacement. I am impressed so far - if it were not for critical need to import legacy Evernote notes, I would 100% migrate to

      • Cannot see Application from Lookup field

        Hi all, I am trying to access data for an application on our account via a lookup field; however, the application doesn't appear in the dropdown at all. Can anyone shed any light on this, please? I have asked Zoho support; however, they're just as confused,

      • Zoho CRM Meetings Module Issues

        We have a use-case that is very common in today's world, but won't work in Zoho CRM. We have an SDR (Sales Development Rep) who makes many calls per day to Leads and Contacts, and schedules meetings for our primary Sales Reps. He does this by logging

      • Zoho Books integration sync from Zoho CRM does not work

        Hi Zoho Community & Zoho Support We just tried to get a sync some products into Zoho Books from CRM using the native sync and we're getting an error: "It looks like some mandatory fields you're trying to map are empty. Please provide valid field names

      • Disable Auto Send Email Notification in Invoice Creation

        Here is what I am facing Make payment via Razor Pay (All Good) Capture payload data on my webapp Send this data, along with other data to Zoho books to create the invoice However in this step - An email goes to the client + The Invoice remains as draft

      • P & L Sub-categorized accounts

        How can I show sub-categorized Income and Expense accounts on the P & L report?

      • Report showing Bill Details with Project and Sales Invoice Number

        Hi There, I am hoping that someone can help, I am looking for report that can show the bill and expense details along with project its as assigned to and the invoice number that the sales has been raised in. The goal is I can filter a customer/project

      • Advanced Payment for Inventory Items with serial numbers

        Hello, We sell equipment that we track the unique serial numbers on using Sales Orders. We can charge the customers an advanced payment, then the balance on delivery. We cannot figure out a way to do this in Books/Inventory: - Cannot part invoice a SO

      • Is it possible to restrict ZCRM user to see only custom views created by administrator

        I have segmented data in my CRM and I want to allow different users to be able to see only parts of it based on some criteria. I've tried to create and share a custom view, but then there is always an option for user to see all open lead for example.

      • Issues Logging into ZOHO

        Hello, one of my coworkers is having issues logging into ZOHO, she has requested a code when entering and the email is correct but she has not received the code. can you help us with this?

      • Google Fonts Integration in Pagesense Popup Editor

        Hello Zoho Pagesense Team, We hope you're doing well. We’d like to submit a feature request to enhance Zoho Pagesense’s popup editor with Google Fonts support. Current Limitation: Currently, Pagesense offers a limited set of default fonts. Google Fonts

      • Add Popup Rejection Metrics to Reports

        Hello Zoho PageSense Team, We would like to request improved reporting for popup interactions. Current Limitation: PageSense currently provides conversion data, but there is no clear visibility into: Popup rejections Popup closes (✕ button clicks) Dismissals

      • Ability to Reset / Reinitialize Popup Cookies

        Hello Zoho PageSense Team, We would like to request the ability to manually reset popup cookies. Current Limitation: At the moment, it is not possible to initiate a new popup cookie from the our side. Visitors who rejected or closed a popup will not see

      • Control Popup Cookie Expiration Duration

        Hello Zoho PageSense Team, We would like to request an enhancement related to popup cookie management. Current Limitation: Currently, PageSense popup cookies remain active for 365 days, and this duration cannot be modified by us. If a visitor closes or

      • Clone / Export Popup Design Across PageSense Projects

        Hello Zoho PageSense Team, We hope you’re doing well. We would like to request an enhancement that allows popup designs to be reused across different PageSense projects. Problem Statement: Currently, Zoho PageSense allows popups to be duplicated only

      • Are there settings for hyperlinks?

        Clicking a hyperlinked cell in Sheet creates this little pop-up with the actual hyperlink inside. Is it possible to have a 1-click link where if you click the cell it opens the link directly with no pop-up?

      • Automatically include all ticket attachments in the ticket resolution email

        Hello Zoho Community, We are implementing Zoho Desk in a real customer-facing production environment and have run into a limitation that is becoming a blocking requirement for our clients. The problem When a ticket is closed or resolved, Zoho Desk sends

      • Keep Zoho People Feature Requests in the Zoho People Forum

        Hello Zoho People Product Team, Greetings. We would like to submit a feature request regarding the handling of feature requests themselves, specifically for Zoho People. Issue: Feature Requests Being Moved to Zoho One Zoho People feature requests are

      • Can I write a check in Zoho Books with no associated bill?

        This currently does not seem possible, and I have a client that desperately needs this function if I am able to convert them with Quickbooks. Thank you in advance for your reply. 

      • Next Page