Kaizen #140 - Integrating Blog feed scraping service into Zoho CRM Dashboard
Howdy Tech Wizards! Welcome to a fresh week of kaizen.
This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget.
Overview
For the use case mentioned above,
- Build and deploy a web scraping service on a cloud.
- Create a Zoho CRM variable to store the response from this web scraping service.
- Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
- Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.
1. Create a Web Scraping Service
Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
- Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
- Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post.
Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function.
Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.
2. Create a Zoho CRM Variable
Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.
Step 2: Click on the Create New Variable button and provide the necessary details.
Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.
For a step-by-step guide on creating a variable, check here.
3. Write a Custom Schedule Function
Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.
Step 2: Click on the Create New Schedule button and provide a name to the schedule.
Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.
Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements.
First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API.
You can also copy and paste the code from here.
response = invokeurl [ url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/" type :GET ]; info response; scrape_response = response.toString(); variable = Map(); variable.put("id","5545974000007587001"); variable.put("value",scrape_response); variables_list = List(); variables_list.add(variable); param = Map(); param.put("variables",variables_list); response = invokeurl [ url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001" type :PUT parameters:param.toString() connection:"crm_oauth_connection" ]; info response; |
For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.
Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.
For more information about Schedules, check this help page.
Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable.
4. Code your Widget
Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.
Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.
Copy and paste the following code in your widget.html file.
<html> <head> </head> <body> <div id="blogs-table-container"></div> <script type="text/javascript"> ZOHO.embeddedApp.on("PageLoad", function () { var conn_name = "test_api_connection"; var req_data = { "method": "GET", "param_type": 1 }; ZOHO.CRM.CONNECTION.invoke(conn_name, req_data) .then(function (data) { console.log(data) const div = document.getElementById("blogs-table-container"); div.innerHTML = data.details.statusMessage.variables[0].value; }) }) ZOHO.embeddedApp.init(); </script> </body> </html> |
5. Upload the Widget and Check Your Results
Step 1: Execute the zet validate and zet pack commands to validate and pack the widget.
Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button.
Fill in the necessary details and upload the package.
Step 3: Let us now check the dashboard widget from the Zoho CRM Home page.
We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites.
Hope this post was insightful and help your interest.
If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!
Stay tuned until we circle back to you next Friday!
Cheers!
-------------------------------------------------------------------------------------------------------------------
Recommended Reads
- Variables - Overview, GET Variable API, UPDATE Variable API
- Scheduler - Create Schedules in Automation
- Functions - Overview, Deluge, Kaizen posts on Functions
- Widgets - Overview, Installation and Package, Use Cases
-------------------------------------------------------------------------------------------------------------------
Previous Kaizen: Kaizen #139: Using Functions in Client Script and Handling Timeouts | Kaizen: Home
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!
Access your files securely from anywhere
New to Zoho Recruit?
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Kiruthiga G
Danny
Sticky Posts
Kaizen #152 - Client Script Support for the new Canvas Record Forms
Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achievedKaizen #142: How to Navigate to Another Page in Zoho CRM using Client Script
Hello everyone! Welcome back to another exciting Kaizen post. In this post, let us see how you can you navigate to different Pages using Client Script. In this Kaizen post, Need to Navigate to different Pages Client Script ZDKs related to navigation A.Kaizen #210 - Answering your Questions | Event Management System using ZDK CLI
Hello Everyone, Welcome back to yet another post in the Kaizen Series! As you already may know, for the Kaizen #200 milestone, we asked for your feedback and many of you suggested topics for us to discuss. We have been writing on these topics over theKaizen #197: Frequently Asked Questions on GraphQL APIs
🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Kaizen #198: Using Client Script for Custom Validation in Blueprint
Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Sending workflow notifications using popular chat services
Hello everyone, We have introduced instant and scheduled notifications on some of the most popular chat platforms to facilitate easy collaboration, quick action, and wider reach. Workflow notifications can be sent to the following chat platforms: ZohoWebhook not firing.
I created a webhook using the Web UI, it looks very nice and the testing worked without an issue, but when i save/ update a ticket, the webhook is not firing. Here are the details of the web-hook i get from using the API "modifiedTime": "2019-10-22T09:23:37.380Z",Exported Slide Image Lacks Portion of the Background Image
This does not always happen, but when I export (or "download") the rendered image of a slide, it sometimes lacks a portion of the background image. I created a sample slide deck to demonstrate it and shared it with the admins. It is also publicly availableZoho Projects - Task Owner filter at Project level
Hi Projects Team, The feature requests I would like to raise is the ability to create a custom view at the project level for projects with tasks owned by a user or users. For example "Ashley's Projects" custom view might contain a list of project in whichHow do I increase the email attachment size in Zoho CRM ?
It looks like I'm limited to 10MB when sending an attachment using the email widget on a record in Zoho CRM. Is there a way to increase the size? Or can I use some other tool? From what I'm reading online, I'm maxed out at 10MB. Any insight would be greatlyAdding Images to a Quote in Zoho CRM
We are currently preparing to use Quotes in Zoho CRM and we are building out our Quote templates. We came across an issue of not being to add Images of the products to the Quote - specifically in the body of the Quote templates. This is a problem,Applications built with Zoho Creator
Hi, I’m really interested in seeing how others have built their application using Zoho Creator, especially those designed for external users (clients, vendors, or the public). If you’ve developed something along those lines and don’t mind sharing, I’dSuper Admin Logging in as another User
How can a Super Admin login as another user. For example, I have a sales rep that is having issues with their Accounts and I want to view their Zoho Account with out having to do a GTM and sharing screens. Moderation Update (8th Aug 2025): We are workingIn arattai received message can't be deleted
The issue has been noticed in following: arattai app (Android) arattai app (Window) arattai web While the message posted by me may be deleted, the ones received from others can't be. The item <Delete> change to <Report> when the message is a receivedPrevent new Record Association
Hello all, We have a small sales organization therefore, it's helpful for everyone on the sales team to be able to view the full list of accounts to assist in preventing duplicate accounts from being created. However we want to prevent people from creatingUnified customer portal login
As I'm a Zoho One subscriber I can provide my customers with portal access to many of the Zoho apps. However, the customer must have a separate login for each app, which may be difficult for them to manage and frustrating as all they understand is thatIs it posssible to add Asap Widget on Wordpress?
I have tried to add the ASAP widget so users could iniciate chats and see the KB information but nothings seems to work. I have tried to add the script using a php snippet that adds the to the footers and also tried one for the header in the functioIs it possible to embed Youtube shorts?
Hi Zoho desk support, This is Ryan from Accuver America. While I'm trying to create a knowledge base article with embed video, I ran into this issue. "www.youtube.com refuse to connect" A little bit background is that because this video is recorded onRequire ticket resolution
Hi Zoho team, Is there a way to require resolution even if an agent did not use a blueprint? for example, our blueprint has a "resolve" transition but what if agent revoked blueprint and manually set the status of ticket to closed? Is there a way whereTables from ZohoSheets remove images when updated from source
I have a few tables from a ZohoSheet in a ZohoWriter document that will remove the images in the cells when I refresh from the source. The source still has the images in the table when I go to refresh. After updating from the source, as you can see theUnable to Receive Emails on Zoho Mail After Office 365 Coexistence Setup – Error: 553 Relaying Disallowed
Hello, My domain name is bigniter.com, and I’ve been using Zoho Mail as my email service provider without any issues. Recently, I followed the steps outlined in the Zoho documentation to enable Coexistence with Office 365: 🔗 https://www.zoho.com/mail/help/adminconsole/coexistence-with-office365.html#multi-serverAPI Pagination Error: 'from' Parameter Limit
Hello, I am encountering an error while paging through the Zoho Desk API results: Status code: 422 - {"errorCode":"UNPROCESSABLE_ENTITY","message":"The value passed for field 'from' exceeds the range of '0-4999'."} Is 5000 the maximum number of recordsHow to go to the next open ticket in the queue when agents closes ticket
Zoho Desk When agent closes a ticket - eg when they choose 'Send and Close" - where is the setting that automatically redirects them to the next open ticket in the queue?External download link limit
Can You please help us to understand this For Zoho WorkDrive external users, the download limit is a maximum of 5 GB total download size and a maximum of 50 first-level files and folders What is the meaning of first level? We are using these files inCRM verify details pop-up
Was there a UI change recently that involves the Verify Details pop-up when changing the Stage of a Deal to certain things? I can't for the life of me find a workflow or function, blueprint, validation rule, layout rule ect that would randomly make itAutomatically Add Recurring Zoho Meeting Events to Zoho Calendar / Zoho Meeting Calendar
Hello Zoho Meeting Team, Hope you are doing well. We would like to request an enhancement regarding recurring meetings created inside Zoho Meeting. At the moment, when we schedule a recurring meeting in Zoho Meeting, it does not appear in Zoho CalendarThe difference between Zoho Marketing Automation and Zoho Campaigns
Greetings Marketers! This post aims to differentiate between Zoho Marketing Automation and Zoho Campaigns. By the time you get to the end of the post, you will be able to choose a product that objectively suits you. What is Zoho Marketing Automation?Custom templates for calendar report
What about being able to design custom templates for the calendar report, as well as for other types of reports? I think more users are waiting for this.Print a price list or price book
Hi Community. Am I right in concluding that Zoho has no functionality to print a price list from either Zoho CRM, Zoho Inventory or Zoho Books? I won't get stuck on the fact that Zoho doesn't sync price books between Zoho CRM and Books/Inventory (moreDisable payment thank-you emails
Hello, can someone please tell me how to disable sending of the "Payment Thank-You" emails?Maximum tags possible in Contacts Records
I read in some documentation that Zoho allows a total of 200 tags across all records. Is this correct? So is it not possible to have one tag per record for 500 records?Zoho inbuilt Telephony made a lot of issues!
Hi there, I am a user that I am working with zoho inbuilt telephony around 1 month. Non of my colleagues are happy with this app! most of the time customer cannot hear my customer service team, customers say our voice is breaking. whenever Telephony supportIntegrate your Outlook/ Office 365 inbox with Zoho CRM via Graph API
Hello folks, In addition to the existing IMAP and POP options, you can now integrate your Outlook/Office 365 inbox with Zoho CRM via Graph API. Why did we add this option? Microsoft Graph API offers a single endpoint to access data from across Microsoft’sAny way to "Pay with Check" or "Refund with Check" for Credit Notes?
When we have a Bill in Zoho Books, we can select the "Pay with Check" option which then allows us to print/cut the check directly out of Zoho Books. When we created a Credit Note and want to refund the customer, is there any way to Refund with Check,CRM Mobile reports
When our engineers finish a job they like to email the customer a job report. This is best done todate as an email template but we can find no way to include an image field from that module. Is there any other options?Items Below Reorder Point Report?
Is there a way to run a report of Items that are below the Reorder Point? I don't see this as a specific report, nor can I figure out how to customize any of the other stock reports to give me this information. Please tell me I'm missing something sUnify All Zoho Video Meeting Experiences into One Standardized Platform
Hi Zoho Team, We would like to share an important user experience concern regarding the current state of video meeting functionality across the Zoho ecosystem. The Problem Within Zoho, there are multiple ways to initiate or schedule a video meeting: ZohoWhen Zoho Tables Beta will be open to EU data center
Hello all, We in EU are looking at you all using and testing and are getting jealous :) When we will be able to get into the beta also? We don't mind testing and playing with beta software. Thank you!Start Form on a different page (i.e., hide form pages)?
I have a Zoho form that uses the `Field Alias - Prefill URL` feature. My goal is to have a pre-filled field that directs the user to a specific starting page in the form. For example: The URL will have a field alias that will auto-populate a field withMicrosoft Teams now available as an online meeting provider
Hello everyone, We're pleased to announce that Zoho CRM now supports Microsoft Teams as an online meeting provider—alongside the other providers already available. Admins can enable Microsoft Teams directly from the Preferences tab under the MeetingsStandard Payment Term is not pulled from account to quotation
Hey Team There seems to be something off. I do have "Net 30" as my default payment term in Zoho Books for my customers. If, from the customer overview or quote section, I create a new Quotation, the payment terms field stays blank and doesn't get thePassing the CRM
Hi, I am hoping someone can help. I have a zoho form that has a CRM lookup field. I was hoping to send this to my publicly to clients via a text message and the form then attaches the signed form back to the custom module. This work absolutely fine whenHow can we disable "My Requests"?
We are not using this functionality in our system at all and our users get confused.Update Subform in specific field(Status) without affecting other fields in Zoho CRM using Deluge?
Scenario: PRODUCT Module(change name Plot) in any product status change From Available to Booked then on PROJECT Custom Module have Subform So, Subform name is Property Details now in that Subform 1 field is STATUS that update according to product statusPayPal payment received recording problem
Hi, A little while back one of our customers used the PayPal payment option to pay an invoice For some reason though the payment is showing up twice within the Payments section of the invoice! Instead of setting the invoice value to ZERO, it now shows a negative value Anyone else face this problem? I've checked PayPal and there is only 1 payment in reality... A bug? Actonia ps: for anyone from Zoho Customer Service or tech team, its Invoice 785 in our accountNext Page