Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard
Howdy Tech Wizards! Welcome to a fresh week of kaizen.
This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget.
Overview
For the use case mentioned above,
- Build and deploy a web scraping service on a cloud.
- Create a Zoho CRM variable to store the response from this web scraping service.
- Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
- Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.
1. Create a Web Scraping Service
Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
- Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
- Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post.
Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function.
Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.
2. Create a Zoho CRM Variable
Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.
Step 2: Click on the Create New Variable button and provide the necessary details.
Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.
For a step-by-step guide on creating a variable, check here.
3. Write a Custom Schedule Function
Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.
Step 2: Click on the Create New Schedule button and provide a name to the schedule.
Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.
Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements.
First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API.
You can also copy and paste the code from here.
response = invokeurl [ url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/" type :GET ]; info response; scrape_response = response.toString(); variable = Map(); variable.put("id","5545974000007587001"); variable.put("value",scrape_response); variables_list = List(); variables_list.add(variable); param = Map(); param.put("variables",variables_list); response = invokeurl [ url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001" type :PUT parameters:param.toString() connection:"crm_oauth_connection" ]; info response; |
For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.
Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.
For more information about Schedules, check this help page.
Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable.
4. Code your Widget
Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.
Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.
Copy and paste the following code in your widget.html file.
<html> <head> </head> <body> <div id="blogs-table-container"></div> <script type="text/javascript"> ZOHO.embeddedApp.on("PageLoad", function () { var conn_name = "test_api_connection"; var req_data = { "method": "GET", "param_type": 1 }; ZOHO.CRM.CONNECTION.invoke(conn_name, req_data) .then(function (data) { console.log(data) const div = document.getElementById("blogs-table-container"); div.innerHTML = data.details.statusMessage.variables[0].value; }) }) ZOHO.embeddedApp.init(); </script> </body> </html> |
5. Upload the Widget and Check Your Results
Step 1: Execute the zet validate and zet pack commands to validate and pack the widget.
Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button.
Fill in the necessary details and upload the package.
Step 3: Let us now check the dashboard widget from the Zoho CRM Home page.
We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites.
Hope this post was insightful and help your interest.
If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!
Stay tuned until we circle back to you next Friday!
Cheers!
-------------------------------------------------------------------------------------------------------------------
Recommended Reads
- Variables - Overview, GET Variable API, UPDATE Variable API
- Scheduler - Create Schedules in Automation
- Functions - Overview, Deluge, Kaizen posts on Functions
- Widgets - Overview, Installation and Package, Use Cases
-------------------------------------------------------------------------------------------------------------------
Previous Kaizen: Kaizen #139: Using Functions in Client Script and Handling Timeouts | Kaizen: Home
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!
Access your files securely from anywhere
New to Zoho Recruit?
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Kiruthiga G
Danny
Sticky Posts
Kaizen #197: Frequently Asked Questions on GraphQL APIs
🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Kaizen #198: Using Client Script for Custom Validation in Blueprint
Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Celebrating 200 posts of Kaizen! Share your ideas for the milestone post
Hello Developers, We launched the Kaizen series in 2019 to share helpful content to support your Zoho CRM development journey. Staying true to its spirit—Kaizen Series: Continuous Improvement for Developer Experience—we've shared everything from FAQsKaizen #193: Creating different fields in Zoho CRM through API
🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! Commands
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Is there a way to set Document Owner/Sender via the API
When sending requests for zoho sign, it would seem zoho uses the id of the person that created the zoho api cred to determine the owner_id, is there a way to set a default for this?Transition Criteria Appearing on Blueprint Transitions
On Monday, Sept. 8th, the Transition criteria started appearing on our Blueprints when users hover over a Transition button. See image. We contacted Zoho support because it's confusing our users (there's really no reason for them to see it), but we haven'tDelegates should be able to delete expenses
I understand the data integrity of this request. It would be nice if there was a toggle switch in the Policy setting that would allow a delegate to delete expenses from their managers account. Some managers here never touch their expense reports, andAdd Save button to Expense form
A save button would be very helpful on the expense form. Currently there is a Save and Close button. When we want to itemize an expense, this option would be very helpful. For example, if we have a hotel expense that also has room service, which is aMultiple organizations under Zoho One
Hello. I have a long and complicated question. I have a Zoho One account and want to set it up to serve the needs of 6 organizations under the same company. Some of the Zoho One users need to be able to work in more than 1 organization’s CRM and otherIMAP Server not responding.
Trying to connect a phone via IMAP and getting "imap.zoho.com not responding." Is the server down, for maintenance or otherwise? I've tried this on two different devices and got the same error on both.Error AS101 when adding new email alias
Hi, I am trying to add apple@(mydomain).com The error AS101 is shown while I try to add the alias.Unbundle feature for composite items
We receive composite items from our vendors and sell them either individually or create other composite items out of them. So, there is a lot of bundling and unbundling involved with our composite items. Previously, this feature was supported in formRegarding the integration of Apollo.io with Zoho crm.
I have been seeing for the last 3 months that your Apollo.io beta version is available in Zoho Flow, and this application has not gone live yet. We requested this 2 months ago, but you guys said that 'we are working on it,' and when we search on GoogleOpen filtered deals from campaign
Do you think a feature like this would be feasible? Say you are seeing campaign "XYZ" in CRM. The campaign has a related list of deals. If you want to see the related deals in a deal view, you should navigate to the Deals module, open the campaign filter,How do you print a refund check to customer?
Maybe this is a dumb question, but how does anyone print a refund check to a customer? We cant find anywhere to either just print a check and pick a customer, or where to do so from a credit note.MTD SA in the UK
Hello ID 20106048857 The Inland Revenue have confirmed that this tax account is registered as Cash Basis In Settings>Profile I have set ‘Report Basis’ as “Cash" However, I see on Zoho on Settings>Taxes>Income Tax that the ‘Tax Basis’ is marked ‘Accrual'workflow not working in subform
I have the following code in a subform which works perfectly when i use the form alone but when i use the form as a subform within another main form it does not work. I have read something about using row but i just cant seem to figure out what to changeFetch data from another table into a form field
I have spent the day trying to work this out so i thought i would use the forum for the first time. I have two forms in the same application and when a user selects a customer name from a drop down field and would like the customer number field in theZoho Creator as LMS and Membership Solution
My client is interested in using Zoho One apps to deploy their membership academy offer. Zoho Creator was an option that came up in my research: Here are the components of the program/offer: 1. Membership portal - individual login credentials for eachRecord comment filter
Hi - I have a calendar app that we use to track tasks. I have the calendar view set up so that the logged in user only sees the record if they are assigned to the task. BUT there are instances when someone is @ mentioned in the record when they are notHow to View Part Inventory and Warehouse Location When Creating a Work Order in Zoho FSM
Hi everyone, We’re currently setting up Zoho FSM and would like to improve how our team selects parts when creating a Work Order. Right now, when we add a part or item to a Work Order, we can select it from our Zoho Inventory list but we don’t see anyFSM too slow today !!
Anybody else with problem today to loading FSM (WO, AP etc.)?CRM x WorkDrive: File storage for new CRM signups is now powered by WorkDrive
Availability Editions: All DCs: All Release plan: Released for new signups in all DCs. It will be enabled for existing users in a phased manner in the upcoming months. Help documentation: Documents in Zoho CRM Manage folders in Documents tab Manage filesNot able to Sign In in Zoho OneAuth in Windows 10
I recently reset my Windows 10 system, after the reset when I downloaded the OAuth app and tried to Sign In It threw an error at me. Error: Token Fetch Error. Message: Object Reference not set to an instance of an object I have attached the screenshotMapping a custom preferred date field in the estimate with the native field in the workorder
Hi Zoho, I created a field in the estimate : "Preferred Date 1", to give the ability to my support agent to add a preferred date while viewing the client's estimate. However, in the conversion mapping (Estimate to Workorder), I'm unable to map my customRuning RPA Agents on Headless Windows 11 Machines
Has anyone tried this? Anything to be aware of regarding screen resolution?The sending IP (136.143.188.15) is listed on spamrl.com as a source of spam.
Hi, it just two day when i am using zoho mail for my business domain, today i was sending email and found that message "The sending IP (136.143.188.15) is listed on https://spamrl.com as a source of spam" I hope to know how this will affect the deliveryDelegates - Access to approved reports
We realized that delegates do not have access to reports after they are approved. Many users ask questions of their delegates about past expense reports and the delegates can't see this information. Please allow delegates see all expense report activity,Split functionality - Admins need ability to do this
Admins should be able to split an expense at any point of the process prior to approval. The split is very helpful for our account coding, but to have to go back to a user and ask them to split an invoice that they simply want paid is a bit of an inWhat is a a valid JavaScript Domain URI when creating a client-based application using the Zoho API console?
No idea what this is. Can't see what it is explained anywhere.Is there a way to request a password?
We add customers info into the vaults and I wanted to see if we could do some sort of "file request" like how dropbox offers with files. It would be awesome if a customer could go to a link and input a "title, username, password, url" all securely and it then shows up in our team vault or something. Not sure if that is safe, but it's the best I can think of to be semi scalable and obviously better than sending emails. I am open to another idea, just thought this would be a great feature. Thanks,Single Task Report
I'd like a report or a way to print to PDF the task detail page. I'd like at least the Task Information section but I'd also like to see the Activity Stream, Status Timeline and Comments. I'd like to export the record and save it as a PDF. I'd like theAuto-response for closed tickets
Hi, We sometimes have users that (presumably) search their email inbox for the last correspondence with us and just hit reply - even if it's a 6 month old ticket... - this then re-opens the 6 month old ticket because of the ticket number in the email's subject. Yes, it's easy to 'Split as new Ticket', but I'd like something automated to respond to the user saying "this ticket has already been resolved and closed, please submit a new ticket". What's the best way to achieve this? Thanks, EdHow to Push Zoho Desk time logged to Zoho Projects?
I am on the last leg of my journey of finally automating time tracking, payments, and invoicing for my minutes based contact center company - I just have one final step to solve - I need time logged in zoho desk to add time a project which is associatedCannot access KB within Help Center
Im working with my boss to customize our knowledge base, but for some reason I can see the KB tab, and see the KB categories, but I cannot access the articles within the KB. We have been troubleshooting for weeks, and we have all permissions set up, customersExport to excel stored amounts as text instead of numbers or accounting
Good Afternoon, We have a quarterly billing report that we generate from our Requests. It exports to excel. However if we need to add a formula (something as simple as a sum of the column), it doesn't read the dollar amounts because the export storeswhy my account is private?
when i post on zohodesk see only agent onlyGetting ZOHO Invoice certified in Portugal?
Hello, We are ZOHO partners in Portugal and here, all the invoice software has to be certified by the government and ZOHO Invoice still isn´t certified. Any plans? Btw, we can help on this process, since we have a client that knows how to get the software certified. Thank you.Show/ hide specific field based on user
Can someone please help me with a client script to achieve the following? I've already tried a couple of different scripts I've found on here (updating to match my details etc...) but none of them seem to work. No errors flagged in the codes, it just500 Internal Server Error
I have been trying to create my first app in Creator, but have been getting the 500: Internal Server Error. When I used the Create New Application link, it gave me the error after naming the application. After logging out, and back in, the application that I created was in the list, but when I try to open it to start creating my app, it gives me the 500: Internal Server Error. Please help! Also, I tried making my named app public, but I even get the error when trying to do that.Client Script | Update - Client Script Support For Portals
Dear All! We are excited to announce the highly anticipated feature: Client Script support for Portals. We understand that many of you have been eagerly awaiting this enhancement, and we are pleased to inform you that this support is now live for allProfessional Plan not activated after payment
I purchased the Professional Plan for 11 users (Subscription ID: RPEU2000980748325) on 12 September 2025, and the payment has been successfully processed. However, even after more than 24 hours, my CRM account still shows “Upgrade” and behaves like aZoho Books - France
L’équipe de Zoho France reçoit régulièrement des questions sur la conformité de ses applications de finances (Zoho Books/ Zoho Invoice) pour le marché français. Voici quelques points pour clarifier la question : Zoho Books est un logiciel de comptabilitéhow to edit the converted lead records?
so I can fetch the converted leads records using API (COQL), using this endpoint https://www.zohoapis.com/crm/v5/coql and using COQL filter Converted__s=true for some reasons I need to change the value from a field in a converted lead record. When I tryNext Page