Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard
Howdy Tech Wizards! Welcome to a fresh week of kaizen.
This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget.
Overview
For the use case mentioned above,
- Build and deploy a web scraping service on a cloud.
- Create a Zoho CRM variable to store the response from this web scraping service.
- Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
- Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.
1. Create a Web Scraping Service
Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
- Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
- Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post.
Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function.
Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.
2. Create a Zoho CRM Variable
Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.
Step 2: Click on the Create New Variable button and provide the necessary details.
Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.
For a step-by-step guide on creating a variable, check here.
3. Write a Custom Schedule Function
Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.
Step 2: Click on the Create New Schedule button and provide a name to the schedule.
Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.
Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements.
First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API.
You can also copy and paste the code from here.
response = invokeurl [ url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/" type :GET ]; info response; scrape_response = response.toString(); variable = Map(); variable.put("id","5545974000007587001"); variable.put("value",scrape_response); variables_list = List(); variables_list.add(variable); param = Map(); param.put("variables",variables_list); response = invokeurl [ url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001" type :PUT parameters:param.toString() connection:"crm_oauth_connection" ]; info response; |
For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.
Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.
For more information about Schedules, check this help page.
Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable.
4. Code your Widget
Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.
Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.
Copy and paste the following code in your widget.html file.
<html> <head> </head> <body> <div id="blogs-table-container"></div> <script type="text/javascript"> ZOHO.embeddedApp.on("PageLoad", function () { var conn_name = "test_api_connection"; var req_data = { "method": "GET", "param_type": 1 }; ZOHO.CRM.CONNECTION.invoke(conn_name, req_data) .then(function (data) { console.log(data) const div = document.getElementById("blogs-table-container"); div.innerHTML = data.details.statusMessage.variables[0].value; }) }) ZOHO.embeddedApp.init(); </script> </body> </html> |
5. Upload the Widget and Check Your Results
Step 1: Execute the zet validate and zet pack commands to validate and pack the widget.
Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button.
Fill in the necessary details and upload the package.
Step 3: Let us now check the dashboard widget from the Zoho CRM Home page.
We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites.
Hope this post was insightful and help your interest.
If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!
Stay tuned until we circle back to you next Friday!
Cheers!
-------------------------------------------------------------------------------------------------------------------
Recommended Reads
- Variables - Overview, GET Variable API, UPDATE Variable API
- Scheduler - Create Schedules in Automation
- Functions - Overview, Deluge, Kaizen posts on Functions
- Widgets - Overview, Installation and Package, Use Cases
-------------------------------------------------------------------------------------------------------------------
Previous Kaizen: Kaizen #139: Using Functions in Client Script and Handling Timeouts | Kaizen: Home
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!
Access your files securely from anywhere
New to Zoho Recruit?
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Kiruthiga G
Danny
Sticky Posts
Kaizen #197: Frequently Asked Questions on GraphQL APIs
🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Kaizen #198: Using Client Script for Custom Validation in Blueprint
Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Celebrating 200 posts of Kaizen! Share your ideas for the milestone post
Hello Developers, We launched the Kaizen series in 2019 to share helpful content to support your Zoho CRM development journey. Staying true to its spirit—Kaizen Series: Continuous Improvement for Developer Experience—we've shared everything from FAQsKaizen #193: Creating different fields in Zoho CRM through API
🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! Commands
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Replace Lookup fields ID value with their actual name and adding inormation from subforms
Hi everyone, I wanted to see if someone smarter than me has managed to find any solutions to two problems we have. I will explain both below. To start we are syncing data from Zoho CRM to Zoho Analytics and I will use the Sales Order module when givingWebhook from Zobot to Zoho Flow fails
I'm trying to connect from zobot to zoho flow. When testing in zflow, I am receiving all entered data from the connector correctly. The SalesIQ connector's "outputreaction" is {} (is this normal or is there a problem?). But as soon as I try my chat botTransition from Sole Proprietorship to GmbH (Limited Liability Company) – Best Approach in Zoho Books / Zoho One
Hello everyone, I am currently operating under a Zoho One plan with a sole proprietorship in Switzerland. As of January 1st, 2026, I will be incorporating a new legal entity – a GmbH (Swiss equivalent of a Limited Liability Company). While the businessPresenting ABM for Zoho CRM: Expand and retain your customers with precision
Picture this scenario: You're a growing SaaS company ready to launch a powerful business suite, and are looking to gain traction and momentum. But as a business with a tight budget, you know acquiring new customers is slow, expensive, and often deliversBest way to display complex Bookings Consultation Descriptions on Zoho Site?
I am a new user so apologies if this has been asked before. I couldn't find any answers in the forum. We offer 18 complex Consultations to our subscribers. Our current platform lets me put detail on these Consultations thoroughly (200-300 words) duringNo Response from Zoho Support in 8 Days - Typical?
I have a couple of issues I'm trying to work through. Initially, I was getting support from support@zohofsm.com, but I have not received a response in 8 days (11 on another question). Is this typical? Can I pay for support? For context, I am not spammingCliq iOS can't see shared screen
Hello, I had this morning a video call with a colleague. She is using Cliq Desktop MacOS and wanted to share her screen with me. I'm on iPad. I noticed, while she shared her screen, I could only see her video, but not the shared screen... Does Cliq iOS is able to display shared screen, or is it somewhere else to be found ? RegardsAge Calculation
I've attempted to calculate the age of someone based on their birthday input by using the formula field. It works but I don't want all those decimals on there. I then tried to use "set variable" after birthday input but I get a field type mismatch, long vs. floating. Any ideas would be wonderful.Access Denied
I am iOS Developer and updating our clients project and shifted ZohoDeskPortalCore SDKs from cocoapods to SPM and changed few lines of code but now i am get access denied, the help center app is unavailable. please contact administrator.Using Zoho Desk to support ISMS process
Hi, I am evaluating using Zoho Desk for security incident management. This seems to be aligned with Zoho Desk purpose as its just another type of incident. However in security incident management, ideally I can link incidents (tickets) with a risk fromFiltering Tickets based on Email headers
We're starting to get a lot more junk coming into our Zoho Desk, which is then triggering unnecessary email alerts to agents. Once thing we could do to cut this junk in half, is to filter tickets based on email headers. Any email containing the `List-Unsubscribe`Field Dependency Not Working on Detail Page in Zoho Desk
Hi Support Team, I’ve created field dependencies between two fields in Zoho Desk, and they are working correctly on the Create and Edit layouts. However, on the Detail page, the fields are not displaying according to the dependencies I’ve set — they appearTaxJar vs Avalara
Hi, I'm evaluating adoption of a sales-tax service for US based business. Anyone else have experience with TaxJar and Zoho Books? I am a Zoho One subscriber so anticipate needing to use Flow to make this work. It seems like Avalara are simply too expensiveHow to check Leads with no Task (open activity)
Hi everyone, I was wondering if there’s a way to view leads that don’t have any tasks assigned or open activities linked to them.What can we do on our end to improve the Answer bot answers?
Hi, I'm using the Answer bot card in the Codeless bot builder. I've input several questions and their answers in the FAQ section to feed the Answer bot. The text is all in French, as this is the language our customers communicate in. I've tried testingZoho Books Sandbox environment
Hello. Is there a free sandbox environment for the developers using Zoho Books API? I am working on the Zoho Books add-on and currently not ready to buy a premium service - maybe later when my add-on will start to bring money. Right now I just need aTaxes for EU B2B Transactions
Currently, ZC doesn't seem to have a procedure for validating VAT numbers of businesses purchasing in another EU state, and removing local VAT is valid. This is essential for all inter EU B2B trade.Zoho Down
I have a drop in my Zoho One services.Customer Parent Account or Sub-Customer Account
Some of clients as they have 50 to 300 branches, they required separate account statement with outlet name and number; which means we have to open new account for each branch individually. However, the main issue is that, when they make a payment, theyIssue with Inline Images in Email Reply via Zoho Desk API
Hi, I am attempting to send inline images in an email reply using the Zoho Desk API, but the images are not being displayed inline for the recipient. I have followed this documentation: https://desk.zoho.com/DeskAPIDocument#Uploads https://desk.zoho.com/DeskAPIDocument#Threads#Threads_SendEmailReplyHow to upload file to Connect using API?
Hi there. I looked at the API documentation and nowhere did it mention how to use the API method to upload a file even though it is mentioned that it is possible to be done so. Please help.Problem for EU users connecting Zoho CRM through Google Ads for Enhanced conversions
Has anyone else experienced this problem when trying to connect Zoho CRM through Google Ads interface to setup enhanced conversions? Did you guys get it fixed somehow? The Problem: The current Google Ads integration is hardcoded to use Zoho's US authenticationHow can I setup Zoho MCP with Chat GPT
I can set up custom connections with Chat GPT but I cat an error when I try to set it up. The error is: "This MCP server can't be used by ChatGPT to search information because it doesn't implement our specification: search action not found" Thoughts?Group Tax in Service Line Items
Hi FSM Team! I noticed that when you update a tax in the service line item the group tax is not showing up as an option. Let me know what can be done thank you!Zoho Campaigns - Why do contacts have owners?
When searching for contacts in Zoho Campaigns I am sometimes caught out when I don't select the filter option "Inactive users". So it appears that I have some contacts missing, until I realise that I need to select that option. Campaigns Support haveFSM Improvement Idea - Show an Import button when there is no data
I am setting up FSM for a client and I noticed that there is no option to import data, see screenshot below. Even when you click Create Contact there is only an option to Import from Zoho Invoice. It is only after you add at lease 1 record that the ImportWhatsapp Limitation Questions
Good day, I would like to find out about the functionality or possibility of all the below points within the Zoho/WhatsApp integration. Will WhatsApp buttons ever be possible in the future? Will WhatsApp Re-directs to different users be possible basedZoho FSM API Delete Record
Hi FSM Team, It would be great if you could delete a record via API. Thank you,Insert auto number from main form into subform rows
Hello. I'm trying to take from my main form "order number" which i have setup as an auto generated number into every line created in my subform. So when a row is created in my subform i want the "order number " from the main form to be inserted automatically.Adding a developer for editing the client application with a single user license
Hi, I want to know that I as a developer I developed one application and handed over to the customer who is using the application on a single user license. Now after6 months customer came back to me and needs some changes in the application. Can a customerFunction #4: Schedule Customer Statements
Regularly sending statements to customers is an imperative part of many business processes as it helps foster strong customer relationships and provides timely guidance on payments. While you can generate the statement of accounts and have it sent overLimiting search or dependencies with an asterisk "*".
I have a form with several dependency fields with options still developing for each field. Since these options were developing and not yet ready to be a selection in the field, I placed a filter for the dropdown field. In this filter, I selected fieldsCollaps Notes
There are times when long/large notes are added to a record i.e. Accounts or Deals etc. Currently, the full note is displayed in the notes related list section. It would be great if by default only 5 to 10 rows of the note are displayed when the noteImproved RingCentral Integration
We’d like to request an enhancement to the current RingCentral integration with Zoho. RingCentral now automatically generates call transcripts and AI-based call summaries (AI Notes) for each call, which are extremely helpful for support and sales teams.How to overcome limitations in meetings
As a company, one of our deliverables is a meeting between two other companies, where we act as facilitators. So, if we recorded this meeting in Zoho CRM, it should be connected to 2 accounts, 2 contacts, and 1 campaign (a campaign, in our use, is theCross Data Center Support for 1:1 Chats with External Users
Hello Zoho Cliq Team, We hope you're doing well. We appreciate the recent enhancement that enables cross data center collaboration in external channels, which has already improved communication across distributed teams. However, we’ve noticed that thisSupport Bots and Automations in External Channels
Hello Zoho Cliq Team, How are you? We actively use Zoho Cliq for collaboration, including with our external developers. For this purpose, external channels are a key tool since they work seamlessly within the same interface as all of our other channelsAnswer Bot and Personalized Questions
Hi there, I have the same problem using the SalesIQ Answer Bot and the Zoho Desk Answer Bot (which really need different names, to be honest, in order to avoid confusion...) Customers that visit our website ask questions in the form of "What do you do?"Handling Greetings/Small Talk at the Beginning of a Zobot Conversation
Hello everyone, I’m currently configuring a **Zobot** in Zoho SalesIQ and everything is working as expected, except for one specific scenario at the very beginning of the conversation. My target audience has the habit of starting with a **greeting orRegex in Zoho Mail custom filters is not supported - but it works!
I recently asked Zoho for help using regex in Zoho Mail custom filters and was told it was NOT supported. This was surprising (and frustrating) as regex in Zoho Mail certainly works, although it does have some quirks* To encourage others, here are 3 regexNext Page