Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard
Howdy Tech Wizards! Welcome to a fresh week of kaizen.
This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget.
Overview
For the use case mentioned above,
- Build and deploy a web scraping service on a cloud.
- Create a Zoho CRM variable to store the response from this web scraping service.
- Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
- Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.
1. Create a Web Scraping Service
Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
- Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
- Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post.
Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function.
Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.
2. Create a Zoho CRM Variable
Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.
Step 2: Click on the Create New Variable button and provide the necessary details.
Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.
For a step-by-step guide on creating a variable, check here.
3. Write a Custom Schedule Function
Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.
Step 2: Click on the Create New Schedule button and provide a name to the schedule.
Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.
Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements.
First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API.
You can also copy and paste the code from here.
response = invokeurl [ url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/" type :GET ]; info response; scrape_response = response.toString(); variable = Map(); variable.put("id","5545974000007587001"); variable.put("value",scrape_response); variables_list = List(); variables_list.add(variable); param = Map(); param.put("variables",variables_list); response = invokeurl [ url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001" type :PUT parameters:param.toString() connection:"crm_oauth_connection" ]; info response; |
For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.
Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.
For more information about Schedules, check this help page.
Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable.
4. Code your Widget
Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.
Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.
Copy and paste the following code in your widget.html file.
<html> <head> </head> <body> <div id="blogs-table-container"></div> <script type="text/javascript"> ZOHO.embeddedApp.on("PageLoad", function () { var conn_name = "test_api_connection"; var req_data = { "method": "GET", "param_type": 1 }; ZOHO.CRM.CONNECTION.invoke(conn_name, req_data) .then(function (data) { console.log(data) const div = document.getElementById("blogs-table-container"); div.innerHTML = data.details.statusMessage.variables[0].value; }) }) ZOHO.embeddedApp.init(); </script> </body> </html> |
5. Upload the Widget and Check Your Results
Step 1: Execute the zet validate and zet pack commands to validate and pack the widget.
Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button.
Fill in the necessary details and upload the package.
Step 3: Let us now check the dashboard widget from the Zoho CRM Home page.
We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites.
Hope this post was insightful and help your interest.
If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!
Stay tuned until we circle back to you next Friday!
Cheers!
-------------------------------------------------------------------------------------------------------------------
Recommended Reads
- Variables - Overview, GET Variable API, UPDATE Variable API
- Scheduler - Create Schedules in Automation
- Functions - Overview, Deluge, Kaizen posts on Functions
- Widgets - Overview, Installation and Package, Use Cases
-------------------------------------------------------------------------------------------------------------------
Previous Kaizen: Kaizen #139: Using Functions in Client Script and Handling Timeouts | Kaizen: Home
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!
Access your files securely from anywhere
New to Zoho Recruit?
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Kiruthiga G
Danny
Sticky Posts
Kaizen #197: Frequently Asked Questions on GraphQL APIs
🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Kaizen #198: Using Client Script for Custom Validation in Blueprint
Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Celebrating 200 posts of Kaizen! Share your ideas for the milestone post
Hello Developers, We launched the Kaizen series in 2019 to share helpful content to support your Zoho CRM development journey. Staying true to its spirit—Kaizen Series: Continuous Improvement for Developer Experience—we've shared everything from FAQsKaizen #193: Creating different fields in Zoho CRM through API
🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! Commands
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Importing a new list into campaigns
I'm in the middle of switching my email platform to campaigns. I have a list that I want to import, and it overlaps with my existing Zoho CRM list. The fields in my Zoho CTM are more robust. Will this new list that I upload into my campaigns overwriteClient Script Payload Size Bug
var createParams = { "data": [{ "Name": "PS for PR 4050082000024714556", "Price_Request": { "id": "4050082000024714556" }, "Account": { "id": "4050082000021345001" }, "Deal": { "id": "4050082000023972001" }, "Owner": { "id": "4050082000007223004" }, "Approval_Status":Webform & spam
Hi, We set up 2 webform on our website, fowarding the content to Zoho CRM. Since it has been opened up, we are getting lot of spam message (for now about 20 a day). To lower the amount of false new leads we added the captcha field and new enquieries are send to the Approval Leads list. However we still get some spam. Is there any "anti spam" mechanism built in Zoho CRM, or how is the best way to avoid these kind of spam ? Thankswhen I email a invoice how can i see it was sent and also were i can go to see all emails sent
when I email a invoice how can i see it was sent and also were i can go to see all emails sent?Kaizen #208 - Answering your Questions | Functions, AI and Extensions
Hello Developers! Welcome back to a fresh week of Kaizen! We are grateful for your active participation in sharing feedback and queries for our 200th milestone. This week, we will answer the queries related to Functions and Extensions in Zoho CRM. 1.Export Invoices to XML file
Namaste! ZOHO suite of Apps is awesome and we as Partner, would like to use and implement the app´s from the Financial suite like ZOHO Invoice, but, in Portugal, we can only use certified Invoice Software and for this reason, we need to develop/customize on top of ZOHO Invoice to create an XML file with specific information and after this, go to the government and certified the software. As soon as we have for example, ZOHO CRM integrated with ZOHO Invoice up and running, our business opportunitiesShowing description in timesheet and timelogs.
I am wondering if it’s possible in version 5 of Zoho People to have the description show by default or with a manipulation on the user’s part. Let me show you what I mean. As you can see this is the view for the users. Now if they want to see the fullHow can I see content of system generated mails from zBooks?
System generated mails for offers or invices appear in the mail tab of the designated customer. How can I view the content? It also doesn't appear in zMail sent folder.New in Cadences: WhatsApp follow-ups, upgraded limits, and options for add-ons
Hello everyone, We're rolling out two key updates to help you engage better and scale smarter with Cadences in Zoho CRM. Reach customers on WhatsApp, directly from Cadences Previously, Cadences have enabled you to automate follow-ups through emails, calls,CRM Blueprint Notification by Cliq
Dear Zoho team, In Workflow, there is nofication by cliq, but in blueprint, there is no option as cliq notification. I think it is very convenient to get notified by Cliq , as there are multi modules in apps, but we will always check CliqsZoho People Attendance Regularization – Wrong Total Hours Displayed
While using Zoho People, I observed that the attendance regularization is showing wrong total hours when applied to past dates. For example, if a check-in is added at 10:00 AM and check-out at 6:00 PM for a previous date, the system sometimes calculatesSync Contacts in iOS
What does the "Sync Contacts" feature in the iOS Zoho Mail app do?Live webinar: Craft the ideal sales pitch deck with Show
Every great sale starts with a great story. And your pitch deck? That’s where the story takes shape. But too often, these presentations end up looking generic, overloaded with text, or lacking structure. The good news is, it's easier to fix than you think!Project Statuses
Hi All, We have projects that sometimes may not make it through to completion. As such, they were being marked as "Cancelled". I noticed that these projects still show as "Active" though which seems counter intuitive. In fact, the only way I can get them👋 Welcome to the Zoho MCP Community
Hello all, glad to have you here! This is your space for everything AI agents, MCP tools, and intelligent business apps. This community is for you — developers, partners, creators, and businesses exploring how agents can transform work. Whether you’reSuitability of Zoho One (Single User License) for Multi-State GST Compliance & Cost Analysis
Hello Zoho Team, I am an e-commerce business owner selling on platforms like Amazon, Flipkart, and Meesho, and I'm currently using their fulfillment warehouses. I have two GSTIN registrations and am planning to register for an additional 2-3 to expandDNS Manager
Where Can I find my DNS manager so I can link this to click funnels or AWEBERForwarder
Hi, I tried to add a forwarder from which emails are sent to my main zoho account email . However, it asks me for a code that should be received at the forwarder email, which is still not activated to send to my zoho emial account. So how can I get theForwarder
Hi, I tried to add a forwarder from which emails are sent to my main zoho account email . However, it asks me for a code that should be received at the forwarder email, which is still not activated to send to my zoho emial account. So how can I get theHow do I sync multiple Google calendars?
I'm brand new to Zoho and I figured out how to sync my business Google calendar but I would also like to sync my personal Google calendar. How can I do this so that, at the very least, when I have personal engagements like doctor's appointments, I canNeed to extract date from datetime field
I have a datetime field and need only the date part from it. I am unable to find a built-in function that would be <DateTime>.Date(). I don't think I want to go the string conversion route of converting the datetime to string and then parsing out values and create a date out of it. Any one out there has a better solution to this? Thanks in adavnce. Regards Moiz Tankiwala Smart Training & IT SolutionsHow to Hide Article Links in SalesIQ Answer Bot Responses
I have published an article in SalesIQ, and the Answer Bot is fetching the data and responding correctly. However, it is also displaying the article link, which I don’t want. How can I remove the link so that only the message is shown?India Tech Support
Is there no phone tech support number for India? And no chat facility either?additional accounts
If I brought 5 emails to my account. Can I later buy additional emails.Issue in Zoho People Regularization – Incorrect Hour Calculation
I have noticed that when applying attendance regularization in Zoho People for previous dates, the total working hours are not calculated correctly. For example, even if the check-in is 10:00 AM and check-out is 6:00 PM, the system shows an incorrectWhy I am unable to configure Zoho Voice with my Zoho CRM account?
I have installed Zoho Voice in my Zoho CRM, but as per the message there is some config needed in Zoho Voice interface. But when I click on the link given in the above message, I get an access denied page.Issue with Hour Calculation in Zoho People Attendance Module
I have noticed an issue in the attendance regularization feature of Zoho People. When trying to regularize past dates, the total working hours are not calculated correctly. For example, if I enter a check-in and check-out time for a previous day, theCliq Meeting Calls No Audio and Screen Share
When in a Cliq channel meeting, the audio does not work at all on pc. When i use my phone as audio source, screen share on pc does not work. I have updated audio drivers but the strangest thing is that during a 1 on 1 call, it works well. Therefore theBug in Total Hour Calculation in Regularization for past dates
There is a bug in Zoho People Regularization For example today is the date is 10 if I choose a previous Date like 9 and add the Check in and Check out time The total hours aren't calculated properly, in the example the check in time is 10:40 AM checkWork anniversary and birthdays on connect
Hello, I like the idea of having employee's work anniversary and birthdays on the dashbaord. Do you have to have the employee complete this information themselves in connect settings, or does it pull from their directory settings? (ie. we use Zoho oneAlias Email Id already exists
Hi I'm trying to create an alias : contact @ yoavarielevy.co.il but i get the message Alias Email Id already exists I had an account with the same name but I deleted it. Can you help? Thanx YoavBANK FEED - MAYBANK , provider from YODLEE IS NOT WORKING
As per topic, the provider YODLEE is not working for the BANK FEED. It have been reported since 2023 Q3, and second report on 2023 Q4. now almost end of 2024 Q1, and coming to 2024 Q2. Malaysia Bank Maybank is NOT working. can anyone check on this issue?Feature Request: Ability to Set a Custom List View as Default for All Users
Dear Zoho CRM Support Team, We would like to request a new feature in Zoho CRM regarding List Views. Currently, each user has to manually select or favorite a custom list view in order to make it their default. However, as administrators, we would likeAdding Multiple Products (Package) to a Quote
I've searched the forums and found several people asking this question, but never found an answer. Is ti possible to add multiple products to a quote at once, like a package deal? This seems like a very basic function of a CRM that does quotes but I can'tZoho Commerce in multiple languages
When will you be able to offer Zoho Commerce in more languages? We sell in multiple markets and want to be able to offer a local version of our webshop. What does the roadmap look like?Compensation | Salary Packages - Hourly Wage Needed
The US Bureau of Labor Statistics says 55.7% of all workers in the US are paid by the hour. I don't know how that compares to the rest of the world, but I would think that this alone would justify the need for having an hourly-based salary package option.Introducing Assemblies and Kits in Zoho Inventory
Hello customers, We’re excited to share a major revamp to Zoho Inventory that brings both clarity and flexibility to your inventory management experience! Presenting Assemblies and Kits We’re thrilled to introduce Assemblies and Kits, which replaces theHow to create auto populate field based on custom module in Zoho CRM?
Hello, i'm still new to Zoho CRM and work as administrator in my company. Currently, I'm configuring layout for Quotes Module. So, the idea is, I've created a read-only field in Quotes called "Spec". I want this field automatically filled with Specificationwebinar registration confirmation are landing in SPMA folders
I am trialing zoho webinar and do not have access to custom domains. When I test user registrations, they are working but the resulting confirmation email is landing in a spam folder. How can I avoid this?Making digital signatures accessible to all: Introducing accessibility controls in Zoho Sign
Hi there! At Zoho Sign, we are committed to building an inclusive digital experience for all our users. As part of our ongoing efforts to align with Web Content Accessibility Guidelines (WCAG), we’re updating the application with support that will goNext Page