Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard

Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard



Howdy Tech Wizards! Welcome to a fresh week of kaizen.

This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of  Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget. 

Overview

For the use case mentioned above, 
  • Build and deploy a web scraping service on a cloud. 
  • Create a Zoho CRM variable to store the response from this web scraping service.
  • Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
  • Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.

1. Create a Web Scraping Service 

Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
  • Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
  • Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post. 

Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function. 

Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.

2. Create a Zoho CRM Variable

Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.

Step 2: Click on the Create New Variable button and provide the necessary details.



Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.

For a step-by-step guide on creating a variable, check here. 

3. Write a Custom Schedule Function 

Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.

Step 2: Click on the Create New Schedule button and provide a name to the schedule.



Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.



Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements. 

First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API. 

You can also copy and paste the code from here. 

response = invokeurl
[
url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/"
type :GET
];
info response;
scrape_response = response.toString();
variable = Map();
variable.put("id","5545974000007587001");
variable.put("value",scrape_response);
variables_list = List();
variables_list.add(variable);
param = Map();
param.put("variables",variables_list);
response = invokeurl
[
url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001"
type :PUT
parameters:param.toString()
connection:"crm_oauth_connection"
];
info response;

For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.

Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.



For more information about Schedules, check this help page. 

Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable. 

4. Code your Widget

Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.

Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.

Copy and paste the following code in your widget.html file. 

<html>
<head>
</head>
<body>
<div id="blogs-table-container"></div>
<script type="text/javascript">
ZOHO.embeddedApp.on("PageLoad", function () {
var conn_name = "test_api_connection";
var req_data = {
"method": "GET",
"param_type": 1
};
ZOHO.CRM.CONNECTION.invoke(conn_name, req_data)
.then(function (data) {
console.log(data)
const div = document.getElementById("blogs-table-container");
div.innerHTML = data.details.statusMessage.variables[0].value;
})
})
ZOHO.embeddedApp.init();
</script>
</body>
</html>

5. Upload the Widget and Check Your Results

Step 1: Execute the zet validate and zet pack commands to validate and pack the widget. 

Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button. 

Fill in the necessary details and upload the package. 



Step 3: Let us now check the dashboard widget from the Zoho CRM Home page. 



We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites. 

Hope this post was insightful and help your interest. 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!

Stay tuned until we circle back to you next Friday!

Cheers!

-------------------------------------------------------------------------------------------------------------------

Recommended Reads

-------------------------------------------------------------------------------------------------------------------
Alert
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!  


    • Recent Topics

    • Iteration through a list - Coming up against a "Failure to update function" error

      Hi there! I've been attempting to get a deluge script working and am running into an error that I have been unable to resolve. The error I am getting is Failed to update function Error at line :18. Improper Statement. Error might be due to missing ';'

    • Can you modify "Last Activity Time" in deluge? If so what's the field name?

      I need to perform some bulk modifications on records in the Leads module, but I need to avoid changing the "last activity time" or "date modified" because I am using those fields to filter and sort leads for follow-up action. I cannot find an answer anywhere

    • How do I define a weekend

      I noticed the default for weekends does not seem to include Saturdays. How can i define weekends to include both Saturdays and Sundays? Thank you.

    • 【Zoho CRM】ポータル機能のアップデート

      ユーザーの皆さま、こんにちは。コミュニティチームの藤澤です。 今回は「Zoho CRM アップデート情報」の中からポータル機能のアップデートをご紹介します。 今回の機能アップデートにより、CRMのポータルへのログインがより簡単にできるようになりました。 【目次】 SAMLベースのシングルサインオンについて ポータルへのアクセスリンク送付について 今回のアップデートにより、アイデンティティプロバイダー（IdP）を利用している組織において、SAMLベースのSSO（シングルサインオン）を有効化できるようになりました！

    • Getting error during inserting a record in form of zoho people using zoho api

      import requests import json # Set your access token and Zoho People API base URL access_token = '1000.XXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXX' api_base_url = 'https://people.zoho.in/people/api/' # Set the form name and data to be inserted form_link_name

    • UI Arabic

      can i change the member portal UI to arabic in zoho community?

    • Resume template to include profile photo

      Hi, We would love the Resume Template to be able to include their profile photo. How can this be done? Miriam

    • Marking a form as 'done'?

      Hi! I've got a form that staff fill in when moving stock from location to location. I see the entries they've sent across and I input them into my stock control system. Is there a way to mark a form as 'done'? So that I know which entries I've input,

    • Zoho CRM - best way to search an account and assign to a deal

      Hi Everyone I am looking for some advice. I want to find the best way to complete the below steps. We have a deal and once it reaches a certain stage we need to allocate a supplier / vendor to this deal along with the salesperson. I want to add (ideally

    • Zoho Rating

      When I close a ticket, the zoho system automatically sends an email to the customer for them to leave a rating. When they leave a rating, the ticket re-opens and I have to close it again. When I close it again, the zoho system sends the customer an email

    • Zoho GC Flow is not showing in IM Channels

      I have added my social IM channels in zoho desk. Created a GC Flow to automate conversations. But Guided conversations are not showing in any platforms I have chosen; WhatsApp Instagram Messenger Web (Business Messaging) In the flow I mentioned these

    • Sharing Knowledge Base articles across multiple departments

      It would be useful to share some Knowledge Base articles across multiple departments where they are applicable, rather than having to go into other departments to find the article you're looking for. For example. Our reception uses the 'Admin' desk whereas our IT guys use the 'Support' desk, however both divisions would find KB articles about our company intranet useful. Reception does not have access to the support desk, so cannot see articles created in the Support KB. Perhaps you could install

    • Contacts marked as Spam are shown as Contacts in tickets.

      Hi Zoho, If a ticket is marked manually as Spam you get an option to set the contact as Spam as well. It would be great if there was another option to delete this Contact. This would be based on the understanding that it wouldn't be marked automatically

    • Is possible to use ${webhookTrigger.payload.street==null?"No street":${webhookTrigger.payload.street}} for to detect inline a null parameter

      Hi Team I need to know if it's possible to evaluate inline the following parameter, with the objetive to detect a null value ? I would like to use ${webhookTrigger.payload.street==null?"No street":${webhookTrigger.payload.street}} Regards, Pablo

    • Real Estate CRM

      How can I tailor my CRM for real estate? I had seen an image where the CRM included property tabs.

    • Issue with Bulk Edit Triggering Function Multiple Times in Zoho Report

      In our Zoho report, we have observed an issue where executing a function during a bulk edit operation runs the function multiple times—once for each selected report entry. This behavior is causing errors due to overlapping or redundant executions. Our

    • Set recurring tasks in Zoho Support?

      Hello,   We use the task function in Zoho Support to set reminders for "due dates" of important events, such as lease expirations, etc. Is it possible to set some of these as recurring yearly? As of now, we have to reset every task manually after one year. Is there a way to have it reset automatically?   Additionally, is there a way to import a list of tasks and their dates into Zoho Calendar? I have not been able to do it from Zoho Support to Calendar. I tried exporting tasks from Zoho Support,

    • Help - How Can I Enter a Hourly Rate (Global Cost) for our Team for Profitability Reporting?

      Hi everyone, I'm hoping someone can help as I am at a loss and am quite shocked (if true) at the lack of functionality in Zoho Books. We run a company in which we do hourly jobs for clients and we bill based on the hours of work completed. Projects are

    • Schedule a regular backup

      In CRM I have a regular backup setup. I cannot find an option to backup Desk? I would like to take a backup before I test syncing products between CMR and Desk.

    • Limit who can submit a support ticket to your site?

      Does anyone have a use case for needing to limit who can submit tickets via your help portal? Some of our customers want to have end-users view our knowledge base articles and see tickets their IT groups have submitted but they do not want end-users to

    • Is there a way to request a password?

      We add customers info into the vaults and I wanted to see if we could do some sort of "file request" like how dropbox offers with files. It would be awesome if a customer could go to a link and input a "title, username, password, url" all securely and it then shows up in our team vault or something. Not sure if that is safe, but it's the best I can think of to be semi scalable and obviously better than sending emails. I am open to another idea, just thought this would be a great feature.  Thanks,

    • Allow us to disable the Reopen Ticket button after a ticket is closed

      Disabling the Reopen Ticket will allow us to have more control over the ticket's flow. According to support, this is currently not possible. Please add this to let us, the customers, mold the ticketing system into something that we can use for our business

    • How to Enforce Mandatory Fields (Category/Sub-category) Before Closing a Ticket?

      Hi, I'm currently using Zoho Desk and encountering an issue where agents are able to close tickets without filling out mandatory fields, specifically Category and Sub-category. I’ve already set them both as a mandatory field, but it can still be bypassed

    • Admin Control for Default Email Templates in Zoho Desk

      Hi Zoho Desk Team, We would like to request a feature enhancement related to default email templates. Currently, agents can select and set their own default email templates when replying to tickets. However, we believe this setting should be managed centrally

    • Lock out of account after password update

      This is a bit of a catch 22 situation. I updated my account password using password generator in ZohoValut. Now I am locked out of everything, as I cannot get into OneAuth for 2FA, as the password is randomly generated, without getting into ZohoValut,

    • Guidance on Making Zoho Desk Connections Available for All Data Centers

      Hi Team, I’m currently developing an application using Zoho Desk connections to manage OAuth for my third-party products. Could you please advise on the steps required to make it available across all data centers? Looking forward to your thoughts on

    • Multibrand Help Center - Share knowledge base catgories between multiple departments

      Hello, I would like to know if it is possible to share categories between multiple departments when the multi brand feature is enabled. So that then one portal exists per department, but certain categories are visible in multiple portals. After all, we

    • Unable to upload document error on zoho sign

      i am creating a document using laravel php and here is my code and i get this reponse {"code":2005,"message":"Unable to upload the document","status":"failure"} $path = storage_path($file); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'https://sign.zoho.com/api/v1/requests');

    • Cloud to cloud folder migration from Sharepoint to Workdrive

      I am aware of OneDrive migration: https://help.zoho.com/portal/en/kb/workdrive/migrations/microsoft/articles/migrate-from-onedrive-to-zoho-workdrive#Whats_migrated would this cover a Sharepoint migration meaning I can move a whole Sharepoint (team) folder

    • Zoho Sign / Prefill By You - Editable by Them!

      Zoho Sign seems to have two field options available - an empty field for signer(s) to complete or a field that has been prefilled by you, that becomes read only once sent.  What I would like is an option on the pre-filled by you fields, to leave them

    • Kiosk Studio wrap-up | How our community used kiosks in 2024

      Hello, everyone! Happy new year! The end of 2024 has been busy, and 2025 promises to be bigger and better. As we ring in the new year, let's rewind and look at Kiosk Studio, our no-code customization tool. The past 300 days have seen the CRM community

    • Zoho Crm - Inventory Integration

      Hello, I need help about Map Fields between CRM and Inventory. For Example, I can't match amount of stock between these 2. I can put opening stock number  in Inventory but I can't see it on CRM. Also I want to see categories and parent categories on Inventory, in CRM too. Best Regards.

    • Inventory Valuation Method Feature

      Zoho added another feature so called Inventory Valuation Method of each Item. This is actually good to see and it will benefit us so much but I have a question on this. For Existing Items that already have transactions, Zoho made it a default and assumed

    • Where is the customization and extendibility of zoho inventory?

      After delving into zoho one subscription to test out systems we need for our business, I'm really disappointed after working in Zoho Inventory. Its features and customizability are extremely lacking compared to the other tools like CRM. In our case we

    • Organization Variables - Restrict Access

      Currently, there is no way to restrict the access to organization variables. This leads to a problem when storing API related values that should be kept secret as anyone with access to create and edit email templates, workflow rules, or inventory templates

    • Zoho takes too long to load...

      I am based in Europe and in the last couple of weeks, Zoho CRM takes minutes even to load modules. Is there a specific reason this is happening? Is this happening to other people?

    • Modules are continuously loading

      Hi! We are not able to open the task modules, it keeps continuously loading

    • Execution failure when fetching a Name field from another app

      In creator I have one app [Employee Directory] with a form [Employees]. One of the fields is a name field called [Name]. Here is a sample that works, if executed from the Employees app: test_Employee = Employees[Name.first_name == "Alex"]; info test_Employee.Name;

    • Accessing Subform Data using getRecordByID

      I am getting data from Zoho Creator using deluge zoho.creator.getRecordByID task, from another application. The report contains a subform, and that subform is in the detail view of the report. I do not appear to have any subform data in the JSON response,

    • Allow customer to select meeting duration

      I would love to see a feature where I can give the customer the option to select the meeting duration they require based on a dropdown list which I have predefined in the service settings. For example: 15, 30, 45, 60, 90 minutes.

    • Next Page