Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard

Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard



Howdy Tech Wizards! Welcome to a fresh week of kaizen.

This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of  Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget. 

Overview

For the use case mentioned above, 
  • Build and deploy a web scraping service on a cloud. 
  • Create a Zoho CRM variable to store the response from this web scraping service.
  • Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
  • Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.

1. Create a Web Scraping Service 

Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
  • Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
  • Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post. 

Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function. 

Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.

2. Create a Zoho CRM Variable

Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.

Step 2: Click on the Create New Variable button and provide the necessary details.



Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.

For a step-by-step guide on creating a variable, check here. 

3. Write a Custom Schedule Function 

Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.

Step 2: Click on the Create New Schedule button and provide a name to the schedule.



Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.



Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements. 

First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API. 

You can also copy and paste the code from here. 

response = invokeurl
[
url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/"
type :GET
];
info response;
scrape_response = response.toString();
variable = Map();
variable.put("id","5545974000007587001");
variable.put("value",scrape_response);
variables_list = List();
variables_list.add(variable);
param = Map();
param.put("variables",variables_list);
response = invokeurl
[
url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001"
type :PUT
parameters:param.toString()
connection:"crm_oauth_connection"
];
info response;

For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.

Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.



For more information about Schedules, check this help page. 

Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable. 

4. Code your Widget

Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.

Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.

Copy and paste the following code in your widget.html file. 

<html>
<head>
</head>
<body>
<div id="blogs-table-container"></div>
<script type="text/javascript">
ZOHO.embeddedApp.on("PageLoad", function () {
var conn_name = "test_api_connection";
var req_data = {
"method": "GET",
"param_type": 1
};
ZOHO.CRM.CONNECTION.invoke(conn_name, req_data)
.then(function (data) {
console.log(data)
const div = document.getElementById("blogs-table-container");
div.innerHTML = data.details.statusMessage.variables[0].value;
})
})
ZOHO.embeddedApp.init();
</script>
</body>
</html>

5. Upload the Widget and Check Your Results

Step 1: Execute the zet validate and zet pack commands to validate and pack the widget. 

Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button. 

Fill in the necessary details and upload the package. 



Step 3: Let us now check the dashboard widget from the Zoho CRM Home page. 



We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites. 

Hope this post was insightful and help your interest. 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!

Stay tuned until we circle back to you next Friday!

Cheers!

-------------------------------------------------------------------------------------------------------------------

Recommended Reads

-------------------------------------------------------------------------------------------------------------------
Alert
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!  


    • Recent Topics

    • Schedule Zoho CRM reports only on Business Days

      Hello, Is it possible to schedule reports only on business days ? We currently get daily sales reports on weekend which has no value since the sales team doesn't work on weekends. Thanks

    • Zoho Payroll's Year in Review 2024

      As we roll into 2025, we'd like to pay tribute to all the milestones we hit in 2024! From releasing out new features that streamlined your workflows to updates that made payroll management smoother, we’ve had a prolific year—all while keeping you, our

    • Recurring Events Not Appearing in "My Events" and therefore not syncing with Google Apps

      We use the Google Sync functionality for our events, and it appears to have been working fine except: I've created a set of recurring events that I noticed were missing from my Google Apps calendar. Upon further research, it appears this is occurring

    • Multiplying Weight of product by Quantity

      I am facing an issue with creating a report that consolidates the total sales volume in kilograms. I have already specified the weight for each product. I have also aggregated the total sales quantity. The key question is: how can I create a report that

    • Confirmation prompt before a custom button action is triggered

      Have you ever created a custom button and just hoped that you/your users are prompted first to confirm the action? Well, Zoho knows this concept. For example, in blueprint, whenever we want to advance to the next state by clicking the transition, it is

    • Frontal interview scheduling - room availability in office using Google Workspace?

      Hi, We're using Zoho Recruit as our ATS and Google Workspace as our email, calendar and resources management. We want to use the interview feature to schedule an in-person (frontal) interview with the applicants. How can we sync the room resources availability

    • Add and Remove Agents from Departments and Groups in Zoho One

      Hi Zoho Flow Team, We hope you're doing well. Currently, Zoho Flow provides an action to add an agent to a group in zoho one, but there is no action to remove an agent from a group or a department. Another action that we find missing is the option to

    • Explication sur comment mettre en place des règles d'affichage ou "layout Rules"

      J'ai passé plus d'une heure hier avec le support et je n'ai rien compris !! Je suis lecteur assidu des guides (je "RTFM") qui ne sont absolument pas orienté "client" chez Zoho, et je tiens à le rappeler ici . Dans la documentation on m'indique un cas

    • Introducing Hiring Pipeline for Vendor Portal

      Keeping vendors informed about candidate progress is often challenging, leading to communication gaps and repeated follow-ups. To address this issue, we've released an update to the Vendor Portal feature that lets you choose to display candidates' hiring

    • Unlocking New Horizons: A Year in Review

      As we bid farewell to 2024, let's celebrate and revisit the key highlights of the year. From adding a new edition to cross-platform enhancements, here’s a roundup of all the feature updates designed to simplify accounting, optimize financial management,

    • Address Grabber function for Zoho

      I converted from ACT to Zoho. With ACT, I used an add-on called AddressGrabber to scrape the contact information from leads that I buy and contact information contained on emails and websites and directly add it as a new lead or contact. Does anyone know

    • Integrate zoom with zoho bookings please; or add optional times in zoho meetings

      Just like events - these online meetings like zoom need to be integrated with zoho bookings, and there needs to be option for customer to book a time slot. It should not be dictated by CRM user.

    • Add Owner to deluge-created module record note

      Is it possible to include the "owner" aka "creator", of a Note when creating it via delulge? This sets "superadmin" as the Note creator. I need to override it. notemap = Map(); notemap.put("Parent_Id",program_contact_id); notemap.put("Note_Content",program_contact_data.get('Note'));

    • Directly Edit, Filter, and Sort Subforms on the Details Page

      Hello everyone, As you know, subforms allow you to associate multiple line items with a single record, greatly enhancing your data organization. For example, a sales order subform neatly lists all products, their quantities, amounts, and other relevant

    • Unique and Random IDs in Zoho Forms: Organize and Secure Your Data Efficiently

      When it comes to form submissions, organizing and identifying entries effectively is crucial. Zoho Forms offers two versatile ID generation options for submissions: Unique ID and Random ID. Each serves distinct purposes, providing flexibility to meet

    • Sort By Date - Deluge

      I have the following code, which normally works to sort calls by created time. Every once in a while, it doesn't work and something sneaks through in the wrong order and I can't figure out why. calls = zoho.crm.searchRecords("Calls","(Owner:equals:" +

    • Iteration through a list - Coming up against a "Failure to update function" error

      Hi there! I've been attempting to get a deluge script working and am running into an error that I have been unable to resolve. The error I am getting is Failed to update function Error at line :18. Improper Statement. Error might be due to missing ';'

    • Can you modify "Last Activity Time" in deluge? If so what's the field name?

      I need to perform some bulk modifications on records in the Leads module, but I need to avoid changing the "last activity time" or "date modified" because I am using those fields to filter and sort leads for follow-up action. I cannot find an answer anywhere

    • How do I define a weekend

      I noticed the default for weekends does not seem to include Saturdays. How can i define weekends to include both Saturdays and Sundays? Thank you.

    • 【Zoho CRM】ポータル機能のアップデート

      ユーザーの皆さま、こんにちは。コミュニティチームの藤澤です。 今回は「Zoho CRM アップデート情報」の中からポータル機能のアップデートをご紹介します。 今回の機能アップデートにより、CRMのポータルへのログインがより簡単にできるようになりました。 【目次】 SAMLベースのシングルサインオンについて ポータルへのアクセスリンク送付について 今回のアップデートにより、アイデンティティプロバイダー（IdP）を利用している組織において、SAMLベースのSSO（シングルサインオン）を有効化できるようになりました！

    • Getting error during inserting a record in form of zoho people using zoho api

      import requests import json # Set your access token and Zoho People API base URL access_token = '1000.XXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXX' api_base_url = 'https://people.zoho.in/people/api/' # Set the form name and data to be inserted form_link_name

    • UI Arabic

      can i change the member portal UI to arabic in zoho community?

    • Resume template to include profile photo

      Hi, We would love the Resume Template to be able to include their profile photo. How can this be done? Miriam

    • Marking a form as 'done'?

      Hi! I've got a form that staff fill in when moving stock from location to location. I see the entries they've sent across and I input them into my stock control system. Is there a way to mark a form as 'done'? So that I know which entries I've input,

    • Zoho CRM - best way to search an account and assign to a deal

      Hi Everyone I am looking for some advice. I want to find the best way to complete the below steps. We have a deal and once it reaches a certain stage we need to allocate a supplier / vendor to this deal along with the salesperson. I want to add (ideally

    • Zoho Rating

      When I close a ticket, the zoho system automatically sends an email to the customer for them to leave a rating. When they leave a rating, the ticket re-opens and I have to close it again. When I close it again, the zoho system sends the customer an email

    • Zoho GC Flow is not showing in IM Channels

      I have added my social IM channels in zoho desk. Created a GC Flow to automate conversations. But Guided conversations are not showing in any platforms I have chosen; WhatsApp Instagram Messenger Web (Business Messaging) In the flow I mentioned these

    • Sharing Knowledge Base articles across multiple departments

      It would be useful to share some Knowledge Base articles across multiple departments where they are applicable, rather than having to go into other departments to find the article you're looking for. For example. Our reception uses the 'Admin' desk whereas our IT guys use the 'Support' desk, however both divisions would find KB articles about our company intranet useful. Reception does not have access to the support desk, so cannot see articles created in the Support KB. Perhaps you could install

    • Contacts marked as Spam are shown as Contacts in tickets.

      Hi Zoho, If a ticket is marked manually as Spam you get an option to set the contact as Spam as well. It would be great if there was another option to delete this Contact. This would be based on the understanding that it wouldn't be marked automatically

    • Is possible to use ${webhookTrigger.payload.street==null?"No street":${webhookTrigger.payload.street}} for to detect inline a null parameter

      Hi Team I need to know if it's possible to evaluate inline the following parameter, with the objetive to detect a null value ? I would like to use ${webhookTrigger.payload.street==null?"No street":${webhookTrigger.payload.street}} Regards, Pablo

    • Real Estate CRM

      How can I tailor my CRM for real estate? I had seen an image where the CRM included property tabs.

    • Issue with Bulk Edit Triggering Function Multiple Times in Zoho Report

      In our Zoho report, we have observed an issue where executing a function during a bulk edit operation runs the function multiple times—once for each selected report entry. This behavior is causing errors due to overlapping or redundant executions. Our

    • Is there a Kanban view of Tasks across all Projects?

      As the title indicates, I could use a Kanban view of my Tasks across my Projects. If it's there, I don't see it. If it isn't there, I'd like to submit this as a feature request. Thx.

    • Set recurring tasks in Zoho Support?

      Hello,   We use the task function in Zoho Support to set reminders for "due dates" of important events, such as lease expirations, etc. Is it possible to set some of these as recurring yearly? As of now, we have to reset every task manually after one year. Is there a way to have it reset automatically?   Additionally, is there a way to import a list of tasks and their dates into Zoho Calendar? I have not been able to do it from Zoho Support to Calendar. I tried exporting tasks from Zoho Support,

    • Help - How Can I Enter a Hourly Rate (Global Cost) for our Team for Profitability Reporting?

      Hi everyone, I'm hoping someone can help as I am at a loss and am quite shocked (if true) at the lack of functionality in Zoho Books. We run a company in which we do hourly jobs for clients and we bill based on the hours of work completed. Projects are

    • Schedule a regular backup

      In CRM I have a regular backup setup. I cannot find an option to backup Desk? I would like to take a backup before I test syncing products between CMR and Desk.

    • Limit who can submit a support ticket to your site?

      Does anyone have a use case for needing to limit who can submit tickets via your help portal? Some of our customers want to have end-users view our knowledge base articles and see tickets their IT groups have submitted but they do not want end-users to

    • Is there a way to request a password?

      We add customers info into the vaults and I wanted to see if we could do some sort of "file request" like how dropbox offers with files. It would be awesome if a customer could go to a link and input a "title, username, password, url" all securely and it then shows up in our team vault or something. Not sure if that is safe, but it's the best I can think of to be semi scalable and obviously better than sending emails. I am open to another idea, just thought this would be a great feature.  Thanks,

    • Allow us to disable the Reopen Ticket button after a ticket is closed

      Disabling the Reopen Ticket will allow us to have more control over the ticket's flow. According to support, this is currently not possible. Please add this to let us, the customers, mold the ticketing system into something that we can use for our business

    • How to Enforce Mandatory Fields (Category/Sub-category) Before Closing a Ticket?

      Hi, I'm currently using Zoho Desk and encountering an issue where agents are able to close tickets without filling out mandatory fields, specifically Category and Sub-category. I’ve already set them both as a mandatory field, but it can still be bypassed

    • Next Page