Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard

Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard



Howdy Tech Wizards! Welcome to a fresh week of kaizen.

This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of  Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget. 

Overview

For the use case mentioned above, 
  • Build and deploy a web scraping service on a cloud. 
  • Create a Zoho CRM variable to store the response from this web scraping service.
  • Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
  • Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.

1. Create a Web Scraping Service 

Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
  • Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
  • Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post. 

Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function. 

Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.

2. Create a Zoho CRM Variable

Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.

Step 2: Click on the Create New Variable button and provide the necessary details.



Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.

For a step-by-step guide on creating a variable, check here. 

3. Write a Custom Schedule Function 

Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.

Step 2: Click on the Create New Schedule button and provide a name to the schedule.



Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.



Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements. 

First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API. 

You can also copy and paste the code from here. 

response = invokeurl
[
url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/"
type :GET
];
info response;
scrape_response = response.toString();
variable = Map();
variable.put("id","5545974000007587001");
variable.put("value",scrape_response);
variables_list = List();
variables_list.add(variable);
param = Map();
param.put("variables",variables_list);
response = invokeurl
[
url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001"
type :PUT
parameters:param.toString()
connection:"crm_oauth_connection"
];
info response;

For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.

Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.



For more information about Schedules, check this help page. 

Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable. 

4. Code your Widget

Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.

Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.

Copy and paste the following code in your widget.html file. 

<html>
<head>
</head>
<body>
<div id="blogs-table-container"></div>
<script type="text/javascript">
ZOHO.embeddedApp.on("PageLoad", function () {
var conn_name = "test_api_connection";
var req_data = {
"method": "GET",
"param_type": 1
};
ZOHO.CRM.CONNECTION.invoke(conn_name, req_data)
.then(function (data) {
console.log(data)
const div = document.getElementById("blogs-table-container");
div.innerHTML = data.details.statusMessage.variables[0].value;
})
})
ZOHO.embeddedApp.init();
</script>
</body>
</html>

5. Upload the Widget and Check Your Results

Step 1: Execute the zet validate and zet pack commands to validate and pack the widget. 

Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button. 

Fill in the necessary details and upload the package. 



Step 3: Let us now check the dashboard widget from the Zoho CRM Home page. 



We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites. 

Hope this post was insightful and help your interest. 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!

Stay tuned until we circle back to you next Friday!

Cheers!

-------------------------------------------------------------------------------------------------------------------

Recommended Reads

-------------------------------------------------------------------------------------------------------------------
Alert
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!  


    • Recent Topics

    • Zoho account sign in with passkey

      Hello, I am trying to sign in using passkey, but the option doesn't show up in the web and is disabled in Oneauth on mobile, saying the admin has restricted the use. On the Admin page in Security MFA I can find no option for passkey. Help would be greatly

    • Pivot table with Text values - "Matrix Report"

      User Story - As a user, I would like the ability to display textual data in a two-way table, matrix format (text datatypes, not numerical datatypes displayed as a dimension) One major feature missing from the Pivot tables in Zoho Analytics is the ability

    • Canvas View in Zoho Recruit

      Is it possible or would it be possible to have the new 'Canvas View' in Zoho Recruit?

    • Can Wisestamp email signature be use with Zoho mail?

      Does a Wisestamp email signature work with Zoho mail?

    • WiseStamp

      WiseStamp is an excellent social media signature tool. It integrated seamlessly into Gmail and Thunderbird, plus a few more. Are there any plans to get this incredible app integrated into Zoho. check it out here: http://www.wisestamp.com/ thanks Tim

    • Merge tickets

      Hello, I am attempting to merge two tickets; however, the option to merge does not appear to be available. Could you kindly confirm if this feature has been relocated or removed? Additionally, could you please provide guidance on the correct process to

    • Add "Merge by Ticket ID" Option on Ticket Screen

      Dear Zoho Team, We would like to request a new feature for the ticket interface in Zoho Desk. Specifically, we propose adding a button to the ticket screen that allows users to merge the current ticket with another ticket by directly providing the second

    • Power BI connector (Zoho Creator) to Zoho Projects

      How can i connect power bi to Zoho Projects? domain is zoho.com How can i find workspace name, application link name and Report link name?

    • Update your Google connection with Zoho TeamInbox

      Dear all, Wishing you a Happy New Year! Google has recently updated its security policy to enhance user protection, requiring all third-party apps and services to use OAuth authentication and password-less login methods. This update impacts users who

    • Possible to backdate Stage shifts in Deals module?

      I want to enter my organisation's historical dealflow data into the CRM and it's important that the Stage history is showing the correct dates in the past of when it happened. Is there a way for me to modify the "Modified Time" in the Stage History Related

    • Easy way to delete attachments

      I've reached my data limit and would like to run a view/report, and mass delete attachments. Is there an easy, fast way to do this? Moderation Update: Post Summary: There are two features the post discusses a) Easy way to remove Email attachments Will

    • Sites Speed and Performance Grades

      I noticed that there are no recent inquiries or complaints about load speed or performance issues with Zoho Sites websites. However, I wanted to understand what Zoho has done to ensure that speed remains optimized, images are compressed and lazy loaded,

    • Integrating Quotes with leads Information

      Our business requires giving a quote to a lead (not Account / Potential - as we define it differently). Currently, Zoho CRM's Quotes are integrated with Accounts / Potentials and not with Leads. Is there a way I could get the Lead information to the Quotes

    • Flow to follow up on trade fair contacts

      Hi, Before we moved to Zoho we had some flows (sequences) in HubSpot to follow up on trade fair contacts. To explain further on this it had the following characteristics: New contacts could be added to the sequence When added a flow of communication started.

    • Unknown error occurred

      Hi, When we want to publish or edit a page in our website, we encounter with "Unknown error occurred" problem. I share a screenshot here. Our website is www.essoft.com. It happens every page. We want to solve this ASAP.

    • Zoho one

      I am starting Zoho one as a beginner and there is Zoho crm and Zoho sales which i needed to work on but it appears in unassigned apps. What Should i do now?

    • Dialing Microsoft Teams Phone Service via Zoho CRM

      I am using the VOIP option in Microsoft teams for my office phone system. I was hoping to have a way to dial numbers directly from Zoho CRM, but don't see anything in the Teams Integration or in the Telephony integration that will enable this. Does anyone

    • Emailing lookup field but placing this as an ID or number rather than text

      Hi there, First time poster and have been a user of Zoho Creator for approx 6weeks so forgive my ignorance as I learn to code. We have a need to send an email to a specific email address with some of the fields triggered by the submission of a form. In

    • Search mails in shared mailbox

      Hi everyone, is there a way to search mails in shared mailbox's? Search in streams or mail doesn't return anything from mails in shared mailboxes. Thanks! Rafal

    • User Emails Blocked

      Community: I keep running into issues where our users stop receiving notifications from CRM because their email addresses get blocked in on the backend some how. I reach out to support, they confirm, they fix, and we carry on, but then it happens again.

    • Why wont Zoho Support Grammarly!! --- PLEASE VOTE FOR THIS to show Zoho we need this

      The spell check and grammar in ZOHO are so buggy and a waste of time. Please support Grammarly! I'm sure I'm not the only one — there are other CRMs that support this. If you're not planning to add this feature, Please let others know before accepting

    • Is it possible to hide Developer Space for all user in Zoho Projects

      Hello! I am Zoho admin in a company and we want to use Zoho Project to manage projects, but after a few days of testing we are not able to "hide" the Developer Space from all kind of users except the admin. To sum up, I want to hide this for all users.

    • API (v2) Search Criteria using CONCAT

      With API I can search for a contact using First_Name and Last_Name. However, when I need to search the Contact Module using a full name — and because CRM does not provide an API for full name — I am not finding a way to do this in the traditional way

    • Weekly Tips: Stay Focused with Email Snooze!

      New Year, New Resolutions Being back at work also means being back to the constant barrage of messages from work and clients. The constant flood of incoming emails can lead to the missing of important messages, especially when you can't respond right

    • Schedule Zoho CRM reports only on Business Days

      Hello, Is it possible to schedule reports only on business days ? We currently get daily sales reports on weekend which has no value since the sales team doesn't work on weekends. Thanks

    • Zoho Payroll's Year in Review 2024

      As we roll into 2025, we'd like to pay tribute to all the milestones we hit in 2024! From releasing out new features that streamlined your workflows to updates that made payroll management smoother, we’ve had a prolific year—all while keeping you, our

    • Using tickets to train Zia

      Hi Team, I would like to know if there is any way that Zia can also learn from previous tickets in addition to the articles from the knowledge base. Since we have most of our knowledge curerently in the tickets and that this is hard to combine into a

    • Recurring Events Not Appearing in "My Events" and therefore not syncing with Google Apps

      We use the Google Sync functionality for our events, and it appears to have been working fine except: I've created a set of recurring events that I noticed were missing from my Google Apps calendar. Upon further research, it appears this is occurring

    • How can the direct link to a ticket be created from ticket #?

      Our agents will be using ZDesk but we will have to create direct link to tickets in another ERP by providing the Desk Ticket # Is there a way to create the direkt links from the ticket # alone? Do we have to use the API to get the direct links?

    • Multiplying Weight of product by Quantity

      I am facing an issue with creating a report that consolidates the total sales volume in kilograms. I have already specified the weight for each product. I have also aggregated the total sales quantity. The key question is: how can I create a report that

    • Confirmation prompt before a custom button action is triggered

      Have you ever created a custom button and just hoped that you/your users are prompted first to confirm the action? Well, Zoho knows this concept. For example, in blueprint, whenever we want to advance to the next state by clicking the transition, it is

    • Frontal interview scheduling - room availability in office using Google Workspace?

      Hi, We're using Zoho Recruit as our ATS and Google Workspace as our email, calendar and resources management. We want to use the interview feature to schedule an in-person (frontal) interview with the applicants. How can we sync the room resources availability

    • Add and Remove Agents from Departments and Groups in Zoho One

      Hi Zoho Flow Team, We hope you're doing well. Currently, Zoho Flow provides an action to add an agent to a group in zoho one, but there is no action to remove an agent from a group or a department. Another action that we find missing is the option to

    • Explication sur comment mettre en place des règles d'affichage ou "layout Rules"

      J'ai passé plus d'une heure hier avec le support et je n'ai rien compris !! Je suis lecteur assidu des guides (je "RTFM") qui ne sont absolument pas orienté "client" chez Zoho, et je tiens à le rappeler ici . Dans la documentation on m'indique un cas

    • Introducing Hiring Pipeline for Vendor Portal

      Keeping vendors informed about candidate progress is often challenging, leading to communication gaps and repeated follow-ups. To address this issue, we've released an update to the Vendor Portal feature that lets you choose to display candidates' hiring

    • Unlocking New Horizons: A Year in Review

      As we bid farewell to 2024, let's celebrate and revisit the key highlights of the year. From adding a new edition to cross-platform enhancements, here’s a roundup of all the feature updates designed to simplify accounting, optimize financial management,

    • Free user licenses across all Portal user types

      Greetings everyone, We're here with some exciting and extensive changes to the availability of free user licenses in CRM Portals. This update provides users with access to all Portal user types for free to help them diversify their user licenses and explore

    • Address Grabber function for Zoho

      I converted from ACT to Zoho. With ACT, I used an add-on called AddressGrabber to scrape the contact information from leads that I buy and contact information contained on emails and websites and directly add it as a new lead or contact. Does anyone know

    • Integrate zoom with zoho bookings please; or add optional times in zoho meetings

      Just like events - these online meetings like zoom need to be integrated with zoho bookings, and there needs to be option for customer to book a time slot. It should not be dictated by CRM user.

    • Add Owner to deluge-created module record note

      Is it possible to include the "owner" aka "creator", of a Note when creating it via delulge? This sets "superadmin" as the Note creator. I need to override it. notemap = Map(); notemap.put("Parent_Id",program_contact_id); notemap.put("Note_Content",program_contact_data.get('Note'));

    • Next Page