Kaizen #140 - Integrating Blog feed scraping service into Zoho CRM Dashboard

Kaizen #140 - Integrating Blog feed scraping service into Zoho CRM Dashboard



Howdy Tech Wizards! Welcome to a fresh week of kaizen.

This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of  Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget. 

Overview

For the use case mentioned above, 
  • Build and deploy a web scraping service on a cloud. 
  • Create a Zoho CRM variable to store the response from this web scraping service.
  • Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
  • Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.

1. Create a Web Scraping Service 

Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
  • Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
  • Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post. 

Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function. 

Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.

2. Create a Zoho CRM Variable

Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.

Step 2: Click on the Create New Variable button and provide the necessary details.



Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.

For a step-by-step guide on creating a variable, check here. 

3. Write a Custom Schedule Function 

Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.

Step 2: Click on the Create New Schedule button and provide a name to the schedule.



Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.



Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements. 

First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API. 

You can also copy and paste the code from here. 

response = invokeurl
[
url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/"
type :GET
];
info response;
scrape_response = response.toString();
variable = Map();
variable.put("id","5545974000007587001");
variable.put("value",scrape_response);
variables_list = List();
variables_list.add(variable);
param = Map();
param.put("variables",variables_list);
response = invokeurl
[
url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001"
type :PUT
parameters:param.toString()
connection:"crm_oauth_connection"
];
info response;

For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.

Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.



For more information about Schedules, check this help page. 

Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable. 

4. Code your Widget

Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.

Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.

Copy and paste the following code in your widget.html file. 

<html>
<head>
</head>
<body>
<div id="blogs-table-container"></div>
<script type="text/javascript">
ZOHO.embeddedApp.on("PageLoad", function () {
var conn_name = "test_api_connection";
var req_data = {
"method": "GET",
"param_type": 1
};
ZOHO.CRM.CONNECTION.invoke(conn_name, req_data)
.then(function (data) {
console.log(data)
const div = document.getElementById("blogs-table-container");
div.innerHTML = data.details.statusMessage.variables[0].value;
})
})
ZOHO.embeddedApp.init();
</script>
</body>
</html>

5. Upload the Widget and Check Your Results

Step 1: Execute the zet validate and zet pack commands to validate and pack the widget. 

Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button. 

Fill in the necessary details and upload the package. 



Step 3: Let us now check the dashboard widget from the Zoho CRM Home page. 



We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites. 

Hope this post was insightful and help your interest. 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!

Stay tuned until we circle back to you next Friday!

Cheers!

-------------------------------------------------------------------------------------------------------------------

Recommended Reads

-------------------------------------------------------------------------------------------------------------------
Alert
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!  


    • Sticky Posts

    • Kaizen #197: Frequently Asked Questions on GraphQL APIs

      🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Kaizen #198: Using Client Script for Custom Validation in Blueprint

      Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Celebrating 200 posts of Kaizen! Share your ideas for the milestone post

      Hello Developers, We launched the Kaizen series in 2019 to share helpful content to support your Zoho CRM development journey. Staying true to its spirit—Kaizen Series: Continuous Improvement for Developer Experience—we've shared everything from FAQs

    • Kaizen #193: Creating different fields in Zoho CRM through API

      🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Client Script | Update - Introducing Commands in Client Script!

      Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! Commands

    • Recent Topics

    • CRM : Function to add user name to text field

      I have a lookup field in a module that is linked to the CRM users so we can assign a Project Lead to the customer. Sadly Zoho Marketing Automation doesn't sync Lookup fields so I need to extract information from the lookup to text fields: Lookup field

    • Upload API

      I'm trying to use the Upload API to upload some images and attach them to comments (https://desk.zoho.com/DeskAPIDocument#Uploads#Uploads_Uploadfile) - however I can only ever get a 401 or bad request back. I'm using an OAuth token with the Desk.tickets.ALL

    • Losing description after merging tickets

      Hello, We merge tickets when they are about the same topic from the same client. It happens sometimes. We recently noticed that after the merger only the description from the master ticket is left in a thread. And the slave-ticket description is erased.

    • Option to Empty Entire Mailbox or Folder in Zoho Mail

      Hello Zoho Mail Team, How are you? We would like to request an enhancement to Zoho Mail that would allow administrators and users to quickly clear out entire folders or mailboxes, including shared mailboxes. Current Limitation: At present, Zoho Mail only

    • update linked contacts when update happens in account

      Hi, I have a custom field called Licence in the Accounts module. When someone buys a licence, I’d like to update a custom field in the related Contacts. How can I achieve this? I noticed that workflows triggered on Accounts only allow me to update fields

    • Problem Management Module

      I am looking for a Problem Management module within Zoho Desk. I saw in some training videos that this is available, and some even provided an annual price for it. I want an official confirmation on whether this is indeed available. This is not a particularly

    • Deluge sendmail in Zoho Desk schedule can't send email from a verified email address

      I am trying to add a scheduled action with ZDesk using a Deluge function that sends a weekly email to specific ticket client contacts I've already verified the email address for use in ZDesk, but sendmail won't allow it in its "from:" clause. I've attached

    • Unable to explore desk.zoho.com

      Greetings, I have an account with zoho which already has a survey subscription. I would like to explore desk.zoho.com, but when I visit it while logged in (https://desk.zoho.com/agent?action=CreatePortal) I just get a blank page. I have tried different

    • Offline support for mobile app

      Accessing your files and folders from your mobile devices is now quicker and simpler, thanks to the power of offline support. Whether on an Android or iOS device, you can use the Offline function to save files and folders, so you can review them even

    • Zoho Desk KB article embedded on another site.

      We embed KB articles from Zoho Desk on another site (our application). When opening the article in a new tab, there is no issue, but if we choose lightbox, we are getting an error "To protect your security, help.ourdomain.com will not allow Firefox to

    • Transitioning to API Credits in Zoho Desk

      At Zoho Desk, we’re always looking for ways to help keep your business operations running smoothly. This includes empowering teams that rely on APIs for essential integrations, functions and extensions. We’ve reimagined how API usage is measured to give

    • List of packaged components and if they are upgradable

      Hello, In reference to the article Components and Packaging in Zoho Vertical Studio, can you provide an overview of what these are. Can you also please provide a list of of components that are considered Packaged and also whether they are Upgradable?

    • Does Attari Messaging app have Bot option and APIB

      Hi, Does Attari also have Bot and API as we use in WhatsApp??

    • how to use validation rules in subform

      Is it possible to use validation rules for subforms? I tried the following code: entityMap = crmAPIRequest.toMap().get("record"); sum = 0; direct_billing = entityMap.get("direct_billing_details"); response = Map(); for each i in direct_billing { if(i.get("type")

    • How to add application logo

      I'm creating an application which i do not want it to show my organization logo so i have changed the setting but i cannot find where to upload/select the logo i wish to use for my application. I have seen something online about using Deluge and writing

    • Introducing Keyboard Shortcuts for Zoho CRM

      Dear Customers, We're happy to introduce keyboard shortcuts for Zoho CRM features! Until now, you might have been navigating to modules manually using the mouse, and at times, it could be tedious, especially when you had to search for specific modules

    • Empowered Custom Views: Cross-Module Criteria Now Supported in Zoho CRM

      Hello everyone, We’re excited to introduce cross-module criteria support in custom views! Custom views provide personalized perspectives on your data and that you can save for future use. You can share these views with all users or specific individuals

    • Send Automated WhatsApp Messages and Leverage the Improved WhatsApp Templates

      Greetings, I hope all of you are doing well. We're excited to announce a major upgrade to Bigin's WhatsApp integration that brings more flexibility, interactivity, and automation to your customer messaging. WhatsApp message automation You can now use

    • Verifying Zoho Mail Functionality After Switching DNS from Cloudflare to Hosting Provider

      I initially configured my domain's (https://roblaxmod.com/) email with Zoho Mail while using Cloudflare to manage my DNS records (MX, SPF, etc.). All services were working correctly. Recently, I have removed my site from Cloudflare and switched my domain's

    • Zoho Analytics Regex Support

      When can we expect full regex support in Zoho Analytics SQL such as REGEXP_REPLACE? Sometimes I need to clean the data and using regex functions is the easiest way to achieve this.

    • Change of Blog Author

      Hi, I am creating the blog post on behalf of my colleague. When I publish the post, it is showing my name as author of the post which is not intended and needs to be changed to my colleague's name. How can I change the name of the author in the blogs?? Thanks, Ramanan

    • Show Attachments in the customer portal

      Hi, is it possible to show the Attachments list in the portal for the particular module? Bests.

    • Does Zoho Docs have a Line Number function ?

      Hi, when collaborating with coding tasks, I need an online real time share document that shows line numbers. Does Zoho's docs offer this feature ? If yes, how can I show them ? Regards, Frank

    • Please make it easier to Pause syncing

      right now it takes 3 clicks to get there. sounds silly, but can you make it just 2 clicks to get it done instead? thats how dropbox does it, 2 clicks to pause instead of 3.

    • Feature Request - Insert URL Links in Folders

      I would love to see the ability to create simple URL links with titles in WorkDrive. or perhaps a WorkDrive extension to allow it. Example use case: A team is working on a project and there is project folder in WordDrive. The team uses LucidChart to create

    • Organization Emails in Email History

      How can I make received Org Emails to show up here?

    • Converting Sales Order to Invoice via API; Problem with decimal places tax

      We are having problems converting a Sales Order to an Invoice via API Call. The cause of the issue is, that the Tax value in a Sales Order is sometimes calculated with up to 16 decimal places (e.g. 0.8730000000000001). The max decimal places allowed in

    • how to differentiate if whatsapp comes from certain landing page?

      I create a Zobot in SalesIQ to create a Whatsapp bot to capture the lead. I have 2 landing pages, one is SEO optimized and the other want is optimized for leads comes from Google Ads. I want to know from which landing page this lead came through WhatsApp

    • How to sync from Zoho Projects into an existing Sprint in Zoho Sprints?

      Hi I have managed to integrate Zoho Projects with Zoho Sprints and I can see that the integration works as a project was created in Zoho Sprints. But, what I would like to do is to sync into an existing Zoho Sprints project. Is there a way to make that

    • How to record company set up fees?

      Hi all, We are starting out our company in Australia and would appreciate any help with setting up Books accounts. We paid an accountant to do company registration, TFN, company constitution, etc. I heard these all can be recorded as Incorporation Costs, which is an intangible asset account, and amortised over 5 years. Is this the correct way to do it under the current Australian tax regulations? How and when exactly should I record the initial entry and each year's amortasation in Books? Generally

    • How to create a drop down menu in Zoho Sheets

      I am trying to find out, how do I create a drop down option in Zoho sheet. I tried Data--> Data Validation --> Criteria --> Text  --> Contains. But that is not working, is there any other way to do it.  Thanks in Advance.

    • Show Payment terms in Estimates

      Hi,  we are trying to set up that estimates automatically relates payment terms for the payment terms we introduced on Edit contact (Field Payment terms).  How can it be done? Our aim is to avoid problems on payment terms introduced and do not need to introduce it manually on each client (for the moment we are introducing this information on Terms and Conditions.  Kind Regards, 

    • Rich-text fields in Zoho CRM

      Hello everyone, We're thrilled to announce an important enhancement that will significantly enhance the readability and formatting capabilities of your information: rich text options for multi-line fields. With this update, you can now enjoy a more versatile

    • How can I calculate the physical stock available for sale?

      Hey Zoho Team,  I've tried to calculate the physical stock on hand in various ways - but always receive a mismatch between what's displayed in Zoho Inventory & analytics.  Can you please let me know how the physical stock available for sale is calculated?

    • When dispatched to crew, assigning lead missing

      Hello, For the past two or three weeks, whenever an officer assigns Service Appointment to a team, the lead person is missing from the assigned service list. Therefore, we have to reschedule the SA and then the lead person becomes visible in the assigned

    • Create Lead Button in Zoho CRM Dashboard

      Right now to create Leads in the CRM our team is going into the Lead module, selecting the "Create Lead" button, then building out the lead. Is there anyway to add the "Create Lead" button or some sort of short cut to the Zoho CRM Dashboard to cut out

    • open word file in zoho writer desktop version

      "How can I open a Microsoft Word (.doc or .docx) file in Zoho Writer if I only have the file saved on my computer and Zoho Writer doesn't appear as an option when I try 'Open with'? Is there a way to directly open the .doc file in Zoho Writer?"

    • Generate leads from instagram

      hello i have question. If connect instagram using zoho social, it is possible to get lead from instagram? example if someone send me direct message or comment on my post and then they generate to lead

    • I want to transfer the project created in this account to another account

      Dear Sir I want to transfer the project created in one account to another account

    • Inactive User Auto Response

      We use Zoho One, and we have a couple employees that are no longer with us, but people are still attempting to email them. I'd like an autoresponder to let them no the person is no longer here, and how they can reach us going forward. I saw a similar

    • Next Page