Kaizen #140 - Integrating Blog feed scraping service into Zoho CRM Dashboard

Kaizen #140 - Integrating Blog feed scraping service into Zoho CRM Dashboard



Howdy Tech Wizards! Welcome to a fresh week of kaizen.

This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of  Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget. 

Overview

For the use case mentioned above, 
  • Build and deploy a web scraping service on a cloud. 
  • Create a Zoho CRM variable to store the response from this web scraping service.
  • Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
  • Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.

1. Create a Web Scraping Service 

Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
  • Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
  • Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post. 

Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function. 

Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.

2. Create a Zoho CRM Variable

Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.

Step 2: Click on the Create New Variable button and provide the necessary details.



Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.

For a step-by-step guide on creating a variable, check here. 

3. Write a Custom Schedule Function 

Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.

Step 2: Click on the Create New Schedule button and provide a name to the schedule.



Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.



Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements. 

First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API. 

You can also copy and paste the code from here. 

response = invokeurl
[
url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/"
type :GET
];
info response;
scrape_response = response.toString();
variable = Map();
variable.put("id","5545974000007587001");
variable.put("value",scrape_response);
variables_list = List();
variables_list.add(variable);
param = Map();
param.put("variables",variables_list);
response = invokeurl
[
url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001"
type :PUT
parameters:param.toString()
connection:"crm_oauth_connection"
];
info response;

For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.

Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.



For more information about Schedules, check this help page. 

Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable. 

4. Code your Widget

Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.

Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.

Copy and paste the following code in your widget.html file. 

<html>
<head>
</head>
<body>
<div id="blogs-table-container"></div>
<script type="text/javascript">
ZOHO.embeddedApp.on("PageLoad", function () {
var conn_name = "test_api_connection";
var req_data = {
"method": "GET",
"param_type": 1
};
ZOHO.CRM.CONNECTION.invoke(conn_name, req_data)
.then(function (data) {
console.log(data)
const div = document.getElementById("blogs-table-container");
div.innerHTML = data.details.statusMessage.variables[0].value;
})
})
ZOHO.embeddedApp.init();
</script>
</body>
</html>

5. Upload the Widget and Check Your Results

Step 1: Execute the zet validate and zet pack commands to validate and pack the widget. 

Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button. 

Fill in the necessary details and upload the package. 



Step 3: Let us now check the dashboard widget from the Zoho CRM Home page. 



We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites. 

Hope this post was insightful and help your interest. 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!

Stay tuned until we circle back to you next Friday!

Cheers!

-------------------------------------------------------------------------------------------------------------------

Recommended Reads

-------------------------------------------------------------------------------------------------------------------
Alert
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!  


    • Sticky Posts

    • Kaizen #197: Frequently Asked Questions on GraphQL APIs

      🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Kaizen #198: Using Client Script for Custom Validation in Blueprint

      Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Celebrating 200 posts of Kaizen! Share your ideas for the milestone post

      Hello Developers, We launched the Kaizen series in 2019 to share helpful content to support your Zoho CRM development journey. Staying true to its spirit—Kaizen Series: Continuous Improvement for Developer Experience—we've shared everything from FAQs

    • Kaizen #193: Creating different fields in Zoho CRM through API

      🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Client Script | Update - Introducing Commands in Client Script!

      Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! Commands

    • Recent Topics

    • Create Lead Button in Zoho CRM Dashboard

      Right now to create Leads in the CRM our team is going into the Lead module, selecting the "Create Lead" button, then building out the lead. Is there anyway to add the "Create Lead" button or some sort of short cut to the Zoho CRM Dashboard to cut out

    • open word file in zoho writer desktop version

      "How can I open a Microsoft Word (.doc or .docx) file in Zoho Writer if I only have the file saved on my computer and Zoho Writer doesn't appear as an option when I try 'Open with'? Is there a way to directly open the .doc file in Zoho Writer?"

    • Generate leads from instagram

      hello i have question. If connect instagram using zoho social, it is possible to get lead from instagram? example if someone send me direct message or comment on my post and then they generate to lead

    • I want to transfer the project created in this account to another account

      Dear Sir I want to transfer the project created in one account to another account

    • Inactive User Auto Response

      We use Zoho One, and we have a couple employees that are no longer with us, but people are still attempting to email them. I'd like an autoresponder to let them no the person is no longer here, and how they can reach us going forward. I saw a similar

    • Weekly Tips : Customize your Compose for a smoother workflow

      You are someone who sends a lot of emails, but half the sections in the composer just get in your way — like fields you never use or sections that clutter the space. You find yourself always hunting for the same few formatting tools, and the layout just

    • Zoho Slowness - Workarounds

      Hi all, We've been having intermittent slowness and Zoho just asks for same stuff each time but never fix it. It usually just goes away on it's own after a couple weeks. Given that speed is a very important thing for companies to be able to keep up with

    • Custom Bulk Select Button

      Zoho CRM offers the ability to select multiple records and invoke a Custom Button This functionality is missing from Recruit Currently we can only add buttons in the detail page and list But we cannot select Multiple Records and invoke a function with

    • How to create a Zoho CRM report with 2 child modules

      Hi all, Is it possible to create a Zoho CRM report or chart with 2 child modules? After I add the first child module, the + button only adds another parent module. It won't let me add multiple child modules at once. We don't have Zoho Analytics and would

    • Zoho CRM still doesn't let you manage timezones (yearly reminder)

      This is something I have asked repeatedly. I'll ask once again. Suppose that you work in France. Next month you have a trip to Guatemala. You call a contact there, close a meeting, record that meeting in CRM. On the phone, your contact said: "meet me

    • Power of Automation :: Smart Ticket Management Between Zoho Desk and Projects

      Hello Everyone, A custom function is a software code that can be used to automate a process and this allows you to automate a notification, call a webhook, or perform logic immediately after a workflow rule is triggered. This feature helps to automate

    • First day of trying FSM in the field.

      What we found. 1. with out a network connection we were unable to start a service call? 2. if you go to an appointment and then want to add an asset it does not seem possible. 3. disappointed not to be able to actually take a payment from within the app

    • BUG - Google Business Buttons - Add a button to GBP Post

      I am experiencing an issue with the "Add a button" feature when creating posts for my Google Business Profile (GBP) through Zoho Social. When I schedule or publish a GBP post and include a call-to-action button with a specific URL, the post itself publishes

    • Rich text Merge field - Not using font specified in HTML

      I have a rich text merge field in a writer template which is creating a table. I have chosen to use this method instead of a repeat region because I need to specify specific cell background colours which change every time the document is created. The

    • Support for Custom Fonts in Zoho Recruit Career Site and Candidate Portal

      Dear Zoho Recruit Team, I hope you're doing well. We would like to request the ability to use custom fonts in the Zoho Recruit Career Site and Candidate Portal. Currently only the default fonts (Roboto, Lato, and Montserrat) are available. While these

    • CC and/or BCC users in email templates

      I would like the ability to automatically assign a CC and BCC "User (company employee)" into email templates. Specifically, I would like to be able to add the "User who owns the client" as a CC automatically on any interview scheduled or candidate submitted

    • Trying to export a report to Excel via a deluge script

      I have this code from other posts but it gives me an error of improper statement, due to missing ; at end of line or incomplete expression. Tried lots of variations to no avail. openUrl(https://creatorapp.zoho.com/<username>/<app name>/XLSX/#Report:<reportname>,"same

    • Zoho Reports Duplicating Entries

      I have a custom costing tab with a table where we entre invoices. These are under a Heading (PO Subject) and notes added in the form with different line items. In the reports, I have organised the report to group per PO Subject, with the total of the

    • Need help to create a attach file api

      https://www.zoho.com/crm/developer/docs/api/v8/upload-attachment.html Please help me to create it... It's not working for while. Do you have some example?

    • Export view via deluge.

      Hi, Is it possible to export a view (as a spreadsheet) via deluge? I would like to be able to export a view as a spreadsheet when a user clicks a button. Thanks     

    • how to add subform over sigma in the CRM

      my new module don't have any subform available any way to add this from sigma or from the crm

    • Outdated state in mexico

      Hello Zoho team, the drop down to add the state for customers, when they introduce their state in mexico has a city named “Distrito Federal” that name changed many years ago to “ciudad de mexico”. could you please update this so my clients can find the

    • Support new line in CRM Multiline text field display in Zoho Deluge

      Hi brainstrust, We have a Zoho CRM field which is a Muti Line (Small) field. It has data in it that has a carriage return after each line: When I pull that data in via Deluge, it displays as: I'm hoping a way I can change it from: Freehand : ENABLED Chenille

    • Possible to generate/download Quote PDF using REST API?

      See title. Is there any way after a quote has been created to export to a PDF using a specified template and then download it? Seems like something that should be doable. Is this not supported in the API v2.0?

    • Creating an invoice to be paid in two installments?

      Hi there, I own a small Photographic Services business and have not been able to find a way to fit my billing system into Zoho, or any other Accounting software. The way my payments work is: 1. Customer pays 50% of total price of service to secure their

    • Bug in allowing the user to buy out of stock items

      Hi i want to allow the user to buy out of stock items, according to the commerce documentation if i disable Restrict "Out of stock" purchases it will, but it doesnt work, so i want to know if it had any relation with zoho inventory, and if theres any

    • Zoho CRM Calendar | Custom Buttons

      I'm working with my sales team to make our scheduling process easier for our team. We primary rely on Zoho CRM calendar to organize our events for our sales team. I was wondering if there is a way to add custom button in the Calendar view on events/meeting

    • Replace Lookup fields ID value with their actual name and adding inormation from subforms

      Hi everyone,  I wanted to see if someone smarter than me has managed to find any solutions to two problems we have. I will explain both below.  To start we are syncing data from Zoho CRM to Zoho Analytics and I will use the Sales Order module when giving

    • Can a Zoho Sites page be embedded into another website (outside Zoho)

      Hi All, We have a request from a client - they'd like to take one of our information pages created in Zoho Sites and embed it into their own website? I was told through an email with Zoho that this was possible >>Thank you for your patience regarding

    • Bug in allowing the user to buy out of stock items

      Hi i want to allow the user to buy out of stock items, according to the commerce documentation if i disable Restrict "Out of stock" purchases it will, but it doesnt work, so i want to know if it had any relation with zoho inventory, and if theres any

    • Transition Criteria Appearing on Blueprint Transitions

      On Monday, Sept. 8th, the Transition criteria started appearing on our Blueprints when users hover over a Transition button. See image. We contacted Zoho support because it's confusing our users (there's really no reason for them to see it), but we haven't

    • Zoho CRM Sales Targets for Individual Salespeople

      Our organistion has salespeople that are allocated to different regions and have different annual sales targets as a result. I am building an CRM analytics dashboard for the sales team, which will display a target meter for the logged in salesperson.

    • All new Address Field in Zoho CRM: maintain structured and accurate address inputs

      The address field will be available exclusively for IN DC users. We'll keep you updated on the DC-specific rollout soon. It's currently available for all new sign-ups and for existing Zoho CRM orgs which are in the Professional edition. Managing addresses

    • Transfer all Related Data to new Account Owner

      Currently when I change the account Owner I only see the option to change only the open deals But I want the new account owner to take over all the related modules and all the deal stages Is it not possible right now? Am I missing something? Do I really

    • Can i connect 2 instagram accounts to 1 brand?

      Can i connect 2 instagram accounts to 1 brand? Or Do i need to create 2 brands for that? also under what subscription package will this apply?

    • How to Calculate MTTR (Mean Time to Resolve)

      We want to calculate MTTR (Mean Time to Resolve) in our Zoho Analytics report under Tickets. Currently, we are using the following fields: Ticket ID Ticket Created Time Ticket Closed Time Ticket On Hold Time We are planning to calculate MTTR (in days)

    • How to export project tasks, including the comments

      Hi, how can I export the project tasks, whereby I can also see the comments associated to a specific task? The use-case is that often we use comments to discuss or update a task related ideas. I would like to export the tasks, where we can also see the

    • How to Install Zoho Workdrive Desktop Sync for Ubuntu?

      Hi. I am newbie to Linux / Ubuntu. I downloaded a tar.gz file from Workdrive for installing the Workdrive Desktop Sync tool. Can someone give me step by step guide on how to install this on Ubuntu? I am using Ubuntu 19.04. Regards Senthil

    • Introducing Version-3 APIs - Explore New APIs & Enhancements

      Happy to announce the release of Version 3 (V3) APIs with an easy to use interface, new APIs, and more examples to help you understand and access the APIs better. V3 APIs can be accessed through our new link, where you can explore our complete documentation,

    • Round robin

      Hi, I'm trying to set up a round robin to automatically distribute tickets between agents in my team but only those tickets that are not otherwise distributed by other workflows or direct assignments. Is that possible and if so which criteria should I

    • Next Page