Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard

Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard



Howdy Tech Wizards! Welcome to a fresh week of kaizen.

This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of  Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget. 

Overview

For the use case mentioned above, 
  • Build and deploy a web scraping service on a cloud. 
  • Create a Zoho CRM variable to store the response from this web scraping service.
  • Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
  • Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.

1. Create a Web Scraping Service 

Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
  • Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
  • Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post. 

Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function. 

Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.

2. Create a Zoho CRM Variable

Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.

Step 2: Click on the Create New Variable button and provide the necessary details.



Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.

For a step-by-step guide on creating a variable, check here. 

3. Write a Custom Schedule Function 

Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.

Step 2: Click on the Create New Schedule button and provide a name to the schedule.



Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.



Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements. 

First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API. 

You can also copy and paste the code from here. 

response = invokeurl
[
url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/"
type :GET
];
info response;
scrape_response = response.toString();
variable = Map();
variable.put("id","5545974000007587001");
variable.put("value",scrape_response);
variables_list = List();
variables_list.add(variable);
param = Map();
param.put("variables",variables_list);
response = invokeurl
[
url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001"
type :PUT
parameters:param.toString()
connection:"crm_oauth_connection"
];
info response;

For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.

Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.



For more information about Schedules, check this help page. 

Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable. 

4. Code your Widget

Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.

Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.

Copy and paste the following code in your widget.html file. 

<html>
<head>
</head>
<body>
<div id="blogs-table-container"></div>
<script type="text/javascript">
ZOHO.embeddedApp.on("PageLoad", function () {
var conn_name = "test_api_connection";
var req_data = {
"method": "GET",
"param_type": 1
};
ZOHO.CRM.CONNECTION.invoke(conn_name, req_data)
.then(function (data) {
console.log(data)
const div = document.getElementById("blogs-table-container");
div.innerHTML = data.details.statusMessage.variables[0].value;
})
})
ZOHO.embeddedApp.init();
</script>
</body>
</html>

5. Upload the Widget and Check Your Results

Step 1: Execute the zet validate and zet pack commands to validate and pack the widget. 

Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button. 

Fill in the necessary details and upload the package. 



Step 3: Let us now check the dashboard widget from the Zoho CRM Home page. 



We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites. 

Hope this post was insightful and help your interest. 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!

Stay tuned until we circle back to you next Friday!

Cheers!

-------------------------------------------------------------------------------------------------------------------

Recommended Reads

-------------------------------------------------------------------------------------------------------------------
Alert
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!  


    Access your files securely from anywhere

          Zoho Developer Community

              Bootcamp
              Community
              Pricing
              Webinars
              What's New
              Integrations
              Contact Support
              Contact Sales

                Zoho Survey Resources
                  Features
                  Survey Templates
                  Integrations
                  Community
                  iOS Mobile Surveys
                  Pricing
                  Android Mobile Surveys
                  Insurvey Blogs
                  Insight Blogs
                  Tips & Tricks





                                Latest Feature Updates


                                    Zoho Desk Resources

                                    • Desk Community Learning Series


                                    • Digest


                                    • Functions


                                    • Meetups


                                    • Kbase


                                    • Resources


                                    • Glossary


                                    • Desk Marketplace


                                    • MVP Corner


                                    • Word of the Day




                                        Zoho Marketing Automation
                                          Zoho SalesIQ Resources


                                                Manage your brands on social media
                                                Access Zoho Social

                                                  Zoho Pagesense Resources


                                                      Zoho TeamInbox Resources

                                                        Zoho DataPrep Resources



                                                          Zoho CRM Plus Resources

                                                            Zoho Books Resources


                                                              Zoho Subscriptions Resources

                                                                Zoho Projects Resources


                                                                  Zoho Sprints Resources


                                                                    Qntrl Resources


                                                                      Zoho Creator Resources



                                                                          Zoho Campaigns Resources


                                                                            Zoho CRM Resources

                                                                            • CRM Community Learning Series

                                                                              CRM Community Learning Series


                                                                            • Kaizen

                                                                              Kaizen

                                                                            • Functions

                                                                              Functions

                                                                            • Meetups

                                                                              Meetups

                                                                            • Kbase

                                                                              Kbase

                                                                            • Resources

                                                                              Resources

                                                                            • Digest

                                                                              Digest

                                                                            • CRM Marketplace

                                                                              CRM Marketplace

                                                                            • MVP Corner

                                                                              MVP Corner





                                                                                Design. Discuss. Deliver.

                                                                                Create visually engaging stories with Zoho Show.

                                                                                Get Started Now


                                                                                  Zoho Show Resources


                                                                                    Zoho Writer Writer

                                                                                    Get Started. Write Away!

                                                                                    Writer is a powerful online word processor, designed for collaborative work.

                                                                                      Zoho CRM コンテンツ






                                                                                        Nederlandse Hulpbronnen


                                                                                            ご検討中の方





                                                                                                  • Recent Topics

                                                                                                  • Separate One-Time Mentions from Ongoing Ticket Following in Zoho Desk

                                                                                                    Dear Zoho Desk Support Team, We are writing to request an enhancement to the mention functionality within Zoho Desk. Currently, when an agent is mentioned in a ticket, they receive notifications for all subsequent activity on that ticket, similar to being

                                                                                                  • Mass update Accounts in Zoho Desk

                                                                                                    We used the integration feature to bring in all of our accounts from Zoho CRM. While all of the names and links are in place, on the Zoho Desk side the email, phone, website, and address fields are blank. By going into each account I can select "Update

                                                                                                  • Vivaldi will not open the Zoho Mail app when I click a email link in the browser

                                                                                                    I'm running Vivaldi on Arch with the Plasma desktop and have under default applications Zoho Mail set as my default email client. Whenever I click a email link in Vivaldi I get the below when it should be launching Zoho. Any ideas on rectifying this issue?

                                                                                                  • Digital publisher

                                                                                                    Kinetic Digital Publisher provides comprehensive digital publishing services, including eBook creation, formatting, cover design, and marketing. We help authors publish and promote their work across online platforms with ease.

                                                                                                  • Time Zone Correction with Daylight Savings Time

                                                                                                    Hey, I'm writing a series of reports where the source data is synced from an external source which stores the date/time information in GMT, but I want the reports to be in local time. Now, I understand there is the CONVERT_TZ function which allows me

                                                                                                  • Load form in iframe without header

                                                                                                    I am trying to load a form into an iframe without the header, but I am not having any luck. I am using openUrl() to load the iframe with the form URL and zc_Header set to false, e.g. #Form:Add_Case?zc_Header=false but it is still loading the header. Any

                                                                                                  • Filter embedded report

                                                                                                    How to filter embedded report in a page, below code is not working. dateField => startDate & dateField=< endDate The report should print on page containing records from startDate to endDate. params='zc_Header=true&amp;Service_Date__gte=<%=startDate%>&amp;Service_Date__lte=<%=endDate%>'

                                                                                                  • Creator Simplified #5: Set file upload restrictions in Zoho Creator

                                                                                                    Hey Creators, Welcome to the next post in the Creator Simplified series. Today, we’ll explore how to implement file upload restrictions to limit user submissions to specific file types. By implementing an allowed list for file uploads, you can optimize

                                                                                                  • Field Type: Address, Change District/State to Dropdown with picklist??

                                                                                                    Using the Address Field type, is there a way to make the District/State field be a dropdown with a picklist so the users can select rather than type the state name every time? I know this can be done if I use a drowdown field for the State (or entire address information), but that isn't as tidy as using the address field type. I apologize if this is a duplicate. I posted this question the other day, or so I though. Can't find that post.

                                                                                                  • cutomized fields are not showing up in zoho creator from zoho crm

                                                                                                    We have customized fields in zoho crm under accounts module like "Last production upgrade" --> This field is a date. When created a solution in zoho creator i couldn't find any of the customized fields. Would you please help me on this matter? Thanks

                                                                                                  • How to Add Bulk Data in Zoho Creator Forms Using Deluge Without Exceeding Execution Time Limit

                                                                                                    I have a database form with a column named 'Product Name' containing 8000 values in a Zoho Creator form. In another form named 'Returns Data,' I have a column with the same name, 'Product Name.' How do I add these 8000 values to the 'Returns Data' form

                                                                                                  • Open New Free Zoho Account

                                                                                                    Hi Team, Do you guys offer a free email hosting? I do have a domain already. If yes, what is the process to open the new account? Thanks,

                                                                                                  • 554 5.7.1 : Recipient address rejected: user info@intimspace.de does not exist

                                                                                                    554 5.7.1 : Адрес получателя отклонен: пользователь info@intimspace.de не существует I can't send an email to Google at info@intimspace.de. An error comes. I entered everything correctly in DNS https://zohomail.tools/#domainDetails/intimspace.de/ALL

                                                                                                  • Emails going back unread

                                                                                                    Hi all, When in Zoho mail - when I recieve a new mail it puts back all emails read that day back to unread - I then have to go back through and open all emails I have already read! Gets very annoying... Any idea on the bug fix?

                                                                                                  • Been getting this error, every now and then "Get count limit exceeded, please try again after 3 mins"

                                                                                                    it is really annoying.

                                                                                                  • Constraints on Tasks

                                                                                                    We have a use case where we have certain fixed date tasks and need to schedule predecessor tasks around these. Predecessor tasks need to be completed with a lag before the fixed date. We should be able to schedule the start and end date for predecessor

                                                                                                  • Moving Project Dependencies Not Moving Precedessors

                                                                                                    Most of the time we want to base our start dates around an event that is in the middle of the project template. If I set a bar up at the date we want it, it doesn't move the predecessors up. Is there a way to change this? eg. there is no point starting

                                                                                                  • Can you set task due dates to be "x" days before the milestone?

                                                                                                    We have a milestone set as the date of our first event. All of the tasks need to happen in increments prior to the milestone event. Is there a way to configure this without having to set up each task due date? Thanks!

                                                                                                  • Default ticket template in helpcenter

                                                                                                    Hello, I have a web form and a ticket template created. How can I make that my default ticket template? If an user clicks New ticket or create a ticket, I want that template to be the default one. Thank you for the time and info.

                                                                                                  • Expanded data-capturing capabilities with enhanced tabular sections

                                                                                                    We are thrilled to announce an update to Zoho Recruit that brings even more flexibility and customization to your recruiting process. With the addition of 10 new field types to the tabular sections, you now have the power to enhance your tabular sections

                                                                                                  • Integration of Business Hours in Email Templates

                                                                                                    Dear Zoho Desk Team, We would like to propose a feature enhancement to Zoho Desk that would greatly improve the utility of the Business Hours settings and streamline communication with our clients. Feature Request: Integration of Business Hours in Email

                                                                                                  • Add Owner to deluge-created module record note

                                                                                                    Is it possible to include the "owner" aka "creator", of a Note when creating it via delulge? This sets "superadmin" as the Note creator. I need to override it. notemap = Map(); notemap.put("Parent_Id",program_contact_id); notemap.put("Note_Content",program_contact_data.get('Note'));

                                                                                                  • Blueprint - Field Validation Criteria (During)

                                                                                                    When setting validation criteria elsewhere in Zoho, or even workflow criteria etc., there are Is Empty and Isn't Empty options.  Within the Field Validation Criteria within Blueprint, those options aren't available.  Is there a particular reason for this? 

                                                                                                  • Delete Field that is used in a Zoho Flow connection

                                                                                                    I'm trying to delete a Field used in a Webhook created by Zoho Flow with CRM Connection and i get the following alert: When going to the alert i get to the following issue, can't edit it since its been deployed by a pluggin But yes i have here the prompted

                                                                                                  • Use image on img HTML tag

                                                                                                    Hi how could I do to use my image saved in Workdrive to use it in an HTML img tag ? I need to display it on my website without having to use iframes. Regards,

                                                                                                  • ZOHO Compain emails going to spam after authentication is successful

                                                                                                    Hello, I am frustrated right now. I have recently setup the zoho email compaign, The auto responder email went to receipient spam folder. then, I researched a lot and completed authentication (SPF, DKIM) in email deliverability, email relay in zoho crm.

                                                                                                  • Security Policies

                                                                                                    To protect against cyber threats and attacks, organizations need to set up security policies for their employees' accounts. Security policies are rules and regulations for every individual or group using the organization's assets and resources. Enabling

                                                                                                  • Zoho CRM functions editor is not in the programming language deluge

                                                                                                    I am trying to write a function for a button. I helped someone before in deluge and I'm using this new editor I'm not familiar with - I guess it is new. Why is the default code statically typed? The editor will not let me create a variable without a type.

                                                                                                  • "Age in Days" calculation in Advanced Analytics

                                                                                                    Hi Can someone advise how this is calculated? I am getting values on this report which I cannot understand. Thank you

                                                                                                  • Automatically set quotes to "lost" if deal is set to lost

                                                                                                    Hi, Is there a way to automate that if a deal (opportunity) is lost the related quotes are also set to lost? Thanks!

                                                                                                  • Subdomain

                                                                                                    How can i make subdomain in my zoho website

                                                                                                  • A/R Aging Details shows wrong aging days

                                                                                                    In the A/R Summary Report all of the invoices are in the right aging buckets. When I run the A/R Aging Details report I get aged dates of +300 days when they should be in the 0-90 day range.

                                                                                                  • Global Choice List share ownership

                                                                                                    I have created several forms that use one or more Global Choice Lists. These lists have been published to Org. I would like to allow one or more admins to edit the choices in these lists. Any help appreciated. Geoff

                                                                                                  • Domain Transfer

                                                                                                    I have a Godaddy domain, how i can transfer it to Zoha? and how i can move my website to Zoho server? With my best wishes.

                                                                                                  • Project Templates & Reminders

                                                                                                    I am getting projects all set up to work for our company and am running into a problem that I'm hoping is easily fixable. I have created a project template and within that project, there are reminders set on certain tasks. When I create a project from

                                                                                                  • Kaizen #126 - Circuits in Zoho CRM - Part 1

                                                                                                    Hello everyone! Welcome back to another week of Kaizen! Today, we will discuss an exciting topic—Circuits in Zoho CRM. For starters, we will discuss what Circuits are, how beneficial they are for businesses, different views of a Circuit, and the different

                                                                                                  • Create customized SLAs for your customer base with support plans

                                                                                                    Managing customer expectations, prioritizing critical issues, and resolving customer inquiries on time is quite a juggle. Without a clear timelines or defined priorities, a support team may struggle with delays in response, SLA violations, and pending

                                                                                                  • Zoho Flow or Schedules

                                                                                                    I have a process where we text our leads 7 times over a 14 day with different content for each text. I created one flow in Zoho Flow to do this, but wondering if there is a more efficient way to accomplish this via Schedules. It goes on for 6 more times

                                                                                                  • Free webinar: Zoho Sign 2024 wrap-up - Everything that is new and has changed

                                                                                                    Hello, Are you looking up to catch up on all the updates made to Zoho Sign in 2024? Or are you still figuring out how you can use Zoho Sign better to get business paperwork done more efficiently? If so, we invite you to join us this Thursday, December

                                                                                                  • How to Customize Task Creation to Send a Custom Alert Using JavaScript in Zoho CRM?

                                                                                                    Hello Zoho CRM Community, I’m looking to customize Zoho CRM to send a custom alert whenever a task is created. I understand that Zoho CRM supports client scripts using JavaScript, and I would like to leverage this feature to implement the alert functionality.

                                                                                                  • Next Page