Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard

Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard



Howdy Tech Wizards! Welcome to a fresh week of kaizen.

This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of  Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget. 

Overview

For the use case mentioned above, 
  • Build and deploy a web scraping service on a cloud. 
  • Create a Zoho CRM variable to store the response from this web scraping service.
  • Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
  • Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.

1. Create a Web Scraping Service 

Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
  • Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
  • Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post. 

Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function. 

Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.

2. Create a Zoho CRM Variable

Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.

Step 2: Click on the Create New Variable button and provide the necessary details.



Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.

For a step-by-step guide on creating a variable, check here. 

3. Write a Custom Schedule Function 

Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.

Step 2: Click on the Create New Schedule button and provide a name to the schedule.



Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.



Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements. 

First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API. 

You can also copy and paste the code from here. 

response = invokeurl
[
url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/"
type :GET
];
info response;
scrape_response = response.toString();
variable = Map();
variable.put("id","5545974000007587001");
variable.put("value",scrape_response);
variables_list = List();
variables_list.add(variable);
param = Map();
param.put("variables",variables_list);
response = invokeurl
[
url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001"
type :PUT
parameters:param.toString()
connection:"crm_oauth_connection"
];
info response;

For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.

Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.



For more information about Schedules, check this help page. 

Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable. 

4. Code your Widget

Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.

Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.

Copy and paste the following code in your widget.html file. 

<html>
<head>
</head>
<body>
<div id="blogs-table-container"></div>
<script type="text/javascript">
ZOHO.embeddedApp.on("PageLoad", function () {
var conn_name = "test_api_connection";
var req_data = {
"method": "GET",
"param_type": 1
};
ZOHO.CRM.CONNECTION.invoke(conn_name, req_data)
.then(function (data) {
console.log(data)
const div = document.getElementById("blogs-table-container");
div.innerHTML = data.details.statusMessage.variables[0].value;
})
})
ZOHO.embeddedApp.init();
</script>
</body>
</html>

5. Upload the Widget and Check Your Results

Step 1: Execute the zet validate and zet pack commands to validate and pack the widget. 

Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button. 

Fill in the necessary details and upload the package. 



Step 3: Let us now check the dashboard widget from the Zoho CRM Home page. 



We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites. 

Hope this post was insightful and help your interest. 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!

Stay tuned until we circle back to you next Friday!

Cheers!

-------------------------------------------------------------------------------------------------------------------

Recommended Reads

-------------------------------------------------------------------------------------------------------------------
Alert
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!  


    Access your files securely from anywhere

          Zoho Developer Community

              Bootcamp
              Community
              Pricing
              Webinars
              What's New
              Integrations
              Contact Support
              Contact Sales

                Zoho Survey Resources
                  Features
                  Survey Templates
                  Integrations
                  Community
                  iOS Mobile Surveys
                  Pricing
                  Android Mobile Surveys
                  Insurvey Blogs
                  Insight Blogs
                  Tips & Tricks





                                Latest Feature Updates


                                    Zoho Desk Resources

                                    • Desk Community Learning Series


                                    • Digest


                                    • Functions


                                    • Meetups


                                    • Kbase


                                    • Resources


                                    • Glossary


                                    • Desk Marketplace


                                    • MVP Corner


                                    • Word of the Day




                                        Zoho Marketing Automation
                                          Zoho SalesIQ Resources


                                                Manage your brands on social media
                                                Access Zoho Social

                                                  Zoho Pagesense Resources


                                                      Zoho TeamInbox Resources

                                                        Zoho DataPrep Resources



                                                          Zoho CRM Plus Resources

                                                            Zoho Books Resources


                                                              Zoho Subscriptions Resources

                                                                Zoho Projects Resources


                                                                  Zoho Sprints Resources


                                                                    Qntrl Resources


                                                                      Zoho Creator Resources



                                                                          Zoho Campaigns Resources


                                                                            Zoho CRM Resources

                                                                            • CRM Community Learning Series

                                                                              CRM Community Learning Series


                                                                            • Kaizen

                                                                              Kaizen

                                                                            • Functions

                                                                              Functions

                                                                            • Meetups

                                                                              Meetups

                                                                            • Kbase

                                                                              Kbase

                                                                            • Resources

                                                                              Resources

                                                                            • Digest

                                                                              Digest

                                                                            • CRM Marketplace

                                                                              CRM Marketplace

                                                                            • MVP Corner

                                                                              MVP Corner





                                                                                Design. Discuss. Deliver.

                                                                                Create visually engaging stories with Zoho Show.

                                                                                Get Started Now


                                                                                  Zoho Show Resources


                                                                                    Zoho Writer Writer

                                                                                    Get Started. Write Away!

                                                                                    Writer is a powerful online word processor, designed for collaborative work.

                                                                                      Zoho CRM コンテンツ






                                                                                        Nederlandse Hulpbronnen


                                                                                            ご検討中の方





                                                                                                  • Recent Topics

                                                                                                  • Power of Automation :: Automatically start / pause / stop timer on task status update.

                                                                                                    Hello Everyone, A Custom function is a user-written set of code to achieve a specific requirement. Set the required conditions needed as when to trigger using the Workflow rules (be it Tasks / Project) and associate the custom function to it. Requirement:-

                                                                                                  • Editor limitations to define screen types

                                                                                                    Guys I have noticed that even in version 2.0 of the editor (which is this new one we use) we still have a lot to improve... When I compare to some more global solutions like Wix, Zyro, Go Daddy, Squarespace and Weebly feel that we have some limitations

                                                                                                  • Dúvidas do Zoho Creator

                                                                                                    Pessoal, Estou colocando um tópico para dúvidas do Zoho Creator. Um abraço, Leandro

                                                                                                  • Tropicalize Books

                                                                                                    Books is an incredibly powerful tool that works well in many countries. But I feel that it is a product that is not yet "tropicalized" for Brazil as we speak (this would be like adapting the local reality). We have many strong competitors who do more

                                                                                                  • Automatic Sitemap Generation

                                                                                                    Guys are all right? Doesn't make sense for me to have to generate a map site and upload it... because it's not automatically generated just as it is done in WIX? where the customer doesn’t have to worry about this.

                                                                                                  • SEO improvements with ZIA

                                                                                                    Are you okay? I would like to bring an idea that would be amazing to improve the product that is the possibility of being able to improve the SEO of the pages (this of each page or each article on the blog) through ZIA so that it could create page Summary

                                                                                                  • Automatically updating field(s) of lookup module

                                                                                                    I have a lookup field, which also pulls through the Status field from the linked record. When the lookup is first done, the Status is pulled through - this works perfectly. If that Status is later updated, the lookup field does not update as well. As

                                                                                                  • Initiating a SalesIQ Zobot from a custom button on Zoho Sites

                                                                                                    I have created a Zobot set to initiate on a custom action called "Fast_Answers". On Zoho Sites, I created a code snippet button and set it to on-click run the event called "Fast_Answers". I installed the SalesIQ integration code into the Zoho Sites Page

                                                                                                  • Zoho Desk -> Zoho Analytics : Where is the field for "Layout" ?

                                                                                                    I have many different layouts on my helpdesks and I want to be able to identify the stats for each one, however I can not file the field in the raw data from the Zoho Desk datasource. I thought it might be under "Tickets" but there is nothing. There is

                                                                                                  • Set resolution mandatory field

                                                                                                    Hi,  i have 2 questions:) : - i want to set the resolution field mandatory before close the ticket. Because for now, i can close the ticket without writing how i solve it - how can i setup zoho desk to receive ticket by email(e.g. clients sent email to support@mydomain.com) and it create a ticket in zoho desk

                                                                                                  • Impuesto automatico en cotizaciones

                                                                                                    Buen dia Mi pregunta es como se puede poner alguna operacion para que las cotizaciones me salgan automaticamente con impuestos ya que uno al cargarla al final tienen que añadir el porcentaje de impuesto Saludos

                                                                                                  • Zoho Assist "Agree and Download" Button "Greyed Out" ("Light Blued" Out)

                                                                                                    Anyone else having issue where support clients are unable to click "Agree and Download" to access the client so that we can provide remote support? This is for "on demand" support via accessing the support page and entering the support key and name. This

                                                                                                  • Project Billing with the Staff Hours Method in Zoho Projects

                                                                                                    The Staff Hours Billing Method in Zoho Projects allows you to bill your clients based on the actual time spent by each team member on a project, at the rate set for each user. This is useful for projects where different skill sets are needed and service

                                                                                                  • Creator Subform to CRM Subform

                                                                                                    Hello all, Has anyone successfully written data from a Creator Subform into CRM subform? I have been able to get the rows to populate but none of the data will come through. I'll add my code and the result in CRM. Creator Subform is 'Delivery_Receiving_Hours'.

                                                                                                  • custom fields not populating from deluge script into invoice

                                                                                                    Hello, I've created some Deluge script that is meant to take a few inputted invoice custom fields and calculate a few others. I can see when I execute the function that my inputted custom fields are being passed, yet im still ending up with all "null"

                                                                                                  • tax summation function - getting error

                                                                                                    Hello, I'm trying to create a function that adds all of individual tax rates from a few jurisdictions. I'm getting an error on line 9 - Value is empty and 'get' function cannot be applied. I've checked that I have data in each of the required fields,

                                                                                                  • Backstage / Zoho Books integration

                                                                                                    Hello. We have Zoho One and have slowly started using Backstage. Loving it. Problem is, we have ZERO accounting control over what is sold through this product. When will we be able to connect it to our existing Zoho Books tenant? Thank you very much!

                                                                                                  • Getting list of calendar events over api for zoho mail calendar

                                                                                                    Hi, I am using just Zoho mail without using Zoho CRM. I wanted to get all events booked in my zoho mail calendar through an api at regular intervals. I could find such API support for Zoho CRM calendar but not for zoho mail calendar. Can you kindly let

                                                                                                  • Allocate emails to user in a shared mailbox

                                                                                                    Hi, This might be obvious, but I cannot find the answer. I have 3 shared mailboxes so any team member can see the emails. Is there a way of allocating a specific email to a user so that it is their responsibility to deal with it? Thanks in advance.

                                                                                                  • Introducing Zoho Campaigns' own gateway for SMS campaigns

                                                                                                    We are excited to announce the launch of our SMS Gateway to send SMS through Zoho Campaigns. We have also made a few other changes in our current SMS Campaign model to improve your over all user experience. These updates are planned with an aim to expand

                                                                                                  • Update a lookup field in CRM from Creator using deluge

                                                                                                    I have a Creator form that creates a new account. When it creates the new account in the Accounts Module, I need it to also populate the Parent Account, which is a lookup field coming from the Module Parent Accounts, field Parent Account Name. I have

                                                                                                  • SMS Keyword Tracking in Zoho CRM From Zoho Campaigns

                                                                                                    Is there a way to track SMS leads in campaigns by associating them with specific keywords or codes? Additionally, can these leads be pushed to the CRM while retaining the keyword for tracking and reporting purposes?

                                                                                                  • Projects Multiselect API

                                                                                                    Having troubles setting a mutli select field via API. updateMap = map(); updateMap.put("UDF_MULTI1","picklist_id1,picklist_id2"); updateProjectRes = invokeurl [ url :"https://projects.zoho.com/restapi/portal/XXXX/projects/" + projectID + "/" type :POST

                                                                                                  • Bill quantity received / PO quantity

                                                                                                    PO's are raised & often the quantity received is greater than the PO quantity, so when we receive the bill & adjust the quantity on the bill we get.... Quantity recorded cannot be more than quantity ordered. This necessitates the adjustment of the PO

                                                                                                  • ZOHO BOOKS - RECEIVING MORE ITEMS THAN ORDERED

                                                                                                    Hello, When trying to enter a vendor's bill that contains items with bigger quantity than ordered in the PO (it happens quite often) - The system would not let us save the bill and show this error: "Quantity recorded cannot be more than quantity ordered." 

                                                                                                  • Adding Bluesky channel

                                                                                                    Hello, Is Bluesky (AT protocol) soon added on Social ? Bluesky is being developped and is now open to anyone (no more invitation) Thank you

                                                                                                  • Using Queries with dynamic parameters in Kiosk Studio

                                                                                                    Hi, I'm pretty new when it comes to developing within Zoho (I'm really a .NET developer), as it was just added to my responsibilities. For a new feature in the CRM, I'm trying to develop a Kiosk function to show a list of records (retrieved by the new

                                                                                                  • Unused items should not count into the available number of custom fields

                                                                                                    Hey, I realized that unused Items reduce the number of available custom fields. I can't see a case where that makes sense. Especially in our case where we have two different layouts in Deals with a lot of different fields, this causes problems.

                                                                                                  • Introducing Bot Filtering for Accurate Email Campaign Analytics

                                                                                                    Dear Marketers, We're excited to announce a new feature designed to enhance the accuracy of your email campaign analytics: bot filtering. This feature helps you filter out bot-generated opens and clicks, ensuring your campaign reports reflect genuine

                                                                                                  • Tip 37: Time Log Restriction in Zoho Projects

                                                                                                    Timesheet in Zoho Projects helps you big time in entering log hours for the tasks and issues and approving them. Now, with the new Time Log Restriction option, you can set daily and weekly log hour limits. You can restrict users from entering extra log hours than the permissible limit. The limits are restricted to 24 hours per day and 168 hours per week by default based on business hours. To customize, navigate to Task & Timesheet settings under Portal Configuration in Zoho Projects setup and enable

                                                                                                  • Chart showing schedule

                                                                                                    I want to be able to create a chart for everyday to check and which of which driver is available on the timeframe. Here's my table Name City Day Start Time End Time Driver1 Medicine Hat Monday 11:45 AM 4:45 PM Driver 2 Medicine Hat Tuesday 11:00 AM 7:00

                                                                                                  • Multiple Vendor SKUs

                                                                                                    One of the big concerns we have with ZOHO Inventory is lack of Vendor Skus like many other inventory software packages offer. Being able to have multiple vendor skus for the same product would be HUGE! It would populate the appropriate vendor Sku for

                                                                                                  • In Zoho Projects, is there a way to create a folders template under documents that can be used once a project is created?

                                                                                                    We have a specific folder structure that we would like to use that is standard across every project. Instead of having to create this structure every time a project is created, is there a way to create a template for the folders that can be added?

                                                                                                  • Zoho Payroll in France

                                                                                                    When will Zoho Payroll be available in France ?

                                                                                                  • Blueprint - 'On hold' state with an automatic transition?

                                                                                                    I think I'm missing something here so I'm hoping if I explain what I'm trying to achieve someone might be able to give me a way around it. We sometimes get request far in advance but we don't want to action them unless it is 7 days from when they are

                                                                                                  • HOW TO VIEW INDIVIDUAL COST OF NEWLY PURCHASED GOODS AFTER ALLOCATING LANDED COSTS

                                                                                                    Hello, I have been able to allocate landed costs to the purchase cost of the new products. however, what i need to see now is the actual cost price (original cost plus landed cost), of only my newly purchased products to enable me set a selling price

                                                                                                  • Client Script: $Client.refresh({ triggerOnLoad: true }); not triggering onLoad Client Scripts

                                                                                                    Hey friends! I'm trying to store a temporary var, refresh the page for the user, then check that temporary var and do some actions. Theoretically using the title's code: $Client.refresh({ triggerOnLoad: true }); should refresh the page and trigger on

                                                                                                  • Calculate months and years between 2 dates on subform

                                                                                                    I am looking for a function syntax for an employment candidate to calculate the number of years and months (decimal format. eg 1.2 years) they are employed. I have their start date entered, but if the end date is blank, that tells me they are still employed

                                                                                                  • Combine related grouping values into categories in CRM analytical components

                                                                                                    Hello everyone, Analyzing large datasets can be challenging when dealing with numerous individual data points. It's often difficult to extract meaningful insights when information is scattered and ungrouped. To address this, we're adding options to create

                                                                                                  • How Kiosk Studio can simplify sales for bank employees | Kiosk Studio Session #4

                                                                                                    Hello everyone, Banks can boost revenue by cross-selling to their current customers. For example, they can sell credit cards, personal loans, and more to existing account holders. To do this, bank employees move all around the CRM, open and close records,

                                                                                                  • Next Page