Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard
Howdy Tech Wizards! Welcome to a fresh week of kaizen.
This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget.
Overview
For the use case mentioned above,
- Build and deploy a web scraping service on a cloud.
- Create a Zoho CRM variable to store the response from this web scraping service.
- Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
- Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.
1. Create a Web Scraping Service
Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
- Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
- Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post.
Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function.
Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.
2. Create a Zoho CRM Variable
Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.
Step 2: Click on the Create New Variable button and provide the necessary details.
Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.
For a step-by-step guide on creating a variable, check here.
3. Write a Custom Schedule Function
Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.
Step 2: Click on the Create New Schedule button and provide a name to the schedule.
Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.
Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements.
First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API.
You can also copy and paste the code from here.
response = invokeurl [ url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/" type :GET ]; info response; scrape_response = response.toString(); variable = Map(); variable.put("id","5545974000007587001"); variable.put("value",scrape_response); variables_list = List(); variables_list.add(variable); param = Map(); param.put("variables",variables_list); response = invokeurl [ url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001" type :PUT parameters:param.toString() connection:"crm_oauth_connection" ]; info response; |
For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.
Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.
For more information about Schedules, check this help page.
Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable.
4. Code your Widget
Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.
Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.
Copy and paste the following code in your widget.html file.
<html> <head> </head> <body> <div id="blogs-table-container"></div> <script type="text/javascript"> ZOHO.embeddedApp.on("PageLoad", function () { var conn_name = "test_api_connection"; var req_data = { "method": "GET", "param_type": 1 }; ZOHO.CRM.CONNECTION.invoke(conn_name, req_data) .then(function (data) { console.log(data) const div = document.getElementById("blogs-table-container"); div.innerHTML = data.details.statusMessage.variables[0].value; }) }) ZOHO.embeddedApp.init(); </script> </body> </html> |
5. Upload the Widget and Check Your Results
Step 1: Execute the zet validate and zet pack commands to validate and pack the widget.
Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button.
Fill in the necessary details and upload the package.
Step 3: Let us now check the dashboard widget from the Zoho CRM Home page.
We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites.
Hope this post was insightful and help your interest.
If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!
Stay tuned until we circle back to you next Friday!
Cheers!
-------------------------------------------------------------------------------------------------------------------
Recommended Reads
- Variables - Overview, GET Variable API, UPDATE Variable API
- Scheduler - Create Schedules in Automation
- Functions - Overview, Deluge, Kaizen posts on Functions
- Widgets - Overview, Installation and Package, Use Cases
-------------------------------------------------------------------------------------------------------------------
Previous Kaizen: Kaizen #139: Using Functions in Client Script and Handling Timeouts | Kaizen: Home
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!
Access your files securely from anywhere
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Kiruthiga G
Sticky Posts
Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! CommandsKaizen #165 : How to call Zoho CRM APIs using Client Script
Welcome back to another exciting Client Script post! In this post, we will discuss one of the most frequently asked questions: How do you call Zoho CRM APIs from Client Scripts? In this kaizen post, 1. Ways to make calls to Zoho CRM APIs using ClientKaizen#162 : Add Tags and Open Pre-filled Email Drafts via Client Scripts
Hello everyone! Welcome to another informative Kaizen post. In this post, let us see how to accomplish the following using Client Script. 1. How to auto-tag a record based on field update? 2. How to Open a pre-filled email draft This post will provideExtension Pointers - JS SDK series #4: Managing data from a widget using ZOHO.CRM.HTTP
Handling and managing the data between two applications is a major factor for an efficient business. There are a variety of ways to handle data between Zoho CRM and other third-party applications. You can use deluge CRM tasks, create connectors, use APIKaizen #152 - Client Script Support for the new Canvas Record Forms
Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achieved
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho DataPrep Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho Campaigns Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Managing Prepaid Hours for Consulting
We are a consulting firm that bills clients a flat upfront annual fee plus an hourly rate and offer a discount for pre-paying a block of hours. Hours that surpass the pre-paid block are billed monthly at the normal rate. If there are any pre-paid hours remaining at the end of the project they are banked for future use. I'm not seeing a method of doing this in Projects/Books/CRM... thoughts?ZOHO Widget SDK not loading in html
I have this code below, I have imported the widgetsdk however I get the error shown in the image, I have tried many different ways of importing and initiating the function ZOHO but nothing is working. can someone explain what I'm doing wrong, if I amEnhancements to Zoho Corp Help Center "Team Requests" View
Dear Zoho Team, I hope this message finds you well. The ability to view both my tickets and my team’s tickets in the Zoho Corp Help Center is a fantastic feature, especially as the focal point for Zoho in our organization. However, we’ve encountered aAllow Multiple Scheduled Appointments with Zoho Support
Dear Zoho Team, I hope you're doing well. First, thank you for introducing the option to schedule support calls via the Zoho CRM booking link. This has been a fantastic enhancement, eliminating the need for back-and-forth coordination when schedulingProjectwise budget ---
Can we have a Project wise subject in addition to the Monthly, and quarterly ACCOUNT LEVEL budget?WorkDrive API Documentation
WorkDrive provides users and developers an extensive set of APIs to help integrate functionalities of Zoho WorkDrive with other Zoho applications and third-party tools. We have published the official WorkDrive API Documentation page for all external users.Error 403: Forbidden When Updating Email Signature via API
Hi Zoho Desk team, First, congratulations again on the excellent Zoho API. But, I’m encountering an issue while attempting to update an email signature via the API. Whenever I make a request to update the signature, the response returns an HTTP 403 ForbiddenWho can see draft replies on tickets?
We have noticed that we are able to see draft replies made by other agents. Which settings can limit this visibility? It makes sense to me that admins and the agent who created the draft would be able to see the draft, but no one else. How can we makeSerious question: Are there actually "solo-preneurs"/small business owners who made Zoho-one work well for them?
L.S. After already many years of continued struggle with Zoho-One, I am seriously wondering if there are actually solo-preneurs (one person small business owners - without a large, dedicated IT dept.) who got it (Zoho-One) to work well for their businesses.Major iOS issues when accessing forms via the browser
Hi, We have been using forms for some time, while the office staff are accessing the forms via the app on Android mobiles, we have a fleet of sub contractors that we would not like them having access to the main app as some of the forms are confidentialAll notes disappeared
I've been using the notebook app for over five years on my phone without being logged into an account. A few days ago I opened the app and all my notes had disappeared. Since then I tried restarting my phone, updating the app and logging into my account,How to Iterate a Function in Zoho Desk Workflow with Delay Between Calls?
Hi everyone, I’m working on a function in Zoho Desk that searches for a specific ticket record. If the ticket is not found, I need to retry the search multiple times with a delay between each attempt until the ticket is located or a maximum number ofHow to Iterate a Function in Zoho Desk Workflow with Delay Between Calls?
Hi everyone, I’m working on a function in Zoho Desk that searches for a specific ticket record. If the ticket is not found, I need to retry the search multiple times with a delay between each attempt until the ticket is located or a maximum number ofWork Orders / Bundle Requests
Zoho Inventory needs a work order / bundle request system. This record would be analogous to a purchase order in the purchasing workflow or a sales order in the sales cycle. It would be non-journaling, but it would reserve the appropriate inventory ofZoho Books API Limit Is RIDICULOUS!!!!!!!!!!!!!
The 2,500 API call limit in Zoho Books is about as useful as AOL dialup. Seriously Zoho, not only can I use up 2,500 API calls in no time with my own app but YOUR OWN STUPID IPAD APP blows through them super fast too, so if any one of my clients wantsQR codes in templates
I'm excited about the new QR code generator. I have included a QR code that contains the record ID setting "${ID}" as input data. In the report detail it works perfectly but when printing it in a template the code is not shown.Button Display Conditions
Hi Guys, Is it at all possible to have extra button conditions? Context: We have data in our deals module which has a custom button which converts the deal into contacts + set up relationships between them. At the end of the conversion we set a fieldKnowledge base: The nitty-gritty of SEO tags
A well-optimized knowledge base with great SEO can benefit your company by allowing customers to find help articles and support resources using search engines. This enables customers to quickly and efficiently find the information they need without directSocial Media Simplified with Zoho Social: Make the best out of the publishing calendar
Are you a marketer who likes visualizing your plan of action before you start social media posting? Are you part of a team that works on social media on a rotational basis, so the most important task is to collaborate to avoid overlap and confusion? OrCustom function daily limit and procedural programming
Dearest Zoho Today, support confirmed that if I call a custom function from another custom function then I will use up two, with regards to my daily limit. A few times, we have blown our daily limit and that means that ordinary business processes don't run for the rest of the day. I have to mop these up the following day and there is no guarantee that I will get it right. Therefore, I can't afford to waste any. Procedural programming has been around for over 50 years now and it greatly simplifiesUnified customer portal login
As I'm a Zoho One subscriber I can provide my customers with portal access to many of the Zoho apps. However, the customer must have a separate login for each app, which may be difficult for them to manage and frustrating as all they understand is thatWelcome Link Expired
Hi The links sent to the users didn't get clicked on in time and now all the links have expired. Is there a way to send a new link without deleting them and re-adding them>New enhancements: Changing portal users' email addresses and new customization options for templates
Dear All, Portals have enabled organizations to extend access to various CRM modules to their customers, vendors, partners, and end users, per their business requirements. When a portal is created, an invitation email is sent to portal users with a linkGranular Time Frame Settings for Message Deletion and Editing in Zoho Cliq
Dear Zoho Team, I hope you're doing well. Currently, the settings for message deletion and editing in Zoho Cliq are configured globally under: Admin Panel > Organisation > Configurations > Conversations Delete messages: Time frame to allow message deletionNew Built In QR/Barcode Generator Print Settings
I'm trying out the new QR/Barcode generator field in Creator. I would think most people will want to print these, like I do. I am not seeing any way to control the height or width of the barcode for printing (inside the print/pdf template builder). TheZoho One. Client Script
Hi, I would like to know if the Client Script feature is available in Zoho One. If it's, how can I enable it?Calendar View for Zoho Tickets
Is there a way to view your tickets with due dates on a calendar view? I can not find a way to merge my Zoho Calendar and Tickets. This would be extremely helpful to my team.Delete / Modify Default Career Site - Zoho Recruit
Hello, It would be very useful if we could delete a default career site or change which of our career site is the default. Our Career site was created when there were issues with Zoho Recruit creating English CTA buttons on French Career sites. The onlyWorkflows for Timesheet
Good day, Any way to have timesheet as triggers? I looked into Zoho Flow and into Zoho Project automation but no where can I have timesheet as a trigger. Basically, I would like to trigger something upon timesheet approval. Right now, the only way toIs it possible to hide Developer Space for all user in Zoho Projects
Hello! I am Zoho admin in a company and we want to use Zoho Project to manage projects, but after a few days of testing we are not able to "hide" the Developer Space from all kind of users except the admin. To sum up, I want to hide this for all users.Introducing automation and utility conversations in WhatsApp marketing
We’re excited to announce the addition of two new features to our WhatsApp integration: Automation and Utility conversations. These enhancements will allow you to streamline your marketing efforts and engage with your customers more effectively by automatingExtracting data from cells in zoho sheets for zoho books
I am currently uploading my bank statment in excel format to zoho workdrive. I would like flow to extract certain data and send it to zoho books. Would scripting in zoho flow be able to help me with this? By this I mean should I attempt this in zoho flowWithin the Basic KPI component in Analytics, it is impossible to set "next" day range as a filter
Hi there, I am currently setting up a deal dashboard for the Sales team. While it is possible to filter deal records to show records that were created LAST X days only, it looks like a NEXT X days Closing date filter is not available. Would it be possiblePulling Specific Products from Sales Orders in Books to a CRM Record
We currently process orders directly through our website (woocommerce) as well as through manual sales orders in zoho books. When an order comes through the website, all of the individual products from that order show up in the CRM record of that customer.Você já viu os cursos do Zoho Mind?
Pessoal, Tem uma plataforma da Zoho chamada Zoho Mind, muito interessante os cursos e vídeos tutoriais que lá possui. Para a turma do Zoho Creator, tem uma dica de Buscar dados em Formulário, segue o link e clique em Zoho Creator. https://www.zohomind.com.br/#/videostutoriaisComo gerar gatilhos para pagamento de impostos no Zoho Books?
Olá Pessoal, boa tarde! Gostaria de saber como vocês estão escriturando os impostos a pagar no Zoho Books. Vi que temos a opção de Bills, porém se eu escriturar nesta aba do Zoho Books para gerar lembretes de tempo de vencimento por exemplo vai refletirSubform Time field to string.
Good afternoon All. I have a Subform 'Delivery_Receiving_Hours' that captures Day (Dropdown), Time_Open (Time), and Time_Close (Time). I need to capture this data and send it to a multiline field in the CRM. The code, posted below, below will captureworkflow for bounced email gets triggered, but email is status = opened
Hello, I have a workflow that sends me an email if outgoing email are bounced. Now I got some kind of this emails, but the corrosponding contacts have status = open at the email. Why this bounce-workflow is triggered? Reports > Email Reports > BounceData export
I need to export our customer's data and projects' data for our purpose but am unable to export full data i only get around 3160 projects and around 2k customer can you please help me to get full data, pleaseAdjusting Physical Inventory
Not getting very far with support on this one, they say they are going to fix it but nothings happened since November. Please give this a thumbs up if you would like to see this feature or comment if you have some insight. Use Case: Inventory set to beNext Page