Kaizen 140 - Integrating Blog feed scraping service into Zoho CRM Dashboard
Howdy Tech Wizards! Welcome to a fresh week of kaizen.
This week, we will look at how to create a dashboard widget that displays the most recent blog post of your preferred products/services, updated daily at a specific time. We will leverage the potential of Zoho CRM's Function, Variables, and Automation Schedules to achieve this widget.
Overview
For the use case mentioned above,
- Build and deploy a web scraping service on a cloud.
- Create a Zoho CRM variable to store the response from this web scraping service.
- Write a custom schedule function to fetch the web scraping response and update the previously created CRM variable.
- Create a dashboard widget that makes a GET API call to that specific CRM variable on page load and displays the response.
1. Create a Web Scraping Service
Develop a web scraping script in your preferred language based on your products/services' blog sites. In this demo, we have used NodeJS and the blog sites of various Zoho products. Our data requirements in the dashboard widget are the title of the latest blog, its URL, and mapping these to their respective products. Since recent posts on these pages often appear first, we have applied the following logic to scrape the sites:
- Use the axios and cheerio libraries in NodeJS to fetch and load the HTML data of the blogs in the DOM format.
- Use a getLatestBlog function that fetches the HTML data and utilizes selectors to retrieve the required information (latest blog title and page URL). This data is then mapped to their respective products and fed into a table.
The sample code for web scrapping is provided as an attachment to this post.
Deploy this NodeJS web scraping script to a server or a cloud function. In our case, we have uploaded it to Catalyst Serverless Function.
Make a note of the Function URL for the NodeJS web scraping script provided by Catalyst after hosting it. We will invoke this URL from a Custom Schedule Function.
2. Create a Zoho CRM Variable
Step 1: In your Zoho CRM, go to Setup > Developer Hub > Variables.
Step 2: Click on the Create New Variable button and provide the necessary details.
Initially, set a dummy value. Make a note of this variable ID as the response from our web scraping script will be stored in it with the help of Custom Schedule Function.
For a step-by-step guide on creating a variable, check here.
3. Write a Custom Schedule Function
Step 1: Go to Setup > Automation > Schedules in your Zoho CRM.
Step 2: Click on the Create New Schedule button and provide a name to the schedule.
Step 3: In the Function To Be Executed field, choose Writing Function. A pop-up will appear with the necessary details to create a function.
Step 4: Once you hit the Create button, the deluge code editor appears within which you have to code the following requirements.
First, invoke the function URL of the web scraping script. This action will run the script and provide a response. Then, use this response to update the designated CRM variable that was previously set up for this task by using the Update Variable API.
You can also copy and paste the code from here.
response = invokeurl [ url :"https://blog-scrapper-789629878.development.catalystserverless.com/server/blog_scrapper_function/" type :GET ]; info response; scrape_response = response.toString(); variable = Map(); variable.put("id","5545974000007587001"); variable.put("value",scrape_response); variables_list = List(); variables_list.add(variable); param = Map(); param.put("variables",variables_list); response = invokeurl [ url :"https://www.zohoapis.com/crm/v6/settings/variables/5545974000007587001" type :PUT parameters:param.toString() connection:"crm_oauth_connection" ]; info response; |
For information on invoking API URLs, please refer to the connections help page. Note that the web scraping URL does not require any authentication, as it is a public URL.
Step 5: Click the Save button to associate the function with your schedule. Configure the frequency of the scheduler to run the function every day at a specific time.
For more information about Schedules, check this help page.
Every day at 11:00 AM, the scheduler will invoke our web scraping script hosted in Catalyst and store its response in a variable. Next, we need to create a dashboard widget that displays the response stored in this variable.
4. Code your Widget
Step 1: Refer to this help page and initiate a widget project in your local environment using Zoho CLI.
Step 2: Everytime, the dashboard widget is loaded, the code in widget.html file executes an API call to the Zoho CRM Variable, which we updated in the Scheduled Function. It loads the response from the GET Variable API call into the widget.
Copy and paste the following code in your widget.html file.
<html> <head> </head> <body> <div id="blogs-table-container"></div> <script type="text/javascript"> ZOHO.embeddedApp.on("PageLoad", function () { var conn_name = "test_api_connection"; var req_data = { "method": "GET", "param_type": 1 }; ZOHO.CRM.CONNECTION.invoke(conn_name, req_data) .then(function (data) { console.log(data) const div = document.getElementById("blogs-table-container"); div.innerHTML = data.details.statusMessage.variables[0].value; }) }) ZOHO.embeddedApp.init(); </script> </body> </html> |
5. Upload the Widget and Check Your Results
Step 1: Execute the zet validate and zet pack commands to validate and pack the widget.
Step 2: Go to Zoho CRM > Setup > Developer Hub > Widgets and click the Create New Widget button.
Fill in the necessary details and upload the package.
Step 3: Let us now check the dashboard widget from the Zoho CRM Home page.
We have used the blog sites of a few Zoho products for this demo. You can replace them with your required products/services' sites.
Hope this post was insightful and help your interest.
If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We eagerly await your thoughts and feedback on this!
Stay tuned until we circle back to you next Friday!
Cheers!
-------------------------------------------------------------------------------------------------------------------
Recommended Reads
- Variables - Overview, GET Variable API, UPDATE Variable API
- Scheduler - Create Schedules in Automation
- Functions - Overview, Deluge, Kaizen posts on Functions
- Widgets - Overview, Installation and Package, Use Cases
-------------------------------------------------------------------------------------------------------------------
Previous Kaizen: Kaizen #139: Using Functions in Client Script and Handling Timeouts | Kaizen: Home
Join us for our upcoming Zoho CRM Developer Series: Zoho CRM APIs, where you can explore more about Zoho CRM APIs. Register Now!
Access your files securely from anywhere
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Kiruthiga G
Sticky Posts
Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! CommandsKaizen #165 : How to call Zoho CRM APIs using Client Script
Welcome back to another exciting Client Script post! In this post, we will discuss one of the most frequently asked questions: How do you call Zoho CRM APIs from Client Scripts? In this kaizen post, 1. Ways to make calls to Zoho CRM APIs using ClientKaizen#162 : Add Tags and Open Pre-filled Email Drafts via Client Scripts
Hello everyone! Welcome to another informative Kaizen post. In this post, let us see how to accomplish the following using Client Script. 1. How to auto-tag a record based on field update? 2. How to Open a pre-filled email draft This post will provideExtension Pointers - JS SDK series #4: Managing data from a widget using ZOHO.CRM.HTTP
Handling and managing the data between two applications is a major factor for an efficient business. There are a variety of ways to handle data between Zoho CRM and other third-party applications. You can use deluge CRM tasks, create connectors, use APIKaizen #152 - Client Script Support for the new Canvas Record Forms
Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achieved
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho DataPrep Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho Campaigns Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Zoho Analytics Pivot Table - How to compare month vs last year same month
Hi, I had created a pivot table with setup as below: Column: - Delivery Date Row: - Customer Group Data as column - Total amount (sum > normal) (show data of the month) - Total amount (sum > % difference from previous value) (compare between this monthRecorded imported goods
I have imported some goods from a foreign vendor called ABC. to Bahrain. My Customs duty and VAT are paid by an agency called XYZ to authorities. I have paid the amounts to XYZ and they have paid to authorities in behalf of us using our TRN Number. NowNotes created in mobile app not syncing with web notebook
I have created certain notes in a new notebook in mobile app. The new notebook is also created using mobile app and when I logged in through web I am unable to see notebook that I have created using mobile app. there is no trace of notes in which i have created in mobile app on the web even after days of creating in mobile app This is not the case vise-a-versa. the notes and note book created on web are accessible on mobile app on real time. this is very inconvenient and big demotivating factorMatch Transaction > Filter | Allow wildcards in search
The Filter function is useful for narrowing down to a specific set of transactions. It would be useful to have the ability to use wildcards in search. e.g. customer name "starts with" or " * " as a placeholder. e.g. "National * " instead of "NationalMarketing Automation Emails Going to Spam
Google is trapping all the marketing automation emails in spam. My domain has a perfect reputation and it looks like Zoho has a low reputation which is sending it to spam. When I pull the email out of spam and click on a link in the email, I get thisConvert Request to Forum Topic
I know you can convert a forum topic to a request and a request to a solution in the knowledge base, but how do you convert a request to a forum topic? Is this possible? If not is this a feature coming down the pipe in the near future? Thanks!Zoho Sign / Prefill By You - Editable by Them!
Zoho Sign seems to have two field options available - an empty field for signer(s) to complete or a field that has been prefilled by you, that becomes read only once sent. What I would like is an option on the pre-filled by you fields, to leave themIs the Contacts sync between Campaigns and CRM bi-directional?
Is the Contacts sync between Campaigns and CRM bi-directional?Task does not syncing to google calendar shedual
Hi why does the tasks activities do not syncing with the google calendar like the event option does?Bug with Zobot Human transfer
I have configured the brand, the operator, and the Zobot in Spanish. However, no matter what I do, it always says "YES" instead of "Sí" or "Sure, why not " instead of "Claro, ¿por qué no?" when asking if the user wants to contact a human operator. TheThis mobile number has been marked spam. Please contact support.
Hi Support, Can you tell me why number was marked as spam. I have having difficult to add my number as you keep requesting i must use it. My number is +63....163 Or is Zoho company excluding Philippines from their services?Something went wrong. One or more fields contain errors
I am getting this error but there is no way to debug what field is causing the issue . I have over 100 fields. Everything was working fine and then i fixed some fields that should have the same field names but had a typo and i am getting this error. ItIntegrating Zoho Desk Instances from two separate organizations
Is it possible to integrate Zoho Desk with an instance from another organization? For example, creating a ticket in one organization can cause the creation of a ticket in the second organization? Or certain tickets from one organization be viewable byKnowledge base bug - Error: Article couldn't be updated.
It took me a while to figure this out, truly one of the most irritating bugs in Zoho. I hope you find this information helpful. When using the knowledge base to create articles, make sure your keywords are in lowercase and separated by commas. Otherwise,Is this possible with Campaigns?
My company is currently moving CRM's from Monday to Zoho One. Currently, our marketing lead process is send out 7 sms messages over 14 days if the lead is in a certain status. If we don't get a response the lead is put into a "Closed" status. Do we buyHow to view two portals (Zoho CRM & Zoho Books) in single login
Hello there, I need to create a portal access for the Customers. Customer data present in two applications namely Zoho CRM & Zoho Books. My requirement is to show the two different portals(Zoho CRM Portal & Zoho books Portal) in single login. I couldUpdating Bounced Contacts in Campaigns to the CRM
The article explains how to update Opt-outs from Campaigns to CRM, but not bounces. While Campaigns will automatically remove any bounced emails from future campaigns, is there a process by which a bounce will update or make a notification in the CRMBandwidth, voice and poor video
Dear Team, The webinar has made things quite easy and convenient for educators around the world. We are an art & culture-based organization where students and researchers are our major audience. While using the Zoho webinar platform, we have observed that the audio lags and the video is of poor quality and often freezes. Some participants don't hear what is being spoken. We also found that even a slightly slow internet connection disrupts the webinar while it should be working well even on lowerZoho books and venmo
Hi, Is there a way to hook Venmo into zoho books? I have a Venmo business account and want to be able to sync that. I know you can do it with the paypal integration but I dont want to use paypal for the fees and that doesnt allow me use/integrate my currentGlobal Sets for Multi-Select pick lists
When is this feature coming to Zoho CRM? It would be very useful now we have got used to having it for the normal pick lists.550 5.4.6 Unusual sending activity detected. Please try after sometime. <a href=https://www.zoho.com/mail/help/usage-policy.html target=_blank>Learn more.</a>
Please help me with this. Sending of mails is blocked.The Social Wall: November 2024
Hey everyone, As we move into December, we're excited to share all the updates that went live in Social during November. View, monitor, and respond to your WhatsApp and Telegram messages from Inbox Take your communication a step further by integratingSegmenting Contacts Based on Product Purchased
I am trying to organize our main Marketing Automation email list in a way that segments contacts based on products they have purchased (for example in this case it is 3 different products). To my knowledge, this would require the sync from Zoho CRM toUrgent Assistance Needed with DKIM Verification
Hi, I have been trying to verify the DKIM for the past month using your instructions as well as other resources. Unfortunately, I have not been able to resolve the issue, and it remains persistent. I need to address this problem as soon as possible soHow do I attach tasks from one task to other tasks in the same project
How do I attach tasks from one task to other tasks in the same projectIncorrect Handling of XLSX data
Trying to import an XLSX schedule of bills into Zoho Books I ran across the problem of date formatting. To replicate: Build a CSV file with bill dates in whatever format you like and import it - this should work if you match the "dd/MM/yyy" etc. formatZoho Books Invoice Salesperson: requires ID but there is no "Fetch salesperson" action
Hi, I am trying to attach a Salesperson to a Zoho Books invoice. In Zoho Flow, the salesperson field required an ID. However, I only have the salesperson name, I need to fetch salesperson by name and then provide the ID. There is no option to fetch salesperson...Change format of quantity format
Hi, I would like to change the qunatity format from 1,00 to 1. Is this possible? thanks!Ticket Views: filter criteria -> dynamic date values in relation to the current date
Hello all, It would be very helpful if you could build custom views in such a way that you do not have to adjust the criteria daily or at whatever interval in order to change the fixed date value as needed. For example, I would like to create a view that,Captchas: No support for Google reCAPTCHA or similar
Hi all, The current captcha integrated into Creator is very basic, and often near-impossible to read. I'm building an app for a charity which includes a couple of public-facing forms, and this is a real issue for people with dyslexia and related conditions - it's effectively excluding them if you want any form of spam protection at all on your published forms (and that being said, the current captcha doesn't even seem very effective - bots can often read it more easily than humans). I've raised thisMaintain consistency in ticket responses with shared snippets
Hello everyone! We are excited to announce that our highly anticipated snippet sharing feature is now available to all users. As you know, snippets are pre-defined message templates, or canned messages, that help agents respond to tickets with efficiency.Tickets - Zoho Desk
Hi Team, My Clients need to see their tickets created and the status of the ticket in the Zoho Support Desk itself. How can I do this? My Client doesn't have a Zoho Account. They need to access the ticket by the provided link without signing in.Automatically assign Contacts to Account owners
Hi, I have a finite number of accounts set up in the CRM, and each new contact that comes in is automatically assigned to an Account according to a rule I set up. I want the Contact owner in the Contacts module to be assigned to the relevant Account owner.Making Copies/Duplicates of Zoho Forms (Shared)
Question to the community: is there a way to take a 'shared form' , make a duplicate copy and save under My Forms, so that i can use that which was already created as a template to make updates to and use as a test form and be able to have full access,Using IMAP configuration for shared email inboxes
Our customer service team utilizes shared email boxes to allow multiple people to view and handle incoming customer requests. For example, the customer sends an email to info@xxxx.com and multiple people can view it and handle the request. How can I configureCan you help us creating a customised form with payment link?
I would like to create a customised Transport form where the user will be asked to make payment basis the drop/ pick up they select.Deleting or disabeling predefined ticket list views
Is it possible to delete or disable predefined views or is this still not possible? For instance, we are not using the chat function and therefore have no use for the "Missed Chats" view. Thanks!Email an Invoice from API
When calling the API to email an invoice, it seems it does not actually send it. Below is a screen shot where I used an API call for the bottom record, and it updated its' status to "Due Today", but no email was actually sent. The top record, I hit theUnable to load your extension. Please check your plugin-manifest or Resources.json.
Hi Team, I am using the config module with multiple fields of different types, such as checkboxes and picklists. However, I am encountering the following issues: Error Message: When loading the extension, I get the error: "Unable to load your extension.Layout Rules / Quick create
Hello, is there a way to create a layout rule for quick create option? Regards, KatarzynaNext Page