Kaizen #122 - Data Synchronization using Bulk Read and Notification APIs
Hello everyone!
Welcome back to yet another insightful post in our Kaizen series.
Welcome back to yet another insightful post in our Kaizen series.
Consider that your organization is involved in selling laptops. Your sales team uses Zoho CRM for their daily activities, managing Leads, Deals, and inventory modules, whereas your Accounts team relies on a legacy application. It is important to maintain data synchronization from Zoho CRM to the legacy system to ensure that the Accounts team always works with the latest information.
To ensure that this synchronization happens in near real-time, the legacy application uses Zoho CRM's Bulk Read API and Notification API.
In this article, we'll discuss how one-way data sync will be achieved for the "Leads" module.
Bulk Read API
This asynchronous API facilitates the export of up to 200,000 records from a module in a single job. If there are more than 200,000 records, additional API calls are required to fetch all the records.
Notification API
This API allows you to subscribe to events such as creating, updating, and deleting a record in a module through a webhook URL. Each of these actions triggers a corresponding HTTP request to your webhook.
Flow Diagram
The above diagram represents the initial sync on how the legacy application syncs data from Zoho CRM.
Step 1: Sync Start
This is the starting point of the code for the data sync from Zoho CRM to the legacy application and keeping the data updated in near real-time.
Step 2: Subscribe to the channel via Notification API
First, subscribe to events (create, update, and delete) occurring in the Leads module through a channel created via the Notification API, and then trigger the Bulk Read API (Step 3) to synchronize the entire data in Zoho CRM.
Reason:
During the asynchronous execution of the Bulk Read API job, all records from the specific module are saved to a CSV file. Let's consider a scenario where a record that has already undergone processing by the Bulk Read API is updated through the web UI or any other source. This means that the data in the Bulk Read CSV may become outdated by the time the job concludes.
To address this issue, you can leverage the Notification API. You subscribe to a channel, actively monitoring any actions performed on records in that module. When an action occurs, you will receive a notification through the configured URL. Subsequently, upon receiving this notification, you can store the updated data in the legacy database, including the modified time. This ensures that only the most recent data is stored in the database.
Sample Request URL: {{api-domain}}/crm/v6/actions/watch
Request Method: POST
Sample Input
{ "watch": [ { "channel_id": "10000", "events": [ "Leads.create", "Leads.edit", "Leads.delete" ], "channel_expiry": "2024-01-21T00:13:59-08:00", "token": "leads.all.notif", "notify_url": "https://webhook.site/6c3d08c2-2c21-4d16-9134-712acb513915" } ] } |
Sample Response
{ "watch": [ { "code": "SUCCESS", "details": { "events": [ { "channel_expiry": "2024-01-21T00:13:59-08:00", "resource_uri": "https://www.zohoapis.com/crm/v6/Leads", "resource_id": "5725767000000002175", "resource_name": "Leads", "channel_id": "10000" } ] }, "message": "Successfully subscribed for actions-watch of the given module", "status": "success" } ] } |
The response confirms a successful subscription for actions-watch on the "Leads" module. For more information about the Notification API, refer to this Kaizen post on the Notification API.
Step 2.1: On each notification, update that record in DB along with the modified time
If a record gets updated or a new record is created, or a record is deleted in the Leads module, you are notified about the change in data via the subscribed notification channel and stored in the legacy database along with the modified time.
Sample JSON Notification Response
Note: The primary purpose of subscribing to the channel via notification is not to miss any data that has been modified or created during the data backup.
Step 3: Bulk Read Initialize
Initiate the data backup by using the Bulk Read API after subscribing to the channel via Notification API. As the API is an asynchronous one, a bulk read job is scheduled. After the job is completed on the Zoho CRM end, a notification will be sent via the callback URL. Also, the application can periodically check the job status using the Get the Status of the Bulk Read Job API.
Note: For the Bulk Read API, the records will be sorted based on the id field in ascending order.
Sample Request for Initial Bulk Read
Request URL: {{api-domain}}/crm/bulk/v6/read
Request Method: POST
Sample Input
{ "callback": { "url": "https://www.xyz.com/callback", "method": "post" }, "query": { "module": { "api_name": "Leads", "page" : 1 } } } |
Sample Response
{ "data": [ { "status": "success", "code": "ADDED_SUCCESSFULLY", "message": "Added successfully", "details": { "id": "5725767000002002008", //job_id "operation": "read", "state": "ADDED", "created_by": { "id": "5725767000000411001", "name": "Patricia Boyle" }, "created_time": "2024-01-20T04:01:48-08:00" } } ], "info": {} } |
Get the Status of Scheduled Bulk Read Job
Check the status of the scheduled bulk read job using the job_id you received.
Request URL: {{api-domain}}/crm/bulk/v6/read/{job_id}
Request Method: GET
Sample Response
{ "data": [ { "id": "5725767000002025007", "operation": "read", "state": "COMPLETED", "result": { "page": 1, "per_page": 200000, "count": 142, "download_url": "/crm/bulk/v6/read/5725767000002025007/result", "more_records": false }, "query": { "module": { "id": "5725767000000002175", "api_name": "Leads" }, "page": 1 }, "created_by": { "id": "5725767000000411001", "name": "Patricia Boyle" }, "created_time": "2024-01-24T21:35:11-08:00", "file_type": "csv" } ] } |
The above response contains the status of the scheduled job as either ADDED, IN PROGRESS, or COMPLETED.
When the job is complete, the response contains the result JSON object with the keys page, count, more_records, and download_url. The download_url in the callback response from which you can download the zip file containing the CSV file.
Step 4: Sync the Bulk Read data with the legacy DB, if it has a more recent modified time
Once the bulk-read data CSV is available, the data will be updated in the database. If any record's modified time in the CSV file is less than the modified time already present in the database, then it's an outdated record and need not be updated in the legacy's database, as it has already been handled via the Notification API's subscribed channel.
If the CSV contains exactly 200,000 records, there is a possibility of having more records. In the response of the Status of the Bulk Read Job API, if the more_records key is true, it indicates additional records to be exported. To retrieve them, simply update the value of the page key in the POST request and schedule another bulk read job to fetch the next set of records. By default, you can fetch up to 200,000 records in a single API call. Repeat this process until all the records are fetched from the CRM side. For more details on how to download, refer to this Kaizen post about the Bulk Read API.
Channel Resubscription
The channel subscription will remain active for a maximum of one day. After this period, it needs to be re-subscribed using the Enable Notification API to continue receiving notifications for create/update/delete actions in the Leads module. It is recommended to perform the re-subscription every 23 hours and 55 minutes, just short of 24 hours. Note that if the expiry time is not specified during the subscription, it will expire within one hour.
Troubleshooting
There are some scenarios where data synchronization may fail.
For instance,
1. Unreachable Webhook URL: If the webhook URL is down, Zoho CRM cannot notify data updates through it.
2. Notification Expiry : If the notification expires before resubscribing to the channel, there is a risk of losing newly registered leads in between.
3. The code logic in the Webhook URL may break due to unforeseen reasons.
To handle such scenarios, store the last successful data sync time. Use this stored time as the modified time criteria in the bulk-read API to fetch any missed data and update the database.
We trust that this post meets your needs and is helpful. Let us know your thoughts in the comment section or reach out to us at support@zohocrm.com
Stay tuned for more insights in our upcoming Kaizen posts!
------------------------------------------------------------------------------------------------------------------------------
Previous Kaizen Post : Kaizen #121 - Customize List Views using Client Script
-------------------------------------------------------------------------------------------------------------------------------
Related Links
- API Help Documentations :
- Kaizen Posts :
- Kaizen Series
Access your files securely from anywhere
Centralize Knowledge. Transform Learning.
All-in-one knowledge management and training platform for your employees and customers.
New to Zoho Recruit?
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Subramanian K
Sticky Posts
Kaizen #198: Using Client Script for Custom Validation in Blueprint
Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Kaizen #226: Using ZRC in Client Script
Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In thisKaizen #222 - Client Script Support for Notes Related List
Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore howKaizen #217 - Actions APIs : Tasks
Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.Kaizen #216 - Actions APIs : Email Notifications
Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
First Name in Mail
While sending a mail/message to the user, I want only the first name to be displayed—for example: “Hi John” instead of the full name using "Hi ${Name_Field}"Add Reauthentication Option for Zoho Bug Tracker Integration in Zoho Desk
Hello Zoho Desk Team, We hope you're doing well. We would like to request an enhancement to the Zoho Bug Tracker integration within Zoho Desk. Current Limitation: At the moment, there is no option to reauthenticate the Zoho Bug Tracker integration inNewby Questions
Q1. The top bar of Zoho Books has a "Search in Banking (/) " field. What is the proper use of this text box? - Searching for Amazon for example has no results but there are transactions. - Is the search case sensitive? - Are regular expressions allowed?Transaction Rule Matching
Q1. Is there a method to have the Transaction Rules applied across multiple Bank Accounts? Q2. To match "contains" text in Transaction Rules, is a wildcard or regular expression required? Is it case sensitive? Would searching for "issue" in the followingIntroducing parent-child ticketing in Zoho Desk [Early access]
Hello Zoho Desk users! We have introduced the parent-child ticketing system to help customer service teams ensure efficient resolution of issues involving multiple, related tickets. You can now combine repetitive and interconnected tickets into parent-childThree Zoho Billing Limitations Blocking Standard Subscription Operations
After working through Zoho Billing support for over a year on these three issues without resolution, we wanted to flag them to the broader community. We are curious whether other businesses are running into the same walls. 1. Cannot Prepone (Move Earlier)Automation #3 - Auto-sync email attachments to tickets
This is a monthly series where we pick some common use cases that have been either discussed or most asked about in our community and explain how they can be achieved using one of the automation capabilities in Zoho Desk. Most of our customers use emailCan't add a sender adress from zoho campaigns
hi, I need to change the sender address for a campaign. When i try to add it i get a message to say 'duplicated email address found while adding your sender address'. This is the first campaign i'm sending so I don't understand why this message is displayed? Thanks JaneUsing a custom single line External ID form as merge fields in templates
Hey everyone, We're looking to integrate a few external systems better with our Zoho CRM, and we had hoped to use external fields for this purpose. In this case, it would mean being able to use our own inoice system's invoice numbers are a direct id compatibleIntroducing Workqueue: your all-in-one view to manage daily work
Hello all, We’re excited to introduce a major productivity boost to your CRM experience: Workqueue, a dynamic, all-in-one workspace that brings every important sales activity, approval, and follow-up right to your fingertips. What is Workqueue? SalesZoho CRM - Feature Request - Analytics Components Group By Week Alternative Formats
On the Zoho CRM Analytics Components, please consider adding an option to allow group by week format to be changed to other formats such as Week Commencing or Week Ending dates, rather than the current Week Number. This would provide improved usabilityDashboard target enhancements
Often individuals in IT are creating dashboards for their sales team. The ability to create a single dashboard that can be used by multiple people is key. A components for a dashboard have the ability to filter by logged in user which is great. HoweverZoho Campaigns - Feature Request - Re-Send Existing Email Action in Automations/Journeys
Hi Zoho Campaigns and Zoho Marketing Automation teams, I would like to suggest a feature that would make building and managing complex automations significantly easier. The Feature Introduce a “Re-Send” or “Send Existing Email” action within AutomationsCan I write a check in Zoho Books with no associated bill?
This currently does not seem possible, and I have a client that desperately needs this function if I am able to convert them with Quickbooks. Thank you in advance for your reply.ISO 3166-2 Compliant Country and Regions
Zoho should consider implementing ISO 3166-2 standards for countries and regions in address dropdowns. It adds a lot of noise in the data when items added programmatically conform to ISO 3166-2 (e.g., TX for Texas, CA for Canada, etc.), but the few itemsUS State abbreviations in Address fields
In regards to all Address fields within Zoho, Is there a way to change the State field to be the 2 letter abbreviation vs the full spelled out US State name? Example: "Washington" should be WA. I am able to type in the abbreviated state, but it's notZoho Booking - TIN vs ATIN & ITIN
Zoho Booking Vendors allows for TAX ID values of SSN, EIN, ATIN an ITIN. There is no option for TIN. What is the method to properly add TIN to the list of taxable values for companies? For reference: Social Security Numbers (SSN) Individual Taxpayer IdentificationAll new Address Field in Zoho CRM: maintain structured and accurate address inputs
Availability Update: 29 September 2025: It's currently available for all new sign-ups and for existing Zoho CRM orgs which are in the Professional edition exclusively for IN DC users. 2 March 2026: Available to users in all DCs except US and EU DC. 24How to see Statement Details Shown in Unclassified Transactions in All Transactions?
All, The list of Unclassified Transactions show the Statement Details and Descriptions. What is the method to see that column in the All Transactions list? -Thanks!Zoho Books | Product updates | October 2025
Hello users, We’ve rolled out new features and enhancements in Zoho Books. From iOS 26 updates to viewing reports as charts, explore the updates designed to enhance your bookkeeping experience. Zoho Books Updates for Apple Devices At WWDC 2025, AppleBill sent for Payment Approval
Zoho Books currently supports three levels of transaction approval for bills: Draft, Submit, and Approved. We are looking to add a layer of approval like a bill marked for payment before initiating payment requests through HSBC Bank. Can you please guideAsk the Experts - Live Q&A webinar
Hello Community, We’re excited to host our very first Ask the Experts session! Join us on 7 April 2026 from 11 a.m. to 12 p.m. (IST) for this live webinar Q&A session, where you will have an opportunity to connect directly with our product experts, gainKaizen #235 - Automating record restoration using Recycle Bin APIs in Zoho CRM
Welcome to another Kaizen week. Here's a question that came up in our developer forum: "I'm working on a piece of software to automate conversion of Leads into Deals based on order status from my company's website. There are some cases where a previouslyOpen PDFs using Zoho PDF Editor
Hello users, In this article, we will be exploring the open PDF capability in Zoho PDF Editor to collaboratively edit PDFs, insert text and images, add fillable and e-signature fields, and more. You can upload PDFs upto 50 MB or 150 pages. Password-protectedOne Contact with Multiple Accounts with Portal enabled
I have a contact that manages different accounts, so he needs to see the invoices of all the companies he manage in Portal but I found it not possible.. any idea? I tried to set different customers with the same email contact with the portal enabled andBank Feeds Breaking Constantly
Hey Everyone, I have already reached out to support about this issue but I am wondering if anyone else is having the same issue. My bank feeds keep breaking within days of me fixing them by updating the credentials. Its been happening for a while andTimed addition to segments
Hi there - hoping you can help me figure out a graceful way of doing this: • I want to add contacts to a Segment in Campaigns based on a Stage pick-list field value X in CRM - that bit's fine. The problem is that I only want to add them to the SegmentZoho Developer AI Agent = Claude AI + MCP Server + Zoho Ecosystem
Hello Zoho Community 👋 I’m excited to share a recent integration we’ve worked on at Officehub Tech: ✅Claude + MCP Server + Zoho Creator Zoho Developer AI Agent – an AI-powered Zoho automation platform This solution connects Zoho applications with anCliq iOS can't see shared screen
Hello, I had this morning a video call with a colleague. She is using Cliq Desktop MacOS and wanted to share her screen with me. I'm on iPad. I noticed, while she shared her screen, I could only see her video, but not the shared screen... Does Cliq iOS is able to display shared screen, or is it somewhere else to be found ? RegardsGoogle Fonts Integration in Pagesense Popup Editor
Hello Zoho Pagesense Team, We hope you're doing well. We’d like to submit a feature request to enhance Zoho Pagesense’s popup editor with Google Fonts support. Current Limitation: Currently, Pagesense offers a limited set of default fonts. Google FontsCRM
How do I build an email funnel of 10 emails with decision tree ie; if yes use this email if no use this email. Also I cannot figure out how to add more groups or see my whole groups. I've set up different groups like attorneys is one group of companies/peopleWebDAV support
I need WebDAV support so that I can upload/download (and modify) documents from my local file system. Is anything planned in his direction?Replying from same domain as a catch-all?
I have 2 domains setup on Zoho, both with associated email addresses. They look something like this: john@example.com (primary address) john@test.com (this domain also has a catch-all setup) I use the catch-all for test.com as a public-facing email addressIs there a way to show contact emails in the Account?
I know I can see the emails I have sent and received on a Contact detail view, but I want to be able to see all the emails that have been sent and received between all an Accounts Contacts on the Account Detail view. That way when I see the Account detailInternal Fillable Contract with Zoho Writer (Before Sending to Client)
Hi everyone, I’m trying to automate the following process in Zoho CRM and would appreciate some guidance. Process: When a Deal moves to a specific stage, CRM triggers an automation. CRM sends a contract template to an internal team member so they canStandard practice rerun updated deluge function over existing recordes
Hi folks, we have a function which is triggered via webhook from a third-party solution which then calls another api with a contact-id and gets a json payload back and then parses that data to custom fields in our CRM to the record id. As requirementsImport Quote from CRM into document in Writer?
Hi, I created a quote for a proposal in the Zoho CRM and was wondering if there is a way to embed the quote into a document in Zoho Writer so I can make our sales proposals look a little nicer. Is that possible?Zoho Sheet for Desktop
Does Zoho plans to develop a Desktop version of Sheet that installs on the computer like was done with Writer?Can I convert MSG to HTML on Mac?
Yes, you can convert MSG files to HTML on a Mac using software Aryson MSG file Converter. This tool allows you to convert Outlook MSG emails into multiple formats, including HTML, PDF, EML, PST, and more. It preserves all email content, attachments, metadata,Duplicate Leads Notification Help!
Hello! I have several web forms that have a duplicate lead notification that are being sent to the creator of the web form. I understand how to change the form entry notification, but I am specifically looking to change the recipient of the "Duplicate Lead" notification. Any help you can offer is greatly appreciated! TIA, ~ JennNext Page