Data Synchronization using Bulk Read and Notification APIs
Hello everyone!
Welcome back to yet another insightful post in our Kaizen series.
Welcome back to yet another insightful post in our Kaizen series.
Consider that your organization is involved in selling laptops. Your sales team uses Zoho CRM for their daily activities, managing Leads, Deals, and inventory modules, whereas your Accounts team relies on a legacy application. It is important to maintain data synchronization from Zoho CRM to the legacy system to ensure that the Accounts team always works with the latest information.
To ensure that this synchronization happens in near real-time, the legacy application uses Zoho CRM's Bulk Read API and Notification API.
In this article, we'll discuss how one-way data sync will be achieved for the "Leads" module.
Bulk Read API
This asynchronous API facilitates the export of up to 200,000 records from a module in a single job. If there are more than 200,000 records, additional API calls are required to fetch all the records.
Notification API
This API allows you to subscribe to events such as creating, updating, and deleting a record in a module through a webhook URL. Each of these actions triggers a corresponding HTTP request to your webhook.
Flow Diagram
The above diagram represents the initial sync on how the legacy application syncs data from Zoho CRM.
Step 1: Sync Start
This is the starting point of the code for the data sync from Zoho CRM to the legacy application and keeping the data updated in near real-time.
Step 2: Subscribe to the channel via Notification API
First, subscribe to events (create, update, and delete) occurring in the Leads module through a channel created via the Notification API, and then trigger the Bulk Read API (Step 3) to synchronize the entire data in Zoho CRM.
Reason:
During the asynchronous execution of the Bulk Read API job, all records from the specific module are saved to a CSV file. Let's consider a scenario where a record that has already undergone processing by the Bulk Read API is updated through the web UI or any other source. This means that the data in the Bulk Read CSV may become outdated by the time the job concludes.
To address this issue, you can leverage the Notification API. You subscribe to a channel, actively monitoring any actions performed on records in that module. When an action occurs, you will receive a notification through the configured URL. Subsequently, upon receiving this notification, you can store the updated data in the legacy database, including the modified time. This ensures that only the most recent data is stored in the database.
Sample Request URL: {{api-domain}}/crm/v6/actions/watch
Request Method: POST
Sample Input
{ "watch": [ { "channel_id": "10000", "events": [ "Leads.create", "Leads.edit", "Leads.delete" ], "channel_expiry": "2024-01-21T00:13:59-08:00", "token": "leads.all.notif", "notify_url": "https://webhook.site/6c3d08c2-2c21-4d16-9134-712acb513915" } ] } |
Sample Response
{ "watch": [ { "code": "SUCCESS", "details": { "events": [ { "channel_expiry": "2024-01-21T00:13:59-08:00", "resource_uri": "https://www.zohoapis.com/crm/v6/Leads", "resource_id": "5725767000000002175", "resource_name": "Leads", "channel_id": "10000" } ] }, "message": "Successfully subscribed for actions-watch of the given module", "status": "success" } ] } |
The response confirms a successful subscription for actions-watch on the "Leads" module. For more information about the Notification API, refer to this Kaizen post on the Notification API.
Step 2.1: On each notification, update that record in DB along with the modified time
If a record gets updated or a new record is created, or a record is deleted in the Leads module, you are notified about the change in data via the subscribed notification channel and stored in the legacy database along with the modified time.
Sample JSON Notification Response
Note: The primary purpose of subscribing to the channel via notification is not to miss any data that has been modified or created during the data backup.
Step 3: Bulk Read Initialize
Initiate the data backup by using the Bulk Read API after subscribing to the channel via Notification API. As the API is an asynchronous one, a bulk read job is scheduled. After the job is completed on the Zoho CRM end, a notification will be sent via the callback URL. Also, the application can periodically check the job status using the Get the Status of the Bulk Read Job API.
Note: For the Bulk Read API, the records will be sorted based on the id field in ascending order.
Sample Request for Initial Bulk Read
Request URL: {{api-domain}}/crm/bulk/v6/read
Request Method: POST
Sample Input
{ "callback": { "url": "https://www.xyz.com/callback", "method": "post" }, "query": { "module": { "api_name": "Leads", "page" : 1 } } } |
Sample Response
{ "data": [ { "status": "success", "code": "ADDED_SUCCESSFULLY", "message": "Added successfully", "details": { "id": "5725767000002002008", //job_id "operation": "read", "state": "ADDED", "created_by": { "id": "5725767000000411001", "name": "Patricia Boyle" }, "created_time": "2024-01-20T04:01:48-08:00" } } ], "info": {} } |
Get the Status of Scheduled Bulk Read Job
Check the status of the scheduled bulk read job using the job_id you received.
Request URL: {{api-domain}}/crm/bulk/v6/read/{job_id}
Request Method: GET
Sample Response
{ "data": [ { "id": "5725767000002025007", "operation": "read", "state": "COMPLETED", "result": { "page": 1, "per_page": 200000, "count": 142, "download_url": "/crm/bulk/v6/read/5725767000002025007/result", "more_records": false }, "query": { "module": { "id": "5725767000000002175", "api_name": "Leads" }, "page": 1 }, "created_by": { "id": "5725767000000411001", "name": "Patricia Boyle" }, "created_time": "2024-01-24T21:35:11-08:00", "file_type": "csv" } ] } |
The above response contains the status of the scheduled job as either ADDED, IN PROGRESS, or COMPLETED.
When the job is complete, the response contains the result JSON object with the keys page, count, more_records, and download_url. The download_url in the callback response from which you can download the zip file containing the CSV file.
Step 4: Sync the Bulk Read data with the legacy DB, if it has a more recent modified time
Once the bulk-read data CSV is available, the data will be updated in the database. If any record's modified time in the CSV file is less than the modified time already present in the database, then it's an outdated record and need not be updated in the legacy's database, as it has already been handled via the Notification API's subscribed channel.
If the CSV contains exactly 200,000 records, there is a possibility of having more records. In the response of the Status of the Bulk Read Job API, if the more_records key is true, it indicates additional records to be exported. To retrieve them, simply update the value of the page key in the POST request and schedule another bulk read job to fetch the next set of records. By default, you can fetch up to 200,000 records in a single API call. Repeat this process until all the records are fetched from the CRM side. For more details on how to download, refer to this Kaizen post about the Bulk Read API.
Channel Resubscription
The channel subscription will remain active for a maximum of one day. After this period, it needs to be re-subscribed using the Enable Notification API to continue receiving notifications for create/update/delete actions in the Leads module. It is recommended to perform the re-subscription every 23 hours and 55 minutes, just short of 24 hours. Note that if the expiry time is not specified during the subscription, it will expire within one hour.
Troubleshooting
There are some scenarios where data synchronization may fail.
For instance,
1. Unreachable Webhook URL: If the webhook URL is down, Zoho CRM cannot notify data updates through it.
2. Notification Expiry : If the notification expires before resubscribing to the channel, there is a risk of losing newly registered leads in between.
3. The code logic in the Webhook URL may break due to unforeseen reasons.
To handle such scenarios, store the last successful data sync time. Use this stored time as the modified time criteria in the bulk-read API to fetch any missed data and update the database.
We trust that this post meets your needs and is helpful. Let us know your thoughts in the comment section or reach out to us at support@zohocrm.com
Stay tuned for more insights in our upcoming Kaizen posts!
------------------------------------------------------------------------------------------------------------------------------
Previous Kaizen Post : Kaizen #121 - Customize List Views using Client Script
-------------------------------------------------------------------------------------------------------------------------------
Related Links
- API Help Documentations :
- Kaizen Posts :
- Kaizen Series
Access your files securely from anywhere
New to Zoho Recruit?
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Subramanian K
Sticky Posts
Kaizen #197: Frequently Asked Questions on GraphQL APIs
🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Kaizen #198: Using Client Script for Custom Validation in Blueprint
Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Celebrating 200 posts of Kaizen! Share your ideas for the milestone post
Hello Developers, We launched the Kaizen series in 2019 to share helpful content to support your Zoho CRM development journey. Staying true to its spirit—Kaizen Series: Continuous Improvement for Developer Experience—we've shared everything from FAQsKaizen #193: Creating different fields in Zoho CRM through API
🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! Commands
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
User marked as SPAMMER. Mail Fetch has also been disabled for any active POP accounts.
I am the administrator for joelles.com One of our accounts has been blocked saying this: User marked as SPAMMER. Mail Fetch has also been disabled for any active POP accounts. I cannot change the disabled account in the control panel as it says that itMy domain did not activate
Hi, my domain (apsaindustrial.com.ar) did not activate, and the phone verification message never arrived. Please would you solve this problem? Thanks.ME SALE ESTE ERROR: No fue posible enviar el mensaje;Motivo:554 5.1.8 Email Outgoing Blocked
Ayuda!! Me sale este error al intentar enviar mensajes desde mi correo electronico de Zoho! Tampoco recibo correos pues cuando me envia rebotan. Ayuda, Me urge enviar unos correo importantes!! Quedo atenta MAGDA HERNANDEZ +5731120888408Search Records returning different values than actually present
Hey! I have this following line in my deluge script: accountSearch = zoho.crm.searchRecords("Accounts","(RS_Enroll_ID:equals:" + rsid + ")",1,200,{"cvid":864868001088693817}); info "Account search size: " + accountSearch.size(); listOfAccounts = zoho.crm.searchRecords("Accounts","(RS_Enroll_ID:equals:"Is there a way to sync Tags between CRM and Campaigns/Marketing Hub?
I wonder if there is a way to synch the tags between CRM and Marketing-Hub / Campaigns?how to see if a specific contact opened an email in zoho campaign?
how to see if a specific contact opened an email in zoho campaign?Rich Text For Notes in Zoho CRM
Hello everyone, As you know, notes are essential for recording information and ensuring smooth communication across your records. With our latest update, you can now use Rich Text formatting to organize and structure your notes more efficiently. By usingRevenue Management: #8 Revenue Recognition in Educational & Training Institutions
Educational Institutions and training centres typically collect course fees at the time of enrolment, sometimes for a one-day workshop and sometimes for a year-long certification course. You might also charge separately for course materials or evaluation.How to Customize Task Creation to Send a Custom Alert Using JavaScript in Zoho CRM?
Hello Zoho CRM Community, I’m looking to customize Zoho CRM to send a custom alert whenever a task is created. I understand that Zoho CRM supports client scripts using JavaScript, and I would like to leverage this feature to implement the alert functionality.how to use validation rules in subform
Is it possible to use validation rules for subforms? I tried the following code: entityMap = crmAPIRequest.toMap().get("record"); sum = 0; direct_billing = entityMap.get("direct_billing_details"); response = Map(); for each i in direct_billing { if(i.get("type")Using files from Zoho CRM in Gemini/ChatGPT/Claude
Hi all, I’ve got subscriptions to Gemini and a few other AI tools which I use for tasks like data enrichment, email composition, etc. In our workflow, we often receive various documents from clients — such as process workflows, BRDs/requirement documentsEnhancements to the formula field in Zoho CRM: Auto-refresh formulas with the "Now" function, stop formula executions based on criteria, and include formulas within formulas
Dear Customers, We hope you're well! By their nature, modern businesses rely every day on computations, whether it's to calculate the price of a product, assess ROI, evaluate the lifetime value of a customer, or even determine the age of a record. WithMaximum file limit in zoho people LMS
Dear Team, I am having approximately 4.9 GB of material, including PPTs and videos for uploading in zoho people LMS course. May I know what is the maximum limit limit for the course files Thanking you, With regards, Logeswar V Executive _ OperationsDisplaying related quotes in sales order and back
Hi, My colleague liked to see to which sales orders, the quote has been converted. Quote shows Invoices, but not SO. Same, they would like to see the quotes in the sales order, as they can see invoices, packages, shipment, How can we achieve this ? Thankautomations: Can I execute a step on a specific date?
I have created a form in Zoho forms, and created a contacts list. I have also begun setting up an automation with the intention of sending the form to the contact list on a specific date every month (via email) for the entire year (essentially sendingDynamically prefill ticket fields
Hello, I am using Zoho Desk to collect tickets of our clients about orders they placed on our website. I would like to be able to prefill two tickets fields dynamically, in this case a readonly field for the order id, and a hidden field for the sellerOptimize your Knowledge Base for better visibility by allowing search engine crawling and indexing
All you need to know about no-follow and no-index in KB. What are article crawlability and indexability? Crawlability and indexability are vital for making an article visible and accessible to search engines. When a search engine crawls an article, itHas anyone created a public ASAP Guide that I can check out?
I am thinking of adding an ASAP guide to my web application, but I have noticed that the ASAP widget itself can be really slow to load sometimes. Has anyone created a public ASAP Guide that I can check to see how performant it is? I don't want to spendZoho Desk Partners with Microsoft's M365 Copilot for seamless customer service experiences
Hello Zoho Desk users, We are happy to announce that Zoho Desk has partnered with Microsoft's M365 to empower customer service teams with enhanced capabilities and seamless experiences for agents. Microsoft announced their partnership during their keynoteWhat’s New in Zoho Analytics – September 2025
Hello Users!! In this month’s update, we’re raising the bar across multiple touchpoints, from how you bring in data, plan and track projects to how you design and brand your dashboards. We’ve added the all-new Gantt chart for project visualization, expandedZoho MCP has no tools for Creator or 3rd Party Apps?
I don't see a Zoho MCP community forum so putting this here. Two big problems I see: 1) Although Zoho advertises "over 950 3rd party apps" as available through their MCP, when I go to "Add Tools" there are ZERO 3rd party apps available to choose from.Systematic SPF alignment issues with Zoho subdomains
Analysis Period: August 19 - September 1, 2025 PROBLEM SUMMARY Multiple Zoho services are causing systematic SPF authentication failures in DMARC reports from major email providers (Google, Microsoft, Zoho). While emails are successfully delivered dueZoho Forms - Zoho Drive connection - Shared Drives not supported
Hello i am stuck with Google Drive Connection There is no supported shared drives Connection is not support shared drives boolean Query Parameters - supportsAllDrives=true&supportsTeamDrives=true to activate fetch files from the shared drives. Ahat needApply Advance option not shown in report
We are facing an issue in Zoho Expenses. While approving an Expense Report, the "Apply Advance" option is not appearing under the three dots (More Options). Details: Module: Expense Reports Issue: "Apply Advance" option not visible Status of Report: AwaitingCan't create package until Bill created?
I can't understand why we cannot create a package until a Bill is created? We are having to created draft Bills to create a package when the item is received, but we may not have received a Bill from the supplier. Also, Bill # is required, but we normallyWhats the Time out Limit for API Calls from Deluge?
Hi Creator Devs, We are making API calls to third party server via Deluge. Getting this error message: Error at line : 24, The task has been terminated since the API call is taking too long to respond. Please try again after sometime. Whats the defaultCommunity Digest Agosto 2025 - Todas las novedades en Español Zoho Community
¡Hola, Zoho Community! Agosto llega a su fin y septiembre nos trae aire fresco a la comunidad: más inteligencia con IA, actualizaciones que elevan la productividad y la recta final hacia Zoholics España 2025. Aquí tienes lo más destacado del mes paraTip #3: How to change your booking page language
Displaying your booking page in your target audience's language can greatly increase customer satisfaction. By speaking their language, you will help customers feel more comfortable scheduling with you and create a stronger connection with them. Let'sHow can I optimize a Zoho Site page for SEO when embedding external menu or restaurant links?
Hi everyone, I’m experimenting with building small content hubs on Zoho Sites and want to make sure I’m doing it in an SEO-friendly way. For example, I tried creating a page that highlights restaurant menu items and linked out to a resource like this:Diff signature for compose new email and replies
Hi, How do i have different signature for replies and new emails. its inconvenient to have one large signature for replies. Usually on Outlook we have the option to keep separate signatures for new emails and for replies.Zoho Website Site Speed Up & Setting
We are experiencing slow loading speeds on our Zoho website and would like assistance in optimizing its performance. Kindly review the site and suggest or implement necessary improvements to enhance speed, especially related to: > Caching mechanisms >Clickjacking: Zoho Vault's Response
Issue Password manager browser extensions are found vulnerable to clickjacking security vulnerabilities that could lead attackers to steal account credentials, TFA codes, card details, under certain conditions. Reported by Marek Toth, Independent SecurityNo option for pick up in Zoho Books / Inventory but yes on commerce
Is it planned to release soon on books/inventory?Archiving Contacts
How do I archive a list of contacts, or individual contacts?Zoho Books - France
L’équipe de Zoho France reçoit régulièrement des questions sur la conformité de ses applications de finances (Zoho Books/ Zoho Invoice) pour le marché français. Voici quelques points pour clarifier la question : Zoho Books est un logiciel de comptabilitéBlueprint Issue - Being able to set a subform field as mandatory
I'm creating a blueprint. My record involves a subform which is only shown once field is set but the field gets set in step two of the process. My problem - I can't save the record as the subform field is set to mandatory - If I unset the mandatory field,Blueprint - Mandatory file upload field
Hi, File upload (as we as image upload) field cannot be set as mandatory during a blueprint transition. Is there a workaround? Setting attachments as mandatory doesn't solve this need as we have no control over which attachments are added, nor can weZoho Books - Include Quote Status in Workflow Field Triggers
Hi Zoho Books team, I recently tried to create a Workflow rule based on when a Quote is Accepted by the customer. This is something which I thought would be very easy to do, however I discovered that Status is not listed as a field which can be monitoredZoho Books - Show Related Sales Orders on Quotes
Hi Books team, I've noticed that the Quotes don't show show the related Sales Order. My feature request is to also show related Sales Orders above the Quote so it's easy to follow the thread of records in the sales and fulfilment process. Below screenshotAdd VAT/Tax line to bank adjustments
When categorising transactions and matching bank feeds with transactions such as customer payments, we use the "Add Adjustment" to add things like fees/bank fees. It would be useful to choose a VAT/Tax rate here. Whilst there is a bank charges option when adding a payment, this goes into the default bank charges account. We use the adjustments so that we can choose the account and separate our fees. We use different card providers and Worldpay charges VAT so we are stuck. We cannot integrate withNext Page