Kaizen #122 - Data Synchronization using Bulk Read and Notification APIs

Kaizen #122 - Data Synchronization using Bulk Read and Notification APIs

Hello everyone!
Welcome back to yet another insightful post in our Kaizen series.

Consider that your organization is involved in selling laptops. Your sales team uses Zoho CRM for their daily activities, managing Leads, Deals, and inventory modules, whereas your Accounts team relies on a legacy application. It is important to maintain data synchronization from Zoho CRM to the legacy system to ensure that the Accounts team always works with the latest information.
To ensure that this synchronization happens in near real-time, the legacy application uses Zoho CRM's Bulk Read API and Notification API.

In this article, we'll discuss how one-way data sync will be achieved for the "Leads" module.

Bulk Read API 

This asynchronous API facilitates the export of up to 200,000 records from a module in a single job. If there are more than 200,000 records, additional API calls are required to fetch all the records.

Notification API

This API allows you to subscribe to events such as creating, updating, and deleting a record in a module through a webhook URL. Each of these actions triggers a corresponding HTTP request to your webhook.

Flow Diagram

The above diagram represents the initial sync on how the legacy application syncs data from Zoho CRM.

Step 1: Sync Start

This is the starting point of the code for the data sync from Zoho CRM to the legacy application and keeping the data updated in near real-time.

Step 2: Subscribe to the channel via Notification API

First, subscribe to events (create, update, and delete) occurring in the Leads module through a channel created via the Notification API, and then trigger the Bulk Read API (Step 3) to synchronize the entire data in Zoho CRM.

Reason:

During the asynchronous execution of the Bulk Read API job, all records from the specific module are saved to a CSV file. Let's consider a scenario where a record that has already undergone processing by the Bulk Read API is updated through the web UI or any other source. This means that the data in the Bulk Read CSV may become outdated by the time the job concludes.

To address this issue, you can leverage the Notification API. You subscribe to a channel, actively monitoring any actions performed on records in that module. When an action occurs, you will receive a notification through the configured URL. Subsequently, upon receiving this notification, you can store the updated data in the legacy database, including the modified time. This ensures that only the most recent data is stored in the database.

Sample Request URL: {{api-domain}}/crm/v6/actions/watch
Request Method: POST

Sample Input
{
    "watch": [
        {
            "channel_id": "10000",
            "events": [
                "Leads.create",
                "Leads.edit",
                "Leads.delete"
            ],
            "channel_expiry": "2024-01-21T00:13:59-08:00",
            "token": "leads.all.notif",
        }
    ]
}

Sample Response
{
    "watch": [
        {
            "code": "SUCCESS",
            "details": {
                "events": [
                    {
                        "channel_expiry": "2024-01-21T00:13:59-08:00",
                        "resource_uri": "https://www.zohoapis.com/crm/v6/Leads",
                        "resource_id": "5725767000000002175",
                        "resource_name": "Leads",
                        "channel_id": "10000"
                    }
                ]
            },
            "message": "Successfully subscribed for actions-watch of the given module",
            "status": "success"
        }
    ]
}

The response confirms a successful subscription for actions-watch on the "Leads" module. For more information about the Notification API, refer to this Kaizen post on the Notification API.

Step 2.1: On each notification, update that record in DB along with the modified time

If a record gets updated or a new record is created, or a record is deleted in the Leads module, you are notified about the change in data via the subscribed notification channel and stored in the legacy database along with the modified time.

Sample JSON Notification Response 

Note: The primary purpose of subscribing to the channel via notification is not to miss any data that has been modified or created during the data backup.

Step 3: Bulk Read Initialize

Initiate the data backup by using the Bulk Read API after subscribing to the channel via Notification API.  As the API is an asynchronous one, a bulk read job is scheduled. After the job is completed on the Zoho CRM end, a notification will be sent via the callback URL. Also, the application can periodically check the job status using the Get the Status of the Bulk Read Job API. 
Note: For the Bulk Read API, the records will be sorted based on the id field in ascending order.

Sample Request for Initial Bulk Read

Request URL:  {{api-domain}}/crm/bulk/v6/read
Request Method:  POST

Sample Input
{
    "callback": {
        "url": "https://www.xyz.com/callback",
        "method": "post"
    },
    "query": {
        "module": {
            "api_name": "Leads",
            "page" : 1
        }
    }
}

Sample Response
{
    "data": [
        {
            "status": "success",
            "code": "ADDED_SUCCESSFULLY",
            "message": "Added successfully",
            "details": {
                "id": "5725767000002002008", //job_id
                "operation": "read",
                "state": "ADDED",
                "created_by": {
                    "id": "5725767000000411001",
                    "name": "Patricia Boyle"
                },
                "created_time": "2024-01-20T04:01:48-08:00"
            }
        }
    ],
    "info": {}
}
For more details and examples, refer to the Create Bulk Read Job API help document. 

Get the Status of Scheduled Bulk Read Job

Check the status of the scheduled bulk read job using the job_id you received.
Request URL: {{api-domain}}/crm/bulk/v6/read/{job_id}
Request Method: GET

Sample Response
{
    "data": [
        {
            "id": "5725767000002025007",
            "operation": "read",
            "state": "COMPLETED",
            "result": {
                "page": 1,
                "per_page": 200000,
                "count": 142,
                "download_url": "/crm/bulk/v6/read/5725767000002025007/result",
                "more_records": false
            },
            "query": {
                "module": {
                    "id": "5725767000000002175",
                    "api_name": "Leads"
                },
                "page": 1
            },
            "created_by": {
                "id": "5725767000000411001",
                "name": "Patricia Boyle"
            },
            "created_time": "2024-01-24T21:35:11-08:00",
            "file_type": "csv"
        }
    ]
}

The above response contains the status of the scheduled job as either ADDED, IN PROGRESS, or COMPLETED.
When the job is complete, the response contains the result JSON object with the keys page, count, more_records, and download_url. The download_url in the callback response from which you can download the zip file containing the CSV file.

Step 4: Sync the Bulk Read data with the legacy DB, if it has a more recent modified time

Once the bulk-read data CSV is available, the data will be updated in the database. If any record's modified time in the CSV file is less than the modified time already present in the database, then it's an outdated record and need not be updated in the legacy's database, as it has already been handled via the Notification API's subscribed channel.

If the CSV contains exactly 200,000 records, there is a possibility of having more records. In the response of the Status of the Bulk Read Job API, if the more_records key is true, it indicates additional records to be exported. To retrieve them, simply update the value of the page key in the POST request and schedule another bulk read job to fetch the next set of records. By default, you can fetch up to 200,000 records in a single API call. Repeat this process until all the records are fetched from the CRM side. For more details on how to download, refer to this Kaizen post about the Bulk Read API.

Channel Resubscription

The channel subscription will remain active for a maximum of one day. After this period, it needs to be re-subscribed using the Enable Notification API to continue receiving notifications for create/update/delete actions in the Leads module. It is recommended to perform the re-subscription every 23 hours and 55 minutes, just short of 24 hours. Note that if the expiry time is not specified during the subscription, it will expire within one hour.

Troubleshooting

There are some scenarios where data synchronization may fail. 
For instance,
1. Unreachable Webhook URL: If the webhook URL is down, Zoho CRM cannot notify data updates through it.
2. Notification Expiry : If the notification expires before resubscribing to the channel, there is a risk of losing newly registered leads in between.
3. The code logic in the Webhook URL may break due to unforeseen reasons.
To handle such scenarios, store the last successful data sync time. Use this stored time as the modified time criteria in the bulk-read API to fetch any missed data and update the database.

We trust that this post meets your needs and is helpful. Let us know your thoughts in the comment section or reach out to us at support@zohocrm.com
Stay tuned for more insights in our upcoming Kaizen posts!
------------------------------------------------------------------------------------------------------------------------------
Previous Kaizen Post : Kaizen #121 - Customize List Views using Client Script
-------------------------------------------------------------------------------------------------------------------------------

Related Links





    • Sticky Posts

    • Kaizen #198: Using Client Script for Custom Validation in Blueprint

      Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Kaizen #226: Using ZRC in Client Script

      Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this

    • Kaizen #222 - Client Script Support for Notes Related List

      Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how

    • Kaizen #217 - Actions APIs : Tasks

      Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.

    • Kaizen #216 - Actions APIs : Email Notifications

      Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are

    • Recent Topics

    • Quotes Approval

      Hey all, Could you please help in the following: When creating quotes, how to configure it in a way, that its approval would work according to the quoted items description, not according to quote information. In my case, the quote should be sent to approval

    • Mandatory Field - but only at conversion

      Hello! We use Zoho CRM and there are times where the "Lead Created Date & Time" field isn't populated into a "Contractor" (Account is the default phrase i believe). Most of my lead tracking is based on reading the Lead Created field above, so it's important

    • Different Task Layouts for Subtasks

      I was wondering how it would be possible for a subtask to have a different task layout to the parent task.

    • Enable Free External Collaboration on Notecards in Zoho Notebook

      Hi Zoho Notebook Team, I would like to suggest a feature enhancement regarding external collaboration in Zoho Notebook. Currently, we can share notes with external users, and they are able to view the content without any issue. However, when these external

    • Using data fields in Zoho Show presentations to extract key numbers from Zia insights based on a report created

      Is it possible to use data fields in Zoho Show presentations along with Zoho Analytics to extract key numbers from Zia insights based on a report created? For example, using this text below: (note that the numbers in bold would be from Zia Insights) Revenue

    • Free webinar: AI-powered agreement management with Zoho Sign

      Hi there! Does preparing an agreement feel like more work than actually signing it? You're definitely not alone. Between drafting the document, managing revisions, securing internal approvals, and rereading clauses to make sure everything still reflects

    • WhatsApp Channels in Zoho Campaigns

      Now that Meta has opened WhatsApp Channels globally, will you add it to Zoho Campaigns? It's another top channel for marketing communications as email and SMS. Thanks.

    • CRM For Everyone - Bring Back Settings Tile View

      I've been using CRM for Everyone since it was in early access and I just can't stand the single list settings menu down the left-hand side. It takes so much longer to find the setting I need. Please give users the option to make the old sytle tile view

    • Lets have Dynamics 365 integration with Zohobooks

      Lets have Dynamics 365 integration with Zohobooks

    • Add notes in spreadsheet view

      It would be great if we could Add/edit notes in the spreadsheet view of contacts/leads. This would enable my sales teams to greatly increase their number of calls. Also viewing the most recent note in the Contact module would also be helpful.

    • Opening balances - Accounts Receivable and Payable

      Our accounting year starts on 1st August 2013 and I have a Trial Balance as at that date, including Accounts Receivableand Accounts Payable balances, broken down by each customer and supplier. Q1 - do I show my opening balance date as 31st July 2013 or

    • Announcing Kiosk 1.1 - Customize screen titles, configure new fields & actions, use values from your Kiosk to update fields, and more.

      Hello all We are back again with more enhancements to Kiosk. So what's new? Enhancements made to the Components Add titles for your Kiosk screens and adjust its width to suit your viewing preferences. Three new fields can be added to your screen: Percentage,

    • Limited System because of Limited Number of Fields for Car Dealership

      Dear Zoho Support, we want to have all the information about a car inside of a car record. We want to have Zoho CRM as our single source of truth for our data, but the limited number of fields are not allowing that. The data consist of: technical data

    • Cancel Subscription

      Hi , Im want to cancel my account but without success please help me to do it

    • Making an email campaign into a Template

      I used a Zoho Campaign Template to create an email. Now I want to use this email and make it a new template, but this seems to be not possible. Am I missing something?

    • Direct Access and Better Search for Zoho Quartz Recordings

      Hi Zoho Team, We would like to request a few enhancements to improve how Zoho Quartz recordings are accessed and managed after being submitted to Zoho Support. Current Limitation: After submitting a Quartz recording, the related Zoho Support ticket displays

    • Multiple Cover Letters

      We are using the staffing firm edition of Recruit and we have noticed that candidates cannot add more than one cover letter. This is a problem as they might be applying for multiple jobs on our career site and when we submit their application to a client,

    • URGENT: Deluge issue with Arabic text Inbox

      Dear Deluge Support, We are facing an issue that started on 12/Feb/2026 with custom functions written using Deluge within Qntrl platform. Currently, custom functions do not accept Arabic content; it is replaced with (???) characters. Scenario 1: If we

    • File Conversion from PDF to JPG/PNG

      Hi, I have a question did  anyone every tried using custom function to convert a PDF file to JPG/PNG format? Any possibility by using the custom function to achieve this within zoho apps.  I do know there are many third parties API provide this with

    • Prevent accidental duplicate entry of Customer Ordersome

      Zoho Support has confirmed that Zoho currently does not have any method (using Deluge, flow or any other method) to alert a user when a sales order has been entered twice using the same customer reference number (i.e. a duplicate). Most ERP platforms

    • Now in Zoho One: Orchestrate customer journeys across apps with Zoho CommandCenter

      Hello Zoho One Community! We’re excited to introduce Zoho CommandCenter as a new capability available in Zoho One. For the whole customer journey As Zoho One customers adopt more apps across sales, marketing, finance, and support, a common challenge emerges:

    • annualy customer report

      we need a report per customer that looks like this invoic number cleaning laundry repair management 01 january xxx euro xx euro xx euro xxx euro 02 february xxx euro xxx euro x euro xxxx euro and so on the years 12 months is that possible to make and

    • Totals for Sales Tax Report

      On the sales tax report, the column totals aren't shown for any column other than Total Tax. I can't think of a good reason that they shouldn't be included for the other columns, as well. It would help me with my returns, for sure. It seems ludicrous

    • Free Webinar: Zoho Sign for Zoho Projects: Automate tasks and approvals with e-signatures

      Hi there! Handling multiple projects at once? Zoho Projects is your solution for automated and streamlined project management, and with the Zoho Sign extension, you can sign, send, and manage digital paperwork directly from your project workspace. Join

    • Exported Report File Name

      Hi, We often export reports for information. It is time consuming to rename all the reports we export on a weekly basis, as when exported their default name is a seemingly random string of numbers. These numbers may be important, I'm not sure, but I am

    • Automatic Refresh on Page?

      Hi everyone, We use a page as a dashboard which shows data for the laboratory and tasks pending etc. Is there a way to set the page to automatically refresh on a X time? Many thanks TOG

    • Full Context of Zoho CRM Records for Zia in Zoho Desk for efficient AI Usage

      Hello everyone, I have a question regarding the use of Zia in Zoho Desk in combination with CRM data. Is it possible to automatically feed the complete context of a CRM record into Zia, so that it can generate automated and highly accurate responses for

    • Hiding Pre-defined Views

      You can enhance Zoho with custom views - but you cannot hide the pre-defined views. Most users focus on 4 or 5 views. Right now for EVERY user EVERY time they want to move to one of their 4 or 5 views - they have to scroll down past a long list of pre-defined views - we don't use ANY of the pre-defined views. Adding the feature to allow a predefined view to be hidden completely or only visible to certain users would be a big improvement in usability for Zoho CRM. This feature is already available

    • [Webinar] Top 10 Most Used Zoho Analytics Features in 2025

      Zoho Analytics has evolved significantly over the past year. Discover the most widely adopted features in Zoho Analytics in 2025, based on real customer usage patterns, best practices, and high-impact use cases. Learn how leading teams are turning data

    • How do I edit the Calendar Invite notifications for Interviews in Recruit?

      I'm setting up the Zoho Recruit Interview Calendar system but there's some notifications I don't have any control over. I've turned off all Workflows and Automations related to the Calendar Scheduling and it seems that it's the notification that is sent

    • Add RTL and Hebrew Support for Candidate Portal (and Other Zoho Recruit Portals)

      Dear Zoho Recruit Team, I hope you're doing well. We would like to request the ability to set the Candidate Portal to be Right-to-Left (RTL) and in Hebrew, similar to the existing functionality for the Career Site. Currently, when we set the Career Site

    • App for Mac OS X please!

      It would be awesome to have a mail app for Mac OS X that included all the cool features such as steams, calendar, tasks, contacts, etc. Most people prefer native apps, rather than running it through a web browser. I know that we can use the IMAP, CalDAV,

    • How to integrate Zoho CRM, Zoho Forms and a WIX Web Site

      Attached video demonstrates how to use Zoho Forms included in Zoho One, to design a Contact Us form to be embedded into a WIX web site and integrated into Zoho CRM.

    • CRM x WorkDrive: File storage for new CRM signups is now powered by WorkDrive

      Availability Editions: All DCs: All Release plan: Released for new signups in all DCs. It will be enabled for existing users in a phased manner in the upcoming months. Help documentation: Documents in Zoho CRM Manage folders in Documents tab Manage files

    • Introducing Workqueue: your all-in-one view to manage daily work

      Hello all, We’re excited to introduce a major productivity boost to your CRM experience: Workqueue, a dynamic, all-in-one workspace that brings every important sales activity, approval, and follow-up right to your fingertips. What is Workqueue? Sales

    • Default Reminder Time in New Tasks or New Event?

      Any way to change this from 1:00am? Thanks, Gary Moderation Update (February 2026): With the Calendar preferences, the default reminder time for Meetings, Appointments and All-Day Meetings can be set. Read more: Calendar preferences in Zoho CRM Regarding

    • Exporting All Custom Functions in ZohoCRM

      Hello, All I've been looking for a way to keep about 30 functions that I have written in Zoho CRM updated in my own repository to use elsewhere in other instances. A github integration would be great, but a way to export all custom functions or any way

    • Dynamic Field Folders in OneDrive

      Hi, With the 2 options today we have either a Dynamic Parent Folder and lots of attachments all in that one folder with only the ability to set the file name (Which is also not incremented so if I upload 5 photos to one field they are all named the same

    • Right Shift key not sending to Zoho Assist environments

      I'm unable to use Right Shift key in Zoho environments. Zoho environments are Win10. Computer I access from is Win 11. Issue started when I changed to Win 11. Have tried: - Multiple browsers - web client AND desktop client - 3rd party mapping tools to

    • Unable to Filter Retail Sales Orders & Need Paid/Unpaid Filter – Zoho Books

      Hi Zoho Team, Recently you introduced Retail – Standard and Retail – Premium templates for Sales Orders. However, in the Sales Order module we still cannot filter or segregate Retail Sales Orders separately from normal B2B sales orders. Because of this,

    • Next Page