Kaizen #122 - Data Synchronization using Bulk Read and Notification APIs

Kaizen #122 - Data Synchronization using Bulk Read and Notification APIs

Hello everyone!
Welcome back to yet another insightful post in our Kaizen series.

Consider that your organization is involved in selling laptops. Your sales team uses Zoho CRM for their daily activities, managing Leads, Deals, and inventory modules, whereas your Accounts team relies on a legacy application. It is important to maintain data synchronization from Zoho CRM to the legacy system to ensure that the Accounts team always works with the latest information.
To ensure that this synchronization happens in near real-time, the legacy application uses Zoho CRM's Bulk Read API and Notification API.

In this article, we'll discuss how one-way data sync will be achieved for the "Leads" module.

Bulk Read API 

This asynchronous API facilitates the export of up to 200,000 records from a module in a single job. If there are more than 200,000 records, additional API calls are required to fetch all the records.

Notification API

This API allows you to subscribe to events such as creating, updating, and deleting a record in a module through a webhook URL. Each of these actions triggers a corresponding HTTP request to your webhook.

Flow Diagram

The above diagram represents the initial sync on how the legacy application syncs data from Zoho CRM.

Step 1: Sync Start

This is the starting point of the code for the data sync from Zoho CRM to the legacy application and keeping the data updated in near real-time.

Step 2: Subscribe to the channel via Notification API

First, subscribe to events (create, update, and delete) occurring in the Leads module through a channel created via the Notification API, and then trigger the Bulk Read API (Step 3) to synchronize the entire data in Zoho CRM.

Reason:

During the asynchronous execution of the Bulk Read API job, all records from the specific module are saved to a CSV file. Let's consider a scenario where a record that has already undergone processing by the Bulk Read API is updated through the web UI or any other source. This means that the data in the Bulk Read CSV may become outdated by the time the job concludes.

To address this issue, you can leverage the Notification API. You subscribe to a channel, actively monitoring any actions performed on records in that module. When an action occurs, you will receive a notification through the configured URL. Subsequently, upon receiving this notification, you can store the updated data in the legacy database, including the modified time. This ensures that only the most recent data is stored in the database.

Sample Request URL: {{api-domain}}/crm/v6/actions/watch
Request Method: POST

Sample Input
{
    "watch": [
        {
            "channel_id": "10000",
            "events": [
                "Leads.create",
                "Leads.edit",
                "Leads.delete"
            ],
            "channel_expiry": "2024-01-21T00:13:59-08:00",
            "token": "leads.all.notif",
        }
    ]
}

Sample Response
{
    "watch": [
        {
            "code": "SUCCESS",
            "details": {
                "events": [
                    {
                        "channel_expiry": "2024-01-21T00:13:59-08:00",
                        "resource_uri": "https://www.zohoapis.com/crm/v6/Leads",
                        "resource_id": "5725767000000002175",
                        "resource_name": "Leads",
                        "channel_id": "10000"
                    }
                ]
            },
            "message": "Successfully subscribed for actions-watch of the given module",
            "status": "success"
        }
    ]
}

The response confirms a successful subscription for actions-watch on the "Leads" module. For more information about the Notification API, refer to this Kaizen post on the Notification API.

Step 2.1: On each notification, update that record in DB along with the modified time

If a record gets updated or a new record is created, or a record is deleted in the Leads module, you are notified about the change in data via the subscribed notification channel and stored in the legacy database along with the modified time.

Sample JSON Notification Response 

Note: The primary purpose of subscribing to the channel via notification is not to miss any data that has been modified or created during the data backup.

Step 3: Bulk Read Initialize

Initiate the data backup by using the Bulk Read API after subscribing to the channel via Notification API.  As the API is an asynchronous one, a bulk read job is scheduled. After the job is completed on the Zoho CRM end, a notification will be sent via the callback URL. Also, the application can periodically check the job status using the Get the Status of the Bulk Read Job API. 
Note: For the Bulk Read API, the records will be sorted based on the id field in ascending order.

Sample Request for Initial Bulk Read

Request URL:  {{api-domain}}/crm/bulk/v6/read
Request Method:  POST

Sample Input
{
    "callback": {
        "url": "https://www.xyz.com/callback",
        "method": "post"
    },
    "query": {
        "module": {
            "api_name": "Leads",
            "page" : 1
        }
    }
}

Sample Response
{
    "data": [
        {
            "status": "success",
            "code": "ADDED_SUCCESSFULLY",
            "message": "Added successfully",
            "details": {
                "id": "5725767000002002008", //job_id
                "operation": "read",
                "state": "ADDED",
                "created_by": {
                    "id": "5725767000000411001",
                    "name": "Patricia Boyle"
                },
                "created_time": "2024-01-20T04:01:48-08:00"
            }
        }
    ],
    "info": {}
}
For more details and examples, refer to the Create Bulk Read Job API help document. 

Get the Status of Scheduled Bulk Read Job

Check the status of the scheduled bulk read job using the job_id you received.
Request URL: {{api-domain}}/crm/bulk/v6/read/{job_id}
Request Method: GET

Sample Response
{
    "data": [
        {
            "id": "5725767000002025007",
            "operation": "read",
            "state": "COMPLETED",
            "result": {
                "page": 1,
                "per_page": 200000,
                "count": 142,
                "download_url": "/crm/bulk/v6/read/5725767000002025007/result",
                "more_records": false
            },
            "query": {
                "module": {
                    "id": "5725767000000002175",
                    "api_name": "Leads"
                },
                "page": 1
            },
            "created_by": {
                "id": "5725767000000411001",
                "name": "Patricia Boyle"
            },
            "created_time": "2024-01-24T21:35:11-08:00",
            "file_type": "csv"
        }
    ]
}

The above response contains the status of the scheduled job as either ADDED, IN PROGRESS, or COMPLETED.
When the job is complete, the response contains the result JSON object with the keys page, count, more_records, and download_url. The download_url in the callback response from which you can download the zip file containing the CSV file.

Step 4: Sync the Bulk Read data with the legacy DB, if it has a more recent modified time

Once the bulk-read data CSV is available, the data will be updated in the database. If any record's modified time in the CSV file is less than the modified time already present in the database, then it's an outdated record and need not be updated in the legacy's database, as it has already been handled via the Notification API's subscribed channel.

If the CSV contains exactly 200,000 records, there is a possibility of having more records. In the response of the Status of the Bulk Read Job API, if the more_records key is true, it indicates additional records to be exported. To retrieve them, simply update the value of the page key in the POST request and schedule another bulk read job to fetch the next set of records. By default, you can fetch up to 200,000 records in a single API call. Repeat this process until all the records are fetched from the CRM side. For more details on how to download, refer to this Kaizen post about the Bulk Read API.

Channel Resubscription

The channel subscription will remain active for a maximum of one day. After this period, it needs to be re-subscribed using the Enable Notification API to continue receiving notifications for create/update/delete actions in the Leads module. It is recommended to perform the re-subscription every 23 hours and 55 minutes, just short of 24 hours. Note that if the expiry time is not specified during the subscription, it will expire within one hour.

Troubleshooting

There are some scenarios where data synchronization may fail. 
For instance,
1. Unreachable Webhook URL: If the webhook URL is down, Zoho CRM cannot notify data updates through it.
2. Notification Expiry : If the notification expires before resubscribing to the channel, there is a risk of losing newly registered leads in between.
3. The code logic in the Webhook URL may break due to unforeseen reasons.
To handle such scenarios, store the last successful data sync time. Use this stored time as the modified time criteria in the bulk-read API to fetch any missed data and update the database.

We trust that this post meets your needs and is helpful. Let us know your thoughts in the comment section or reach out to us at support@zohocrm.com
Stay tuned for more insights in our upcoming Kaizen posts!
------------------------------------------------------------------------------------------------------------------------------
Previous Kaizen Post : Kaizen #121 - Customize List Views using Client Script
-------------------------------------------------------------------------------------------------------------------------------

Related Links





    • Recent Topics

    • Slow Performance on desk.zoho.com. 11/3/2025

      I’m not seeing any active service alerts for desk.zoho.com, but everyone on our account is currently experiencing very slow load times when opening or navigating tickets. We’ve already tried the standard troubleshooting steps — clearing cache and cookies,

    • Enhance Sign CRM integration

      Hello all, I'm working on a custom Deluge script to enhance the integration between Zoho CRM and Sign by using a writer merge template for additional flexibility. I want to replicate the post-sign document integration that exists between CRM and Sign

    • March 15, 2023: Zoho Docs is discontinued

      As of today (March 15, 2023) Zoho Docs is discontinued for all users. We would like to thank our customers for trusting us for so many years! Going forward, we're confident you'll enjoy using Zoho WorkDrive for all your advanced file management and collaboration

    • Create View to See Tickets Closed within the last 3 days

      I'm trying to create a view in Zoho Desk that shows me "recently closed ticket," which I will define as tickets closed in the last 3 days. I want this view to update so that whenever I click to view it is recalculates and shows me tickets closed within

    • Canvas View in Zoho Recruit

      Is it possible or would it be possible to have the new 'Canvas View' in Zoho Recruit?

    • Allow Multiple usage units to items while adding them to sales/purchase transactions

      The usage unit of items added in zoho books are static right now and can not be changed. But certain items are received or sold in multiple usage units. One example is fabric. It can be bought in Meters, inches, kgs or other units. Another example would

    • Will I Get a Refund If I Downgrade Zoho Mail?

      Hello, We upgraded an email account for our new employee. However, the employee left after one month, and now I've reduced the number of Zoho Mail users from 7 to 6. Can we get a refund for the remaining portion of our annual payment?

    • Introducing Profile Summary: Faster Candidate Insights with Zia

      We’re excited to launch Profile Summary, a powerful new feature in Zoho Recruit that transforms how you review candidate profiles. What used to take minutes of resume scanning can now be assessed in seconds—thanks to Zia. A Quick Example Say you’re hiring

    • How to copy value from a single line field into a picklist field within a module's subform?

      Hello there, I have a single line field in a module's subform. I would like the value in the field to automatically update a picklist field within the same subform (both have items with the same names). Is this possible via function? Unfortunately, workflows

    • Send Whatsapp message from Whatsapp template with custom variables

      Hi, I'm trying to do some basic integration for sending WhatsApp messages from Zoho CRM using Zoho Desk whatsapp templates. When creating new whatsapp template in Zoho Desk we can choose ticket related fields as variables but it's not clear how to use

    • Recurring Events Not Appearing in "My Events" and therefore not syncing with Google Apps

      We use the Google Sync functionality for our events, and it appears to have been working fine except: I've created a set of recurring events that I noticed were missing from my Google Apps calendar. Upon further research, it appears this is occurring

    • Dissociate account from contact

      Hello, When I make the call through the API to "Dissociate account from contact" using, /api/v1/contacts/**integer**/accounts/**integer**/dissociate I get this response back, 422Unprocessable Entity (WebDAV) (RFC 4918) { "errorCode": "INVALID_DATA", "message":

    • Zoho Desk and Zoho People Integration - is it possible in order to alert whether an agent is available

      Hi, We use Zoho People for our employees to log absence (Paid Time Off etc). I was wondering if it was possible that this information can be shared in to Zoho Desk, so that when allocating a ticket to an agent, if they are on leave there is an onscreen

    • Add Save button to Expense form

      A save button would be very helpful on the expense form. Currently there is a Save and Close button. When we want to itemize an expense, this option would be very helpful. For example, if we have a hotel expense that also has room service, which is a

    • Change Currency symbol

      I would like to change the way our currency displays when printed on quotes, invoices and purchase orders. Currently, we have Australian Dollars AUD as our Home Currency. The only two symbol choices available for this currency are "AU $" or "AUD". I would

    • Call transcrition working for ringcentral?

      I don't see anything about what telephony providers can be used. The Zoho support person A said that RingCentral isn't supported. Zoho support person B said that it works, just make sure the call recording link works. Excellent instructions here: Call

    • How to Send Email from within a custom module (with or without an email template)

      It is possible to send an email from the Deals module. However, I can't find a way to send an email from any of our custom modules. I have tried adding an email field to the modules (even though we don't really want one or need it there). That doesn't

    • All new Address Field in Zoho CRM: maintain structured and accurate address inputs

      The address field will be available exclusively for IN DC users. We'll keep you updated on the DC-specific rollout soon. It's currently available for all new sign-ups and for existing Zoho CRM orgs which are in the Professional edition. Managing addresses

    • Zoho Inventory - Managing Items With Multiple Suppliers

      Hi community, I'm working on a project at the moment for a car parts wholesale business. Each Item (part) has its own original manufacturer part number (sometimes more than one part number for the same item). When purchasing parts from 3rd party suppliers,

    • Empowered Custom Views: Cross-Module Criteria Now Supported in Zoho CRM

      Hello everyone, We’re excited to introduce cross-module criteria support in custom views! Custom views provide personalized perspectives on your data and that you can save for future use. You can share these views with all users or specific individuals

    • Zoho Creator customer portal limitation | Zoho One

      I'm asking you all for any feedback as to the logic or reasoning behind drastically limiting portal users when Zoho already meters based on number of records. I'm a single-seat, Zoho One Enterprise license holder. If my portal users are going to add records, wouldn't that increase revenue for Zoho as that is how Creator is monetized? Why limit my customer portal to only THREE external users when more users would equate to more records being entered into the database?!? (See help ticket reply below.)

    • Super Admin Logging in as another User

      How can a Super Admin login as another user. For example, I have a sales rep that is having issues with their Accounts and I want to view their Zoho Account with out having to do a GTM and sharing screens. Moderation Update (8th Aug 2025): We are working

    • Bug tracking

      Hi, does anyone know how to track errors during picking or packing? This way I can keep track and see how to improve and prevent errors in this area.

    • Retainer invoice in Zoho Finance modlue

      Hello, Is there a way of creating retainer invoices in the Zoho Finance module? If not can I request this is considered for future updates please.

    • Is it really true that I can't set the default 'deposit to' account in 2025?

      I've been using Books for 7 years and the default account has never been a problem. I usually manually reconcile invoices and have never had a thought about which account. It has always been my account. However, I recently noticed that for the past 4

    • How do I see the total leads during a certain period?

      I understand I can get the count of leads and potentials but the total number of leads in a certain period should be equal to Leads+potentials because when we convert a lead it gets moved to potentials and no longer exists there. is there a way i could

    • Unable to Add Asset to Work Order Due to Different Contact and Company

      Hello Zoho Team, I’m facing an issue in our environment related to asset linking in Work Orders. Here’s the situation: When our engineers go for field activities, they initially create the Work Order using the salesperson’s contact details. After completing

    • Where is the Global Search field?

      I am looking for an alternative to SF.com. Zoho CRM seems to be work fine, and be customizable in terms of the fields and reports. But there's one big thing missing and it's going to prevent us from using it: there's no global search box at the top of

    • API 500 Error

      Hello amazing ZOHO Projects Community, I get this message. How can we solve this? { "error": { "status_code": "500", "method": "GET", "instance": "/api/v3/portal/2010147XXXX/projects/2679160000003XXXX/timesheet", "title": "INTERNAL_SERVER_ERROR", "error_type":

    • In place field editing for candidates

      Wondering about any insight/best practices for efficiently updating candidate records while reviewing them in a Job Opening pipeline. We can do in-field editing (e.g. update job title or City) only when we have the full candidate record open, however

    • Audio/video quality issues with Zoho Meeting – Any roadmap for improvement?

      Hi Zoho Team, We’ve been using Zoho Meeting for both internal and external meetings, and unfortunately, the experience has been consistently poor. The video and audio quality are so unreliable that it often renders meetings ineffective—especially with

    • Default tax type for mileage?

      Where we are, mileage includes a government tax. Is there any way to set a default tax for the Personal Car Mileage category of expense? (Or any other way?)

    • Analytics Portal

      I have the "standard plan" and want to explore the portal option; I activated the 15-day trial but do not see the pricing for the add-on. How can I get the price under "Upgrade add-ons." Thanks Rudy

    • The Social Wall: October 2025

      Hello everyone, As we head toward the end of the year, we’re bringing you a few updates to help give your social media efforts a strong finish. This month, we’re rolling out new enhancements across both the web and mobile app. Post Preview Have you ever

    • Show price book list price

      When using price books, once you add products to the price book in the Products related list you can display the Unit price which is the default list price; however, there is no option to show the price book list price. To see the price book list price

    • Zoho CRM - Scheduled Reports Which Contain Chart

      Hi Zoho CRM Team, I'm requesting that the Report Export and Scheduling feature be enhanced to include a chart, if one has been created on a report. At the moment I have a report which shows Sales This Week by Deal Owner and a pie chart at the top of the

    • Cliq File Upload

      While uploading large file like 500MB, it takes time, that fines. But if you resize window or move window in other screen, that uploading disappears. After upload complete & sent it will be visible

    • Resizing a Record Template Background Inage

      Hi everyone, I have an issue which I can't seem to resolve: Basically, I'm designing a record template in certificate form. I've specified A5 landscape. I've set my background image the same dimensions with total pixels at 443,520. Whatever I try, when

    • Analytics : How to share to an external client ?

      We have a use case where a client wants a portal so that several of his users can view dashboards that we have created for them in Zoho Analytics. They are not part of our company or Zoho One account. The clients want the ability to have user specific,

    • Zia Actions: AI-powered Workflow Automation for Faster and Smarter Execution

      Hello everyone, Workflows got a notch better with AI-based actions. Actions such as field extraction, prediction, auto reply, and content generation facilitate quick execution with improved speed and accuracy. Zia can intercept useful details in newly

    • Next Page