Kaizen #123 Data Synchronization from a third party application

Kaizen #123 Data Synchronization from a third party application

Welcome back to the Kaizen series!

In last week's post in the Kaizen series, we discussed one-way data synchronization from Zoho CRM to a legacy application, utilizing Zoho CRM's Bulk Read API and Notification API. This week, we will discuss data synchronization from a third party app to Zoho CRM.

The APIs available in Zoho CRM to insert, update or upsert records are :

1) Insert, Update, and Upsert Records APIs

These are synchronous APIs that are used to perform createupdate or upsert operations for up to 100 records in a Zoho CRM module. While the Insert Records API and Update Records API is used to create and update a record respectively, the  Upsert Records API is used to upsert (insert or update) a record depending on the availability of the record in your org . In Upsert Records API, the system checks for duplicate records based on the values in the duplicate_check_fields json array in the input json. If the record is already present, it gets updated. If not, the record is inserted. You will get the status of these operations immediately. Read more about the Upsert Records API in this Kaizen post.

2) Bulk Write API

 The Bulk Write API is an asynchronous API that is used to perform create/update/upsert operations for up to 25K records in a Zoho CRM module. A background job will be scheduled in Zoho CRM. You can check the status of the scheduled job using the Bulk Write Job Details API. Alternatively, you can configure a callback URL when you schedule the job. The details of the Bulk Write Job result are posted on the callback URL upon successful completion or failure of the job. This bulk write API involves the following steps:

To learn more about these steps in the Bulk Write API and about the Bulk Write API in general, refer to the Kaizen post on the Bulk Write API.


  Insert, Update, Upsert Records


Bulk Write API


Insert, update, or upsert up to 100 records per API call.

Insert, update, or upsert up to 25000 records per API call.
The response is available instantly.


The response to the Bulk Write API request will not be available immediately. You can check the status of the job by polling for it, or you can wait for the status to be available in the callback URL after the job has completed.

These APIs consume one credit per 10 records. 

Bulk Write API consumes 500 credits per API call. 
Input should be in JSON format

The input file should be in CSV format.




Let us now examine four different scenarios for record insertion in a module, specifically with varying quantities: 100, 2000, 5000, and 25000 records, and discover how these APIs perform in terms of number of API calls and credits consumed.


Number of Records

Insert,Update or Upsert Records API

Bulk Write API
100
1 API Call
10 API Credits
1 API Call
500 API Credits
2000
20 API Calls
200 API Credits
1 API Call
500 API Credits
5000
50 API Calls
500 API Credits
1 API Call
500 API Credits
25000
250 API Calls
2500 API Credits
1 API Call
500 API Credits

Clearly, if you have more than 5000 records, it is advisable to use the bulk write API, given that you are ok with the operation being run asynchronously. If you want instant results, you should use insert/update/upsert APIs. However, more API credits will be consumed in this case.

Data Sync

Consider an e-commerce company where the Order management team uses a legacy system and the Accounts team uses Zoho CRM. To ensure that the Accounts team works with near real-time data in Zoho CRM, data synchronization from the legacy application is essential. Assume that in Zoho CRM, there is a custom module named "Orders" that contains the details from the legacy system. Imagine that you are in charge of the data synchronization between Zoho CRM and the legacy system. How should you call these APIs to achieve this synchronization? The synchronization process involves two steps. 
Step 1: Initial Data Sync : Initial data sync from the legacy system to Zoho CRM.
Step 2: Subsequent Data Sync : Push any new orders or updated orders to Zoho CRM.

The solution we propose here is one of the many ways in which the data sync can be achieved.

This solution is built on the below three key points: 

1) Unique field in Order Module: Maintain a unique field "parent_id" in the "Orders"  module. The unique identifier of the orders in your legacy system should be mapped to this parent_id field. In create bulk write job, this can be achieved by the section field_mappings. For ensuring correct working of upsert operations, you can make use of "find_by" field inside resource JSON Array in the Create Bulk Write Job API Request.  You should provide the value of "find_by" as the unique field "parent_id" .  To read more about find_by field refer to Create Bulk Write Job page.

2) Sync Flag in the Orders table in legacy system: For the sync, maintain a flag in your legacy system orders table for indicating whether the record is synced with Zoho CRM or not. Once the sync is done, update the flag to  true. Whenever a record is modified or sync fails, the flag value should be updated to false. This ensures that in case a record that is synced gets updated during the initial sync process, this flag column helps us re-sync this record again.

3) Grouping of data to batches:  For initial sync, the data volume in the table will be huge. So they have to be batched in a group of 25k records. This grouping can be done based on sorting the table with a particular column & filtering records based on the sync column with criteria sync=false.

Step 1: Initial Data Sync 

Since you have multiple batches of 25k records, you have to iterate them one by one and send them to the Bulk Write API.

Initial Data Sync
For each batch of data, prepare them as a CSV file and initiate a bulk write Job. Next, check the status of the Bulk Write job, and once complete, look for any records that have been skipped during the process. If there are any, fix them, update the sync flag as false, and include them in the next round of data sync. Continue this loop until all records, including any that were missed, are processed. Take the next batch and repeat till all the batches are over.

Step 2: Subsequent Data Sync

The Order Management team continues to use the legacy application, leading to continuous creation or update of records in the Orders module.
It is not recommended to invoke Upsert Records API whenever an operation happens in Orders in the legacy system. Subsequent data syncs can be done in two ways: either by  (a)near real-time sync or (b) Interval based sync.


(a) Near Real-time Sync

 As we want near real-time data in Zoho CRM, it is ok for the records that are updated in the legacy system from last synchronization to be unavailable in Zoho CRM for 10 minutes.

Near Real-time Sync

Instead of making API calls to Zoho CRM on every data operation, store the data in a temporary data store like a DB table or Redis. Once 100 records are accumulated in that data-store, the Upsert Records API can be promptly invoked with all these record details and followed by emptying of the data store. This approach will lead to lower consumption of API credits.
There are chances that in a period of 10 mins, 100 record operations may not happen. Scheduler jobs or CRON jobs can be created to handle the records in the data store to update records to Zoho CRM. Once these records are handled, the data store should be emptied.

(b) Interval based synchronization

For interval based synchronization, schedule bulk write API calls at suitable intervals to upsert the Orders module in Zoho CRM. Whenever there is a record operation in the legacy table, update the sync flag to "false". At regular intervals, such as every 12 or 24 hours, the bulk write API or Upsert Records API can be employed depending on the volume of the data changes.

We hope you found this article useful. We will be back next week with another interesting topic. If you have any questions, write to us at support@zohocrm.com or let us know in the comment section. 


Related Links




    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                                Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
                                                              • Kaizen #226: Using ZRC in Client Script

                                                                Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this
                                                              • Kaizen #222 - Client Script Support for Notes Related List

                                                                Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how
                                                              • Kaizen #217 - Actions APIs : Tasks

                                                                Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.
                                                              • Kaizen #216 - Actions APIs : Email Notifications

                                                                Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • I want to create a mailing list, NOT a group.

                                                                                                                  Can I create a mailing list in Zoho mail? I just want to be able to make a list of email addresses and give the list a name. Then when I type the list name, the list of email addresses will be automatically listed. When I create a group it sends an email
                                                                                                                • ERROR CODE :554 - Your access to this mail system has been rejected due to poor reputation of a domain used in message transfer

                                                                                                                  This message was created automatically by mail delivery software. A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. *****, ERROR CODE :554 - Your access to this mail system has been rejected due
                                                                                                                • no me llegan los correos a Zoho mail

                                                                                                                  No puedo recibir correos pero sí enviarlos, ya hice la modificación de MX y la verificación de teléfonos, qué es lo que ocurre? gracias
                                                                                                                • Group Calendar as Default for adding new events, etc?

                                                                                                                  Hi, I want to make the group calendar (that I created, if that makes a difference) the default for anything new I add to the calendar. How can I do that? thanks.
                                                                                                                • Bookmark Loading is Buffering

                                                                                                                  Hi, i clicked on the bookmark tab, around yesterday and since then it's been constantly buffering and doesn't allow me to access the mail's i have tried login in and out but of no help also trying to share a screenshot of the issue around 232 kb size,
                                                                                                                • Zoho Webinar custom registration fields into Zoho CRM

                                                                                                                  I am pushing webinar registrations into zoho crm as leads and this is working fine. I have added a few custom fields to my webinar registration and I wish for these fields values to get mapped into the resulting CRM lead record. I am not seeing anywhere
                                                                                                                • GitLab Extension for Zoho Desk: Connecting support and development for faster resolutions

                                                                                                                  Hello everyone! We’re excited to introduce the GitLab Extension for Zoho Desk, an integration that bridges the gap between support and development teams. This allows tickets to be converted into actionable GitLab issues for faster resolutions, better
                                                                                                                • Download Google Drive Shared Image/File Using Deluge – Complete Guide

                                                                                                                  Download Google Drive Shared Image/File Using Deluge – Complete Guide When working with Google Drive share links inside Deluge, many developers face a common challenge: 👉 How can we download a Google Drive file directly using Deluge? This article explains
                                                                                                                • How to update Acuity appointment fields from Zoho Flow?

                                                                                                                  Hi, I need help with Zoho Flow → Acuity Scheduling integration. What I’m doing Appointment created in Acuity Zoho Flow creates a Zoho Calendar event I want to send the Zoho Calendar Event UID back into Acuity (into a custom form field) This is so that
                                                                                                                • No horizontal scroll bar in reports

                                                                                                                  Hi support team, I created a pivot table report which contains many columns (in my application about 15). The problem is that there is no bottom scroll bar to display colums that does not fit within the browser page. I saw that it was possible to resize colums, but colums sizes are not saved within the session. So could you add scroll bar, and eventually make colums sizes stored in the user session ? Best regards,
                                                                                                                • Zoho Map integration tasks have changed - you need to "Locate all instances of Zoho Map integration tasks in your Deluge scripts by searching for the v1 marker... before 16 January 2026"

                                                                                                                  Details of an issue specific to Zoho CRM + Deluge Functions (originally posted in this forum) can be found here: https://help.zoho.com/portal/en/community/topic/zoho-map-integration-tasks-have-changed-you-need-to-locate-all-instances-of-zoho-map-integration-tasks-in-your-deluge-scripts-by-searching-for-the-v1-marker
                                                                                                                • filter broke my data

                                                                                                                  I uploaded a file recently from Sheets and it has top 2 rows frozen, with table headers in second row and each one is filterable. somehow my first 2 columns became unfiltered and no matter what I do I cannot reapply the filter?? also didn't realize they
                                                                                                                • Request to remove domain "greentechcoatings.vn" from previous Zoho account

                                                                                                                  Hello Zoho Support, I would like to register the domain "greentechcoatings.vn" in my new Zoho Mail account, but I receive the message "This domain is already associated with another account". Please remove the domain from any previous/unknown account
                                                                                                                • Trigger actions in third-party apps using Zoho Flow

                                                                                                                  Greetings, I hope you're all doing well. We're excited to share an enhancement to Bigin's workflow capabilities. Zoho Flow Actions are now available in Bigin, enabling you to automate tasks across third-party applications directly from your workflow and
                                                                                                                • Enroll, Script, Win: Hackathon 2025!

                                                                                                                  Hello CRM Developers! Are you ready to create some magic with Client Script, Widgets and Functions in Zoho CRM? Let’s make Hackathon 2025 an unforgettable adventure! The Zoho Developer Community Hackathon 2025 is here, and it’s your time to shine! REGISTER
                                                                                                                • Zoho Help Desk - Not receiving emails

                                                                                                                  Hello… I am on your trial version of help desk… and I am having trouble getting emails from Zoho Help Desk to my MS Outlook Exchange Server… So when a customer sends ticket request I am not getting an email with a notification? I have read your articles
                                                                                                                • Almost all Flows are being queued

                                                                                                                  A few days ago I saw one of my Flows status was Queued. This was the first time this had ever happened. Now, almost every Flow that triggers is Queued for at least a few minutes. If I re-trigger the Flow (which causes separate problems) they sometimes
                                                                                                                • Employment Contract / HRMS Documents - Zoho People

                                                                                                                  How do I create customized HR documents for disbursal from Zoho People ? Example, say offer letters ? appointment letters ? Memos ? We want to be able to preset these in one or ther other form and issue them to employees who are in Zoho People.
                                                                                                                • Live webinar: Craft and deliver impactful slides with Show’s desktop apps

                                                                                                                  If you love the comfort of working on your desktop and want a setup that keeps you focused and uninterrupted, this session is made for you. We’re excited to invite you to our upcoming live webinar: "Craft and deliver impactful slides with Show's desktop
                                                                                                                • Enhancements to Bigin's forms

                                                                                                                  Greetings, I hope all of you are doing well. We're happy to announce a few recent enhancements we've made to Bigin's forms. We'll go over each one in detail. Merge field support in auto-filled forms The auto-fill option in Bigin's forms lets you predefine
                                                                                                                • Table dimensions

                                                                                                                  I try changing the dimensions of the table on my computer but it doesn't change. Do I have to be a premium member or does it only work on the app?
                                                                                                                • Direct link to Record Summary

                                                                                                                  Hi everyone, In one of my reports, I have built a Record Summary template to display the details of one record. I would like to be able to link directly to this Record Summary once I submit a new record, without having to go to the list of records first and click on View. Is there a possibility to do so ?  Should I use the URL by passing some parameters ? Thank you very much for your help ! Guillaume
                                                                                                                • Amendment effective date

                                                                                                                  Hi everyone, I noticed that the amendment effective date mentionned in my amendment is not right. Indeed, when a contract is amended several times, it states the previous amendment and their effective date. However, the effective date stated is always
                                                                                                                • Show both Vendor and Customers in contact statement

                                                                                                                  Dear Sir, some companies like us working with companies as Vendor and Customers too !!! it mean we send invoice and also receive bill from them , so we need our all amount in one place , but in contact statement , is separate it as Vendor and Customer, 
                                                                                                                • STOCK history in zohosheets

                                                                                                                  is it possible to get historical stock value using stock function in zoho sheets? i could not see from and to period in the helper document.
                                                                                                                • Auto sync Photo storage

                                                                                                                  Hello I am new to Zoho Workdrive and was wondering if the is a way of automatically syncing photos on my Android phone to my workdrive as want to move away from Google? Thanks
                                                                                                                • Agent password reset

                                                                                                                  Hi Zoho support, I would like to ask if there is a way the admin can reset a password of an agent? Regards
                                                                                                                • Can receive but not send messages in Zoho Mail

                                                                                                                  Hello! I was able to configure my email client successfully in that I can receive messages just fine. However, when I send messages out, they seem to go out fine (I don't receive any errors or anything), but the recipient does NOT receive those messages.
                                                                                                                • Mail is sent twice!

                                                                                                                  Been using Zoho for a while now. Installed Zoho for someone else and some weird things are happening. Mails are being sent twice. He is using Thunderbird as an email client. I already read about email being duplicated in the sent folder. But in my case
                                                                                                                • Can't login IMAP suddenly

                                                                                                                  Since this evening I'm getting the error: You are yet to enable IMAP for your account. Please contact your administrator... IMAP always been enabled in my account and was workign fine for the past 7 years. Already tried turning IMAP off and on again.
                                                                                                                • Sending of username did not succeed: Mail server pop.zoho.com responded: User already specified

                                                                                                                  I am having issues receiving emails from Zoho in Thunderbird. I am getting the above error. The first error tells me Authentication failed, and prompts me to enter in my password. Then I get the above error. I can receive emails when I log in online to
                                                                                                                • Bug tracking

                                                                                                                  Hi, does anyone know how to track errors during picking or packing? This way I can keep track and see how to improve and prevent errors in this area.
                                                                                                                • Migration of corporate mail environment from Yandex 360 to Zoho mail

                                                                                                                  I have to migrate a corporate mail environment with an existing domain from Yandex 360 to Zoho mail. It is vital to migrate all users with all the data. I have read the article on this topic using MacMister Email Backup Software just now and have some
                                                                                                                • I'm unable to send mail pthrough Zoho SMTP programmatically

                                                                                                                  This has been working for years, but today it's been offline all day long. I see nothing anywhere on your site about this. I'm not the only one experiencing this. Downdetector has a spike of reports today
                                                                                                                • Can no longer send email via Django site

                                                                                                                  This was working fine as of 11/7/25. Now I am unable to send user verification emails from a Django site on a AWS lightsail sever. When a user attempts to register the following error occurs. I have also attempted to send a test email via the shell and
                                                                                                                • unable to send email but able to receive email

                                                                                                                  my email address is info@securityforceservices.ca
                                                                                                                • Login to server failing

                                                                                                                  When trying to retrieve my mail, I am getting this error message -- Login to server pop.zoho.com with username (my email address) failed. It gives me the option to retry, enter password, or cancel. Then I get this message -- Sending of username did not
                                                                                                                • Configuration failed: 200 response not received for POST request.

                                                                                                                  Hello, I am trying to set up a webhook to connect with an Salesforce but I receive the following error from Zoho: Configuration failed: 200 response not received for POST request I have tried testing it on webhook.site as well and receive the same error
                                                                                                                • Zoho Migration Assistant not working

                                                                                                                  Hello, I am trying to use you Migration assistant to migrate emails from Rediff to Zoho. I am stuck in the first step. After downloading the migration tool, I copied the link to verify user credentials, however, after pasting the link in the browser,
                                                                                                                • Contacts Missing — PeopleSync/Zoho Mail

                                                                                                                  English: In our company we use ManageEngine Mobile Device Manager (MDM), Free edition, to manage corporate mobile devices. Our usage policy does not allow personal Google accounts on these devices; therefore, Google account sync is blocked through MDM.
                                                                                                                • Next Page