Kaizen #123 Data Synchronization from a third party application

Kaizen #123 Data Synchronization from a third party application

Welcome back to the Kaizen series!

In last week's post in the Kaizen series, we discussed one-way data synchronization from Zoho CRM to a legacy application, utilizing Zoho CRM's Bulk Read API and Notification API. This week, we will discuss data synchronization from a third party app to Zoho CRM.

The APIs available in Zoho CRM to insert, update or upsert records are :

1) Insert, Update, and Upsert Records APIs

These are synchronous APIs that are used to perform createupdate or upsert operations for up to 100 records in a Zoho CRM module. While the Insert Records API and Update Records API is used to create and update a record respectively, the  Upsert Records API is used to upsert (insert or update) a record depending on the availability of the record in your org . In Upsert Records API, the system checks for duplicate records based on the values in the duplicate_check_fields json array in the input json. If the record is already present, it gets updated. If not, the record is inserted. You will get the status of these operations immediately. Read more about the Upsert Records API in this Kaizen post.

2) Bulk Write API

 The Bulk Write API is an asynchronous API that is used to perform create/update/upsert operations for up to 25K records in a Zoho CRM module. A background job will be scheduled in Zoho CRM. You can check the status of the scheduled job using the Bulk Write Job Details API. Alternatively, you can configure a callback URL when you schedule the job. The details of the Bulk Write Job result are posted on the callback URL upon successful completion or failure of the job. This bulk write API involves the following steps:

To learn more about these steps in the Bulk Write API and about the Bulk Write API in general, refer to the Kaizen post on the Bulk Write API.


  Insert, Update, Upsert Records


Bulk Write API


Insert, update, or upsert up to 100 records per API call.

Insert, update, or upsert up to 25000 records per API call.
The response is available instantly.


The response to the Bulk Write API request will not be available immediately. You can check the status of the job by polling for it, or you can wait for the status to be available in the callback URL after the job has completed.

These APIs consume one credit per 10 records. 

Bulk Write API consumes 500 credits per API call. 
Input should be in JSON format

The input file should be in CSV format.




Let us now examine four different scenarios for record insertion in a module, specifically with varying quantities: 100, 2000, 5000, and 25000 records, and discover how these APIs perform in terms of number of API calls and credits consumed.


Number of Records

Insert,Update or Upsert Records API

Bulk Write API
100
1 API Call
10 API Credits
1 API Call
500 API Credits
2000
20 API Calls
200 API Credits
1 API Call
500 API Credits
5000
50 API Calls
500 API Credits
1 API Call
500 API Credits
25000
250 API Calls
2500 API Credits
1 API Call
500 API Credits

Clearly, if you have more than 5000 records, it is advisable to use the bulk write API, given that you are ok with the operation being run asynchronously. If you want instant results, you should use insert/update/upsert APIs. However, more API credits will be consumed in this case.

Data Sync

Consider an e-commerce company where the Order management team uses a legacy system and the Accounts team uses Zoho CRM. To ensure that the Accounts team works with near real-time data in Zoho CRM, data synchronization from the legacy application is essential. Assume that in Zoho CRM, there is a custom module named "Orders" that contains the details from the legacy system. Imagine that you are in charge of the data synchronization between Zoho CRM and the legacy system. How should you call these APIs to achieve this synchronization? The synchronization process involves two steps. 
Step 1: Initial Data Sync : Initial data sync from the legacy system to Zoho CRM.
Step 2: Subsequent Data Sync : Push any new orders or updated orders to Zoho CRM.

The solution we propose here is one of the many ways in which the data sync can be achieved.

This solution is built on the below three key points: 

1) Unique field in Order Module: Maintain a unique field "parent_id" in the "Orders"  module. The unique identifier of the orders in your legacy system should be mapped to this parent_id field. In create bulk write job, this can be achieved by the section field_mappings. For ensuring correct working of upsert operations, you can make use of "find_by" field inside resource JSON Array in the Create Bulk Write Job API Request.  You should provide the value of "find_by" as the unique field "parent_id" .  To read more about find_by field refer to Create Bulk Write Job page.

2) Sync Flag in the Orders table in legacy system: For the sync, maintain a flag in your legacy system orders table for indicating whether the record is synced with Zoho CRM or not. Once the sync is done, update the flag to  true. Whenever a record is modified or sync fails, the flag value should be updated to false. This ensures that in case a record that is synced gets updated during the initial sync process, this flag column helps us re-sync this record again.

3) Grouping of data to batches:  For initial sync, the data volume in the table will be huge. So they have to be batched in a group of 25k records. This grouping can be done based on sorting the table with a particular column & filtering records based on the sync column with criteria sync=false.

Step 1: Initial Data Sync 

Since you have multiple batches of 25k records, you have to iterate them one by one and send them to the Bulk Write API.

Initial Data Sync
For each batch of data, prepare them as a CSV file and initiate a bulk write Job. Next, check the status of the Bulk Write job, and once complete, look for any records that have been skipped during the process. If there are any, fix them, update the sync flag as false, and include them in the next round of data sync. Continue this loop until all records, including any that were missed, are processed. Take the next batch and repeat till all the batches are over.

Step 2: Subsequent Data Sync

The Order Management team continues to use the legacy application, leading to continuous creation or update of records in the Orders module.
It is not recommended to invoke Upsert Records API whenever an operation happens in Orders in the legacy system. Subsequent data syncs can be done in two ways: either by  (a)near real-time sync or (b) Interval based sync.


(a) Near Real-time Sync

 As we want near real-time data in Zoho CRM, it is ok for the records that are updated in the legacy system from last synchronization to be unavailable in Zoho CRM for 10 minutes.

Near Real-time Sync

Instead of making API calls to Zoho CRM on every data operation, store the data in a temporary data store like a DB table or Redis. Once 100 records are accumulated in that data-store, the Upsert Records API can be promptly invoked with all these record details and followed by emptying of the data store. This approach will lead to lower consumption of API credits.
There are chances that in a period of 10 mins, 100 record operations may not happen. Scheduler jobs or CRON jobs can be created to handle the records in the data store to update records to Zoho CRM. Once these records are handled, the data store should be emptied.

(b) Interval based synchronization

For interval based synchronization, schedule bulk write API calls at suitable intervals to upsert the Orders module in Zoho CRM. Whenever there is a record operation in the legacy table, update the sync flag to "false". At regular intervals, such as every 12 or 24 hours, the bulk write API or Upsert Records API can be employed depending on the volume of the data changes.

We hope you found this article useful. We will be back next week with another interesting topic. If you have any questions, write to us at support@zohocrm.com or let us know in the comment section. 


Related Links




    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                                Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
                                                              • Kaizen #226: Using ZRC in Client Script

                                                                Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this
                                                              • Kaizen #222 - Client Script Support for Notes Related List

                                                                Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how
                                                              • Kaizen #217 - Actions APIs : Tasks

                                                                Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.
                                                              • Kaizen #216 - Actions APIs : Email Notifications

                                                                Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • Internal Server Error (500) When Attempting to View Banking Transactions

                                                                                                                  I am experiencing an Internal Server Error (500) when attempting to view transactions across all of my banking accounts. Despite multiple attempts to resolve this, I have received little more than runaround from support, and the issue remains unresolved.
                                                                                                                • How do I add a blank line to the Organisation Address Format?

                                                                                                                  I'd like to have my VAT number, for example, shown prominently by having a clear gap between it and the address block above, but any blank lines in the address format get ignored in PDF outputs.
                                                                                                                • Automatic Invoice Number generation for createRecord

                                                                                                                  Hello, while testing some custom Buttons in my Zoho Books application, I noticed that I get an error that previously did not occur. After some further digging I found that the automatic transaction numbering of invoices no longer work in my organization.
                                                                                                                • Adding number of days to an estimate.

                                                                                                                  I need both QTY of item and "number of days hire" in my estimates at the line item level. Any clues as to how this is done would be greatly appreciated. It needs to calculate. Thanks J
                                                                                                                • Books Api: listing expenses created after certain dates

                                                                                                                  Is there any parameter I can add to the List Expenses endpoint that will let me look up expenses by when they were created?
                                                                                                                • Why can't we change the Account type from an Expense to an Asset?

                                                                                                                  Like the question. Why in QuickBooks for example if I mistakenly created an account as an expenses and I already captured information in those accounts, I can just change the account type from expense to asset
                                                                                                                • Is it possible to do validation for the Actions added to Reports?

                                                                                                                  We have an all-around On Validate function that checks all the possibilities before the Created/Edited form submissions. We want to have a button in the report view, so we can change records without entering. We are able to add this button, and it does
                                                                                                                • Ability to Edit Ticket Subject when Splitting a Ticket

                                                                                                                  Often someone will make an additional or new request within an existing ticket that requires we split the ticket. The annoying part is that the new ticket maintains the subject of the original ticket after the split so when the new ticket email notification
                                                                                                                • [Free Webinar] Environments in Zoho Creator - Creator Tech Connect

                                                                                                                  Hello Everyone, We’re delighted to kick off the 2026 edition of the Zoho Creator Tech Connect Series and are excited to continue this learning journey with you. The Creator Tech Connect series is a free monthly webinar featuring deep-dive technical sessions
                                                                                                                • Zoho Voice API SMS

                                                                                                                  I made a post request in Zapier to try to send an SMS. Authentication appears to be configured correctly. I followed the documentation and I'm getting a 400 Error "Internal Server Error". Not helpful at all. Anyways, I could not find any information on
                                                                                                                • How do I change (error) display messages

                                                                                                                  Hi! I would like to edit display messages like "invalid entries" and "enter a value for Nome" and so on into Italian language messages. How can I do that? Already tried on Validate on form submit. Where I am failing? Thanks in advance.
                                                                                                                • Record Overseas Transactions Along with CC charges

                                                                                                                  Hi All, We receive payments via stripe in multiple currencies and stripe takes around 2.5% fees. The amount is settled in INR into our local bank. What should be the currency of the income account used for Stripe payments? Here is a sample flow that we
                                                                                                                • Easier onboarding for new users with stage descriptions

                                                                                                                  Greetings, I hope all of you are doing well. We're happy to announce a recent enhancement we've made to Bigin. You can now add descriptions to the stages in your pipeline. Previously, when creating a pipeline, you could only add stages. With this update,
                                                                                                                • Pieds de page personnalisé - Document Zoho Writer

                                                                                                                  Bonjour à tous, Je rencontre un souci avec l’ajout d’un pied de page personnalisé dans un document Zoho Writer. Je souhaite insérer les informations de mon entreprise (notamment un logo + adresse) dans le pied de page. Le problème, c’est que lorsque j’ajoute
                                                                                                                • Credit card transactions are backwards after importing statements

                                                                                                                  I am new to Zoho Books so I'm importing my firm's bank and credit card statements in. My credit card statements have a single column with negative numbers so that is the option I chose. But when I went to categorize the credit card transactions, I can
                                                                                                                • Permissions on Views

                                                                                                                  Having the option of any agent creating custom views is firing back and got a situation where there are a hundred different views across the team and tickets are not being dealt in the most efficient of ways. Tickets seems to be missed by some agents,
                                                                                                                • Function #8: Add additional charges to invoices

                                                                                                                  Here goes one of the highly sought-after custom functions in Zoho Books. If you find yourself needing to apply additional charges to customers on their invoices (say credit card surcharges, or fuel charges applicable to customers from a certain region,
                                                                                                                • VIsual maindmaps in Zoho notebook Ai

                                                                                                                  Can I create and export VIsual maindmaps in Zoho notebook Ai
                                                                                                                • 1stDibs Integration to Zoho Inventory

                                                                                                                  Hello is it possible to integrate my Zoho inventory and 1stDibs?
                                                                                                                • Community Question: Renewal vs Invoicing

                                                                                                                  This is a question for the community. Does anyone else consider there to be a difference between a subscription renewal event and sending out a recurring invoice for a subscription? For example, let's say customer XYZ purchases a 1-year subscription to
                                                                                                                • cloud console support for music websites

                                                                                                                  Hi Friends, I am not from a very technical background.. So need support from the Zoho family. I want to build a e-commerce website which is in Musical Niche. It will help people learn music & play instruments of all types. There are a few players in the market like Chordify , Guitaa, Guitar Dashboard, ChordU & few others. But I all these websites allow only a few instruments  to play. So I want to build a better  website than the one I mentioned. So wnated to know what UI & AI should I use so that
                                                                                                                • Scheduled Reports - Do not send empty report

                                                                                                                  Hello, We are intensively using reports in the CRM, especially for sales managers. When data is empty, they still receive an email. Can you add an option to avoid sending the report when data is empty?
                                                                                                                • Is Zoho One Desktop more secure than Zoho One Web SaaS?

                                                                                                                  Is Zoho One Desktop more secure than using Zoho One in browser? Inherently, it seems desktop would be unless you don't do things like share your pw or leave pc on. I am concerned about data being on the cloud or someone else's server and database.
                                                                                                                • Reminder Settings - Time Tracker

                                                                                                                  On the time tracker in Reminder Settings I created a reminder according to the Zoho manual. But a manager asked me to change the message that goes in the email. Is it possible to change the message? I didn't find that in the Zoho manual. Another question: I configured to receive the reminder everyone who logged in less than 40 hours. Does Zoho consider less than 40 hours of the current week or the whole month? Another situation, I put it so that I and another specific user would receive the notification,
                                                                                                                • Create Item group from a composite Item

                                                                                                                  I have applied my mind for hours but cannot figure this out. Can you have a composite item in an item group?. E.g. We bundle different color and size SKU's together as composite items. Also Using composite items as Bill of Materials. We want to create
                                                                                                                • Main difference of Zoho Recruit Corporte version and Staffing HR vesion

                                                                                                                  Hi Zoho, I need help to fully understand what is the main key point differences of Recruit Corporate version versus the Staffing HR version? We are currently using Corporate HR version and we are looking on having an insightful automated reporting, does
                                                                                                                • Out of Stock items showing in Commerce

                                                                                                                  I have over 6000 items and most are not in stock, but all items are showing up in Commerce whether they are inventory or not. What option or feature can you use to hide items in Commerce at zero or negative quantities? I currently am using Commerce for
                                                                                                                • Is it possible to transfer data from one related list to another within the same module ?

                                                                                                                  In the Leads module, there is an existing default Product related list that already contains data. Recently, I added a custom multi-lookup field, which created a new related list in the same Leads module. Now, I need to move the existing data from the
                                                                                                                • How to implement new online payment gateway?

                                                                                                                  Hello, Can you tell me how to proceed to implement my local payment gateway? DIBS has an open avaiable API that should be easy to implement into ZOHO BOOKS. http://tech.dibspayment.com/dibs_payment_window
                                                                                                                • Formula working in MS Excel , is not working in Zoho Sheets, Filter fuction which is working in MS sheets per the attached workbook / Daily update.

                                                                                                                  Filter function , which is working in MS Excel is not working in Zoho sheets. Please help on function of filter of particular class sheets and extract students name who has more than 3.5 hours of study hours on particular day depending on date in D7 cell.
                                                                                                                • New in Smart Prompt: Record Assistant for contextual assistance, and support for new AI models

                                                                                                                  Smart Prompt helps teams stay informed and move faster by providing relevant suggestions where work happens in CRM. With this update, Smart Prompt becomes more adaptable to your organization’s AI preferences. You can now choose which Large Language Model
                                                                                                                • Improved Integration Failure Information (And Notification Options)

                                                                                                                  Hi, When an attachment service for example fails, you just get "Field x - Error", I can check the field it is failing on and find nothing wrong, same file size, type, dimensions, etc. so more information as to the actual issue would be helpful. And an
                                                                                                                • Change User Role in a Form

                                                                                                                  Hi, When in a form, it would be good (And consistent) to be able to change the user role/permission like you can with Shared Reports, All Entries or the actual User itself, rather than having to delete the users permission and then add it back again with
                                                                                                                • Reassign Partially Saved Entries

                                                                                                                  Hi, I would like to be able to go to Partially Saved Entries and like the option to delete them I would like the option to multi-select and be able to reassign them to another user to complete (Such as when a user has left the company). Thanks Dan
                                                                                                                • Increase the "Maximum Saved Entries per User" Options Limit

                                                                                                                  Hi, You can create lots of saved entries, yet the Limit when you apply one is 25, we may often expect 32 to be in draft, and therefore want to enforce that, can we increase the limit of this field from 25 to 100 (As you can just turn it off and have more
                                                                                                                • Product Updates in Zoho Workplace applications | December 2025

                                                                                                                  Hello Workplace Community, Let’s take a look at the new features and enhancements that went live across all Workplace applications this December. Zoho Mail Block emails without notifying the sender Block emails without sending a rejection notification
                                                                                                                • Zoho CRM for Gmail Extension Not Working in Brave Browser?

                                                                                                                  Is anyone able to get the Zoho CRM Chrome Extension working in the Brave browser? They're both built on the Chromium platform and every other Chrome Extension works with the exception of Zoho CRM for Gmail so any ideas here?
                                                                                                                • Set Frozen Cells in the Report Settings

                                                                                                                  Hi, It would be nice to be able to set the frozen cells in the report Settings, and have an option if this is fixed or can be changed after loading (On the next load it still goes back to the Settings). Thanks Dan
                                                                                                                • Microsoft Teams now available as an online meeting provider

                                                                                                                  Hello everyone, We're pleased to announce that Zoho CRM now supports Microsoft Teams as an online meeting provider—alongside the other providers already available. Admins can enable Microsoft Teams directly from the Preferences tab under the Meetings
                                                                                                                • Tip of the week #26: Import/ Export calendars in Zoho Calendar.

                                                                                                                  Any calendar on the web or calendars that you create in any other calendar application can be imported in to Zoho Calendar. This will help you to add the events from the calendars that you import to your Zoho Calendar. You also have the option to export
                                                                                                                • Next Page