Kaizen #123 Data Synchronization from a third party application

Kaizen #123 Data Synchronization from a third party application

Welcome back to the Kaizen series!

In last week's post in the Kaizen series, we discussed one-way data synchronization from Zoho CRM to a legacy application, utilizing Zoho CRM's Bulk Read API and Notification API. This week, we will discuss data synchronization from a third party app to Zoho CRM.

The APIs available in Zoho CRM to insert, update or upsert records are :

1) Insert, Update, and Upsert Records APIs

These are synchronous APIs that are used to perform createupdate or upsert operations for up to 100 records in a Zoho CRM module. While the Insert Records API and Update Records API is used to create and update a record respectively, the  Upsert Records API is used to upsert (insert or update) a record depending on the availability of the record in your org . In Upsert Records API, the system checks for duplicate records based on the values in the duplicate_check_fields json array in the input json. If the record is already present, it gets updated. If not, the record is inserted. You will get the status of these operations immediately. Read more about the Upsert Records API in this Kaizen post.

2) Bulk Write API

 The Bulk Write API is an asynchronous API that is used to perform create/update/upsert operations for up to 25K records in a Zoho CRM module. A background job will be scheduled in Zoho CRM. You can check the status of the scheduled job using the Bulk Write Job Details API. Alternatively, you can configure a callback URL when you schedule the job. The details of the Bulk Write Job result are posted on the callback URL upon successful completion or failure of the job. This bulk write API involves the following steps:

To learn more about these steps in the Bulk Write API and about the Bulk Write API in general, refer to the Kaizen post on the Bulk Write API.


  Insert, Update, Upsert Records


Bulk Write API


Insert, update, or upsert up to 100 records per API call.

Insert, update, or upsert up to 25000 records per API call.
The response is available instantly.


The response to the Bulk Write API request will not be available immediately. You can check the status of the job by polling for it, or you can wait for the status to be available in the callback URL after the job has completed.

These APIs consume one credit per 10 records. 

Bulk Write API consumes 500 credits per API call. 
Input should be in JSON format

The input file should be in CSV format.




Let us now examine four different scenarios for record insertion in a module, specifically with varying quantities: 100, 2000, 5000, and 25000 records, and discover how these APIs perform in terms of number of API calls and credits consumed.


Number of Records

Insert,Update or Upsert Records API

Bulk Write API
100
1 API Call
10 API Credits
1 API Call
500 API Credits
2000
20 API Calls
200 API Credits
1 API Call
500 API Credits
5000
50 API Calls
500 API Credits
1 API Call
500 API Credits
25000
250 API Calls
2500 API Credits
1 API Call
500 API Credits

Clearly, if you have more than 5000 records, it is advisable to use the bulk write API, given that you are ok with the operation being run asynchronously. If you want instant results, you should use insert/update/upsert APIs. However, more API credits will be consumed in this case.

Data Sync

Consider an e-commerce company where the Order management team uses a legacy system and the Accounts team uses Zoho CRM. To ensure that the Accounts team works with near real-time data in Zoho CRM, data synchronization from the legacy application is essential. Assume that in Zoho CRM, there is a custom module named "Orders" that contains the details from the legacy system. Imagine that you are in charge of the data synchronization between Zoho CRM and the legacy system. How should you call these APIs to achieve this synchronization? The synchronization process involves two steps. 
Step 1: Initial Data Sync : Initial data sync from the legacy system to Zoho CRM.
Step 2: Subsequent Data Sync : Push any new orders or updated orders to Zoho CRM.

The solution we propose here is one of the many ways in which the data sync can be achieved.

This solution is built on the below three key points: 

1) Unique field in Order Module: Maintain a unique field "parent_id" in the "Orders"  module. The unique identifier of the orders in your legacy system should be mapped to this parent_id field. In create bulk write job, this can be achieved by the section field_mappings. For ensuring correct working of upsert operations, you can make use of "find_by" field inside resource JSON Array in the Create Bulk Write Job API Request.  You should provide the value of "find_by" as the unique field "parent_id" .  To read more about find_by field refer to Create Bulk Write Job page.

2) Sync Flag in the Orders table in legacy system: For the sync, maintain a flag in your legacy system orders table for indicating whether the record is synced with Zoho CRM or not. Once the sync is done, update the flag to  true. Whenever a record is modified or sync fails, the flag value should be updated to false. This ensures that in case a record that is synced gets updated during the initial sync process, this flag column helps us re-sync this record again.

3) Grouping of data to batches:  For initial sync, the data volume in the table will be huge. So they have to be batched in a group of 25k records. This grouping can be done based on sorting the table with a particular column & filtering records based on the sync column with criteria sync=false.

Step 1: Initial Data Sync 

Since you have multiple batches of 25k records, you have to iterate them one by one and send them to the Bulk Write API.

Initial Data Sync
For each batch of data, prepare them as a CSV file and initiate a bulk write Job. Next, check the status of the Bulk Write job, and once complete, look for any records that have been skipped during the process. If there are any, fix them, update the sync flag as false, and include them in the next round of data sync. Continue this loop until all records, including any that were missed, are processed. Take the next batch and repeat till all the batches are over.

Step 2: Subsequent Data Sync

The Order Management team continues to use the legacy application, leading to continuous creation or update of records in the Orders module.
It is not recommended to invoke Upsert Records API whenever an operation happens in Orders in the legacy system. Subsequent data syncs can be done in two ways: either by  (a)near real-time sync or (b) Interval based sync.


(a) Near Real-time Sync

 As we want near real-time data in Zoho CRM, it is ok for the records that are updated in the legacy system from last synchronization to be unavailable in Zoho CRM for 10 minutes.

Near Real-time Sync

Instead of making API calls to Zoho CRM on every data operation, store the data in a temporary data store like a DB table or Redis. Once 100 records are accumulated in that data-store, the Upsert Records API can be promptly invoked with all these record details and followed by emptying of the data store. This approach will lead to lower consumption of API credits.
There are chances that in a period of 10 mins, 100 record operations may not happen. Scheduler jobs or CRON jobs can be created to handle the records in the data store to update records to Zoho CRM. Once these records are handled, the data store should be emptied.

(b) Interval based synchronization

For interval based synchronization, schedule bulk write API calls at suitable intervals to upsert the Orders module in Zoho CRM. Whenever there is a record operation in the legacy table, update the sync flag to "false". At regular intervals, such as every 12 or 24 hours, the bulk write API or Upsert Records API can be employed depending on the volume of the data changes.

We hope you found this article useful. We will be back next week with another interesting topic. If you have any questions, write to us at support@zohocrm.com or let us know in the comment section. 


Related Links




    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                                Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
                                                              • Kaizen #226: Using ZRC in Client Script

                                                                Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this
                                                              • Kaizen #222 - Client Script Support for Notes Related List

                                                                Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how
                                                              • Kaizen #217 - Actions APIs : Tasks

                                                                Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.
                                                              • Kaizen #216 - Actions APIs : Email Notifications

                                                                Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • Reply and react to comments

                                                                                                                  Hi everyone! We're excited to bring to you a couple of new features that'll make your sprint process simpler. A cloud application brings with it an array of social media features that can be efficiently used in your organizational setup. As an agile scrum
                                                                                                                • Module Customisation - Lookup function not available

                                                                                                                  Good evening, Within my business, I can have multiple customers, who have multiple mobile assets. When I set these assets up, I enter information such as vehicle registration, Vehicle identification number (VIN), Unit number, YOM, in addition to others.
                                                                                                                • zoho click, and nord VPN

                                                                                                                  Unfortunately, we've been having problems with Zoho Click, where essentially the line cuts off after about a minute's worth of conversation every time we are on VPN. Is there a way we can change this within the settings so it does not cut the line off
                                                                                                                • Zoho Calender

                                                                                                                  a) does the clanender in zoho project allow you to see the name of the event in the celnder view, it currently says either "Task (1) or "Milestoen (1)" b) Alternatively does the calender in Zoho project integrate with zoho calender?
                                                                                                                • Matching ZOHO Payments in Banking

                                                                                                                  Our company has recently integrated ZOHO Payments into our system. This seemed really convenient at first because our customers could pay their account balance by clicking on a link imbedded in the emailed invoice. Unfortunately, we can't figure out how
                                                                                                                • New Enhancements to Zoho CRM and Zoho Creator Integration

                                                                                                                  Hello Everyone, We’ve rolled out enhancements to the Zoho Creator and Zoho CRM integration to align with recent updates made to the Zoho Creator platform. With enhancements to both the UI and functionality, This update also tightens access control by
                                                                                                                • Work Type - Limitation

                                                                                                                  Hello, I'm setting up work types and have noticed, a limitation on the parts area to 10 lines. Can this be increased to 20 or greater? In addition to this, when I attempt to add the work type to a work order, the correct labour hours doesn't flow through.
                                                                                                                • Locked out of MFA due to changing phones

                                                                                                                  I have been locked out of my Zoho Books account since i changed my mobile phone over. I can get the OTP through the ZOHO Books software but i cant confirm using my phone app as nothing appears to authorise. I need to know how to register/set up my new
                                                                                                                • Emails sent through Bigin are not posting in IMAP Sent folder

                                                                                                                  I have set up my email to work from within Bigin using IMAP.  I am using IMAP so I can sync my email across multiple devices - phone / laptop / desktop / iPad / etc.  I want all my emails to populate my email client (outlook & iphone email) whether or
                                                                                                                • Turning the page for Zoho SalesIQ: 2025 to 2026

                                                                                                                  As we wrap up 2025, we would like to take a moment to reflect on what we set out to achieve this year, what we’ve delivered, and where we’re headed next. What we focused on in 2025 This year was all about strengthening the core of engagement and AI, making
                                                                                                                • Temporary Outage in Zoho Cliq Affecting US Users – July 23, 2025

                                                                                                                  We experienced a service disruption in Zoho Cliq that impacted core functionality for users in the US region. The issue occurred between Jul 23, 2025, 06:54:00 PM IST and 07:13:13 PM IST, lasting approximately 19 minutes. To restore service stability,
                                                                                                                • Zoho Cliq not working on airplanes

                                                                                                                  Hi, My team and I have been having this constant issue of cliq not working when connected to an airplane's wifi. Is there a reason for this? We have tried on different Airlines and it doesn't work on any of them. We need assistance here since we are constantly
                                                                                                                • Create an Eye-Catching Announcement Widget for Your Help Center

                                                                                                                  Hello Everyone! In this week’s edition, let’s explore how to keep your customers updated with exciting news in the Help Center. See how ZylkerMobile wowed their customers by bringing updates right to their portal. ZylkerMobile, the renowned brand for
                                                                                                                • Customer Management: #6 Common Mistakes in Customer Handling

                                                                                                                  Managing customers doesn't usually fall apart overnight. More often, slight gaps in the process slowly become bigger problems. Incidents like missed follow-ups, billing confusion, and unhappy customers will lead to revenue loss. Many businesses don't
                                                                                                                • Zoho Desk iOS app update: UI enhancement of picklist and multi picklist fields

                                                                                                                  Hello everyone! We have enhanced the UI of the picklist and multiselect picklist fields on the Zoho Desk iOS app to provide a more refined, efficient, and user-friendly experience. We have now supported an option to Search within the picklist and multiselect
                                                                                                                • Zoho Desk iOS app update: Revamped scribbles with Apple pencil kit

                                                                                                                  Hello everyone! We’re excited to introduce a revamped Scribble experience, rebuilt from the ground up using Apple PencilKit for smooth strokes, proper scaling, and seamless image uploads. Please update the app to the latest version directly from the App
                                                                                                                • Zoho Desk Android app update: Norwegian language support

                                                                                                                  Hello everyone! In the most recent Android version of the Zoho Desk app update, we have brought in support to access the app in Norwegian language. We have introduced the Norwegian language on the IM module of the Zoho Desk app as well. Please update
                                                                                                                • Payment Vouchers

                                                                                                                  Is there any Payment Vouchers in Zoho? How can we create payment for non-trade vendors, i.e. professional fees, rent, and payment to commissioner income tax?
                                                                                                                • When I click on PDF/PRINT it makes the invoice half size

                                                                                                                  When I click PDF / Print for my invoice in Zoho Books, the generated PDF appears at half size — everything is scaled down, including the logo, text, and layout. The content does not fill the page as it should. Could someone advise what causes Zoho Books
                                                                                                                • Search by contain letter in a column

                                                                                                                  Hello, everyone I need a filter function that searches by letter in a cell, and it should be a macro. To clarify further, if I have a column with several names and I chose a search cell and what I want is search by a single letter, for example, "a" then
                                                                                                                • Can add a colum to the left of the item in Zoho Books?

                                                                                                                  I would need to add a column to the left of the item column in Books. When i create custom fields, i can only display them to the right of the item.
                                                                                                                • AI Bot and Advanced Automation for WhatsApp

                                                                                                                  Most small businesses "live" on WhatsApp, and while Bigin’s current integration is helpful, users need more automation to keep up with volume. We are requesting features based on our customer Feedbacks AI Bot: For auto-replying to FAQs. Keyword Triggers:
                                                                                                                • 2025 Ask the Experts sessions wrap-up : Key highlights from the experts

                                                                                                                  Here is a rewind journey of our Ask the Experts (ATE) Sessions, where we brought you expert insights and practical best practices together in one place. This recap highlights the key takeaways, learnings, and best practices from all these sessions so
                                                                                                                • How to disable the edit option in subform

                                                                                                                  How to disable the edit option in subform
                                                                                                                • Move record from one custom module to another custom module

                                                                                                                  Is it possible to create a button or custom field that will transfer a record from one custom module to another? I already have the 'Leads' module used for the Sr. Sales department, once the deal is closed they convert it to the 'Accounts' module. I would like to create a 'Convert' button for a custom module ('Locations') for the department that finds locations for each account. Once the location is secured, I want to move the record to another custom module called 'Secured Locations'. It's basically
                                                                                                                • Notes Not Saving

                                                                                                                  Hello,  My notes are continuously not saving.  I make sure to save them, I know the process to save them.  It is not operator error.  I go back into a Leads profile a while later and do not see the previous notes that I have made.  I then have to go back and do unnecessary research that would have been in the notes in the first place.  Not a good experience and it is frustrating.  Slows me down and makes me do unnecessary work.  Please resolve.   As a quick heads up, deleting cookies is not a fix
                                                                                                                • Prefill form with CRM/Campaigns

                                                                                                                  I created a form in zForms and created prefill fields. I added this to the CRM and selected the fields so when sending from the CRM, the form works great. However, I want to use the same form in Campaigns and I want it to pull the data from CRM (which
                                                                                                                • Triggering a campaign automation from a Form

                                                                                                                  I used Forms to create a lead form that is accessed by a button on my website. The field information flows into the CRM. However, I am trying to figure out how to use Campaign automations to start a workflow (series of campaign emails) that is triggered
                                                                                                                • Employee Appraisal Applicability - Why is Date of Joining Hard-Coded?

                                                                                                                  In the new (to me, at least) Performance Appraisal Cycle wizard, it's possible to set criteria to determine for whom the appraisal process should apply. This makes sense on its face. However, one MUST use the Date of Joining criterion as a filter. Why
                                                                                                                • Formula fields

                                                                                                                  Zoho People now supports formula fields. This post illustrates it. Formula fields are fields whose value is calculated instead of being entered by the user. Using this, number, decimal and date manipulations can be done. The value of this field could be numeric or date depending on the output of the formula. In date manipulations, the result will be given in milliseconds, which you can format as per you need. The operators we support are +, - , *, /. Formula fields get recalculated automatically
                                                                                                                • Zobot operator information

                                                                                                                  I am working on a zobot that calls a plug to send data to our API. I want the response data from that API to be shown only to the operator and not the customer. I have tried writing this data out to the visitorCity field but nothing gets updated on the
                                                                                                                • Copy paste from word document deletes random spaces

                                                                                                                  Hello Dear Zoho Team, When copying from a word document into Notebook, often I face a problem of the program deleting random spaces between words, the document become terribly faulty, eventhough it is perfect in its original source document (and without
                                                                                                                • You are not a part of any org. So Campaigns permission(s) will not be granted.

                                                                                                                  I have an issue while calling the access token by my email "kessam@axisapp.com" please give me suggestions for resolving this problem!
                                                                                                                • Is it possible to use module field filters via URL parameters?

                                                                                                                  It would be really convenient if I could quickly link to a filter. For reference, this is the filter functionality I'm referring to: https://help.zoho.com/portal/en/kb/crm/customize-crm-account/advanced-filters/articles/advanced-filters For example: My
                                                                                                                • Transitioning FESCO Bill Project to Zoho Sheets and Integration Options

                                                                                                                  Hello Zoho Support, I'm considering transitioning my FESCO bill project from Google Sheets to Zoho Sheets and wanted to know if there are integration options to seamlessly migrate our existing work. You can view our platform here, any guidance would be
                                                                                                                • Credit card

                                                                                                                  Coming from another software, is it possible here to typically have credit card payment off, but independently per customer, click a box (possibly on the invoice when I go to to send) allowing that 1 allowed customer credit card payment, NOT ALL of my
                                                                                                                • Lightbox Pop-up form

                                                                                                                  I would like to embed my form using the lightbox pop up. I don't want it to load automatically. I want it to load when some clicks the button. I can see this option, however when I use the "show pop-up launch button" on the website, the button automatically
                                                                                                                • Lightbox Pop-up form

                                                                                                                  I would like to embed my form using the lightbox pop up. I don't want it to load automatically. I want it to load when some clicks the button. I can see this option, however when I use the "show pop-up launch button" on the website, the button automatically
                                                                                                                • Connecting Portals from different Zoho apps

                                                                                                                  Hi, I note that Zoho has functionality for customer portals for several of the Zoho apps, like CRM, Projects, Desk etc. Is there any way to connect these portals?  It would be great if we could give our customers access to a portal in which they could
                                                                                                                • Customer Management: #5 Never Let the Customer Slip

                                                                                                                  When Rahul started Knight's Watch Consulting, his focus was simple: deliver good work and keep clients happy. He offered one-time consulting projects, monthly advisory retainers and usage-based support for growing clients. Business was steady, and customers
                                                                                                                • Next Page