Kaizen #123 Data Synchronization from a third party application

Kaizen #123 Data Synchronization from a third party application

Welcome back to the Kaizen series!

In last week's post in the Kaizen series, we discussed one-way data synchronization from Zoho CRM to a legacy application, utilizing Zoho CRM's Bulk Read API and Notification API. This week, we will discuss data synchronization from a third party app to Zoho CRM.

The APIs available in Zoho CRM to insert, update or upsert records are :

1) Insert, Update, and Upsert Records APIs

These are synchronous APIs that are used to perform createupdate or upsert operations for up to 100 records in a Zoho CRM module. While the Insert Records API and Update Records API is used to create and update a record respectively, the  Upsert Records API is used to upsert (insert or update) a record depending on the availability of the record in your org . In Upsert Records API, the system checks for duplicate records based on the values in the duplicate_check_fields json array in the input json. If the record is already present, it gets updated. If not, the record is inserted. You will get the status of these operations immediately. Read more about the Upsert Records API in this Kaizen post.

2) Bulk Write API

 The Bulk Write API is an asynchronous API that is used to perform create/update/upsert operations for up to 25K records in a Zoho CRM module. A background job will be scheduled in Zoho CRM. You can check the status of the scheduled job using the Bulk Write Job Details API. Alternatively, you can configure a callback URL when you schedule the job. The details of the Bulk Write Job result are posted on the callback URL upon successful completion or failure of the job. This bulk write API involves the following steps:

To learn more about these steps in the Bulk Write API and about the Bulk Write API in general, refer to the Kaizen post on the Bulk Write API.


  Insert, Update, Upsert Records


Bulk Write API


Insert, update, or upsert up to 100 records per API call.

Insert, update, or upsert up to 25000 records per API call.
The response is available instantly.


The response to the Bulk Write API request will not be available immediately. You can check the status of the job by polling for it, or you can wait for the status to be available in the callback URL after the job has completed.

These APIs consume one credit per 10 records. 

Bulk Write API consumes 500 credits per API call. 
Input should be in JSON format

The input file should be in CSV format.




Let us now examine four different scenarios for record insertion in a module, specifically with varying quantities: 100, 2000, 5000, and 25000 records, and discover how these APIs perform in terms of number of API calls and credits consumed.


Number of Records

Insert,Update or Upsert Records API

Bulk Write API
100
1 API Call
10 API Credits
1 API Call
500 API Credits
2000
20 API Calls
200 API Credits
1 API Call
500 API Credits
5000
50 API Calls
500 API Credits
1 API Call
500 API Credits
25000
250 API Calls
2500 API Credits
1 API Call
500 API Credits

Clearly, if you have more than 5000 records, it is advisable to use the bulk write API, given that you are ok with the operation being run asynchronously. If you want instant results, you should use insert/update/upsert APIs. However, more API credits will be consumed in this case.

Data Sync

Consider an e-commerce company where the Order management team uses a legacy system and the Accounts team uses Zoho CRM. To ensure that the Accounts team works with near real-time data in Zoho CRM, data synchronization from the legacy application is essential. Assume that in Zoho CRM, there is a custom module named "Orders" that contains the details from the legacy system. Imagine that you are in charge of the data synchronization between Zoho CRM and the legacy system. How should you call these APIs to achieve this synchronization? The synchronization process involves two steps. 
Step 1: Initial Data Sync : Initial data sync from the legacy system to Zoho CRM.
Step 2: Subsequent Data Sync : Push any new orders or updated orders to Zoho CRM.

The solution we propose here is one of the many ways in which the data sync can be achieved.

This solution is built on the below three key points: 

1) Unique field in Order Module: Maintain a unique field "parent_id" in the "Orders"  module. The unique identifier of the orders in your legacy system should be mapped to this parent_id field. In create bulk write job, this can be achieved by the section field_mappings. For ensuring correct working of upsert operations, you can make use of "find_by" field inside resource JSON Array in the Create Bulk Write Job API Request.  You should provide the value of "find_by" as the unique field "parent_id" .  To read more about find_by field refer to Create Bulk Write Job page.

2) Sync Flag in the Orders table in legacy system: For the sync, maintain a flag in your legacy system orders table for indicating whether the record is synced with Zoho CRM or not. Once the sync is done, update the flag to  true. Whenever a record is modified or sync fails, the flag value should be updated to false. This ensures that in case a record that is synced gets updated during the initial sync process, this flag column helps us re-sync this record again.

3) Grouping of data to batches:  For initial sync, the data volume in the table will be huge. So they have to be batched in a group of 25k records. This grouping can be done based on sorting the table with a particular column & filtering records based on the sync column with criteria sync=false.

Step 1: Initial Data Sync 

Since you have multiple batches of 25k records, you have to iterate them one by one and send them to the Bulk Write API.

Initial Data Sync
For each batch of data, prepare them as a CSV file and initiate a bulk write Job. Next, check the status of the Bulk Write job, and once complete, look for any records that have been skipped during the process. If there are any, fix them, update the sync flag as false, and include them in the next round of data sync. Continue this loop until all records, including any that were missed, are processed. Take the next batch and repeat till all the batches are over.

Step 2: Subsequent Data Sync

The Order Management team continues to use the legacy application, leading to continuous creation or update of records in the Orders module.
It is not recommended to invoke Upsert Records API whenever an operation happens in Orders in the legacy system. Subsequent data syncs can be done in two ways: either by  (a)near real-time sync or (b) Interval based sync.


(a) Near Real-time Sync

 As we want near real-time data in Zoho CRM, it is ok for the records that are updated in the legacy system from last synchronization to be unavailable in Zoho CRM for 10 minutes.

Near Real-time Sync

Instead of making API calls to Zoho CRM on every data operation, store the data in a temporary data store like a DB table or Redis. Once 100 records are accumulated in that data-store, the Upsert Records API can be promptly invoked with all these record details and followed by emptying of the data store. This approach will lead to lower consumption of API credits.
There are chances that in a period of 10 mins, 100 record operations may not happen. Scheduler jobs or CRON jobs can be created to handle the records in the data store to update records to Zoho CRM. Once these records are handled, the data store should be emptied.

(b) Interval based synchronization

For interval based synchronization, schedule bulk write API calls at suitable intervals to upsert the Orders module in Zoho CRM. Whenever there is a record operation in the legacy table, update the sync flag to "false". At regular intervals, such as every 12 or 24 hours, the bulk write API or Upsert Records API can be employed depending on the volume of the data changes.

We hope you found this article useful. We will be back next week with another interesting topic. If you have any questions, write to us at support@zohocrm.com or let us know in the comment section. 


Related Links




    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                                Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
                                                              • Kaizen #226: Using ZRC in Client Script

                                                                Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this
                                                              • Kaizen #222 - Client Script Support for Notes Related List

                                                                Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how
                                                              • Kaizen #217 - Actions APIs : Tasks

                                                                Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.
                                                              • Kaizen #216 - Actions APIs : Email Notifications

                                                                Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • Note sync turn off

                                                                                                                  Hi, Is it possible to turn off notes sync between task notes and the parent module? (Account/Deal)
                                                                                                                • Basic Plan Active but Survey Creation Still Limited to 3 Surveys

                                                                                                                  I have an active Basic (Monthly) subscription (valid period: Dec 24, 2025 – Jan 24, 2026), but the system still limits survey creation to only 3 surveys, which matches Free plan behavior. The subscription appears active in Portal Information, however
                                                                                                                • 2025 in Review: Powering Field Services Forward—One Feature at a Time

                                                                                                                  As 2025 draws to a close, it’s time to pause and reflect on a year of meaningful progress at Zoho FSM. This year was all about listening closely to our customers and partners, building with purpose, and continuously improving the way field service teams
                                                                                                                • API Support for Creating Invoices with Batch-Tracked Items

                                                                                                                  Hi Zoho Community, I am working on an integration where we create invoices in ERPNext and push them to Zoho Books. I need to send batch-tracked items (batch numbers) when creating invoices. I could not find any reference in the Zoho Books API documentation.
                                                                                                                • New to automation - please help

                                                                                                                  Hi there! We are new to automations in Zoho. We built out one automation campaign and it seems to be firing off for some people but others not. Please advise what's the best practice or if anything we should tweak. Thank you so much!
                                                                                                                • Unable to Create Zoho Booking via the Book Appointment API

                                                                                                                  Its giving the below error {     "response": {         "errormessage": "Error setting value for the variable:customer_details\n null",         "status": "Error"     } } Request: POST Url: https://www.zohoapis.in/bookings/v1/json/appointment attached Zoho-oauthtoken
                                                                                                                • Task Details on task template

                                                                                                                  When creating a task template in settings that task details seem to be missing. Is it not possible to set the details of a task, such as the priority, type, reminder settings, and custom field values?
                                                                                                                • Building Toppings #3 -Testing and publishing Bigin toppings

                                                                                                                  Hey Biginners! In previous forum posts, we discussed what a Bigin topping is and explored the Bigin Developer Console's features. In this post, we're going to create a topping from scratch and discuss how to test and publish it. Our topping will automate
                                                                                                                • markdown files?

                                                                                                                  How can I import a markdown file into a note?
                                                                                                                • Mail to Zoho Notebook

                                                                                                                  In the Instuctiosn I faound as email add@notebook.app In my account I see add@eu.notebook.app What is correct please
                                                                                                                • Auto-Invite Users to Portals in Zoho CRM based on Conditions

                                                                                                                  Hello Everyone, You can now automate portal invitations in Zoho CRM with the new Auto-Invite users feature in Portal management. No more manually enabling portal access one by one. With this enhancement, you can automatically send invites for users to
                                                                                                                • Growth You can Count On: Zoho Billing's 2025 Journey

                                                                                                                  As we wrap up 2025, we’re reflecting on a year rooted in steady progress and meaningful growth for Zoho Billing. Every enhancement we have introduced was built to support scalable and reliable billing growth for your business. From launching the Enterprise
                                                                                                                • Need help with creating workflow with Bigin and Clickup

                                                                                                                  Can you please help me with this? I am trying to create a workflow in Zoho Flow wherein a bigin Pipeline deal moves to Decision Closing, it will create a task and subtasks in Clickup, under the condition/filter that the Deal name contains the word Social
                                                                                                                • [Webinar] Automate generation of wills, trusts, POAs, and other estate planning documents with Zoho Writer

                                                                                                                  Managing the lifecycle of the estate planning documents such as wills, trusts, and POAs, from client intake to final storage, can be complex and time-consuming. Join our live webinar to learn how Zoho Writer transforms this process by automating document
                                                                                                                • Create / Update Rule in TeamInbox when record is updated in ZohoCRM using flow?

                                                                                                                  I'm currently evaluating options and trying to work out if the possible scenario is possible; If a record in ZohoCRM is updated, take 2 of the fields from that - lets call them 'OrderRef' and 'OrderTo'. Using flow, if a record is added or updated, add
                                                                                                                • Plans to allow more columns of monitoring, and monitoring not only your own channels?

                                                                                                                  Are their any plans to allow more columns of monitoring, and monitoring not only your own channels? Here's why - I'm sure I'm not alone in that we sell other brands products, so not only am I interested in my own brand social channels, but also the social
                                                                                                                • Unwrapping the best practices for the Zoho Desk mobile app: Part 1

                                                                                                                  In focus: Empowering the field and frontline customer support representatives Imagine having to carry your computer everywhere to respond to customers, handle escalations, and update your process. This series presents you with efficient tips to handle
                                                                                                                • dealing with post dated cheques

                                                                                                                  Hi, can you help me please the best way to deal with this We sell an item of three months duration that is paid for with post dated cheques in monthly stages example - item is sold £3000 Cheque 1 is for £1000 due at time of sale (say Sept) cheque 2 is dated 25th of next month (Oct) cheque 3 is dated 25th of next month +1 (Nov) Now, with invoice number one it's simple - i send a standard invoice as usual But with the other two here's what i want zoho to do next month i want it to send an invoice on
                                                                                                                • Is there a way to sell in a practical method multiple subscriptions of the same product? i.e. domain names

                                                                                                                  In evaluating Zoho Billing, a hurdle to adopting it is that Zoho Billing does not seem designed to support businesses that sell multiple subscriptions of the same product. In our case, we need to sell and manage several domain names per client. Am I right
                                                                                                                • Bug Report: "Name A-Z" sorting fails when Note titles start with an Emoji

                                                                                                                  Hello, I recently migrated from Evernote (~2600 notes across 23 notebooks). I use emojis at the start of my note titles for visual organization. The Issue: When I set the view to Sort by: Name (A to Z), the sorting logic breaks for any note starting with
                                                                                                                • Copy & Paste not working

                                                                                                                  I could create and save 10 new documents but when I go to copy and paste content (just text) into them, only 1 of 10 would save the content. It would appear to save the content but when I go back into the documents, they are empty.
                                                                                                                • Currency for Petty Cash

                                                                                                                  Currencies for Petty Cash should be available
                                                                                                                • editing/applying online payments

                                                                                                                  We have customers who pay part or all of an invoice and then cancel their service and want the payment as a credit to future work. It would be ideal if we could make it an excess payment and then void the invoice. I can't make it an excess payment because
                                                                                                                • Contacts per department

                                                                                                                  Hello, Is it possible to limit Contacts to a Department? Thanks
                                                                                                                • This site can’t be reached mail.zoho.com took too long to respond

                                                                                                                  In my office at any system, we couldnt able to login zoho email. it shows " This site can’t be reached mail.zoho.com took too long to respond". please fix it soon.
                                                                                                                • Can send email from zoho mail, but can't receive any.

                                                                                                                  Hi, My domain is sattvameditationresort.com. I've updated MX records with those of Zoho. But i can't send any mails to this email id from gmail. I have checked the MX status with MXTOOLS, its showing the correct entry either. The error is as shown below:
                                                                                                                • My domain did not activate

                                                                                                                  Hi, my domain (apsaindustrial.com.ar) did not activate, and the phone verification message never arrived. Please would you solve this problem? Thanks.
                                                                                                                • Host not found?

                                                                                                                  Howdy! So i'm trying to add my custom domain for with the mail server. I have 100% control of my DNS and have tried every single option (TXT, CNAME, and even HTML) multiple times, ensuring i did it properly, to no luck. I get the same error message every
                                                                                                                • Auto End Chats After Customer Inactivity

                                                                                                                  Hello Everyone, I may be missing something, but is there a way to auto-end chats when a customer has been inactive for an amount of time? I have several operators (we use a shared set of account with schedules for people... that's another story) but at
                                                                                                                • Report on opportunities showing only the last note added.

                                                                                                                  Hi I need to create a report that shows the most recent note added to each opportunity. This is so management can see what the latest update is according to the assigned salesperson. One workaround is to use the status field but this implies added manual work and mistakes as the salesperson would have to copy the existing status to a note before adding the latest status... otherwise the activity history would be lost. My current workaround is a report on Notes with Opportunities as the related module.
                                                                                                                • Please add Zelle as an online payment option

                                                                                                                  Hello, I would like to request Zelle be added to the online payment service providers for Zoho Invoice. Considering how ubiquitous Zelle has become as a way to pay people via the major banking institutions, I feel like many freelancers would benefit from
                                                                                                                • Getting Project Template List using the REST API

                                                                                                                  I am trying to confirm that I can use the REST API to create a project using a project template. The API documentation indicates this is possible by providing the Template ID, but it is not clear at all how to get a list of available Project Templates
                                                                                                                • How to display the CONTACT ID in the Contact page

                                                                                                                  Hi, I've seen this conversation below and it is exactly the same question I'm raising now, but unfortunately the last message is not solved https://help.zoho.com/portal/community/topic/show-contact-id-while-editing-contact-form I need to show the ContactId and I don't know how to do this. The last message included in the conversaton shows the way but not it is not completed. "I am sorry by default we do not have the option to show the record ID for the contacts in the field in a record. When you
                                                                                                                • How to change margins in templates

                                                                                                                  Hi, I'm new to Zoho Campaigns, and am trying to figure out how to adjust the margins within an email template.  The default margins make the text go all the way to the edges of the page.  Please advise.
                                                                                                                • Create Canvas list view templates from images powered by Zia

                                                                                                                  Currently available for all paid editions of Zoho CRM in the US, EU, IN, JP & CN DCs. Designing a personalized CRM interface just got even easier. In today’s fast-evolving digital landscape, AI is transforming the way we work by automating complex tasks
                                                                                                                • Painfully Slow Zoho mail

                                                                                                                  Since yesterday Zoho Mail seems to have starting functioning very slowly and having a few bugs. It's slow to open mails, slow to send, slow to change between email accounts. Sometimes clicking on a particular folder (eg Sent folder) stops working and
                                                                                                                • "Wrong password or login" Problem to configure Zoho on MAIL App on my Macbook

                                                                                                                  Hi, I'm having problems to configure my e-mail on my MAIL App(Macbook pro). My e-mail is hari@trespontoum.net Actually was working perfectly, and still working on my Iphone. My MAIL App prompt me that my login or password is wrong. I tried to change 3
                                                                                                                • "User already exist in your org"

                                                                                                                  Hello, I've just read a discussion about this issue, which didn't solve my problem. I'm trying to add the following emails: sales@kiss-my-boutique.co.uk returns@kiss-my-boutique.co.uk orders@kiss-my-boutique.co.uk I'm getting an error message each time I try and add them. None of them are primary or secondary emails and none of them have been created as users before. I know this as when I try and login and do 'forgot my password' all I get is an error message saying 'user invalid'. Please advise.
                                                                                                                • Operation Not Permitted

                                                                                                                  Hi, I have problem in adding user after verifying the domain but it seems like error appeared which is "operation not permitted". For your information, I had delete other domain before did it.
                                                                                                                • Email forwarding setup fails

                                                                                                                  I'm trying to set up email forwarding from my Zoho email to my gmail address. I followed the directions to set up email forwarding here: https://www.zoho.com/mail/help/email-forwarding.html. I did only steps 1-6. After doing this, rather than setting
                                                                                                                • Next Page