Kaizen #123 Data Synchronization from a third party application

Kaizen #123 Data Synchronization from a third party application

Welcome back to the Kaizen series!

In last week's post in the Kaizen series, we discussed one-way data synchronization from Zoho CRM to a legacy application, utilizing Zoho CRM's Bulk Read API and Notification API. This week, we will discuss data synchronization from a third party app to Zoho CRM.

The APIs available in Zoho CRM to insert, update or upsert records are :

1) Insert, Update, and Upsert Records APIs

These are synchronous APIs that are used to perform createupdate or upsert operations for up to 100 records in a Zoho CRM module. While the Insert Records API and Update Records API is used to create and update a record respectively, the  Upsert Records API is used to upsert (insert or update) a record depending on the availability of the record in your org . In Upsert Records API, the system checks for duplicate records based on the values in the duplicate_check_fields json array in the input json. If the record is already present, it gets updated. If not, the record is inserted. You will get the status of these operations immediately. Read more about the Upsert Records API in this Kaizen post.

2) Bulk Write API

 The Bulk Write API is an asynchronous API that is used to perform create/update/upsert operations for up to 25K records in a Zoho CRM module. A background job will be scheduled in Zoho CRM. You can check the status of the scheduled job using the Bulk Write Job Details API. Alternatively, you can configure a callback URL when you schedule the job. The details of the Bulk Write Job result are posted on the callback URL upon successful completion or failure of the job. This bulk write API involves the following steps:

To learn more about these steps in the Bulk Write API and about the Bulk Write API in general, refer to the Kaizen post on the Bulk Write API.


  Insert, Update, Upsert Records


Bulk Write API


Insert, update, or upsert up to 100 records per API call.

Insert, update, or upsert up to 25000 records per API call.
The response is available instantly.


The response to the Bulk Write API request will not be available immediately. You can check the status of the job by polling for it, or you can wait for the status to be available in the callback URL after the job has completed.

These APIs consume one credit per 10 records. 

Bulk Write API consumes 500 credits per API call. 
Input should be in JSON format

The input file should be in CSV format.




Let us now examine four different scenarios for record insertion in a module, specifically with varying quantities: 100, 2000, 5000, and 25000 records, and discover how these APIs perform in terms of number of API calls and credits consumed.


Number of Records

Insert,Update or Upsert Records API

Bulk Write API
100
1 API Call
10 API Credits
1 API Call
500 API Credits
2000
20 API Calls
200 API Credits
1 API Call
500 API Credits
5000
50 API Calls
500 API Credits
1 API Call
500 API Credits
25000
250 API Calls
2500 API Credits
1 API Call
500 API Credits

Clearly, if you have more than 5000 records, it is advisable to use the bulk write API, given that you are ok with the operation being run asynchronously. If you want instant results, you should use insert/update/upsert APIs. However, more API credits will be consumed in this case.

Data Sync

Consider an e-commerce company where the Order management team uses a legacy system and the Accounts team uses Zoho CRM. To ensure that the Accounts team works with near real-time data in Zoho CRM, data synchronization from the legacy application is essential. Assume that in Zoho CRM, there is a custom module named "Orders" that contains the details from the legacy system. Imagine that you are in charge of the data synchronization between Zoho CRM and the legacy system. How should you call these APIs to achieve this synchronization? The synchronization process involves two steps. 
Step 1: Initial Data Sync : Initial data sync from the legacy system to Zoho CRM.
Step 2: Subsequent Data Sync : Push any new orders or updated orders to Zoho CRM.

The solution we propose here is one of the many ways in which the data sync can be achieved.

This solution is built on the below three key points: 

1) Unique field in Order Module: Maintain a unique field "parent_id" in the "Orders"  module. The unique identifier of the orders in your legacy system should be mapped to this parent_id field. In create bulk write job, this can be achieved by the section field_mappings. For ensuring correct working of upsert operations, you can make use of "find_by" field inside resource JSON Array in the Create Bulk Write Job API Request.  You should provide the value of "find_by" as the unique field "parent_id" .  To read more about find_by field refer to Create Bulk Write Job page.

2) Sync Flag in the Orders table in legacy system: For the sync, maintain a flag in your legacy system orders table for indicating whether the record is synced with Zoho CRM or not. Once the sync is done, update the flag to  true. Whenever a record is modified or sync fails, the flag value should be updated to false. This ensures that in case a record that is synced gets updated during the initial sync process, this flag column helps us re-sync this record again.

3) Grouping of data to batches:  For initial sync, the data volume in the table will be huge. So they have to be batched in a group of 25k records. This grouping can be done based on sorting the table with a particular column & filtering records based on the sync column with criteria sync=false.

Step 1: Initial Data Sync 

Since you have multiple batches of 25k records, you have to iterate them one by one and send them to the Bulk Write API.

Initial Data Sync
For each batch of data, prepare them as a CSV file and initiate a bulk write Job. Next, check the status of the Bulk Write job, and once complete, look for any records that have been skipped during the process. If there are any, fix them, update the sync flag as false, and include them in the next round of data sync. Continue this loop until all records, including any that were missed, are processed. Take the next batch and repeat till all the batches are over.

Step 2: Subsequent Data Sync

The Order Management team continues to use the legacy application, leading to continuous creation or update of records in the Orders module.
It is not recommended to invoke Upsert Records API whenever an operation happens in Orders in the legacy system. Subsequent data syncs can be done in two ways: either by  (a)near real-time sync or (b) Interval based sync.


(a) Near Real-time Sync

 As we want near real-time data in Zoho CRM, it is ok for the records that are updated in the legacy system from last synchronization to be unavailable in Zoho CRM for 10 minutes.

Near Real-time Sync

Instead of making API calls to Zoho CRM on every data operation, store the data in a temporary data store like a DB table or Redis. Once 100 records are accumulated in that data-store, the Upsert Records API can be promptly invoked with all these record details and followed by emptying of the data store. This approach will lead to lower consumption of API credits.
There are chances that in a period of 10 mins, 100 record operations may not happen. Scheduler jobs or CRON jobs can be created to handle the records in the data store to update records to Zoho CRM. Once these records are handled, the data store should be emptied.

(b) Interval based synchronization

For interval based synchronization, schedule bulk write API calls at suitable intervals to upsert the Orders module in Zoho CRM. Whenever there is a record operation in the legacy table, update the sync flag to "false". At regular intervals, such as every 12 or 24 hours, the bulk write API or Upsert Records API can be employed depending on the volume of the data changes.

We hope you found this article useful. We will be back next week with another interesting topic. If you have any questions, write to us at support@zohocrm.com or let us know in the comment section. 


Related Links




    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                                Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
                                                              • Kaizen #226: Using ZRC in Client Script

                                                                Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this
                                                              • Kaizen #222 - Client Script Support for Notes Related List

                                                                Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how
                                                              • Kaizen #217 - Actions APIs : Tasks

                                                                Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.
                                                              • Kaizen #216 - Actions APIs : Email Notifications

                                                                Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • Allow Text within a Formula

                                                                                                                  Hi, I would like to be able to use this for others things like taking an existing Date Field and copying its value, so by entering getDay(Date)&"-"&getMonth(Date)&"-"&getYear(Date) it results in 01-02-2026. And then when the Date is changed so is this
                                                                                                                • Change Number Field to Decimal Field

                                                                                                                  Hi, It would be nice to be able to change the field type without having to delete it and create a new one, messing up the database and history. Thanks Dan
                                                                                                                • Pipeline.Company Name field shows up as numbers! [Bigin Developer Console > Component > URL]

                                                                                                                  Hi there, I am setting up to invoke URL to send infromation zoho bigin > zoho forms with company name pre-fill in the form. however when I use : ${Pipelines.Company Name} field it shows up as a string of number instead of words. Help.
                                                                                                                • Add specific field value to URL

                                                                                                                  Hi Everyone. I have the following code which is set to run from a subform when the user selects a value from a lookup field "Plant_Key" the URL opens a report but i want the report to be filtered on the matching field/value. so in the report there is
                                                                                                                • How to Move Behavior, Acquisition, Polls & Forms Data from Zoho PageSense to Zoho Analytics?

                                                                                                                  Hi Zoho Community, I'm looking for a way to transfer data from Zoho PageSense to Zoho Analytics, specifically: Behavioral data (clicks, scrolls, heatmaps, etc.) Acquisition data (traffic sources, campaigns, etc.) Polls and forms data As far as I can tell:
                                                                                                                • Zoho Social - Feature Request - Non-US Date Format

                                                                                                                  Hi Social Team, I have noticed that there is no option to change the date format from US mm/dd/yyyy to others like dd/mm/yyyy. It would be great to see this added as the platform matures. Thanks for considering this feedback.
                                                                                                                • In arattai received message can't be deleted

                                                                                                                  The issue has been noticed in following: arattai app (Android) arattai app (Window) arattai web While the message posted by me may be deleted, the ones received from others can't be. The item <Delete> change to <Report> when the message is a received
                                                                                                                • Zoho CRM Community Digest - December 2025 | Part 1

                                                                                                                  Hello Everyone! In the first half of December, Zoho CRM rolled out auto-invite for portals, smarter CPQ with Zia suggestions and price rules, and Query Workbench for faster query building. We also highlight other noteworthy conversations you shouldn’t
                                                                                                                • Zoho Flow Credits

                                                                                                                  Hi , I would like to ask the reason why every time I added plus credit but few days later I will return back to default? (as below I add credit to 3000 but today It change back to 1000) Most important is, when the credit fully used, not any reminder to
                                                                                                                • Custom Module Send Email not filling To automatically after adding Email field also

                                                                                                                  I create custom module in zohoCRM but problem is in custom module when i click send email there in popup To is filling automatic its coming empty which i need to manually add an email there. How can I fix it? I already added the Email field there, but
                                                                                                                • Introducing Intake Forms

                                                                                                                  We are excited to announce the release of Intake Forms, a new feature in Zoho Contracts designed to make contract requests simple, structured, and efficient. Intake Forms allow organization members to request new contracts through a web form instead of
                                                                                                                • Manage control over Microsoft Office 365 integrations with profile-based sync permissions

                                                                                                                  Greetings all, Previously, all users in Zoho CRM had access to enable Microsoft integrations (Calendar, Contacts, and Tasks) in their accounts, regardless of their profile type. Users with administrator profiles can now manage profile-based permissions
                                                                                                                • eIDAS 2.0: What's changed for digital trust in Europe, and where Zoho Sign stands

                                                                                                                  Hi there! It's that time of year when many of us get our ducks in a row. A new year often makes us reassess priorities, and for businesses in the EU, it means taking a closer look at how digital identities and electronic signatures work across borders,
                                                                                                                • Directly Edit, Filter, and Sort Subforms on the Details Page

                                                                                                                  Hello everyone, As you know, subforms allow you to associate multiple line items with a single record, greatly enhancing your data organization. For example, a sales order subform neatly lists all products, their quantities, amounts, and other relevant
                                                                                                                • Add Israel & Jewish Holidays to Zoho People Holidays Gallery

                                                                                                                  Greetings, We hope you are doing well. This feature request is related to Zoho People - please don't move it to zoho one! We are writing to request an enhancement to the Holidays Gallery in Zoho People. Currently, there are several holidays available,
                                                                                                                • Any update on much requested feature, to delete attachments without deleting the e-mail body?

                                                                                                                  People have been requesting the ability to delete e-mail attachments without deleting the e-mail for more than ten years now. The latest I see is marked "Working On It" and a year ago it was supposedly being added, see here: https://help.zoho.com/portal/en/community/topic/is-there-a-way-to-delete-mail-attachments-without-deleting-the-text
                                                                                                                • How to delete attachments form Zoho mail accounts

                                                                                                                  I can't find a way to delete attachments from Zoho mail messages, either individually or in bulk. Searches here are providing conflicting results and often talk about workspace, whereas I am only interested in how to delete attachments that are seen with
                                                                                                                • Unable to fetch ticket by custom field value

                                                                                                                  I'm trying to set up a Flow to fetch a ticket based on a custom field value. This seems like it should be pretty straightforward, but it's not working for me. I keep getting an error saying Zoho Desk says "Extra query parameter 'cf_creator_record_id'
                                                                                                                • Zoho Inventory - Composite Items - Assembly - Single Line Item Quantity of One

                                                                                                                  Hi Zoho Inventory Team, Please consider relaxing the system rules which prevent an assembly items from consisting of a single line item and outputting a quantity of 1. A client I'm currently working with sells cosmetics and offers testers of their products
                                                                                                                • What's New in Zoho Inventory | Q2 2025

                                                                                                                  Hello Customers, The second quarter have been exciting months for Zoho Inventory! We’ve introduced impactful new features and enhancements to help you manage inventory operations with even greater precision and control. While we have many more exciting
                                                                                                                • Zoho POS App Hanging Issue – Sales Becoming Difficult

                                                                                                                  The Zoho POS app frequently hangs and becomes unresponsive during billing, making it very difficult to complete sales smoothly. This commonly happens while adding items, during checkout, or at payment time, especially during peak hours. These issues cause
                                                                                                                • Writing by Hand in "Write" Notes

                                                                                                                  Hi there! I just downloaded this app a few moments ago, and I was wondering if there was a way to write things by hand in "Write" mode instead of just typing in the keyboard. It would make things a bit more efficient for me in this moment. Thanks!
                                                                                                                • Zoho Mail Verification Following DNS Switch from Cloudflare to Hosting Provider

                                                                                                                  I initially configured my domain’s ( https://roblozapk.com/ ) email with Zoho Mail while using Cloudflare to manage my DNS records such as MX, SPF, and DKIM. At that time, all email services were working correctly without any issues. Recently, I removed
                                                                                                                • "Spreadsheet Mode" for Fast Bulk Edits

                                                                                                                  One of the challenges with using Zoho Inventory is when bulk edits need to be done via the UI, and each value that needs to be changed is different. A very common use case here is price changes. Often, a price increase will need to be implemented, and
                                                                                                                • SAML in Zoho One vs Zoho Accounts

                                                                                                                  What is the difference between setting up SAML in Zoho Accounts: https://help.zoho.com/portal/en/kb/accounts/manage-your-organization/saml/articles/configure-saml-in-zoho-accounts ... vs SAML in Zoho One?: https://help.zoho.com/portal/en/kb/one/admin-guide/custom-authentication/setting-up-custom-authentication-with-popular-idps/articles/zohoone-customauthentication-azure
                                                                                                                • Deactivated Zoho One account can sign in

                                                                                                                  I am concerned by the fact that deactivated users in Zoho One have the ability to sign in even after their account has been deactivated (not deleted). these inactive identities have no access to individual Zoho apps or data. based on my experience they
                                                                                                                • How can I reset the password for a user in Zoho Projects

                                                                                                                  We need to reset the password for a user in Zoho Projects. I am the admin portal owner and there was nothing to be found to do this. very confusing.
                                                                                                                • How can I add a comment to an existing ticket via API?

                                                                                                                  I need to add comments/notes to the history of an existing ticket using the API without overwriting the original ticket description. Thanks!
                                                                                                                • Marketer's Space: Proven tips to improve open rates – Part III

                                                                                                                  Hello Marketers! Welcome back to another post in Marketer's Space! This is the final post in the "open rate series". In the first and second parts, we discussed topics ranging from sender domains to pre-headers—but we're not done yet. A few more important
                                                                                                                • Attention Deluge Developers: Important Update Regarding "Send Email Deluge Task"

                                                                                                                  Hi Deluge Users, We hope this message finds you well. We would like to inform you about a recent update on the Send emails deluge task. If you are using this deluge task (SalesIQ Scripts) for your Zobot, widgets, or form controllers to send emails, please
                                                                                                                • Zoho Social - Feature Request - Reviewer Role

                                                                                                                  Hi Social Team, I've come across this with a couple of clients, where they need a role which can review and comment on posts but who has no access to create content. This is a kind of reviewer role. They just need to be able to see what content is scheduled
                                                                                                                • Zoho Books/Inventory - Update Marketplace Sales Order via API

                                                                                                                  Hi everyone, Does anyone know if there is a way to update Sales Orders created from a marketplace intigration (Shopify in this case) via API? I'm trying to cover a scenario where an order is changed on the Shopify end and the changes must be reflected
                                                                                                                • Zoho Inventory / Finance Suite - Add feature to prevent duplicate values in Item Unit field

                                                                                                                  I've noticed that a client has 2 values the same in the Unit field on edit/create Items. This surprised me as why would you have 2 units with the same name. Please consider adding a feature which prevents this as it seems to serve no purpose.
                                                                                                                • Kaizen #224 - Quote-driven Deal Reconciliation Using Zoho CRM Functions and Automation

                                                                                                                  Hello everyone! Welcome back to another instalment in the Kaizen series. This post covers quote-driven deal reconciliation, emphasizing Functions and Automation to address practical sales challenges. Business Challenge Sales organizations often mark deals
                                                                                                                • BARCODE PICKLIST

                                                                                                                  Hello! Does anyone know how the Picklist module works? I tried scanning the barcode using the UPC and EAN codes I added to the item, but it doesn’t work. Which barcode format does this module use for scanning?
                                                                                                                • Zoho Inventory - Allow Update of Marketplace Generated Sales Orders via API

                                                                                                                  Hi Inventory Team, I was recently asked by a client to create an automation which updated a Zoho Inventory Sales Order if a Shopify Order was updated. I have created the script but I found that the request is blocked as the Sales Order was generated by
                                                                                                                • How do I create an update to the Cost Price from landed costs?

                                                                                                                  Hi fellow Zoho Inventory battlers, I am new to Zoho inventory and was completely baffled to find that the cost price of products does not update when a new purchase order is received. The cost price is just made up numbers I start with when the product
                                                                                                                • VoC in Zoho CRM is now data savvy: Explore response drilldown, summary components and participation in CRM criteria

                                                                                                                  VoC has all the goods when it comes to customer intelligence—which is why we're constantly enhancing it. We recently added the following: A customer drilldown component that shows you the list of prospects and customers behind a chart's attribute Expanded
                                                                                                                • Zoho Browser??

                                                                                                                  hai guys, this sounds awkward but can v get a ZOHO BROWSER same as zoho writer, etc. where i can browse websites @ home and continue browsing the same websites @ my office, as v have the option in Firefox, once i save and close the browser and again when i open it i will be getting the same sites. If u people r not clear with my explanation, plz let me know. Thanks, Sandeep  
                                                                                                                • Let’s Talk Recruit: LinkedIn Cheatsheet

                                                                                                                  Welcome to the first edition of Let’s Talk Recruit for 2026. We are kicking off the year by revisiting one of the most asked about topics in Zoho Recruit. How our LinkedIn integrations actually work and how recruiters can get the most value from them.
                                                                                                                • Next Page