Kaizen #123 Data Synchronization from a third party application

Kaizen #123 Data Synchronization from a third party application

Welcome back to the Kaizen series!

In last week's post in the Kaizen series, we discussed one-way data synchronization from Zoho CRM to a legacy application, utilizing Zoho CRM's Bulk Read API and Notification API. This week, we will discuss data synchronization from a third party app to Zoho CRM.

The APIs available in Zoho CRM to insert, update or upsert records are :

1) Insert, Update, and Upsert Records APIs

These are synchronous APIs that are used to perform createupdate or upsert operations for up to 100 records in a Zoho CRM module. While the Insert Records API and Update Records API is used to create and update a record respectively, the  Upsert Records API is used to upsert (insert or update) a record depending on the availability of the record in your org . In Upsert Records API, the system checks for duplicate records based on the values in the duplicate_check_fields json array in the input json. If the record is already present, it gets updated. If not, the record is inserted. You will get the status of these operations immediately. Read more about the Upsert Records API in this Kaizen post.

2) Bulk Write API

 The Bulk Write API is an asynchronous API that is used to perform create/update/upsert operations for up to 25K records in a Zoho CRM module. A background job will be scheduled in Zoho CRM. You can check the status of the scheduled job using the Bulk Write Job Details API. Alternatively, you can configure a callback URL when you schedule the job. The details of the Bulk Write Job result are posted on the callback URL upon successful completion or failure of the job. This bulk write API involves the following steps:

To learn more about these steps in the Bulk Write API and about the Bulk Write API in general, refer to the Kaizen post on the Bulk Write API.


  Insert, Update, Upsert Records


Bulk Write API


Insert, update, or upsert up to 100 records per API call.

Insert, update, or upsert up to 25000 records per API call.
The response is available instantly.


The response to the Bulk Write API request will not be available immediately. You can check the status of the job by polling for it, or you can wait for the status to be available in the callback URL after the job has completed.

These APIs consume one credit per 10 records. 

Bulk Write API consumes 500 credits per API call. 
Input should be in JSON format

The input file should be in CSV format.




Let us now examine four different scenarios for record insertion in a module, specifically with varying quantities: 100, 2000, 5000, and 25000 records, and discover how these APIs perform in terms of number of API calls and credits consumed.


Number of Records

Insert,Update or Upsert Records API

Bulk Write API
100
1 API Call
10 API Credits
1 API Call
500 API Credits
2000
20 API Calls
200 API Credits
1 API Call
500 API Credits
5000
50 API Calls
500 API Credits
1 API Call
500 API Credits
25000
250 API Calls
2500 API Credits
1 API Call
500 API Credits

Clearly, if you have more than 5000 records, it is advisable to use the bulk write API, given that you are ok with the operation being run asynchronously. If you want instant results, you should use insert/update/upsert APIs. However, more API credits will be consumed in this case.

Data Sync

Consider an e-commerce company where the Order management team uses a legacy system and the Accounts team uses Zoho CRM. To ensure that the Accounts team works with near real-time data in Zoho CRM, data synchronization from the legacy application is essential. Assume that in Zoho CRM, there is a custom module named "Orders" that contains the details from the legacy system. Imagine that you are in charge of the data synchronization between Zoho CRM and the legacy system. How should you call these APIs to achieve this synchronization? The synchronization process involves two steps. 
Step 1: Initial Data Sync : Initial data sync from the legacy system to Zoho CRM.
Step 2: Subsequent Data Sync : Push any new orders or updated orders to Zoho CRM.

The solution we propose here is one of the many ways in which the data sync can be achieved.

This solution is built on the below three key points: 

1) Unique field in Order Module: Maintain a unique field "parent_id" in the "Orders"  module. The unique identifier of the orders in your legacy system should be mapped to this parent_id field. In create bulk write job, this can be achieved by the section field_mappings. For ensuring correct working of upsert operations, you can make use of "find_by" field inside resource JSON Array in the Create Bulk Write Job API Request.  You should provide the value of "find_by" as the unique field "parent_id" .  To read more about find_by field refer to Create Bulk Write Job page.

2) Sync Flag in the Orders table in legacy system: For the sync, maintain a flag in your legacy system orders table for indicating whether the record is synced with Zoho CRM or not. Once the sync is done, update the flag to  true. Whenever a record is modified or sync fails, the flag value should be updated to false. This ensures that in case a record that is synced gets updated during the initial sync process, this flag column helps us re-sync this record again.

3) Grouping of data to batches:  For initial sync, the data volume in the table will be huge. So they have to be batched in a group of 25k records. This grouping can be done based on sorting the table with a particular column & filtering records based on the sync column with criteria sync=false.

Step 1: Initial Data Sync 

Since you have multiple batches of 25k records, you have to iterate them one by one and send them to the Bulk Write API.

Initial Data Sync
For each batch of data, prepare them as a CSV file and initiate a bulk write Job. Next, check the status of the Bulk Write job, and once complete, look for any records that have been skipped during the process. If there are any, fix them, update the sync flag as false, and include them in the next round of data sync. Continue this loop until all records, including any that were missed, are processed. Take the next batch and repeat till all the batches are over.

Step 2: Subsequent Data Sync

The Order Management team continues to use the legacy application, leading to continuous creation or update of records in the Orders module.
It is not recommended to invoke Upsert Records API whenever an operation happens in Orders in the legacy system. Subsequent data syncs can be done in two ways: either by  (a)near real-time sync or (b) Interval based sync.


(a) Near Real-time Sync

 As we want near real-time data in Zoho CRM, it is ok for the records that are updated in the legacy system from last synchronization to be unavailable in Zoho CRM for 10 minutes.

Near Real-time Sync

Instead of making API calls to Zoho CRM on every data operation, store the data in a temporary data store like a DB table or Redis. Once 100 records are accumulated in that data-store, the Upsert Records API can be promptly invoked with all these record details and followed by emptying of the data store. This approach will lead to lower consumption of API credits.
There are chances that in a period of 10 mins, 100 record operations may not happen. Scheduler jobs or CRON jobs can be created to handle the records in the data store to update records to Zoho CRM. Once these records are handled, the data store should be emptied.

(b) Interval based synchronization

For interval based synchronization, schedule bulk write API calls at suitable intervals to upsert the Orders module in Zoho CRM. Whenever there is a record operation in the legacy table, update the sync flag to "false". At regular intervals, such as every 12 or 24 hours, the bulk write API or Upsert Records API can be employed depending on the volume of the data changes.

We hope you found this article useful. We will be back next week with another interesting topic. If you have any questions, write to us at support@zohocrm.com or let us know in the comment section. 


Related Links




    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                                Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
                                                              • Kaizen #226: Using ZRC in Client Script

                                                                Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this
                                                              • Kaizen #222 - Client Script Support for Notes Related List

                                                                Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how
                                                              • Kaizen #217 - Actions APIs : Tasks

                                                                Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.
                                                              • Kaizen #216 - Actions APIs : Email Notifications

                                                                Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • Dashboard

                                                                                                                  I’m currently working on a project using the Zoho IoT Cloud with an ESP32 and MQTT. I’ve successfully connected the ESP32 to Wi-Fi and MQTT, and the data from the ESP32 is displayed in the telemetry section of the cloud. However, when I try to create
                                                                                                                • Can i connect 2 instagram accounts to 1 brand?

                                                                                                                  Can i connect 2 instagram accounts to 1 brand? Or Do i need to create 2 brands for that? also under what subscription package will this apply?
                                                                                                                • Zoho Forms - Improved Sub-Forms

                                                                                                                  Hi Forms team, I'm helping a client move from JotForms to Zoho Forms and I've found another opportunity for improvement. In the below screenshot, JotForm left and Zoho Forms right. The Zoho Forms Sub-Form is quite a poor visually. There is no way to make
                                                                                                                • What's New in Zoho Billing | Q3 2025

                                                                                                                  Hello everyone, We are excited to share the latest set of updates and enhancements made to Zoho Billing in Q3 2025. From the latest GST updates to multi-level discounting, Zia-powered report summaries, and customizable web forms, these updates are designed
                                                                                                                • How to Delete Personal Account Linked with My Mobile Number in past or by someone else

                                                                                                                  How to Delete Account Created with My Mobile Number in past or by someone else This is creating issues in making or sync with my credentials mobile and email address..
                                                                                                                • WhatsApp Business Calling API

                                                                                                                  Dear Zoho SalesIQ Team, I would like to request a feature that allows users to call WhatsApp numbers directly via Zoho SalesIQ. This integration would enable sending and receiving calls to and from WhatsApp numbers over the internet, without the need
                                                                                                                • Does Zoho Learn integrate with Zoho Connect,People,Workdrive,Project,Desk?

                                                                                                                  Can we propose Zoho LEarn as a centralised Knowledge Portal tool that can get synched with the other Zoho products and serve as a central Knowledge repository?
                                                                                                                • Marketer's Space - Going beyond basics: Smarter ecommerce marketing with Zoho Campaigns

                                                                                                                  Hello Marketers, Welcome back to this week's Marketer's Space. In the last post, we discussed the basics of email marketing and how to get started with email marketing in ecommerce. In this part, we'll dive much deeper into some other advanced features
                                                                                                                • Connecting two modules - phone number

                                                                                                                  Hi, I’d like some guidance on setting up an automation in Zoho CRM that links records between the Leads module and a custom module called Customer_Records whenever the phone numbers match. Here’s what I’m trying to achieve: When a new Lead is created
                                                                                                                • Zoho Marketing Automation 2.0 - Landing Page function not working

                                                                                                                  Dear Zoho Team, I am working on implementing Zoho Marketing Automation 2.0, and am now looking into the section "Lead Generation". If I open the "Landing Pages" section, I immediately get an Error code: Error: internal error occurred. Can you help me
                                                                                                                • Unable to use Sign "You have entereed some invalid characters"

                                                                                                                  Unable to use Sign "You have entered some invalid characters" I do not see any invalid characters. The text in "Leave a Note" is plain text which I entered directly into the field. See attached screenshot
                                                                                                                • cannot login from outlook on mobile

                                                                                                                  Hi there,   I tried to set up my email accounts on my iphone using outlook, but it always return email address and password error. Please find the attachment for the screenshot. I am pretty sure that I typed in the correct email address and password and the imap has been enabled from zoho admin account. I can configure the email address on iphone default mail service as well as outlook on pc, the problem only occurs on the outlook app on mobile. I do need to use this app instead of the iphone default
                                                                                                                • Elevate your CX delivery using CommandCenter 2.0: Simplified builder; seamless orchestration

                                                                                                                  Most businesses want to create memorable customer experiences—but they often find it hard to keep them smooth, especially as they grow. To achieve a state of flow across their processes, teams often stitch together a series of automations using Workflow
                                                                                                                • Messages not displayed from personal LinkedIn profile

                                                                                                                  Hello. I connected both our company profile and my personal profile to Zoho social. I do see all messages from our company page but none from my private page. not even the profile is being added on top to to switch between company or private profile,
                                                                                                                • "Performed changes in the query is not allowed due to following reason" when adding columns or reordering data

                                                                                                                  I'm trying to make changes to a query but every time i try to save it i get this error message. I'm not touching the data it's flagging. All I've tried to do is reorder a couple of fields and add a new one. Why won't it let me do this? It's a core query
                                                                                                                • Onboard new users easily with native screen recording in Zoho Projects Plus

                                                                                                                  Communication involving both visual and audio elements tends to capture more interest among the audience. Whenever we onboard new employees to the organization, the task of engaging them in the induction sessions, and keeping the spark going in their
                                                                                                                • Request to Add Support for PDF Templates and More Flexibility in Email Editor

                                                                                                                  Hi Zoho Campaigns Team, I hope you're doing well. I wanted to share some feedback and request a few improvements in the Email Template feature of Zoho Campaigns. Right now, we can create email templates using the HTML editor, which is helpful. But we’re
                                                                                                                • Can you stop Custom View Cadences from un-enrolling leads?

                                                                                                                  I'm testing Cadences for lead nurture. I have set un-enroll properties to trigger on email bounce/unsubscribe, and do NOT have a view criteria un-enroll trigger. However, help documents say that emails are automatically un-enrolled from a Cadence when
                                                                                                                • Zoho Desk Android and iOS app update: Agent detail overview

                                                                                                                  Hello everyone! We’ve introduced agent detail overview on the Zoho Desk Android and iOS app update. With this feature, you can get a complete view of agent’s tickets, response times, logged hours (for both tickets and activities), customer happiness ratings
                                                                                                                • Blog Widget: Show recent blog posts on my homepage

                                                                                                                  Hey there I am using the Zoho Sites Blog feature. On my homepage, on the bottom I'd like to have a featured content section where I show some of my blog posts (selected, most recent, filtered by category and so on...). It would be nice to have a blog
                                                                                                                • How to iterate through excel data using zoho rpa

                                                                                                                  I wanted to iterate or loop through excel data ( datatable). How can i do that
                                                                                                                • Zoho Commerce

                                                                                                                  Hi, I have zoho one and use Zoho Books. I am very interested in Zoho Commerce , especially with how all is integrated but have a question. I do not want my store to show prices for customers that are not log in. Is there a way to hide the prices if not
                                                                                                                • Mass (Pull back) (Close WOs) (Close APs)

                                                                                                                  Hi develop team. Just idea because Zoho FSM is great but its too manual. we have a lot of task to do every day and the system needs to have more features to automatic our tasks. I need to close several WOs, APs and Pull Back per day. Please we need mass
                                                                                                                • Customising Zohidesk portal default widgets

                                                                                                                  Hello, I'm wondering how could I customise the default widgets such as "popular topics", "popular articles" using CSS or to make the tiles at the bottom match the ones that I made in the picture attached. I have made some custom widgets and edited them
                                                                                                                • If there’s a batch at the end, the ‘mark as inactive’ function doesn’t work.

                                                                                                                  when my batch has a date to batch end and I "mark as inactive" he batch does not obey the status and reactivates the batch.
                                                                                                                • Seamless multilingual messaging and tone control in Zia IM Composer

                                                                                                                  Hello everyone! Managing customer conversations across multiple languages can be time-consuming. Using Zia in the Instant Messaging (IM) Composer helps you overcome this by providing tools to translate content, adjust your tone, and refine your messages
                                                                                                                • Advanced Deluge help needed with custom function and COQL limits

                                                                                                                  I have a scheduled function that takes a snapshot of specific metrics and saves them to a custom module. However I'm struggling with the limits of COQL. I attach my full code, but an example of the issues I'm having is that it's erroring with a 'Limit
                                                                                                                • Tip of the Week - Conditional Formatting Chart

                                                                                                                  Conditional formatting allows you to highlight your data series based on a pattern or a trend in your data. This makes it easy for you to identify when your data reaches certain values or when it deviates from the trend.  Zoho Analytics allows you to
                                                                                                                • Zoho Campaign says "null"

                                                                                                                  Here is what I did: 1. Created a list in CRM, made a form to fill information in the list. - This is working fine. 2. Created a flow to connect CRM (chose a the list) and want to add an event in Google Calendar. (Calendar is connected). Problem 1 - When
                                                                                                                • Zoho Learn - AI Translate

                                                                                                                  Hi Learn team, I had a situation today where I was providing training to a client team and I had written articles into their Zoho Learn to support their learning. I realised that one of the team members was a non-native English speaker and they were struggling
                                                                                                                • Power of Automation :: Auto-Sync Custom Date Field Between Related Tasks via Blueprint Transition

                                                                                                                  Hello Everyone, A custom function is a software code that can be used to automate a process and this allows you to automate a notification, call a webhook, or perform logic immediately after a workflow rule is triggered. This feature helps to automate
                                                                                                                • Introducing auto-upgrade in Zoho LandingPage

                                                                                                                  Hi everyone, We’re excited to share an upcoming update to help you get the most out of your landing pages — the auto-upgrade option, launching on October 22, 2025. Your landing pages are designed to bring in leads, and we want to make sure you never miss
                                                                                                                • Webhook Authentication Key

                                                                                                                  As Google Ads' lead form extension does not integrate with Zoho CRM at the moment other than Zapier. I want to use the Zoho Flow webhook feature to send leads to Zoho CRM. Meaning creating a flow in Zoho Flow that triggers based on Webhook and then webhook
                                                                                                                • Workdrive comment links stopped working.

                                                                                                                  I have marked a PDF file with 95 comments, they have worked for a while. When i clicked on them they brought me to the correct page and showed me the outline that i had drew on the plans. The comments included the page number of the outline, but now it
                                                                                                                • How to Bulk-Update Sales Orders in CRM

                                                                                                                  Hi - I need to bulk update existing sales orders with dates from our ERP of when the sales orders were created. I made a date field on the Sales Order module where I want to insert that data. I can't Mass Update because I am not updating the fields to
                                                                                                                • Zoho ToDo in Cliq

                                                                                                                  Our organization utilizes Zoho ToDo in the Zoho Mail Desktop app. Is there a way for these to show up in Cliq Desktop app as well?
                                                                                                                • 【Zoho CRM】サンドボックス機能のアップデート:カスタムビューが利用可能になりました。

                                                                                                                  ユーザーの皆さま、こんにちは。コミュニティチームの中野です。 今回は「Zoho CRM アップデート情報」の中から、サンドボックス機能のアップデートをご紹介します。 目次 1. カスタムビューとは 2. 今回の機能アップデートについて 1. カスタムビューとは カスタムビューは、ユーザーが設定した条件に基づいてデータをフィルタリング・整理し、 重要な情報へ効率的にアクセスできるようにする機能です。 「過去15日間の見込み客」や「受注間近の商談」、「特定の優先度レベルが設定された案件」など 条件を指定してわずか数クリックで設定できます。
                                                                                                                • Changing an agents email address

                                                                                                                  How do you change an agent's email address? I keep getting a red circle even though I am an admin. And on one of my agents he has two email addresses? How is that possible?
                                                                                                                • Zoho CRM - Potentials Tab

                                                                                                                  Hi! When I create a Deal (Potentials tab) the header looks like this: After a refresh on the page it looks like this: What should I do so that it is displayed like in the second pic without refresh? Also I have a user that as of recently, cannot see this
                                                                                                                • Unable to create embed code for resource of workdrive using API

                                                                                                                  Hello Team, I am trying to create embed code for a resource using workdrive api in powershell, however facing some issues with injecting data in body. Followed Doc: https://workdrive.zoho.com/apidocs/v1/filefoldersharing/shareeveryone Please help, below
                                                                                                                • Next Page