Kaizen #123 Data Synchronization from a third party application

Kaizen #123 Data Synchronization from a third party application

Welcome back to the Kaizen series!

In last week's post in the Kaizen series, we discussed one-way data synchronization from Zoho CRM to a legacy application, utilizing Zoho CRM's Bulk Read API and Notification API. This week, we will discuss data synchronization from a third party app to Zoho CRM.

The APIs available in Zoho CRM to insert, update or upsert records are :

1) Insert, Update, and Upsert Records APIs

These are synchronous APIs that are used to perform createupdate or upsert operations for up to 100 records in a Zoho CRM module. While the Insert Records API and Update Records API is used to create and update a record respectively, the  Upsert Records API is used to upsert (insert or update) a record depending on the availability of the record in your org . In Upsert Records API, the system checks for duplicate records based on the values in the duplicate_check_fields json array in the input json. If the record is already present, it gets updated. If not, the record is inserted. You will get the status of these operations immediately. Read more about the Upsert Records API in this Kaizen post.

2) Bulk Write API

 The Bulk Write API is an asynchronous API that is used to perform create/update/upsert operations for up to 25K records in a Zoho CRM module. A background job will be scheduled in Zoho CRM. You can check the status of the scheduled job using the Bulk Write Job Details API. Alternatively, you can configure a callback URL when you schedule the job. The details of the Bulk Write Job result are posted on the callback URL upon successful completion or failure of the job. This bulk write API involves the following steps:

To learn more about these steps in the Bulk Write API and about the Bulk Write API in general, refer to the Kaizen post on the Bulk Write API.


  Insert, Update, Upsert Records


Bulk Write API


Insert, update, or upsert up to 100 records per API call.

Insert, update, or upsert up to 25000 records per API call.
The response is available instantly.


The response to the Bulk Write API request will not be available immediately. You can check the status of the job by polling for it, or you can wait for the status to be available in the callback URL after the job has completed.

These APIs consume one credit per 10 records. 

Bulk Write API consumes 500 credits per API call. 
Input should be in JSON format

The input file should be in CSV format.




Let us now examine four different scenarios for record insertion in a module, specifically with varying quantities: 100, 2000, 5000, and 25000 records, and discover how these APIs perform in terms of number of API calls and credits consumed.


Number of Records

Insert,Update or Upsert Records API

Bulk Write API
100
1 API Call
10 API Credits
1 API Call
500 API Credits
2000
20 API Calls
200 API Credits
1 API Call
500 API Credits
5000
50 API Calls
500 API Credits
1 API Call
500 API Credits
25000
250 API Calls
2500 API Credits
1 API Call
500 API Credits

Clearly, if you have more than 5000 records, it is advisable to use the bulk write API, given that you are ok with the operation being run asynchronously. If you want instant results, you should use insert/update/upsert APIs. However, more API credits will be consumed in this case.

Data Sync

Consider an e-commerce company where the Order management team uses a legacy system and the Accounts team uses Zoho CRM. To ensure that the Accounts team works with near real-time data in Zoho CRM, data synchronization from the legacy application is essential. Assume that in Zoho CRM, there is a custom module named "Orders" that contains the details from the legacy system. Imagine that you are in charge of the data synchronization between Zoho CRM and the legacy system. How should you call these APIs to achieve this synchronization? The synchronization process involves two steps. 
Step 1: Initial Data Sync : Initial data sync from the legacy system to Zoho CRM.
Step 2: Subsequent Data Sync : Push any new orders or updated orders to Zoho CRM.

The solution we propose here is one of the many ways in which the data sync can be achieved.

This solution is built on the below three key points: 

1) Unique field in Order Module: Maintain a unique field "parent_id" in the "Orders"  module. The unique identifier of the orders in your legacy system should be mapped to this parent_id field. In create bulk write job, this can be achieved by the section field_mappings. For ensuring correct working of upsert operations, you can make use of "find_by" field inside resource JSON Array in the Create Bulk Write Job API Request.  You should provide the value of "find_by" as the unique field "parent_id" .  To read more about find_by field refer to Create Bulk Write Job page.

2) Sync Flag in the Orders table in legacy system: For the sync, maintain a flag in your legacy system orders table for indicating whether the record is synced with Zoho CRM or not. Once the sync is done, update the flag to  true. Whenever a record is modified or sync fails, the flag value should be updated to false. This ensures that in case a record that is synced gets updated during the initial sync process, this flag column helps us re-sync this record again.

3) Grouping of data to batches:  For initial sync, the data volume in the table will be huge. So they have to be batched in a group of 25k records. This grouping can be done based on sorting the table with a particular column & filtering records based on the sync column with criteria sync=false.

Step 1: Initial Data Sync 

Since you have multiple batches of 25k records, you have to iterate them one by one and send them to the Bulk Write API.

Initial Data Sync
For each batch of data, prepare them as a CSV file and initiate a bulk write Job. Next, check the status of the Bulk Write job, and once complete, look for any records that have been skipped during the process. If there are any, fix them, update the sync flag as false, and include them in the next round of data sync. Continue this loop until all records, including any that were missed, are processed. Take the next batch and repeat till all the batches are over.

Step 2: Subsequent Data Sync

The Order Management team continues to use the legacy application, leading to continuous creation or update of records in the Orders module.
It is not recommended to invoke Upsert Records API whenever an operation happens in Orders in the legacy system. Subsequent data syncs can be done in two ways: either by  (a)near real-time sync or (b) Interval based sync.


(a) Near Real-time Sync

 As we want near real-time data in Zoho CRM, it is ok for the records that are updated in the legacy system from last synchronization to be unavailable in Zoho CRM for 10 minutes.

Near Real-time Sync

Instead of making API calls to Zoho CRM on every data operation, store the data in a temporary data store like a DB table or Redis. Once 100 records are accumulated in that data-store, the Upsert Records API can be promptly invoked with all these record details and followed by emptying of the data store. This approach will lead to lower consumption of API credits.
There are chances that in a period of 10 mins, 100 record operations may not happen. Scheduler jobs or CRON jobs can be created to handle the records in the data store to update records to Zoho CRM. Once these records are handled, the data store should be emptied.

(b) Interval based synchronization

For interval based synchronization, schedule bulk write API calls at suitable intervals to upsert the Orders module in Zoho CRM. Whenever there is a record operation in the legacy table, update the sync flag to "false". At regular intervals, such as every 12 or 24 hours, the bulk write API or Upsert Records API can be employed depending on the volume of the data changes.

We hope you found this article useful. We will be back next week with another interesting topic. If you have any questions, write to us at support@zohocrm.com or let us know in the comment section. 


Related Links




    Access your files securely from anywhere

        All-in-one knowledge management and training platform for your employees and customers.







                              Zoho Developer Community




                                                    • Desk Community Learning Series


                                                    • Digest


                                                    • Functions


                                                    • Meetups


                                                    • Kbase


                                                    • Resources


                                                    • Glossary


                                                    • Desk Marketplace


                                                    • MVP Corner


                                                    • Word of the Day


                                                    • Ask the Experts



                                                              • Sticky Posts

                                                              • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                                Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
                                                              • Kaizen #226: Using ZRC in Client Script

                                                                Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this
                                                              • Kaizen #222 - Client Script Support for Notes Related List

                                                                Hello everyone! Welcome to another week of Kaizen. The final Kaizen post of the year 2025 is here! With the new Client Script support for the Notes Related List, you can validate, enrich, and manage notes across modules. In this post, we’ll explore how
                                                              • Kaizen #217 - Actions APIs : Tasks

                                                                Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.
                                                              • Kaizen #216 - Actions APIs : Email Notifications

                                                                Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are


                                                              Manage your brands on social media



                                                                    Zoho TeamInbox Resources



                                                                        Zoho CRM Plus Resources

                                                                          Zoho Books Resources


                                                                            Zoho Subscriptions Resources

                                                                              Zoho Projects Resources


                                                                                Zoho Sprints Resources


                                                                                  Qntrl Resources


                                                                                    Zoho Creator Resources



                                                                                        Zoho CRM Resources

                                                                                        • CRM Community Learning Series

                                                                                          CRM Community Learning Series


                                                                                        • Kaizen

                                                                                          Kaizen

                                                                                        • Functions

                                                                                          Functions

                                                                                        • Meetups

                                                                                          Meetups

                                                                                        • Kbase

                                                                                          Kbase

                                                                                        • Resources

                                                                                          Resources

                                                                                        • Digest

                                                                                          Digest

                                                                                        • CRM Marketplace

                                                                                          CRM Marketplace

                                                                                        • MVP Corner

                                                                                          MVP Corner









                                                                                            Design. Discuss. Deliver.

                                                                                            Create visually engaging stories with Zoho Show.

                                                                                            Get Started Now


                                                                                              Zoho Show Resources

                                                                                                Zoho Writer

                                                                                                Get Started. Write Away!

                                                                                                Writer is a powerful online word processor, designed for collaborative work.

                                                                                                  Zoho CRM コンテンツ




                                                                                                    Nederlandse Hulpbronnen


                                                                                                        ご検討中の方




                                                                                                                • Recent Topics

                                                                                                                • Custom Function to increment a value by one

                                                                                                                  Hi, I'm trying to find a solution to set up a counter on CRM records that tracks how many times certain actions have taken place. As a specific example: We have a field on Deals called "Times Close Updated". This starts at 0 on record creation. I'd like
                                                                                                                • Access token generate from the refresh token not working for API

                                                                                                                  Dear Sir/Madam, When I use my refresh token to obtain new access_token, that token return INVALID_TOKEN when the same API is called. I made sure that my api site is correct and match the auth web site. However the original access_token work fine.
                                                                                                                • why does my campaign move back to draft?

                                                                                                                  Every time I try to send my email campaign, it reverts back to Draft status.. this has happened three times in a row..how do i find out what the problem is? Thanks
                                                                                                                • Service and Parts Line Item Limitations

                                                                                                                  Hi FSM Team, We work with clients who deliver large-scale field service projects. In many cases, a single work order can contain 200+ service and parts line items. Currently, Zoho FSM limits work orders to 30 service and parts line items. While this works
                                                                                                                • 【Zoho CRM】やりとりの要約機能リリースのお知らせ

                                                                                                                  ユーザーの皆さま、こんにちは。コミュニティチームの藤澤です。 今回は「Zoho CRM アップデート情報」の中から、新機能のやりとり要約機能をご紹介します。 Zoho CRMのやりとりの要約は、Ziaが生成する機能で、データに関連付けられた最近のやりとりを簡潔にまとめて提供します。 メール、通話、ミーティング、メモなどを1つの明確な要約にまとめ、重要なポイント、過去の話し合いの要点、結果、次のステップを表示することで、日常的な課題に対処します。 Ziaの強みは、単なる要約にとどまらず、内容の解釈も行える点です。やりとりの要約では、顧客との会話に潜む遅延、フォローアップ漏れ、約束、期限、感情、意図を特定します。
                                                                                                                • Recent post analytics not very useful

                                                                                                                  Hi, I'm enjoying some aspects of the Zoho Social. like being able to schedule posts to multiple channels. However, the recent posts analytics is disappointing. For example, I put up a reel on Instagram and Facebook yesterday. Instagram insights shows
                                                                                                                • What happens to the files created by user, whose account is deleted.

                                                                                                                  Hello, I created a folder in the My folders names "Quote" I shared the same with my colleague. She created some files and folders in that folder over the period of 1 and half year. Now she left company, and I deleted her account from Zoho. What happens
                                                                                                                • Internal Error When Accessing Team Inbox.

                                                                                                                  All our users are seeing this error in teaminbox. Because its a critical tool kindly resolve this issue ASAP.
                                                                                                                • Sharing URLs and direct access

                                                                                                                  Hello, I am storing my team's email signature images on Workdrive. I am creating a public image download share and adding “?directDownload=true” so that the image can be accessed without the Workdrive interface. A few questions: 1) Can we generate friendly
                                                                                                                • Zoho Mail Android app update: Calendar enhancements

                                                                                                                  Hello everyone! In the most recent version of the Zoho Mail Android app update, we have introduced various new enhancements for the calendar module. Let's explore what's new. Drag and drop events to update the date and time We have enhanced the calendar
                                                                                                                • Enhance Delay Configuration in Zoho Flow

                                                                                                                  Dear Zoho Flow Support Team, We are writing to request an improvement to the delay configuration process within Zoho Flow. Currently, users are required to manually enter the exact delay duration (e.g., "2 days") in the delay block. This can be time-consuming
                                                                                                                • Contratação ProdutivosX

                                                                                                                  Bem-vindo(a) ao processo de contratação da ProdutivosX. Este formulário tem como objetivo coletar informações essenciais para análise de perfil profissional, alinhamento de competências e possível integração ao equipamento ProdutivosX. A ProdutivosX é
                                                                                                                • Enhance Sign CRM integration

                                                                                                                  Hello all, I'm working on a custom Deluge script to enhance the integration between Zoho CRM and Sign by using a writer merge template for additional flexibility. I want to replicate the post-sign document integration that exists between CRM and Sign
                                                                                                                • Unified WhatsApp Number Management in Zoho Desk and SalesIQ

                                                                                                                  Dear Zoho Desk Support Team, We are currently utilizing both Zoho Desk and Zoho SalesIQ for our customer support operations. While both platforms offer WhatsApp integration, we are facing challenges due to the requirement of separate WhatsApp numbers
                                                                                                                • Email Field Validation Incorrectly Rejects RFC-Compliant Addresses (Forward Slashes)

                                                                                                                  I've encountered a validation issue with Zoho Creator's Email field that rejects RFC-compliant email addresses containing forward slashes, and I'm hoping the Zoho team can address this in a future update. The Issue When entering an email address containing
                                                                                                                • Plug Sample #15 - Enable Human-Like, Contextual Interactions in SalesIQ with Zia Agents

                                                                                                                  Zia Agents are conversational AI assistants designed to understand user intent and respond intelligently, helping businesses automate conversations and offer personalized support at scale. While Zia Agents are yet to be publicly released, access is currently
                                                                                                                • Best way to schedule bill payments to vendors

                                                                                                                  I've integrated Forte so that I can convert POs to bills and make payments to my vendors all through Books. Is there a way to schedule the bill payments as some of my vendors are net 30, net 60 and even net 90 days. If I can't get this to work, I'll have
                                                                                                                • Seamless Round-Trip Navigation for Related Blocks (Detail View)

                                                                                                                  As highlighted previously in this post (and here, here, here, and here), we still lack a fundamental capability for seamless navigation in Related Blocks. The popup that appears when adding a related record doesn't exist for viewing/editing existing records,
                                                                                                                • Social Profile Logo Format

                                                                                                                  Hello, I'm using Zoho Sites and am attempting to add a couple of social media accounts to my profile that aren't included in the default platforms that are available. What format is the logo required to be in? I have tried .png and .jpg with no success.
                                                                                                                • How to create Sepa Direct Debit XML file: solution and code

                                                                                                                  Even though Books provides a payment integration for Stripe and Gocardless (in Europe) there are customers that want to use the Sepa services of their own bank. Mainly because Stripe and Gocardless are quite expensive. In that case they would need a Sepa
                                                                                                                • Traditional Spreadsheet vs Zoho Tables

                                                                                                                  Hello everyone, This article explains the differences between Spreadsheet applications and no-code databases, such as Zoho Tables. While both tools belong to the "grid" family, the core difference lies in their purpose. A spreadsheet (such as Zoho Sheet,
                                                                                                                • RFQ MODEL

                                                                                                                  A Request for quotation model is used for Purchase Inquiries to multiple vendors. The Item is Created and then selected to send it to various vendors , once the Prices are received , a comparative chart is made for the user. this will help Zoho books
                                                                                                                • Delegate Access - Mobile iOS/iPad

                                                                                                                  We’re over the moon that delegate access is now available in Zoho Mail as we were nearly ready to switch platforms because of it! Is there a timeline on when delegate mailboxes will be accessible from the iOS and iPad OS applications? Thanks, Jake
                                                                                                                • Request For Quotation (RFQ) module

                                                                                                                  Hello, Do you have any plans to implement a RFQ module in to ZOHO Inventory? I would like to chose items that I require a price for, select a number of different suppliers to e-mail and have them submit there pricing online. I would then like to see a
                                                                                                                • Support for Developing Zoho Recruit Extensions via Zoho Sigma

                                                                                                                  Hi, I’m interested in building an extension for Zoho Recruit using Zoho Sigma. However, when I navigate to Sigma and attempt to create a new extension (via the "New Extension" option), Zoho Recruit does not appear as a listed service—only options like
                                                                                                                • FOLDER DISAPPEARED

                                                                                                                  Hello Zoho Community. I recently found a problem with no explanation: a folder in Zoho Analytics just disappeared, but the tables and SQL sentences still existed, the folder was not deleted. The solution for me in this case was to create a new folder,
                                                                                                                • How to create estimates/Invoices with sub-totals

                                                                                                                  Every other accounting package can create estimates and invoices with Sub-totals. How can I do that in ZohoBooks?
                                                                                                                • Deluge scripts

                                                                                                                  Why is there not a search function to make it easier to find the script of interest when modifications are required.
                                                                                                                • Social media simplified with Zoho Social: Why should brands have a Threads profile?

                                                                                                                  Just over a year ago, Instagram launched Threads, the all new social media app primarily focusing on sharing text online. It was welcomed by people worldwide with more than 10 million users in just seven hours, and it currently has over 175 million active
                                                                                                                • Auto tracking URL generation based on Carrier

                                                                                                                  Hi, While creating a shipment order for a package in Zoho Books, I have a requirement that for example, if the carrier is Delhivery and tracking number is 1234, then can automatically the tracking link/URL be generated as www.delhivery.com/1234. Similary,
                                                                                                                • New Beginnings with Zoho Desk mobile app best practices: Part 3

                                                                                                                  In focus: Scaling consistency, intelligence, and customization In our third installment, we'd like to share tips to help you elevate your customer experience as you handle support operations seamlessly on mobile. Let's reconnect with Omniserve, the field
                                                                                                                • Highlights of 2025: Milestones and moments

                                                                                                                  Hey everyone! As we step into 2026, it’s a great time to look back at everything 2025 brought to Zoho Social. This year was big, packed with powerful new features, smart enhancements, exciting events, and a major milestone: a whole decade of Zoho Social.
                                                                                                                • Introducing the sandbox environment in Zoho Sign

                                                                                                                  Hey there! Customer and partners across the globe have been requesting a testing environment—also called a sandbox—for quite some time. Sandboxes help you try out document workflows before using them in your production setup. This new year, we are excited
                                                                                                                • Can I share the drive with another Windows user?

                                                                                                                  We use the WorkDrive Sync application on a server with several remote desktop users. Our question is: Can the drive created by WorkDrive Sync be shared from one of these remote users with the other users so they can access the files, or do we need to
                                                                                                                • Syncing Bills in Zoho Books to Zoho CRM

                                                                                                                  Is there any way to sync the Bills in Zoho Books in Zoho CRM
                                                                                                                • Auto CheckOut Based On Shift.

                                                                                                                  This Deluge script runs on a scheduled basis to automatically set the 'Actual_Check_Out' time for employees who haven't manually checked out. If the current time is past their scheduled 'Shift_End_Time', the script updates the check-out time to match
                                                                                                                • ERROR: Product type cannot be changed for Items having transactions.

                                                                                                                  I have mistakenly added a product type as goods for an item that was a digital service. Now when HSN/SAC became mandatory, this brought my attention to this error I did. So I tried changing the product type but it displayed this error message Product
                                                                                                                • Boost your CRM accuracy with smart Data Enrichment

                                                                                                                  Hello all! In this post let's look at data enrichment from a holistic perspective to gain more clarity on how you can use it. So, here's a quick recap of what data enrichment is and how you set it up it to enhance your existing module data! What is data
                                                                                                                • Is it possible to set a region lookup table so that my deal country can lookup this lookup table

                                                                                                                  Hi there, I would like to be able to add a data lookup table with the following fields: Country, Region. And then in my deal report, i can then lookup this table to fish out the region in my report. This will be important for my sales process to categorise
                                                                                                                • Test Emails Show Sent "via zcsend.net" in My Gmail Account

                                                                                                                  I noticed the following info... However, a few email service providers such as Outlook and Gmail will display that the email was sent by zcsend.net (Zoho campaigns’ server) on your behalf instead of just your from email address.  In order to prevent this, you can go for either of these options: Do not select the check box meant for DomainKey Signature which is listed under Unverified sender domains. Include our mail servers by advanced authentication method such as SPF/ Sender ID and DomainKey/ DKIM.
                                                                                                                • Next Page