Kaizen #123 Data Synchronization from a third party application

Kaizen #123 Data Synchronization from a third party application

Welcome back to the Kaizen series!

In last week's post in the Kaizen series, we discussed one-way data synchronization from Zoho CRM to a legacy application, utilizing Zoho CRM's Bulk Read API and Notification API. This week, we will discuss data synchronization from a third party app to Zoho CRM.

The APIs available in Zoho CRM to insert, update or upsert records are :

1) Insert, Update, and Upsert Records APIs

These are synchronous APIs that are used to perform create, update or upsert operations for up to 100 records in a Zoho CRM module. While the Insert Records API and Update Records API is used to create and update a record respectively, the  Upsert Records API is used to upsert (insert or update) a record depending on the availability of the record in your org . In Upsert Records API, the system checks for duplicate records based on the values in the duplicate_check_fields json array in the input json. If the record is already present, it gets updated. If not, the record is inserted. You will get the status of these operations immediately. Read more about the Upsert Records API in this Kaizen post.

2) Bulk Write API

 The Bulk Write API is an asynchronous API that is used to perform create/update/upsert operations for up to 25K records in a Zoho CRM module. A background job will be scheduled in Zoho CRM. You can check the status of the scheduled job using the Bulk Write Job Details API. Alternatively, you can configure a callback URL when you schedule the job. The details of the Bulk Write Job result are posted on the callback URL upon successful completion or failure of the job. This bulk write API involves the following steps:

To learn more about these steps in the Bulk Write API and about the Bulk Write API in general, refer to the Kaizen post on the Bulk Write API.


  Insert, Update, Upsert Records


Bulk Write API


Insert, update, or upsert up to 100 records per API call.

Insert, update, or upsert up to 25000 records per API call.
The response is available instantly.


The response to the Bulk Write API request will not be available immediately. You can check the status of the job by polling for it, or you can wait for the status to be available in the callback URL after the job has completed.

These APIs consume one credit per 10 records. 

Bulk Write API consumes 500 credits per API call. 
Input should be in JSON format

The input file should be in CSV format.




Let us now examine four different scenarios for record insertion in a module, specifically with varying quantities: 100, 2000, 5000, and 25000 records, and discover how these APIs perform in terms of number of API calls and credits consumed.


Number of Records

Insert,Update or Upsert Records API

Bulk Write API
100
1 API Call
10 API Credits
1 API Call
500 API Credits
2000
20 API Calls
200 API Credits
1 API Call
500 API Credits
5000
50 API Calls
500 API Credits
1 API Call
500 API Credits
25000
250 API Calls
2500 API Credits
1 API Call
500 API Credits

Clearly, if you have more than 5000 records, it is advisable to use the bulk write API, given that you are ok with the operation being run asynchronously. If you want instant results, you should use insert/update/upsert APIs. However, more API credits will be consumed in this case.

Data Sync

Consider an e-commerce company where the Order management team uses a legacy system and the Accounts team uses Zoho CRM. To ensure that the Accounts team works with near real-time data in Zoho CRM, data synchronization from the legacy application is essential. Assume that in Zoho CRM, there is a custom module named "Orders" that contains the details from the legacy system. Imagine that you are in charge of the data synchronization between Zoho CRM and the legacy system. How should you call these APIs to achieve this synchronization? The synchronization process involves two steps. 
Step 1: Initial Data Sync : Initial data sync from the legacy system to Zoho CRM.
Step 2: Subsequent Data Sync : Push any new orders or updated orders to Zoho CRM.

The solution we propose here is one of the many ways in which the data sync can be achieved.

This solution is built on the below three key points: 

1) Unique field in Order Module: Maintain a unique field "parent_id" in the "Orders"  module. The unique identifier of the orders in your legacy system should be mapped to this parent_id field. In create bulk write job, this can be achieved by the section field_mappings. For ensuring correct working of upsert operations, you can make use of "find_by" field inside resource JSON Array in the Create Bulk Write Job API Request.  You should provide the value of "find_by" as the unique field "parent_id" .  To read more about find_by field refer to Create Bulk Write Job page.

2) Sync Flag in the Orders table in legacy system: For the sync, maintain a flag in your legacy system orders table for indicating whether the record is synced with Zoho CRM or not. Once the sync is done, update the flag to  true. Whenever a record is modified or sync fails, the flag value should be updated to false. This ensures that in case a record that is synced gets updated during the initial sync process, this flag column helps us re-sync this record again.

3) Grouping of data to batches:  For initial sync, the data volume in the table will be huge. So they have to be batched in a group of 25k records. This grouping can be done based on sorting the table with a particular column & filtering records based on the sync column with criteria sync=false.

Step 1: Initial Data Sync 

Since you have multiple batches of 25k records, you have to iterate them one by one and send them to the Bulk Write API.

Initial Data Sync
For each batch of data, prepare them as a CSV file and initiate a bulk write Job. Next, check the status of the Bulk Write job, and once complete, look for any records that have been skipped during the process. If there are any, fix them, update the sync flag as false, and include them in the next round of data sync. Continue this loop until all records, including any that were missed, are processed. Take the next batch and repeat till all the batches are over.

Step 2: Subsequent Data Sync

The Order Management team continues to use the legacy application, leading to continuous creation or update of records in the Orders module.
It is not recommended to invoke Upsert Records API whenever an operation happens in Orders in the legacy system. Subsequent data syncs can be done in two ways: either by  (a)near real-time sync or (b) Interval based sync.


(a) Near Real-time Sync

 As we want near real-time data in Zoho CRM, it is ok for the records that are updated in the legacy system from last synchronization to be unavailable in Zoho CRM for 10 minutes.

Near Real-time Sync

Instead of making API calls to Zoho CRM on every data operation, store the data in a temporary data store like a DB table or Redis. Once 100 records are accumulated in that data-store, the Upsert Records API can be promptly invoked with all these record details and followed by emptying of the data store. This approach will lead to lower consumption of API credits.
There are chances that in a period of 10 mins, 100 record operations may not happen. Scheduler jobs or CRON jobs can be created to handle the records in the data store to update records to Zoho CRM. Once these records are handled, the data store should be emptied.

(b) Interval based synchronization

For interval based synchronization, schedule bulk write API calls at suitable intervals to upsert the Orders module in Zoho CRM. Whenever there is a record operation in the legacy table, update the sync flag to "false". At regular intervals, such as every 12 or 24 hours, the bulk write API or Upsert Records API can be employed depending on the volume of the data changes.

We hope you found this article useful. We will be back next week with another interesting topic. If you have any questions, write to us at support@zohocrm.com or let us know in the comment section. 


Related Links




    Access your files securely from anywhere

          Zoho FSM Video Tutorials


                All-in-one knowledge management and training platform for your employees and customers.






                                      Zoho Developer Community




                                                            • Desk Community Learning Series


                                                            • Digest


                                                            • Functions


                                                            • Meetups


                                                            • Kbase


                                                            • Resources


                                                            • Glossary


                                                            • Desk Marketplace


                                                            • MVP Corner


                                                            • Word of the Day


                                                            • Ask the Experts



                                                                      • Sticky Posts

                                                                      • Announcing the new SKILL.md for Zoho CRM and the updated OAS repository!

                                                                        We are introducing a new zoho-crm skill to make working with Zoho CRM Developer tools (like APIs, functions, widgets, client scripts, queries etc) easier and faster, with the help of AI in your preferred AI harness like Claude Code, Codex, Cursor, VSCode
                                                                      • Kaizen #261 : Using Zoho CRM Skill to generate complex COQL queries

                                                                        Hello everyone, In the previous Kaizen posts, we introduced the zoho-crm SKILL.md and discussed how it can help you when you are working with Zoho CRM APIs and Client Script. In this Kaizen, we will focus on using the Zoho CRM `SKILL.md` with GitHub Copilot
                                                                      • Kaizen #256 - Build an Arrival Readiness Web Tab in Zoho CRM

                                                                        Hi everyone! Welcome back to the Kaizen series! In the post, we discuss a use case in hospitality industry: how an Arrival Readiness web tab widget can be used to let reception staff identify and resolve issues before arrival of guests. Use case In the
                                                                      • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                                        Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.
                                                                      • Kaizen #226: Using ZRC in Client Script

                                                                        Hello everyone! Welcome to another week of Kaizen. In today's post, lets see what is ZRC (Zoho Request Client) and how we can use ZRC methods in Client Script to get inputs from a Salesperson and update the Lead status with a single button click. In this


                                                                      Manage your brands on social media



                                                                            Zoho TeamInbox Resources



                                                                                Zoho CRM Plus Resources

                                                                                  Zoho Books Resources


                                                                                    Zoho Billing Resources

                                                                                      Zoho Projects Resources


                                                                                        Zoho Sprints Resources


                                                                                          Qntrl Resources


                                                                                            Zoho Creator Resources



                                                                                                Zoho CRM Resources

                                                                                                • CRM Community Learning Series

                                                                                                  CRM Community Learning Series


                                                                                                • Kaizen

                                                                                                  Kaizen

                                                                                                • Functions

                                                                                                  Functions

                                                                                                • Meetups

                                                                                                  Meetups

                                                                                                • Kbase

                                                                                                  Kbase

                                                                                                • Resources

                                                                                                  Resources

                                                                                                • Digest

                                                                                                  Digest

                                                                                                • CRM Marketplace

                                                                                                  CRM Marketplace

                                                                                                • MVP Corner

                                                                                                  MVP Corner






                                                                                                    Design. Discuss. Deliver.

                                                                                                    Create visually engaging stories with Zoho Show.

                                                                                                    Get Started Now


                                                                                                      Zoho Show Resources

                                                                                                        Zoho Writer

                                                                                                        Get Started. Write Away!

                                                                                                        Writer is a powerful online word processor, designed for collaborative work.

                                                                                                          Zoho CRM コンテンツ


                                                                                                            Nederlandse Hulpbronnen


                                                                                                                ご検討中の方





                                                                                                                          • Recent Topics

                                                                                                                          • CNIL - Suivi de Pixel

                                                                                                                            Bonjour à tous, À la suite de la nouvelle recommandation de la CNIL sur les pixels de suivi dans les e-mails, savez-vous si Zoho Campaigns permet : de conditionner le suivi des ouvertures au consentement de chaque contact ; de proposer un lien permettant
                                                                                                                          • Meet PageHealth by Zoho PageSense: a free way to find out what's holding your page back

                                                                                                                            We all know the feeling. You've spent days tweaking headlines, moving buttons, and rewriting copy, and you're still not sure why visitors aren't sticking around. Revamping your website is hard, and figuring out what's actually getting in the way is even
                                                                                                                          • Important: Whitelist new proxy outgoing IPs for IN, SA, and UAE data centers

                                                                                                                            Hello everyone, We're expanding the pool of proxy outgoing IP addresses for our IN, SA, and UAE data centers by introducing new IPs alongside the existing ones. This helps ensure high availability, optimal load distribution, and redundancy for outgoing
                                                                                                                          • Zoho Flow Custom Function Issue

                                                                                                                            I have a use case where I will be needing to post data to a wordpress site. Specifically to a custom post type with a bunch of ACF fields. I have setup trigger event correctly for Tookan and can receive data from the webhook. In order to post the data
                                                                                                                          • How to Open .mbox file in Gmail with Attachments?

                                                                                                                            Gmail offer users to backup INBOX folder in .mbox file format via Google Takeout Feature. Howver there is no such option to Restore Gmail MBOX. Thus I would like to suggest you to choose an alternate approach i.e. MBOX to Gmail Wizard. This utility will open .mbox file in Gmail with attachments.  Steps to open MBOX file in Gmail are; Run MBOX to Gmail Wizard Click Add File and add .mbox file. Enter your Gmail login credentials. Click Convert button. Finished! This is how you can open MBOX file in
                                                                                                                          • Custom Validation for On Duty

                                                                                                                            Hi Team, We would like to have custom validation support for the On Duty form in the Attendance module. We need to apply some restrictions, such as: Limit the number of On Duty requests per month. Restrict On Duty requests on selected days. Allow employees
                                                                                                                          • Forms data export bug

                                                                                                                            The export csv with a password has changed for the worst. It is less secure by not allowing complex passwords. Just last week and the last year (at least) I was able to use a complex, secure password. Export CSV moreover doesn't even work if I use a simple
                                                                                                                          • Placeholders for Multi-Level Discount

                                                                                                                            I am using custom templates for Invoices and all other modules. Since i have activated multi-level discount (an early-access feature from zoho) "discount at line-item and transaction level", I need placeholders for using them in my custom templates. With
                                                                                                                          • Integrating FaceBook Ads into MarketingHub

                                                                                                                            I would like to track the clicks from a FB Ad campaign into my website via marketinghub.  I cannot currently add a MH form directly to my FB page, as I receive an error telling me I need at least 2000 subscribers on FB to be allows to do that (yikes!) What is the best way to set up MH so I can track which FB are converting well?  I know I can reach the FB reports directly, but these do not track conversions from the MH forms on my website. Best regards, Joe
                                                                                                                          • Migration from GoDaddy to Zohomail

                                                                                                                            Hi All, I am trying to migrate all the emails from Go Daddy to zohomail account using the migration tool. i tried with imap method, its not working. when i try it with pop method, then only the emails from the go daddy account inbox ( not other folders), is getting transferred, how can i transfer all the folders in the email account from go daddy to zoho. and also the emails which are coming from go daddy are like new unread emails ( is there any where i can differentiate from read and unread emails)
                                                                                                                          • How do I migrate from Office 365 to Zoho Mail?

                                                                                                                            Check out Advik Email Migration Wizard, this software is specially developed to move mailboxes from Office 365 to Zoho Webmail. In addition you can migrate from Gmail, Yahoo, Rediffmail and 80+ webmail servers to ZOHO MAIL. Isn't it amazing? This is an all in one email migration solution. Steps to export emails from Office 365 to Zoho Webmail are as follows; Run Advik Email Migration Tool in your system. Select Office 365 as source and enter its login credentials. Select mailbox folders and choose
                                                                                                                          • How to Backup Zoho to PST?

                                                                                                                            I'm looking for a simple way to backup Zoho Mail emails to PST format. I tried the IMAP method with Outlook, but it seems slow and complicated for large mailboxes. I need a solution that can: Export Zoho emails to PST Preserve attachments and folder hierarchy
                                                                                                                          • Shopify connector – product cost ("Cost per item") not synced, so COGS/gross margin can't be calculated

                                                                                                                            Hi Zoho Analytics team, I use the native Shopify connector in Zoho Analytics (cloud) to build reporting dashboards for Shopify stores. As far as I can tell, the connector does not sync product cost, so there's no way to calculate Cost of Goods Sold (COGS),
                                                                                                                          • Zoho Sprints - Q3 Updates for 2026

                                                                                                                            Another quarter, another set of productive features. This time our focus was mainly to simplify in-app navigations and control all your work related activities within Zoho Sprints. We delivered 7 major features and 9 enhancements, which include Zoho Sprints
                                                                                                                          • Zoho Webinar - Sharing System Audio (NOT AVAILABLE)

                                                                                                                            Hi, We are having a serious problem with Zoho Webinar. In the webinars we run, we very often share the audio from a video we are streaming directly from YouTube or other applications. Until recently we were using Zoom, but as we use other Zoho applications
                                                                                                                          • Which Zoho CRM tasks would you want an AI assistant to handle?

                                                                                                                            I’ve started delegating my Zoho CRM work to AI – and many tasks now take me half the time. It has been so useful that we’re now working to make it available to everyone. Instead of clicking through records and tabs, I simply describe what needs to be
                                                                                                                          • How to present an Analytic on a CRM dashboard and filter it on loginuser id

                                                                                                                            How to present an Analytic on a CRM dashboard and filter it on loginuser id? I want to display an Analytics pivot table (as a widget) on a CRM dashboard and have the pivot filtered on the loginuser. I want additional user-filters available, such as date,
                                                                                                                          • Create a single record from multiple images with ICR

                                                                                                                            Hello all, Zia's Intelligent Character Recognition can now create a single record from multiple images. Users can upload up to three images at once, such as an ID, a passport, and a business card, and Zia will extract the relevant information from all
                                                                                                                          • The Social Wall: August 2024

                                                                                                                            Hi everyone, We're thrilled to bring you the latest updates, features, and improvements to help you get the most out of Zoho Social. Let's see what was new for us at Social in August. Post documents on LinkedIn You can now reach a lot more people and
                                                                                                                          • Free PST Password Recovery Tool

                                                                                                                            Recoveryfix PST Password Recovery is a specialized standalone utility designed to instantly unlock encrypted Microsoft Outlook PST files . It provides an efficient way to recover, reset, or entirely remove passwords when access credentials are lost or
                                                                                                                          • Change from OST to PST on Windows 10/11 Using Free Method

                                                                                                                            There is an advantage in converting OST files to PST format if you need to access your Outlook data on a different computer, want to keep a local copy, or are trying to recover items from an old Outlook profile. The good point is that users of Windows
                                                                                                                          • How can I add a Custom Field (or Customer_name) to a picklist?

                                                                                                                            Our workflow is Shopify-->SO -->Picklist-->Package-->Shipment. We would like to have the Picklist template to include a custom field "Customer_Notes" (we have created this field already). This way we get alerted when a customer has notes on their account.
                                                                                                                          • Cash Flow Statement : Direct Method

                                                                                                                            I can not see how can I see the cash flow statement using direct method. AS 3 recommends two formats of cash flow statement. The indirect method is there on Zoho but there report in Direct method. How can I see that by Direct method.
                                                                                                                          • ZeptoMail is now Zoho CPaaS!

                                                                                                                            ZeptoMail has a new name and it’s expanding beyond email. With ZeptoMail, our focus was simple : help developers and businesses with reliable transactional emails. At the time, email handled a big part of those customer moments. But transactional communication
                                                                                                                          • Button Display Conditions

                                                                                                                            Hi Guys, Is it at all possible to have extra button conditions? Context: We have data in our deals module which has a custom button which converts the deal into contacts + set up relationships between them. At the end of the conversion we set a field
                                                                                                                          • WhatsApp Business Message Pricing Update: Effective October 1, 2026

                                                                                                                            Hi everyone, Starting October 1, 2026, Meta is updating how WhatsApp Business messages are charged. This guide explains the new per-message pricing model, how message categories work, and what changes for service messages specifically. How Meta charges
                                                                                                                          • Cannot login to Zoho Mail on Mobile App

                                                                                                                            Hi, I am the super administrator for our company's email account. I just created 4 new email accounts, and my users have downloaded the zoho mail app. However they are unable to login to their email on the app. It kept saying "User does not exist". Please help! Thank you.
                                                                                                                          • Zoho Projects - Linking CRM Deals to Projects via API without sending customer contact email invitation to project portal

                                                                                                                            We are using the Zoho CRM and Zoho Projects APIs to automatically create a project in Zoho Projects and then associate that project with the relevant Deal in Zoho CRM. The API process is working in that the project is created successfully and appears
                                                                                                                          • Live connection to DS files in Zoho Creator / ChatGPT

                                                                                                                            As an idea, we are using ChatGPT to build with Zoho Creator (outside of Zia), which is works brilliantly to assist write deluge, provide test suggesitons etc. Currently we are uploading the DS files which are very point in time, it would be great to have
                                                                                                                          • In ZOHO Books, it appears that there is a maximum of 200 Shipping addresses.

                                                                                                                            I am using ZOHO Books and what you call a shipping address, we use as a Service address. I have a couple of customers that have Many Many service addresses, I am starting to get an error saying too many shipping addresses. Additionally, is there a way
                                                                                                                          • Setting default From address when replying to request

                                                                                                                            At the moment, if I want to reply to a request, the From field has three options, company@zohosupport.com, support@company.zohosupport.com, and support@company.com. The first two are really internal address that should never be seen by the customer and
                                                                                                                          • Notice to Zoho Community Members: Updated Zoho Community Guidelines

                                                                                                                            Hello everyone, We have updated the Zoho Community Guidelines to keep the exchanges original, relevant, and meaningful. The major focus of this update is on AI use, and on sharing apps, extensions or other software in the community. This community works
                                                                                                                          • How to add a fee based on payment method

                                                                                                                            I currently accept both ACH payments and credit card. I want to add a 3% fee to a number of my subscription plans if the customer chooses to use a credit card (and not ACH/checking). How do I do this? Is there an in-built feature, or would i have to create
                                                                                                                          • How to charge Convenience fee OR payment gateway charges to the end client who is paying the invoice?

                                                                                                                            Hello, I am creating this topic after having discussions with various sets of users and have understood that with people moving more and more to digital payments, it is important for the client to enable the "Convenience fee" kind of scenario. I have
                                                                                                                          • Can't copy and paste from email in Zoho email client

                                                                                                                            I don't know what happened, but I have lost the ability to cut and paste information from an email.
                                                                                                                          • What's New in Zoho Inventory | July & August 2026

                                                                                                                            Hello users, The July and August 2026 releases bring a range of new features and improvements across the product. From the Nigeria Edition and Super Admin enhancements to Sri Lanka IRD-compliant invoicing, Loyalty Program Management, and improvements
                                                                                                                          • Hide Inactive Social Sign-In Providers from Login Screen

                                                                                                                            Hello Zoho Team, We hope you are doing well. Currently, Zoho One allows admins to configure security policies and enable or disable Social Sign-In options for third-party providers such as Apple, Google, Microsoft, LinkedIn, Yahoo, Twitter, Facebook,
                                                                                                                          • How does Zoho handle the daylight savings time change in Alberta?

                                                                                                                            Alberta has just made the decision to get rid of daylight savings. This means that as of November 1, 2026 we remain on UTC-6 year-round. We have timing errors in meetings scheduled after that For example, we scheduled meetings in late November at 9:00
                                                                                                                          • Need to set workflow or journey wait time (time delay) in minutes, not hours

                                                                                                                            Minimum wait time for both Campaigns workflows and Marketing Automation journeys is one hour. I need one or the other to be set to several minutes (fraction of the hour). I tried to solve this by entering a fraction but the wait time data type is an integer
                                                                                                                          • Shared Mailbox in iOS app

                                                                                                                            Hi, On the desktop I am using a Shared mailbox for our company's general mailbox that somebody else should be able to view. In the iOS app however I can't find it? Old sources on the internet says this isn't possible but that can't be right? Somebody
                                                                                                                          • Next Page