Kaizen #177: Duplicate Check Preferences API vs. Upsert API

Kaizen #177: Duplicate Check Preferences API vs. Upsert API

Hello all!!!

Welcome back to another week of Kaizen. Last week, we discussed Optimizing the Use of Record ID Variables in Zoho CRM Queries.

This week, we will explore two important APIs for managing duplicate records in Zoho CRM - Duplicate Check Preferences API and Upsert Records API. While both of these APIs help manage duplicate records, they serve different purposes and are to be used in different scenarios. 




This post covers:
  • What is the Duplicate Check Preferences API?
    • Use Case
    • Duplicate Check Configurations
    • Updating Duplicate Check Configuration
    • How Duplicate Check Preferences Work During Record Creation/Update?
  • What is the Upsert API?
    • Use Case
  • Differences between Duplicate Check Preferences API and Upsert API

What is the Duplicate Check Preferences API

The Duplicate Check Preferences API is a configuration API that allows you to set up criteria using unique fields to manage duplicates across Leads module including Converted Leads and Contacts
in Zoho CRM. Based on the given criteria, the system will restrict the creation or update of duplicate records. For example, if the Phone field is marked as unique and set for duplicate checks, the system will make sure that no two records have the same phone number during records creation or update.

With the Duplicate Check Preferences API, you can define criteria to find duplicates, such as matching email addresses, phone numbers, or custom fields. During record creation or update, Zoho CRM checks the configured criteria against converted leads or contacts. If a match is found, it triggers a "DUPLICATE_DATA" error and blocks duplicates based on your configuration.

By default, the Insert Records API and Update Records API automatically check for this configuration. If a record matches the defined duplicate criteria, the system prevents its creation or update.

Use case

Zylker, a sales team managing customer contacts in Zoho CRM, often imports leads from various sources like website forms, events, and campaigns. However, the team noticed that many duplicate contacts are being created because the same phone number is associated with some of the contacts. This creates confusion and results in redundant work in their sales process.
By using the Duplicate Check Preferences API, Zylker can prevent the anymore duplicate contacts based on phone numbers (phone is used for an example).

Duplicate Check Configurations

In Zoho CRM, you can configure duplicate check preferences via API in two ways. 

i. Configuration in Leads module
The system checks for duplicates on all the Leads using the specified unique fields. By selecting this option, the duplicates will be checked on all the Converted Leads (custom view) as well. This configuration is applicable across all the layouts.
The following is the configuration for duplicate check in the Leads module via API. 

Request Method: POST

Note:
  • The "module" parameter is mandatory, and its supported value is Leads. 
  • The above URL applies to both Leads and Contacts module configurations.
  •  You must mark at least one field as mandatory. So, the system can check for duplicates based on the value in the unique field within the Leads module when creating a record.
Input JSON

{
    "duplicate_check_preference": {
        "type": "converted_records", //Converted Leads custom view in Leads module
    }
}


Response

{
    "duplicate_check_preference": {
        "code": "SUCCESS",
        "details": {},
        "message": "Duplicate check enabled for converted_records successfully.",
        "status": "success"
    }
}




ii. Configuration in Contacts module

By selecting this option, the system will check for duplicates in the Contacts module using mapped fields.

Input JSON

{
    "duplicate_check_preference": {
        "type": "mapped_module_records",
        "type_configurations": [
            {
                "field_mappings": [
                    {
                        "current_field": {
                            "api_name": "Email",
                            "id": "5725767000000002601"
                        },
                        "mapped_field": {
                            "api_name": "Email",
                            "id": "5725767000000002503"
                        }
                    }
                ],
                "mapped_module": {
                    "api_name": "Contacts",
                    "id": "5725767000000002179"
                }
            }
        ]
    }
}


The "Phone" field from both the Leads and Contacts modules is mapped to identify duplicate records.

How Duplicate Check Preferences Work During Record Creation/Update

Consider Zylker has configured the Contacts module ("type":"mapped_module_records") to check duplicate contacts.
For example, a sales representative at Zylker attempts to add a contact from the Cold Call lead source. However, the same contact already exists in the Contacts module, either from the same or a different lead source.
Since the Contacts module is configured for duplicate check preferences, the system will check for duplicates in the Contacts module using mapped fields. During the record insertion action, the system verifies the Contacts module for any matching records based on these unique identifiers. Based on the result, it either creates the record or throws an error.

Use the Insert Records API to create records in a module.

Request Method: POST

Input JSON

{
    "data": [
        {
            "Company": "Villa Margarita",
            "Last_Name": "Dolan",
            "First_Name": "Brian",
            "Email": "brian@villa.com",
            "Phone": "12345",
            "State": "Texas"
        }
    ]
}


Response


 {

    "data": [

        {

            "code": "DUPLICATE_DATA",

            "details": {

                "api_name": "Email",

                "duplicate_record": {

                    "Owner": {

                        "name": "Patricia Boyle",

                        "id": "5725767000000411001",

                        "zuid": "808233918"

                    },

                    "module": {

                        "api_name": "Contacts",

                        "id": "5725767000000002179"

                    },

                    "id": "5725767000000774010" //The record that already exists in the Contacts module.

                },

                "json_path": "$.data[0].Email",

                "more_records": false

            },

            "message": "duplicate data",

            "status": "error"

        }

    ]

}


Duplicates are also checked during record updates. If a record is updated with a value that already exists in a unique field, the system indicates it as a duplicate and returns a "DUPLICATE_DATA" error.

Update Duplicate Check Configuration

To update the previously enabled Duplicate Check option configured in your account, use the Update Duplicate Check Option API.


The Contacts ("type": "mapped_module_records") configuration is used in the below sample input. 


{
    "duplicate_check_preference": {
        "type": "mapped_module_records",
        "type_configurations": [
            {
                "field_mappings": [
                    {
                        "mapped_field": {
                            "api_name": "Phone", //The previously configured Email field as a unique identifier in Contacts has now been replaced with the Phone field.
                            "name": "Contacts",
                            "id": "5725767000000411001"
                        },
                        "current_field": {
                            "api_name": "Phone",
                            "name": "Leads", //The previously configured Email field as a unique identifier in Leads has now been replaced with the Phone field.
                            "id": "5725767000005381030"
                        }
                    }
                ],
                "mapped_module": {
                    "api_name": "Contacts",
                    "name": "Contacts",
                    "id": "5725767000000002179"
                }
            }
        ]
    }
}

You can also retrieve and delete duplicate configuration.

What is the Upsert API?

The Upsert API is a combination of "update" and "insert". 


How it works?

  • You provide a unique identifier (e.g., email, phone number, or a custom field) to check for existing records.

  • If a record with the same identifier exists, it updates that record.

  • If no matching record is found, it creates a new one.

Flow diagram

Use Case 

Zylker, an e-commerce company, uses Zoho CRM to manage customer data. Customers sign up on Zylker’s online store, and their details are stored in the e-commerce platform like Shopify. Over time, customers may update their contact information, or new customers may sign up. To keep customer records in Zoho CRM accurate and avoid duplicates, Zylker uses the Upsert API to:
  • Update existing customer records details if the given unique identifier, such as email or phone number, matches an existing record in Zoho CRM.
  • Insert a new record if no match is found.  This way, Zylker ensures that new customers are added efficiently.

There are two types of duplicate check fields:

i. System-defined duplicate check fields.
ii. User-defined duplicate check fields.

i. System-defined duplicate check fields
The table below lists the system-defined unique fields. When you create a record using the Upsert API, the system checks these fields for duplicate entries. 

Leads

Email

Contacts

Email

Accounts

Account_Name

Deals

Deal_Name

Campaigns

Campaign_Name

Cases

Subject

Solutions

Solution_Title

Products

Product_Name

Vendors

Vendor_Name

PriceBooks

Price_Book_Name

Quotes

Subject

SalesOrders

Subject

PurchaseOrders

Subject

Invoices

Subject

Custom Module

Name


ii. User-defined duplicate check fields
You can set a normal field as a duplicate check field by enabling "Do not allow duplicate values." Note that you can only set fields with the following field-types as unique— Single Line, Email, Phone, Number, Long Integer, and URL.


Request Method: POST

Sample Input JSON

Here, "Email," a system-defined field, has been used as a duplicate check field.
{
    "data": [
        {
            "Email": "alice@mail.com",
            "First_Name": "Alice",
            "Last_Name": "Brown",
            "Phone": "+1-555-222-3333",
            "Order_Date": "2024-01-10",
            "Order_Value": "$175.00"
        },
        {
            "Email": "pat@mail.com",
            "First_Name": "pat",
            "Last_Name": "Davis",
            "Phone": "+1-555-666-7777",
            "Order_Date": "2024-01-15",
            "Order_Value": "$300.00"
        }
    ],
    "duplicate_check_fields": [
        "Email" //You can set the order in which the system checks for duplicate records by specifying the duplicate_check_field array in the input. 
    ]
}

Response


{
    "data": [
        {
            "code": "SUCCESS",
            "duplicate_field": null,
            "action": "insert",
            "details": {
                "Modified_Time": "2025-02-08T09:43:08-08:00",
                "Modified_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                },
                "Created_Time": "2025-02-08T09:43:08-08:00",
                "id": "5725767000005425036",
                "Created_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                }
            },
            "message": "record added",
            "status": "success"
        },
        {
            "code": "SUCCESS",
            "duplicate_field": "Email",
            "action": "update",
            "details": {
                "Modified_Time": "2025-02-08T09:43:08-08:00",
                "Modified_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                },
                "Created_Time": "2025-02-08T08:52:22-08:00",
                "id": "5725767000005425002",
                "Created_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                }
            },
            "message": "record updated",
            "status": "success"
        }
    ]
}


For more details on how the Upsert API works, refer to the Kaizen #56 - Upsert Records #API.

Differences between Duplicate Check Preferences API and Upsert API

 

                Duplicate Check API

                           Upsert API

Duplicate Check Preferences can be enabled only in the Leads module.

Supports all modules.

Throws a "DUPLICATE_DATA" error if a duplicate is detected.

Updates the existing record with new field values if a match is found for the unique fields; otherwise, inserts a new record.


Use the above Zoho CRM APIs which fit your use cases.
---------------------------------------------------------------------------------------------------------------------

We trust that this post meets your needs and is helpful. Let us know your thoughts in the comment section or reach out to us at support@zohocrm.com 
Stay tuned for more insights in our upcoming Kaizen posts!

Info

    • Sticky Posts

    • Kaizen #197: Frequently Asked Questions on GraphQL APIs

      🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Kaizen #198: Using Client Script for Custom Validation in Blueprint

      Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Celebrating 200 posts of Kaizen! Share your ideas for the milestone post

      Hello Developers, We launched the Kaizen series in 2019 to share helpful content to support your Zoho CRM development journey. Staying true to its spirit—Kaizen Series: Continuous Improvement for Developer Experience—we've shared everything from FAQs

    • Kaizen #193: Creating different fields in Zoho CRM through API

      🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

    • Client Script | Update - Introducing Commands in Client Script!

      Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! Commands

    • Recent Topics

    • Forwarder

      Hi, I tried to add a forwarder from which emails are sent to my main zoho account email . However, it asks me for a code that should be received at the forwarder email, which is still not activated to send to my zoho emial account. So how can I get the

    • Forwarder

      Hi, I tried to add a forwarder from which emails are sent to my main zoho account email . However, it asks me for a code that should be received at the forwarder email, which is still not activated to send to my zoho emial account. So how can I get the

    • How do I sync multiple Google calendars?

      I'm brand new to Zoho and I figured out how to sync my business Google calendar but I would also like to sync my personal Google calendar. How can I do this so that, at the very least, when I have personal engagements like doctor's appointments, I can

    • Need to extract date from datetime field

      I have a datetime field and need only the date part from it. I am unable to find a built-in function that would be <DateTime>.Date(). I don't think I want to go the string conversion route of converting the datetime to string and then parsing out values and create a date out of it. Any one out there has a better solution to this? Thanks in adavnce. Regards Moiz Tankiwala Smart Training & IT Solutions

    • How to Hide Article Links in SalesIQ Answer Bot Responses

      I have published an article in SalesIQ, and the Answer Bot is fetching the data and responding correctly. However, it is also displaying the article link, which I don’t want. How can I remove the link so that only the message is shown?

    • New in Cadences: WhatsApp follow-ups, upgraded limits, and options for add-ons

      Hello everyone, We're rolling out two key updates to help you engage better and scale smarter with Cadences in Zoho CRM. Reach customers on WhatsApp, directly from Cadences Previously, Cadences have enabled you to automate follow-ups through emails, calls,

    • additional accounts

      If I brought 5 emails to my account. Can I later buy additional emails.

    • Issue in Zoho People Regularization – Incorrect Hour Calculation

      I have noticed that when applying attendance regularization in Zoho People for previous dates, the total working hours are not calculated correctly. For example, even if the check-in is 10:00 AM and check-out is 6:00 PM, the system shows an incorrect

    • Why I am unable to configure Zoho Voice with my Zoho CRM account?

      I have installed Zoho Voice in my Zoho CRM, but as per the message there is some config needed in Zoho Voice interface. But when I click on the link given in the above message, I get an access denied page.

    • Issue with Hour Calculation in Zoho People Attendance Module

      I have noticed an issue in the attendance regularization feature of Zoho People. When trying to regularize past dates, the total working hours are not calculated correctly. For example, if I enter a check-in and check-out time for a previous day, the

    • Cliq Meeting Calls No Audio and Screen Share

      When in a Cliq channel meeting, the audio does not work at all on pc. When i use my phone as audio source, screen share on pc does not work. I have updated audio drivers but the strangest thing is that during a 1 on 1 call, it works well. Therefore the

    • Bug in Total Hour Calculation in Regularization for past dates

      There is a bug in Zoho People Regularization For example today is the date is 10 if I choose a previous Date like 9 and add the Check in and Check out time The total hours aren't calculated properly, in the example the check in time is 10:40 AM check

    • Work anniversary and birthdays on connect

      Hello, I like the idea of having employee's work anniversary and birthdays on the dashbaord. Do you have to have the employee complete this information themselves in connect settings, or does it pull from their directory settings? (ie. we use Zoho one

    • Alias Email Id already exists

      Hi I'm trying to create an alias : contact @ yoavarielevy.co.il but i get the message  Alias Email Id already exists I had an account with the same name but I deleted it. Can you help? Thanx Yoav

    • BANK FEED - MAYBANK , provider from YODLEE IS NOT WORKING

      As per topic, the provider YODLEE is not working for the BANK FEED. It have been reported since 2023 Q3, and second report on 2023 Q4. now almost end of 2024 Q1, and coming to 2024 Q2. Malaysia Bank Maybank is NOT working. can anyone check on this issue?

    • Feature Request: Ability to Set a Custom List View as Default for All Users

      Dear Zoho CRM Support Team, We would like to request a new feature in Zoho CRM regarding List Views. Currently, each user has to manually select or favorite a custom list view in order to make it their default. However, as administrators, we would like

    • Adding Multiple Products (Package) to a Quote

      I've searched the forums and found several people asking this question, but never found an answer. Is ti possible to add multiple products to a quote at once, like a package deal? This seems like a very basic function of a CRM that does quotes but I can't

    • Zoho Commerce in multiple languages

      When will you be able to offer Zoho Commerce in more languages? We sell in multiple markets and want to be able to offer a local version of our webshop. What does the roadmap look like?

    • webinar registration confirmation are landing in SPMA folders

      I am trialing zoho webinar and do not have access to custom domains. When I test user registrations, they are working but the resulting confirmation email is landing in a spam folder. How can I avoid this?

    • Making digital signatures accessible to all: Introducing accessibility controls in Zoho Sign

      Hi there! At Zoho Sign, we are committed to building an inclusive digital experience for all our users. As part of our ongoing efforts to align with Web Content Accessibility Guidelines (WCAG), we’re updating the application with support that will go

    • Is there a way to set Document Owner/Sender via the API

      When sending requests for zoho sign, it would seem zoho uses the id of the person that created the zoho api cred to determine the owner_id, is there a way to set a default for this?

    • Delegates should be able to delete expenses

      I understand the data integrity of this request. It would be nice if there was a toggle switch in the Policy setting that would allow a delegate to delete expenses from their managers account. Some managers here never touch their expense reports, and

    • Add Save button to Expense form

      A save button would be very helpful on the expense form. Currently there is a Save and Close button. When we want to itemize an expense, this option would be very helpful. For example, if we have a hotel expense that also has room service, which is a

    • Multiple organizations under Zoho One

      Hello. I have a long and complicated question. I have a Zoho One account and want to set it up to serve the needs of 6 organizations under the same company. Some of the Zoho One users need to be able to work in more than 1 organization’s CRM and other

    • Error AS101 when adding new email alias

      Hi, I am trying to add apple@(mydomain).com The error AS101 is shown while I try to add the alias.

    • Unbundle feature for composite items

      We receive composite items from our vendors and sell them either individually or create other composite items out of them. So, there is a lot of bundling and unbundling involved with our composite items. Previously, this feature was supported in form

    • Regarding the integration of Apollo.io with Zoho crm.

      I have been seeing for the last 3 months that your Apollo.io beta version is available in Zoho Flow, and this application has not gone live yet. We requested this 2 months ago, but you guys said that 'we are working on it,' and when we search on Google

    • MTD SA in the UK

      Hello ID 20106048857 The Inland Revenue have confirmed that this tax account is registered as Cash Basis In Settings>Profile I have set ‘Report Basis’ as “Cash" However, I see on Zoho on Settings>Taxes>Income Tax that the ‘Tax Basis’ is marked ‘Accrual'

    • workflow not working in subform

      I have the following code in a subform which works perfectly when i use the form alone but when i use the form as a subform within another main form it does not work. I have read something about using row but i just cant seem to figure out what to change

    • Fetch data from another table into a form field

      I have spent the day trying to work this out so i thought i would use the forum for the first time. I have two forms in the same application and when a user selects a customer name from a drop down field and would like the customer number field in the

    • Record comment filter

      Hi - I have a calendar app that we use to track tasks. I have the calendar view set up so that the logged in user only sees the record if they are assigned to the task. BUT there are instances when someone is @ mentioned in the record when they are not

    • How to View Part Inventory and Warehouse Location When Creating a Work Order in Zoho FSM

      Hi everyone, We’re currently setting up Zoho FSM and would like to improve how our team selects parts when creating a Work Order. Right now, when we add a part or item to a Work Order, we can select it from our Zoho Inventory list but we don’t see any

    • FSM too slow today !!

      Anybody else with problem today to loading FSM (WO, AP etc.)?

    • Not able to Sign In in Zoho OneAuth in Windows 10

      I recently reset my Windows 10 system, after the reset when I downloaded the OAuth app and tried to Sign In It threw an error at me. Error: Token Fetch Error. Message: Object Reference not set to an instance of an object I have attached the screenshot

    • Mapping a custom preferred date field in the estimate with the native field in the workorder

      Hi Zoho, I created a field in the estimate : "Preferred Date 1", to give the ability to my support agent to add a preferred date while viewing the client's estimate. However, in the conversion mapping (Estimate to Workorder), I'm unable to map my custom

    • The sending IP (136.143.188.15) is listed on spamrl.com as a source of spam.

      Hi, it just two day when i am using zoho mail for my business domain, today i was sending email and found that message "The sending IP (136.143.188.15) is listed on https://spamrl.com as a source of spam" I hope to know how this will affect the delivery

    • Delegates - Access to approved reports

      We realized that delegates do not have access to reports after they are approved. Many users ask questions of their delegates about past expense reports and the delegates can't see this information. Please allow delegates see all expense report activity,

    • Split functionality - Admins need ability to do this

      Admins should be able to split an expense at any point of the process prior to approval. The split is very helpful for our account coding, but to have to go back to a user and ask them to split an invoice that they simply want paid is a bit of an in

    • Is there a way to request a password?

      We add customers info into the vaults and I wanted to see if we could do some sort of "file request" like how dropbox offers with files. It would be awesome if a customer could go to a link and input a "title, username, password, url" all securely and it then shows up in our team vault or something. Not sure if that is safe, but it's the best I can think of to be semi scalable and obviously better than sending emails. I am open to another idea, just thought this would be a great feature.  Thanks,

    • Single Task Report

      I'd like a report or a way to print to PDF the task detail page. I'd like at least the Task Information section but I'd also like to see the Activity Stream, Status Timeline and Comments. I'd like to export the record and save it as a PDF. I'd like the

    • Next Page