Kaizen #177: Duplicate Check Preferences API vs. Upsert API

Kaizen #177: Duplicate Check Preferences API vs. Upsert API

Hello all!!!

Welcome back to another week of Kaizen. Last week, we discussed Optimizing the Use of Record ID Variables in Zoho CRM Queries.

This week, we will explore two important APIs for managing duplicate records in Zoho CRM - Duplicate Check Preferences API and Upsert Records API. While both of these APIs help manage duplicate records, they serve different purposes and are to be used in different scenarios. 




This post covers:
  • What is the Duplicate Check Preferences API?
    • Use Case
    • Duplicate Check Configurations
    • Updating Duplicate Check Configuration
    • How Duplicate Check Preferences Work During Record Creation/Update?
  • What is the Upsert API?
    • Use Case
  • Differences between Duplicate Check Preferences API and Upsert API

What is the Duplicate Check Preferences API

The Duplicate Check Preferences API is a configuration API that allows you to set up criteria using unique fields to manage duplicates across Leads module including Converted Leads and Contacts
in Zoho CRM. Based on the given criteria, the system will restrict the creation or update of duplicate records. For example, if the Phone field is marked as unique and set for duplicate checks, the system will make sure that no two records have the same phone number during records creation or update.

With the Duplicate Check Preferences API, you can define criteria to find duplicates, such as matching email addresses, phone numbers, or custom fields. During record creation or update, Zoho CRM checks the configured criteria against converted leads or contacts. If a match is found, it triggers a "DUPLICATE_DATA" error and blocks duplicates based on your configuration.

By default, the Insert Records API and Update Records API automatically check for this configuration. If a record matches the defined duplicate criteria, the system prevents its creation or update.

Use case

Zylker, a sales team managing customer contacts in Zoho CRM, often imports leads from various sources like website forms, events, and campaigns. However, the team noticed that many duplicate contacts are being created because the same phone number is associated with some of the contacts. This creates confusion and results in redundant work in their sales process.
By using the Duplicate Check Preferences API, Zylker can prevent the anymore duplicate contacts based on phone numbers (phone is used for an example).

Duplicate Check Configurations

In Zoho CRM, you can configure duplicate check preferences via API in two ways. 

i. Configuration in Leads module
The system checks for duplicates on all the Leads using the specified unique fields. By selecting this option, the duplicates will be checked on all the Converted Leads (custom view) as well. This configuration is applicable across all the layouts.
The following is the configuration for duplicate check in the Leads module via API. 

Request Method: POST

Note:
  • The "module" parameter is mandatory, and its supported value is Leads. 
  • The above URL applies to both Leads and Contacts module configurations.
  •  You must mark at least one field as mandatory. So, the system can check for duplicates based on the value in the unique field within the Leads module when creating a record.
Input JSON

{
    "duplicate_check_preference": {
        "type": "converted_records", //Converted Leads custom view in Leads module
    }
}


Response

{
    "duplicate_check_preference": {
        "code": "SUCCESS",
        "details": {},
        "message": "Duplicate check enabled for converted_records successfully.",
        "status": "success"
    }
}




ii. Configuration in Contacts module

By selecting this option, the system will check for duplicates in the Contacts module using mapped fields.

Input JSON

{
    "duplicate_check_preference": {
        "type": "mapped_module_records",
        "type_configurations": [
            {
                "field_mappings": [
                    {
                        "current_field": {
                            "api_name": "Email",
                            "id": "5725767000000002601"
                        },
                        "mapped_field": {
                            "api_name": "Email",
                            "id": "5725767000000002503"
                        }
                    }
                ],
                "mapped_module": {
                    "api_name": "Contacts",
                    "id": "5725767000000002179"
                }
            }
        ]
    }
}


The "Phone" field from both the Leads and Contacts modules is mapped to identify duplicate records.

How Duplicate Check Preferences Work During Record Creation/Update

Consider Zylker has configured the Contacts module ("type":"mapped_module_records") to check duplicate contacts.
For example, a sales representative at Zylker attempts to add a contact from the Cold Call lead source. However, the same contact already exists in the Contacts module, either from the same or a different lead source.
Since the Contacts module is configured for duplicate check preferences, the system will check for duplicates in the Contacts module using mapped fields. During the record insertion action, the system verifies the Contacts module for any matching records based on these unique identifiers. Based on the result, it either creates the record or throws an error.

Use the Insert Records API to create records in a module.

Request Method: POST

Input JSON

{
    "data": [
        {
            "Company": "Villa Margarita",
            "Last_Name": "Dolan",
            "First_Name": "Brian",
            "Email": "brian@villa.com",
            "Phone": "12345",
            "State": "Texas"
        }
    ]
}


Response


 {

    "data": [

        {

            "code": "DUPLICATE_DATA",

            "details": {

                "api_name": "Email",

                "duplicate_record": {

                    "Owner": {

                        "name": "Patricia Boyle",

                        "id": "5725767000000411001",

                        "zuid": "808233918"

                    },

                    "module": {

                        "api_name": "Contacts",

                        "id": "5725767000000002179"

                    },

                    "id": "5725767000000774010" //The record that already exists in the Contacts module.

                },

                "json_path": "$.data[0].Email",

                "more_records": false

            },

            "message": "duplicate data",

            "status": "error"

        }

    ]

}


Duplicates are also checked during record updates. If a record is updated with a value that already exists in a unique field, the system indicates it as a duplicate and returns a "DUPLICATE_DATA" error.

Update Duplicate Check Configuration

To update the previously enabled Duplicate Check option configured in your account, use the Update Duplicate Check Option API.


The Contacts ("type": "mapped_module_records") configuration is used in the below sample input. 


{
    "duplicate_check_preference": {
        "type": "mapped_module_records",
        "type_configurations": [
            {
                "field_mappings": [
                    {
                        "mapped_field": {
                            "api_name": "Phone", //The previously configured Email field as a unique identifier in Contacts has now been replaced with the Phone field.
                            "name": "Contacts",
                            "id": "5725767000000411001"
                        },
                        "current_field": {
                            "api_name": "Phone",
                            "name": "Leads", //The previously configured Email field as a unique identifier in Leads has now been replaced with the Phone field.
                            "id": "5725767000005381030"
                        }
                    }
                ],
                "mapped_module": {
                    "api_name": "Contacts",
                    "name": "Contacts",
                    "id": "5725767000000002179"
                }
            }
        ]
    }
}

You can also retrieve and delete duplicate configuration.

What is the Upsert API?

The Upsert API is a combination of "update" and "insert". 


How it works?

  • You provide a unique identifier (e.g., email, phone number, or a custom field) to check for existing records.

  • If a record with the same identifier exists, it updates that record.

  • If no matching record is found, it creates a new one.

Flow diagram

Use Case 

Zylker, an e-commerce company, uses Zoho CRM to manage customer data. Customers sign up on Zylker’s online store, and their details are stored in the e-commerce platform like Shopify. Over time, customers may update their contact information, or new customers may sign up. To keep customer records in Zoho CRM accurate and avoid duplicates, Zylker uses the Upsert API to:
  • Update existing customer records details if the given unique identifier, such as email or phone number, matches an existing record in Zoho CRM.
  • Insert a new record if no match is found.  This way, Zylker ensures that new customers are added efficiently.

There are two types of duplicate check fields:

i. System-defined duplicate check fields.
ii. User-defined duplicate check fields.

i. System-defined duplicate check fields
The table below lists the system-defined unique fields. When you create a record using the Upsert API, the system checks these fields for duplicate entries. 

Leads

Email

Contacts

Email

Accounts

Account_Name

Deals

Deal_Name

Campaigns

Campaign_Name

Cases

Subject

Solutions

Solution_Title

Products

Product_Name

Vendors

Vendor_Name

PriceBooks

Price_Book_Name

Quotes

Subject

SalesOrders

Subject

PurchaseOrders

Subject

Invoices

Subject

Custom Module

Name


ii. User-defined duplicate check fields
You can set a normal field as a duplicate check field by enabling "Do not allow duplicate values." Note that you can only set fields with the following field-types as unique— Single Line, Email, Phone, Number, Long Integer, and URL.


Request Method: POST

Sample Input JSON

Here, "Email," a system-defined field, has been used as a duplicate check field.
{
    "data": [
        {
            "Email": "alice@mail.com",
            "First_Name": "Alice",
            "Last_Name": "Brown",
            "Phone": "+1-555-222-3333",
            "Order_Date": "2024-01-10",
            "Order_Value": "$175.00"
        },
        {
            "Email": "pat@mail.com",
            "First_Name": "pat",
            "Last_Name": "Davis",
            "Phone": "+1-555-666-7777",
            "Order_Date": "2024-01-15",
            "Order_Value": "$300.00"
        }
    ],
    "duplicate_check_fields": [
        "Email" //You can set the order in which the system checks for duplicate records by specifying the duplicate_check_field array in the input. 
    ]
}

Response


{
    "data": [
        {
            "code": "SUCCESS",
            "duplicate_field": null,
            "action": "insert",
            "details": {
                "Modified_Time": "2025-02-08T09:43:08-08:00",
                "Modified_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                },
                "Created_Time": "2025-02-08T09:43:08-08:00",
                "id": "5725767000005425036",
                "Created_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                }
            },
            "message": "record added",
            "status": "success"
        },
        {
            "code": "SUCCESS",
            "duplicate_field": "Email",
            "action": "update",
            "details": {
                "Modified_Time": "2025-02-08T09:43:08-08:00",
                "Modified_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                },
                "Created_Time": "2025-02-08T08:52:22-08:00",
                "id": "5725767000005425002",
                "Created_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                }
            },
            "message": "record updated",
            "status": "success"
        }
    ]
}


For more details on how the Upsert API works, refer to the Kaizen #56 - Upsert Records #API.

Differences between Duplicate Check Preferences API and Upsert API

 

                Duplicate Check API

                           Upsert API

Duplicate Check Preferences can be enabled only in the Leads module.

Supports all modules.

Throws a "DUPLICATE_DATA" error if a duplicate is detected.

Updates the existing record with new field values if a match is found for the unique fields; otherwise, inserts a new record.


Use the above Zoho CRM APIs which fit your use cases.
---------------------------------------------------------------------------------------------------------------------

We trust that this post meets your needs and is helpful. Let us know your thoughts in the comment section or reach out to us at support@zohocrm.com 
Stay tuned for more insights in our upcoming Kaizen posts!

Info

    • Recent Topics

    • Is it possible to create a word cloud chart in ZoHo Analystics?

      Hi there, I have a volume of transaction text that I would like to analyse using word cloud (or other approcah to detect and present word frequency in a dataset). For example, I have 50,000 records describing menu items in restaurants. I want to be able

    • Help integrating Aircall into Zoho CRM

      Hi all, We are need to get better AirCall integration into Zoho. We have configured in the Aircall dashboard but we have not done the Zoho side. We cannot see Zoho CRM information on incoming calls. And we calls and text's are not being logged. Specifically

    • Shopify Extension no longer working in CRM

      Zoho CRM and Extension are no longer working properly. We have used Shopify Basic for years. All of the sudden it stopped working, then started working again, and now no longer working again. There are several ways for Personal Information to sync from

    • How to keep track of bags, cans, drums of inventory?

      We buy and sell products that are packaged in bags 🛍️, cans🥫, drums🛢️, etc. with batch numbers. When we get a shipment of one of the products, how do we track we received (say) 10 cans each of 5L of a product and maybe we received 10 cans of another

    • Easy third party collaboration: Zoho Flow Integration

      Third-party integrations extend your project functionality with improved workflow, and data processing. With Zoho Flow integration you can include multiple third party actions into your project or task automation. Add Zoho Flow action in Workflow rules

    • Introducing Configure, Price, Quote (CPQ) in Zoho CRM | Public early access 2023

      Greetings Customers! We're excited to announce that the CPQ feature will soon be available in Zoho CRM and accessible to all accounts using the Professional, Enterprise, and Ultimate editions. We've opened this feature to select customers, and it'll be

    • how to edit the converted lead records?

      so I can fetch the converted leads records using API (COQL), using this endpoint https://www.zohoapis.com/crm/v5/coql and using COQL filter Converted__s=true for some reasons I need to change the value from a field in a converted lead record. When I try

    • Zoho Bookings <> CRM integration

      Hello Zoho community! We are enabling our Zoho Bookings <> CRM integration. What is the workflow if the integration detects that the contact already exists in the CRM? Does it create a duplicate record? Overwrite the record? Merge the record? (in this

    • Cannot reject empty expense report

      Hello, We are currently having issues with two empty expense reports where if we try to reject them, either manually or through the REST API, we get error 114016, which says some of the expenses have already been billed and must be removed. I'd appreciate

    • Use of Zia within Forms

      I have noticed that recently you have added Zia to help create forms from scratch. Would it be possible to add Zia functionality to free text fields for form submissions or if this is already on the roadmap an ETA for this please? We have a lot of field

    • Default in fields on Form B based on the user selection in Form A

      Hi Everyone, I have added an action button to a form report to bring up a new form based on user selection, see it indicated in red below: Then when the ne form loads, I want to default in some of the fields based on the record the user was selected on.

    • Auto-sync field of lookup value

      This feature has been requested many times in the discussion Field of Lookup Announcement and this post aims to track it separately. At the moment the value of a 'field of lookup' is a snapshot but once the parent lookup field is updated the values diverge.

    • Last activity time is acting like last modified time

      When i edit the description or any field in the potential, account, contact and lead, the Last Activity Time is being updated like the Modified Time. This is messing all workflows and reports and we are unable to track real last time of activities like

    • Enhancements to the formula field in Zoho CRM: Auto-refresh formulas with the "Now" function, stop formula executions based on criteria, and include formulas within formulas

      Dear Customers, We hope you're well! By their nature, modern businesses rely every day on computations, whether it's to calculate the price of a product, assess ROI, evaluate the lifetime value of a customer, or even determine the age of a record. With

    • HEX/RGB Color Input in Visual Editor

      Hello Zoho Pagesense Team, We hope you're doing well. We’d like to submit a feature request to improve the color selection options in the Pagesense popup editor. Current Limitation: Currently, to set text colors, users must move the color slider manually.

    • Add Comprehensive Accessibility Features to Zoho Writer

      Hello Zoho Writer Team, We hope you are doing well. We would like to submit a feature request to enhance Zoho Writer with a full set of accessibility tools, similar to the accessibility options already available in the Zoho Desk agent interface. 🚧 Current

    • CRM and Campaigns- tags not integrating?

      Hi! I am setting up an automation in zoho campaigns and it says the crm is integrated...but when I want to use a trigger of a certain tag I created for people in the crm to denote those who purchased, it doesnt have that tag available as an option in

    • Maximum limit of rows exceeded

      I am trying to add a row to a spreadsheets that has fewer than 60 rows. I keep getting an error message that says I have exceeded the maximum limit of 65,536 rows. Any ideas out there?

    • Can't change form's original name in URL

      Hi all, I have been duplicating + editing forms for jobs regarding the same department to maintain formatting + styling. The issue I've not run into is because I've duplicated it from an existing form, the URL doesn't seem to want to update with the new

    • Tip #7: 5 formas de mantener tus listas de correo electrónico

      Tips and Tricks #7, adaptado de Susmit Sen ¡Hola, comunidad! Esta semana volvemos con nuestros Tips and Tricks para Zoho Campaigns. En esta ocasión, voy a darte algunos consejos para mantener limpias tus listas de correo electrónico, y empezar el 2021

    • Mail Merge related Lists

      When I try to merge an associated list the fields do not have a checkbox to choose them as it does in the tutorials. Am I missing something? Any ideas would be appreciated.

    • create-a-purchase-receive API issues

      Hello all, I'm trying to use the create a purchase receive inventory API documented here (https://www.zoho.com/inventory/api/v1/purchasereceives/#create-a-purchase-receive) however when I do am getting the following error {"code":9,"message":"The purchase

    • Zoho Pagesense really this slow??? 5s delay...

      I put the pagesense on my website (hosted by webflow and fast) and it caused a 5s delay to load. do other people face similar delays?

    • Heatmap: Missing Content Elements - Zoho Page Sense

      Hi there, I'm trying out Zoho Page Sense to generate heatmaps for my Wix pages. My problem: Some parts of my Wix pages don't show on my heatmap, the heatmap is just empty there. Maybe Wix default lazy loading setting causes this error? Is there a way

    • Marketing Tip #3: Use social proof to build trust

      People trust people. Showcasing reviews, testimonials, or “bestseller” badges on your product pages can nudge hesitant buyers toward purchase. Try this today: Add one customer testimonial or highlight your top-selling product on your homepage. Or, do

    • Writing by Hand in "Write" Notes

      Hi there! I just downloaded this app a few moments ago, and I was wondering if there was a way to write things by hand in "Write" mode instead of just typing in the keyboard. It would make things a bit more efficient for me in this moment. Thanks!

    • Make CAMPAIGNS email look as simple as possible

      Hi there I'm trying to make my Campaigns email look as much like a normal email as possible. I'm a bit stuck with the "justification" of the email email block. Can I LEFT JUSTIFY the "whole email" to make it look "normal"? (Please see screenshot attached)

    • Zoho Sign Reminder email template

      Is there a template we can edit for the reminder emails? I don't see it in Settings / Choose a template

    • Preview an upload PDF or File

      I have a form where the customer has to upload a file (normally PDF - never jpeg)  When in report view I want to be able to preview the uploaded file without having to download it.  If I click on the upload it downloads to my computer, I want to be able

    • How to filter emails by Reply-to field?

      I receive a very particular newsletter from an association A registered on a website W (that is used by many associations), and the emails fields are not great: the From just contains the generic website's W's email, while A is only mentioned in the Reply-to

    • How to invite friends on other social media platforms to one of my group chats in arattai?

      Hello, I have formed chat groups in arattai. I want to invite my friends on other social media platforms like WhatsApp/ FB to one of my groups. Different friends would be invited to different groups. How to share an invite link of one of my groups to

    • Line spacing

      I coudn't decrease the line spacing to space smaller then a single line. There is too much space between the lines that make the document look ugly. Please fix that. Liran. fonar

    • Google Fonts Integration in Pagesense Popup Editor

      Hello Zoho Pagesense Team, We hope you're doing well. We’d like to submit a feature request to enhance Zoho Pagesense’s popup editor with Google Fonts support. Current Limitation: Currently, Pagesense offers a limited set of default fonts. Google Fonts

    • Control Position of “X” (Close) Button in Popup Editor

      Hello Zoho PageSense Team, We hope you're doing well. We would like to request a customization improvement in the PageSense popup editor. Current Limitation: Currently, the position of the “X” (close) button is fixed and cannot be customized in the popup

    • Add Standalone “Save” Button in Pagesense Popup Editor

      Hello Zoho Pagesense Team, We hope you're doing well. We would like to request an important usability improvement in the Pagesense popup editor. Current Limitation: There is currently no dedicated Save button while building a popup. The only way to save

    • Text Direction Control in Pagesense Popup Editor

      Hello Zoho Pagesense Team, We hope you're doing well. We’d like to submit a feature request to add text direction control in the Pagesense popup editor. Current Limitation: Currently, the popup editor does not provide native support for RTL (Right-to-Left)

    • Autosave in Pagesense Popup Editor

      Hello Zoho Pagesense Team, We hope you're doing well. We’d like to submit a feature request to enhance the Pagesense editor with an autosave functionality. Current Limitation: Currently, changes made in the Pagesense editor must be manually saved. In

    • Billing frequency is not displayed correctly.

      Hello There is an issue while displaying the billing frequency on a subscription quote. I am able to activate the subscription details and get this lovely overview: If I am adding a Plan which I charge quarterly, first of all it should be possible to

    • Creating Secret via Vault API

      Hi I am trying to create a secret through vault api.  This is the response I get. One thing I am not sure is how to decrypt the secretdata, how to get the secrettypeid? {     "operation": {         "result": {             "error_code": "",             "message": "Sorry, we are unable to process your request.",

    • Zoho CRM custom fields not showing in zoho creator

      Hi Team, I have created a Products form with Zoho CRM integration and connected it to Products module of CRM. But when I see the reports of Products in Zoho creator then I am not able to see custom fields of Products module. Only standard fields of Products

    • Next Page