Kaizen #177: Duplicate Check Preferences API vs. Upsert API

Kaizen #177: Duplicate Check Preferences API vs. Upsert API

Hello all!!!

Welcome back to another week of Kaizen. Last week, we discussed Optimizing the Use of Record ID Variables in Zoho CRM Queries.

This week, we will explore two important APIs for managing duplicate records in Zoho CRM - Duplicate Check Preferences API and Upsert Records API. While both of these APIs help manage duplicate records, they serve different purposes and are to be used in different scenarios. 




This post covers:
  • What is the Duplicate Check Preferences API?
    • Use Case
    • Duplicate Check Configurations
    • Updating Duplicate Check Configuration
    • How Duplicate Check Preferences Work During Record Creation/Update?
  • What is the Upsert API?
    • Use Case
  • Differences between Duplicate Check Preferences API and Upsert API

What is the Duplicate Check Preferences API

The Duplicate Check Preferences API is a configuration API that allows you to set up criteria using unique fields to manage duplicates across Leads module including Converted Leads and Contacts
in Zoho CRM. Based on the given criteria, the system will restrict the creation or update of duplicate records. For example, if the Phone field is marked as unique and set for duplicate checks, the system will make sure that no two records have the same phone number during records creation or update.

With the Duplicate Check Preferences API, you can define criteria to find duplicates, such as matching email addresses, phone numbers, or custom fields. During record creation or update, Zoho CRM checks the configured criteria against converted leads or contacts. If a match is found, it triggers a "DUPLICATE_DATA" error and blocks duplicates based on your configuration.

By default, the Insert Records API and Update Records API automatically check for this configuration. If a record matches the defined duplicate criteria, the system prevents its creation or update.

Use case

Zylker, a sales team managing customer contacts in Zoho CRM, often imports leads from various sources like website forms, events, and campaigns. However, the team noticed that many duplicate contacts are being created because the same phone number is associated with some of the contacts. This creates confusion and results in redundant work in their sales process.
By using the Duplicate Check Preferences API, Zylker can prevent the anymore duplicate contacts based on phone numbers (phone is used for an example).

Duplicate Check Configurations

In Zoho CRM, you can configure duplicate check preferences via API in two ways. 

i. Configuration in Leads module
The system checks for duplicates on all the Leads using the specified unique fields. By selecting this option, the duplicates will be checked on all the Converted Leads (custom view) as well. This configuration is applicable across all the layouts.
The following is the configuration for duplicate check in the Leads module via API. 

Request Method: POST

Note:
  • The "module" parameter is mandatory, and its supported value is Leads. 
  • The above URL applies to both Leads and Contacts module configurations.
  •  You must mark at least one field as mandatory. So, the system can check for duplicates based on the value in the unique field within the Leads module when creating a record.
Input JSON

{
    "duplicate_check_preference": {
        "type": "converted_records", //Converted Leads custom view in Leads module
    }
}


Response

{
    "duplicate_check_preference": {
        "code": "SUCCESS",
        "details": {},
        "message": "Duplicate check enabled for converted_records successfully.",
        "status": "success"
    }
}




ii. Configuration in Contacts module

By selecting this option, the system will check for duplicates in the Contacts module using mapped fields.

Input JSON

{
    "duplicate_check_preference": {
        "type": "mapped_module_records",
        "type_configurations": [
            {
                "field_mappings": [
                    {
                        "current_field": {
                            "api_name": "Email",
                            "id": "5725767000000002601"
                        },
                        "mapped_field": {
                            "api_name": "Email",
                            "id": "5725767000000002503"
                        }
                    }
                ],
                "mapped_module": {
                    "api_name": "Contacts",
                    "id": "5725767000000002179"
                }
            }
        ]
    }
}


The "Phone" field from both the Leads and Contacts modules is mapped to identify duplicate records.

How Duplicate Check Preferences Work During Record Creation/Update

Consider Zylker has configured the Contacts module ("type":"mapped_module_records") to check duplicate contacts.
For example, a sales representative at Zylker attempts to add a contact from the Cold Call lead source. However, the same contact already exists in the Contacts module, either from the same or a different lead source.
Since the Contacts module is configured for duplicate check preferences, the system will check for duplicates in the Contacts module using mapped fields. During the record insertion action, the system verifies the Contacts module for any matching records based on these unique identifiers. Based on the result, it either creates the record or throws an error.

Use the Insert Records API to create records in a module.

Request Method: POST

Input JSON

{
    "data": [
        {
            "Company": "Villa Margarita",
            "Last_Name": "Dolan",
            "First_Name": "Brian",
            "Email": "brian@villa.com",
            "Phone": "12345",
            "State": "Texas"
        }
    ]
}


Response


 {

    "data": [

        {

            "code": "DUPLICATE_DATA",

            "details": {

                "api_name": "Email",

                "duplicate_record": {

                    "Owner": {

                        "name": "Patricia Boyle",

                        "id": "5725767000000411001",

                        "zuid": "808233918"

                    },

                    "module": {

                        "api_name": "Contacts",

                        "id": "5725767000000002179"

                    },

                    "id": "5725767000000774010" //The record that already exists in the Contacts module.

                },

                "json_path": "$.data[0].Email",

                "more_records": false

            },

            "message": "duplicate data",

            "status": "error"

        }

    ]

}


Duplicates are also checked during record updates. If a record is updated with a value that already exists in a unique field, the system indicates it as a duplicate and returns a "DUPLICATE_DATA" error.

Update Duplicate Check Configuration

To update the previously enabled Duplicate Check option configured in your account, use the Update Duplicate Check Option API.


The Contacts ("type": "mapped_module_records") configuration is used in the below sample input. 


{
    "duplicate_check_preference": {
        "type": "mapped_module_records",
        "type_configurations": [
            {
                "field_mappings": [
                    {
                        "mapped_field": {
                            "api_name": "Phone", //The previously configured Email field as a unique identifier in Contacts has now been replaced with the Phone field.
                            "name": "Contacts",
                            "id": "5725767000000411001"
                        },
                        "current_field": {
                            "api_name": "Phone",
                            "name": "Leads", //The previously configured Email field as a unique identifier in Leads has now been replaced with the Phone field.
                            "id": "5725767000005381030"
                        }
                    }
                ],
                "mapped_module": {
                    "api_name": "Contacts",
                    "name": "Contacts",
                    "id": "5725767000000002179"
                }
            }
        ]
    }
}

You can also retrieve and delete duplicate configuration.

What is the Upsert API?

The Upsert API is a combination of "update" and "insert". 


How it works?

  • You provide a unique identifier (e.g., email, phone number, or a custom field) to check for existing records.

  • If a record with the same identifier exists, it updates that record.

  • If no matching record is found, it creates a new one.

Flow diagram

Use Case 

Zylker, an e-commerce company, uses Zoho CRM to manage customer data. Customers sign up on Zylker’s online store, and their details are stored in the e-commerce platform like Shopify. Over time, customers may update their contact information, or new customers may sign up. To keep customer records in Zoho CRM accurate and avoid duplicates, Zylker uses the Upsert API to:
  • Update existing customer records details if the given unique identifier, such as email or phone number, matches an existing record in Zoho CRM.
  • Insert a new record if no match is found.  This way, Zylker ensures that new customers are added efficiently.

There are two types of duplicate check fields:

i. System-defined duplicate check fields.
ii. User-defined duplicate check fields.

i. System-defined duplicate check fields
The table below lists the system-defined unique fields. When you create a record using the Upsert API, the system checks these fields for duplicate entries. 

Leads

Email

Contacts

Email

Accounts

Account_Name

Deals

Deal_Name

Campaigns

Campaign_Name

Cases

Subject

Solutions

Solution_Title

Products

Product_Name

Vendors

Vendor_Name

PriceBooks

Price_Book_Name

Quotes

Subject

SalesOrders

Subject

PurchaseOrders

Subject

Invoices

Subject

Custom Module

Name


ii. User-defined duplicate check fields
You can set a normal field as a duplicate check field by enabling "Do not allow duplicate values." Note that you can only set fields with the following field-types as unique— Single Line, Email, Phone, Number, Long Integer, and URL.


Request Method: POST

Sample Input JSON

Here, "Email," a system-defined field, has been used as a duplicate check field.
{
    "data": [
        {
            "Email": "alice@mail.com",
            "First_Name": "Alice",
            "Last_Name": "Brown",
            "Phone": "+1-555-222-3333",
            "Order_Date": "2024-01-10",
            "Order_Value": "$175.00"
        },
        {
            "Email": "pat@mail.com",
            "First_Name": "pat",
            "Last_Name": "Davis",
            "Phone": "+1-555-666-7777",
            "Order_Date": "2024-01-15",
            "Order_Value": "$300.00"
        }
    ],
    "duplicate_check_fields": [
        "Email" //You can set the order in which the system checks for duplicate records by specifying the duplicate_check_field array in the input. 
    ]
}

Response


{
    "data": [
        {
            "code": "SUCCESS",
            "duplicate_field": null,
            "action": "insert",
            "details": {
                "Modified_Time": "2025-02-08T09:43:08-08:00",
                "Modified_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                },
                "Created_Time": "2025-02-08T09:43:08-08:00",
                "id": "5725767000005425036",
                "Created_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                }
            },
            "message": "record added",
            "status": "success"
        },
        {
            "code": "SUCCESS",
            "duplicate_field": "Email",
            "action": "update",
            "details": {
                "Modified_Time": "2025-02-08T09:43:08-08:00",
                "Modified_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                },
                "Created_Time": "2025-02-08T08:52:22-08:00",
                "id": "5725767000005425002",
                "Created_By": {
                    "name": "Patricia Boyle",
                    "id": "5725767000000411001"
                }
            },
            "message": "record updated",
            "status": "success"
        }
    ]
}


For more details on how the Upsert API works, refer to the Kaizen #56 - Upsert Records #API.

Differences between Duplicate Check Preferences API and Upsert API

 

                Duplicate Check API

                           Upsert API

Duplicate Check Preferences can be enabled only in the Leads module.

Supports all modules.

Throws a "DUPLICATE_DATA" error if a duplicate is detected.

Updates the existing record with new field values if a match is found for the unique fields; otherwise, inserts a new record.


Use the above Zoho CRM APIs which fit your use cases.
---------------------------------------------------------------------------------------------------------------------

We trust that this post meets your needs and is helpful. Let us know your thoughts in the comment section or reach out to us at support@zohocrm.com 
Stay tuned for more insights in our upcoming Kaizen posts!

Info

      • Recent Topics

      • File upload support for subforms

        Dear All, Subforms help you add and track data related to your CRM records. We've always supported most of the commonly used field types in subforms, so as to cater to your business requirements. Now we're happy to extend support to file upload fields.

      • Circular record relationship in CRM

        Given there is a lookup in custom module Transactions creating a related list on the chosen Contact record. I. e. a lookup field on Transactions module points to Contacts. The related list on a Contact record can thus have have many transaction records.

      • Multi-currency and Products

        One of the main reasons I have gone down the Zoho route is because I need multi-currency support.  However, I find that products can only be priced in the home currency, We sell to the US and UK.  However, we maintain different price lists for each. 

      • Is it possible to pull the Zoho desk data into Zoho analytics in real time

        Hi - I am looking to add more dashboards and reports of Zoho Desk in analytics. I see there is a schedule to pull the data into analytics, but I'm wondering if there is an option to pull the data in real time instead of a specific interval?

      • Enhance productivity with the revamped Zoho Sheet View

        Hello folks, For some time now, you've been able to use the Zoho Sheet View to quickly edit multiple records or to insert a batch of new records. Its tabular interface allows users to engage in these tasks productively. Despite this, the existing Sheet

      • How to set value of dropdown field to variable

        For the life of me, I can't figure out why the following code won't set input.status to the previousSelection. Steps to reproduce: 1. Click input.complete 2. input.status dropdown updates from '1-Backlog' to '4-Done' 3. Click input.complete again to deselect

      • [Product Update] TimeSheets module is now renamed as Time Logs in Zoho Projects.

        Dear Zoho Analytics customers, As part of the ongoing enhancements in Zoho Projects, the Timesheets module has been renamed to Time Logs. However, the module name will continue to be displayed as Timesheets in Zoho Analytics until the relevant APIs are

      • Issue in Annual Leave

        We created a policy to credit 21 days at the Start of the Year An employee has taken 16 days of leaves thought the year, so we expect to see 5 Leaves remaining right? But Zoho People is Showing 12 Days of Leave Balance Available If we check the Leave

      • [WEBINAR][MEA] Learn how to control your inventory and multi-channel sales with the Zoho Inventory & Zoho Books integration

        Hello there, We are hosting an exclusive live webinar tailored for businesses across the Middle East and African countries, where you'll learn how to take full control of your inventory and multi-channel sales while keeping your accounting perfectly in

      • Less clicks, more connection – the usability edge of Zoho Desk

        Imagine joining a new workplace, eager to prove your skills, partner with peers, learn new tools, and build a strong foundation for your career. The standards you've set could drive pressure into your head. You now discover the organization is using Zoho

      • Tip #3 Automating total item weight calculation for your sales orders in Zoho Inventory

        Hello, Hope the day is treating you well. Last week, we saw how we could automate the calculation of total shipping charges from numerous shipments for your sales orders and invoices. This week, we will see how you can automate the calculation of total item weight for your sales orders.  How does this work? First, you are required to capture the weight of all your items using a custom field. And when you create a sales order, you can either have the system display the individual weights for you or

      • Why Format section gets disabled when we create Merge Template over PDF

        I need some assistance I have a Client who is going to give certificates to users who passes his exam. So, I am using mail merge but in ZOHO writer after I upload the PDF and create merge Template over PDF the format Section gets disabled. My problem

      • Whatsapp Integration on Zoho Campaign

        Team: Can the messages from Zoho Campaign delivered through Whatsapp... now customers no longer are active on email, but the entire campaign module is email based.... when will it be available on whatsapp.... are there any thirdparty providers who can

      • How to calculate separate totals for Product Line Items filtered by category in Quotes?

        Hello! I'm working with Quotes in Zoho CRM and need help calculating conditional totals for line items. Current setup: I have two tables in my Quote template (Zoho Writer), both displaying Quoted Line Items Table 1 (top): Shows all products where Product

      • I Need Help Verifying Ownership of My Zoho Help Desk on Google Search Console

        I added my Zoho desk portal to Google Search Console, but since i do not have access to the html code of my theme, i could not verify ownership of my portal on Google search console. I want you to help me place the html code given to me from Google search

      • Zoho Desk API Documentation missing a required field

        We are trying to create a section using this information. Even after preparing everything based on that page, we still get an error. The error we get is this: {"errorCode":"INVALID_DATA","message":"The data is invalid due to validation restrictions","errors":[{"fieldName":"/translations","errorType":"missing","errorMessage":""}]}

      • Fetch Subform values through a lookup of a lookup

        Hi, I'm having an issue fetching the values of a subform through a lookup field through another lookup field. The code works perfectly to show me the subform row ID, but the second I try to get any readable data, it says "Error occurred. Please contact

      • Good news! Calendar in Zoho CRM gets a face lift

        Dear Customers, We are delighted to unveil the revamped calendar UI in Zoho CRM. With a complete visual overhaul aligned with CRM for Everyone, the calendar now offers a more intuitive and flexible scheduling experience. What’s new? Distinguish activities

      • Turn Decision Box to a button

        Dear all, I need your help on CSS to turn a Decision Box to a Button. I have a Decision Box like: Turn on/off to refresh the information. (on User Input, either True or False, will refresh the information) Now I want to simulate to treat it like a Refresh

      • Emails not being received from a particular domain

        Cannot receive any emails sent from atco.com Domain is in the spam whitelist so should be no reason for it not to be coming through. Have filed a ticket and besides a generic response of we are looking at it - it seems there is no actual support workers

      • Stock Count

        The stock count is a nice new feature, but we cannot figure out how to: 1. Use it without assigning to a person, we have a team or one of multiple do stock counts as do most any company. 2. Add any extra fields to what the "counter" sees. The most important

      • Can you import projects into Zoho Projects yet?

        I see some very old posts asking about importing project records into Zoho Projects. But I can't find anything up to date about the topic. Has this functionality been added? Importing tasks is helpful. But we do have a project where importing projects

      • Send Whatsapp message from Whatsapp template with custom variables

        Hi, I'm trying to do some basic integration for sending WhatsApp messages from Zoho CRM using Zoho Desk whatsapp templates. When creating new whatsapp template in Zoho Desk we can choose ticket related fields as variables but it's not clear how to use

      • How to remove some users in zoho accounts

        How to remove some users in Zoho accounts.

      • Outgoing blocked: Unusual activity detected.

        I just made payment for my Zohomail Today and have been debited so i will like to be Unblocked because this is what it says (Outgoing blocked: Unusual activity detected) Thank you i await your swift responses

      • Integrating with My Own Application and ZOHO CRM Simultaneously

        I have my own WhatsApp bot that uses my WhatsApp business account. I want to use the same phone number to integrate with ZOHO as well. What is the recommended way to do that? Should I integrate my application with ZOHO, forwarding messages whenever the

      • How do i move multiple tickets to a different department?

        Hello, i have several tickets that have been assigned to the wrong department.  I am talking about hundreds of automatically generated ones that come from a separate system. How can i select them all at once to move them to another department in one go? I can select them in "unsassigned open tickets view" but i can't find a "move to another department" option. I also can't seem to assign multiple tickets to the same agent in that same view. Could somebody advice?

      • Transferring CRM Attachments to Workdrive

        relatedrecords = zoho.crm.getRelatedRecords("Attachments","Conditions",conId); attachid = List(); for each ele in relatedrecords { attachementId = ele.get("id"); attachid.add(attachementId); } for each ele in attachid { counter = 1; downloadFile = invokeurl [ url: "https://www.zohoapis.com/crm/v2/Conditions/" + conId + "/Attachments/" + ele type: GET connection : "work_drive" ]; resp2 = zoho.crm.attachFile("Deals",dealId,downloadFile); resp3 = zoho.workdrive.uploadFile(downloadFile, dealWD, "PlaceHolder"+counter+"",

      • New Customization options in the module builder: Quick Create and Detail view

        Hello everyone, We have introduced two new components to the module builder: Quick create and Detail view. The Quick Create Component It is a mini form used to create a record and associate it to the parent record from a lookup field. For example, if you have a Deals lookup in the Contacts module, then you can associate existing deals or create a deal and associate it with the contact. You can customize this Quick Create form by adding standard as well as custom fields. There is no limit to the number

      • unblock my zoho mail account. outlines@zoho.com

        please unblock my zoho mail account, outlines@zoho.com

      • SMTP email sending problem

        Hello, I've sent emails before, but you haven't responded. Please respond. My work is being disrupted. I can't send emails via SMTP. Initially, there were no problems, but now I'm constantly receiving 550 bounce errors. I can't use the service I paid

      • Mailk got blocked / Inquiry About Email Sending Limits and Upgrade Options

        Dear Zoho Support Team, My name is Kamr Elsayed I created this account to use for applying for vocational training in Germany. As part of this process, I send multiple emails to different companies. However, after sending only 8 emails today, I received

      • Forwarder

        Hi, I tried to add a forwarder from which emails are sent to my main zoho account email . However, it asks me for a code that should be received at the forwarder email, which is still not activated to send to my zoho emial account. So how can I get the

      • No chat option

        Chat option is not supported.

      • Direct “Add to Google Calendar” Option in Zoho Meeting

        Hello Zoho Meeting Team, Hope you are doing well. We would like to request an enhancement related to the “Add to Calendar” functionality in Zoho Meeting. Currently, when we open Zoho Meeting and view our meetings under My Calendar, there is an Add to

      • Default Tagging on API-generated Transactions

        If one assigns tags to an Item or Customer, those tags get auto-populated in each line item of an Invoice or Sales Order when one creates those documents. However, if one creates the Sales Order or Invoice via the API (either directly coding or using

      • DKIM cannot be enabled for the domain as no verified default selector present

        Hi Support Team, For Domain DKIM record trying to enable status. but showing error "DKIM cannot be enabled for the domain as no verified default selector present" So, please resolve the issue. Thank you.

      • unable to send message reason 554 5.1.8 Email outgoing blocked

        unable to send message reason 554 5.1.8 Email outgoing blocked

      • I can't log in to my account on Thunderbird

        I've just had to rebuild my PC (calamitous mess from Microsoft with Win10/Win 11 'upgrade' - they confirmed I had to start with a new build). I have used Zoho mail for years via Mozilla Thunderbird, but now I've had to download the latest version of TBird,

      • Should I Use DMARC?

        When I configure Zoho Mail's DMARC settings, it's mandatory to fill in the RUA and RUF (Aggregate notification email address*, Forensic notification email address*) addresses. When we enter an email address in these fields, we receive reports from the

      • Next Page