Kaizen #92 - Bulk Read in PHP SDK

Kaizen #92 - Bulk Read in PHP SDK

Heya! Welcome back to the PHP SDK fold in our Kaizen series.

In this post, we will explore how to use bulk read APIs and how to structure the criteria format for different field data types in PHP SDK based on v4 APIs. This will ease your process of handling large volumes of data in an efficient and streamlined manner. 

What does Bulk Read API do?

The Bulk Read API helps you to fetch large volumes of data that meet specific criteria in a single request. Unlike the GET Records API , which has a limitation of 200 records per call, the Bulk Read API allows you to extract bulk data, 200k records, for data synchronisation purposes. This prevents the need for you to fire multiple API calls which can easily drain your daily credits.

This Bulk Read is asynchronous, thus the result for your request will not be available immediately. So, you will have to either poll the bulk read job status API to know the status or specify a callback URL for Zoho CRM to send notifications regarding the process. 

The CRM system processes your request and returns the results in either a CSV or ICS (for Meetings/Events module) zipped file. To download your result, use the job ID given in the response of bulk read job API .

This entire process can be wrapped into three stages as given below, 
  • Creating a Bulk Read Job 
  • Checking the status of a Bulk Read Job
  • Downloading the CSV/ICS data zip

Create a Bulk Read Job

1. Adding Callback URL

To get notified of the progress and results of the bulk read job, add a callback URL to the request body using the following format. 
$requestWrapper = new RequestWrapper();

$callbackVar = new CallBack();

$callbackVar->setUrl(" your_callback_url ");

$callbackVar->setMethod(new Choice("post"));

$requestWrapper->setCallback($callbackVar);
The method for callback URL is typically set to POST as it helps the client to send the required data along with the url to handle the callback appropriately.

2. Structuring a Query

A query comprises multiple parameters such as your desired module, fields, custom view ID, page value, file-type and criteria.

A criterion is typically a filter/condition used in a query to narrow down the data to your desired set of records. Criteria can include one or more conditions and you can combine them using group operators like 'and' and 'or' to create more complex filtering logic. 

Each condition in the criteria comprises three components.
  • Field - It refers to the field API name of the record, which you want to use for filtering purpose.
  • Comparator - It determines how the value of the specified field has to be evaluated.
  • Value - It represents the value of the field against which the record has to be evaluated.
This helps you to retrieve only the relevant records, thereby optimising the performance and reducing the amount of data transferred.

To enhance the understanding of how to structure a criterion, it is necessary to address these group operators and comparators before delving into it.

Group Operators 

Group Operators are used to perform logical operations on multiple conditions within a query. They help you in grouping multiple conditions together to create complex search criteria.The possible group operators are 'and' and 'or'.
  • and - It specifies that all conditions within the group must be satisfied for a record to be included in the query result.
  • or - It specifies that any of the conditions within the group can be satisfied for a record to be included in the query result.
These operators provide flexibility in constructing queries and retrieve the desired set of records from your CRM system with ease.

Comparators

Comparators are used in a query condition to specify the type of comparison that should be performed between a field and a value. They define the relationship between the field and the value and determine how the records are filtered. Refer to this help document to find the possible comparators with their field datatypes. 

An example of each datatype is given in PHP for your better understanding. 
Data Type
Example
  Number

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field ->setAPIName("Amount");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("greater_than"));
$conditionVar->setValue(10000);
  Multi-select 

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field ->setAPIName("owner");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("in"));
$owner = ["5545974393001", "5545974393011"]
$conditionVar->setValue($owner);
  DateTime                                   

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field->setAPIName("Created_Time");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("between"));
$createdTime = [date_create("2023-05-01T17:58:47+05:30")->setTimezone(new \DateTimeZone(date_default_timezone_get())), date_create("2023-06-01T17:58:47+05:30")->setTimezone(new \DateTimeZone(date_default_timezone_get()))];
$conditionVar->setValue($createdTime);
  Boolean

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field ->setAPIName("Email_Opt_Out");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("equal"));
$conditionVar->setValue(false);
  Lookup

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field ->setAPIName("owner.last_name");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("equal"));
$conditionVar->setValue("Boyle");

Structuring Simple Criteria

Below is a query format with a simple criterion of two conditions to filter your records from the given module. 

$criteriaVar = new Criteria();

$criteriaVar->setGroupOperator(new Choice("operatorValue"));

$conditionVar1 = new Criteria();

$fieldVar = new MinifiedFields();

$fieldVar->setAPIName("Field_API_Name");

$conditionVar1->setField($fieldVar);

$conditionVar1->setComparator(new Choice("comparatorValue"));

$conditionVar1->setValue("Field_Value");

$conditionVar2 = new Criteria();

$field = new MinifiedFields();

$field->setAPIName("Field_API_Name");

$conditionVar2->setField($fieldVar);

$conditionVar2->setComparator(new Choice("comparatorValue"));

$conditionVar2->setValue("Field_Value");

$criteriaVar->setGroup([$conditionVar1,$conditionVar2]);

$query->setCriteria($criteriaVar);

$requestWrapper->setQuery($query);

In the above format, the group operator is used for two different conditions.

Following is the json format of this simple criteria.
{
    "query": {
        "criteria": {
            "group_operator": "operatorValue",
            "group": [
                {
                    "comparator": "comparatorValue",
                    "field": {
                        "api_name": "Field_API_Name"
                    },
                    "value": "Field_Value"
                },
                {
                    "comparator": "comparatorValue",
                    "field": {
                        "api_name": "Field_API_Name"
                    },
                    "value": "Field_Value"
                }
            ]
        }
    }
}

Structuring Complex Criteria

To structure a criterion with four different conditions follow the below format.

//top-level group criteria with 2 different groups comprising of 2 different conditions each

$criteria = new Criteria();

$criteria->setGroupOperator(new Choice("operatorValue"));

//group1 with 2 different conditions

$groupVar1 = new Criteria();

$groupVar1->setGroupOperator(new Choice("operatorValue"));

$conditionVar11 = new Criteria();

$field11 = new MinifiedFields();

$field11->setAPIName("Field_API_Name");

$conditionVar11->setField($field11);

$conditionVar11->setComparator(new Choice("comparatorValue"));

$conditionVar11->setValue("Value");

$conditionVar12 = new Criteria();

$field12 = new MinifiedFields();

$field12->setAPIName("Field_API_Name");

$conditionVar12->setField($field21);

$conditionVar12->setComparator(new Choice("comparatorValue"));

$conditionVar12->setValue("Value");

$groupVar1->setGroup([$conditionVar11, $conditionVar12]);

//group2 with 2 different conditions

$groupVar2 = new Criteria();

$groupVar2->setGroupOperator(new Choice("operatorValue"));

$conditionVar21 = new Criteria();

$field21 = new MinifiedFields();

$field21->setAPIName("Field_API_Name");

$conditionVar21->setField($field21);

$conditionVar21->setComparator(new Choice("comparatorValue"));

$conditionVar21->setValue("Value");

$conditionVar22 = new Criteria();

$field22 = new MinifiedFields();

$field22->setAPIName("Field_API_Name");

$conditionVar22->setField($field22);

$conditionVar22->setComparator(new Choice("comparatorValue"));

$conditionVar22->setValue("Value");

$groupVar2->setGroup([$conditionVar21, $conditionVar22]);

//pushing the groups to the top-level group criteria

$criteria->setGroup([$groupVar1, $groupVar2]);

In the above format, two group variables ($groupVar1 and $groupVar2) are created, each representing a nested group of conditions. Both the groups are further assigned to the top-level group variable ($criteria). 

The json arrangement of the above format is given for your clarity.
{
    "query": {
        "criteria": {
            "group_operator": "operatorValue",
            "group": [
                {
                    "group_operator": "operatorValue",
                    "group": [
                        {
                            "comparator": "comparatorValue",
                            "field": {
                                "api_name": "Field_API_Name"
                            },
                            "value": "Field_Value"
                        },
                        {
                            "comparator": "comparatorValue",
                            "field": {
                                "api_name": "Field_API_Name"
                            },
                            "value": "Field_Value"
                        }
                    ]
                },
                {
                    "group_operator": "operatorValue",
                    "group": [
                        {
                            "comparator": "comparatorValue",
                            "field": {
                                "api_name": "Field_API_Name"
                            },
                            "value": "Field_Value"
                        },
                        {
                            "comparator": "comparatorValue",
                            "field": {
                                "api_name": "Field_API_Name"
                            },
                            "value": "Field_Value"
                        }
                    ]
                }
            ]
        }
     }
}

The default file type is CSV. To bulk read the Events module always prefer the ICS file type. Refer to this resource to fetch the custom view IDs, module and field API names. 

Below is a sample code for creating a bulk read job with compound criteria. The criteria follow this arrangement ((1 and 2) or (3 or 4)) of conditions.

<?php

use com\zoho\crm\api\bulkread\BulkReadOperations;
use com\zoho\crm\api\bulkread\CallBack;
use com\zoho\crm\api\util\Choice;
use com\zoho\crm\api\bulkread\Query;
use com\zoho\crm\api\bulkread\Criteria;
use com\zoho\crm\api\bulkread\RequestWrapper;
use com\zoho\crm\api\modules\MinifiedModule;
use com\zoho\crm\api\fields\MinifiedFields;
require_once "vendor/autoload.php";

class CreateBulkReadJob
{
public static function initialize()
    {
        // Add initialisation code
        // Refer to this article for more help
    }
    public static function createBulkReadJob(string $moduleAPIName)
    {
        $bulkReadOperations = new BulkReadOperations();
        $requestWrapper = new RequestWrapper();
        $callback = new CallBack();
        $callback->setUrl("https://www.callback.com/example");
        $callback->setMethod(new Choice("post"));
        $requestWrapper->setCallback($callback);

        //query
        $query = new Query();
        $module = new MinifiedModule();
        $module->setAPIName($moduleAPIName);
        $query->setModule($module);
        $query->setCvid("34770610087501");
        $query->setFields(["Last_Name"]);
        $query->setPage(1);

        //top-level group criteria with 2 different groups comprising of 2 different conditions each
        $criteria = new Criteria();
        $criteria->setGroupOperator(new Choice("or"));
        $criteriaList = array();

        //group1 with 2 different conditions
        $groupVar1 = new Criteria();
        $groupVar1->setGroupOperator(new Choice("and"));

        $conditionVar11 = new Criteria();
        $field11= new MinifiedFields();
        $field11->setAPIName("Last_Name");
        $conditionVar11->setField($field11);
        $conditionVar11->setComparator(new Choice("equal"));
        $conditionVar11->setValue("Boyle");
        
        $conditionVar12 = new Criteria();
        $field12 = new MinifiedFields();
        $field->setAPIName("Owner");
        $conditionVar12->setField($field12);
        $conditionVar12->setComparator(new Choice("in"));
        $owner = array("5545974000000393001","5545974000000393011");
        $conditionVar12->setValue($owner);
        $groupVar1->setGroup([$conditionVar11, $conditionVar12]);

         //group2 with 2 different conditions
        $groupVar2 = new Criteria();
        $groupVar2->setGroupOperator(new Choice("or"));

        $conditionVar21 = new Criteria();
        $field21 = new MinifiedFields();
        $field21->setAPIName("Company");
        $conditionVar21->setField($field21);
        $conditionVar21->setComparator(new Choice("equal"));
        $conditionVar21->setValue("Morlong Associates");
        
        $conditionVar22 = new Criteria();
        $field22= new MinifiedFields();
        $field22->setAPIName("Created_Time");
        $conditionVar22->setField($field22);
        $conditionVar22->setComparator(new Choice("between"));
        $createdTime = array(date_create("2023-04-15T17:58:47+05:30")->setTimezone(new \DateTimeZone(date_default_timezone_get())), date_create("2023-06-01T17:58:47+05:30")->setTimezone(new \DateTimeZone(date_default_timezone_get())));
        $conditionVar22->setValue($createdTime);
        $groupVar2->setGroup([$conditionVar21, $conditionVar22]);

         //pushing the groups to the top-level group criteria
        $criteria->setGroup($criteriaList);
        $query->setCriteria($criteria);
        $requestWrapper->setQuery($query);
        // $requestWrapper->setFileType(new Choice("ics")); for Events module
        $response = $bulkReadOperations->createBulkReadJob($requestWrapper);
        //Add your code to handle the response received in $response
        // For more details, refer here.
    }
}
CreateBulkReadJob::initialize();
$moduleAPIName = "Leads";
CreateBulkReadJob::createBulkReadJob($moduleAPIName);
?>

2. Status of a Bulk Read Job

To keep track of the status of your bulk read job, use the id provided in the details section of your bulk read job response. 

Here is a sample code for checking the status of a bulk read job.

<?php

use com\zoho\crm\api\bulkread\BulkReadOperations;

require_once "vendor/autoload.php";

class GetBulkReadJobDetails
{
public static function initialize()        
    {
        // Add initialisation code
        // Refer to this article for more help 
    }

    public static function getBulkReadJobDetails(string $jobId)
    {
        $bulkReadOperations = new BulkReadOperations();
        $response = $bulkReadOperations->getBulkReadJobDetails($jobId);
        //Add your code to handle the response received in $response
        // For more details, refer here.
    }
}
GetBulkReadJobDetails::initialize();
$jobId = "55459743108003";
GetBulkReadJobDetails::getBulkReadJobDetails($jobId);
?>

3. Download Bulk Read Result

To download your bulk read job result, use the job id mentioned in the bulk read job response. Additionally, provide the path of the destination folder where you would like to download the result. The result will be downloaded as a zip file containing CSV/ICS files.
Here is a sample code for downloading a bulk read job result.

<?php

namespace samples\bulkread;

use com\zoho\crm\api\bulkread\BulkReadOperations;

require_once "vendor/autoload.php";

class DownloadResult
{
public static function initialize()
    {
        // Add initialisation code
        // Refer to this article for more help  
    }
    public static function downloadResult(string $jobId, string $destinationFolder)
    {
        $bulkReadOperations = new BulkReadOperations();
        $response = $bulkReadOperations->downloadResult($jobId);
        $streamWrapper = $response->getObject()->getFile();
        fputs(fopen($destinationFolder . "/" . $streamWrapper->getName(), "w"), $streamWrapper->getStream());
        //Add your code to handle the response received in $response
        // For more details, refer here.
    }
}
DownloadResult::initialize();
$jobId = "55459743108003";
$destinationFolder = "/Downloads";
DownloadResult::downloadResult($jobId, $destinationFolder);
?>

We hope you found this post useful and engaging! 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We value your thoughts and eagerly look forward to hearing from you.

Stay tuned for more enriching posts coming your way soon!

Cheers!

    Access your files securely from anywhere








                            Zoho Developer Community



                                Bootcamp
                                Community
                                Pricing
                                Webinars
                                What's New
                                Integrations
                                Contact Support
                                Contact Sales
                                  Zoho Survey Resources
                                    Features
                                    Survey Templates
                                    Integrations
                                    Community
                                    iOS Mobile Surveys
                                    Pricing
                                    Android Mobile Surveys
                                    Insurvey Blogs
                                    Insight Blogs
                                    Tips & Tricks





                                                Latest Feature Updates

                                                  • Desk Community Learning Series


                                                  • Digest


                                                  • Functions


                                                  • Meetups


                                                  • Kbase


                                                  • Resources


                                                  • Glossary


                                                  • Desk Marketplace


                                                  • MVP Corner


                                                  • Word of the Day


                                                  • Ask the Experts




                                                      Zoho SalesIQ Resources

                                                            • Sticky Posts

                                                            • Kaizen #217 - Actions APIs : Tasks

                                                              Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.

                                                            • Kaizen #216 - Actions APIs : Email Notifications

                                                              Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are

                                                            • Kaizen #152 - Client Script Support for the new Canvas Record Forms

                                                              Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achieved

                                                            • Kaizen #142: How to Navigate to Another Page in Zoho CRM using Client Script

                                                              Hello everyone! Welcome back to another exciting Kaizen post. In this post, let us see how you can you navigate to different Pages using Client Script. In this Kaizen post, Need to Navigate to different Pages Client Script ZDKs related to navigation A.

                                                            • Kaizen #210 - Answering your Questions | Event Management System using ZDK CLI

                                                              Hello Everyone, Welcome back to yet another post in the Kaizen Series! As you already may know, for the Kaizen #200 milestone, we asked for your feedback and many of you suggested topics for us to discuss. We have been writing on these topics over the


                                                            Manage your brands on social media
                                                            Access Zoho Social

                                                              Zoho Pagesense Resources


                                                                  Zoho TeamInbox Resources

                                                                    Zoho DataPrep Resources


                                                                      Zoho CRM Plus Resources

                                                                        Zoho Books Resources


                                                                          Zoho Subscriptions Resources

                                                                            Zoho Projects Resources


                                                                              Zoho Sprints Resources


                                                                                Qntrl Resources


                                                                                  Zoho Creator Resources



                                                                                      Zoho CRM Resources

                                                                                      • CRM Community Learning Series

                                                                                        CRM Community Learning Series


                                                                                      • Kaizen

                                                                                        Kaizen

                                                                                      • Functions

                                                                                        Functions

                                                                                      • Meetups

                                                                                        Meetups

                                                                                      • Kbase

                                                                                        Kbase

                                                                                      • Resources

                                                                                        Resources

                                                                                      • Digest

                                                                                        Digest

                                                                                      • CRM Marketplace

                                                                                        CRM Marketplace

                                                                                      • MVP Corner

                                                                                        MVP Corner









                                                                                          Design. Discuss. Deliver.

                                                                                          Create visually engaging stories with Zoho Show.

                                                                                          Get Started Now


                                                                                            Zoho Show Resources

                                                                                              Zoho Writer

                                                                                              Get Started. Write Away!

                                                                                              Writer is a powerful online word processor, designed for collaborative work.

                                                                                                Zoho CRM コンテンツ



                                                                                                  Nederlandse Hulpbronnen


                                                                                                      ご検討中の方