Kaizen #92 - Bulk Read in PHP SDK

Kaizen #92 - Bulk Read in PHP SDK

Heya! Welcome back to the PHP SDK fold in our Kaizen series.

In this post, we will explore how to use bulk read APIs and how to structure the criteria format for different field data types in PHP SDK based on v4 APIs. This will ease your process of handling large volumes of data in an efficient and streamlined manner. 

What does Bulk Read API do?

The Bulk Read API helps you to fetch large volumes of data that meet specific criteria in a single request. Unlike the GET Records API , which has a limitation of 200 records per call, the Bulk Read API allows you to extract bulk data, 200k records, for data synchronisation purposes. This prevents the need for you to fire multiple API calls which can easily drain your daily credits.

This Bulk Read is asynchronous, thus the result for your request will not be available immediately. So, you will have to either poll the bulk read job status API to know the status or specify a callback URL for Zoho CRM to send notifications regarding the process. 

The CRM system processes your request and returns the results in either a CSV or ICS (for Meetings/Events module) zipped file. To download your result, use the job ID given in the response of bulk read job API .

This entire process can be wrapped into three stages as given below, 
  • Creating a Bulk Read Job 
  • Checking the status of a Bulk Read Job
  • Downloading the CSV/ICS data zip

Create a Bulk Read Job

1. Adding Callback URL

To get notified of the progress and results of the bulk read job, add a callback URL to the request body using the following format. 
$requestWrapper = new RequestWrapper();

$callbackVar = new CallBack();

$callbackVar->setUrl(" your_callback_url ");

$callbackVar->setMethod(new Choice("post"));

$requestWrapper->setCallback($callbackVar);
The method for callback URL is typically set to POST as it helps the client to send the required data along with the url to handle the callback appropriately.

2. Structuring a Query

A query comprises multiple parameters such as your desired module, fields, custom view ID, page value, file-type and criteria.

A criterion is typically a filter/condition used in a query to narrow down the data to your desired set of records. Criteria can include one or more conditions and you can combine them using group operators like 'and' and 'or' to create more complex filtering logic. 

Each condition in the criteria comprises three components.
  • Field - It refers to the field API name of the record, which you want to use for filtering purpose.
  • Comparator - It determines how the value of the specified field has to be evaluated.
  • Value - It represents the value of the field against which the record has to be evaluated.
This helps you to retrieve only the relevant records, thereby optimising the performance and reducing the amount of data transferred.

To enhance the understanding of how to structure a criterion, it is necessary to address these group operators and comparators before delving into it.

Group Operators 

Group Operators are used to perform logical operations on multiple conditions within a query. They help you in grouping multiple conditions together to create complex search criteria.The possible group operators are 'and' and 'or'.
  • and - It specifies that all conditions within the group must be satisfied for a record to be included in the query result.
  • or - It specifies that any of the conditions within the group can be satisfied for a record to be included in the query result.
These operators provide flexibility in constructing queries and retrieve the desired set of records from your CRM system with ease.

Comparators

Comparators are used in a query condition to specify the type of comparison that should be performed between a field and a value. They define the relationship between the field and the value and determine how the records are filtered. Refer to this help document to find the possible comparators with their field datatypes. 

An example of each datatype is given in PHP for your better understanding. 
Data Type
Example
  Number

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field ->setAPIName("Amount");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("greater_than"));
$conditionVar->setValue(10000);
  Multi-select 

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field ->setAPIName("owner");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("in"));
$owner = ["5545974393001", "5545974393011"]
$conditionVar->setValue($owner);
  DateTime                                   

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field->setAPIName("Created_Time");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("between"));
$createdTime = [date_create("2023-05-01T17:58:47+05:30")->setTimezone(new \DateTimeZone(date_default_timezone_get())), date_create("2023-06-01T17:58:47+05:30")->setTimezone(new \DateTimeZone(date_default_timezone_get()))];
$conditionVar->setValue($createdTime);
  Boolean

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field ->setAPIName("Email_Opt_Out");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("equal"));
$conditionVar->setValue(false);
  Lookup

$conditionVar = new Criteria();
$field = new MinifiedFields();
$field ->setAPIName("owner.last_name");
$conditionVar>setField($field);
$conditionVar->setComparator(new Choice("equal"));
$conditionVar->setValue("Boyle");

Structuring Simple Criteria

Below is a query format with a simple criterion of two conditions to filter your records from the given module. 

$criteriaVar = new Criteria();

$criteriaVar->setGroupOperator(new Choice("operatorValue"));

$conditionVar1 = new Criteria();

$fieldVar = new MinifiedFields();

$fieldVar->setAPIName("Field_API_Name");

$conditionVar1->setField($fieldVar);

$conditionVar1->setComparator(new Choice("comparatorValue"));

$conditionVar1->setValue("Field_Value");

$conditionVar2 = new Criteria();

$field = new MinifiedFields();

$field->setAPIName("Field_API_Name");

$conditionVar2->setField($fieldVar);

$conditionVar2->setComparator(new Choice("comparatorValue"));

$conditionVar2->setValue("Field_Value");

$criteriaVar->setGroup([$conditionVar1,$conditionVar2]);

$query->setCriteria($criteriaVar);

$requestWrapper->setQuery($query);

In the above format, the group operator is used for two different conditions.

Following is the json format of this simple criteria.
{
    "query": {
        "criteria": {
            "group_operator": "operatorValue",
            "group": [
                {
                    "comparator": "comparatorValue",
                    "field": {
                        "api_name": "Field_API_Name"
                    },
                    "value": "Field_Value"
                },
                {
                    "comparator": "comparatorValue",
                    "field": {
                        "api_name": "Field_API_Name"
                    },
                    "value": "Field_Value"
                }
            ]
        }
    }
}

Structuring Complex Criteria

To structure a criterion with four different conditions follow the below format.

//top-level group criteria with 2 different groups comprising of 2 different conditions each

$criteria = new Criteria();

$criteria->setGroupOperator(new Choice("operatorValue"));

//group1 with 2 different conditions

$groupVar1 = new Criteria();

$groupVar1->setGroupOperator(new Choice("operatorValue"));

$conditionVar11 = new Criteria();

$field11 = new MinifiedFields();

$field11->setAPIName("Field_API_Name");

$conditionVar11->setField($field11);

$conditionVar11->setComparator(new Choice("comparatorValue"));

$conditionVar11->setValue("Value");

$conditionVar12 = new Criteria();

$field12 = new MinifiedFields();

$field12->setAPIName("Field_API_Name");

$conditionVar12->setField($field21);

$conditionVar12->setComparator(new Choice("comparatorValue"));

$conditionVar12->setValue("Value");

$groupVar1->setGroup([$conditionVar11, $conditionVar12]);

//group2 with 2 different conditions

$groupVar2 = new Criteria();

$groupVar2->setGroupOperator(new Choice("operatorValue"));

$conditionVar21 = new Criteria();

$field21 = new MinifiedFields();

$field21->setAPIName("Field_API_Name");

$conditionVar21->setField($field21);

$conditionVar21->setComparator(new Choice("comparatorValue"));

$conditionVar21->setValue("Value");

$conditionVar22 = new Criteria();

$field22 = new MinifiedFields();

$field22->setAPIName("Field_API_Name");

$conditionVar22->setField($field22);

$conditionVar22->setComparator(new Choice("comparatorValue"));

$conditionVar22->setValue("Value");

$groupVar2->setGroup([$conditionVar21, $conditionVar22]);

//pushing the groups to the top-level group criteria

$criteria->setGroup([$groupVar1, $groupVar2]);

In the above format, two group variables ($groupVar1 and $groupVar2) are created, each representing a nested group of conditions. Both the groups are further assigned to the top-level group variable ($criteria). 

The json arrangement of the above format is given for your clarity.
{
    "query": {
        "criteria": {
            "group_operator": "operatorValue",
            "group": [
                {
                    "group_operator": "operatorValue",
                    "group": [
                        {
                            "comparator": "comparatorValue",
                            "field": {
                                "api_name": "Field_API_Name"
                            },
                            "value": "Field_Value"
                        },
                        {
                            "comparator": "comparatorValue",
                            "field": {
                                "api_name": "Field_API_Name"
                            },
                            "value": "Field_Value"
                        }
                    ]
                },
                {
                    "group_operator": "operatorValue",
                    "group": [
                        {
                            "comparator": "comparatorValue",
                            "field": {
                                "api_name": "Field_API_Name"
                            },
                            "value": "Field_Value"
                        },
                        {
                            "comparator": "comparatorValue",
                            "field": {
                                "api_name": "Field_API_Name"
                            },
                            "value": "Field_Value"
                        }
                    ]
                }
            ]
        }
     }
}

The default file type is CSV. To bulk read the Events module always prefer the ICS file type. Refer to this resource to fetch the custom view IDs, module and field API names. 

Below is a sample code for creating a bulk read job with compound criteria. The criteria follow this arrangement ((1 and 2) or (3 or 4)) of conditions.

<?php

use com\zoho\crm\api\bulkread\BulkReadOperations;
use com\zoho\crm\api\bulkread\CallBack;
use com\zoho\crm\api\util\Choice;
use com\zoho\crm\api\bulkread\Query;
use com\zoho\crm\api\bulkread\Criteria;
use com\zoho\crm\api\bulkread\RequestWrapper;
use com\zoho\crm\api\modules\MinifiedModule;
use com\zoho\crm\api\fields\MinifiedFields;
require_once "vendor/autoload.php";

class CreateBulkReadJob
{
public static function initialize()
    {
        // Add initialisation code
        // Refer to this article for more help
    }
    public static function createBulkReadJob(string $moduleAPIName)
    {
        $bulkReadOperations = new BulkReadOperations();
        $requestWrapper = new RequestWrapper();
        $callback = new CallBack();
        $callback->setUrl("https://www.callback.com/example");
        $callback->setMethod(new Choice("post"));
        $requestWrapper->setCallback($callback);

        //query
        $query = new Query();
        $module = new MinifiedModule();
        $module->setAPIName($moduleAPIName);
        $query->setModule($module);
        $query->setCvid("34770610087501");
        $query->setFields(["Last_Name"]);
        $query->setPage(1);

        //top-level group criteria with 2 different groups comprising of 2 different conditions each
        $criteria = new Criteria();
        $criteria->setGroupOperator(new Choice("or"));
        $criteriaList = array();

        //group1 with 2 different conditions
        $groupVar1 = new Criteria();
        $groupVar1->setGroupOperator(new Choice("and"));

        $conditionVar11 = new Criteria();
        $field11= new MinifiedFields();
        $field11->setAPIName("Last_Name");
        $conditionVar11->setField($field11);
        $conditionVar11->setComparator(new Choice("equal"));
        $conditionVar11->setValue("Boyle");
        
        $conditionVar12 = new Criteria();
        $field12 = new MinifiedFields();
        $field->setAPIName("Owner");
        $conditionVar12->setField($field12);
        $conditionVar12->setComparator(new Choice("in"));
        $owner = array("5545974000000393001","5545974000000393011");
        $conditionVar12->setValue($owner);
        $groupVar1->setGroup([$conditionVar11, $conditionVar12]);

         //group2 with 2 different conditions
        $groupVar2 = new Criteria();
        $groupVar2->setGroupOperator(new Choice("or"));

        $conditionVar21 = new Criteria();
        $field21 = new MinifiedFields();
        $field21->setAPIName("Company");
        $conditionVar21->setField($field21);
        $conditionVar21->setComparator(new Choice("equal"));
        $conditionVar21->setValue("Morlong Associates");
        
        $conditionVar22 = new Criteria();
        $field22= new MinifiedFields();
        $field22->setAPIName("Created_Time");
        $conditionVar22->setField($field22);
        $conditionVar22->setComparator(new Choice("between"));
        $createdTime = array(date_create("2023-04-15T17:58:47+05:30")->setTimezone(new \DateTimeZone(date_default_timezone_get())), date_create("2023-06-01T17:58:47+05:30")->setTimezone(new \DateTimeZone(date_default_timezone_get())));
        $conditionVar22->setValue($createdTime);
        $groupVar2->setGroup([$conditionVar21, $conditionVar22]);

         //pushing the groups to the top-level group criteria
        $criteria->setGroup($criteriaList);
        $query->setCriteria($criteria);
        $requestWrapper->setQuery($query);
        // $requestWrapper->setFileType(new Choice("ics")); for Events module
        $response = $bulkReadOperations->createBulkReadJob($requestWrapper);
        //Add your code to handle the response received in $response
        // For more details, refer here.
    }
}
CreateBulkReadJob::initialize();
$moduleAPIName = "Leads";
CreateBulkReadJob::createBulkReadJob($moduleAPIName);
?>

2. Status of a Bulk Read Job

To keep track of the status of your bulk read job, use the id provided in the details section of your bulk read job response. 

Here is a sample code for checking the status of a bulk read job.

<?php

use com\zoho\crm\api\bulkread\BulkReadOperations;

require_once "vendor/autoload.php";

class GetBulkReadJobDetails
{
public static function initialize()        
    {
        // Add initialisation code
        // Refer to this article for more help 
    }

    public static function getBulkReadJobDetails(string $jobId)
    {
        $bulkReadOperations = new BulkReadOperations();
        $response = $bulkReadOperations->getBulkReadJobDetails($jobId);
        //Add your code to handle the response received in $response
        // For more details, refer here.
    }
}
GetBulkReadJobDetails::initialize();
$jobId = "55459743108003";
GetBulkReadJobDetails::getBulkReadJobDetails($jobId);
?>

3. Download Bulk Read Result

To download your bulk read job result, use the job id mentioned in the bulk read job response. Additionally, provide the path of the destination folder where you would like to download the result. The result will be downloaded as a zip file containing CSV/ICS files.
Here is a sample code for downloading a bulk read job result.

<?php

namespace samples\bulkread;

use com\zoho\crm\api\bulkread\BulkReadOperations;

require_once "vendor/autoload.php";

class DownloadResult
{
public static function initialize()
    {
        // Add initialisation code
        // Refer to this article for more help  
    }
    public static function downloadResult(string $jobId, string $destinationFolder)
    {
        $bulkReadOperations = new BulkReadOperations();
        $response = $bulkReadOperations->downloadResult($jobId);
        $streamWrapper = $response->getObject()->getFile();
        fputs(fopen($destinationFolder . "/" . $streamWrapper->getName(), "w"), $streamWrapper->getStream());
        //Add your code to handle the response received in $response
        // For more details, refer here.
    }
}
DownloadResult::initialize();
$jobId = "55459743108003";
$destinationFolder = "/Downloads";
DownloadResult::downloadResult($jobId, $destinationFolder);
?>

We hope you found this post useful and engaging! 

If you have any queries, feel free to drop them in the comments section below or reach out to us directly at support@zohocrm.com. We value your thoughts and eagerly look forward to hearing from you.

Stay tuned for more enriching posts coming your way soon!

Cheers!

    Access your files securely from anywhere






                      Zoho Campaigns Resources


                            Zoho Developer Community



                                Bootcamp
                                Community
                                Pricing
                                Webinars
                                What's New
                                Integrations
                                Contact Support
                                Contact Sales

                                  Zoho Survey Resources
                                    Features
                                    Survey Templates
                                    Integrations
                                    Community
                                    iOS Mobile Surveys
                                    Pricing
                                    Android Mobile Surveys
                                    Insurvey Blogs
                                    Insight Blogs
                                    Tips & Tricks





                                                Latest Feature Updates
                                                  What's New in Zoho Desk

                                                  Use cases

                                                  Make the most of Zoho Desk with the use cases.

                                                   
                                                    

                                                  eBooks

                                                  Download free eBooks and access a range of topics to get deeper insight on successfully using Zoho Desk.

                                                   
                                                    

                                                  Videos

                                                  Watch comprehensive videos on features and other important topics that will help you master Zoho Desk.

                                                   
                                                    

                                                  Webinar

                                                  Sign up for our webinars and learn the Zoho Desk basics, from customization to automation and more

                                                   
                                                    
                                                  What's New in Zoho Desk

                                                  • Desk Community Learning Series


                                                  • Meetups


                                                  • Ask the Experts


                                                  • Kbase


                                                  • Resources


                                                  • Glossary


                                                  • Desk Marketplace


                                                  • MVP Corner





                                                      Zoho SalesIQ Resources

                                                            • Sticky Posts

                                                            • Kaizen #197: Frequently Asked Questions on GraphQL APIs

                                                              🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

                                                            • Kaizen #198: Using Client Script for Custom Validation in Blueprint

                                                              Nearing 200th Kaizen Post – 1 More to the Big Two-Oh-Oh! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

                                                            • Celebrating 200 posts of Kaizen! Share your ideas for the milestone post

                                                              Hello Developers, We launched the Kaizen series in 2019 to share helpful content to support your Zoho CRM development journey. Staying true to its spirit—Kaizen Series: Continuous Improvement for Developer Experience—we've shared everything from FAQs

                                                            • Kaizen #193: Creating different fields in Zoho CRM through API

                                                              🎊 Nearing 200th Kaizen Post – We want to hear from you! Do you have any questions, suggestions, or topics you would like us to cover in future posts? Your insights and suggestions help us shape future content and make this series better for everyone.

                                                            • Client Script | Update - Introducing Commands in Client Script!

                                                              Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! Commands


                                                            Manage your brands on social media
                                                            Access Zoho Social

                                                              Zoho Pagesense Resources


                                                                  Zoho TeamInbox Resources

                                                                    Zoho DataPrep Resources


                                                                      Zoho CRM Plus Resources

                                                                        Zoho Books Resources


                                                                          Zoho Subscriptions Resources

                                                                            Zoho Projects Resources


                                                                              Zoho Sprints Resources


                                                                                Qntrl Resources


                                                                                  Zoho Creator Resources



                                                                                      Zoho CRM Resources

                                                                                      • CRM Community Learning Series

                                                                                        CRM Community Learning Series


                                                                                      • Kaizen

                                                                                        Kaizen

                                                                                      • Functions

                                                                                        Functions

                                                                                      • Meetups

                                                                                        Meetups

                                                                                      • Kbase

                                                                                        Kbase

                                                                                      • Resources

                                                                                        Resources

                                                                                      • Digest

                                                                                        Digest

                                                                                      • CRM Marketplace

                                                                                        CRM Marketplace

                                                                                      • MVP Corner

                                                                                        MVP Corner







                                                                                          Design. Discuss. Deliver.

                                                                                          Create visually engaging stories with Zoho Show.

                                                                                          Get Started Now


                                                                                            Zoho Show Resources


                                                                                              Zoho Writer Writer

                                                                                              Get Started. Write Away!

                                                                                              Writer is a powerful online word processor, designed for collaborative work.

                                                                                                Zoho CRM コンテンツ








                                                                                                  Nederlandse Hulpbronnen


                                                                                                      ご検討中の方