Kaizen #12 - Bulk Read API
Hello everyone!
We hope you all are having a wonderful start in the New Year!
Welcome back to another post in the Kaizen series!
What is the Bulk Read API?
The Bulk Read API allows you to export data from a module in Zoho CRM in bulk. The primary difference between the GET records API and the Bulk Read API is the number of records you can retrieve.
While you can get only 200 records per Get records API call, you can fetch 200,000 records per bulk read API call.
When should you use this API?
- When you want to export more than 200 records through an API call.
- When you want to perform background processes like migration, data backup, and initial data sync between Zoho CRM and external services.
What is the difference between GET Records API and Bulk Read API?
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
What kind of export does the Bulk Read API support?
You can export
- Records available in a custom view. You can also apply the criteria on the records available in that custom view to filter the records further. Note that you cannot export records in the Co-owner, Shared by Me, and Shared to Me views.
- Lookup fields of records in a module using the dot (.) operator.
- Records in the events module as an ICS file.
How does the Bulk Read API work?
Scheduling a bulk read job encompasses the following steps.
- Creating a bulk read job.
- Getting the status of the scheduled job.
- Downloading the result.
We will now see each of these steps in detail.
Step-1: Creating a bulk read job
This involves making a POST API call to schedule a job to fetch records that match certain criteria, along with the required fields.
When this call is successful, the response returns an id. You can use this id to know the status of the job by polling, or check the job completion in the callback URL.
Request URL: {{api-domain}}/crm/bulk/v2/read
Request Method: POST
Consider an example where you want to export records from the Contacts module.
The below query fetches the records that matches one of the three criteria—
- The lead source is Advertisement (or)
- The owner's last name is Boyle (or)
- The phone number of the account associated with the contact contains the numbers 99807.
The fields to be fetched are mentioned in the fields array. To fetch all fields in the module, do not input the fields key.
In this query, Account_Name is the default lookup field in the Contacts module. Owner.last_name returns the last name of the owner of the contact, Account_Name returns the ID and Account_Name.Account_Name returns the name of the account associated with the contact, and Account_Name.Phone returns the phone number of the account associated with the contact.
{ "callback": { "method": "post" }, "query": { "module": "Contacts", "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1 } } |
The below table gives the description of the keys in the sample input.
Key and Data Type | Mandatory | Description |
callback JSON Object | Yes | Contains a valid URL, which should allow the HTTP POST method. The Bulk Read Job's details are posted to this URL on successful completion or failure of the job. |
query JSON Object | Yes | Contains the module, fields that you want to export, criteria based on which you want to export the records, and the page. Refer to the query properties table below for more details. |
file_type String | Yes, when you want to export the events as an ICS file. | Specify the value of this key as "ics" to export all records in the Events module as an ICS file. |
query properties
Key and Data Type | Mandatory | Description |
module String | Yes | The API name of the module from which you want to export the records. Specify the module name as "Events" if you want to export the records in the Events module as an ics file. |
cvid String | Yes, when you want to export records in a custom view | The unique ID of the custom view whose records you want to export. You can obtain the cvid from the Custom View Metadata API. |
fields JSON Array | No | The API Name of the fields you want to export from the module. Example: First_Name, Last_Name, Email, Owner.last_name. Do not input this key when you want to export the records in the Events module as an ICS file. The system throws FIELDS_NOT_SUPPORTED error, otherwise. |
page Integer | No | Default value for page is 1. The page value '1' means that the first 200,000 records matching your query will be exported. If you want to fetch the records from the range of 200,001 to 400,000, then you should mention the page as '2'. The maximum value for this key is 500. |
criteria JSON Object | No | To filter the records you want to export. This JSON object contains the API name of the field, comparator, and a group. Refer to the criteria properties table below for more details. |
criteria properties
Key and Data Type | Mandatory | Description |
api_name String | Yes, if group and group_operator are not specified. | API name of the field you want to compare. Example: First_Name, Last_Name, Owner.last_name etc,. Example: "criteria": { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" } |
value String or JSON array | Yes, if group and group_operator are not specified. | Positive integer values only. |
group_operator String | Yes, if api_name, comparator & value are not specified. | Logical operators. Supported values are and, or. |
group JSON array | Yes, if api_name, comparator & value are not specified. | Array of criteria objects. |
comparator String | Yes, if group and group_operator are not specified. | Specifies the comparator. Example: equal, greater_than. Refer to the comparators table for more details. |
comparators
Data Type | Comparator | Value and Limits |
Number (Integer), Decimal/BigInteger/ Currency/Percent) | equal, not_equal, in, not_in, less_than, less_equal, greater_than, greater_equal | Any number values or ${EMPTY} for empty value. Not more than 19 digits for big integer, allows decimal values for decimal and currency fields. In multi-currency enabled accounts, only home currency value is supported. |
Text (Email, Phone, URL, Picklist, Multi-select, etc) | equal, not_equal, in, not_in, contains, not_contains, starts_with, ends_with | Any text or ${EMPTY} for empty value. Not more than 255 characters. |
Date | equal, not_equal, in, not_in, between, not_between | Any date value in ISO 8601 format or ${EMPTY} for empty value. |
DateTime | equal, not_equal, in, not_in, between, not_between | Any date time value in ISO 8601 format or ${EMPTY} for empty value. Example: 2019-04-01T14:24:04+05:30. Milliseconds are not supported. |
Boolean | equal | true or false. |
Lookup | equal, not_equal, in, not_in | Biginteger value of the lookup, ${EMPTY} for empty value, or use the .(dot) operator to establish a relation between two modules. Example: In the Contacts module, Owner fetches the ID of the Owner, whereas Owner.last_name fetches the last name of the owner. Account_Name fetches the ID of the Account associated with the base module, whereas Account_Name.Phone fetches the phone number of the account associated with the base module. |
Text Area (Multi-line) | Not supported | Not supported |
The response to the sample query is as follows. You can use the id in the details key to check the job status periodically as explained in step-2.
{ "data": [ { "status": "success", "code": "ADDED_SUCCESSFULLY", "message": "Added successfully.", "details": { "id": "3652397000000646004", "operation": "read", "state": "ADDED", "created_by": { "id": "3652397000000186017", "name": "Patricia Boyle" }, "created_time": "2019-04-01T14:24:04+05:30" } } ], "info": {} } |
For more details and examples, refer to the Create Bulk Read Job page of our API guide.
Step-2: Getting the status of the scheduled bulk read job
The bulk read API supports polling and callback.
Polling
You can poll (check the status of the scheduled bulk read job) with the job ID you received in the previous step.
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}
Request Method: GET
The response contains the status of the scheduled job as either ADDED, IN PROGRESS, or COMPLETED.
When the job is complete, the response contains the result JSON object with the keys page, count, more_records, and download_url.
You can also find the download_url in the callback response from which you can download the zip file containing the CSV or ICS file.
If the more_records key is true, there are more records you need to export. Simply change the value of the key page in the POST request, and schedule another bulk read job to fetch the next set of records.
The download_url contains the URL to download the CSV file. Follow the instructions in step-3 to download the CSV file.
Here's a sample response to the above query when the job is completed.
{ "data": [ { "id": "3652397000000646004", "operation": "read", "state": "COMPLETED", "result": { "page": 1, "count": 3, "download_url": "/crm/bulk/v2/read/3652397000000646004/result", "per_page": 200000, "more_records": false }, "query": { "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "module": "Contacts", "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1, "cvid": "554023000000093005" }, "created_by": { "id": "554023000000235011", "name": "Patricia Boyle" }, "created_time": "2019-05-09T14:01:24+05:30" } ] } |
The other possible values for the key state are
- ADDED - Indicates that the job is scheduled.
- IN PROGRESS - Indicates that the job is under processing.
Note that only a completed job will have the download URL in its response.
Callback
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
- The state indicates the successful completion ("state":"COMPLETED") or failure ("state":"FAILED") of the job.
- The callback response will also contain the download URL if the job was completed successfully.
Below is a sample callback response for a completed job.
{ "job_id": "554023000000568002", "operation": "read", "state": "COMPLETED", "query": { "module": "Contacts", "criteria": { "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ], "group_operator": "or" }, "page": 1, "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "cvid": "554023000000093005" }, "result": { "page": 1, "count": 1588, "download_url": "/crm/bulk/v2/read/554023000000568002/result", "per_page": 200000, "more_records": false } } |
For more details, refer to the Get Bulk Read Job Details page of our API guide.
This is the final step to complete exporting records from a module in Zoho CRM.
Make a GET request to the download_url you received in the response of step-2.
Step-3: Downloading the CSV file
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}/result
Request Method: GET
This request downloads the zip file. Extract it to get the CSV or ICS file.
Limitations
- Only 10 requests for download are allowed for a one-minute interval. Crossing the limit throws the error with the HTTP code 429. All subsequent downloads will be unsuccessful.
- After completing the bulk read job, you can access the downloadable file only for a period of one day. After that, you cannot access the file via the endpoints.
- You can specify a maximum of 200 select fields via an endpoint. If you specify more than that, the system exports all fields available in that module.
- The maximum value of the page key in the export request body is 500.
- You can specify a maximum of 25 criteria in a query. This also includes the number of criteria in a custom view or a standard view. For example, if you specify the ID of the standard view "My converted Leads", which uses two criteria, the remaining criteria that you can use in the query will be 23.
- in and not_in comparators can accept up to 20 values. For example: 'Lead Status' - 'in' - 'Cold,Warm,Hot,Junk,Contacted,Not Contacted,....(20 values)'.
- You cannot retrieve multi-line, multi-select lookup fields.
- Custom views in the Activities module, Co-Owner view in the Contacts module, the standard views Shared By Me and Shared To Me are not supported in this API.
For more details, refer to the Limitations page of our API guide.
We hope you found this post useful. Let us know your thoughts in the comment section, or reach out to us at support@zohocrm.com if you have any questions.
Cheers!
Access your files securely from anywhere
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Shylaja S
Parikshit
Praveen Sankar
Sticky Posts
Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! CommandsKaizen #165 : How to call Zoho CRM APIs using Client Script
Welcome back to another exciting Client Script post! In this post, we will discuss one of the most frequently asked questions: How do you call Zoho CRM APIs from Client Scripts? In this kaizen post, 1. Ways to make calls to Zoho CRM APIs using ClientKaizen#162 : Add Tags and Open Pre-filled Email Drafts via Client Scripts
Hello everyone! Welcome to another informative Kaizen post. In this post, let us see how to accomplish the following using Client Script. 1. How to auto-tag a record based on field update? 2. How to Open a pre-filled email draft This post will provideExtension Pointers - JS SDK series #4: Managing data from a widget using ZOHO.CRM.HTTP
Handling and managing the data between two applications is a major factor for an efficient business. There are a variety of ways to handle data between Zoho CRM and other third-party applications. You can use deluge CRM tasks, create connectors, use APIKaizen #152 - Client Script Support for the new Canvas Record Forms
Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achieved
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho DataPrep Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Custom Status for Purchase Orders
Currently Zoho books has functionality to create custom statuses for Sales Orders. Can this be extended to include custom status for purchase orders as well? It was a great decision to add this functionality to sales orders. Our use case is for trackingMy workflows not working
Hi I have created my first workflow I want it so when a customer responds to a ticket that the status is changed to "reopened" We want this so its clear there is a ticket to deal with again. We only have 2 agents responding to tickets and do not haveWaiting Room
Is there any plans to implement a Virtual Waiting Room for Attendees on Meetings and or Webinars?Error AS101 when adding new email alias
Hi, I am trying to add apple@(mydomain).com The error AS101 is shown while I try to add the alias.Global Sets for Multi-Select pick lists
When is this feature coming to Zoho CRM? It would be very useful now we have got used to having it for the normal pick lists.What's New - January 2025 | Zoho Backstage
Hi there! As we step into a brand-new year, we’re thrilled to bring you some exciting updates to make event management more enjoyable and efficient. Here’s what we rolled out in January to kick off 2025: Backstage 2.0 now enforced for all users We'veHow do you do ticket add ons in Backstage?
Hi Everyone, If you wanted to have general admin tickets and allow for add ons, like camping, or car or Carbon offset. What would you do? Peace RobinRe-create auth token
I need to re-create a self-client auth token, with the same scopes (and additional ones) as the current token. Is there a way to view the scopes assigned to the current token to ensure that no scopes are missing when creating the new one?Writing on sketch cards is bugged when zoomed in
When zoomed in, it writes a noticeable distance above or to the side of where you're actually trying to write. The further you're zoomed in, the more noticeable it is. Zooming is also entirely absent on the desktop version.Doesn't let me test my newsletter.
Hello, I click "Preview and Test". I select the email address to send the test (usually my own email). I send the test email, appears the message "Your test email has been sent", but nothing happens. I mean, there is nothing in my inbox. I also triedCustom Related List anchor tag with tel protocol
The following code worked great for about a year to create clickable tel and mailto links in a related list in the Deals module. Just this morning, it started displaying the HTML as text. For example: <a href='tel:8882490100'>8882490100</a> Zoho supportreset of user password
User forgot her password, so she requests a password rest, but never gets the email. How do I reset her password, since that function is not an option, even for an administrator?In what case it's better to use Developer Console? Only for marketplace?
As I can see from the documentation, the limitations are challenging. No Custom API No Page Variables No Custom Connections.commision tracker
I am after a zoho developer to finish my commission tracker as I hired someone on oDesk that started but now will respond to my emails.I really need to get It finished as it is holding up my plans Thanks Budget : $500 | Duration : a.s.a.p To leave a private message, please click the link for private response in post Action section.Generate a Zoho Sign link
From time to time I get a response "I never received your you e-document for electronic signature" is there a way to generate a Zoho Sign link to share.Error 403: Forbidden When Updating Email Signature via API
Hi Zoho Desk team, First, congratulations again on the excellent Zoho API. But, I’m encountering an issue while attempting to update an email signature via the API. Whenever I make a request to update the signature, the response returns an HTTP 403 ForbiddenQueues Management ?
I need to assign the new requirements equally between my support agents. How I can configure this feature in Zoho? ThanksClient Can't View Ticket
Hi, I created a ticket for my Client since he had an issue. I added him as a Contact as well. My client got a notification in his email that the ticket has been created. When he went to view the ticket on his desktop, the screen was blank. He then viewedAutomating Custom Web Link Messages to Customers via Instant Messaging in Zoho Desk
Hello, I am looking for assistance with Zoho Desk's Instant Messaging feature. Is there a way to automate the sending of a custom web link to customers every day at 6:30 PM? Any guidance or suggestions on how to achieve this would be greatly appreciated.How do you apply field validation to a date field?
We are using Sign for remote e-signature of liability waivers. The signer must enter a date of birth, and if they are a minor their legal guardian must sign for them. However we have had numerous problems with signers returning an invalid date of birthImplement Regex in Layout and Validation rules
Hello all, We are excited to announce that users can now implement Regular Expressions (Regex) in our layout and validation rules. This new functionality allows for more flexible rules to be created when designing and validating forms. What is Regex?Zoho Desk Validation Rule Using Custom Function
Hi all, I tried to find the way to validate fields using custom function just like in Zoho CRM but to no avail. Is there a way to do this?Editing landing page after signup for a webinar
Hi, how can I edit the landing page after signing up for a webinar. I personalized the email "registration confirmation", which is working fine. Nevertheless after submitting the form, the participant gets redirected to a page, which looks like the standardKiosk GetRecords can't compare formulas
(submitted as a help request by accident - should have been a topic) Scenario: I have an "email domain" formula whose value is "@xyz.com" if the e-mail address is brent@xyz.com . I have the same field in Leads and Contacts. So, theoretically, I shouldKiosk can't merge picklist or multiselect
There is no ability to load a multiselect or picklikst field into a kiosk with the values that have been previously selected. So, I essentially have 3 unacceptable options: 1.)Load the value into a text string and include instructions like this: "PicklistFillable template with dynamic tables?
Is there a way to build a fillable template so that users can add rows to a table? To describe what I'm trying to accomplish the table has 3 sections; a header row, some number of rows with custom information, and a summary row with totals. I can't figureTip of the Week - Timeline User Filter for Dashboard
Does the Date User Filter in your Dashboard leave out a few reports from filtering? This happens when the reports are not created over the same table and the (time) column over which the user filter is created. To solve this, Zoho Reports has a Timeline User Filter. This filter applies the criteria on all reports, matching it with the best possible date column associated with a report. i.e., if your reports are based on a date column, then it will filter it for that data column. To know more onZoho Forms Webhooks - Only sends on submission not on edit.
Hi There, We have a slightly unique use case for Zoho Forms. We used Zoho Forms for the collection of Guest Lists. When a customer submits their guest list they get an email confirmation. Within this is a link for them to update their guest list beforeAdd Knowledge Base KB Articles to multiple categories
Greetings, Love you help center system. One item that would be incredibly helpful to many of us would be able to add a single Knowledge Base KB article to multiple categories in our system. It seems it could be quite easy to use a checkbox form, insteadWhat does "Tickets for Review" do?
What is the purpose of the va nilla view " Tickets for Review?"Report of Leads/Accounts/Deals with no open activities
How can we replicate the option in filter to only show Leads/Accounts/Deals with no open activities ? The option is present in the filters (please find screenshot) but I am unable to find them in the report. ThanksCannot set Subform Multiline field to read-only using Client Script
I am using client script to set different subform fields as read-only It works for Single Line Fields, but when I try to set a Multiline field as Read-only it doesn't work var subform = ZDK.Page.getSubform("Subform_1"); var item_Code_old = subform.getField('Item_Code_Old').setReadOnly(true);Changing Color Theme of Guided Conversations
Hello, We have recently added Guided Conversations to one of our websites, but I am wondering if there is a way to customize the color scheme so it matches the appearance of the website? Thank you in advance!Lost the ability to sort by ticket owner
Hi all, in the last week or so, we have lost the ability to sort tickets by Ticket Owner. Unlike the other columns which we can hover over and click on to sort, Ticket Owner is no longer clickable. Is it just us, or are other customers seeing this too?Can't login to Zoho mail
I'm logged into Zoho but when I try to go in zoho mail I get: Invalid request! The input passed is invalid or the URL is invoked without valid parameters. Please check your input and try again. I just set up my mx records and stuff with namecheap a fewSocial Media Simplified with Zoho Social: Bridge sales and social media marketing
Social media marketing has become the go-to strategy for most businesses who are trying to boost awareness of their product/service, create a brand image in the minds of their audience, increase website traffic and engagement, and generate leads. PostingThis domain is not allowed to add. Please contact support-as@zohocorp.com for further details
I am trying to setup the free version of Zoho Mail. When I tried to add my domain, theselfreunion.com I got the error message that is the subject of this Topic. I've read your other community forum topics, and this is NOT a free domain. So what is theWork Orders / Bundle Requests
Zoho Inventory needs a work order / bundle request system. This record would be analogous to a purchase order in the purchasing workflow or a sales order in the sales cycle. It would be non-journaling, but it would reserve the appropriate inventory ofCreate Quote does not show the "Product Description" entered as part of the Product setup.
The product description created as part of the product setup page, does not show in the Create Quote module; The module allows for an additional description to be added but without access to the original stored description. By selecting the product from the "Product Name Lookup" pop-up, the "Product Description" part should be populated from the product record where the user is allowed to further modify it.Allow Itemization for Recurring Expenses
For whatever reason, one cannot itemize a Recurring Expense. This capability should be added. The use cases to support this is largely the same as what they were to allow for itemization in Expenses. Anything that would need to be itemized for a regularNext Page