Kaizen #12 - Bulk Read API
Hello everyone!
We hope you all are having a wonderful start in the New Year!
Welcome back to another post in the Kaizen series!
What is the Bulk Read API?
The Bulk Read API allows you to export data from a module in Zoho CRM in bulk. The primary difference between the GET records API and the Bulk Read API is the number of records you can retrieve.
While you can get only 200 records per Get records API call, you can fetch 200,000 records per bulk read API call.
When should you use this API?
- When you want to export more than 200 records through an API call.
- When you want to perform background processes like migration, data backup, and initial data sync between Zoho CRM and external services.
What is the difference between GET Records API and Bulk Read API?
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
What kind of export does the Bulk Read API support?
You can export
- Records available in a custom view. You can also apply the criteria on the records available in that custom view to filter the records further. Note that you cannot export records in the Co-owner, Shared by Me, and Shared to Me views.
- Lookup fields of records in a module using the dot (.) operator.
- Records in the events module as an ICS file.
How does the Bulk Read API work?
Scheduling a bulk read job encompasses the following steps.
- Creating a bulk read job.
- Getting the status of the scheduled job.
- Downloading the result.
We will now see each of these steps in detail.
Step-1: Creating a bulk read job
This involves making a POST API call to schedule a job to fetch records that match certain criteria, along with the required fields.
When this call is successful, the response returns an id. You can use this id to know the status of the job by polling, or check the job completion in the callback URL.
Request URL: {{api-domain}}/crm/bulk/v2/read
Request Method: POST
Consider an example where you want to export records from the Contacts module.
The below query fetches the records that matches one of the three criteria—
- The lead source is Advertisement (or)
- The owner's last name is Boyle (or)
- The phone number of the account associated with the contact contains the numbers 99807.
The fields to be fetched are mentioned in the fields array. To fetch all fields in the module, do not input the fields key.
In this query, Account_Name is the default lookup field in the Contacts module. Owner.last_name returns the last name of the owner of the contact, Account_Name returns the ID and Account_Name.Account_Name returns the name of the account associated with the contact, and Account_Name.Phone returns the phone number of the account associated with the contact.
{ "callback": { "method": "post" }, "query": { "module": "Contacts", "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1 } } |
The below table gives the description of the keys in the sample input.
Key and Data Type | Mandatory | Description |
callback JSON Object | Yes | Contains a valid URL, which should allow the HTTP POST method. The Bulk Read Job's details are posted to this URL on successful completion or failure of the job. |
query JSON Object | Yes | Contains the module, fields that you want to export, criteria based on which you want to export the records, and the page. Refer to the query properties table below for more details. |
file_type String | Yes, when you want to export the events as an ICS file. | Specify the value of this key as "ics" to export all records in the Events module as an ICS file. |
query properties
Key and Data Type | Mandatory | Description |
module String | Yes | The API name of the module from which you want to export the records. Specify the module name as "Events" if you want to export the records in the Events module as an ics file. |
cvid String | Yes, when you want to export records in a custom view | The unique ID of the custom view whose records you want to export. You can obtain the cvid from the Custom View Metadata API. |
fields JSON Array | No | The API Name of the fields you want to export from the module. Example: First_Name, Last_Name, Email, Owner.last_name. Do not input this key when you want to export the records in the Events module as an ICS file. The system throws FIELDS_NOT_SUPPORTED error, otherwise. |
page Integer | No | Default value for page is 1. The page value '1' means that the first 200,000 records matching your query will be exported. If you want to fetch the records from the range of 200,001 to 400,000, then you should mention the page as '2'. The maximum value for this key is 500. |
criteria JSON Object | No | To filter the records you want to export. This JSON object contains the API name of the field, comparator, and a group. Refer to the criteria properties table below for more details. |
criteria properties
Key and Data Type | Mandatory | Description |
api_name String | Yes, if group and group_operator are not specified. | API name of the field you want to compare. Example: First_Name, Last_Name, Owner.last_name etc,. Example: "criteria": { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" } |
value String or JSON array | Yes, if group and group_operator are not specified. | Positive integer values only. |
group_operator String | Yes, if api_name, comparator & value are not specified. | Logical operators. Supported values are and, or. |
group JSON array | Yes, if api_name, comparator & value are not specified. | Array of criteria objects. |
comparator String | Yes, if group and group_operator are not specified. | Specifies the comparator. Example: equal, greater_than. Refer to the comparators table for more details. |
comparators
Data Type | Comparator | Value and Limits |
Number (Integer), Decimal/BigInteger/ Currency/Percent) | equal, not_equal, in, not_in, less_than, less_equal, greater_than, greater_equal | Any number values or ${EMPTY} for empty value. Not more than 19 digits for big integer, allows decimal values for decimal and currency fields. In multi-currency enabled accounts, only home currency value is supported. |
Text (Email, Phone, URL, Picklist, Multi-select, etc) | equal, not_equal, in, not_in, contains, not_contains, starts_with, ends_with | Any text or ${EMPTY} for empty value. Not more than 255 characters. |
Date | equal, not_equal, in, not_in, between, not_between | Any date value in ISO 8601 format or ${EMPTY} for empty value. |
DateTime | equal, not_equal, in, not_in, between, not_between | Any date time value in ISO 8601 format or ${EMPTY} for empty value. Example: 2019-04-01T14:24:04+05:30. Milliseconds are not supported. |
Boolean | equal | true or false. |
Lookup | equal, not_equal, in, not_in | Biginteger value of the lookup, ${EMPTY} for empty value, or use the .(dot) operator to establish a relation between two modules. Example: In the Contacts module, Owner fetches the ID of the Owner, whereas Owner.last_name fetches the last name of the owner. Account_Name fetches the ID of the Account associated with the base module, whereas Account_Name.Phone fetches the phone number of the account associated with the base module. |
Text Area (Multi-line) | Not supported | Not supported |
The response to the sample query is as follows. You can use the id in the details key to check the job status periodically as explained in step-2.
{ "data": [ { "status": "success", "code": "ADDED_SUCCESSFULLY", "message": "Added successfully.", "details": { "id": "3652397000000646004", "operation": "read", "state": "ADDED", "created_by": { "id": "3652397000000186017", "name": "Patricia Boyle" }, "created_time": "2019-04-01T14:24:04+05:30" } } ], "info": {} } |
For more details and examples, refer to the Create Bulk Read Job page of our API guide.
Step-2: Getting the status of the scheduled bulk read job
The bulk read API supports polling and callback.
Polling
You can poll (check the status of the scheduled bulk read job) with the job ID you received in the previous step.
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}
Request Method: GET
The response contains the status of the scheduled job as either ADDED, IN PROGRESS, or COMPLETED.
When the job is complete, the response contains the result JSON object with the keys page, count, more_records, and download_url.
You can also find the download_url in the callback response from which you can download the zip file containing the CSV or ICS file.
If the more_records key is true, there are more records you need to export. Simply change the value of the key page in the POST request, and schedule another bulk read job to fetch the next set of records.
The download_url contains the URL to download the CSV file. Follow the instructions in step-3 to download the CSV file.
Here's a sample response to the above query when the job is completed.
{ "data": [ { "id": "3652397000000646004", "operation": "read", "state": "COMPLETED", "result": { "page": 1, "count": 3, "download_url": "/crm/bulk/v2/read/3652397000000646004/result", "per_page": 200000, "more_records": false }, "query": { "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "module": "Contacts", "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1, "cvid": "554023000000093005" }, "created_by": { "id": "554023000000235011", "name": "Patricia Boyle" }, "created_time": "2019-05-09T14:01:24+05:30" } ] } |
The other possible values for the key state are
- ADDED - Indicates that the job is scheduled.
- IN PROGRESS - Indicates that the job is under processing.
Note that only a completed job will have the download URL in its response.
Callback
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
- The state indicates the successful completion ("state":"COMPLETED") or failure ("state":"FAILED") of the job.
- The callback response will also contain the download URL if the job was completed successfully.
Below is a sample callback response for a completed job.
{ "job_id": "554023000000568002", "operation": "read", "state": "COMPLETED", "query": { "module": "Contacts", "criteria": { "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ], "group_operator": "or" }, "page": 1, "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "cvid": "554023000000093005" }, "result": { "page": 1, "count": 1588, "download_url": "/crm/bulk/v2/read/554023000000568002/result", "per_page": 200000, "more_records": false } } |
For more details, refer to the Get Bulk Read Job Details page of our API guide.
This is the final step to complete exporting records from a module in Zoho CRM.
Make a GET request to the download_url you received in the response of step-2.
Step-3: Downloading the CSV file
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}/result
Request Method: GET
This request downloads the zip file. Extract it to get the CSV or ICS file.
Limitations
- Only 10 requests for download are allowed for a one-minute interval. Crossing the limit throws the error with the HTTP code 429. All subsequent downloads will be unsuccessful.
- After completing the bulk read job, you can access the downloadable file only for a period of one day. After that, you cannot access the file via the endpoints.
- You can specify a maximum of 200 select fields via an endpoint. If you specify more than that, the system exports all fields available in that module.
- The maximum value of the page key in the export request body is 500.
- You can specify a maximum of 25 criteria in a query. This also includes the number of criteria in a custom view or a standard view. For example, if you specify the ID of the standard view "My converted Leads", which uses two criteria, the remaining criteria that you can use in the query will be 23.
- in and not_in comparators can accept up to 20 values. For example: 'Lead Status' - 'in' - 'Cold,Warm,Hot,Junk,Contacted,Not Contacted,....(20 values)'.
- You cannot retrieve multi-line, multi-select lookup fields.
- Custom views in the Activities module, Co-Owner view in the Contacts module, the standard views Shared By Me and Shared To Me are not supported in this API.
For more details, refer to the Limitations page of our API guide.
We hope you found this post useful. Let us know your thoughts in the comment section, or reach out to us at support@zohocrm.com if you have any questions.
Cheers!
Access your files securely from anywhere
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Shylaja S
Parikshit
Praveen Sankar
Sticky Posts
Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! CommandsKaizen #165 : How to call Zoho CRM APIs using Client Script
Welcome back to another exciting Client Script post! In this post, we will discuss one of the most frequently asked questions: How do you call Zoho CRM APIs from Client Scripts? In this kaizen post, 1. Ways to make calls to Zoho CRM APIs using ClientKaizen#162 : Add Tags and Open Pre-filled Email Drafts via Client Scripts
Hello everyone! Welcome to another informative Kaizen post. In this post, let us see how to accomplish the following using Client Script. 1. How to auto-tag a record based on field update? 2. How to Open a pre-filled email draft This post will provideExtension Pointers - JS SDK series #4: Managing data from a widget using ZOHO.CRM.HTTP
Handling and managing the data between two applications is a major factor for an efficient business. There are a variety of ways to handle data between Zoho CRM and other third-party applications. You can use deluge CRM tasks, create connectors, use APIKaizen #152 - Client Script Support for the new Canvas Record Forms
Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achieved
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho DataPrep Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho Campaigns Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Blueprint - 'On hold' state with an automatic transition?
I think I'm missing something here so I'm hoping if I explain what I'm trying to achieve someone might be able to give me a way around it. We sometimes get request far in advance but we don't want to action them unless it is 7 days from when they areHOW TO VIEW INDIVIDUAL COST OF NEWLY PURCHASED GOODS AFTER ALLOCATING LANDED COSTS
Hello, I have been able to allocate landed costs to the purchase cost of the new products. however, what i need to see now is the actual cost price (original cost plus landed cost), of only my newly purchased products to enable me set a selling priceClient Script: $Client.refresh({ triggerOnLoad: true }); not triggering onLoad Client Scripts
Hey friends! I'm trying to store a temporary var, refresh the page for the user, then check that temporary var and do some actions. Theoretically using the title's code: $Client.refresh({ triggerOnLoad: true }); should refresh the page and trigger onCalculate months and years between 2 dates on subform
I am looking for a function syntax for an employment candidate to calculate the number of years and months (decimal format. eg 1.2 years) they are employed. I have their start date entered, but if the end date is blank, that tells me they are still employedCombine related grouping values into categories in CRM analytical components
Hello everyone, Analyzing large datasets can be challenging when dealing with numerous individual data points. It's often difficult to extract meaningful insights when information is scattered and ungrouped. To address this, we're adding options to createHow Kiosk Studio can simplify sales for bank employees | Kiosk Studio Session #4
Hello everyone, Banks can boost revenue by cross-selling to their current customers. For example, they can sell credit cards, personal loans, and more to existing account holders. To do this, bank employees move all around the CRM, open and close records,Tip #19 - Create checkbox tracker in your spreadsheet
Hello Sheet users! We are here with yet another tip to help you get the most out of Zoho Sheet. Spreadsheets can be used to handle a variety of tasks, but ever tried using checkboxes to track the progress of your action items dynamically? Here's a sampleIntegrate Oracle with ZOHO CRM
Please let me know if its possible to Integrate Oracle with ZOHO CRM. If yes then help me in doing it.UUIDs
Has anyone coded a Universal Unique Identifier (UUID) generator in Deluge?Introducing signer groups: Streamline signature collection and make it even faster
Hello everyone, We're excited to introduce signer groups, a feature designed to make your signing process quicker, more efficient, and collaborative. With signer groups, you can send an envelope to a group of people, and any member of the group can openCalendar - Recurring Event - End of Month and Last Weekday of Month
How do I set a calendar event to recur on the last day of every month? How do I set a calendar event to recur on the last weekday of the month?Duplicate Zoho Invoices and Sales Receipts
We have been running into an issue where upon saving an invoice or a sales receipt, we get a duplicate: same information saved twice but with a different invoice id/number and sales receipt id/number. I have logged a ticket but so far no response. ItIntroducing revamped Zoho Creator Developer Console—a powerful platform for developing and distributing apps
Hello everyone! We're thrilled to announce the launch of the revamped Zoho Creator Developer Console—a dedicated platform designed specifically for Creator developers and Partners to build, test, and distribute apps to your clients. Developer ConsolePhone App of CRM doesn't find contacts
I open the crm to and do a contact search but nothing comes up. If I do a full zia search it will find an old note that is attached to a contact and I can open the contact. BUT I still can't enter notes of calls, set appointments, etc. It requests I go[Product Update] Deprecation of 'Inbound Transport Details' module in Zoho Analytics - Amazon Seller Central integration
Dear Customers, Please be informed that the Amazon Seller Central will be deprecating the 'Inbound Transport Details' module APIs effective December 20, 2024. As a result, this module will no longer be supported in the Amazon Seller Central - Zoho AnalyticsIs there any support for Drivers license and other forms of ID scanning capabilities?
When scanning a drivers license barcode for data input to a Form, the scanning tool receives all the raw data but there doesnt seem to be a function to limit or remove the unnecessary fragmrnts, like a prefix. Is there any support or info in the helpExciting Update: Multi WhatsApp Business Account (WABA) Support Now Available in SalesIQ!
We’re pleased to share an important update that will enhance the way you manage your WhatsApp Business accounts (WABAs) within SalesIQ. With the launch of Multi WABA support, you can now connect and manage multiple brands more effectively, each underGravity Forms plugin not passing some fields
I use the gravity form zoho plugin to push data from my lead form into my lead page in Zoho CRM. Everything was working file for about 6 months. Suddenly on Oct 1st, some of the fields are no longer getting passed to Zoho. The fields with the problemCan't delete bank transactions (i changed from 14 days trail to free just now)
Hi, I manually added one bank transaction When i try to delete it, it say below: What should I do?Feature Request: Search in the PC client. Some thoughts about the search.
Hi all. I'm really excited to start using Zoho Notebook, but I'm missing some of the search capabilities on my desktop. There are also some thoughts on improving search in general. Search is very important to me, without it it is difficult for me to useLeave Policy for Brazil
Hi, Brazil asked us to configure Zoho People to apply the following policy: · To block starting vacations 2 business days before holidays or weekends; Employees cannot start their vacations 2 days before holidays or weekends. Example: If December 25thAdding date field to each individual Items when creating Invoices?
When adding items into an invoice I need to be able to have a date for each item. Example below: Date Item Description Qty Rate Discount(%) Tax Amount 31/07/13 Brown Sugar Performance 1.00 3,000.00 0.00 10% $3300 Is this possible or will it be in theAuto-Create OneDrive Folder Structure Upon Lead Creation
Hello, New to Zoho and looking for help on a critical process automation I'm looking to implement. My company currently utilizes OneDrive for file management and the folder structure is Proposals -> Client Name -> Address (where I need to initially createProfile Page View Customization
I need to change the fields, sections from the profile view of an emplyoyee.What do the Image Sizes mean in Zoho CRM Email Templates?
Below you can see the image options in email templates: Banner, Fit, Small, Medium, Original, Custom. Can someone from Zoho please share with me: What each is/means? How each will look on desktop AND mobile? How to edit "custom"? If I choose "Custom"Translate any published presentation to the language of your choice
As part of our constant effort to enrich your presentation experience, Zoho Show has rolled out a new feature for translating published presentations. Consider the following scenario: Zylker IT services, a multinational corporation, has announced productBlueprint: multi-select lookup field not available in the criteria option
I read this old forum post which stated that multi-select lookup fields are now selectable as an option in a Blueprint transition configuration: https://help.zoho.com/portal/en/community/topic/blueprint-multi-select-lookup-field-not-available-within-blueprint-transitionApply Credit Note Automatically
We need the ability to apply open Credit Notes toward invoices generated by recurring invoice as the first, priority payment. This should be an option that we can enable/disable in the recurring profile and/or the Credit Note. Other invoicing systems can do this. I'm not sure why Zoho Invoice doesn't have it. Here's an example for a Recurring Invoice... If a customer has open Credit Notes, and a saved credit card set for auto-pay, once an invoice goes out, the credit balance gets automatically appliedWorkflow for "Expenses" module?
Hi there, over the last 2 years, Zoho Expense has seen tremendous growth and we are happy with it. But, sometimes it is frustrating to see things are being implemented halfheartedly, or so it seems. For example, There is the possibility to create workflowsRecord Template - Conditionally printing sections
Is there a way to conditionally print a section of a Record Template? More specifically I am printing records from a Form "Invoice". That Form has 3 SubForms. I'd like to print the content of those SubForms using a Record Template but only if they have at least one line item. If they have no line items, I'd like to hide the headers for that section on the printout.User can choose the PDF report
Hi. I would like to find out if a user (Creator or customer portal) to choose from the different PDF customised reports that have been built?Query table pull last 12 months
I am tying to pull the following criteria and the date is always what causes me the issue. I want to pull people (pco_id) who have entries of "event_id" being these 2 events and whos "kind" is Regular or Guest and where the event_starts_at (date column)PLEASE FIX Search options and consider a Global seach option
A recent update has removed the ability to search for addresses and phone numbers under contacts. We cannot find where this moved to (If it is still available). Please put these options back as we cannot locate specific projects anymore. Also please consider allowing for a Global search. This would really improve the search engine. For example: If I search for "Sally Jones" then all invoices , estimates, vendors etc.. would populate.. Please let me know if you need any more information. Thank You....Customize your calendar based on personal preferences
Greetings, We're happy to introduce a few new capabilities to the Activities module's Calendar View! Now you can tailor your calendar's appearance and notification settings to suit your needs. In the past, the Calendar View lacked customization optionsBUG ALERT: Client Script + Commands -> $Page contextual data is not updated
When using the new Client Script Commands feature, there is an issue with the Client Script $Page contextual data not accurately being updated each time a Command is run. Assuming a Client Script Command called "Client Script Command Bug" with the followingShow iFrame of related List inside of Blueprint Transition
Hey, is it possible to show an iFrame of a related list like this inside of a Blueprint transition?Lookup Fields not Converting
I manage holiday properties. I have a lookup to the Accounts (Properties) in the Leads module. The lookup is connected to the property address field. When I convert it the lookup field does not update in Deals, although the property address does. There2024: A Year of Transformation with Zoho Forms
As we close the curtain on another exciting year, it’s time to reflect on the strides Zoho Forms has taken in 2024. From empowering businesses with advanced tools to simplifying workflows and enhancing user experiences, our updates this year were allStop selling out of stock Items.
Hi I have been using Zohobooks for a around 8 month now. I am not involved in selling process but my staff cant stop selling product which they do not hold in stock, this is a big headache for me as physical count never matches what is shown on the books.Bigin API Token Request ("invalid_client")
Hi people, I tried to connect to the API without success, I've read all of the documentation multiple time and tried just about everything. I tried to do it with Python Request module and with Postman, passing the information through both the URL parameterNext Page