Kaizen #12 - Bulk Read API
Hello everyone!
We hope you all are having a wonderful start in the New Year!
Welcome back to another post in the Kaizen series!
What is the Bulk Read API?
The Bulk Read API allows you to export data from a module in Zoho CRM in bulk. The primary difference between the GET records API and the Bulk Read API is the number of records you can retrieve.
While you can get only 200 records per Get records API call, you can fetch 200,000 records per bulk read API call.
When should you use this API?
- When you want to export more than 200 records through an API call.
- When you want to perform background processes like migration, data backup, and initial data sync between Zoho CRM and external services.
What is the difference between GET Records API and Bulk Read API?
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
What kind of export does the Bulk Read API support?
You can export
- Records available in a custom view. You can also apply the criteria on the records available in that custom view to filter the records further. Note that you cannot export records in the Co-owner, Shared by Me, and Shared to Me views.
- Lookup fields of records in a module using the dot (.) operator.
- Records in the events module as an ICS file.
How does the Bulk Read API work?
Scheduling a bulk read job encompasses the following steps.
- Creating a bulk read job.
- Getting the status of the scheduled job.
- Downloading the result.
We will now see each of these steps in detail.
Step-1: Creating a bulk read job
This involves making a POST API call to schedule a job to fetch records that match certain criteria, along with the required fields.
When this call is successful, the response returns an id. You can use this id to know the status of the job by polling, or check the job completion in the callback URL.
Request URL: {{api-domain}}/crm/bulk/v2/read
Request Method: POST
Consider an example where you want to export records from the Contacts module.
The below query fetches the records that matches one of the three criteria—
- The lead source is Advertisement (or)
- The owner's last name is Boyle (or)
- The phone number of the account associated with the contact contains the numbers 99807.
The fields to be fetched are mentioned in the fields array. To fetch all fields in the module, do not input the fields key.
In this query, Account_Name is the default lookup field in the Contacts module. Owner.last_name returns the last name of the owner of the contact, Account_Name returns the ID and Account_Name.Account_Name returns the name of the account associated with the contact, and Account_Name.Phone returns the phone number of the account associated with the contact.
{ "callback": { "method": "post" }, "query": { "module": "Contacts", "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1 } } |
The below table gives the description of the keys in the sample input.
Key and Data Type | Mandatory | Description |
callback JSON Object | Yes | Contains a valid URL, which should allow the HTTP POST method. The Bulk Read Job's details are posted to this URL on successful completion or failure of the job. |
query JSON Object | Yes | Contains the module, fields that you want to export, criteria based on which you want to export the records, and the page. Refer to the query properties table below for more details. |
file_type String | Yes, when you want to export the events as an ICS file. | Specify the value of this key as "ics" to export all records in the Events module as an ICS file. |
query properties
Key and Data Type | Mandatory | Description |
module String | Yes | The API name of the module from which you want to export the records. Specify the module name as "Events" if you want to export the records in the Events module as an ics file. |
cvid String | Yes, when you want to export records in a custom view | The unique ID of the custom view whose records you want to export. You can obtain the cvid from the Custom View Metadata API. |
fields JSON Array | No | The API Name of the fields you want to export from the module. Example: First_Name, Last_Name, Email, Owner.last_name. Do not input this key when you want to export the records in the Events module as an ICS file. The system throws FIELDS_NOT_SUPPORTED error, otherwise. |
page Integer | No | Default value for page is 1. The page value '1' means that the first 200,000 records matching your query will be exported. If you want to fetch the records from the range of 200,001 to 400,000, then you should mention the page as '2'. The maximum value for this key is 500. |
criteria JSON Object | No | To filter the records you want to export. This JSON object contains the API name of the field, comparator, and a group. Refer to the criteria properties table below for more details. |
criteria properties
Key and Data Type | Mandatory | Description |
api_name String | Yes, if group and group_operator are not specified. | API name of the field you want to compare. Example: First_Name, Last_Name, Owner.last_name etc,. Example: "criteria": { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" } |
value String or JSON array | Yes, if group and group_operator are not specified. | Positive integer values only. |
group_operator String | Yes, if api_name, comparator & value are not specified. | Logical operators. Supported values are and, or. |
group JSON array | Yes, if api_name, comparator & value are not specified. | Array of criteria objects. |
comparator String | Yes, if group and group_operator are not specified. | Specifies the comparator. Example: equal, greater_than. Refer to the comparators table for more details. |
comparators
Data Type | Comparator | Value and Limits |
Number (Integer), Decimal/BigInteger/ Currency/Percent) | equal, not_equal, in, not_in, less_than, less_equal, greater_than, greater_equal | Any number values or ${EMPTY} for empty value. Not more than 19 digits for big integer, allows decimal values for decimal and currency fields. In multi-currency enabled accounts, only home currency value is supported. |
Text (Email, Phone, URL, Picklist, Multi-select, etc) | equal, not_equal, in, not_in, contains, not_contains, starts_with, ends_with | Any text or ${EMPTY} for empty value. Not more than 255 characters. |
Date | equal, not_equal, in, not_in, between, not_between | Any date value in ISO 8601 format or ${EMPTY} for empty value. |
DateTime | equal, not_equal, in, not_in, between, not_between | Any date time value in ISO 8601 format or ${EMPTY} for empty value. Example: 2019-04-01T14:24:04+05:30. Milliseconds are not supported. |
Boolean | equal | true or false. |
Lookup | equal, not_equal, in, not_in | Biginteger value of the lookup, ${EMPTY} for empty value, or use the .(dot) operator to establish a relation between two modules. Example: In the Contacts module, Owner fetches the ID of the Owner, whereas Owner.last_name fetches the last name of the owner. Account_Name fetches the ID of the Account associated with the base module, whereas Account_Name.Phone fetches the phone number of the account associated with the base module. |
Text Area (Multi-line) | Not supported | Not supported |
The response to the sample query is as follows. You can use the id in the details key to check the job status periodically as explained in step-2.
{ "data": [ { "status": "success", "code": "ADDED_SUCCESSFULLY", "message": "Added successfully.", "details": { "id": "3652397000000646004", "operation": "read", "state": "ADDED", "created_by": { "id": "3652397000000186017", "name": "Patricia Boyle" }, "created_time": "2019-04-01T14:24:04+05:30" } } ], "info": {} } |
For more details and examples, refer to the Create Bulk Read Job page of our API guide.
Step-2: Getting the status of the scheduled bulk read job
The bulk read API supports polling and callback.
Polling
You can poll (check the status of the scheduled bulk read job) with the job ID you received in the previous step.
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}
Request Method: GET
The response contains the status of the scheduled job as either ADDED, IN PROGRESS, or COMPLETED.
When the job is complete, the response contains the result JSON object with the keys page, count, more_records, and download_url.
You can also find the download_url in the callback response from which you can download the zip file containing the CSV or ICS file.
If the more_records key is true, there are more records you need to export. Simply change the value of the key page in the POST request, and schedule another bulk read job to fetch the next set of records.
The download_url contains the URL to download the CSV file. Follow the instructions in step-3 to download the CSV file.
Here's a sample response to the above query when the job is completed.
{ "data": [ { "id": "3652397000000646004", "operation": "read", "state": "COMPLETED", "result": { "page": 1, "count": 3, "download_url": "/crm/bulk/v2/read/3652397000000646004/result", "per_page": 200000, "more_records": false }, "query": { "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "module": "Contacts", "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1, "cvid": "554023000000093005" }, "created_by": { "id": "554023000000235011", "name": "Patricia Boyle" }, "created_time": "2019-05-09T14:01:24+05:30" } ] } |
The other possible values for the key state are
- ADDED - Indicates that the job is scheduled.
- IN PROGRESS - Indicates that the job is under processing.
Note that only a completed job will have the download URL in its response.
Callback
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
- The state indicates the successful completion ("state":"COMPLETED") or failure ("state":"FAILED") of the job.
- The callback response will also contain the download URL if the job was completed successfully.
Below is a sample callback response for a completed job.
{ "job_id": "554023000000568002", "operation": "read", "state": "COMPLETED", "query": { "module": "Contacts", "criteria": { "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ], "group_operator": "or" }, "page": 1, "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "cvid": "554023000000093005" }, "result": { "page": 1, "count": 1588, "download_url": "/crm/bulk/v2/read/554023000000568002/result", "per_page": 200000, "more_records": false } } |
For more details, refer to the Get Bulk Read Job Details page of our API guide.
This is the final step to complete exporting records from a module in Zoho CRM.
Make a GET request to the download_url you received in the response of step-2.
Step-3: Downloading the CSV file
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}/result
Request Method: GET
This request downloads the zip file. Extract it to get the CSV or ICS file.
Limitations
- Only 10 requests for download are allowed for a one-minute interval. Crossing the limit throws the error with the HTTP code 429. All subsequent downloads will be unsuccessful.
- After completing the bulk read job, you can access the downloadable file only for a period of one day. After that, you cannot access the file via the endpoints.
- You can specify a maximum of 200 select fields via an endpoint. If you specify more than that, the system exports all fields available in that module.
- The maximum value of the page key in the export request body is 500.
- You can specify a maximum of 25 criteria in a query. This also includes the number of criteria in a custom view or a standard view. For example, if you specify the ID of the standard view "My converted Leads", which uses two criteria, the remaining criteria that you can use in the query will be 23.
- in and not_in comparators can accept up to 20 values. For example: 'Lead Status' - 'in' - 'Cold,Warm,Hot,Junk,Contacted,Not Contacted,....(20 values)'.
- You cannot retrieve multi-line, multi-select lookup fields.
- Custom views in the Activities module, Co-Owner view in the Contacts module, the standard views Shared By Me and Shared To Me are not supported in this API.
For more details, refer to the Limitations page of our API guide.
We hope you found this post useful. Let us know your thoughts in the comment section, or reach out to us at support@zohocrm.com if you have any questions.
Cheers!
Access your files securely from anywhere
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Shylaja S
Parikshit
Praveen Sankar
Sticky Posts
Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! CommandsKaizen #165 : How to call Zoho CRM APIs using Client Script
Welcome back to another exciting Client Script post! In this post, we will discuss one of the most frequently asked questions: How do you call Zoho CRM APIs from Client Scripts? In this kaizen post, 1. Ways to make calls to Zoho CRM APIs using ClientKaizen#162 : Add Tags and Open Pre-filled Email Drafts via Client Scripts
Hello everyone! Welcome to another informative Kaizen post. In this post, let us see how to accomplish the following using Client Script. 1. How to auto-tag a record based on field update? 2. How to Open a pre-filled email draft This post will provideExtension Pointers - JS SDK series #4: Managing data from a widget using ZOHO.CRM.HTTP
Handling and managing the data between two applications is a major factor for an efficient business. There are a variety of ways to handle data between Zoho CRM and other third-party applications. You can use deluge CRM tasks, create connectors, use APIKaizen #152 - Client Script Support for the new Canvas Record Forms
Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achieved
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho DataPrep Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho Campaigns Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Record Asset Received as Payment
How exactly would you account for this in books? For example, I receive a mini computer for a review and I get to keep it after the video is published. Would debit my normal asset account (e.g. Computers) and credit an income account (e.g. Other Income).Invoice Line Item Report
Is it possible to run an 'Invoice Line Item Report'? The 'Invoice Details Report' shows one row per invoice. I would like one row per Invoice Line Item.Transform Your Customer Support with AI-Powered Chatbots in Zoho SalesIQ
Ever wondered how some companies seem to have superhuman customer support? Let's uncover their secret! In the digital age, customer expectations are skyrocketing. Did you know that according to McKinsey, 75% of consumers expect a response within fiveProgressive Invoicing
Progressing invoicing is needed for many industries. It would be great to see it implemented into Zoho Books as well. Set up and send progress invoices in QuickBooks Desktop (intuit.com)Client Script - mapping data from different module
Dear ZOHO Team Firstly I need to describe the need - I need to have data from Contacts module based on lookup field - the 5 map limit is not enough for me because I have almost 20 fields to copy So I have decided to make a Customer Script - and from unknownDORA compliance
For DORA (Digital Operational Resilience Act) compliance, I’ll want to check if Zoho provides specific features or policies aligned with DORA requirements, particularly for managing ICT risk, incident reporting, and ensuring operational resilience inFiles Uploaded to Zoho WorkDrive Not Being Indexed by Search Engines
Hello, I have noticed that the files I upload to Zoho WorkDrive are not being indexed by search engines, including Google. I’d like to understand why this might be happening and what steps I can take to resolve it. Here are the details of my issue: FileZoho Creator Upcoming Updates - December 2024
Hi all, We're excited to be back with the latest updates and developments on the Creator platform. Here's what we're going over this month: Deluge AI assistance Rapid error messages in Deluge editor QR code & barcode generator Expandable RTF and multiCustomer can't comment on SO or Invoice
Hi I just saw that my customers are not able to submit a comment either on invoices or sales order. What happens if my customer hits submit is just nothing. only a red line appears on top of the page which probalby indicates an error. I'm not able toZoho Creator customer portal limitation | Zoho One
I'm asking you all for any feedback as to the logic or reasoning behind drastically limiting portal users when Zoho already meters based on number of records. I'm a single-seat, Zoho One Enterprise license holder. If my portal users are going to add records, wouldn't that increase revenue for Zoho as that is how Creator is monetized? Why limit my customer portal to only THREE external users when more users would equate to more records being entered into the database?!? (See help ticket reply below.)See Calendar When Creating Meetings On Record Page
It would be a great user experience to see you calendar while you are creating a meeting on a record page. Here is how I imagine it could look:Saved filters, layout rule support, related list quick navigation, and more
Hello Everyone, We're excited to share some new features and enhancements in the Zoho CRM iOS and Android apps that will improve your mobile experience. These updates will make your CRM journey more efficient and user-friendly. Here's a look at what'sPower of Automation: Automatically send an email to all portal users with today's list of Open tasks.
Hello Everyone, A Custom Function is a user-written set of code to achieve a specific requirement. Set the required conditions needed as to when to trigger using the Workflow rules (be it Tasks / Project) and associate the custom function to it. Requirement:-Introduction of Robotics Process Automation in Zoho products
It will be great if Zoho can start advancing from automation to robotics process automation. For a start, it can be started with smart document understanding. Provide OCR engines Google cloud, Microsoft Azure Computer vision OCR, Microsoft OCR, OmnipageLock a custom field on a deal record but keep all other fields editable?
I have a custom field, which auto-populates a job number upon converting a lead to a deal but the automation breaks if someone accidentally edits that field. I want to lock that field but keep all other fields open. Is this possible? I've tried throughAdd Feature To Hide Plugin Sections On Record View
Hi team, I'm trying to help a client tidy up their CRM. When it comes to record view some sections and fields are visible no matter what Layout Rules are applies and they are not removeable from the layout editor. I would like to see an option to hideCreator Simplified #10: Predefine Form Field Values and Make Them Read-Only for Users
Hey Creators, Ready for this week's tip in the Creator Simplified series? Today, we will explore how to have read only fields in a form. Use Case: Assume a scenario where the default value for a Department field needs to be English Literature, but youfetch records from analytics table from creator
I have a creator workflow that I am working in that will compare data from within the app to a table in zoho analytics. Is there a way to fetch a record from Analytics? I have attempted a custom connector with analytics and tried to use it with invokeDirectly Edit, Filter, and Sort Subforms on the Details Page
Hello everyone, As you know, subforms allow you to associate multiple line items with a single record, greatly enhancing your data organization. For example, a sales order subform neatly lists all products, their quantities, amounts, and other relevantAbility to Change Custom View After Cadence Creation
Dear Zoho Team, I hope you are well. We would like to request an enhancement to the Cadence feature in Zoho CRM. Currently, during the creation of a Cadence, we can select a Custom View under the "Who is this for?" section. However, once the Cadence isZoho Creator Integration with QuickBooks: A Step-by-Step Guide
Introduction: Integrating Zoho Creator with QuickBooks allows you to sync your business data between the two platforms, providing a seamless experience for managing accounting, invoicing, and financial data. This integration helps automate workflows andNote not being pulled for other modules in email template
Hi there, Currently i am creating an email template that is able to pull the data from notes field in estimate module and email it to procurement team where they will be able to receive the email with the contents of the note, i am unable to replicateNo Sales Returns on SO's with Dropped Shipped items + Inventory Items
We have encountered an issue in Zoho related to sales orders that include both dropshipped items and inventory items. Specifically, it is currently not possible to create sales returns for the company’s own inventory items from these sales orders. ThisPre-fill TO and CC fields for Email Templates
This would be a game changer to be able to set either specific email addresses or merge fields based on deal role titles into email templates. Please pass this along to *hopefully* add to future features of Zoho CRM.Pick list - Cannot save list "Special Characters not Allowed" error message
Bulk uploading values. All values are pretty standard - with the exception of a "-" (dash). Like: Industry - Prepared Food Is the simple dash a special character too? JanFlow with CRM
Hello, I have a simple flow that uses a web hook to enter data into a Sales Order. I have the web hook sending Flow data which has a PO field. If the PO has a special character like - or / or \ the task fails. How can I get the flow to be okay with theMaking the Resolution Tab Mandatory
Hello Everyone! This edition is here to show you how to make the Resolution mandatory when closing a ticket. The Ticket Resolution tab helps keep a record of the solution provided for the ticket query. The resolution can serve as a quick reference toRemove County field from Customer Address input screen (or allow input to be deleted)
We are in the USA and have just noticed that there is now a County field in the Customer Address input screen (and maybe other areas of Zoho Books, but this is the one affecting us at the moment). County is not important to our business, and in fact weNotificación de cumpleaños
Hola: Se puede enviar alguna alerta de felicitación al personal que cumple años, que se dispare solo? Si existe como se puede hacer? SaludosAutomation#25: Move Tickets to Unassigned When the Owner Is Offline
Hello Everyone, Welcome to this week's Community Series! 'Tis the holiday season—a time when work often takes a brief pause. The holiday spirit is in full swing at Zylker Techfix too, with employees taking some well-deserved time off. During this period,Callback URLs
I need to connect to an external service through an API that requires me to provide a Callback URL so that a status update can be sent back when the API request has been processed. Is there a way to do this in Creator without having to use a middlewareEmail signature not being included if user creates ticket / sends email
When I create a ticket (send email), the signature doesn't appear to be added to the ticket. Can you confirm if this is the case? It would obviously be useful to include the user's signature even when sending a client an email and not only on replieZoho Notebook window ignores taskbar
When maximized to full screen, the Zoho Notebook window ignores the presence of the taskbar and overlaps it. What could be the problem? Linux Mint 22 Cinnamon. Zoho Notebook 3.2.0URL for job opening
I would like to ask you to add this feature to the product: create an unique url for each job opening on Zoho Recruit, so that I can add a link to it when I post the opportunity in a mailing list or on a forum.Document images
We used to be able to rotate the images but this has now been removed ???VENDORS ARE NOT SYNCHED WITH CONTACTS IN CRM
Hello, While the ACCOUNTS and CONTACTS (Including the primary contact) are synced with the CONTACTS module in CRM, the vendor's CONTACTS are not synced with CRM - which basically forces the users to re-enter all vendor's contacts twice and then updateInvolved account types are not applicable when create journals
{ "journal_date": "2016-01-31", "reference_number": "20160131", "notes": "SimplePay Payroll", "line_items": [{ "account_id": "538624000000035003", "description": "Net Pay", "amount": 26690.09, "debit_or_credit": "credit" }, { "account_id": "538624000000000403", "description": "Gross", "amount": 32000, "debit_or_credit": "debit" }, { "account_id": "538624000000000427", "description": "CPP", "amount": 1295.64, "debit_or_credit": "debit" }, { "account_id": "538624000000000376", "description":KB Templates
* It would be nice if Zoho can provide users an option to create custom templates for KB articles. Also, it would be nice as well if the users can have an option to 1.) select a default template and 2.) declare default tag/tags, for KB articles created through Ticket's resolution.Zoho CRM Reports Module on Mobil App
I have the mobile app and the reports module doesn't appear in the sidebar for some reason. I saw a Youtube video where the user had the Reports module on mobile. Is there a setting to show it on mobile? Thanks.Contacts Don't Always Populate
I've noticed that some contacts can easily be added to an email when I type their name. Other times, a contact doesn't appear even though I KNOW it is in my contact list. It is possible the ones I loaded from a spreadsheet are not an issue and the onesNext Page