Kaizen #12 - Bulk Read API
Hello everyone!
We hope you all are having a wonderful start in the New Year!
Welcome back to another post in the Kaizen series!
What is the Bulk Read API?
The Bulk Read API allows you to export data from a module in Zoho CRM in bulk. The primary difference between the GET records API and the Bulk Read API is the number of records you can retrieve.
While you can get only 200 records per Get records API call, you can fetch 200,000 records per bulk read API call.
When should you use this API?
- When you want to export more than 200 records through an API call.
- When you want to perform background processes like migration, data backup, and initial data sync between Zoho CRM and external services.
What is the difference between GET Records API and Bulk Read API?
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
What kind of export does the Bulk Read API support?
You can export
- Records available in a custom view. You can also apply the criteria on the records available in that custom view to filter the records further. Note that you cannot export records in the Co-owner, Shared by Me, and Shared to Me views.
- Lookup fields of records in a module using the dot (.) operator.
- Records in the events module as an ICS file.
How does the Bulk Read API work?
Scheduling a bulk read job encompasses the following steps.
- Creating a bulk read job.
- Getting the status of the scheduled job.
- Downloading the result.
We will now see each of these steps in detail.
Step-1: Creating a bulk read job
This involves making a POST API call to schedule a job to fetch records that match certain criteria, along with the required fields.
When this call is successful, the response returns an id. You can use this id to know the status of the job by polling, or check the job completion in the callback URL.
Request URL: {{api-domain}}/crm/bulk/v2/read
Request Method: POST
Consider an example where you want to export records from the Contacts module.
The below query fetches the records that matches one of the three criteria—
- The lead source is Advertisement (or)
- The owner's last name is Boyle (or)
- The phone number of the account associated with the contact contains the numbers 99807.
The fields to be fetched are mentioned in the fields array. To fetch all fields in the module, do not input the fields key.
In this query, Account_Name is the default lookup field in the Contacts module. Owner.last_name returns the last name of the owner of the contact, Account_Name returns the ID and Account_Name.Account_Name returns the name of the account associated with the contact, and Account_Name.Phone returns the phone number of the account associated with the contact.
{ "callback": { "method": "post" }, "query": { "module": "Contacts", "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1 } } |
The below table gives the description of the keys in the sample input.
Key and Data Type | Mandatory | Description |
callback JSON Object | Yes | Contains a valid URL, which should allow the HTTP POST method. The Bulk Read Job's details are posted to this URL on successful completion or failure of the job. |
query JSON Object | Yes | Contains the module, fields that you want to export, criteria based on which you want to export the records, and the page. Refer to the query properties table below for more details. |
file_type String | Yes, when you want to export the events as an ICS file. | Specify the value of this key as "ics" to export all records in the Events module as an ICS file. |
query properties
Key and Data Type | Mandatory | Description |
module String | Yes | The API name of the module from which you want to export the records. Specify the module name as "Events" if you want to export the records in the Events module as an ics file. |
cvid String | Yes, when you want to export records in a custom view | The unique ID of the custom view whose records you want to export. You can obtain the cvid from the Custom View Metadata API. |
fields JSON Array | No | The API Name of the fields you want to export from the module. Example: First_Name, Last_Name, Email, Owner.last_name. Do not input this key when you want to export the records in the Events module as an ICS file. The system throws FIELDS_NOT_SUPPORTED error, otherwise. |
page Integer | No | Default value for page is 1. The page value '1' means that the first 200,000 records matching your query will be exported. If you want to fetch the records from the range of 200,001 to 400,000, then you should mention the page as '2'. The maximum value for this key is 500. |
criteria JSON Object | No | To filter the records you want to export. This JSON object contains the API name of the field, comparator, and a group. Refer to the criteria properties table below for more details. |
criteria properties
Key and Data Type | Mandatory | Description |
api_name String | Yes, if group and group_operator are not specified. | API name of the field you want to compare. Example: First_Name, Last_Name, Owner.last_name etc,. Example: "criteria": { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" } |
value String or JSON array | Yes, if group and group_operator are not specified. | Positive integer values only. |
group_operator String | Yes, if api_name, comparator & value are not specified. | Logical operators. Supported values are and, or. |
group JSON array | Yes, if api_name, comparator & value are not specified. | Array of criteria objects. |
comparator String | Yes, if group and group_operator are not specified. | Specifies the comparator. Example: equal, greater_than. Refer to the comparators table for more details. |
comparators
Data Type | Comparator | Value and Limits |
Number (Integer), Decimal/BigInteger/ Currency/Percent) | equal, not_equal, in, not_in, less_than, less_equal, greater_than, greater_equal | Any number values or ${EMPTY} for empty value. Not more than 19 digits for big integer, allows decimal values for decimal and currency fields. In multi-currency enabled accounts, only home currency value is supported. |
Text (Email, Phone, URL, Picklist, Multi-select, etc) | equal, not_equal, in, not_in, contains, not_contains, starts_with, ends_with | Any text or ${EMPTY} for empty value. Not more than 255 characters. |
Date | equal, not_equal, in, not_in, between, not_between | Any date value in ISO 8601 format or ${EMPTY} for empty value. |
DateTime | equal, not_equal, in, not_in, between, not_between | Any date time value in ISO 8601 format or ${EMPTY} for empty value. Example: 2019-04-01T14:24:04+05:30. Milliseconds are not supported. |
Boolean | equal | true or false. |
Lookup | equal, not_equal, in, not_in | Biginteger value of the lookup, ${EMPTY} for empty value, or use the .(dot) operator to establish a relation between two modules. Example: In the Contacts module, Owner fetches the ID of the Owner, whereas Owner.last_name fetches the last name of the owner. Account_Name fetches the ID of the Account associated with the base module, whereas Account_Name.Phone fetches the phone number of the account associated with the base module. |
Text Area (Multi-line) | Not supported | Not supported |
The response to the sample query is as follows. You can use the id in the details key to check the job status periodically as explained in step-2.
{ "data": [ { "status": "success", "code": "ADDED_SUCCESSFULLY", "message": "Added successfully.", "details": { "id": "3652397000000646004", "operation": "read", "state": "ADDED", "created_by": { "id": "3652397000000186017", "name": "Patricia Boyle" }, "created_time": "2019-04-01T14:24:04+05:30" } } ], "info": {} } |
For more details and examples, refer to the Create Bulk Read Job page of our API guide.
Step-2: Getting the status of the scheduled bulk read job
The bulk read API supports polling and callback.
Polling
You can poll (check the status of the scheduled bulk read job) with the job ID you received in the previous step.
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}
Request Method: GET
The response contains the status of the scheduled job as either ADDED, IN PROGRESS, or COMPLETED.
When the job is complete, the response contains the result JSON object with the keys page, count, more_records, and download_url.
You can also find the download_url in the callback response from which you can download the zip file containing the CSV or ICS file.
If the more_records key is true, there are more records you need to export. Simply change the value of the key page in the POST request, and schedule another bulk read job to fetch the next set of records.
The download_url contains the URL to download the CSV file. Follow the instructions in step-3 to download the CSV file.
Here's a sample response to the above query when the job is completed.
{ "data": [ { "id": "3652397000000646004", "operation": "read", "state": "COMPLETED", "result": { "page": 1, "count": 3, "download_url": "/crm/bulk/v2/read/3652397000000646004/result", "per_page": 200000, "more_records": false }, "query": { "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "module": "Contacts", "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1, "cvid": "554023000000093005" }, "created_by": { "id": "554023000000235011", "name": "Patricia Boyle" }, "created_time": "2019-05-09T14:01:24+05:30" } ] } |
The other possible values for the key state are
- ADDED - Indicates that the job is scheduled.
- IN PROGRESS - Indicates that the job is under processing.
Note that only a completed job will have the download URL in its response.
Callback
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
- The state indicates the successful completion ("state":"COMPLETED") or failure ("state":"FAILED") of the job.
- The callback response will also contain the download URL if the job was completed successfully.
Below is a sample callback response for a completed job.
{ "job_id": "554023000000568002", "operation": "read", "state": "COMPLETED", "query": { "module": "Contacts", "criteria": { "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ], "group_operator": "or" }, "page": 1, "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "cvid": "554023000000093005" }, "result": { "page": 1, "count": 1588, "download_url": "/crm/bulk/v2/read/554023000000568002/result", "per_page": 200000, "more_records": false } } |
For more details, refer to the Get Bulk Read Job Details page of our API guide.
This is the final step to complete exporting records from a module in Zoho CRM.
Make a GET request to the download_url you received in the response of step-2.
Step-3: Downloading the CSV file
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}/result
Request Method: GET
This request downloads the zip file. Extract it to get the CSV or ICS file.
Limitations
- Only 10 requests for download are allowed for a one-minute interval. Crossing the limit throws the error with the HTTP code 429. All subsequent downloads will be unsuccessful.
- After completing the bulk read job, you can access the downloadable file only for a period of one day. After that, you cannot access the file via the endpoints.
- You can specify a maximum of 200 select fields via an endpoint. If you specify more than that, the system exports all fields available in that module.
- The maximum value of the page key in the export request body is 500.
- You can specify a maximum of 25 criteria in a query. This also includes the number of criteria in a custom view or a standard view. For example, if you specify the ID of the standard view "My converted Leads", which uses two criteria, the remaining criteria that you can use in the query will be 23.
- in and not_in comparators can accept up to 20 values. For example: 'Lead Status' - 'in' - 'Cold,Warm,Hot,Junk,Contacted,Not Contacted,....(20 values)'.
- You cannot retrieve multi-line, multi-select lookup fields.
- Custom views in the Activities module, Co-Owner view in the Contacts module, the standard views Shared By Me and Shared To Me are not supported in this API.
For more details, refer to the Limitations page of our API guide.
We hope you found this post useful. Let us know your thoughts in the comment section, or reach out to us at support@zohocrm.com if you have any questions.
Cheers!
Access your files securely from anywhere
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Shylaja S
Parikshit
Praveen Sankar
Sticky Posts
Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! CommandsKaizen #165 : How to call Zoho CRM APIs using Client Script
Welcome back to another exciting Client Script post! In this post, we will discuss one of the most frequently asked questions: How do you call Zoho CRM APIs from Client Scripts? In this kaizen post, 1. Ways to make calls to Zoho CRM APIs using ClientKaizen#162 : Add Tags and Open Pre-filled Email Drafts via Client Scripts
Hello everyone! Welcome to another informative Kaizen post. In this post, let us see how to accomplish the following using Client Script. 1. How to auto-tag a record based on field update? 2. How to Open a pre-filled email draft This post will provideExtension Pointers - JS SDK series #4: Managing data from a widget using ZOHO.CRM.HTTP
Handling and managing the data between two applications is a major factor for an efficient business. There are a variety of ways to handle data between Zoho CRM and other third-party applications. You can use deluge CRM tasks, create connectors, use APIKaizen #152 - Client Script Support for the new Canvas Record Forms
Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achieved
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho DataPrep Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho Campaigns Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
missing video-urgent
hi..I have a problem regarding zoho meeting. I already record almost 2hrs for my interview session. After end my session, i'm stop the recording. Its happened when i didn't received any meeting recording at my email.But i received recording 7minutes afterDuplicate New Ticket Emails
I am getting two Ticket emails when a ticket is created. The first one goes to me as the Admin helpdesk@ And because I am an Agent as well it also send to helpdesk@, support1@ and support2@ Is there any way to stop this?Ticket Response Email Template
Is there a way to have an email template automatically selected when I click on Reply for a ticket?Unpublishing Job Post for LinkedIn
Hi, We have experienced issue with the above. We did unpublish a job in Recruit but it still appear under Job section in our LinkedIn page. It did not remove from LinkedIn as it should be. Please check and advise. Regards, SnyderPower of Automation :: Automatically start / pause / stop timer on task status update.
Hello Everyone, A Custom function is a user-written set of code to achieve a specific requirement. Set the required conditions needed as when to trigger using the Workflow rules (be it Tasks / Project) and associate the custom function to it. Requirement:-Editor limitations to define screen types
Guys I have noticed that even in version 2.0 of the editor (which is this new one we use) we still have a lot to improve... When I compare to some more global solutions like Wix, Zyro, Go Daddy, Squarespace and Weebly feel that we have some limitationsDúvidas do Zoho Creator
Pessoal, Estou colocando um tópico para dúvidas do Zoho Creator. Um abraço, LeandroTropicalize Books
Books is an incredibly powerful tool that works well in many countries. But I feel that it is a product that is not yet "tropicalized" for Brazil as we speak (this would be like adapting the local reality). We have many strong competitors who do moreAutomatic Sitemap Generation
Guys are all right? Doesn't make sense for me to have to generate a map site and upload it... because it's not automatically generated just as it is done in WIX? where the customer doesn’t have to worry about this.SEO improvements with ZIA
Are you okay? I would like to bring an idea that would be amazing to improve the product that is the possibility of being able to improve the SEO of the pages (this of each page or each article on the blog) through ZIA so that it could create page SummaryAutomatically updating field(s) of lookup module
I have a lookup field, which also pulls through the Status field from the linked record. When the lookup is first done, the Status is pulled through - this works perfectly. If that Status is later updated, the lookup field does not update as well. AsInitiating a SalesIQ Zobot from a custom button on Zoho Sites
I have created a Zobot set to initiate on a custom action called "Fast_Answers". On Zoho Sites, I created a code snippet button and set it to on-click run the event called "Fast_Answers". I installed the SalesIQ integration code into the Zoho Sites PageZoho Desk -> Zoho Analytics : Where is the field for "Layout" ?
I have many different layouts on my helpdesks and I want to be able to identify the stats for each one, however I can not file the field in the raw data from the Zoho Desk datasource. I thought it might be under "Tickets" but there is nothing. There isSet resolution mandatory field
Hi, i have 2 questions:) : - i want to set the resolution field mandatory before close the ticket. Because for now, i can close the ticket without writing how i solve it - how can i setup zoho desk to receive ticket by email(e.g. clients sent email to support@mydomain.com) and it create a ticket in zoho deskImpuesto automatico en cotizaciones
Buen dia Mi pregunta es como se puede poner alguna operacion para que las cotizaciones me salgan automaticamente con impuestos ya que uno al cargarla al final tienen que añadir el porcentaje de impuesto SaludosZoho Assist "Agree and Download" Button "Greyed Out" ("Light Blued" Out)
Anyone else having issue where support clients are unable to click "Agree and Download" to access the client so that we can provide remote support? This is for "on demand" support via accessing the support page and entering the support key and name. ThisProject Billing with the Staff Hours Method in Zoho Projects
The Staff Hours Billing Method in Zoho Projects allows you to bill your clients based on the actual time spent by each team member on a project, at the rate set for each user. This is useful for projects where different skill sets are needed and serviceCreator Subform to CRM Subform
Hello all, Has anyone successfully written data from a Creator Subform into CRM subform? I have been able to get the rows to populate but none of the data will come through. I'll add my code and the result in CRM. Creator Subform is 'Delivery_Receiving_Hours'.custom fields not populating from deluge script into invoice
Hello, I've created some Deluge script that is meant to take a few inputted invoice custom fields and calculate a few others. I can see when I execute the function that my inputted custom fields are being passed, yet im still ending up with all "null"tax summation function - getting error
Hello, I'm trying to create a function that adds all of individual tax rates from a few jurisdictions. I'm getting an error on line 9 - Value is empty and 'get' function cannot be applied. I've checked that I have data in each of the required fields,Backstage / Zoho Books integration
Hello. We have Zoho One and have slowly started using Backstage. Loving it. Problem is, we have ZERO accounting control over what is sold through this product. When will we be able to connect it to our existing Zoho Books tenant? Thank you very much!Getting list of calendar events over api for zoho mail calendar
Hi, I am using just Zoho mail without using Zoho CRM. I wanted to get all events booked in my zoho mail calendar through an api at regular intervals. I could find such API support for Zoho CRM calendar but not for zoho mail calendar. Can you kindly letAllocate emails to user in a shared mailbox
Hi, This might be obvious, but I cannot find the answer. I have 3 shared mailboxes so any team member can see the emails. Is there a way of allocating a specific email to a user so that it is their responsibility to deal with it? Thanks in advance.Introducing Zoho Campaigns' own gateway for SMS campaigns
We are excited to announce the launch of our SMS Gateway to send SMS through Zoho Campaigns. We have also made a few other changes in our current SMS Campaign model to improve your over all user experience. These updates are planned with an aim to expandUpdate a lookup field in CRM from Creator using deluge
I have a Creator form that creates a new account. When it creates the new account in the Accounts Module, I need it to also populate the Parent Account, which is a lookup field coming from the Module Parent Accounts, field Parent Account Name. I haveSMS Keyword Tracking in Zoho CRM From Zoho Campaigns
Is there a way to track SMS leads in campaigns by associating them with specific keywords or codes? Additionally, can these leads be pushed to the CRM while retaining the keyword for tracking and reporting purposes?Projects Multiselect API
Having troubles setting a mutli select field via API. updateMap = map(); updateMap.put("UDF_MULTI1","picklist_id1,picklist_id2"); updateProjectRes = invokeurl [ url :"https://projects.zoho.com/restapi/portal/XXXX/projects/" + projectID + "/" type :POSTBill quantity received / PO quantity
PO's are raised & often the quantity received is greater than the PO quantity, so when we receive the bill & adjust the quantity on the bill we get.... Quantity recorded cannot be more than quantity ordered. This necessitates the adjustment of the POZOHO BOOKS - RECEIVING MORE ITEMS THAN ORDERED
Hello, When trying to enter a vendor's bill that contains items with bigger quantity than ordered in the PO (it happens quite often) - The system would not let us save the bill and show this error: "Quantity recorded cannot be more than quantity ordered."Adding Bluesky channel
Hello, Is Bluesky (AT protocol) soon added on Social ? Bluesky is being developped and is now open to anyone (no more invitation) Thank youUsing Queries with dynamic parameters in Kiosk Studio
Hi, I'm pretty new when it comes to developing within Zoho (I'm really a .NET developer), as it was just added to my responsibilities. For a new feature in the CRM, I'm trying to develop a Kiosk function to show a list of records (retrieved by the newUnused items should not count into the available number of custom fields
Hey, I realized that unused Items reduce the number of available custom fields. I can't see a case where that makes sense. Especially in our case where we have two different layouts in Deals with a lot of different fields, this causes problems.Introducing Bot Filtering for Accurate Email Campaign Analytics
Dear Marketers, We're excited to announce a new feature designed to enhance the accuracy of your email campaign analytics: bot filtering. This feature helps you filter out bot-generated opens and clicks, ensuring your campaign reports reflect genuineTip 37: Time Log Restriction in Zoho Projects
Timesheet in Zoho Projects helps you big time in entering log hours for the tasks and issues and approving them. Now, with the new Time Log Restriction option, you can set daily and weekly log hour limits. You can restrict users from entering extra log hours than the permissible limit. The limits are restricted to 24 hours per day and 168 hours per week by default based on business hours. To customize, navigate to Task & Timesheet settings under Portal Configuration in Zoho Projects setup and enableChart showing schedule
I want to be able to create a chart for everyday to check and which of which driver is available on the timeframe. Here's my table Name City Day Start Time End Time Driver1 Medicine Hat Monday 11:45 AM 4:45 PM Driver 2 Medicine Hat Tuesday 11:00 AM 7:00Multiple Vendor SKUs
One of the big concerns we have with ZOHO Inventory is lack of Vendor Skus like many other inventory software packages offer. Being able to have multiple vendor skus for the same product would be HUGE! It would populate the appropriate vendor Sku forIn Zoho Projects, is there a way to create a folders template under documents that can be used once a project is created?
We have a specific folder structure that we would like to use that is standard across every project. Instead of having to create this structure every time a project is created, is there a way to create a template for the folders that can be added?Zoho Payroll in France
When will Zoho Payroll be available in France ?Blueprint - 'On hold' state with an automatic transition?
I think I'm missing something here so I'm hoping if I explain what I'm trying to achieve someone might be able to give me a way around it. We sometimes get request far in advance but we don't want to action them unless it is 7 days from when they areHOW TO VIEW INDIVIDUAL COST OF NEWLY PURCHASED GOODS AFTER ALLOCATING LANDED COSTS
Hello, I have been able to allocate landed costs to the purchase cost of the new products. however, what i need to see now is the actual cost price (original cost plus landed cost), of only my newly purchased products to enable me set a selling priceNext Page