Kaizen #12 - Bulk Read API
Hello everyone!
We hope you all are having a wonderful start in the New Year!
Welcome back to another post in the Kaizen series!
What is the Bulk Read API?
The Bulk Read API allows you to export data from a module in Zoho CRM in bulk. The primary difference between the GET records API and the Bulk Read API is the number of records you can retrieve.
While you can get only 200 records per Get records API call, you can fetch 200,000 records per bulk read API call.
When should you use this API?
- When you want to export more than 200 records through an API call.
- When you want to perform background processes like migration, data backup, and initial data sync between Zoho CRM and external services.
What is the difference between GET Records API and Bulk Read API?
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
GET Records
Bulk Read API
You can fetch a maximum of 200 records per API call.
You can fetch a maximum of 200,000 records per API call.
The response is available instantly.
The response is not available instantly; the bulk read job is scheduled, and the status is available after job completion in the callback URL.
The records are available as JSON objects in the response.
The records are available in a downloadable CSV file or ICS file (for events).
What kind of export does the Bulk Read API support?
You can export
- Records available in a custom view. You can also apply the criteria on the records available in that custom view to filter the records further. Note that you cannot export records in the Co-owner, Shared by Me, and Shared to Me views.
- Lookup fields of records in a module using the dot (.) operator.
- Records in the events module as an ICS file.
How does the Bulk Read API work?
Scheduling a bulk read job encompasses the following steps.
- Creating a bulk read job.
- Getting the status of the scheduled job.
- Downloading the result.
We will now see each of these steps in detail.
Step-1: Creating a bulk read job
This involves making a POST API call to schedule a job to fetch records that match certain criteria, along with the required fields.
When this call is successful, the response returns an id. You can use this id to know the status of the job by polling, or check the job completion in the callback URL.
Request URL: {{api-domain}}/crm/bulk/v2/read
Request Method: POST
Consider an example where you want to export records from the Contacts module.
The below query fetches the records that matches one of the three criteria—
- The lead source is Advertisement (or)
- The owner's last name is Boyle (or)
- The phone number of the account associated with the contact contains the numbers 99807.
The fields to be fetched are mentioned in the fields array. To fetch all fields in the module, do not input the fields key.
In this query, Account_Name is the default lookup field in the Contacts module. Owner.last_name returns the last name of the owner of the contact, Account_Name returns the ID and Account_Name.Account_Name returns the name of the account associated with the contact, and Account_Name.Phone returns the phone number of the account associated with the contact.
{ "callback": { "method": "post" }, "query": { "module": "Contacts", "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1 } } |
The below table gives the description of the keys in the sample input.
Key and Data Type | Mandatory | Description |
callback JSON Object | Yes | Contains a valid URL, which should allow the HTTP POST method. The Bulk Read Job's details are posted to this URL on successful completion or failure of the job. |
query JSON Object | Yes | Contains the module, fields that you want to export, criteria based on which you want to export the records, and the page. Refer to the query properties table below for more details. |
file_type String | Yes, when you want to export the events as an ICS file. | Specify the value of this key as "ics" to export all records in the Events module as an ICS file. |
query properties
Key and Data Type | Mandatory | Description |
module String | Yes | The API name of the module from which you want to export the records. Specify the module name as "Events" if you want to export the records in the Events module as an ics file. |
cvid String | Yes, when you want to export records in a custom view | The unique ID of the custom view whose records you want to export. You can obtain the cvid from the Custom View Metadata API. |
fields JSON Array | No | The API Name of the fields you want to export from the module. Example: First_Name, Last_Name, Email, Owner.last_name. Do not input this key when you want to export the records in the Events module as an ICS file. The system throws FIELDS_NOT_SUPPORTED error, otherwise. |
page Integer | No | Default value for page is 1. The page value '1' means that the first 200,000 records matching your query will be exported. If you want to fetch the records from the range of 200,001 to 400,000, then you should mention the page as '2'. The maximum value for this key is 500. |
criteria JSON Object | No | To filter the records you want to export. This JSON object contains the API name of the field, comparator, and a group. Refer to the criteria properties table below for more details. |
criteria properties
Key and Data Type | Mandatory | Description |
api_name String | Yes, if group and group_operator are not specified. | API name of the field you want to compare. Example: First_Name, Last_Name, Owner.last_name etc,. Example: "criteria": { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" } |
value String or JSON array | Yes, if group and group_operator are not specified. | Positive integer values only. |
group_operator String | Yes, if api_name, comparator & value are not specified. | Logical operators. Supported values are and, or. |
group JSON array | Yes, if api_name, comparator & value are not specified. | Array of criteria objects. |
comparator String | Yes, if group and group_operator are not specified. | Specifies the comparator. Example: equal, greater_than. Refer to the comparators table for more details. |
comparators
Data Type | Comparator | Value and Limits |
Number (Integer), Decimal/BigInteger/ Currency/Percent) | equal, not_equal, in, not_in, less_than, less_equal, greater_than, greater_equal | Any number values or ${EMPTY} for empty value. Not more than 19 digits for big integer, allows decimal values for decimal and currency fields. In multi-currency enabled accounts, only home currency value is supported. |
Text (Email, Phone, URL, Picklist, Multi-select, etc) | equal, not_equal, in, not_in, contains, not_contains, starts_with, ends_with | Any text or ${EMPTY} for empty value. Not more than 255 characters. |
Date | equal, not_equal, in, not_in, between, not_between | Any date value in ISO 8601 format or ${EMPTY} for empty value. |
DateTime | equal, not_equal, in, not_in, between, not_between | Any date time value in ISO 8601 format or ${EMPTY} for empty value. Example: 2019-04-01T14:24:04+05:30. Milliseconds are not supported. |
Boolean | equal | true or false. |
Lookup | equal, not_equal, in, not_in | Biginteger value of the lookup, ${EMPTY} for empty value, or use the .(dot) operator to establish a relation between two modules. Example: In the Contacts module, Owner fetches the ID of the Owner, whereas Owner.last_name fetches the last name of the owner. Account_Name fetches the ID of the Account associated with the base module, whereas Account_Name.Phone fetches the phone number of the account associated with the base module. |
Text Area (Multi-line) | Not supported | Not supported |
The response to the sample query is as follows. You can use the id in the details key to check the job status periodically as explained in step-2.
{ "data": [ { "status": "success", "code": "ADDED_SUCCESSFULLY", "message": "Added successfully.", "details": { "id": "3652397000000646004", "operation": "read", "state": "ADDED", "created_by": { "id": "3652397000000186017", "name": "Patricia Boyle" }, "created_time": "2019-04-01T14:24:04+05:30" } } ], "info": {} } |
For more details and examples, refer to the Create Bulk Read Job page of our API guide.
Step-2: Getting the status of the scheduled bulk read job
The bulk read API supports polling and callback.
Polling
You can poll (check the status of the scheduled bulk read job) with the job ID you received in the previous step.
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}
Request Method: GET
The response contains the status of the scheduled job as either ADDED, IN PROGRESS, or COMPLETED.
When the job is complete, the response contains the result JSON object with the keys page, count, more_records, and download_url.
You can also find the download_url in the callback response from which you can download the zip file containing the CSV or ICS file.
If the more_records key is true, there are more records you need to export. Simply change the value of the key page in the POST request, and schedule another bulk read job to fetch the next set of records.
The download_url contains the URL to download the CSV file. Follow the instructions in step-3 to download the CSV file.
Here's a sample response to the above query when the job is completed.
{ "data": [ { "id": "3652397000000646004", "operation": "read", "state": "COMPLETED", "result": { "page": 1, "count": 3, "download_url": "/crm/bulk/v2/read/3652397000000646004/result", "per_page": 200000, "more_records": false }, "query": { "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "module": "Contacts", "criteria": { "group_operator": "or", "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ] }, "page": 1, "cvid": "554023000000093005" }, "created_by": { "id": "554023000000235011", "name": "Patricia Boyle" }, "created_time": "2019-05-09T14:01:24+05:30" } ] } |
The other possible values for the key state are
- ADDED - Indicates that the job is scheduled.
- IN PROGRESS - Indicates that the job is under processing.
Note that only a completed job will have the download URL in its response.
Callback
If you do not want to poll for the status of the job, you can wait for the system to notify you of job completion on the callback URL provided in the POST request.
- The state indicates the successful completion ("state":"COMPLETED") or failure ("state":"FAILED") of the job.
- The callback response will also contain the download URL if the job was completed successfully.
Below is a sample callback response for a completed job.
{ "job_id": "554023000000568002", "operation": "read", "state": "COMPLETED", "query": { "module": "Contacts", "criteria": { "group": [ { "api_name": "Lead_Source", "comparator": "equal", "value": "Advertisement" }, { "api_name": "Owner.last_name", "comparator": "equal", "value": "Boyle" }, { "api_name": "Account_Name.Phone", "comparator": "contains", "value": "99807" } ], "group_operator": "or" }, "page": 1, "fields": [ "Last_Name", "Owner", "Owner.last_name", "Account_Name.Account_Name", "Account_Name.Phone", "Lead_Source", "Created_Time" ], "cvid": "554023000000093005" }, "result": { "page": 1, "count": 1588, "download_url": "/crm/bulk/v2/read/554023000000568002/result", "per_page": 200000, "more_records": false } } |
For more details, refer to the Get Bulk Read Job Details page of our API guide.
This is the final step to complete exporting records from a module in Zoho CRM.
Make a GET request to the download_url you received in the response of step-2.
Step-3: Downloading the CSV file
Request URL: {{api-domain}}/crm/bulk/v2/read/{job_id}/result
Request Method: GET
This request downloads the zip file. Extract it to get the CSV or ICS file.
Limitations
- Only 10 requests for download are allowed for a one-minute interval. Crossing the limit throws the error with the HTTP code 429. All subsequent downloads will be unsuccessful.
- After completing the bulk read job, you can access the downloadable file only for a period of one day. After that, you cannot access the file via the endpoints.
- You can specify a maximum of 200 select fields via an endpoint. If you specify more than that, the system exports all fields available in that module.
- The maximum value of the page key in the export request body is 500.
- You can specify a maximum of 25 criteria in a query. This also includes the number of criteria in a custom view or a standard view. For example, if you specify the ID of the standard view "My converted Leads", which uses two criteria, the remaining criteria that you can use in the query will be 23.
- in and not_in comparators can accept up to 20 values. For example: 'Lead Status' - 'in' - 'Cold,Warm,Hot,Junk,Contacted,Not Contacted,....(20 values)'.
- You cannot retrieve multi-line, multi-select lookup fields.
- Custom views in the Activities module, Co-Owner view in the Contacts module, the standard views Shared By Me and Shared To Me are not supported in this API.
For more details, refer to the Limitations page of our API guide.
We hope you found this post useful. Let us know your thoughts in the comment section, or reach out to us at support@zohocrm.com if you have any questions.
Cheers!
Access your files securely from anywhere
Zoho Developer Community
New to Zoho LandingPage?
Zoho LandingPage Resources
New to Zoho Survey?
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Bigin?
Topic Participants
Shylaja S
Parikshit
Praveen Sankar
Sticky Posts
Client Script | Update - Introducing Commands in Client Script!
Have you ever wished you could trigger Client Script from contexts other than just the supported pages and events? Have you ever wanted to leverage the advantage of Client Script at your finger tip? Discover the power of Client Script - Commands! CommandsKaizen #165 : How to call Zoho CRM APIs using Client Script
Welcome back to another exciting Client Script post! In this post, we will discuss one of the most frequently asked questions: How do you call Zoho CRM APIs from Client Scripts? In this kaizen post, 1. Ways to make calls to Zoho CRM APIs using ClientKaizen#162 : Add Tags and Open Pre-filled Email Drafts via Client Scripts
Hello everyone! Welcome to another informative Kaizen post. In this post, let us see how to accomplish the following using Client Script. 1. How to auto-tag a record based on field update? 2. How to Open a pre-filled email draft This post will provideExtension Pointers - JS SDK series #4: Managing data from a widget using ZOHO.CRM.HTTP
Handling and managing the data between two applications is a major factor for an efficient business. There are a variety of ways to handle data between Zoho CRM and other third-party applications. You can use deluge CRM tasks, create connectors, use APIKaizen #152 - Client Script Support for the new Canvas Record Forms
Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achieved
New to Zoho TeamInbox?
Zoho TeamInbox Resources
Zoho DataPrep Resources
Zoho CRM Plus Resources
Zoho Books Resources
Zoho Subscriptions Resources
Zoho Projects Resources
Zoho Sprints Resources
Qntrl Resources
Zoho Creator Resources
Zoho Campaigns Resources
Zoho CRM Resources
Design. Discuss. Deliver.
Zoho Show Resources
Get Started. Write Away!
Writer is a powerful online word processor, designed for collaborative work.
Zoho CRM コンテンツ
-
オンラインヘルプ
-
Webセミナー
-
機能活用動画
-
よくある質問
-
Ebook
-
-
Zoho Campaigns
- Zoho サービスのWebセミナー
その他のサービス コンテンツ
Nederlandse Hulpbronnen
ご検討中の方
Recent Topics
Subform Time field to string.
Good afternoon All. I have a Subform 'Delivery_Receiving_Hours' that captures Day (Dropdown), Time_Open (Time), and Time_Close (Time). I need to capture this data and send it to a multiline field in the CRM. The code, posted below, below will captureworkflow for bounced email gets triggered, but email is status = opened
Hello, I have a workflow that sends me an email if outgoing email are bounced. Now I got some kind of this emails, but the corrosponding contacts have status = open at the email. Why this bounce-workflow is triggered? Reports > Email Reports > BounceData export
I need to export our customer's data and projects' data for our purpose but am unable to export full data i only get around 3160 projects and around 2k customer can you please help me to get full data, pleaseAdjusting Physical Inventory
Not getting very far with support on this one, they say they are going to fix it but nothings happened since November. Please give this a thumbs up if you would like to see this feature or comment if you have some insight. Use Case: Inventory set to beZoho Marketing Plus : Un outil tout-en-un pour la création de pages, la collaboration et la gestion du calendrier marketing
Nous sommes ravis de vous présenter trois nouvelles fonctionnalités puissantes de Zoho Marketing Plus s’enrichit désormais d’un page web (l'éditeur de pages), qui vous permet de créer des pages attrayantes et à fort taux de conversion pour vos campagnesGrouping payments to match deposits
Is there a way to group multiple invoice payments together so they match credit card batches and grouped deposits in the bank account? Basically, we are creating invoices for each of our transactions, and applying a payment to each of the invoices. Our payments are either credit cards or checks. We want to be able to group payments together so when our bank account reflects a credit card batch made up of many transactions, or the deposit we took to the bank that has multiple checks from differentEmployees can not add some expenses suddenly
Zoho expense was working fine and whenever there was a new merchant, it would automatically add and also the same auto added in Zoho Books (due to merchant-vendor sync) untill now. From today, it is having problems in searching the existing vendors andZoho email setup in office365
When i am trying to setup zoho mail setup using my domain in office365 and it is not working and it says that we couldn't log on to the incoming (IMAP) server and please check your email address and password and try again. I was able to login using myiOS 10: Caller ID new feature?
Hi, in the update history of the iOS App (for iOS10) - v.3.2 - i found the point "caller identification" has this feature been deactivated again? i cannot find anything on my iphone on how to activate this feature. or does it just work from the beginning?Recommendations to store meeting notes for easy access from Contacts, Accounts & Deals module records?
I would like your advice on how to achieve this use case for my organization. It’s related to where/how best to store meeting notes from a conversation with Contact(s) working at an Account (Company) in the context of a Deal. The ideal solution (fromBank reconciliation. Match Transaction -filter
When matching an imported bank statement file we only get a match if it is an excact match on both amount and date. Then a suggestions comes up with a very broad selection regarding amount, and no default "between" dates. I can then go an manually adjust the filter, and have to put in from-to amounts and dates. How do I set a default from-to date? As an example, I would like the date to be +- 3 days, Thanks.Added new staff but does not appear in other organization list
Hi, I added the new staff under Sales Manager in the contacts, but it does not appear in the other organization list where I need to create a contact, and I can't select the newly added Sales ManagerIntegrating Calendly with Zoho Calendar in Zoho Mail
I moved my office into a business incubator space that uses Calendly for meeting management and events. Calendly doesn't have a integration with Zoho Calendar and vice versa. I was directed to Zapier for integration but it doesn't have an integrationMap fields from module X to a lookup field in subform in module Y
Hi there In the 1st screenshot attached, you can see a subform in myLeads module. You can see that there is a number already filled there - that is the 'Property ID' and it is a single line field. It is the 'Property ID' of an entry I have in another🎄 Jingle, Mingle, and Automate: Spread Christmas Cheer with Zoho Desk Auto-Replies! 🎄
Hello Everyone! Welcome to this week's episode of the Community Learning Series. Christmas is in the air, and I’m sure we can all feel the jingle and the mingle of the season! The folks at Zylker Techfix are no exception—they’re busy with holiday planshow to create a new line in string in Client Script?
I want to show an alert using client script, I need to add a new line in String, I assume I can use \n\n inside a string, but unfortunately it doesnt work ZDK.Client.showAlert("First Line \n\nI expect this is in second line");Surely it's time Inline editing from views
I think the first request I found for in-line editing from grids was approximately 12 years ago - that post was locked because it was suggested Zoho sheetview solved the problem. However, it's now 2024, and in-line editing from grids is just a basic expectation.Multi branding issue with sender addresses
Hello, I'm currently working on a project involving two (seperate) brands. Named 'Windeck' and 'Prolance'. They've chosen CRM Plus and I'm currently working on CRM, SalesIQ, Social and Marketing Automation. So far, I'm able to make enough separationsHow to Replace an Assessment in a Job Opening on Zoho Recruit
Hi everyone, I’m currently using Zoho Recruit and would like to replace the assessment linked to a specific job opening. I want to remove the existing assessment and add a new one. What is the best way to do this without losing any important data or affectingIs there API Doc for Zoho Survey?
Hi everyone, Is there API doc for Zoho Survey? Currently evaluating a solution - use case to automate survey administration especially for internal use. But after a brief search, I couldn't find API doc for this. So I thought I should ask here. ThanEmail Campaigns overview page is missing SENT DATE and # people sent to!
I would like to see the date the email campaign was sent, so I can understand and track when each email campaign was sent. Right now, unless you go to a contact who received a campaign, you cannot see when the campaign was sent (!!!!!!). So, if my bossSEO recommendation of H1 tag for website tittle
The exact words are “ It is good practice to place the page title inside the H1tag.” Now I already have one H1 tag on my website but it is not website tittle. In the SEO recommendation that is clear too that I have h1 tag on my page. Now I don’t knowHow to choose other payment methodes than creditcards
We have connected stripe as a payment provider in zoho books, booking, commerce and checkout. In stripe we selected al major payment methodes for Belgium (mainly bancontact). However, at checkout customers seems to have only the possibility to pay withIntroducing Zia LLM: Zoho’s in-house Generative AI solution for CRM's AI capabilities
Hello everyone, We're excited to announce the launch of our in-house Large Language Model (LLM) by Zia to power our AI offerings. What is LLM? LLM stands for Large Language Model, a powerful AI technology that processes and generates human-like text basedHow to call a Creator function which is in a different Creator application?
How to call a Creator function which is in a different Creator application?Can the code in my "Successful form submission" WF be invoked from a function?
Can "Successful form submission" be invoked from a function? Data gets into a form manually and programatically. My code in "successful form submission" is good and I want to reuse it/call it, from another function which does Insert Into How to achieveKaizen #169 - Serialization and Schema Management in Queries
Hello everyone! Welcome back to another post in the Kaizen series! In Kaizen #166, we discussed handling Variables in Queries and associating the query in Kiosk. This week, we will discuss Serialization and Schema management in Queries. Business ScenarioIntroducing Keyboard Shortcuts for Zoho CRM
Dear Customers, We're happy to introduce keyboard shortcuts for Zoho CRM features! Until now, you might have been navigating to modules manually using the mouse, and at times, it could be tedious, especially when you had to search for specific modulesFeature Request: Notebooks within notebooks (Tree-like structure)
Dear Zoho! I already migrated all my stuff from Google Keep, Im really fond of Zoho Notebook so far. One thing that could make the service much more powerful is multi-level notebooks (or tree like structure). For example, entering into Notebook namedCan't get authorization for Sandbox environment using the self client
Hello, After creating a self client, and following the client-credentials method (as it's not optimat to manually generate a code for every 10 minutes), after inputting the sandbox org id for SOID parameter, im getting the error: "error": "no_org". ForCreate landing pages from Zoho Marketing Plus
Hey everyone, Over the last few months, we've introduced various features and enhancements to bolster the marketing capabilities of Zoho Marketing Plus and make it simpler for everyone. To that end, we're excited to announce that Zoho LandingPage is nowCustom service report or Zoho forms integration
Hello, So far the experience with Zoho FSM and the integration with Books has been good, however there are limitations with service reports. As with my business, many organisations send technicians to different types of jobs that call for a differentEmail tracking subdomain
The Email Tracking configuration screen of the ZeptoMail asks for a subdomain. I have gone through the documentation but could not find more information about how that subdomain is used by ZeptoMail to track the emails. Can someone throw some light aboutChart View group X-axis values above a value
I have a data set with X values ranging from 0 up to 300-400, the Y values are an AVG of the values for the given X. I am interested in the values at the low end of the scale, say 0-10 and want the X values 10 and greater to be grouped into a single categoryHow do I get the Text Account name instead of the Reference number?
Good Morning everyone! I am very new to zoho analytics. I'm trying to create some pivot tables and when I add the Account or Division name it comes up as a reference number. From my research, I need to use a lookup. I'm having a really hard time understandingCustom API - Need to create a string return value, not only MAP
@Support: When creating a Custom API it only allows a return from a function of MAP type. The service I'm using requires a string return, how can this be achieved?missing video-urgent
hi..I have a problem regarding zoho meeting. I already record almost 2hrs for my interview session. After end my session, i'm stop the recording. Its happened when i didn't received any meeting recording at my email.But i received recording 7minutes afterDuplicate New Ticket Emails
I am getting two Ticket emails when a ticket is created. The first one goes to me as the Admin helpdesk@ And because I am an Agent as well it also send to helpdesk@, support1@ and support2@ Is there any way to stop this?Ticket Response Email Template
Is there a way to have an email template automatically selected when I click on Reply for a ticket?Unpublishing Job Post for LinkedIn
Hi, We have experienced issue with the above. We did unpublish a job in Recruit but it still appear under Job section in our LinkedIn page. It did not remove from LinkedIn as it should be. Please check and advise. Regards, SnyderNext Page