Batch Size in Zoho DataPrep
Batch size refers to the maximum number of rows you can process at a time in Zoho DataPrep during every import, data preparation, or export of data.
The default batch size is 1 Million. This means that you can process as many rows at a time as large as your batch size limit. However, you can always scale this by purchasing Batch size Add-Ons from the Zoho Store.
Let's say, you are trying to import a dataset with more than one million rows from Zoho Analytics. If your current batch size is limited to 1M, the import will fail throwing the below error:
To continue importing, you need to upgrade your batch size limit by purchasing batch size add-ons from Zoho Store. You can either purchase batch size add-ons or upgrade your plan to upgrade your batch size limit. Here's the link to
DataPrep's pricing
page.
Purchasing batch size add-ons will simply increase your batch size limit, not the no. of rows processed. Click here
to see a full comparison of DataPrep's plans and features.
You can view your DataPrep organization's allowed batch size limit in the
Manage Subscription
tab in Zoho DataPrep. If you would like to purchase Batch Size Add-Ons, click
Manage your Plan
to access Zoho Store. To know how to purchase Batch Size Add-On, click here.
FAQs
1. What will happen if I try to import datasets that exceed the batch size limit?
If you attempt to import data from cloud databases, Zoho Analytics, or Zoho CRM more than your batch size limit, the import will fail throwing an error. To continue importing, you need to upgrade your batch size limit by purchasing batch size add-ons from Zoho Store. Go to Manage Subscription tab and click Manage your Plan to access Zoho Store and purchase the batch size add-on.
If you try importing data more than your batch size limit from any other data sources such as, files, cloud storage, FTP, URL, etc. the dataset will be imported as we will only know the row count after initial parsing of the file. But the below error will be thrown when you try to apply any transform. Click here to know the allowed and restricted operations on datasets that exceed the batch size limit.
In order to resume prepping your data, you can choose to purchase batch size add-on.
You can also purchase add-ons or upgrade your plan from the
Manage Subscription page.
2. How do I increase my batch size limit?
You can increase your batch size limit by purchasing Batch size Add-On from the Zoho Store.
a. Click the Settings icon from the top-right corner to open the Settings tab.
b. Click the
Manage Your Plan
button, and you will be taken to Zoho Store.
c. Click the
Upgrade Row/Add-Ons
button.
d. Choose an option from the
Batch Size Add-On
dropdown.
d. You can review the price and add-on details in the
After Addition
section.
e. Click to
Proceed
and make payment. Once you complete your purchase successfully, you can refresh the
Manage Subscription
page in DataPrep to see your updated subscription details.
3. What will happen to those datasets that already exist in my DataPrep organization which exceed the batch size limit? Will I lose my data?
No, you will not lose datasets with more than the batch size limit applicable for your DataPrep organization. But you can cannot apply transforms or export such datasets. However, you can continue using Zoho DataPrep seamlessly by purchasing Batch Size Add-Ons from the Zoho Store.
4. Which transforms are allowed and restricted on existing datasets that exceed the batch size limit?
1. You can use the
Advanced filters
option to filter and view data but you cannot choose to Keep or Delete the filtered rows.
2. You can Undo or Redo the rules applied on the existing datasets. However, you cannot apply new transforms and add new rules to your ruleset.
3. You cannot switch sample strategy or generate a new sample.
4. You cannot edit the
Data source details
under the
Data source configuration
settings from the
Ruleset
pane.
SEE ALSO