Export data to Google BigQuery
Zoho DataPrep supports exporting data to Google BigQuery. BigQuery is a fully managed enterprise data warehouse that helps you manage organization data.
To export data to Google BigQuery
1. Open an existing pipeline or create a pipeline from the Home Page, Pipelines tab or Workspaces tab. You can bring your data from 50+sources.
2. On the Pipeline Builder page, once you have completed creating your data flow and applying the necessary transforms in your stages, you can right-click a stage and select the Add Destination option.
2. Choose Google BigQuery under the All destinations tab or from the Cloud databases category.
3. Select an account from the saved connections or add a new account using the Add new option.
4. Enter the Project ID and click the Sign in with Google button.
4. After signing in with your google account, you can click Continue to allow Zoho DataPrep to access data from your Google BigQuery project for the first time.
Note: The connection configuration will be saved for exporting to Google BigQuery project in the future. Credentials are securely encrypted and stored.
5. Click Connect.
Note: If you face trouble connecting to your database, please make sure Zoho DataPrep's IP Addresses are whitelisted in your application to export data to cloud databases. Click here to know about Zoho DataPrep IP addresses.6. Once you have successfully connected to your Google BigQuery project, you can choose when and where to export the data.
7. Choose Existing table if you want to export data to an existing table and select one from the list of dataset and tables available in the database. There are two ways in which you can choose how to add the new rows to the table.
- If the new rows are to be added to the table, choose Append.
- If the newly added rows are to replace the existing rows, select Overwrite from the dropdown.
11. If you want to create a new table and export data, select the New table option, select the dataset, enter a Table name and choose how to add the new rows to the table and proceed to exporting.
12. Click Save. Now that you have added a destination, you may want to try executing your pipeline using a manual run at first. Once you make sure manual run works, you can then set up schedule to automate the pipeline. Learn about the different types of runs here.
Info: Each run is saved as a job. When a pipeline run is executed, the data fetched from your data sources will be prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination. This complete process is captured in the Jobs page.
13. If the manual run succeeds without any errors, your data will be exported successfully. If you are exporting data to an existing table in your cloud database, and if the manual run fails throwing the below target match error, you can fix them by completing the target matching steps.
Target matching is a useful feature in DataPrep that prevents export failures caused due to errors from the data model mismatch.
Note: Target matching will be applied even if you export data to a new table and automate the pipeline using the Schedule run option. Only during the first schedule it will treated as a new table. In the subsequent exports, the new table will be treated as an existing table and target matching will be applied.
Target matching during export to cloud databases
Target matching happens before the data is exported to the destination. Target matching is a useful feature in DataPrep that prevents export failures caused due to errors from the data model mismatch. Using target matching, you can set the required cloud database table as the target and align the source dataset columns to match with your target table. This ensures seamless export of high quality data to the cloud databases.
Note: Target matching failure is not an export failure. Target matching happens before the data is actually exported to the destination. This way the schema or data model errors that could cause export to fail are caught beforehand preventing export failures.
When target match check fails
1. If the target match check fails during export, you can go to the DataPrep Studio page, click the target matching icon 
at the top right corner and choose the Show target option. The target's data model is displayed above the existing source dataset. The columns in the source dataset are automatically aligned to match the columns in the target dataset, if found.
Target matching displays the different icons and suggestions on the matched and unmatched columns. You can click on these suggestions to quickly make changes to match the existing column with the target column. To make it easier for you to fix the errors, the target module in your cloud database is attached as a target to your data. You can view the mapping of your data with the table in the DataPrep Studio page along with the errors wherever there is a mismatch. You can hover over the error icons to understand the issue and click on them to resolve each error.
Note: All columns are displayed in the grid by default. However, you can filter out the required option by clicking the All columns link.
2. Click the View summary link to view the summary of the target match errors. The summary shows the different model match errors and the number of columns associated with each error. You can click on the required error columns and click Apply to filter out specific error columns.
Target match error summary
- The Target match errors section shows the errors and the number of columns associated with each error.
- The section at the top lists the error categories along with the number of errors in each category.
- You can click them to filter errors related to each category in the panel.
- In the default view, all columns are displayed. However, you can click any error category and get a closer look at the columns or view the error columns alone by selecting the Show only errors checkbox.
- Your filter selection in the Target match error summary will also be applied on the grid in the DataPrep Studio page.
Target matching errors
The errors in target matching are explained below:
- Unmatched columns : This option shows all the unmatched columns in the source and target.Note:
- The non-mandatory columns in the target can either be matched with a source column if available or ignored.
- The columns in the source that are missing in the target need to be matched or removed to proceed exporting.
When using the unmatched columns option, you can toggle the Show only mandatory columns option to see if there are any mandatory columns(set as mandatory in the target) and include them. You can also fix only the mandatory columns and proceed to exporting.
- Data type mismatch : This option displays the columns from the source having data types that do not match the columns in the target.
- Data format mismatch : This option displays columns from the source having date, datetime and time formats that differ from those in the target.
- Constraint mismatch : This option displays the columns that do not match the data type constraints of the columns in the target. To know how to add constraints for a column, click here.
- Mandatory column mismatch: This option displays the columns that are set as mandatory in the target but not set as mandatory in your source.Note: The mandatory columns cannot be exported to the destination unless they are matched and set as mandatory. You can click the
icon above the column to set it as mandatory. You can also use the Set as mandatory (not null) check box under the Change data type transform to set a column as mandatory.
- Data size overflow warnings : This option filters the columns with data exceeding the maximum size allowed in the target.
3. After fixing the errors you can go to the Pipeline builder page and run your pipeline to export your data. Once you make sure manual run works, you can then set up schedule to automate the pipeline. Learn about the different types of runs here.
See Also
How to import data from Google BigQuery into Zoho DataPrep?
Access your files securely from anywhere
Zoho CRM Training Programs
Learn how to use the best tools for sales force automation and better customer engagement from Zoho's implementation specialists.
Zoho DataPrep Personalized Demo
If you'd like a personalized walk-through of our data preparation tool, please request a demo and we'll be happy to show you how to get the best out of Zoho DataPrep.
- Zoho Workerly Home
- Forums
- Connect With Us:
- Zoho Recruit Home
- Forums
- Connect With Us:
New to Zoho Writer?
Create. Review. Publish.
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
You are currently viewing the help pages of Qntrl’s earlier version. Click here to view our latest version—Qntrl 3.0's help articles.
New to Zoho Survey?
Zoho Sheet Resources
Zoho Forms Resources
New to Zoho Sign?
Zoho Sign Resources
Sign, Paperless!
New to Zoho TeamInbox?
Zoho TeamInbox Resources
New to Zoho ZeptoMail?
Design. Discuss. Deliver.
New to Zoho Workerly?
New to Zoho Recruit?
New to Zoho CRM?
New to Zoho Projects?
New to Zoho Sprints?
New to Zoho Assist?
New to Bigin?
Related Articles
Import data from Google BigQuery
Zoho DataPrep supports importing data from Google BigQuery. BigQuery is a fully managed enterprise data warehouse that helps you manage organization data. With this connector, you can streamline your ETL workflows, enable smooth data movement, and ...Export data to Google Drive
Zoho DataPrep supports exporting data to Google Drive, a file storage and synchronization service developed by Google. Google Drive allows users to store files in the cloud, synchronize files across devices, and share files. To export data to Google ...Export data to Google Cloud SQL PostgreSQL
Zoho DataPrep allows you to export data to Google Cloud SQL, a cloud database. It is a fully-managed database service that helps you set up, maintain, manage, and administer your relational databases on Google Cloud Platform. To export data to Google ...Export data to Google Cloud SQL - MySQL
Zoho DataPrep allows you to export data to Google Cloud SQL, a cloud database. It is a fully-managed database service that helps you set up, maintain, manage, and administer your relational databases on Google Cloud Platform. To export data to Google ...Export data to cloud storage services
You can export data to the following cloud storage services. Google Drive OneDrive Box Storage Dropbox Zoho WorkDrive Amazon S3 To export data to cloud storage services 1. Open an existing pipeline or create a pipeline from the Home Page, Pipelines ...
New to Zoho LandingPage?
Zoho LandingPage Resources