Prerequisite

The Project Id and the Dataset location details are required to fetch the data from Google BigQuery.

1. Login to your Google account.
2. Choose the Project which has the datasets that you wish to import. The Project ID is displayed in the Project Selector at the top left of the BigQuery console, where you can select or switch between different projects.
3. Click any table name to view the details. The Dataset info pane will list the details about the Dataset Location. Refer to BigQuery locations article to learn more.
   

To import data from Google BigQuery

Info: You can also click the Import data  icon at the top of the pipeline builder and bring data from multiple sources into the pipeline.

4. To add a new connection, enter the Project ID and click the Sign in with Google button.

Note: Project ID can be found on your Google cloud console. Google BigQuery authorizes access to resources based on verified identity.

5. After signing in with your google account, you can click Continue to allow Zoho DataPrep to access data from your Google BigQuery project for the first time.

Note: The connection configuration is securely saved for importing data from the Google BigQuery project in the future. 

6. Select the tables that need to be imported. Click Import.

7. You can also use SQL query to select and import data.

The incremental fetch option is not available when the data is imported using a query from databases. Click here to know more about incremental fetch from cloud database.

8. Click Import to begin importing data from your Google BigQuery project.

9. Once you complete importing data, the Pipeline builder page opens and you can start applying transforms to the ETL pipeline. You can also right-click the stage and choose the Prepare data option to prepare your data using various transforms in the DataPrep Studio page. Click here to know more about the transforms. 

10. Once you are done creating your data flow and applying necessary transforms in your stages, you can right-click a stage and add a destination to complete your data flow.

Note:  After adding a destination to the ETL pipeline, you can try executing your pipeline using a manual run at first. Once you make sure manual run works, you can then set up schedule to automate the pipeline and data movement. Learn about the different types of runs here.

While configuring the Schedule, Backfill, Manual reload, Webhooks, or Zoho Flow, the import configuration needs to be mandatorily setup for all the sources. Without setting up the import configuration, the run cannot be saved. Click here to know more about how to set up import configuration.

11. After configuring a run, a pipeline job will be created at the run time. You can view the status of a job with the granular details in the Job summary. Click here to know more about the job summary.

To edit the Google BigQuery connection  

DataPrep saves your data connections to avoid the hassle of keying in the credentials every time you need to connect to a data source or destination. You can always edit the saved data connection and update them with new parameters or credentials using the Edit connection option.

 

1. Click Saved connections from the left pane under the Choose your data source box while creating a new dataset.

 

2. You can manage your saved connections right from the data import screen. Click the  (ellipses) icon to share, edit, view the connection overview, or remove the connection.

 

3. Click the Edit connection option. You can update the Project ID in the saved connection and click Update.

 

SEE ALSO

How to add a new pipeline?

Learn about importing data using saved data connections.

Import data from cloud storage services

Learn about scheduling pipeline

Import data from local databases

Export data to Google BigQuery