Zoho DataPrep allows you to seamlessly bring in your forms data from Zoho Creator using the Zoho Creator connector. Zoho Creator is a powerful low-code application development platform that helps businesses build custom web and mobile apps faster. With this connector, you can perform ETL operations, automate data movement, and ensure smooth data integration between your Zoho Creator and Zoho DataPrep.

 To import data from Zoho Creator 

1. Open an existing pipeline or create a pipeline from the Home Page, Pipelines tab or Workspaces tab and click the Add data option.

Info: You can also click the Import data  icon at the top of the pipeline builder to bring data from multiple sources into the pipeline.

2. Choose the Zoho Apps category from the left pane and click on Zoho Creator. You can also directly search the option in the search box.

3. Choose the required environment from the Choose environment dropdown.

Production - This is the live, production version of the application.

5. You can choose to import all data, or configure your import since a previous date.    

a) If you choose to import all data -  All data available in the selected forms will be imported.



b) If you choose to import data from a specific date - All data recorded in the selected forms (both added and modified) since (on or after) the chosen date will be imported.

Info: The new and modified data will be fetched from the selected date using the Added_Time and Modified_Time columns generated while importing data from Creator (system generated columns).

Note: The forms created using integration with other applications and data sources (Integrated Datasource) cannot be imported into Zoho DataPrep.

7. The selected forms appear at the top. Click Import to import the selected forms into Zoho DataPrep.

8. Once you have completed importing data, Pipeline builder page will open from where you can start applying transforms to the ETL pipeline. You can also right-click the stage and choose the Prepare data option to prepare your data in the DataPrep Studio page. Click here to know more about the transforms.



9. Once you are done creating your data flow and applying necessary transforms in your stages, you can right-click a stage and add a destination to complete your data flow.

Note:  After adding a destination to the pipeline, you can try executing your ETL pipeline using a manual run at first. Once you make sure manual run works, you can then set up schedule to automate the pipeline and data movement. Learn about the different types of runs here.

Schedule   

 You can schedule your pipeline using the Schedule option. You can also import incremental data from your Creator application while scheduling. Incremental data import is a method used to import new or modified records after the previous sync.

Schedule configuration

1. Select the Schedule option in the pipeline builder.

2. Select a Repeat method (hourly, daily, weekly, monthly) and set frequency using Perform every dropdown. The options of the Perform every dropdown change with the Repeat method. Click here to know more.

Info: The range can be between 2-100. The default value is 2.

Import configuration

Note: The import configuration needs to be mandatorily setup for all the sources in the pipeline. Without setting up the import configuration, the schedule cannot be saved. 

5. Select the Click here link to set the import configuration. 

6. Select the required option from the How to import data from source? drop down. You can choose to import all data, only modified and new data, only new data, or do not import data using this option.

Import all data

If you want to import all data, select the date from when data needs to be imported in the From field. This option will import all available data from the selected date for every scheduled run.

Incremental data fetch

Only modified and new data

Info: The modified and new data will be fetched using the Modified_Time and Added_Time columns from the last imported time. 

Only new data

To import the new data incrementally from the last imported time, select Only new data option from the drop-down.

Info: The new data will be fetched using the Added_Time columns from the last imported time.

Use existing data if new data is not available: 

During incremental import,

1. If the checkbox is checked: When there is no new data in the source, the last fetched data will be imported again.
2. If the checkbox is unchecked: When there is no new data in the source, the import will fail and no files will be imported. This will, in turn, cause the entire pipeline job to fail.

Do not import data

The data is imported only once. The second time, the rules gets applied on the same data and gets exported.

7. Click Save to schedule import for your dataset.

Important: 

• If you have already configured a schedule from Creator, data will be reloaded based on your earlier configuration under the Import configuration section when you click on the Edit schedule option and set a new schedule.

Schedule settings

Stop export if data has invalid values: Enabling this will stop the export when prepared data still has invalid values.

Order exports

You can use this option when you have configured multiple destinations and would like to determine in what order the data has to be exported to destinations.

If not enabled, export will run in the default order.

Note: This option will be visible only if you have added more than one destination in your pipeline.

To rearrange the order of your export destinations

1) Click the Order exports toggle

2) You can drag and drop to change the order of the destinations and then click Save.

Note: Click the Edit order link if you want to rearrange the order again.

8. After you configure the schedule configuration, click Save to execute the schedule. This will start the pipeline.

Each scheduled run is saved as a job. When a pipeline is scheduled, the data will be fetched from your data sources, prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination through data integration at regular intervals. This complete process is captured in the job history.

9. To go to the jobs list of a particular pipeline, go to the  ellipses icon in the pipeline builder, and click on the Job history menu to check the job status of your pipeline.

10. Click the required job ID in the Jobs history page to navigate to the Job summary of a particular job.

The Job summary shows the history of a job executed in a pipeline flow. Click here to know more.

11. When the schedule is completed, the data prepared in your pipeline will be exported to the configured destinations.

Info: You can also view the status of your schedules later on the Jobs page.

Note: If you make any further changes to the pipeline, the changes are saved as a draft version. Choose the Draft option and mark your pipeline as ready for the changes to reflect in the schedule.

After you set your schedule, you can choose to Pause schedule or Resume schedule, Edit schedule and Remove schedule using the Schedule Active option in the pipeline builder.

When you edit and save a schedule, the next job will be from the last schedule run time to the next scheduled data interval.

To export data to Zoho Creator

1. Open an existing pipeline or create a pipeline from the Home Page, Pipelines tab or Workspaces tab. You can bring your data from 50+sources.

2. On the Pipeline Builder page, once you have completed creating your data flow and applying the necessary transforms in your stages, you can right-click a stage and select the Add Destination option.

 

3. You can search for Zoho Creator under the All destinations tab or filter using the Zoho Apps category and click on it.

 

 

4. Choose the required environment where you would like to export your data from the Choose environment dropdown.

Below are the available environments:

Development (BETA) - By default, when an application is created, it will be added to the Development environment. You can make the desired edits to the application in this environment. This section enables users to access the application while it's running in the development environment. 

Stage (BETA) - After making changes in the Development environment, you can test the application in the Stage environment, an intermediate step between development and production before distributing the finalized changes.

Production - This is the live, production version of the application.

5. Choose the Workspace, Application and Forms in Zoho Creator to which you want to export your data.

Note: Please note that exporting data to the forms created using integrated data source is not supported.

6. Choose one of the below options to determine how to handle the data being exported to Zoho Creator:

• Add only new records : Using this option you can only add newly imported records from Zoho Creator.
  

• Only update existing records : Using this option, you can update the existing records in Zoho Creator that match the selected field (match based on field).
  

• Both add and update records : This option will update the records that match the selected field provided and insert those records that do not match the selected field value.

 

 

7. Select a unique and mandatory field to match data from the Match based on field dropdown. Please make sure that the field is set mandatory and marked as no duplicate values in your Zoho Creator form. Click here to know more.

8. By default, the Disable Creator Workflows option is enabled. This prevents the workflows from getting triggered when data is exported to Creator. You can also choose to disable it.

9. When you are exporting data to the same form that you imported from, you may choose to Delete records from Zoho Creator that do not match the data to be exported.

For example, if you've imported records from the Candidates form and removed duplicates, you may choose to delete those duplicate records from Zoho Creator when you export back to the same Candidates form.

You can also review the records before deleting using the Review records for deletion link.

10. Save the destination configuration. 

Check Target Matching

Navigate to the DataPrep Studio page of the stage where Zoho Creator is set as the destination.

Click the target matching iconat the top right corner and choose the Show target option. Make sure the target matching is done to avoid export failure. Click here to know more about target matching.

Now, after checking the target matching, you may want to try executing your pipeline using a manual run at first. Once you make sure the manual run works, you can then set up a schedule to automate the pipeline. Learn about the different types of runs here.

Info: Each run is saved as a job. When a pipeline run is executed, the data fetched from your data sources will be prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination. This complete process is captured in the Jobs page.

11. If the manual run succeeds without any errors, your data will be exported successfully. If the manual run fails throwing the below target match error, you can fix them by completing the target matching steps.

Target matching is a useful feature in DataPrep that prevents export failures caused due to errors from the data model mismatch. Learn more about target matching. 

 

Info : You also have an option to rollback your export because of partial failures or other problems in the Output section of the job summary page. Learn how to rollback.

 

Target matching during export to Zoho Creator

Target matching happens before the data is exported to the destination. Target matching is a useful feature in DataPrep that prevents export failures caused due to errors from the data model mismatch. Using target matching, you can set the required Zoho Creator form as the target and align the source dataset columns to match with your target Zoho Creator form. This ensures seamless export of high quality data to Zoho Creator.

Note: Target matching failure is not an export failure. Target matching happens before the data is actually exported to the destination. This way the schema or data model errors that could cause export to fail are caught beforehand preventing export failures. Learn more about target matching.

 When target match check fails

1. If the target match check fails during Zoho Creator export, you can go to the DataPrep Studio page, click the target matching icon  at the top right corner and choose the Show target option. The target's data model is displayed above the existing source dataset. The columns in the source dataset are automatically aligned to match the columns in the target dataset, if found.

Note: All columns are displayed in the grid by default. However, you can filter out the required option by clicking the All columns link.

3. Click the View summary link to view the summary of the target match errors. The summary shows the different model match errors and the number of columns associated with each error. You can click on the required error columns and click Apply to filter out specific error columns. 

Target match error summary

Target matching errors

• Unmatched columns : This option filters the columns in the source dataset that do not match the columns in the target.
  
  
  
  Info : When using the unmatched columns filter, you can toggle the Show only mandatory columns to verify if there are any mandatory columns (set as mandatory columns) required and include them. You can also fix only the mandatory columns and proceed exporting.
  

• Data type mismatch : This option filters the columns from the source dataset having data types that do not match the columns in the target.

• Data format mismatch : This option filters columns from the source dataset having data formats that differ from those in the target dataset.

• Constraint mismatch : This option filters the columns that do not match the data type constraints of the columns in the target. To know how to add constraints for a column, click here.

• Mandatory columns mismatch : This option displays the columns that are set as mandatory in the target but not set as mandatory in your source.

  
  
  Note: The mandatory columns cannot be exported to the destination unless they are matched and set as mandatory. You can click the  icon above the column to set it as mandatory. 

4. After fixing the errors you can go to the Pipeline builder page and run your pipeline to export your data. Once you make sure manual run works, you can then set up schedule to automate the pipeline and data movement. Learn about the different types of runs here.

Note: When there are multiple duplicate records in a Creator form, only the last duplicate record will be updated based on the unique field; the other duplicate records will be skipped. We recommend using the deduplicate transform in Zoho DataPrep before exporting to Creator to avoid duplicates.

5. In case of partial export failures, you may choose to undo or cancel the changes done to the Zoho Creator form using the Rollback now feature. You can rollback even a successful export if you want to. Learn more about rollback in the next section.

Rollback export    

You can undo or cancel the changes done to your Zoho Creator form in case of partial export failures using the Rollback feature. Rollback will help you delete the inserted records, undo changes in the updated records, and restore the deleted records in your Zoho Creator form.

You can rollback after the pipeline run from the Output tab in the Jobs summary page. 

Rollback now

In case of partial export failures, you may choose to undo or cancel the changes done to the Zoho Creator form using the Rollback now feature. You can rollback even a successful export if you want to. 

1. Click the Rollback now link in the Output tab to rollback the export immediately. Review the rollback details and click Rollback in the confirmation dialog.

2. The rollback progress is shown. Please do not perform any other action while rollback is in progress. The rollback process cannot be reverted once started.

Important:

• You can rollback only the latest export, i.e., the Rollback now link will be visible only for the most recent export that was carried out using the processed data.

• The rollback process cannot be reverted once started. Please do not perform any other action while rollback is in progress.

 Limitations

1. You can import a maximum of 5 million records at a time

4. You can export a maximum of 1 million records at a time.

SEE ALSO

How to schedule a pipeline?

Learn more about Target Matching

How to import data from Zoho Creator?

Learn how Zoho DataPrep can help you achieve different use cases with Creator data.