Zoho DataPrep allows you to trigger your pipeline using Zoho Flow. Zoho Flow is an integration platform which connects cloud applications. It helps you set up workflows to automate information exchange among the apps you use.

1. Configure the pipeline with Zoho Flow in Zoho DataPrep
   
2. Create a new flow in Zoho Flow
3. Configure the trigger in Zoho Flow
   
4. Configure the action in Zoho Flow
5. Save the flow
   
6. Switch on the flow

For instance, let's use this flow as an example, and walk through how to recreate it.
When a new file is added in Zoho Workdrive, a flow can trigger a pipeline that imports the newly added Workdrive file.

Configure Zoho Flow in Zoho DataPrep

Important: When you add an action to run a pipeline in Zoho Flow, it must be configured in Zoho DataPrep.

To configure Zoho Flow in DataPrep follow the below steps:

1. Once you have finished creating your entire pipeline,

Pro tip: You may want to try executing your pipeline using the Run button. Each run is saved as a job.

When a pipeline run is executed, the data fetched from your data sources will be prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination. This complete process is captured in the job history. Once you ensure manual run works, choose the Draft option and mark your pipeline as ready. You can then set up Zoho Flow to trigger your pipeline.

Note: Only one run type can be setup in a pipeline at a time.

  

2. Click the Schedule drop-down icon and click the Zoho Flow option.

 

Zoho Flow Settings 

Stop export if data has invalid values : Enabling this will stop the export when your data still has invalid values.

Order exports  :

This option determines in what order the data has to be exported to destinations when you have configured multiple destinations.

If not enabled, export will run in the default order.

 

Note: This option will be visible only if you have added more than one destination in your pipeline.

To rearrange the order of your export destinations

1) Click the Order exports toggle.

2) You can drag and drop to change the order of the destinations and then click Save.

Note: Click the Edit order link if you want to rearrange the order again. 

Import configuration

3. While configuring the Zoho Flow, the import configuration needs to be mandatorily setup for all the sources. Without setting up the import configuration, the Zoho Flow configuration cannot be saved.

Select the Edit here link to set the import configuration.

 

The import configuration is different for different sources. Click here to know more about how to set up import configuration for various sources.

 

4. After you configure the Zoho Flow configuration, click Save.

Create a new flow in Zoho Flow 

1. Login to your Zoho Flow account. In the My Flows section, click Create Flow in the top right corner, or the + button on the left.

2. Enter the flow name and a description (optional). Click Create to be directed to your builder screen.

 a) Configure the trigger

4. Pick an app to trigger your flow and click Next.

5. Choose the event in the app to trigger your flow and click Next.
Different apps have different types of triggers.

Info: For DataPrep we have only Pipeline job completion trigger.

6. If you have not created any connections already, click Connect and authenticate your connection. Note that different apps have different authentication methods.

7. After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

Note: When renaming the variable,

• All names must start with a letter. Variable names can contain alphanumeric characters and underscores.

• Other characters, such as spaces, are not accepted. Names must be unique to avoid naming clashes.

• If you use the same variable name for multiple actions, the result of the action executed last will be stored in the variable. E.g: new_deal, createTask
  

8. Select the required data from the drop-down and click Done.

Note: When you connect to Zoho DataPrep, the following variables will be provided for filter criteria:

Organization ID

Workspace ID

Pipeline ID

Job executed status

Job completed time in Unix timestamp

Job ID

Creator email ID

Data interval end time in Unix timestamp

Data interval start time in Unix timestamp

Job created time in Unix timestamp

Did the export stop due to invalid data?

Job re-run count

Job type

b) Configure the Actions

Info: For DataPrep we have only Run a pipeline action.

10. In the configuration window that opens, choose the connection and enter the data in the fields that appear.

If you have not created any connections already, click Connect and authenticate your connection. Note that different apps have different authentication methods.

 To connect to DataPrep as an action:

a) Click Connect and give a name to your connection.

b) Click Authorize and accept the pop-up that appears.

c) Zoho DataPrep account gets connected with Zoho Flow.

11. After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

12. Select the Organization ID, Workspace ID, and Pipeline ID from the drop-down and click Done.

Note: You can insert variables to draft your variable name or message according to the app you choose.

DataPrep provides the following variable to be used:

Organization ID	Organization ID of the pipeline.
Workspace ID	Workspace ID of the pipeline.
Pipeline ID	Pipeline ID chosen.
Job executed status	Success or failure status of the job.
Job completed time in Unix timestamp	The total time interval taken for the job.
Job ID	ID of the job executed.
Creator email ID	The email ID of the user who runs the pipeline
Data interval end time in Unix timestamp	The time when the pipeline run was started.
Data interval start time in Unix timestamp	The time when the pipeline run was hit.
Job created time in Unix timestamp	The time when the job was created.
Did the export stop due to invalid data?	If the Stop export if data has invalid values setting is enabled and if the pipeline contains invalid data during export, then yes, if not no.
Job re-run count	The number of times a job was re-run.
Job type	The type of job that triggers the action: Schedule, Manual, Webhooks, Zoho Flow, Backfill.

  

   

 Configure Zoho Flow in Zoho DataPrep

Important: When you add an action to run a pipeline, it must be configured in Zoho DataPrep.

Configure the pipeline with Zoho Flow in DataPrep by navigating to the Pipeline Builder, clicking the Schedule dropdown at the top-right, and selecting Zoho Flow. Click here to know more about how to configure Zoho Flow in DataPrep.

 c) Save the Flow 

13. In Zoho Flow, your flow will be autosaved by default. After you switch on your flow, the draft will be published to live and saved as a version. Further changes made will be saved as a draft. Click Apply changes for those changes to get reflected in the flow.

Note: You won't be able to save your flow if your flow has no actions, or if an action or trigger is not configured completely.

d) Switch on the Flow

Note: Maximum number of trigger-based jobs allowed for a pipeline in an hour is 4.

How does Zoho Flow work?

15. Whenever a new file is uploaded in Zoho Workdrive, Zoho Flow will automatically trigger and run the pipeline configured in Zoho DataPrep.

 

Each Zoho Flow run is saved as a job in DataPrep. When a pipeline is run, the data will be fetched from your data sources, prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination at regular intervals. This complete process is captured in the job history.

 You can view the status of a job with the granular details in the Job summary. Click here to know more about the job summary.

Note: If you make any further changes to the pipeline, the changes are saved as a draft version. Choose the Draft option and mark your pipeline as ready for the changes to reflect in the Zoho Flow run.

After you set your Zoho Flow, you can choose to Edit or Remove using the Zoho Flow Active option in the pipeline builder.

Here is another example for the Flow

1. Connect with pipeline 1 in DataPrep as a trigger

To connect to DataPrep as a trigger

a) Click Connect and give a name to your connection.

b) Click Authorize and accept the pop-up that appears.

c) Zoho DataPrep account gets connected with Zoho Flow.

d) After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

e) Select the Organization ID, Workspace ID, and Pipeline ID from the drop-down.

f) You can also configure the conditions that trigger the flow using the following Filter criteria and click Done.

DataPrep provides the following variable to be used:

After setting pipeline job completion as a trigger (for pipeline 1), set run a pipeline as an action (for pipeline 2) and then set message as Bot on channel as an action.

2. Set "Run a pipeline" as an action

Important: When you add an action to run a pipeline, it must be configured in Zoho DataPrep.

Note: You can insert variables to draft your variable name or message.

DataPrep provides the following variable to be used:

4. Save and run your flow

In Zoho Flow, your flow will be autosaved. Click the slider at the top to switch on your flow in Zoho Flow and let it work with actual data.

When Pipeline 1 completes successfully, Zoho Flow triggers Pipeline 2, and a message is sent to a Zoho Cliq channel.

Important: The action and trigger for a flow should not be the same pipeline, as this would result in the pipeline looping.

SEE ALSO

Pipeline-Backfill run

Pipeline-Manual run

Learn about Jobs in DataPrep

Data sources in Zoho DataPrep

Export destinations in DataPrep