Trigger Pipelines in Zoho DataPrep using Zoho Flow

Zoho DataPrep allows you to trigger your pipeline using Zoho Flow. Zoho Flow is an integration platform which connects cloud applications. It helps you set up workflows to automate information exchange among the apps you use.

To trigger pipelines in Zoho DataPrep using Zoho Flow

For instance, let's use this flow as an example, and walk through how to recreate it.
When a new file is added in Zoho Workdrive, a flow can trigger a pipeline that imports the newly added Workdrive file.

Configure Zoho Flow in Zoho DataPrep

Important: When you add an action to run a pipeline in Zoho Flow, it must be configured in Zoho DataPrep.

To configure Zoho Flow in DataPrep follow the below steps:

1. Once you have finished creating your entire pipeline,

Pro tip: You may want to try executing your pipeline using the Run button. Each run is saved as a job.

When a pipeline run is executed, the data fetched from your data sources will be prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination. This complete process is captured in the job history. Once you ensure manual run works, choose the Draft option and mark your pipeline as ready. You can then set up Zoho Flow to trigger your pipeline.

Note: Only one run type can be setup in a pipeline at a time.

2. Click the Schedule drop-down icon and click the Zoho Flow option.

Zoho Flow Settings

Stop export if data has invalid values : Enabling this will stop the export when your data still has invalid values.

Order exports :

This option determines in what order the data has to be exported to destinations when you have configured multiple destinations.

If not enabled, export will run in the default order.

Note: This option will be visible only if you have added more than one destination in your pipeline.

To rearrange the order of your export destinations

1) Click the Order exports toggle.

2) You can drag and drop to change the order of the destinations and then click Save.

Note: Click the Edit order link if you want to rearrange the order again.

Import configuration

3. While configuring the Zoho Flow, the import configuration needs to be mandatorily setup for all the sources. Without setting up the import configuration, the Zoho Flow configuration cannot be saved.

Select the Edit here link to set the import configuration.

The import configuration is different for different sources. Click here to know more about how to set up import configuration for various sources.

4. After you configure the Zoho Flow configuration, click Save.

Create a new flow in Zoho Flow

1. Login to your Zoho Flow account. In the My Flows section, click Create Flow in the top right corner, or the + button on the left.

2. Enter the flow name and a description (optional). Click Create to be directed to your builder screen.

a) Configure the trigger

3. When you land on the builder, click Configure in the App trigger box. A trigger is the event that starts a flow execution. App triggers when an event occurs in an app.

4. Pick an app to trigger your flow and click Next.

5. Choose the event in the app to trigger your flow and click Next.
Different apps have different types of triggers.

Info: For DataPrep we have only Pipeline job completion trigger.

6. If you have not created any connections already, click Connect and authenticate your connection. Note that different apps have different authentication methods.

7. After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

Note: When renaming the variable,

All names must start with a letter. Variable names can contain alphanumeric characters and underscores.
Other characters, such as spaces, are not accepted. Names must be unique to avoid naming clashes.
If you use the same variable name for multiple actions, the result of the action executed last will be stored in the variable. E.g: new_deal, createTask

8. Select the required data from the drop-down and click Done.

Note: When you connect to Zoho DataPrep, the following variables will be provided for filter criteria. You can use the values in the Values column to configure the filter criteria. For entries marked as N/A, provide the appropriate values as needed.

Variable	Example
Organization ID	N/A
Workspace ID	N/A
Pipeline ID	N/A
Job executed status	FAILURE,RUNNING,SUCCESS,CREATED,SKIPPED
Job completed time in Unix timestamp	N/A
Job ID	N/A
Creator email ID	N/A
Data interval end time in Unix timestamp	N/A
Data interval start time in Unix timestamp	N/A
Job created time in Unix timestamp	N/A
Did the export stop due to invalid data?	TRUE/FALSE
Job re-run count	N/A
Job type	MANUAL,SCHEDULE,RELOAD,BACKFILL,WEBHOOK,ZOHO FLOW

b) Configure the Actions

9. Once you have configured the trigger, you can add multiple actions to a flow to execute sequentially. You will find these actions (app and logic-based) to the left of your builder under the Apps and Logic sections. An action is a task that the workflow executes based on the trigger. Different apps have different types of actions.

Info: For DataPrep we have only Run a pipeline action.

10. In the configuration window that opens, choose the connection and enter the data in the fields that appear.

If you have not created any connections already, click Connect and authenticate your connection. Note that different apps have different authentication methods.

To connect to DataPrep as an action:

a) Click Connect and give a name to your connection.

b) Click Authorize and accept the pop-up that appears.

c) Zoho DataPrep account gets connected with Zoho Flow.

11. After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

12. Select the Organization ID, Workspace ID, and Pipeline ID from the drop-down and click Done.

Note: You can insert variables to draft your variable name or message according to the app you choose.

DataPrep provides the following variables for filter criteria. You can use the values in the Example column to configure the filter criteria. For entries marked as N/A, provide the appropriate values as needed.

Variable	Description	Example
Organization ID	Organization ID of the pipeline.	N/A
Workspace ID	Workspace ID of the pipeline.	N/A
Pipeline ID	Pipeline ID chosen.	N/A
Job executed status	Success or failure status of the job.	FAILURE,RUNNING,SUCCESS,CREATED,SKIPPED
Job completed time in Unix timestamp	The total time interval taken for the job.	N/A
Job ID	ID of the job executed.	N/A
Creator email ID	The email ID of the user who runs the pipeline	N/A
Data interval end time in Unix timestamp	The time when the pipeline run was started.	N/A
Data interval start time in Unix timestamp	The time when the pipeline run was hit.	N/A
Job created time in Unix timestamp	The time when the job was created.	N/A
Did the export stop due to invalid data?	If the Stop export if data has invalid values setting is enabled and if the pipeline contains invalid data during export, then yes, if not no.	TRUE/FALSE
Job re-run count	The number of times a job was re-run.	N/A
Job type	The type of job that triggers the action: Schedule, Manual, Webhooks, Zoho Flow, Backfill.	MANUAL,SCHEDULE,RELOAD,BACKFILL,WEBHOOK,ZOHO FLOW

Configure Zoho Flow in Zoho DataPrep

Important: When you add an action to run a pipeline, it must be configured in Zoho DataPrep.

Configure the pipeline with Zoho Flow in DataPrep by navigating to the Pipeline Builder, clicking the Schedule dropdown at the top-right, and selecting Zoho Flow. Click here to know more about how to configure Zoho Flow in DataPrep.

After you configure the Zoho Flow configuration, click Save.

c) Save the Flow

13. In Zoho Flow, your flow will be autosaved by default. After you switch on your flow, the draft will be published to live and saved as a version. Further changes made will be saved as a draft. Click Apply changes for those changes to get reflected in the flow.

Note: You won't be able to save your flow if your flow has no actions, or if an action or trigger is not configured completely.

d) Switch on the Flow

14. After the Zoho Flow configuration of your pipeline in DataPrep, click the slider at the top to switch on your flow in Zoho Flow and let it work with actual data.

Note: Maximum number of trigger-based jobs allowed for a pipeline in an hour is 4.

How does Zoho Flow work?

15. Whenever a new file is uploaded in Zoho Workdrive, Zoho Flow will automatically trigger and run the pipeline configured in Zoho DataPrep.

Each Zoho Flow run is saved as a job in DataPrep. When a pipeline is run, the data will be fetched from your data sources, prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination at regular intervals. This complete process is captured in the job history.

You can view the status of a job with the granular details in the Job summary. Click here to know more about the job summary.

Note: If you make any further changes to the pipeline, the changes are saved as a draft version. Choose the Draft option and mark your pipeline as ready for the changes to reflect in the Zoho Flow run.

After you set your Zoho Flow, you can choose to Edit or Remove using the Zoho Flow Active option in the pipeline builder.

Sequencing of Pipelines

You can configure your flow as needed using the apps in Zoho Flow. For example, a finance team has two different pipelines in DataPrep. They want to validate expense data in Pipeline 1, which then triggers Pipeline 2 for report generation and notifies in Cliq upon completion.

1. Connect with pipeline 1 in DataPrep as a trigger

To connect to DataPrep as a trigger

a) Click Connect and give a name to your connection.

b) Click Authorize and accept the pop-up that appears.

c) Zoho DataPrep account gets connected with Zoho Flow.

d) After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

e) Select the Organization ID, Workspace ID, and Pipeline ID from the drop-down.

f) You can also configure the conditions that trigger the flow using the following Filter criteria and click Done.

DataPrep provides the following variables for filter criteria. You can use the values in the Example column to configure the filter criteria. For entries marked as N/A, provide the appropriate values as needed.

Variable	Description	Example
Organization ID	Organization ID of the pipeline.	N/A
Workspace ID	Workspace ID of the pipeline.	N/A
Pipeline ID	ID of the Pipeline.	N/A
Job executed status	Success or failure status of the job.	FAILURE,RUNNING,SUCCESS,CREATED,SKIPPED
Job completed time in Unix timestamp	The total time interval taken for the job.	N/A
Job ID	ID of the job executed.	N/A
Creator email ID	The email ID of the user who ran the pipeline	N/A
Data interval end time in Unix timestamp	The time when the pipeline run was started.	N/A
Data interval start time in Unix timestamp	The time when the pipeline run was hit.	N/A
Job created time in Unix timestamp	The time when the job was created.	N/A
Did the export stop due to invalid data?	If the Stop export if data has invalid values setting is enabled and if the pipeline contains invalid data during export, then yes, if not no.	TRUE/FALSE
Job re-run count	The number of times a job was re-run.	N/A
Job type	The type of job that starts the trigger pipeline: Schedule, Manual, Webhook, Backfill,Zoho Flow	MANUAL,SCHEDULE,RELOAD,BACKFILL,WEBHOOK,ZOHO FLOW

After setting pipeline job completion as a trigger (for pipeline 1), set run a pipeline as an action (for pipeline 2) and then set message as Bot on channel as an action.

2. Set "Run a pipeline" as an action

Once you have configured the trigger, you can add Run a pipeline action from Zoho DataPrep located to the left of your builder under the App and Logic sections to a flow to execute sequentially.

Important: When you add an action to run a pipeline, it must be configured in Zoho DataPrep.

Here, pipeline 2 (action) must be configured with Zoho flow configuration. Click here to know how to configure your pipeline in Zoho DataPrep.

3. Set "Message as Bot on channel" as another action.

Note: You can insert variables to draft your variable name or message.

Variable	Description	Example
Organization ID	Organization ID of the pipeline.	N/A
Workspace ID	Workspace ID of the pipeline.	N/A
Pipeline ID	Pipeline ID chosen.	N/A
Job executed status	Success or failure status of the job.	FAILURE,RUNNING,SUCCESS,CREATED,SKIPPED
Job completed time in Unix timestamp	The total time interval taken for the job.	N/A
Job ID	ID of the job executed.	N/A
Creator email ID	The email ID of the user who runs the pipeline	N/A
Data interval end time in Unix timestamp	The time when the pipeline run was started.	N/A
Data interval start time in Unix timestamp	The time when the pipeline run was hit.	N/A
Job created time in Unix timestamp	The time when the job was created.	N/A
Did the export stop due to invalid data?	If the Stop export if data has invalid values setting is enabled and if the pipeline contains invalid data during export, then yes, if not no.	TRUE/FALSE
Job re-run count	The number of times a job was re-run.	N/A
Job type	The type of job that triggers the action: Schedule, Manual, Webhook, Zoho Flow, Backfill.	MANUAL,SCHEDULE,RELOAD,BACKFILL,WEBHOOK,ZOHO FLOW

4. Save and run your flow

In Zoho Flow, your flow will be autosaved. Click the slider at the top to switch on your flow in Zoho Flow and let it work with actual data.

When Pipeline 1 completes successfully, Zoho Flow triggers Pipeline 2, and a message is sent to a Zoho Cliq channel.

Important: The action and trigger for a flow should not be the same pipeline, as this would result in the pipeline looping.

Customize pipeline completion notifications

You can set up a customized flow of notifications to be sent to others who are not users or did not configure schedule. For this you can use Zoho Flow integration to monitor for failures and send notifications to the channels that you would like, this enables you to extend the notification capabilities of Zoho DataPrep to beyond its scope, such as notifying in a slack channel, creating tickets in Zoho Desk, or just an email to someone.

To connect to DataPrep as a trigger

a) Click Connect and give a name to your connection.

b) Click Authorize and accept the pop-up that appears.

c) Zoho DataPrep account gets connected with Zoho Flow.

d) After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

e) Select the Organization ID, Workspace ID, and Pipeline ID from the drop-down.

Variable	Description	Example
Organization ID	Organization ID of the pipeline.	N/A
Workspace ID	Workspace ID of the pipeline.	N/A
Pipeline ID	ID of the Pipeline.	N/A
Job executed status	Success or failure status of the job.	FAILURE,RUNNING,SUCCESS,CREATED,SKIPPED
Job completed time in Unix timestamp	The total time interval taken for the job.	N/A
Job ID	ID of the job executed.	N/A
Creator email ID	The email ID of the user who ran the pipeline	N/A
Data interval end time in Unix timestamp	The time when the pipeline run was started.	N/A
Data interval start time in Unix timestamp	The time when the pipeline run was hit.	N/A
Job created time in Unix timestamp	The time when the job was created.	N/A
Did the export stop due to invalid data?	If the Stop export if data has invalid values setting is enabled and if the pipeline contains invalid data during export, then yes, if not no.	TRUE/FALSE
Job re-run count	The number of times a job was re-run.	N/A
Job type	The type of job that starts the trigger pipeline: Schedule, Manual, Webhook, Backfill,Zoho Flow.	MANUAL,SCHEDULE,RELOAD,BACKFILL,WEBHOOK,ZOHO FLOW

g) For example, if you want to notify when the pipeline job has failed, use the Job executed status filter and enter failure.

h) You can use a decision box for each case, success as well as failure.

i) For example, in case of success, you can notify you team in a Slack or a Cliq channel. In case of failure, directly raise a ticket in Zoho Desk for investigation.

SEE ALSO

Pipeline-Backfill run

Pipeline-Manual run

Learn about Jobs in DataPrep

Data sources in Zoho DataPrep

Export destinations in DataPrep

Create. Review. Publish.

Write, edit, collaborate on, and publish documents to different content management platforms.

Get Started Now

Access your files securely from anywhere

Zoho CRM Training Programs

Learn how to use the best tools for sales force automation and better customer engagement from Zoho's implementation specialists.

Redefine the way you work

with Zoho Workplace

Start your free trial

Zoho DataPrep Personalized Demo

If you'd like a personalized walk-through of our data preparation tool, please request a demo and we'll be happy to show you how to get the best out of Zoho DataPrep.

Create, share, and deliver

beautiful slides from anywhere.

Get Started Now

Zoho Sign now offers specialized one-on-one training for both administrators and developers.

BOOK A SESSION

Zoho Workerly Home
Forums
Connect With Us:

Zoho Recruit Home
Forums
Connect With Us:

Start tracking your website metrics today

Use PageSense to understand what your visitors are looking for, how they are behaving, and where they are facing problems on your website.

Ready to improve your website's performance?

Install the PageSense code snippet on your site in a matter of minutes and start collecting in-depth data about the website visitors to grow your business.

Track and measure every business goal

Set up goals in PageSense to measure every single action performed by visitors on your website like button or link clicks, form submissions, and page engagements.

Understand the customer journey on your website

Create funnels in PageSense to quickly see which pages visitors use to enter your website, where they navigate to next, and which pages they decide to leave without converting.

Visualize your visitor's behavior with color codes

Set up heatmaps in PageSense to see where users have clicked more, how far they've scrolled, and on which parts of a page they've spent the most time using color-coded patterns in reports.

Measure customer engagement with your web forms

Use form analytics in PageSense to see how people interact with different fields in your form, whether they complete the form successfully or not, and where exactly they drop out on your form.

Record real visitor interactions on your website

Use session recordings in PageSense to watch a video of all the visitor actions performed on your website including the pages they navigate, the buttons they click, the UX issues they face, and more.

Test and optimize webpages for better conversions

Run A/B or Split URL tests in PageSense to figure out which version of your web page works best for your business and results in the best conversion rate.

Give each customer a unique website experience

Use personalization in PageSense to deliver customized versions of your website for every individual customer based on their demographics, local weather, browsing history, and more.

Grow your business through customer feedback

Run polls on your website using PageSense to understand what your customers think about your products/services and what needs improvement on your site.

Send timely updates that customers love

Use web push notifications in PageSense to schedule and notify your customers about an upcoming flash sale, product releases, promotional coupons, and a lot more that can spark conversions on your website.

Advertise your business to website visitors

Use pop-ups in PageSense to instantly grab the attention of visitors by showing attractive signup offers, coupon code discounts, or email newsletters that can eventually convert them into subscribers.

Access more advanced settings in PageSense

Use PageSense's advanced features like creating mutually exclusive groups, enabling cross-domain tracking, configuring customized project JS, and more to get deeper insights about your website.

Try easy-to-use extensions

Download the PageSense extension app available for your web browser with a few clicks and start collecting all of your required website metrics in real time.

Discover your favorite integrations with PageSense

Get a deeper look at your website's data by seamlessly integrating PageSense with a host of popular third-party apps like Google Analytics, Mixpanel, Intercom, and more.

Quick Links	Workflow Automation	Data Collection
Web Forms	Retail	Online Data Collection Tool
Embeddable Forms	Banking	Begin Data Collection
Interactive Forms	Workplace	Data Collection App
CRM Forms	Customer Service	Forms for Solopreneurs
Digital Forms	Marketing	Forms for Small Business
HTML Forms	Education	Forms for Enterprise
Contact Forms	E-commerce	Forms for any business
Lead Generation Forms	Healthcare	Forms for Startups
Wordpress Forms	Customer onboarding	Forms for Small Business
No Code Forms	Construction	RSVP tool for holidays
Free Forms	Travel	Features for Order Forms
Prefill Forms	Non-Profit	Forms for Government
Intake Forms	Legal	Mobile App
Form Designer	HR	Mobile Forms
Card Forms	Food	Offline Forms
Assign Forms	Photography	Mobile Forms Features
Translate Forms	Real Estate	Kiosk in Mobile Forms
Electronic Forms	Insurance
Drag & drop form builder
Notification Emails for Forms	Alternatives	Security & Compliance
Holiday Forms	Google Forms alternative	GDPR
Form to PDF	Jotform alternative	HIPAA Forms
Email Forms	Wufoo alternative	Encrypted Forms
Accessible Forms	Typeform alternative	Secure Forms
		WCAG

Quick Links

New to Zoho Writer?

Signup Now

Zoho Writer

What's new

Help Guide

Blogs

Forums

Webinars

API

Developers

Build Extensions

Create. Review. Publish.

Write, edit, collaborate on, and publish documents to different content management platforms.

Get Started Now

Zoho Campaigns Resources

Campaigns Community Learning Series

New to Zoho CRM Plus?

Deliver unforgettable customer experiences

ACCESS ZOHO CRM PLUS

New to Zoho CRM Plus?

Deliver unforgettable customer experiences

ACCESS ZOHO CRM PLUS

New to Zoho Marketing Plus?

Everything you need to run your marketing

ACCESS ZOHO MARKETING PLUS

New to Zoho Marketing Plus?

Everything you need to run your marketing

ACCESS ZOHO MARKETING PLUS

Zoho Pagesense Resources

You are currently viewing the help pages of Qntrl’s earlier version. Click here to view our latest version—Qntrl 3.0's help articles.

New to Zoho Survey?

Access Zoho Survey

Latest Feature Updates

Explore our Pricing & Plans

Zoho Survey Resources

Android Mobile Surveys

Insurvey Blogs

Insight Blogs

Tips & Tricks

New to Zoho Social?

Manage your brands on social media

Access Zoho Social

New to Zoho Desk?

Access Zoho Desk

What's New in Zoho Desk

Use cases

Make the most of Zoho Desk with the use cases.

Learn More

eBooks

Download free eBooks and access a range of topics to get deeper insight on successfully using Zoho Desk.

Download eBooks

Videos

Watch comprehensive videos on features and other important topics that will help you master Zoho Desk.

Watch Now

Webinar

What's New in Zoho Desk

Desk Community Learning Series
Meetups
Ask the Experts
Kbase
Resources
Glossary
Desk Marketplace
MVP Corner

Zoho Sheet Resources

New to Zoho Forms?

Latest Feature Updates

Explore our Pricing & Plans

Zoho Forms Resources

Secure your business

communication with Zoho Mail

Get started

Mail on the move with

Zoho Mail mobile application

Go Mobile

Stay on top of your schedule

at all times

Get started

Carry your calendar with you

Anytime, anywhere

Get started

Latest Announcements

New to Zoho Sign?

Zoho Sign Resources

Sign, Paperless!

Sign and send business documents on the go!

Get Started Now

Zoho SalesIQ Resources

New to Zoho TeamInbox?

Zoho TeamInbox Resources

Connect with us

New to Zoho ZeptoMail?

LEARN MORE

Latest Feature Updates

Zoho DataPrep Resources

New to Zoho DataPrep?

Access Zoho DataPrep

Zoho DataPrep Demo

Get a personalized demo or POC

Design. Discuss. Deliver.

Create visually engaging stories with Zoho Show.

Get Started Now

New to Zoho Workerly?

Access Zoho Workerly

New to Zoho Recruit?

Access Zoho Recruit

New to Zoho CRM?

Signup Now

Access Zoho CRM

Latest Feature Updates

New to Zoho Projects?

Access Zoho Projects

New to Zoho Sprints?

Access Zoho Sprints

New to Zoho Assist?

Access Zoho Assist

New to Bigin?

Signup Now

Access Bigin

Latest Feature Updates

Related Articles
Zoho DataPrep's Home page
Dashboard Getting started The getting started section in the home page is shown to trial users, this shows you the various aspects to get started with Zoho DataPrep. You can take a Product tour to get a walkthrough and understand the end-to-end ETL ...
White labeled Zoho DataPrep
White labeled Zoho DataPrep makes ETL and data preparation implementation easy for independent software vendors, solution developers, business consultants, etc. by providing integrated solution into their product. You can take a look at the features ...
Ruleset in Zoho DataPrep
This page covers the following sections: What is a Ruleset? Data source settings Ruleset operations Options with individual rules Ruleset Templates What is a Ruleset? Each transform applied on the data is stored in DataPrep as a rule, in order of ...
Encryption at Zoho DataPrep
Encryption is primarily used to safeguard the contents of a message so that only the intended recipient could read it. This is done by replacing the contents with unrecognizable data, which could be understood only by the intended recipient. This is ...
White labelled Zoho DataPrep vs Standalone Zoho DataPrep
White Labelled Zoho DataPrep makes data preparation easy for independent software vendors, solution developers, business consultants, etc by providing integrated solution into their product. It streamlines ETL processes, simplifies data movement, and ...