Trigger pipelines in Zoho DataPrep using Zoho Flow

Trigger Pipelines in Zoho DataPrep using Zoho Flow

Zoho DataPrep allows you to trigger your pipeline using Zoho Flow. Zoho Flow is an integration platform which connects cloud applications. It helps you set up workflows to automate information exchange among the apps you use.

To trigger pipelines in Zoho DataPrep using Zoho Flow

  1. Configure the pipeline with Zoho Flow in Zoho DataPrep
  2. Create a new flow in Zoho Flow
  3. Configure the trigger in Zoho Flow
  4. Configure the action in Zoho Flow
  5. Save the flow
  6. Switch on the flow


For instance, let's use this flow as an example, and walk through how to recreate it.

When a new file is added in Zoho Workdrive, a flow can trigger a pipeline that imports the newly added Workdrive file.




Configure Zoho Flow in Zoho DataPrep

Warning

Important: When you add an action to run a pipeline in Zoho Flow, it must be configured in Zoho DataPrep.

To configure Zoho Flow in DataPrep follow the below steps:

1. Once you have finished creating your entire pipeline,

IdeaPro tip: You may want to try executing your pipeline using the Run button. Each run is saved as a job.

When a pipeline run is executed, the data fetched from your data sources will be prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination. This complete process is captured in the job history. Once you ensure manual run works, choose the Draft option and mark your pipeline as ready. You can then set up Zoho Flow to trigger your pipeline.

Notes
Note: Only one run type can be setup in a pipeline at a time.


  

2. Click the Schedule drop-down icon and click the Zoho Flow option.


 

Zoho Flow Settings 


Stop export if data has invalid values
 : Enabling this will stop the export when your data still has invalid values.


Order exports  :

This option determines in what order the data has to be exported to destinations when you have configured multiple destinations.

If not enabled, export will run in the default order.

 

Note: This option will be visible only if you have added more than one destination in your pipeline.


To rearrange the order of your export destinations

1) Click the Order exports toggle.

2) You can drag and drop to change the order of the destinations and then click Save.



NotesNote: Click the Edit order link if you want to rearrange the order again. 

Import configuration

3. While configuring the Zoho Flow, the import configuration needs to be mandatorily setup for all the sources. Without setting up the import configuration, the Zoho Flow configuration cannot be saved.

Select the Edit here link to set the import configuration.

 

The import configuration is different for different sources. Click here to know more about how to set up import configuration for various sources.

 

4. After you configure the Zoho Flow configuration, click Save.

Create a new flow in Zoho Flow 

1. Login to your Zoho Flow account. In the My Flows section, click Create Flow in the top right corner, or the + button on the left.


2. Enter the flow name and a description (optional). Click Create to be directed to your builder screen.


 a) Configure the trigger

3.  When you land on the builder, click Configure in the App trigger box. A trigger is the event that starts a flow execution. App triggers when an event occurs in an app.

4. Pick an app to trigger your flow and click Next.

5. Choose the event in the app to trigger your flow and click Next.
Different apps have different types of triggers.



Info

Info: For DataPrep we have only Pipeline job completion trigger.



6. If you have not created any connections already, click Connect and authenticate your connection. Note that different apps have different authentication methods.

7. After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

Notes

Note: When renaming the variable,

    • All names must start with a letter. Variable names can contain alphanumeric characters and underscores.

    • Other characters, such as spaces, are not accepted. Names must be unique to avoid naming clashes.

    • If you use the same variable name for multiple actions, the result of the action executed last will be stored in the variable. E.g: new_deal, createTask


8. Select the required data from the drop-down and click Done.



Notes

Note: When you connect to Zoho DataPrep, the following variables will be provided for filter criteria:

Organization ID

Workspace ID

Pipeline ID

Job executed status

Job completed time in Unix timestamp

Job ID

Creator email ID

Data interval end time in Unix timestamp

Data interval start time in Unix timestamp

Job created time in Unix timestamp

Did the export stop due to invalid data?

Job re-run count

Job type

   

b) Configure the Actions

9. Once you have configured the trigger, you can add multiple actions to a flow to execute sequentially. You will find these actions (app and logic-based) to the left of your builder under the Apps and Logic sections. An action is a task that the workflow executes based on the trigger. Different apps have different types of actions.
InfoInfo: For DataPrep we have only Run a pipeline action.


10. In the configuration window that opens, choose the connection and enter the data in the fields that appear.

If you have not created any connections already, click Connect and authenticate your connection. Note that different apps have different authentication methods.

 To connect to DataPrep as an action:

a) Click Connect and give a name to your connection.


b) Click Authorize and accept the pop-up that appears.

c) Zoho DataPrep account gets connected with Zoho Flow.


 

11. After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

12. Select the Organization ID, Workspace ID, and Pipeline ID from the drop-down and click Done.

Notes

Note: You can insert variables to draft your variable name or message according to the app you choose.

DataPrep provides the following variable to be used:

Organization ID

Organization ID of the pipeline.

Workspace ID

Workspace ID of the pipeline.

Pipeline ID

Pipeline ID chosen.

Job executed status

Success or failure status of the job.

Job completed time in Unix timestamp

The total time interval taken for the job.

Job ID

ID of the job executed.

Creator email ID

The email ID of the user who runs the pipeline

Data interval end time in Unix timestamp

The time when the pipeline run was started.

Data interval start time in Unix timestamp

The time when the pipeline run was hit.

Job created time in Unix timestamp

The time when the job was created.

Did the export stop due to invalid data?

If the Stop export if data has invalid values setting is enabled and if the pipeline contains invalid data during export, then yes, if not no.

Job re-run count

The number of times a job was re-run.

Job type

The type of job that triggers the action: Schedule, Manual, Webhooks, Zoho Flow, Backfill.

  

   


 Configure Zoho Flow in Zoho DataPrep

Warning

Important: When you add an action to run a pipeline, it must be configured in Zoho DataPrep.

Configure the pipeline with Zoho Flow in DataPrep by navigating to the Pipeline Builder, clicking the Schedule dropdown at the top-right, and selecting Zoho Flow. Click here to know more about how to configure Zoho Flow in DataPrep.

 After you configure the Zoho Flow configuration, click Save.

  

 c) Save the Flow 


13. In Zoho Flow, your flow will be autosaved by default. After you switch on your flow, the draft will be published to live and saved as a version. Further changes made will be saved as a draft. Click Apply changes for those changes to get reflected in the flow.

Notes
Note: You won't be able to save your flow if your flow has no actions, or if an action or trigger is not configured completely.



d) Switch on the Flow

14. After the Zoho Flow configuration of your pipeline in DataPrep, click the slider at the top to switch on your flow in Zoho Flow and let it work with actual data.



Notes
Note: Maximum number of trigger-based jobs allowed for a pipeline in an hour is 4.

How does Zoho Flow work?

15. Whenever a new file is uploaded in Zoho Workdrive, Zoho Flow will automatically trigger and run the pipeline configured in Zoho DataPrep.

 

Each Zoho Flow run is saved as a job in DataPrep. When a pipeline is run, the data will be fetched from your data sources, prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination at regular intervals. This complete process is captured in the job history.

 You can view the status of a job with the granular details in the Job summary. 
Click here to know more about the job summary.



NotesNote: If you make any further changes to the pipeline, the changes are saved as a draft version. Choose the Draft option and mark your pipeline as ready for the changes to reflect in the Zoho Flow run.

After you set your Zoho Flow, you can choose to Edit or Remove using the Zoho Flow Active option in the pipeline builder.


Here is another example for the Flow

You can configure your flow as needed using the apps in Zoho Flow. For example, a finance team has two different pipelines in DataPrep. They want to validate expense data in Pipeline 1, which then triggers Pipeline 2 for report generation and notifies in Cliq upon completion.



1. Connect with pipeline 1 in DataPrep as a trigger

To connect to DataPrep as a trigger

a) Click Connect and give a name to your connection.


b) Click Authorize and accept the pop-up that appears.

c) Zoho DataPrep account gets connected with Zoho Flow.

 

d) After configuring a connection, the variable name will be auto filled. You can edit the name if needed.

e) Select the Organization ID, Workspace ID, and Pipeline ID from the drop-down.

f) You can also configure the conditions that trigger the flow using the following Filter criteria and click Done.

DataPrep provides the following variable to be used:

Organization ID

Organization ID of the pipeline.

Workspace ID

Workspace ID of the pipeline.

Pipeline ID

ID of the Pipeline.

Job executed status

Success or failure status of the job.

Job completed time in Unix timestamp

The total time interval taken for the job.

Job ID

ID of the job executed.

Creator email ID

The email ID of the user who ran the pipeline

Data interval end time in Unix timestamp

The time when the pipeline run was started.

Data interval start time in Unix timestamp

The time when the pipeline run was hit.

Job created time in Unix timestamp

The time when the job was created.

Did the export stop due to invalid data?

If the Stop export if data has invalid values setting is enabled and if the pipeline contains invalid data during export, then yes, if not no.

Job re-run count

The number of times a job was re-run.

Job type

The type of job that starts the trigger pipeline: Schedule, Manual, Webhook, Backfill.


After setting pipeline job completion as a trigger (for pipeline 1), set run a pipeline as an action (for pipeline 2) and then set message as Bot on channel as an action.

2. Set "Run a pipeline" as an action

 Once you have configured the trigger, you can add Run a pipeline action from Zoho DataPrep located to the left of your builder under the App and Logic sections to a flow to execute sequentially.

Warning
Important: When you add an action to run a pipeline, it must be configured in Zoho DataPrep.



Here, pipeline 2 (action) must be configured with Zoho flow configuration. Click here to know how to configure your pipeline in Zoho DataPrep.

3. Set "Message as Bot on channel" as another action.



Note: You can insert variables to draft your variable name or message.

DataPrep provides the following variable to be used:

Organization ID

Organization ID of the pipeline.

Workspace ID

Workspace ID of the pipeline.

Pipeline ID

Pipeline ID chosen.

Job executed status

Success or failure status of the job.

Job completed time in Unix timestamp

The total time interval taken for the job.

Job ID

ID of the job executed.

Creator email ID

The email ID of the user who runs the pipeline

Data interval end time in Unix timestamp

The time when the pipeline run was started.

Data interval start time in Unix timestamp

The time when the pipeline run was hit.

Job created time in Unix timestamp

The time when the job was created.

Did the export stop due to invalid data?

If the Stop export if data has invalid values setting is enabled and if the pipeline contains invalid data during export, then yes, if not no.

Job re-run count

The number of times a job was re-run.

Job type

The type of job that triggers the action: Schedule, Manual, Webhook, Zoho Flow, Backfill.




4. Save and run your flow

In Zoho Flow, your flow will be autosaved. Click the slider at the top to switch on your flow in Zoho Flow and let it work with actual data.


When Pipeline 1 completes successfully, Zoho Flow triggers Pipeline 2, and a message is sent to a Zoho Cliq channel.



Warning

Important: The action and trigger for a flow should not be the same pipeline, as this would result in the pipeline looping.


SEE ALSO

Pipeline-Backfill run

Pipeline-Manual run

Learn about Jobs in DataPrep

Data sources in Zoho DataPrep

Export destinations in DataPrep


    Access your files securely from anywhere

      Zoho CRM Training Programs

      Learn how to use the best tools for sales force automation and better customer engagement from Zoho's implementation specialists.

      Zoho CRM Training
        Redefine the way you work
        with Zoho Workplace

          Zoho DataPrep Personalized Demo

          If you'd like a personalized walk-through of our data preparation tool, please request a demo and we'll be happy to show you how to get the best out of Zoho DataPrep.

          Zoho CRM Training

            Create, share, and deliver

            beautiful slides from anywhere.

            Get Started Now


              Zoho Sign now offers specialized one-on-one training for both administrators and developers.

              BOOK A SESSION





                          Quick LinksWorkflow AutomationData Collection
                          Web FormsEnterpriseBegin Data Collection
                          Interactive FormsWorkplaceData Collection App
                          CRM FormsCustomer ServiceAccessible Forms
                          Digital FormsMarketingForms for Small Business
                          HTML FormsEducationForms for Enterprise
                          Contact FormsE-commerceForms for any business
                          Lead Generation FormsHealthcareForms for Startups
                          Wordpress FormsCustomer onboardingOrder Forms for Small Business
                          No Code FormsConstructionRSVP tool for holidays
                          Free FormsTravel
                          Prefill FormsNon-Profit

                          Intake FormsLegal
                          Mobile App
                          Form DesignerHR
                          Mobile Forms
                          Card FormsFoodOffline Forms
                          Assign FormsPhotography
                          Mobile Forms Features
                          Translate FormsReal EstateKiosk in Mobile Forms
                          Electronic Forms

                          Notification Emails for FormsAlternativesSecurity & Compliance
                          Holiday FormsGoogle Forms alternative GDPR
                          Form to PDFJotform alternativeHIPAA Forms
                          Email Forms
                          Encrypted Forms
                          Embeddable Forms
                          Secure Forms
                          Drag & drop form builder
                          WCAG



                                            You are currently viewing the help pages of Qntrl’s earlier version. Click here to view our latest version—Qntrl 3.0's help articles.




                                                Manage your brands on social media


                                                  • Desk Community Learning Series


                                                  • Digest


                                                  • Functions


                                                  • Meetups


                                                  • Kbase


                                                  • Resources


                                                  • Glossary


                                                  • Desk Marketplace


                                                  • MVP Corner


                                                  • Word of the Day


                                                  • Ask the Experts


                                                    Zoho Marketing Automation

                                                      Zoho Sheet Resources

                                                       

                                                          Zoho Forms Resources


                                                            Secure your business
                                                            communication with Zoho Mail


                                                            Mail on the move with
                                                            Zoho Mail mobile application

                                                              Stay on top of your schedule
                                                              at all times


                                                              Carry your calendar with you
                                                              Anytime, anywhere




                                                                    Zoho Sign Resources

                                                                      Sign, Paperless!

                                                                      Sign and send business documents on the go!

                                                                      Get Started Now




                                                                              Zoho TeamInbox Resources



                                                                                      Zoho DataPrep Resources



                                                                                        Zoho DataPrep Demo

                                                                                        Get a personalized demo or POC

                                                                                        REGISTER NOW


                                                                                          Design. Discuss. Deliver.

                                                                                          Create visually engaging stories with Zoho Show.

                                                                                          Get Started Now









                                                                                                              • Related Articles

                                                                                                              • Zoho DataPrep - FAQs

                                                                                                                1. What is Zoho DataPrep? Zoho DataPrep is an advanced self-service data preparation tool that helps organizations model, cleanse, prepare, enrich and organize large volumes of data from multiple data sources to serve data analytics and data ...
                                                                                                              • Zoho DataPrep's Home page

                                                                                                                Dashboard Getting started The getting started section in the home page is shown to trial users, this shows you the various aspects to get started with Zoho DataPrep. You can take a Product tour to get a walkthrough and understand the end-to-end ...
                                                                                                              • Ruleset in Zoho DataPrep

                                                                                                                This page covers the following sections: What is a Ruleset? Data source settings Ruleset operations Options with individual rules Ruleset Templates What is a Ruleset? Each transform applied on the data is stored in DataPrep as a rule, in order of ...
                                                                                                              • Encryption at Zoho DataPrep

                                                                                                                Encryption is primarily used to safeguard the contents of a message so that only the intended recipient could read it. This is done by replacing the contents with unrecognizable data, which could be understood only by the intended recipient. This is ...
                                                                                                              • White labelled Zoho DataPrep vs Standalone Zoho DataPrep

                                                                                                                White Labelled Zoho DataPrep makes data preparation easy for independent software vendors, solution developers, business consultants, etc by providing integrated solution into their product. You can get in touch with our support team to get a white ...
                                                                                                                Wherever you are is as good as
                                                                                                                your workplace

                                                                                                                  Resources

                                                                                                                  Videos

                                                                                                                  Watch comprehensive videos on features and other important topics that will help you master Zoho CRM.



                                                                                                                  eBooks

                                                                                                                  Download free eBooks and access a range of topics to get deeper insight on successfully using Zoho CRM.



                                                                                                                  Webinars

                                                                                                                  Sign up for our webinars and learn the Zoho CRM basics, from customization to sales force automation and more.



                                                                                                                  CRM Tips

                                                                                                                  Make the most of Zoho CRM with these useful tips.



                                                                                                                    Zoho Show Resources