Pipeline sectional run
When data propagation happens in a pipeline, the changes made in the upstream stages are loaded to the current stage that you're working on. This runs a section of a pipeline and therefore called Sectional runs.
What is data propagation?
Data propagation is the process of loading any changes made in the upstream stages to the current stage that you're working on. Let's say you make some changes rule additions or edits in a parent stage(s), those changes will be propagated to the dependent child stage(s).
In DataPrep, data propagation is processed either manually or automatically based on your preference in the Settings tab. Learn more about data propagation.
Note: Auto or manual data propagation where changes are loaded from the upstream are counted towards rows processed consumption.
To execute sectional run in a pipeline
1. Open your pipeline and go to the pipeline builder view. Click here if you want to know how to create a pipeline in the first place.
2. Click on the Add data option to bring your data from the required data source. Learn more on 50+ data sources.
After data is imported, you will be redirected to the pipeline builder where you can see your data source, and a stage linked to it.
Stages are nodes created for processing data while applying data flow transforms. Every dataset imported from your data source will have a stage created by default.
3. You can right-click the stage and apply the data flow transforms.
4. You can also choose to Prepare Data to open the DataPrep Studio page and explore more data transforms. Click here to know the various elements in a pipeline builder.
Trigger sectional run
A sectional run is automatically triggered when data propagation happens in a pipeline and doesn't require a manual trigger. Let's look into the below example to understand this.
Example
5. I have Leads data in the first stage. Now I would like to view the leads generated during the first quarter and create a report. But I would still need all data later for future. So I'm going to create another stage from my current stage. This will create a copy of my current stage. To do this, right-click on the stage and choose the Add stage option. A new child stage gets created. In the child stage, you can filter the required data using the Filters transform.
6. Now I have made some updates in the contact information for a few leads in the parent stage and would like the same changes in the child stage.
7. Open the child stage and you will find the below screen if you had enabled the Preferences toggle in the Settings tab. The data changes from the upstream will start propagating automatically.
Note: If you had disabled the Preferences toggle in the Settings tab, you will see the below pop-up and the sectional run will be triggered when you propagate data manually.
8. This will initiate the sectional run and execute the pipeline. To view the run details, go to the Job history page and enable the Include Sectional Jobs toggle. This will show the list of sectional runs.
You can click open specific jobs to view more details such as start and end time, duration of the job, etc. in the Jobs summary page. The Job summary shows the history of a job executed in a pipeline flow. Click here to know more about Jobs and Job summary. You can view the status of the run on the Job summary page. There are three different status for a job in DataPrep - Initiated, Success and Failure.
To export data, you can right-click a stage, add a destination and run the pipeline. Here's the different types of runs in DataPrep
In the Overview tab, you can view the status of the job along with the details such as user who started the job, storage used, total rows processed, start time, end time and duration of the job. Click here to know more.
In the Stages tab, you can view the details of each pipeline stage such as Import, Transform and Export. Click here to know more.
9. Once you are done updating your data, you can go back to the Pipeline builder page, right-click a stage and add a destination.
Data destination is a place where you want to export your data to. It can be your local database, business applications like Zoho CRM, Zoho Analytics, etc. You can choose your preferred destination out of 50+ data destinations in DataPrep to export your prepared data
10. After adding a destination for your pipeline, click the Run drop down and choose a run type and execute the pipeline. Runs types in DataPrep.
Important: A sectional run will not export data to your destination. Please make sure to manually run the pipeline to get your data exported.
Access your files securely from anywhere
Zoho CRM Training Programs
Learn how to use the best tools for sales force automation and better customer engagement from Zoho's implementation specialists.
Zoho DataPrep Personalized Demo
If you'd like a personalized walk-through of our data preparation tool, please request a demo and we'll be happy to show you how to get the best out of Zoho DataPrep.
- Zoho Workerly Home
- Forums
- Connect With Us:
- Zoho Recruit Home
- Forums
- Connect With Us:
Centralize Knowledge. Transform Learning.
All-in-one knowledge management and training platform for your employees and customers.
New to Zoho Writer?
Create. Review. Publish.
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho CRM Plus?
Deliver unforgettable customer experiences
New to Zoho Marketing Plus?
Everything you need to run your marketing
New to Zoho Marketing Plus?
Everything you need to run your marketing
You are currently viewing the help pages of Qntrl’s earlier version. Click here to view our latest version—Qntrl 3.0's help articles.
New to Zoho Survey?
Zoho Sheet Resources
Zoho Forms Resources
New to Zoho Sign?
Zoho Sign Resources
Sign, Paperless!
New to Zoho TeamInbox?
Zoho TeamInbox Resources
New to Zoho ZeptoMail?
Design. Discuss. Deliver.
New to Zoho Workerly?
New to Zoho Recruit?
New to Zoho CRM?
New to Zoho Projects?
New to Zoho Sprints?
New to Zoho Assist?
New to Bigin?
Related Articles
Pipeline Manual run
By triggering the manual run option in your pipeline, you can export the data you have prepared to your destination. A manual run will run your pipeline on the existing data without refreshing, which means no new data will be imported. Note: You can ...Create a new pipeline
Pipelines are fundamental entities that allow users to create ETL pipelines / data flows in DataPrep, with multiple data stages and using various flow‑level transforms. You can create a new pipeline from scratch (for your data integration and data ...Pipeline Backfill run
Backfill run can process data that was missed in previous schedules due to a change in data models or data preparation workflow. This way, you can catch up with missed schedules or if you want to import specific data within a timeframe data, or to ...Pipeline
Pipelines are fundamental entities that allow users to create data flows in DataPrep, with multiple data stages and various flow-level transforms. To view the list of pipelines, you can click the Pipeline tab from the top bar. The Pipelines tab shows ...Import pipeline from .zdpt file
Pipelines are fundamental entities that allow users to create ETL pipelines / data flows in DataPrep, with multiple data stages and using various flow‑level transforms. You can create a pipeline by importing a saved pipeline template from a ...
New to Zoho LandingPage?
Zoho LandingPage Resources