1. Open an existing pipeline or create a pipeline from the Home Page, Pipelines tab or Workspaces tab and click the Add data option.

Info: You can also click the Import data  icon at the top of the pipeline builder to bring data from multiple sources into the pipeline.

Import configuration

Import all data  

Schedule settings

Stop export if data has invalid values: Enabling this will stop the export when prepared data still has invalid values.

Order exports

To rearrange the order of your export destinations

1) Click the Order exports toggle.

8. After you configure the schedule configuration, click Save to execute the schedule. This will start the ETL pipeline.

Each scheduled run is saved as a job. When a pipeline is scheduled, the data will be fetched from your data sources, prepared using the series of transforms you have applied in each of the stages, and then data will be exported to your destination through seamless data integration at regular intervals. This complete process is captured in the job history.

9. To go to the jobs list of a particular pipeline, go to the  ellipses icon in the pipeline builder, and click on the Job history menu to check the job status of your pipeline.

10. Click the required job ID in the Jobs history page to navigate to the Job summary of a particular job.

The Job summary shows the history of a job executed in a pipeline flow. Click here to know more.

Note: If you make any further changes to the pipeline, the changes are saved as a draft version. Choose the Draft option and mark your pipeline as ready for the changes to reflect in the schedule.

After you set your schedule, you can choose to Pause schedule or Resume schedule, Edit schedule and Remove schedule using the Schedule Active option in the pipeline builder.

When you edit and save a schedule, the next job will be from the last schedule run time to the next scheduled data interval.

For the Pipedrive connector, incremental import is performed on a page-wise basis, with each page containing a maximum of 500 records.

For example:

• If the Deals module has 510 total records and 300 have been updated, a single page containing 500 records will be fetched.

• If there are 900 total records and 600 have been updated, two pages will be imported - one with 500 records and another with the remaining 400 - based on the total record count.

This means the import is based on total record count per page, not strictly on the number of updated records.

Troubleshooting FAQs

1. What should I do if I see a "Something went wrong!" error or a blank screen during Pipedrive authentication?