Import configuration for local files

Import configuration for local files

You can import data from local files using Zoho Databridge. Look into the following sections to learn about the import configuration of each selection type.

Import configuration for local files without using Databridge

While configuring the Backfill run and Manual reload, the import configuration needs to be mandatorily setup for all the sources. Without setting up the import configuration, the run cannot be saved.

Below is a snapshot from the backfill configuration.


 

How to import data from source? You can choose to Import source file, or Do not import data using this option.

 

Import source file


This option will import the source file during every run.



NotesNote: Import configuration is not applicable when File is the data source for Schedule, Webhook, and Zoho flow. But for Backfill and Manual reload, File upload 
option is provided for during import configuration. You can use the Parse file as option to parse unknown file formats.


Do not import data  

The data is imported only once. The second time, the rules get applied to the same data and get exported.  

 


Import configuration for local files using Databridge

If you import data from local network, you can configure how to import and fetch incremental data from your local network using the below Import configuration options.

You can select the Click here link to set the import configuration.

Below is a snapshot from the backfill configuration.


 


How to import data from source?

Select the way you would like to import your data from the drop-down - Import all data, Incremental file fetch, Do not import data.

 Import all data   

This option will import all available data that matches the file pattern.

 


 

File batch size: Specify the batch size when you choose to import all files. Files will be grouped by this number, based on their first modified time. These files will be imported as a single file and exported to the destination in every run.

Info
Info: File batch size cannot be greater than 10. This option is available only when you import data from local network

 Incremental file fetch   

You can configure how to import and fetch incremental data from your local network using the Import configuration option. Incremental data import is a method used to import new or modified records after the previous sync. 


 



 

Use the previously imported data if no new data is available: 

During incremental import,

  1. If the checkbox is checked: When there is no new data in the source, the last fetched data will be imported again.
  2. If the checkbox is unchecked: When there is no new data in the source, the import will fail and no files will be imported. This will, in turn, cause the entire pipeline job to fail.

 

Which file to import? You can choose to import All files, Newest file, or Oldest file using this option.

 All files   

This option will import all the files that match the given file pattern only in the specific data interval based on the Fetch based on field.


Fetch based on: You can choose to select the time based on which the file must be sorted and imported - Modified time, Created time.

NotesNote: Google Drive, OneDrive, SharePoint, Box has modifiedTime and createdTime.
DropBox, Amazon S3 has only modifiedTime (There is no created time)

File batch size: Specify the batch size when you choose to import all files incrementally. Files will be grouped by this number, based on their first created or modified time in that specific data interval. These files will be imported as a single file and exported to the destination in every run.

InfoInfo: File batch size cannot be greater than 10. This option is available only when you import data from local network.
 

 Newest file   

This option will import the latest file that matches the given file pattern in the specific data interval based on the Fetch based on field.



 Oldest file   

This option will import the oldest file that matches the given file pattern in the specific data interval based on the Fetch based on field.

 


 Do not import data   

The data is imported only once. The second time, the rules get applied to the same data and get exported.

 




SEE ALSO