1. Click the Advanced selection link.

Advanced selection helps you perform dynamic file selection based on regex. This can be used for getting new or incremental data from your Google Drive folder. The newly added or modified file that matches the file pattern during the specific data interval will be taken from your Google Drive folder. Click here to know more about incremental fetch.

Important: Advanced selection is used not only for incremental fetch. You can also use this option for bulk import of files based on the file pattern. 

2. Provide the following details :

• Choose folder : Choose the folder you want to import data from.

• Folder path : The folder path where you want to search for files. Eg. 2023/ 
  
  If the files are stored in the drive without any folder, you can leave this field empty.

  

  The "Folder Path" is a directory path field and does not support regex patterns.
  Info: Folder path is case-sensitive.

• Include subfolders : You can also select the Include subfolders checkbox if you want to include subfolders while searching for a file. Click here to know the limitations of this option.
  

• File pattern : The pattern used to match the file names in the folder. This supports regex type matching. You can also use the pattern, ".*" to match any file in the path specified.
  
  
  
  Info: File pattern is case-sensitive.

Note: The file pattern match is a simple regex type match. For example, to fetch files with file names such as Sales_2022.csv , Sales_2023.csv , Sales_2024.csv , you can input the pattern Sales_.* 

Similarly to fetch files such as PublicData1.csv , PublicData2.csv , PublicData3.csv , use Public.*

If you want to import a single file, then specify the pattern using the exact file name.
Eg: leads_jan_2022.*

• Parse file as : Choose the required extension to parse the file. If your file format is not a commonly used one, you can use this option to parse the file into one of the following formats before importing the data into a readable format. The available formats are CSV, TSV, JSON, XML, and TXT.
  
  

• Merge files and import
  

   - This will merge all the files that match the pattern specified and import them as a single dataset.
  

  You can use this option to merge files together during the import itself, without having to perform unions post import.

  Info: This option can merge a maximum of only 5 files at a time.
  Note: If this checkbox is unchecked then, only 1 file will be fetched at a time.
  Eg. If your Google Drive account has 10 files, the first 5 will be merged into one dataset and imported. During the next reload, the remaining 5 files will be merged and imported.
  
  
  
  Similarly, if your Google Drive account has 8 files, the first 5 will be merged and fetched first, followed by the next 3. 
  

3. Click the Import button.

Note: We support only CSV, TSV, JSON, XML, and TXT file formats for incremental fetch from cloud storage.

Limitations

 
1.  Using the Include subfolder option, you can only fetch files from a single sub-folder or all the sub-folders of the entire My drive or Shared with me folders. You cannot fetch files from all the sub-folders within a specific folder.

2. To fetch files from a single subfolder: Enter the exact path of the subfolder in the folder path. For example, 2023/jan/ . Fill in the required details. The files that match mentioned file pattern will be fetched from the specified folder.

3. To fetch files from all subfolders of Google Drive: Leave the folder path empty and select the Include subfolders checkbox. Fill in the required details. The files that match the mentioned file pattern will be fetched from all the sub-folders of the entire My Drive or Shared with me folders.