Discover Parquet or Delta files on the local file system

Please first read how to use the Discovery Application.

Discover Parquet or Delta files

To discover Parquet or Delta files on the local file system with the Discovery Application, please choose the following source system type:

Then fill in the following information:

• Name: discovery name
  • Example: Parquet_AW2019_Local
• File Path Type:
  
  
  • Single File: select this option if you want to discover a single Parquet or Delta file
    
    • Example of a single Delta file:
  • Folder Contains Multiple Entities: select this option if you want to discover several entities contained in a folder:
    • Example of 4 entities in a source folder:
      
      
    • Each entity folder contains the Parquet file to discover:
      
  • Delta/Parquet Entity: select this option to discover a folder containing all the Parquet or delta files concerning an Entity:
    
    • Example of 1 entity Credit Card with multiple Parquet files stored by partition (date):
      
• File Path: according to the File Path Type selected, fill:
  • Single File: the path to the single file
    
    • Example: C:\Discovery Application\Delta on local file system\source\creditcard

• • Folder Contains Multiple Entities: the path to the folder containing all the entity's folders
    
    • Example: C:\Discovery Application\Parquet on local file system\source

• • Delta/Parquet Entity: the path to the folder containing all the entity files
    
    • Example: C:\Users\Discovery Application\Parquet on local file system\source_partition\CreditCard