Discover a Delta file

Discover a Delta file

To discover a Delta file with the Discovery Companion, please first prepare the source YAML file with the following:

  • The specifications to access the source system
  • The specifications for the output file

 

Here is an example:

  • A Delta file available in a local path ./creditcard_delta:

  • A local output folder for the Discovery JSON file is named outputs.
    The complete path is: ./outputs/delta_discovery_output.json
source:
  type: delta-lake
  config:
    base_path: ./creditcard_delta

sink:
  type: file
  config:
    filename: ./outputs/delta_discovery_output.json

The output folder in your source YAML file should exist.

You can download this YAML file example here or create a .yaml file with the previous example and then adapt it:

  • The Delta files path 
  • The output path

Then, you can use it in the Discovery Companion.

 

All possible configurations are described on the DataHub official website.