Understand the Discovery Companion application

The Discovery Companion app uses metadata ingestion to produce discovery files.

It is a Python package based on DataHub.

We are using pull-based metadata ingestion: we ingest metadata from data systems by connecting to them and extracting metadata in a file.

The Discovery Companion can discover metadata from over 50 sources, such as Microsoft SQL Server, CSV files, Kafka, Snowflake, and Redshift.

You can find all the possible sources here.

Only sources with a certified support status should be used:

In this Knowledge Base, we explain how to use the most common sources with the Discovery Companion: How to use the Discovery Companion app.

Sinks are destinations for metadata.

Within the Discovery Companion, the Sink is always a file. You will use the file in the biGENIUS-X application to discover your source data.

A recipe is the main configuration file that puts it all together.

It is a YAML file. Please look at how to use Discovery Companion for more detailed information.