Skip to content
  • There are no suggestions because the search field is empty.

Discovery Application Release - 3.0.2

New Features

  • App / UI

    • Implemented a uniqueness check for discovery configuration names to ensure they are unique.

    • Added a validation check to ensure source objects are selected before enabling the "Save As" and "Upload" buttons in the source system view.

    • Added support for Source System and Auto-Version APIs.

  • App / CLI

    • Added support for word-by-word navigation in the command line interface.

    • Added support for Source System and Auto-Version APIs.

    • Changed the --property argument to --sensitive-properties to clearly indicate its usage and avoid confusion.

    • Added file arguments to the discover and create-and-discover commands to support the use of external files for sensitive properties and API connection properties.

  • Discovery File Generation

    • Implemented logic to exclude Source Object Models that contain no fields from the output and display a notification message.

  • Parquet - OneLake

    • Introduced support for Parquet - OneLake Discovery.

  • Snowflake Discovery

    • Added optional Warehouse attributes to the discovery process.

  • Unity Catalog - Databricks

    • Enhanced discovery to include tables where the user has metadata access but not read access by adding an additional parameter to the API call.

Bug Fixes

  • App / UI

    • Updated logic to ensure the "Last Execution Time" is only updated upon successful discovery generation.

    • Fixed an issue where description values for a discovery were not properly uploaded to the biGENIUS-X Data Marketplace API.

    • Fixed an issue where the vertical scrollbar was always shown on the source system selection user control.

  • App / CLI

    • Resolved an issue where long commands were truncated in the application, causing exceptions.

    • Fixed an issue where discovery configurations were not properly deleted when using the --delete-after argument in the create-and-discover command.

    • Fixed an issue where log messages were printed directly to the console instead of being routed through the messaging system.

    • Updated help command documentation to reflect the new --sensitive-properties argument.

  • CSV - Local Filesystem Discovery

    • Fixed an issue on macOS where CSV files containing empty column values could not be parsed.

  • Parquet - Local Filesystem Discovery

    • Excluded Parquet files located in _delta_log directories to prevent structure mismatch errors.

Technical Improvements

    • App / UI

      • Updated the database implementation to single-file storage for discovery configurations.

      • Simplified CSV handling and removed dependence on actual data from CSV files for all CSV discoveries.

      • Refactored internal naming conventions: renamed 'FlatFile' to 'CSV' and introduced support for 'Delta' and 'MariaDB' source system types.

      • Refactored Parquet/Delta and CSV handlers to use a unified FileStorage implementation.

      • Improved Delta discovery performance by optimizing the reading of checkpoint files and subsequent JSONs.

    • App / CLI

      • Updated the application startup to support migration to the new single-file data storage structure.

    • Unity Catalog - Databricks

      • Renamed the source system type label from "Parquet" to "Databricks Unity" for better clarity.


You can download this Discovery application release (3.0.2) here: