Understand Input Configuration File Examples for biGENIUS Discovery CLI
Overview
The biGENIUS Discovery CLI supports reading configuration properties from external files using the --properties-file
option. This feature is available in both the create
and create-and-discover
commands. The CLI automatically detects the file format based on the file extension and supports multiple file formats.
Usage
# Using with create command
create <discovery-type> --name <name> --properties-file <file-path>
# Using with create-and-discover command
create-and-discover <discovery-type> --name <name> --properties-file <file-path>
Supported File Formats
The CLI supports the following file formats:
Format |
Extensions |
Description |
---|---|---|
CSV |
.csv |
Comma-separated values file |
Environment |
.env |
Environment variables file |
INI |
.ini |
INI configuration file |
JSON |
.json |
JavaScript Object Notation file |
TOML |
.toml |
Tom's Obvious, Minimal Language file |
XML |
.xml |
Extensible Markup Language file |
YAML |
.yaml, .yml |
YAML Ain't Markup Language file |
File Format Examples
CSV Format (.csv)
FilePathType,FilePath,ColumnDelimiter,FieldQuote,RowDelimiter
"Directory With Partitioned Entities","C:\data","|","""","\r\n"
-
First row must contain property names
-
Values should be properly quoted if they contain special characters
-
Double quotes within values should be escaped with another double quote
Environment Format (.env)
FilePathType="Directory With Partitioned Entities"
FilePath="C:\data"
ColumnDelimiter="|"
FieldQuote="""
RowDelimiter="\r\n"
Notes:
-
Property names are case-sensitive
-
Values can be quoted (recommended for special characters)
-
Backslashes do not have to be escaped
INI Format (.ini)
FilePathType=Directory With Partitioned Entities
FilePath=C:\data
ColumnDelimiter=|
FieldQuote="
RowDelimiter=\r\n
Notes:
-
Values don't need quotes unless they contain special characters
-
Backslashes in paths don't need to be escaped
-
Special characters don’t need to be escaped
JSON Format (.json)
{
"FilePathType": "Directory With Partitioned Entities",
"FilePath": "C:\\data",
"ColumnDelimiter": "|",
"FieldQuote": "\"",
"RowDelimiter": "\\r\\n"
}
Notes:
-
Must be valid JSON format
-
All strings must be quoted
-
Backslashes must be escaped
-
Special characters must be escaped
TOML Format (.toml)
{
"FilePathType": "Directory With Partitioned Entities",
"FilePath": "C:\\data",
"ColumnDelimiter": "|",
"FieldQuote": "\"",
"RowDelimiter": "\\r\\n"
}
Notes:
-
Values must be properly quoted
-
Backslashes must be escaped
-
Special characters must be escaped
XML Format (.xml)
<?xml version="1.0" encoding="utf-8"?>
<FileProcessingConfig>
<FilePathType>Directory With Partitioned Entities</FilePathType>
<FilePath>C:\data</FilePath>
<ColumnDelimiter>|</ColumnDelimiter>
<FieldQuote>"</FieldQuote>
<RowDelimiter>\r\n</RowDelimiter>
</FileProcessingConfig>
Notes:
-
Root element name is ignored
-
Special characters should use XML entities
-
Backslashes in paths don't need to be escaped
-
Property names are case-sensitive
YAML Format (.yaml, .yml)
FilePathType: "Directory With Partitioned Entities"
FilePath: "C:\data"
ColumnDelimiter: "|"
FieldQuote: """
RowDelimiter: "\r\n"
Notes:
-
Simple values don't need quotes
-
Special characters should be quoted
-
Backslashes must not be escaped
Best Practices
-
File Naming:
-
Use descriptive names that indicate the configuration type
-
Always include the correct file extension
-
Example:
sqlserver-config.json
,csv-local.yaml
-
-
Value Formatting:
-
Always quote values containing special characters
-
Properly escape backslashes in paths
-
Use appropriate escaping for quotes within values
-
-
Property Names:
-
Use consistent casing (the CLI is case-sensitive)
-
Match the exact property names from the discovery type
-
Avoid spaces in property names
-
Common Issues and Solutions
-
File Not Found:
-
Ensure the file path is correct
-
Use absolute paths or paths relative to the current directory
-
Check file permissions
-
-
Format Detection:
-
Make sure the file extension matches the actual format
-
Ensure the file content matches the expected format
-
Check for syntax errors in the file
-
-
Property Mapping:
-
Verify property names match the discovery type
-
Check for case sensitivity issues
-
Ensure all required properties are present
-
-
Special Characters:
-
Use proper escaping for backslashes
-
Quote values containing special characters
-
Use appropriate escaping for quotes within values
-
Examples
Here are complete examples of using properties files with different formats:
# Create a CSV Local configuration using a JSON file
create "CSV Local" --name "MyCSV" --properties-file "CsvLocal.json"
# Create and discover a CSV Local configuration using a YAML file
create-and-discover "CSV Local" --name "MyCSV" --properties-file "CsvLocal.yaml"
# Create a configuration using an environment file
create "CSV Local" --name "MyCSV" --properties-file ".env"
# Override a property from the file
create "CSV Local" --name "MyCSV" --properties-file "config.json" --properties FilePath="C:\new\path"
Demonstration video