Column Delimiter (Model Object)

Column Delimiter (ColumnDelimiter) is the character that separates the values of a row in a source file. The Stage loader reads the file with this delimiter.

The property exists in the Stage Files generators of these platforms.

Where to set it

Set Column Delimiter when you edit a model object of type Source. It is a text field. On Databricks and Fabric Lakehouse it is in the group Csv, next to the other CSV options.

Defaults and loader output

A source object that you create from a Discovery gets the delimiter that the Discovery detected, for example |. The Discovery parameters are described in Discover CSV files on the local file system.

On SQL Server, the generator writes one format file per Stage. The Column Delimiter terminates every column except the last one, which ends with the Row Delimiter.

Example

A Discovery found the delimiter | in the file CUSTOMER.csv, and the Source CUSTOMER carries this value. On SQL Server the format file of the Stage starts with these column lines:

1   SQLCHAR  0  0  "|"     1   "CUSTOMER_ID"  ...
2   SQLCHAR  0  0  "|"     2   "CUSTOMER"     ...
...
11  SQLCHAR  0  0  "\r\n"  11  "BU"           ...

On Databricks and Fabric Lakehouse the Stage loader reads the file with:

df = spark.read.format('csv') \
    .option('sep', '|') \

The spacing of the format file is shortened here, and the other columns are left out.

Related articles

• Reference: Row Delimiter (Model Object), Field Quote (Model Object), CSV options (Model Object)
• See also: Create Source Objects from a Discovery
  • Microsoft SQL Server: Stage Files
  • Databricks: Stage Files
  • Fabric Lakehouse: Stage Files