Skip to content
Contact us Go to Customer Portal
  • There are no suggestions because the search field is empty.

BG_SourceFilePath

Applies to
Platform: Databricks · Fabric Lakehouse
Architecture: Stage

BG_SourceFilePath holds the path of the source file that a Stage row was read from. Use it to trace a row back to its file when a Stage reads several files.

Objects that carry it

Stage in the file Stage generators for Databricks and Fabric Lakehouse. Every Stage gets the term when it is created. It is a column of the Stage table and the Result view and cannot be NULL. You do not map it; the loader fills it.

Value

The Stage loader adds the path while it reads the files:

  • Databricks: the file metadata column _metadata.file_path
  • Fabric Lakehouse: the function input_file_name()

Data type

string on both platforms.

Example

The loader notebook of a CSV Stage CUSTOMER on Databricks reads the files and adds the column:

df = spark.read.format('csv') \
    .option('sep', '|') \
    ...
    .load("{stage.spark.databricks.csv#bfm_csv#file_path}CUSTOMER") \
    .withColumn("bg_sourcefilepath", col("_metadata.file_path"))

The placeholder in curly braces is the file path you set when you replace placeholders. Each row of the Stage table then holds the path of the file it came from.

Related articles