Skip to content
Contact us Go to Customer Portal
  • There are no suggestions because the search field is empty.

File Name Pattern (Model Object)

Applies to
Platform: Snowflake
Architecture: Stage

File Name Pattern (FileNamePattern) is the regular expression that selects the files the Stage loader copies from the Snowflake stage folder of a Source. Partition Order (PartitionOrder) marks the terms of a Source whose values come from partition folders in the file path.

Both properties exist only in the Snowflake Stage File generator.

Where to set them

  • File Name Pattern: when you edit a model object of type Source. It is a text field.
  • Partition Order: in the Edit Term dialog of a term of a Source. A Discovery of partitioned files sets it, for example 1, 2 and 3 for the folders year, month and day.

Values and defaults

  • File Name Pattern: a regular expression in the syntax of the PATTERN option of Snowflake COPY INTO. Default .*[.]parquet, which selects every file with the extension .parquet.
  • Partition Order: a number for the level of the partition folder. Default empty, which means that the term is read from the file content.

Effect on the Stage loader

The Stage loader copies the files with COPY INTO. It reads the folder <source object name>/ of the Snowflake stage that the source system placeholders snowflake_stage_database_name, snowflake_stage_schema_name and snowflake_stage_name name, and passes File Name Pattern as PATTERN. For each term with a Partition Order, it takes the value from the folder <term name>=<value> of the file path.

Example

The Source customer reads Parquet files stored in folders such as customer/dt=2024-06-30/. Its term dt has Partition Order 1, and File Name Pattern keeps its default. The Stage loader contains:

,SPLIT_PART(SPLIT_PART(METADATA$FILENAME, '/dt=', 2), '/', 1)::DATE
    AS DT
...
PATTERN = '.*[.]parquet'

With File Name Pattern .*part-0.*[.]parquet on a Source, the COPY INTO statement of its Stage ends with PATTERN = '.*part-0.*[.]parquet'. Snowflake applies the pattern as described in its COPY INTO documentation.

Related articles