A Model Object of type Stage can be created in the Bronze Layer.
What is a Stage
A Stage represents a source object in the target architecture. Therefore, a Stage is created for each source object accessed in the DWH.
Creation steps
There are two possibilities to create a Stage Model Object:
- Create a Model Object with the Wizard:
- Recommended possibility
- Use the Wizard Create a Stage from a Source that contains the following steps:
- Create a Model Object manually:
- Not recommended possibility
- Create a Model Object of type Stage in the Bronze Layer
- Create a Dataflow Set
- Add a Model Object to the Dataflow Set: add the Source Model Object
- Auto-map the Source Terms: select the auto-map mode and auto-map all the Source Terms to create the Target Terms and map them to the Source Terms
If a Business Key is defined in the Source Model Object, the same one will be defined in the Stage Model Object when you auto-map it.
If you used the Wizard and your Source Model Object was a file, the Stage name can contain by default the extension of the file as CreditCard.parquet, for example.
Please remove the extension and rename the Stage object to CreditCard (for this example).
Properties
A Stage Model Object can be customized through the following Properties:
-
Deduplication Method or Optimization Method (Only for Spark Generator)
- Deduplication Partition or Optimization Method Columns (Only for Spark Generator)
- File format (Only for Spark Generator)
Default Terms
A Stage Model Object will include the following Default Terms:
Business Rules
The following Business Rules are checked during the Model Object creation:
- Dataflows amount: min = {1}, max = {1}
- Dataflow Sets amount: min = {1}, max = {null}
- Dataflow Set Model Objects amount: min = {1}, max = {null}
- Relationships amount: min = {null}, max = {0}
- Terms amount: exclude Default Terms = {true}, exclude Business Keys = {false}, exclude Identities = {false}, exclude Foreign Keys = {false}, exclude unmapped Terms = {false}, min = {1}, max = {null}
- Supported Term Data Types: exclude Default Terms = {true}, exclude Business Keys = {false}, exclude Identities = {false}, exclude Foreign Keys = {false}, exclude unmapped Terms = {false}
- Supported Term Data Types: exclude unmapped Terms = {true}, Term name = {IncrementalFilter}
- Supported Implementation Types: supported Implementation Types = {Cached, Virtual}
- Implementation Type Incremental Filter
- Supported Deduplication Methods: supported deduplication Methods = {None, Distinct, Partition}