Create a Raw Vault Composite

A Model Object of type Raw Vault Composite can be created in the Raw Vault Layer.

What is a Raw Vault Composite

A Raw Vault  Composite represents an intermediate object that holds temporary data to build readable and traceable mappings.

Often, they represent a sub-select or common table expression (CTE).

They can contain joins, aggregation, expressions, filters, and default values.

Use case example

Source data

We will use this Model Object Type to split our sales data into reseller and internet sales.

Our source data are from the Stage SalesOrderHeader.

The Term OnlineOrderFlag lets us distinguish between resellers and the internet in sales.

Let's create a Raw Vault Composite with the Wizard from the Stage SalesOrderHeader:

Rename it SC_ResellerSales.

Add the following where clause for the Set1:

s1.OnlineOrderFlag = 0

Load of data

The resulting view after the data load is:

Only the sales from resellers are inside the view.

Creation steps

There are two possibilities to create a Raw Vault Composite Model Object:

• Create a Model Object with the Wizard:
  • Recommended possibility
  • Use the Wizard Create a Raw Vault Composite from a Stage or a Raw Vault Composite that contains the following steps:
    • Create Model Object
    • Create Dataflow
    • Add to View
• Create a Model Object manually:
  • Not recommended possibility
  • Create a Model Object of type Raw Vault Composite in the Raw Vault Layer
  • Create a Dataflow Set 
  • Add a Model Object to the Dataflow Set: add the Stage Model Object
  • Auto-map the Source Terms: select the auto-map mode and auto-map all the Source Terms to create the Target Terms and map them to the Source Terms

Properties

A Raw Vault Composite Model Object can be customized through the following Properties:

• Implementation Type

• Deduplication Method or Optimization Method (Only for Spark Generator)

• Deduplication Partition or Optimization Method Columns (Only for Spark Generator)
• File format (Only for Spark Generator)

Default Terms

A Raw Vault Composite Model Object will include the following Default Terms:

• BG_LoadTimestamp

• BG_SourceSystem

Business Rules

The following Business Rules are checked during the Model Object creation:

• Dataflows amount: min = {1}, max = {null}
• Dataflow Sets amount: min = {1}, max = {null}
• Dataflow Set Model Objects amount: min = {1}, max = {null}
• Relationships amount: min = {null}, max = {0}
• Terms amount: exclude Default Terms = {true}, exclude Business Keys = {false}, exclude Identities = {false}, exclude Foreign Keys = {false}, exclude unmapped Terms = {false}, min = {1}, max = {null}
• Supported Term Data Types: exclude Default Terms = {true}, exclude Business Keys = {false}, exclude Identities = {false}, exclude Foreign Keys = {false}, exclude unmapped Terms = {false}
• Supported Term Data Types: exclude unmapped Terms = {true}, Term name = {IncrementalFilter}
• Supported Implementation Types: supported Implementation Types = {Cached, Virtual}
• Implementation Type Incremental Filter
• Supported Deduplication Methods: supported deduplication Methods = {None, Distinct, Partition}