You have a historized ODS and also perhaps some delete detection activated in your ODS or a source that contains several entries for the same Business Key but with a different timeline: you can load the full history of your source object in your Data Vault DWH with the Multi-Version Load behavior.
We can configure the Multi-Version Load behavior on the following Model Objects:
- Store Entity
As it is possible to mix SCD1 and SCD2 in a Store Entity, please check if the timelines of the historized source Model Object are correct. If not, this can lead to unexpected results.
- Satellite
- Link Satellite
- Business Satellite
- Business Link Satellite
- Mart Dimension
To configure it, you must set, for all the Dataflow Set Model Objects containing historized data, the ValidFrom Source Term Property to the Dataflow Set Model Object Term containing the start date of the source row:
For Data Vault Model Objects, the corresponding Hub and all the related Links must be deduplicated, so have the Deduplication Method Property set.
To have a better understanding, let's take an example in a Data Vault modeling.
Source data
In this example, we have a historized source table named ProductS.
Products 317, 382, 383, and 384 have several rows for different timelines:
Data Vault model
Let's create the following object:
- A Hub named HubProductS (source data ODS_Sales.AWProduction_ProductS)
- Set the Deduplication Method Property to Distinct
- A Satellite named SatProductS (source data ODS_Sales.AWProduction_ProductS) and his relationship to the HubProductS
- Set the ValidFrom Source Term Property with the Term ODS_ValidFrom
Load of data
Generate, replace the placeholders, deploy, and load the project.
Then check the data for the ProductS :
SELECT sat.[BG_LoadTimestamp]
, hub.[Hub_HK]
, hub.[ProductSID]
, sat.[BG_ValidFromTimestamp]
, sat.[Name]
, sat.[ProductSNumber]
FROM [RDV].[RDV_SAT_AW_ProductS1] sat
left join [RDV].[RDV_HUB_AW_ProductS] hub on hub.Hub_HK = sat.Hub_HK
where hub.[ProductSID] in (317, 382, 383, 384)
You should have this result:
The history from the source was loaded, and the load dates (BG_ValidFromTimestamp) are the same as the ODS_ValidFrom Term mapped.