Skip to content
  • There are no suggestions because the search field is empty.

Understand the Generator target objects

Applies to
Platform: Microsoft SQL Server · Snowflake · Snowflake dbt · Databricks · Databricks dbt · Fabric Lakehouse · Fabric Warehouse · Oracle

Multiple target objects are generated for a single model object that you model.

This target object processing chain applies all defined rules to the source data, producing the final target object.

Target objects processing chain

The complete processing chain is illustrated in the following diagram:

Target object processing chain: Source leads to the decision Deduplication?. With Yes the Deduplication view follows. Both paths join and continue through the decisions Delete detection? and Specific Scd management?, with DeleteDetection and Scd, to Hashing. The same junction leads to the DataQuality Loader, which writes the DataQualityTable; the DataQualityBeforeLoadAction Loader writes DataQualityError and updates the DataQualityTable; the DataQualityResult view reads the DataQualityTable and leads to Hashing. Hashing continues through Lookup, RowCondensing, Versioning, Delta and the Final Loader to the Final target object and the Result view.

The following target objects are generated depending on the features you configured:
  • Views: Source, Deduplication, DeleteDetection, Scd, Hashing, Lookup, RowCondensing, Versioning, Delta, DataQualityResult, Result
  • Loader: DataQuality Loader, DataQualityBeforeLoadAction loader, DataQualityAfterLoadAction loader, Final Loader
  • Storage (can be a Table or a File): DataQualityTable, DataQualityError, Final Target object

Source

Purpose: Displays the source data according to the modeled dataflow, including dataflow sets and term mappings.

Predecessor(s): Source System

Successor(s): Deduplication, DeleteDetection, DataQuality Loader, or the next view of the chain, for example Hashing

The Source view accesses the source object(s) of a model object, which may include Source Systems or other model objects.

Deduplication

Purpose: Filters duplicate rows based on the selected deduplication method.

Predecessor(s): Source

Successor(s): DataQuality Loader when "Before Load" data quality checks are configured, otherwise the next view of the chain, for example Hashing

The Deduplication view is optional: it is created and used only when deduplication is activated.

In addition to the view, a DeduplicationTable and a DeduplicationLoader are also created and used.

See Deduplication Method for more details.

DataQuality Loader

Purpose: Loads source data into the data quality table.

Predecessor(s): Source, Deduplication

Successor(s): DataQualityTable, DataQualityBeforeLoadAction Loader

The DataQualityLoader is optional: it is created and used only when "Before Load" data quality checks are configured. In addition to the loader, a DataQualityBeforeLoadAction, a DataQualityTable, a DataQualityError table, and a DataQualityResult view are created.

DataQualityBeforeLoadAction Loader

Purpose: Performs configured "Before Load" data quality checks.

Predecessor(s): DataQuality Loader

Successor(s): DataQualityError, DataQualityTable (the loader updates its rows)

The DataQualityBeforeLoadAction is optional: it is created and used only when "Before Load" data quality checks are configured. In addition to the loader, a DataQualityLoader, a DataQualityTable, a DataQualityError table, and a DataQualityResult view are created.

DataQuality Table

Purpose: Contains the data on which "Before Load" data quality checks are performed.

Predecessor(s): DataQuality Loader, DataQualityBeforeLoadAction Loader

Successor(s): DataQualityResult

The DataQualityTable is optional: it is created and used only when "Before Load" data quality checks are configured. In addition to the table, a DataQualityLoader, a DataQualityBeforeLoadAction, a DataQualityError table and a DataQualityResult view are created.

DataQualityResult

Purpose: Filters rows with data quality errors.

Predecessor(s): DataQualityTable

Successor(s): Hashing, or the next view of the chain

The DataQualityResult is optional: it is created and used only when "Before Load" data quality checks are configured. In addition to the view, a DataQualityLoader, a DataQualityBeforeLoadAction, a DataQualityTable, and a DataQualityError table are created.

DataQualityError

Purpose: Contains suspect rows with data quality issues.

Predecessor(s): DataQualityBeforeLoadAction Loader, DataQualityAfterLoadAction Loader

Successor(s): N/A

The DataQualityError is optional: it is created and used only when data quality checks are configured. In addition to the table, a DataQualityLoader, a DataQualityBeforeLoadAction, a DataQualityAfterLoadAction, a DataQualityTable and a DataQualityResult view are created.

DataQualityAfterLoadAction Loader

Purpose: Performs configured "After Load" data quality checks.

Predecessor(s): N/A

Successor(s): DataQualityError, Final target Object

The DataQualityAfterLoadAction is optional: it is created and used only when "After Load" data quality checks are configured. In addition to the loader, a DataQualityError table is created.

DeleteDetection

Purpose: Filters rows containing deleted business keys.

Predecessor(s): Source

Successor(s): Scd

The DeleteDetection view is optional: it is created and used only when delete detection is activated.

In addition to the view, a DeleteDetectionTable and a DeleteDetectionLoader target objects are also created and used when the FullSet deduplication method is used.

See Configure a delete detection behavior for more details.

Scd

Purpose: Applies the historization type (slowly changing dimensions).

Predecessor(s): DeleteDetection

Successor(s): Hashing

The Scd view is optional: it is created and used when the most up-to-date information must be selected.

This is the case for a Mart Dimension when the source objects are historized (such as a Satellite in a Data Vault modeling, for example).

Hashing

Purpose: Calculates hash values.

Predecessor(s): Scd, DataQualityResult, Deduplication or Source, whichever of them is the last view created before Hashing

Successor(s): Lookup

The Hashing view generates hash values like Hub_HK, Link_HK, and BG_RowHash.

If Load Caching is set to Hashing, additional HashingTable and HashingLoader target objects are also created and used.

See Load Caching (Model Object) for more details.

Lookup

Purpose: Resolves relationships with other model objects by applying foreign keys and identifying the surrogate keys.

Predecessor(s): Hashing

Successor(s): RowCondensing

The Lookup view is created for every Entity, Fact and Mart Fact. A foreign key without a match in the referenced model object gets the surrogate key -1 of the unknown-member row.

See Handle missing Foreign Key in a Relationship for more details.

RowCondensing

Purpose: Filters row versions with no changed data compared to the previous version.

Predecessor(s): Lookup

Successor(s): Versioning

The RowCondensing view is optional: it is created and used only when historized sources are configured (multi-version load).

See Version Valid From Term for more details.

Versioning

Purpose: Builds a single history for one or multiple historized source objects in a dataflowset.

Predecessor(s): RowCondensing

Successor(s): Delta

The Versioning view is optional: it is created and used only when historized sources are configured (multi-version load).

See Version Valid From Term for more details.

Delta

Purpose: Filters for changes compared to the previously loaded data.

Predecessor(s): Versioning

Successor(s): Final Loader

The Delta view shows the difference between the source (Hashing, Lookup, and RowCondensing views) and target (Result view) to identify rows that must be loaded to the final target object.

Final Loader

Purpose: Loads data into storage.

Predecessor(s): Delta

Successor(s): Final target object

The loader inserts the final data into the final target object.

In addition, an IncrementTable and an IncrementSource view target objects are created and used when incremental load behavior is configured.

See Configure an incremental load for more details.

Final target object

Purpose: Stores the persisted or cached data.

Predecessor(s): Final Loader, DataQualityAfterLoadAction Loader

Successor(s): Result

The final target object serves as the ultimate storage location where data is inserted.

Result

Purpose: Presents the final stored data that is ready for consumption.

Predecessor(s): Final target object

Successor(s): N/A

The Result view delivers the final data to external consumers, such as upper layers or other systems.

Related articles