Skip to content
  • There are no suggestions because the search field is empty.

Partition deduplication without business keys (Business Rule)

Applies to
Platform: Microsoft SQL Server · Snowflake · Snowflake dbt · Databricks · Databricks dbt · Fabric Lakehouse · Fabric Warehouse · Oracle

This business rule warns when a dataflow uses Deduplication Method Partition (PartitionOrder) on a model object without business key terms. It raises a warning, and generation continues. The rule ID is ID-10051.

Message

The client shows a warning with this text on the model object. <model object name> and <type> stand for its name and type.

Model object '<model object name>' of type '<type>' has dataflow with
deduplication method 'Partition' has no business keys defined. Therefore
all attributes are used as partition, which is functionally the same as
deduplication method 'Distinct' but with degraded performance.

On the model object card, point to the warning icon to see this message with the rule ID: Warning ID-10051. The Generate Artifacts page lists the message without the ID.

Cause

A dataflow has Deduplication Method Partition, and no term of the model object is marked as business key. The partition then uses all terms that are not Default Terms. The rule checks Stage, Store Composite, Fact, Core Composite, Mart Fact, Mart Composite, Mart Out, RawVault Composite, BusinessVault Composite and Vault Out.

Fix

The model object is generated as configured. Do one of the following:

  • Mark the terms that identify a row as business key in Manage Terms. The partition then uses them.
  • Set Deduplication Method of the dataflow to Distinct, see Edit a Dataflow.

Check the result

The warning icon is gone from the model object.

Example

In a Dimensional project on Microsoft SQL Server, the dataflow of the Fact deduplication_partition_without_bk_fa has Deduplication Method Partition, and the Fact has no business key term. The client shows:

Model object 'deduplication_partition_without_bk_fa' of type 'Fact' has
dataflow with deduplication method 'Partition' has no business keys
defined. Therefore all attributes are used as partition, which is
functionally the same as deduplication method 'Distinct' but with
degraded performance.

Related articles