Take care of entries from the Data Quality rules log

Depending on your Generator Configuration, some default Data Quality Rules are applied during the data load of your data solution.

It is also possible to add custom ones (See Understand Data Quality rules overview).

This article explains how they work during the load and how to care for them.

Default Data Quality rules

There are currently 5 default Data Quality Rules:

• Nulled Non-Nullable Business Keys: checks if the Term set as Non-Nullable Business Keys contains nullable values
• Duplicated Business Keys: checks if the Term set as Business Key has duplicated values
• Nulled Non-Nullable Term: checks if the Term set as Non-Nullable contains nullable values
• Duplicated Term: checks if the Term has duplicated values
• Missing Foreign Keys: check if the Foreign Key Term value exists in the Model Object in relation to

Data Quality rules check during load

For each Model Object where Data Quality rules can be applied, a XXX_DataQualityError table is created during deployment.

When a row violates a Data Quality rule, and the Logging option is set to True for the rule, the row is logged to this table rather than in the usual target table.

Example:

Take care of the Data Quality rule entries

During a load, you may have multiple entries in the XXX_DataQualityError tables.

You need to take care of these entries regularly because they show a potential problem in the design of your Model Object.

An entry in this table can be of different types:

• Nulled Non-Nullable Business Keys:
  • The Term set as Non-Nullable Business Key contains nullable values
  • ACTION TO DO: Define a Business Key that will never be null for any row
• Duplicated Business Keys:
  • The Term set as Business Key has duplicated values
  • ACTION TO DO: Define a Business Key that will be unique for all your rows
• Nulled Non-Nullable Term:
  • The Term set as Non-Nullable contains nullable values
  • ACTION TO DO: Define a Term that will never be null for any line
• Duplicated Term: 
  • The Term has duplicated values
  • ACTION TO DO: Define a Term that will be unique for all your lines
• Missing Foreign Keys:
  • The Foreign Key Term value doesn't exist in the Model Object in relation to
  • ACTION TO DO: Correct the Model Object in order to have all values used in related Model Objects