Core Composite

A Core Composite prepares data in the Cleanse layer with joins, filters, expressions and aggregations on Stages, Store Entities or other Core Composites. Use it for an intermediate result that an Entity, a Fact or another Core Composite reads. By default it is a view; with Implementation Type Cached it is a table that each load empties and refills.

Sources and relationships

A Core Composite reads Stages, Store Entities of a linked Data Store project and other Core Composites, in exactly one dataflow. It has no relationships. An Entity, a Fact or another Core Composite reads its result view. Mart objects of a Dimensional and Mart project do not read it.

Create a Core Composite

• With the wizard: open the menu of a Stage, Core Composite or Store Entity, choose Automate and then Create Core Composite. Its steps are Create Model Object, Create Dataflow, Map Selected Terms and Add to View. See Create a Model Object with the wizard.
• Manually: create a model object of type Core Composite in the Dimensional column, create a dataflow set, add the source model objects and map the terms.

Then add joins, a where clause or expressions to the dataflow sets as the composite needs.

Properties

• Implementation Type: Virtual (default) creates views only, so every query reads the current data of the sources. Cached adds a table that the loader empties and refills on every load.
• Deduplication Method: None (default), Distinct or Partition, with Sort Order within Partition. A method other than None adds a deduplication view. Core Composite terms can be marked as Business Key in Edit Term; with Partition and no business key, validation reports a warning.

Default Terms

• BG_LoadTimestamp: with Cached, the loader writes the timestamp of the load. With Virtual, the term is NULL unless you map it.
• BG_SourceSystem: the source system of the row; empty unless you map it.

Generated objects

These are the names on Microsoft SQL Server for a Core Composite large_branch:

• All objects are in the Cleanse layer. The loader is a stored procedure; CLS_CC_large_branch is a table; all other objects are views.
• The deduplication view exists only with a Deduplication Method other than None.
• The loader and the table exist only with Cached.

The dependencies are:

Virtual: sources -> _Source [-> _Deduplication] -> _Result
Cached:  sources -> _Source [-> _Deduplication] -> _Loader
         _Loader -> CLS_CC_large_branch -> _Result

Example

The Core Composite large_branch on Microsoft SQL Server keeps only the large branches of the Stage branch. It maps BRANCH_ID, BRANCH_NAME and MARKET_SIZE; BG_LoadTimestamp is not mapped. Its dataflow set has this where clause, in which s1 is the alias of the Stage:

s1.MARKET_SIZE = 'large'

The generated source view ends with:

FROM [<database name>].[stg].[STG_ST_branch_Result] AS [s1]
WHERE [s1].[MARKET_SIZE] = 'large'

<database name> is the Stage database that you set when you replace the placeholders, and stg is the Stage schema of the example. The Stage branch holds three branches after load 1 on 2026-10-28 08:00 and two after load 2 on 2026-10-29 08:00:

• Load 1: 1 Basel large, 2 Bern small, 3 Zurich large.
• Load 2: Zurich is deleted, and Bern changes to large.

Check the result

Query the result view, here in the schema cls:

SELECT BRANCH_ID, BRANCH_NAME, BG_LoadTimestamp
FROM cls.CLS_CC_large_branch_Result
ORDER BY BRANCH_ID;

With the default Implementation Type Virtual, after load 1 and after load 2 of the Stage:

After load 1                        After load 2
BRANCH_ID  BRANCH_NAME  LoadTs      BRANCH_ID  BRANCH_NAME  LoadTs
1          Basel        NULL        1          Basel        NULL
3          Zurich       NULL        2          Bern         NULL

With Cached, the result changes only when the loader of the Core Composite runs. The loader ran after Stage load 1 on 2026-10-28; this is the result after Stage load 2, before and after the loader runs again on 2026-10-29:

Before the loader                   After the loader
BRANCH_ID  BRANCH_NAME  LoadTs      BRANCH_ID  BRANCH_NAME  LoadTs
1          Basel        2026-10-28  1          Basel        2026-10-29
3          Zurich       2026-10-28  2          Bern         2026-10-29

• The filter excludes Bern in load 1. Zurich is missing after load 2 because the Stage no longer holds it.
• A Virtual Core Composite follows the Stage at once, because its views read the result view of the Stage.
• A Cached Core Composite keeps the rows of its last load until its loader runs again; the loader writes its load timestamp to every row.

Platform differences

• Snowflake: names are in capitals, and the table of a Cached Core Composite is a transient table. The load control task LC_CLS_CC_<MODEL OBJECT NAME>_TASK calls its loader. The dataflow properties Use Stream and Warehouse Placeholder Variable Name apply.
• Databricks and Fabric Lakehouse: the loader is the notebook CLS_CC_<model object name>_Loader. The properties File Format, Optimization Method and Optimization Method Columns apply.
• Fabric Warehouse: the deployment notebooks create the same objects as on Microsoft SQL Server. Fabric Warehouse is available since Generator 2.2.

Limitations

• A Core Composite has exactly one dataflow, at least one term that is not a Default Term, and no relationships.
• An aggregation must be defined in all dataflow sets or in none.
• Dataflows between Core Composites must not form a cycle.
• A Core Composite keeps no history; it has no validity terms.

Related articles

• Previous: Stage
• Next: Entity, Fact
• Reference: Implementation Type (Model Object), Deduplication Method (Dataflow), Sort Order within Partition (Dataflow), BG_LoadTimestamp, BG_SourceSystem
• See also: Create a Model Object with the wizard, Model a Dimensional Solution