Skip to content
  • There are no suggestions because the search field is empty.

Understand Dataflow, Dataflow Set and Dataflow Set Model Object

To understand the concepts of Dataflow, Dataflow Set, and Dataflow Set Model Object, let's assume we want to load in a Model Object data from the following SQL statement:

SELECT  AccountNumber, 
        CustomerID, 
        PersonID, 
        StoreID, 
        TerritoryID
FROM    Customer cus
LEFT JOIN    Person per on cus.PersonID = per.PersonID
WHERE     cus.PersonID IS NOT NULL

EXCEPT

SELECT  AccountNumber, 
        CustomerID, 
        PersonID, 
        StoreID, 
        TerritoryID
FROM    Customer cus
WHERE     cus.PersonID IN (658,659)

Dataflow

A Dataflow is a loader.

In our example, we can imagine two source applications from which the Model Object data are coming.

We can create 2 Dataflows, so two loaders:

  • 1 to load data daily from a source application A
  • 1 to load similar data every month from a source application B

Dataflow Set

A Dataflow Set is a selection of data (SELECT statement).

In our example, there are 2 Dataflow Sets with the Set Operator Except between the 2 Sets:

Dataflow Set Model Object

A Dataflow Set Model Object is a Model Object used in a Dataflow Set.

In our example, there are:

  • 2 Dataflow Set Model Objects in the Dataflow Set 1 with a Join Operator Left Join between the 2 Model Objects
  • 1 Dataflow Set Model Object in the Dataflow Set 2