LoadControl Options (Project)

LoadControl Options (LoadControlOptions) select how the generated loaders are orchestrated: by a native load control, by Apache Airflow, or both. The generator writes the load-control artifacts for each selected value.

The dbt generators do not offer the property.

Values

A multi-select list in Project Settings. Each platform offers these values, the first one being the default:

• Microsoft SQL Server, Oracle: Native Load Control, Airflow.
• Databricks, Fabric Lakehouse: Multi-Threading Notebook, Airflow.
• Fabric Warehouse: Multi-Threading Notebook.
• Snowflake: Depending Tasks, Stage-Driven Tasks, Near-Realtime Tasks, Airflow.

Meaning of the values:

• Native Load Control: tables and stored procedures in the target database register, plan, run and log the loads. See Understand Load Control.
• Multi-Threading Notebook: a load-control notebook runs the loader notebooks on four threads in dependency order.
• Depending Tasks, Stage-Driven Tasks, Near-Realtime Tasks: Snowflake tasks run the loaders. The dataflow property Ignore Downstream Dependencies (default false) is used only by Stage-Driven Tasks; see Snowflake - Load data - Native Load Control.
• Airflow: the generator writes an Airflow DAG, LoadControl\<project>_dag.py, with one task per loader. See Apache Airflow - Load control environment.

Effect on generated output

On Microsoft SQL Server, Native Load Control adds the load-control scripts and execute.sql. Airflow alone produces only the DAG; deploy.sql then deploys the layers and runs the deployment validation, without load-control scripts. With both values selected, the artifacts contain both.

Example

The Data Vault Test Project on Microsoft SQL Server with Native Load Control and Airflow has these files in LoadControl:

configure_loadcontrol.sql
datavault_mssql_dag.py
deploy_loadcontrol.sql
L000_Loadcontrol_Deployment.sql
L010_Stage_Configuration.sql
L020_Raw Vault_Configuration.sql
L030_Business Vault_Configuration.sql
LoadControl_Execution_Datavault.Mssql.sql

Related articles

• Reference: Configure LoadControl (Project), Deploy LoadControl (Project), Logging Method (Project)
• See also: Understand Load Control, Native - Load control environment, Apache Airflow - Load control environment
  • Microsoft SQL Server: Load data - Native Load Control, Load data - Apache Airflow Load Control
  • Snowflake: Load data - Native Load Control, Load data - Apache Airflow Load Control
  • Databricks: Load data - Native Load Control, Load data - Apache Airflow Load Control
  • Fabric Lakehouse: Load data - Native Load Control, Load data - Apache Airflow Load Control
  • Fabric Warehouse: Load data - Native Load Control
  • Oracle: Load data - Native Load Control, Load data - Apache Airflow Load Control