--- title: Databricks - Load data - Native Load Control description: "Load the deployed Databricks solution with the native load control: a notebook runs every loader notebook in dependency order, four at a time." --- [Skip to content](https://knowledge.bigenius-x.com/databricks-load-data-native-load-control#main-content) [Contact us](https://knowledge.bigenius-x.com/kb-tickets/new?hsLang=en) [Go to Customer Portal](https://knowledge.bigenius-x.com/customer-portal?hsLang=en) [![TRI\_Logo\_biGENIUS\_2020\_RGB\_s](https://knowledge.bigenius-x.com/hs-fs/hubfs/Marketing/TRI_Logo_biGENIUS_2020_RGB_s.png?width=150&height=42&name=TRI_Logo_biGENIUS_2020_RGB_s.png)](https://www.bigenius-x.com/) Open main navigation Close main navigation - [Contact us](https://knowledge.bigenius-x.com/kb-tickets/new) - [Go to Customer Portal](https://knowledge.bigenius-x.com/customer-portal) How can we help you? - There are no suggestions because the search field is empty. 1. [biGENIUS-X Knowledge Base](https://knowledge.bigenius-x.com/?hsLang=en) 2. [Generators](https://knowledge.bigenius-x.com/generators?hsLang=en) 3. [Load data with a native load control](https://knowledge.bigenius-x.com/generators?hsLang=en#load-data-with-a-native-load-control) # Databricks - Load data - Native Load Control **Applies to** Platform: Databricks Load the deployed Databricks solution with the native load control: a notebook runs every loader notebook in dependency order, four at a time. Run `_MultithreadingLoadExecution` once or schedule it as a Databricks job. Afterwards the log table `execution_log` lists every loader run. The same steps apply with Unity Catalog and with the Hive metastore. To load with Apache Airflow instead, see [Databricks - Load data - Apache Airflow Load Control](https://knowledge.bigenius-x.com/databricks-load-data-apache-airflow-load-control?hsLang=en). ### Prerequisites - [LoadControl Options](https://knowledge.bigenius-x.com/loadcontrol-options-project?hsLang=en) contains `Multi-Threading Notebook`, the default, and [Deploy LoadControl](https://knowledge.bigenius-x.com/deploy-loadcontrol-project?hsLang=en) is `true`. - The artifacts are [deployed](https://knowledge.bigenius-x.com/databricks-deployment?hsLang=en) without errors into a workspace folder. - A running all-purpose cluster, see [Databricks - Target environment with Unity Catalog](https://knowledge.bigenius-x.com/databricks-unity-target-environment?hsLang=en) or [Databricks - Target environment with the Hive metastore](https://knowledge.bigenius-x.com/databricks-target-environment?hsLang=en). ### Load the data 1. In the workspace folder of the artifacts, open `LoadControl/_MultithreadingLoadExecution`. 2. Attach the cluster and choose Run all. 3. Wait until the notebook ends with `all tasks completed successfully`. It prints one line per loader when the loader starts and finishes. The notebook takes its start time as the load timestamp and passes it to every loader, so all rows of a run get the same `bg_loadtimestamp`. A loader runs after the loaders of its sources. When a loader fails, the notebook skips all loaders that depend on it. To load on a schedule, create a job in Jobs & Pipelines with a notebook task for this notebook on the cluster, and add a schedule trigger. ### Check the result 1. Open `Helpers/_SelectResults` and run it. It shows the row count and the latest load timestamp of each result view. 2. Query the log of the load control, with your catalog and load control schema: ``` SELECT log_level, task_name, message FROM ..execution_log WHERE task_name = 'main' OR log_level = 'ERROR' ORDER BY log_timestamp; ``` A successful load ends with the row `main`, `tasks completed successfully`. With the Hive metastore, name the tables `.` instead. ### Example A Stage Files project with eight loaders writes its tables to the schema `stage` of the example catalog `dwh`. `stageparquetdatabricks_SelectResults` then shows, shortened and split in two blocks: ``` |table_name |row_count| |`dwh`.`stage`.`stg_st_branch_result` |3 | |`dwh`.`stage`.`stg_st_customer_result`|3 | ``` ``` |table_name |bg_loadtimestamp | |`dwh`.`stage`.`stg_st_branch_result` |2026-11-03 04:09:15.935295| |`dwh`.`stage`.`stg_st_customer_result`|2026-11-03 04:09:15.935295| ``` Both result views show the same load timestamp, the start time of the load notebook. ### Load part of a project To load part of a project, [generate](https://knowledge.bigenius-x.com/generate-artifacts?hsLang=en) the artifacts for a [Dataflow Modeling View](https://knowledge.bigenius-x.com/create-a-modeling-view?hsLang=en), deploy them and run their load notebook. The artifacts of a view contain only its model objects. Several views of one project can be loaded in order with one job: 1. Put each model object in one view only. A model object that depends on another one is in the same view or in a view loaded later. For example, the view `Order` depends on `CreditCard` and `Customer`: ![View flow in which the views CreditCard and Customer both lead to the view Order](https://knowledge.bigenius-x.com/hs-fs/hubfs/image-png-Dec-01-2025-09-30-06-0649-AM.png?width=290&height=153&name=image-png-Dec-01-2025-09-30-06-0649-AM.png) 2. Generate and deploy each view into its own workspace folder. Their notebooks have the same names. 3. Deploy the load control only once, with the first view; for the other views, run `.\deploy.ps1 -SkipDeployment` and then `Jupyter/_Deployment`. Each deployment of the load control replaces the log tables `execution_log` and `deployment_log`. Give each project its own load control schema (placeholder `loadcontrol#loadcontrol#schema_name`). With the default Deployment Options, a project that deploys the load control into a shared schema empties the logs of the other projects. 4. In Jobs & Pipelines, create a job with one notebook task per view. Each task runs the load notebook of its view folder on your cluster: ![Job task form with the fields Task name, Type Notebook, Source Workspace, Path and Compute](https://knowledge.bigenius-x.com/hs-fs/hubfs/image-png-Dec-01-2025-09-12-16-9384-AM.png?width=640&height=177&name=image-png-Dec-01-2025-09-12-16-9384-AM.png) 5. In Depends on, select the tasks of the views it depends on, for example `Load_View_Creditcard` and `Load_View_Customer` for `Load_View_Order`: ![Depends on field with the task Load\_View\_Creditcard selected](https://knowledge.bigenius-x.com/hs-fs/hubfs/image-png-Dec-01-2025-09-17-02-4632-AM.png?width=640&height=36&name=image-png-Dec-01-2025-09-17-02-4632-AM.png) 6. Choose Run now. A task starts when the tasks it depends on have succeeded. ### Failure symptoms and recovery #### The load notebook ends with failed tasks Message: ` tasks failed during execution. Please check the logs for more details.` Cause: A loader notebook failed; the notebook skipped the loaders that depend on it. Fix: Find the loader with `task failed` in `execution_log` and open its notebook run in the output. Correct the cause and run the load notebook again. #### The load notebook does not find a loader Message: `task could not be found when running command` in `execution_log` Cause: The loader notebooks are not in the folder of `loader_folder_path`, `../Jupyter/` by default. Fix: Upload the whole artifacts folder, or correct `loader_folder_path` in `replacement_config.json`, unzip the artifacts again, replace the placeholders and upload them with `.\deploy.ps1 -SkipDeployment`. For failures in other steps, see [Troubleshoot generation, deployment and load failures](https://knowledge.bigenius-x.com/troubleshoot-generation-deployment-and-load-failures?hsLang=en). ### Release differences #### Sequential Notebook Generator 1.8 offered the LoadControl Options value `Sequential Notebook`. It generated `_SimpleLoadExecution.ipynb`, which ran the loaders one after the other from the dependencies file `bg__dependencies.json`. Since Generator 1.9 only Multi-Threading Notebook and Airflow are offered. ### Related articles - Previous: [Databricks - Deployment](https://knowledge.bigenius-x.com/databricks-deployment?hsLang=en) - Reference: [LoadControl Options (Project)](https://knowledge.bigenius-x.com/loadcontrol-options-project?hsLang=en), [Deploy LoadControl (Project)](https://knowledge.bigenius-x.com/deploy-loadcontrol-project?hsLang=en), [Logging Method (Project)](https://knowledge.bigenius-x.com/logging-method-project?hsLang=en), [Databricks - Placeholder values](https://knowledge.bigenius-x.com/databricks-placeholder-values?hsLang=en) - See also: [Understand Load Control](https://knowledge.bigenius-x.com/understand-load-control?hsLang=en), [Native - Load control environment](https://knowledge.bigenius-x.com/native-load-control-environment?hsLang=en), [Databricks - Load data - Apache Airflow Load Control](https://knowledge.bigenius-x.com/databricks-load-data-apache-airflow-load-control?hsLang=en), [Databricks - Artifacts](https://knowledge.bigenius-x.com/databricks-artifacts?hsLang=en), [Use Sequential Dataflow Execution (Model Object)](https://knowledge.bigenius-x.com/use-sequential-dataflow-execution-model-object?hsLang=en) - [Getting started](https://knowledge.bigenius-x.com/getting-started?hsLang=en#main-content) - [Guidelines](https://knowledge.bigenius-x.com/getting-started?hsLang=en#guidelines) - [Start working](https://knowledge.bigenius-x.com/getting-started?hsLang=en#start-working) - [Account](https://knowledge.bigenius-x.com/getting-started?hsLang=en#account) - [General overview](https://knowledge.bigenius-x.com/general-overview?hsLang=en) - [Artificial Intelligence](https://knowledge.bigenius-x.com/artificial-intelligence?hsLang=en) - [Application Modules](https://knowledge.bigenius-x.com/application-modules?hsLang=en#main-content) - [Administration](https://knowledge.bigenius-x.com/application-modules?hsLang=en#administration) - [Solutions](https://knowledge.bigenius-x.com/application-modules?hsLang=en#solutions) - [Global Features](https://knowledge.bigenius-x.com/application-modules?hsLang=en#global-features) - [Projects](https://knowledge.bigenius-x.com/application-modules?hsLang=en#projects) - [Branches](https://knowledge.bigenius-x.com/application-modules?hsLang=en#branches) - [Data Connections](https://knowledge.bigenius-x.com/application-modules?hsLang=en#data-connections) - [Dataflow Modeling - Overview](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-overview) - [Dataflow Modeling - Wizard Steps](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-wizard-steps) - [Dataflow Modeling - Terms](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-terms) - [Dataflow Modeling - Term Mapping](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-term-mapping) - [Dataflow Modeling - Relationships](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-relationships) - [Dataflow Modeling - Data Quality](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-data-quality) - [Dataflow Modeling - Indexes](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-indexes) - [Relationship Modeling](https://knowledge.bigenius-x.com/application-modules?hsLang=en#relationship-modeling) - [Generate Artifacts](https://knowledge.bigenius-x.com/application-modules?hsLang=en#generate-artifacts) - [Project Settings](https://knowledge.bigenius-x.com/application-modules?hsLang=en#project-settings) - [Data Marketplace](https://knowledge.bigenius-x.com/application-modules?hsLang=en#data-marketplace) - [Generators](https://knowledge.bigenius-x.com/generators?hsLang=en#main-content) - [Fabric Warehouse](https://knowledge.bigenius-x.com/generators?hsLang=en#fabric-warehouse) - [Fabric Lakehouse](https://knowledge.bigenius-x.com/generators?hsLang=en#fabric-lakehouse) - [Databricks](https://knowledge.bigenius-x.com/generators?hsLang=en#databricks) - [Snowflake](https://knowledge.bigenius-x.com/generators?hsLang=en#snowflake) - [Microsoft SQL Server](https://knowledge.bigenius-x.com/generators?hsLang=en#microsoft-sql-server) - [Oracle](https://knowledge.bigenius-x.com/generators?hsLang=en#oracle) - [Artifacts](https://knowledge.bigenius-x.com/generators?hsLang=en#artifacts) - [Replace Placeholders](https://knowledge.bigenius-x.com/generators?hsLang=en#replace-placeholders) - [Target solution environment](https://knowledge.bigenius-x.com/generators?hsLang=en#target-solution-environment) - [Deployment](https://knowledge.bigenius-x.com/generators?hsLang=en#deployment) - [Deployment with an Azuze DevOps pipeline](https://knowledge.bigenius-x.com/generators?hsLang=en#deployment-with-an-azuze-devops-pipeline) - [Delta Deployment](https://knowledge.bigenius-x.com/generators?hsLang=en#delta-deployment) - [Load control environment](https://knowledge.bigenius-x.com/generators?hsLang=en#load-control-environment) - [Load data with a native load control](https://knowledge.bigenius-x.com/generators?hsLang=en#load-data-with-a-native-load-control) - [Load data with Apache Airflow](https://knowledge.bigenius-x.com/generators?hsLang=en#load-data-with-apache-airflow) - [Model Object Type](https://knowledge.bigenius-x.com/generators?hsLang=en#model-object-type) - [Properties](https://knowledge.bigenius-x.com/generators?hsLang=en#properties) - [Default Terms](https://knowledge.bigenius-x.com/generators?hsLang=en#default-terms) - [Discovery application](https://knowledge.bigenius-x.com/discovery-application?hsLang=en#main-content) - [Discovery configurations](https://knowledge.bigenius-x.com/discovery-application?hsLang=en#discovery-configurations) - [Best practices](https://knowledge.bigenius-x.com/best-practices?hsLang=en#main-content) - [Modeling Approaches](https://knowledge.bigenius-x.com/best-practices?hsLang=en#modeling-approaches) - [Use Cases](https://knowledge.bigenius-x.com/best-practices?hsLang=en#use-cases) - [Business Rules](https://knowledge.bigenius-x.com/best-practices?hsLang=en#business-rules) - [Data Quality Rules](https://knowledge.bigenius-x.com/best-practices?hsLang=en#data-quality-rules) - [FAQs](https://knowledge.bigenius-x.com/faqs?hsLang=en) - [Product Release Notes](https://knowledge.bigenius-x.com/product-release-notes?hsLang=en#main-content) - [SaaS Application](https://knowledge.bigenius-x.com/product-release-notes?hsLang=en#saas-application) - [Discovery application](https://knowledge.bigenius-x.com/product-release-notes?hsLang=en#discovery-application) - [Generators](https://knowledge.bigenius-x.com/product-release-notes?hsLang=en#generators) - [Legal Documents](https://knowledge.bigenius-x.com/legal-documents?hsLang=en#main-content) - [Current legal docs](https://knowledge.bigenius-x.com/legal-documents?hsLang=en#current-legal-docs) - [Software Product and Limits](https://knowledge.bigenius-x.com/legal-documents?hsLang=en#software-product-and-limits) - [Contact us](https://knowledge.bigenius-x.com/kb-tickets/new?hsLang=en) - [Customer Portal](https://knowledge.bigenius-x.com/customer-portal) - [Sign in](https://knowledge.bigenius-x.com/_hcms/mem/login) [![biGENIUS logo](https://knowledge.bigenius-x.com/hubfs/TRI_Logo_biGENIUS_2020_RGB-1.svg "biGENIUS logo")](https://www.bigenius-x.com) All rights reserved © 2025, biGENIUS AG