--- title: Databricks - Target environment with Unity Catalog description: A Databricks target environment with Unity Catalog needs a workspace enabled for Unity Catalog, a catalog, an all-purpose cluster and the Databricks CLI. --- [Skip to content](https://knowledge.bigenius-x.com/databricks-target-environment-with-unity-catalog#main-content) [Contact us](https://knowledge.bigenius-x.com/kb-tickets/new?hsLang=en) [Go to Customer Portal](https://knowledge.bigenius-x.com/customer-portal?hsLang=en) [![TRI\_Logo\_biGENIUS\_2020\_RGB\_s](https://knowledge.bigenius-x.com/hs-fs/hubfs/Marketing/TRI_Logo_biGENIUS_2020_RGB_s.png?width=150&height=42&name=TRI_Logo_biGENIUS_2020_RGB_s.png)](https://www.bigenius-x.com/) Open main navigation Close main navigation - [Contact us](https://knowledge.bigenius-x.com/kb-tickets/new) - [Go to Customer Portal](https://knowledge.bigenius-x.com/customer-portal) How can we help you? - There are no suggestions because the search field is empty. 1. [biGENIUS-X Knowledge Base](https://knowledge.bigenius-x.com/?hsLang=en) 2. [Generators](https://knowledge.bigenius-x.com/generators?hsLang=en) 3. [Target solution environment](https://knowledge.bigenius-x.com/generators?hsLang=en#target-solution-environment) # Databricks - Target environment with Unity Catalog **Applies to** Platform: Databricks A Databricks target environment with Unity Catalog needs a workspace enabled for Unity Catalog, a catalog, an all-purpose cluster and the Databricks CLI. Prepare it before you replace the placeholders and deploy the artifacts of a Databricks generator. Afterwards `deploy.ps1` can upload the notebooks and run the deployment on the cluster. This article applies when the project property [Use Unity Catalog](https://knowledge.bigenius-x.com/use-unity-catalog-project?hsLang=en) is selected. For projects without it, see [Databricks - Target environment with the Hive metastore](https://knowledge.bigenius-x.com/databricks-target-environment-with-the-hive-metastore?hsLang=en). biGENIUS support does not cover the installation and configuration of Databricks beyond the example in this article. ### Prerequisites - A Databricks workspace enabled for Unity Catalog, for example an Azure Databricks workspace. - A user who can create schemas in the target catalog, for example as its owner, and who can create an all-purpose cluster. - Use Unity Catalog is selected in Project Settings, the default since Generator 2.0: ![Project property Use Unity Catalog selected](https://knowledge.bigenius-x.com/hs-fs/hubfs/image-png-Sep-09-2024-08-29-03-7957-AM.png?width=640&height=40&name=image-png-Sep-09-2024-08-29-03-7957-AM.png) - PowerShell 7 on the deploying machine. `deploy.ps1` requires it. ### Prepare the catalog 1. Create a catalog, for example in Catalog Explorer or with `CREATE CATALOG datalakehouse;`, or choose an existing one. The deployment creates the schemas of the layers and of the load control, but not the catalog. 2. Enter the catalog in the `database_name` placeholders and the schemas in the `schema_name` placeholders, see [Databricks - Placeholder values](https://knowledge.bigenius-x.com/databricks-placeholder-values?hsLang=en). The layer schemas are prefilled with the layer names. If the load control schema is empty, enter one. ### Create the cluster `deploy.ps1` runs the deployment on an all-purpose cluster that you name, and the Airflow jobs run on an existing cluster too. 1. In the workspace, open Compute and choose Create compute. 2. Enter a name, for example `dwh_cluster`, and set: - Access mode: Dedicated (single user) or Standard. - Databricks runtime version: 18 LTS. - Single node with a node type such as `Standard_DS3_v2`. 3. Choose Create compute and wait until the cluster is running. With Databricks Runtime 17.3 LTS, 16.4 LTS or 15.4 LTS, the deployment of tables with a `UNIQUE` constraint, such as hubs, fails while [Use Unique Constraints](https://knowledge.bigenius-x.com/use-unique-constraints-project?hsLang=en) is selected, see [Databricks - Deployment](https://knowledge.bigenius-x.com/databricks-deployment?hsLang=en#failure-symptoms-and-recovery). ### Install the Databricks CLI 1. Install the [Databricks CLI](https://docs.databricks.com/aws/en/dev-tools/cli/install), for example with `winget install Databricks.DatabricksCLI`. 2. Open a new PowerShell 7 window and run `databricks -v`. The CLI prints its version. On its first run, `deploy.ps1` creates a CLI profile and signs you in through the browser, see [Databricks - Deployment](https://knowledge.bigenius-x.com/databricks-deployment?hsLang=en). ### Provide the source data Which source system types a generator configuration accepts is listed in [Source systems per generator configuration](https://knowledge.bigenius-x.com/source-systems-per-generator-configuration?hsLang=en). To create source model objects from tables of a catalog, see [Discover a Databricks Unity Catalog](https://knowledge.bigenius-x.com/discover-a-databricks-unity-catalog?hsLang=en). A Data Mart project has no discovery; it uses model objects of linked projects, see [Manage Linked Projects](https://knowledge.bigenius-x.com/manage-linked-projects?hsLang=en). #### Files with Stage Files Parquet, Delta or CSV files are loaded by a separate Databricks Stage Files project. A second project, for example a Data Vault project, then uses its Stage model objects as a linked project. 1. Upload the files into a Unity Catalog volume, one folder per source model object, for example `/Volumes////sales/customer/`. 2. Enter the parent folder with a trailing slash as `file_path`, here `/Volumes////sales/`. The loader reads the folder named like the source model object below it. A volume needs no storage settings on the cluster. #### Databases with Stage JDBC Tables of MariaDB, Microsoft SQL Server, MySQL, Oracle or PostgreSQL are loaded by a separate Databricks Stage JDBC project, used as a linked project in the same way. The cluster must reach the database. For Azure SQL Database, select Allow Azure services and resources to access this server in the networking settings of the server: ![Azure SQL server networking exception Allow Azure services and resources to access this server selected](https://knowledge.bigenius-x.com/hs-fs/hubfs/image-png-Feb-22-2024-08-57-29-5471-AM.png?width=389&height=77&name=image-png-Feb-22-2024-08-57-29-5471-AM.png) The cluster has the JDBC drivers of Microsoft SQL Server, MariaDB and PostgreSQL. For Oracle and MySQL, install the JDBC driver as a library of the cluster, for example from Maven: `com.oracle.database.jdbc:ojdbc11` or `com.mysql:mysql-connector-j`, and enter its driver class in `jdbc_driver` (`oracle.jdbc.OracleDriver`, `com.mysql.cj.jdbc.Driver`). Enter the connection in the `jdbc_url`, `jdbc_driver`, `jdbc_user` and `jdbc_password` placeholders, and the source schema in `schema_name` of the source system. ### Check the result In a notebook attached to the cluster, run `SHOW SCHEMAS IN ;`. It lists the schemas of the catalog without an error. For Stage Files, `LIST ''` lists one folder per source model object. ### Failure symptoms and recovery #### The deployment script does not find the Databricks CLI Message: `Databricks CLI is not installed, not recognized or the PATH environment variable is not properly configured.` Cause: The CLI is not installed, or its folder is not in `PATH` of the PowerShell window. Fix: Install the CLI, open a new PowerShell 7 window and check `databricks -v`. For failures in other steps, see [Troubleshoot generation, deployment and load failures](https://knowledge.bigenius-x.com/troubleshoot-generation-deployment-and-load-failures?hsLang=en). ### Release differences - Use Unity Catalog is selected by default since Generator 2.0. Up to Generator 1.10 it was cleared by default. - Since Generator 2.2, the deployment creates the schemas with Generate Create Schema If Not Exists Statements, a default of [Deployment Options](https://knowledge.bigenius-x.com/deployment-options-project?hsLang=en). With older artifacts, create the schemas yourself. - Up to Generator 1.9, a Stage JDBC project took the source schema from the model object name, for example `Sales.CreditCard`. Since Generator 1.10 it comes from `schema_name` of the source system. ### Related articles - Next: [Databricks - Placeholder values](https://knowledge.bigenius-x.com/databricks-placeholder-values?hsLang=en), [Databricks - Deployment](https://knowledge.bigenius-x.com/databricks-deployment?hsLang=en) - Reference: [Use Unity Catalog (Project)](https://knowledge.bigenius-x.com/use-unity-catalog-project?hsLang=en), [Use Unique Constraints (Project)](https://knowledge.bigenius-x.com/use-unique-constraints-project?hsLang=en), [Databricks - Stage Files](https://knowledge.bigenius-x.com/databricks-stage-files?hsLang=en), [Databricks - Stage JDBC](https://knowledge.bigenius-x.com/databricks-stage-jdbc?hsLang=en), [Source systems per generator configuration](https://knowledge.bigenius-x.com/source-systems-per-generator-configuration?hsLang=en) - See also: [Databricks - Target environment with the Hive metastore](https://knowledge.bigenius-x.com/databricks-target-environment-with-the-hive-metastore?hsLang=en), [Databricks - Artifacts](https://knowledge.bigenius-x.com/databricks-artifacts?hsLang=en), [Discover a Databricks Unity Catalog](https://knowledge.bigenius-x.com/discover-a-databricks-unity-catalog?hsLang=en), [Manage Linked Projects](https://knowledge.bigenius-x.com/manage-linked-projects?hsLang=en), [Native - Load control environment](https://knowledge.bigenius-x.com/native-load-control-environment?hsLang=en), [Apache Airflow - Load control environment](https://knowledge.bigenius-x.com/apache-airflow-load-control-environment?hsLang=en), [Databricks dbt - Target environment](https://knowledge.bigenius-x.com/databricks-dbt-target-environment?hsLang=en) - [Getting started](https://knowledge.bigenius-x.com/getting-started?hsLang=en#main-content) - [Guidelines](https://knowledge.bigenius-x.com/getting-started?hsLang=en#guidelines) - [Start working](https://knowledge.bigenius-x.com/getting-started?hsLang=en#start-working) - [Account](https://knowledge.bigenius-x.com/getting-started?hsLang=en#account) - [General overview](https://knowledge.bigenius-x.com/general-overview?hsLang=en) - [Artificial Intelligence](https://knowledge.bigenius-x.com/artificial-intelligence?hsLang=en) - [Application Modules](https://knowledge.bigenius-x.com/application-modules?hsLang=en#main-content) - [Administration](https://knowledge.bigenius-x.com/application-modules?hsLang=en#administration) - [Solutions](https://knowledge.bigenius-x.com/application-modules?hsLang=en#solutions) - [Global Features](https://knowledge.bigenius-x.com/application-modules?hsLang=en#global-features) - [Projects](https://knowledge.bigenius-x.com/application-modules?hsLang=en#projects) - [Branches](https://knowledge.bigenius-x.com/application-modules?hsLang=en#branches) - [Data Connections](https://knowledge.bigenius-x.com/application-modules?hsLang=en#data-connections) - [Dataflow Modeling - Overview](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-overview) - [Dataflow Modeling - Wizard Steps](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-wizard-steps) - [Dataflow Modeling - Terms](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-terms) - [Dataflow Modeling - Term Mapping](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-term-mapping) - [Dataflow Modeling - Relationships](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-relationships) - [Dataflow Modeling - Data Quality](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-data-quality) - [Dataflow Modeling - Indexes](https://knowledge.bigenius-x.com/application-modules?hsLang=en#dataflow-modeling-indexes) - [Relationship Modeling](https://knowledge.bigenius-x.com/application-modules?hsLang=en#relationship-modeling) - [Generate Artifacts](https://knowledge.bigenius-x.com/application-modules?hsLang=en#generate-artifacts) - [Project Settings](https://knowledge.bigenius-x.com/application-modules?hsLang=en#project-settings) - [Data Marketplace](https://knowledge.bigenius-x.com/application-modules?hsLang=en#data-marketplace) - [Generators](https://knowledge.bigenius-x.com/generators?hsLang=en#main-content) - [Fabric Warehouse](https://knowledge.bigenius-x.com/generators?hsLang=en#fabric-warehouse) - [Fabric Lakehouse](https://knowledge.bigenius-x.com/generators?hsLang=en#fabric-lakehouse) - [Databricks](https://knowledge.bigenius-x.com/generators?hsLang=en#databricks) - [Snowflake](https://knowledge.bigenius-x.com/generators?hsLang=en#snowflake) - [Microsoft SQL Server](https://knowledge.bigenius-x.com/generators?hsLang=en#microsoft-sql-server) - [Oracle](https://knowledge.bigenius-x.com/generators?hsLang=en#oracle) - [Artifacts](https://knowledge.bigenius-x.com/generators?hsLang=en#artifacts) - [Replace Placeholders](https://knowledge.bigenius-x.com/generators?hsLang=en#replace-placeholders) - [Target solution environment](https://knowledge.bigenius-x.com/generators?hsLang=en#target-solution-environment) - [Deployment](https://knowledge.bigenius-x.com/generators?hsLang=en#deployment) - [Deployment with an Azuze DevOps pipeline](https://knowledge.bigenius-x.com/generators?hsLang=en#deployment-with-an-azuze-devops-pipeline) - [Delta Deployment](https://knowledge.bigenius-x.com/generators?hsLang=en#delta-deployment) - [Load control environment](https://knowledge.bigenius-x.com/generators?hsLang=en#load-control-environment) - [Load data with a native load control](https://knowledge.bigenius-x.com/generators?hsLang=en#load-data-with-a-native-load-control) - [Load data with Apache Airflow](https://knowledge.bigenius-x.com/generators?hsLang=en#load-data-with-apache-airflow) - [Model Object Type](https://knowledge.bigenius-x.com/generators?hsLang=en#model-object-type) - [Properties](https://knowledge.bigenius-x.com/generators?hsLang=en#properties) - [Default Terms](https://knowledge.bigenius-x.com/generators?hsLang=en#default-terms) - [Discovery application](https://knowledge.bigenius-x.com/discovery-application?hsLang=en#main-content) - [Discovery configurations](https://knowledge.bigenius-x.com/discovery-application?hsLang=en#discovery-configurations) - [Best practices](https://knowledge.bigenius-x.com/best-practices?hsLang=en#main-content) - [Modeling Approaches](https://knowledge.bigenius-x.com/best-practices?hsLang=en#modeling-approaches) - [Use Cases](https://knowledge.bigenius-x.com/best-practices?hsLang=en#use-cases) - [Business Rules](https://knowledge.bigenius-x.com/best-practices?hsLang=en#business-rules) - [Data Quality Rules](https://knowledge.bigenius-x.com/best-practices?hsLang=en#data-quality-rules) - [FAQs](https://knowledge.bigenius-x.com/faqs?hsLang=en) - [Product Release Notes](https://knowledge.bigenius-x.com/product-release-notes?hsLang=en#main-content) - [SaaS Application](https://knowledge.bigenius-x.com/product-release-notes?hsLang=en#saas-application) - [Discovery application](https://knowledge.bigenius-x.com/product-release-notes?hsLang=en#discovery-application) - [Generators](https://knowledge.bigenius-x.com/product-release-notes?hsLang=en#generators) - [Legal Documents](https://knowledge.bigenius-x.com/legal-documents?hsLang=en#main-content) - [Current legal docs](https://knowledge.bigenius-x.com/legal-documents?hsLang=en#current-legal-docs) - [Software Product and Limits](https://knowledge.bigenius-x.com/legal-documents?hsLang=en#software-product-and-limits) - [Contact us](https://knowledge.bigenius-x.com/kb-tickets/new?hsLang=en) - [Customer Portal](https://knowledge.bigenius-x.com/customer-portal) - [Sign in](https://knowledge.bigenius-x.com/_hcms/mem/login) [![biGENIUS logo](https://knowledge.bigenius-x.com/hubfs/TRI_Logo_biGENIUS_2020_RGB-1.svg "biGENIUS logo")](https://www.bigenius-x.com) All rights reserved © 2025, biGENIUS AG