Snowflake - Target environment
This article outlines a potential setup for a target environment for Snowflake generators.
Please note that the installation and configuration of this target environment fall outside the scope of biGENIUS support.
We will not offer assistance beyond the provided example in this article.
A Snowflake target environment has numerous alternative configurations and installations.
Below is a potential target environment setup for a Snowflake generator.
Setup environment
The Snowflake target environment needs at least the following items configured into your Snowflake account:
- (Optional) A target Warehouse WH_XXX
- You can also work into the default Snowflake Warehouse COMPUTE_WH
- A source Database SD_XXX into the target Warehouse WH_XXX (or COMPUTE_WH)
- A target Database TD_XXX into the target Warehouse WH_XXX (or COMPUTE_WH)
- A Role R_XXX with the following privileges on the source database SD_XXX and the target database TD_XXX: OWNERSHIP, CREATE SCHEMA, MODIFY, MONITOR, USAGE, and EXECUTE TASK
For example, you can add the EXECUTE TASK privilege for the Role R_XXX with the following script:
GRANT EXECUTE TASK ON ACCOUNT TO ROLE R_XXX;
- A User XXX who can access the role R_XXX
Source data
There are two ways to provide source data to a Snowflake generator:
- From data in existing tables in a Snowflake database
- From Parquet and Delta files by using the Snowflake Stage Files generator as a Linked Project
Parquet and Delta Files
If your source data is stored in Parquet or Delta files, please:
- Create a first Project with the Snowflake Stage Files generator
- In this first Project, discover the Parquet and Delta files, create the Stage Model Object, generate, deploy, and load data in the target storage account.
- Create a second Project with the Snowflake Data Vault or Snowflake DataVault and Mart generators.
- In this second Project, use the first Project Stage Model Object as a source by using the Linked Project feature.
Globally, the source data is loaded according to the following architecture:
The following Azure service is required:
- Azure Blob Storage: Azure Blob Storage helps you create data lakes for your analytics needs, and provides storage to build powerful cloud-native and mobile apps..
If you have already discovered your source data, modeled your project, and generated the artifacts, you're now ready to replace the placeholders in your generated artifacts, deploy these artifacts, and subsequently load the data based on the Generator you are using with the following possible load controls: