Snowflake - Artifacts - 1.4

With the download of the generated Artifacts, a zip file was created in the default download location set for your Web Browser as:

Let's see what this zip file contains now.

Generated Artifacts content

The Generated Artifacts zip file contains the following:

• A LoadControl folder
• A SQL folder
• The placeholder toolkit is in the replace_placeholders.ps1 file
• The placeholder configuration is in the replacement_config.json file
• A ParseError.log file (optional)

LoadControl folder

The LoadControl folder contains:

• The Load Control creation script, which contains all the needed SQL code to create the Load Control structure of your Target System composed of Snowflake Tasks
  • As you can observe, placeholders are used for the target database name and schemas. Before deploying the generated code in your target environment, please Replace the placeholders in the Generated Artifacts.
• a Python file for a load with Airflow
  • If you prefer to load data with Airflow instead of natively with Snowflake

SQL folder

The SQL folder contains one sql file by Layer configured in the Generator.

In this example, four layers are configured for the Generator:

1. Stage layer
2. Raw Vault layer
3. Business Vault Layer
4. Mart

Each file contains all the needed SQL code to create the tables, views, and stored procedures to create the structure of your Target System.

Example of a part of a SQL file:

...
CREATE TRANSIENT TABLE IDENTIFIER('{SNOWFLAKE1.4.0DATAVAULTANDMART#STAGE#SCHEMA_NAME}.STG_ST_CREDITCARD') (
     BG_LOADTIMESTAMP TIMESTAMP_TZ NOT NULL
    ,BG_SOURCESYSTEM VARCHAR NULL
    ,CREDITCARDID NUMBER(38) NULL
    ,CARDTYPE VARCHAR() NULL
    ,CARDNUMBER VARCHAR() NULL
    ,EXPMONTH NUMBER(38,0) NULL
    ,EXPYEAR NUMBER(38,0) NULL
    ,MODIFIEDDATE TIMESTAMP_TZ NULL
)
;
...

As you can observe, a placeholder is used for the target database name and schema.

Before you deploy the generated code in your target environment, please Replace the placeholders in the Generated Artifacts.

ParseError.log file

If some errors occur during the generation of your Artifacts, the file ParseError.log will contain these errors.

Example of error:

ERROR LinkSourceView: Customer_Order_Link_link source view_1 ### TargetName: RDV_LNK_Customer_Order_Link_Source
Invalid amount of dataflows for ArchitectureModelObject name Customer_Order_Link_link source view_1
System.Exception: Invalid amount of dataflows for ArchitectureModelObject name Customer_Order_Link_link source view_1
   at biGENIUS.Implementation.Base.DataflowBuilder.Build(ArchitectureModelObject architectureModelObject) in C:\.sources\bgaas\bigenius.implementation.base\src\biGENIUS.Implementation.Base\StatementBuilder\DataflowBuilder.cs:line 18
   at biGENIUS.Implementation.Datavault.Mssql.SourceViewBuilder.Build(ArchitectureModelObject architectureModelObject) in C:\.sources\bgaas\bigenius.implementation.datavault.mssql\src\biGENIUS.Implementation.Datavault.Mssql\StatementBuilder\SourceViewBuilder.cs:line 22
   at biGENIUS.Implementation.Datavault.Mssql.LinkSourceView.BuildScriptInternal() in C:\.sources\bgaas\bigenius.implementation.datavault.mssql\src\biGENIUS.Implementation.Datavault.Mssql\Template\LinkSourceView.cs:line 16
   at biGENIUS.Implementation.Base.TemplateScriptBase.BuildScript(IGeneratorContext generatorContext, ArchitectureModelObject architectureModelObject, TemplateRenderingContext templateRenderingContext, StatementBuilderContext statementBuilderContext) in C:\.sources\bgaas\bigenius.implementation.base\src\biGENIUS.Implementation.Base\TemplateBase\TemplateScriptBase.cs:line 45
   at biGENIUS.Implementation.Base.GeneratorHookBase.RenderTemplate(IGeneratorContext generatorContext, Assembly assembly, ArchitectureModelObject amo) in C:\.sources\bgaas\bigenius.implementation.base\src\biGENIUS.Implementation.Base\Hook\GeneratorHookBase.cs:line 169

In this case, no Dataflow was configured for the Link named Customer_Order_Link.

So, please now define the dataflow and generate a new time.

Placeholder toolkit

It is composed when you deploy PowerShell script: replace_placeholders.ps1.

It uses a placeholder configuration file to replace the connection needed information in the generated artifacts.

For more information, please review Replace the placeholders in the Generated Artifacts.

Placeholder configuration

It is composed of a JSON file: replacement_config.json.

The placeholder toolkit uses it to replace the connection needed information in the generated artifacts.

For more information, please review Replace the placeholders in the Generated Artifacts.