Artifacts
      Back to home
      1. biGENIUS-X Knowledge Base
      2. Generators
      3. Artifacts

      Snowflake - Artifacts - 1.5

      With the download of the generated Artifacts, a zip file was created in the default download location set for your Web Browser as:

      Let's see what this zip file contains now.

      Generated Artifacts content

      The Generated Artifacts zip file contains the following:

      • A LoadControl folder
      • A SQL folder
      • The placeholder toolkit is in the replace_placeholders.ps1 file
      • The placeholder configuration is in the replacement_config.json file
      • A ParseError.log file (optional)

      LoadControl folder

      If the Property Configure LoadControl is set to False, the LoadControl folder will not be generated.

      The LoadControl folder contains:

      • The Load Control creation script, which contains all the needed SQL code to create the Load Control structure of your Target System composed of Snowflake Tasks

      Snowflake tasks are objects used to execute SQL commands or stored procedures on a scheduled basis.

      • The Load Control creation script for Stream, which contains all the needed SQL code to create the Load Control structure of your Target System composed of Snowflake Streams

      Snowflake streams are objects that track and record changes (inserts, updates, deletes) made to tables over time. They enable "change data capture" (CDC), allowing you to query the sequence of changes to a table.

      • A Python file for a load with Airflow

       

      As you can observe, placeholders are used for the target database name and schemas. Before deploying the generated code in your target environment, please Replace the placeholders in the Generated Artifacts.

      SQL folder

      The SQL folder contains one sql file by Layer configured in the Generator.

      In this example, four layers are configured for the Generator:

      1. Stage layer
      2. Raw Vault layer
      3. Business Vault Layer
      4. Mart

      Each file contains all the SQL code needed to create the tables, views, and stored procedures that make up the structure of your Target System.

      Example of a part of a SQL file:

      ...
      CREATE TRANSIENT TABLE IDENTIFIER('{SNOWFLAKEXXXXDATAVAULTANDMART#STAGE#SCHEMA_NAME}.STG_ST_CREDITCARD') (
           BG_LOADTIMESTAMP TIMESTAMP_TZ NOT NULL
          ,BG_SOURCESYSTEM VARCHAR NULL
          ,CREDITCARDID NUMBER(38) NULL
          ,CARDTYPE VARCHAR() NULL
          ,CARDNUMBER VARCHAR() NULL
          ,EXPMONTH NUMBER(38,0) NULL
          ,EXPYEAR NUMBER(38,0) NULL
          ,MODIFIEDDATE TIMESTAMP_TZ NULL
      )
      ;
      ...

      As you can observe, a placeholder is used for the target database name and schema.

      Before you deploy the generated code in your target environment, please Replace the placeholders in the Generated Artifacts.

      ParseError.log file

      If some errors occur during the generation of your Artifacts, the file ParseError.log will contain these errors.

      Example of error:

      ERROR LinkSourceView: Customer_Order_Link_link source view_1 ### TargetName: RDV_LNK_Customer_Order_Link_Source
      Invalid amount of dataflows for ArchitectureModelObject name Customer_Order_Link_link source view_1
      System.Exception: Invalid amount of dataflows for ArchitectureModelObject name Customer_Order_Link_link source view_1
         at biGENIUS.Implementation.Base.DataflowBuilder.Build(ArchitectureModelObject architectureModelObject) in C:\.sources\bgaas\bigenius.implementation.base\src\biGENIUS.Implementation.Base\StatementBuilder\DataflowBuilder.cs:line 18
         at biGENIUS.Implementation.Datavault.Mssql.SourceViewBuilder.Build(ArchitectureModelObject architectureModelObject) in C:\.sources\bgaas\bigenius.implementation.datavault.mssql\src\biGENIUS.Implementation.Datavault.Mssql\StatementBuilder\SourceViewBuilder.cs:line 22
         at biGENIUS.Implementation.Datavault.Mssql.LinkSourceView.BuildScriptInternal() in C:\.sources\bgaas\bigenius.implementation.datavault.mssql\src\biGENIUS.Implementation.Datavault.Mssql\Template\LinkSourceView.cs:line 16
         at biGENIUS.Implementation.Base.TemplateScriptBase.BuildScript(IGeneratorContext generatorContext, ArchitectureModelObject architectureModelObject, TemplateRenderingContext templateRenderingContext, StatementBuilderContext statementBuilderContext) in C:\.sources\bgaas\bigenius.implementation.base\src\biGENIUS.Implementation.Base\TemplateBase\TemplateScriptBase.cs:line 45
         at biGENIUS.Implementation.Base.GeneratorHookBase.RenderTemplate(IGeneratorContext generatorContext, Assembly assembly, ArchitectureModelObject amo) in C:\.sources\bgaas\bigenius.implementation.base\src\biGENIUS.Implementation.Base\Hook\GeneratorHookBase.cs:line 169

      In this case, no Dataflow was configured for the Link named Customer_Order_Link.

      So, please now define the dataflow and generate a new time.

      If no errors occur during generation, the file ParseError.log will not exist in the Generated Artifact zip file.

      Placeholder toolkit

      It is composed when you deploy PowerShell script: replace_placeholders.ps1.

      It uses a placeholder configuration file to replace the connection needed information in the generated artifacts.

      For more information, please review Replace the placeholders in the Generated Artifacts.

      Placeholder configuration

      It is composed of a JSON file: replacement_config.json.

      The placeholder toolkit uses it to replace the connection needed information in the generated artifacts.

      For more information, please review Replace the placeholders in the Generated Artifacts.