The Generated Artifacts zip file contains files with placeholders for the target environment parameters.
You should replace all the placeholders to deploy your Azure Synapse Target System.
Replace the Placeholders
A toolkit is provided in the generated artifacts to replace the placeholders.
It is composed of a Powershell script to execute: replace_placeholders.ps1.
To execute commands with Powershell later in this article, please use at least version 7 provided by Microsoft. If you would like to install the latest version, please check here.
The process to follow is:
- Update the replacement_config.json file with your values:
- Insert the value to replace into the value node
- Each database_name is filled by default with the layer name (example: rawvault)
- It should contain the database name of the Target Data Lake
- In our example, we want to create the target tables in the Data Lake rawvault
- It should contain the database name of the Target Data Lake
- Each database_directory placeholder should contain the path to create the Target Parquet files.
- If you are using our Azure Synapse environment example, the path is the one inside the Storage Account chosen to store the Target Data Lake
abfss://claire-datalake@bgsynapsedatalake1.dfs.core.windows.net/rawvault/ for the Raw Vault layer placeholder- claire-datalake is the name of the folder we created inside the Storage Account
- bgsynapsedatalake1 is the name of the Storage Account itself
- If you are using our Azure Synapse environment example, the path is the one inside the Storage Account chosen to store the Target Data Lake
All the paths should have a slash appended at the end: abfss://claire-datalake@bgsynapsedatalake1.dfs.core.windows.net/rawvault/
Depending on your source:
-
- Linked Project (from a Stage JDBC or a Stage File Project):
- Each database_name placeholder should contain the database name of the source table created before with the Linked Project
- In our example, it is stage
- Each database_name placeholder should contain the database name of the source table created before with the Linked Project
- Direct source (External table):
-
- Each database_name placeholder should contain the database name of the external table, which contains the source data
-
- In our example, it is stage
-
- Each database_name placeholder should contain the database name of the external table, which contains the source data
-
- Linked Project (from a Stage JDBC or a Stage File Project):
Replace the placeholders in the files:
- Open Windows Powershell (or equivalent) in the replace_placeholders.ps1 location:
- Execute the following command :
-
.\replace_placeholders.ps1
- You should have a similar result:
- The configured values in all generated artifacts replaced the placeholders
- You can now use these files and deploy your Target system
Some parameters can be added to the replace_placeholders.ps1 command.
All are described and available by executing:
.\replace_placeholders.ps1 - help
The -ReplacementConfigPath parameter mainly permits using a replacement_config.json file in another path. It is beneficial when you are in a development mode for your project.
Example of usage:
.\replace_placeholders.ps1 -ReplacementConfigPath "C:\TEMP\Replacement config files\replacement_config_
If you run into any issues, check our FAQs section for Generate Artifacts: