Oracle - Stage

Applies to
Platform: Oracle
Architecture: Stage

Oracle - Stage generates a Stage layer for an Oracle database as SQL*Plus scripts with PL/SQL loader packages. Select it in Project Settings, where Generator Configuration shows Oracle Stage, when your Stage layer runs on Oracle. This article lists what the configuration supports and where it differs from Microsoft SQL Server - Stage.

Scope and output

Platform: Oracle. The configuration is available since Generator 2.0.
Layers: Source and Stage. Dataflow Modeling shows Stage in the Bronze column. See Understand the Layering concept.
Output: a SQL*Plus script for the Stage layer, PL/SQL loader packages, the load control scripts and the PowerShell 7 scripts that deploy and load, described in Oracle - Artifacts.

Prerequisites

One Oracle user for the Stage layer and one for load control (with Native Load Control). The user name is also the schema name of the layer.
PowerShell 7 and SQL*Plus on the machine that deploys.
The privileges listed in Oracle - Target environment, for example CREATE DATABASE LINK for the Stage user.

The placeholders for the users, schemas and connections are listed in Oracle - Placeholder values.

Sources and linked projects

The source system has the type Oracle. Stage views read its tables through a database link or directly, as set in Database Links for Source Systems. To create source model objects from an Oracle schema, see Discover an Oracle database.

You can export this project as a linked project. The configuration cannot import linked projects.

Supported model objects

Source
Stage

Capabilities and limitations

Names: generated names are unquoted and in capitals. Characters other than letters, digits, _, $ and # become _, and names are cut to 128 characters.
Indexes: index management offers B-Tree Index and Bitmap Index and the option Create as local Index.
Table storage: the partition, compression, tablespace and statistics properties of a Stage are described in Partition Clause and table storage (Model Object).
Loading: LoadControl Options offers Native Load Control and Airflow.
Data types: the platform data types CHAR(CHAR), CHAR(BYTE), VARCHAR2(CHAR) and VARCHAR2(BYTE) write the length semantics, for example VARCHAR2(100 CHAR). CHAR and VARCHAR2 write none, so the database setting NLS_LENGTH_SEMANTICS applies. TIMESTAMP WITH TIME ZONE with precision 3 becomes TIMESTAMP(3) WITH TIME ZONE.
Data type validation: a length, precision or scale that does not fit the Oracle data type gets a warning, see Invalid data type configuration (Business Rule). The generator writes the configured value: VARCHAR2(5000) fails with ORA-00910 at deployment on a database with MAX_STRING_SIZE = STANDARD.

Project properties

The configuration offers these project properties, with their defaults:

Hash Casing: None; also Upper and Lower
Hash Concatenation Separator: _
Hash Null Placeholder: _NULL_
Hash Trim: off
Persist Validity: off
Create Result Views: on
Target Object Comment Type: None
Artifact Output Mode: Layer Composites
Deployment Options: Generate Drop Table Statements
Configure LoadControl: on
Deploy LoadControl: on
LoadControl Options: Native Load Control; also Airflow
Database Links for Source Systems: Create Database Links
Create GRANTs for Imported Objects: on

Platform differences

The table compares the configuration with Microsoft SQL Server - Stage of the same release:

Item	Oracle	Microsoft SQL Server
Timestamp Default Terms	`TIMESTAMP`	`datetimeoffset`
Hash Default Terms	`RAW(20)`	`binary(20)`
Text Default Terms	`VARCHAR2`	`nvarchar`
Number Default Terms	`NUMBER`	`int` or `bigint`
Flag Default Terms	`NUMBER(1, 0)`	`bit`
Index types	B-tree, bitmap	rowstore, columnstore
Begin of Universe	not available	available

Database Links for Source Systems, Create GRANTs for Imported Objects, Create as local Index and the table storage properties exist only on Oracle. Included Columns of an index exists only on Microsoft SQL Server.

These Default Terms belong to each group; text Default Terms have the same lengths on both platforms:

Timestamps: BG_LoadTimestamp, BG_ValidFromTimestamp, BG_ValidToTimestamp, BG_UpdateTimestamp, BG_EffectiveTimestamp
Row hashes: BG_RowHash, BG_RowHashScd1, BG_RowHashScd2
Text: BG_SourceSystem, BG_DataflowName, BG_DataflowSetName, BG_ErrorDescription, BG_IncrementLoadStatus
Numbers: BG_ID, BG_ErrorCount, BG_DataQuality_ID
Flags: BG_IsDeleted

Example

A Stage CUSTOMER reads the table CUSTOMER of the Oracle source system BFM Mssql. With Database Links for Source Systems set to Create Database Links, it generates this table and source view. In the excerpt, <stage schema> and <source schema> stand for the placeholders of the two schemas, and eight columns of the view are left out:

CREATE TABLE <stage schema>.STG_ST_CUSTOMER (
     BG_LOADTIMESTAMP TIMESTAMP NOT NULL
    ,BG_SOURCESYSTEM VARCHAR2(255)
    ,CUSTOMER_ID NUMBER NOT NULL
    ,CUSTOMER VARCHAR2(100) NOT NULL
    ,CUSTOMER_TYPE VARCHAR2(100) NOT NULL
    ,CUSTOMER_SINCE DATE
    ,ADDRESS_LINE1 VARCHAR2(100)
    ,ADDRESS_LINE2 VARCHAR2(100)
    ,ADDRESS_LINE3 VARCHAR2(100)
    ,ZIP_CODE VARCHAR2(20)
    ,COUNTRY VARCHAR2(50)
    ,COUNTRY_ISO_CODE VARCHAR2(2)
    ,BU VARCHAR2(2) NOT NULL
)
;
CREATE OR REPLACE VIEW <stage schema>.STG_ST_CUSTOMER_SOURCE
AS
SELECT
     CAST(NULL AS TIMESTAMP) AS BG_LOADTIMESTAMP
    ,CAST(NULL AS VARCHAR2(255)) AS BG_SOURCESYSTEM
    ,S1.CUSTOMER_ID AS CUSTOMER_ID
    ,S1.CUSTOMER AS CUSTOMER
    ...
    ,S1.BU AS BU
FROM <source schema>.CUSTOMER@BFM_Mssql_DBL S1
;

The view reads the source table through the database link BFM_Mssql_DBL, which the Stage script creates for the source system. On Microsoft SQL Server, BG_LOADTIMESTAMP and BG_SOURCESYSTEM get the types datetimeoffset and nvarchar(255).

Release differences

Project Settings shows the Generator Version of your project.

Generator 2.0: first release of the configuration.
Generator 2.0.1: index management, deactivation of model objects, Customize Default Terms, custom code before and after templates, template preview and the export as a linked project.
Generator 2.2: the project properties Persist Validity, Create Result Views, Hash Casing, Hash Concatenation Separator, Hash Null Placeholder and Hash Trim; the model object property Increment Management Type; the platform data types with length semantics and the data type validation; the replacement of characters in generated names. Up to Generator 2.0.1, precision 3 was written as TIMESTAMP(3 WITH TIME ZONE).
Fixes for Oracle are listed in Generators Release - 2.3.0, Generators Release - 2.4.0 and Generators Release - 2.4.1.