What is biGENIUS-X
biGENIUS-X is a SaaS application (Software as a Service).
It provides advanced automation and system support for any data project – data warehouses, data marts, data lakes, operational data stores, event hubs, and big data integrations.
The workflow begins with defining objects and modeling according to business requirements within a business model. This creates a semantic layer. Users can model in a user-friendly interface without needing to code.
Using standard business intelligence layers ensures a robust and easily maintainable architecture.
After modeling, biGENIUS automatically generates all the required code for the project. During the generation process, the generator templates interpret our business model and the configurations designed at a semantic level.
The templates are built based on best practice blueprints derived from extensive experience in numerous business intelligence and big data projects.
We can add more configurations to the model. For example, we can add delete detection or choose slowly changing dimensions.
biGENIUS-X offers a clear distinction between the semantic model defined by the user and the generated implementation.
The software solution consists of several components, which are illustrated in the following solution architecture:
- Source Systems
- biGENIUS-X Discovery application
- biGENIUS-X cloud application
- biGENIUS-X Services
- biGENIUS-X Configuration
- biGENIUS-X metadata
- Code Repository
- CI/CD Pipelines
- Target Solution
What biGENIUS-X is made of
Source Systems
The Source Systems contain the Source Data you need to model your Data Warehouse.
This may include databases, files, or other sources.
Please refer to the article: Use the Discovery Application for the list of Source Systems supported by biGENIUS-X.
To be able to use a Source System, you should have access to it to extract Metadata.
For instance, if the Source System is a database, you need a valid user and password. You also need sufficient access rights to read the database structure.
biGENIUS-X Discovery Application
The Discovery application can produce discovery files from your Source Systems.
biGENIUS-X can then use these discovery files.
It is a portable application that uses metadata ingestion to produce discovery files from your Source Systems.
For more details, see the article Understand Discovery Application.
Since the Discovery Application is entirely separate from the biGENIUS-X cloud application, the cloud application does not directly access your Source Systems.
You will need to use the Discovery Application where you can access the Source Systems.
biGENIUS-X cloud application
The cloud application user interface provides access to various comprehensive features for modeling your project. These include creating model objects, managing users, and visualizing data lineage, among other functionalities.
Please refer to the article Understand the application structure for more details.
biGENIUS-X Services
biGENIUS-X also encompasses several micro-services that interact with the cloud application.
These microservices facilitate various functions, such as managing authentication to the cloud application and controlling the generator's behavior to produce the target solution.
biGENIUS-X Configuration
The biGENIUS-X Configuration contains two types of configurations:
- Semantic configuration: for instance, the ability to create a Hub from a Stage Object or the layers of Data Vault modeling (Source, Stage, and Data Vault)
- Target Technology specific configuration: for example, the mapping between Source System data types and Target technology data types
Please take a look at Understand the Generator Configuration for more details about the generator service and the possible configurations.
See Understand the Layering concept for more details about the Generator Layers.
biGENIUS-X Metadata
The biGENIUS-X Metadata encompasses the components that constitute your projects: Discoveries, Model Objects, Term Mappings, and Relationships.
The Metadata is stored as files in a Git Repository.
Please refer to the article Understand the Project Metadata Repository content for more details.
The Metadata is stored in your infrastructure. So, the creation, maintenance, and access rights definition is part of your responsibility.
Code Repository
The biGENIUS-X Generated Artifacts are the files generated from your modeling process. These files are stored in a git Code Repository.
For further information, please refer to the Generate Artifacts article.
The Code Repository is within your infrastructure. Therefore, the responsibility for creating, maintaining, and defining access rights lies with you.
Pipelines
You can establish CI/CD Pipelines to:
- handle CI (Continuous Integration) thanks to the Code Repository and automated builds you can set up
- orchestrate CD (Continuous Deployment) Pipelines to automate the deployment of newly Generated Artifacts to your Target System
By incorporating these architectural elements with biGENIUS-X, you can achieve compliance with DevOps practices.
While CI/CD Pipelines are not directly integrated into biGENIUS-X, they can be part of an architecture that includes biGENIUS-X. Setting up and maintaining these pipelines is the responsibility of the user.
Target Solution
Based on the Generated Artifacts deployed in your infrastructure, the Target Solution can be a Data Warehouse, a Data Lake, a Lakehouse, or a Data Mesh.
The Target Solution is composed of several target objects whose dependencies are managed by biGENIUS-X.
Learn more about target objects here: Understand the Generator target objects.
biGENIUS-X also generates the Execution Plans for orchestrating loads, monitoring, logging, and debugging.