What is biGENIUS-X
biGENIUS-X is a SaaS application (Software as a Service).
It provides advanced automation and system support for any data project – data warehouses, data marts, data lakes, operational data stores, event hubs, and big data integrations.
The workflow begins with defining objects and modeling according to business requirements within a business model. This creates a semantic layer. Users can model in a user-friendly interface without needing to code.
Using standard business intelligence layers ensures a robust and easily maintainable architecture.
After modeling, biGENIUS automatically generates all the required code for the project. During the generation process, the generator templates interpret our business model and the configurations designed at a semantic level.
The templates are built based on best practice blueprints derived from extensive experience in numerous business intelligence and big data projects.
We can add more configurations to the model. For example, we can add delete detection or choose slowly changing dimensions.
biGENIUS-X offers a clear distinction between the semantic model defined by the user and the generated implementation.
The software solution consists of several components, which are illustrated in the following solution architecture:
- Source Systems
- biGENIUS-X Discovery Companion
- biGENIUS-X cloud application
- biGENIUS-X Services
- biGENIUS-X Configuration
- biGENIUS-X metadata
- Code Repository
- CI/CD Pipelines
- Target Solution
What biGENIUS-X is made of
Source Systems
The Source Systems contain the Source Data you need to model your Data Warehouse.
This may include databases, files, or other sources.
Please refer to the article Discovery Companion for the list of Source Systems supported by biGENIUS-X.
To be able to use a Source System, you should have access to it to extract Metadata.
For instance, if the Source System is a database, you need a valid user and password. You also need sufficient access rights to read the database structure.
biGENIUS-X Discovery Companion
The Discovery Companion application is an application that can be used to produce discovery files from your Source Systems. biGENIUS-X can then use these discovery files.
It is a Python package that uses metadata ingestion to produce discovery files from your Source Systems.
For more details, see the article Understand Discovery Companion.
Since the Discovery Companion is entirely separate from the biGENIUS-X cloud application, the cloud application does not directly access your Source Systems.
You will need to install the Discovery Companion where you can access the Source Systems.
biGENIUS-X cloud application
The cloud application user interface provides access to various comprehensive features for modeling your project. These include creating model objects, managing users, and visualizing data lineage, among other functionalities.
Please refer to the article Understand the application structure for more details.
biGENIUS-X Services
biGENIUS-X also encompasses several micro-services that interact with the cloud application.
These microservices facilitate various functions, such as managing authentication to the cloud application and controlling the generator's behavior to produce the target solution.
biGENIUS-X Configuration
The biGENIUS-X Configuration contains two types of configurations:
- Semantic configuration: for instance, the ability to create a Hub from a Stage Object or the layers of Data Vault modeling (Source, Stage, and Data Vault)
- Target Technology specific configuration: for example, the mapping between Source System data types and Target technology data types
Please take a look at Understand the Generator Configuration for more details about the generator service and the possible configurations.
See Understand the Layering concept for more details about the Generator Layers.
biGENIUS-X Metadata
The biGENIUS-X Metadata encompasses the components that constitute your projects: Discoveries, Model Objects, Term Mappings, and Relationships.
The Metadata is stored as files in a Git Repository.
Please refer to the article Understand the Project Metadata Repository content for more details.
The Metadata is stored in your infrastructure. So, the creation, maintenance, and access rights definition is part of your responsibility.
Code Repository
The biGENIUS-X Generated Artifacts are the files generated from your modeling process. These files are stored in a git Code Repository.
For further information, please refer to the Generate Artifacts article.
The Code Repository is within your infrastructure. Therefore, the responsibility for creating, maintaining, and defining access rights lies with you.
Pipelines
You can establish CI/CD Pipelines to:
- handle CI (Continuous Integration) thanks to the Code Repository and automated builds you can set up
- orchestrate CD (Continuous Deployment) Pipelines to automate the deployment of newly Generated Artifacts to your Target System
By incorporating these architectural elements with biGENIUS-X, you can achieve compliance with DevOps practices.
While CI/CD Pipelines are not directly integrated into biGENIUS-X, they can be part of an architecture that includes biGENIUS-X. Setting up and maintaining these pipelines is the responsibility of the user.
Target Solution
Based on the Generated Artifacts deployed in your infrastructure, the Target Solution can be a Data Warehouse, a Data Lake, a Lakehouse, or a Data Mesh.
biGENIUS-X also generates the Execution Plans for orchestrating loads, monitoring, logging, and debugging.