Infrahub architecture

Infrahub is deployed as a container-based architecture, composed of multiple components. A minimum deployment consists of the following architecture:

Infrahub Architecture

Infrahub components

The main components are:

A Frontend written in React and rendered in the user's browser.
An API server written in Python with FastAPI.
A Task manager based on Prefect to orchestrate workflow tasks.
A Task worker written in Python to execute specific tasks such as managing the interaction with external Git repositories.
A Graph database based on neo4j.
A Relational database based on PostgreSQL.
A Message bus based on RabbitMQ.
A Cache based on redis.
An Object Store based on local storage or Amazon S3.

API server

Language: Python

The API server delivers the REST API and the GraphQL endpoints. Internally, the API server is built with FastAPI as the web framework and Graphene to generate the GraphQL endpoints.

note

Multiple instance of the API Server can run at the same time to process more requests.

Task manager

The Task manager is based on Prefect, and is responsible for orchestration of tasks to be delegated to one or more Task workers.

Task worker

Language: Python

The Task worker is responsible for managing all the content related to the Git repositories. It organizes the file systems in order to quickly access any relevant commit. The Task worker periodically pulls the Git server for updates and listens to the RPC channel on the event bus for tasks to execute.

Currently there are three types of tasks:

Internal tasks
User tasks
Git tasks

Some of the tasks that can be executed on the Task worker include:

Rendering a Jinja template.
Rendering a transform function.
Executing a check.
All Git operations (pull/merge/diff).

note

Multiple instance of the Task worker can run at the same time to process more requests.

Frontend

React-based single page application.

External systems

Graph database

The Graph database is based on Bolt and Cypher. Currently, we have validated both Neo4j 5.x and Memgraph as possible options. Neo4j is a production grade, battle tested graph database that is used in thousands of deployments around the world. Memgraph is a lightweight, very fast, in-memory database that works great for testing and demos.

Message bus

The message bus is based on RabbitMQ. It supports both a fanout channel to distribute messages to all members at the same time and a RPC framework to distribute work synchronously. Infrahub also supports NATS JetStream for the message bus.

Cache

The cache is based on Redis. It's mainly used as a central point to support the distributed lock systems between all the different component of the system. Infrahub also supports NATS JetStream for the cache.

Git server (GitHub/GitLab)

Any Git server. The most popular being: GitHub, GitLab, or Bitbucket.

TLS-secured connections

Infrahub supports TLS connections toward the dependencies above. The following configuration variables are available to configure TLS:

INFRAHUB_BROKER_TLS_CA_FILE=/opt/ssl/ca.pem # File path to a CA certificate or bundle in PEM format
INFRAHUB_BROKER_TLS_ENABLED=true # Boolean to enable TLS connections (allowed values: 1, True, true, 0, False, false)
INFRAHUB_BROKER_TLS_INSECURE=true # Boolean to ignore certificate verification, useful for self-signed certificates (CA is not verified, expiry is not verified, hostname is not verified)

INFRAHUB_CACHE_TLS_CA_FILE
INFRAHUB_CACHE_TLS_ENABLED
INFRAHUB_CACHE_TLS_INSECURE
INFRAHUB_DB_TLS_CA_FILE
INFRAHUB_DB_TLS_ENABLED
INFRAHUB_DB_TLS_INSECURE

Graph database

You can configure TLS for Neo4j using the following Docker environment variables:

NEO4J_dbms_ssl_policy_bolt_enabled=true
NEO4J_dbms_ssl_policy_bolt_base__directory=/opt/ssl
NEO4J_dbms_ssl_policy_bolt_private__key=cert.key
NEO4J_dbms_ssl_policy_bolt_public__certificate=cert.pem
NEO4J_dbms_connector_bolt_tls__level=REQUIRED

Infrahub High Availability deployment

Infrahub may be deployed with no single-point of failure, and horizontally scaled architectures can be deployed. An example of a HA deployment is below:

Infrahub Architecture

An example of HA deployment using Terraform on a 3-nodes Kubernetes cluster

deploy.tf
terraform {
  required_providers {
    kubectl = {
      source  = "alekc/kubectl"
      version = "2.1.3"
    }
  }
}

provider "helm" {
  kubernetes {
    config_path = "~/.kube/config"
  }
}

provider "kubernetes" {
  config_path = "~/.kube/config"
}

provider "kubectl" {
  config_path = "~/.kube/config"
}

locals {
  target_namespace = "infrahub"
}

### Infrahub

resource "helm_release" "infrahub_ha" {
  depends_on = [helm_release.taskmanager_ha, helm_release.cache_ha, helm_release.messagequeue_ha, helm_release.database_ha, helm_release.objectstore_ha]

  name    = "infrahub"
  chart   = "oci://registry.opsmill.io/opsmill/chart/infrahub-enterprise"
  version = "3.1.7"

  create_namespace = true
  namespace        = local.target_namespace

  values = [
    <<EOT
infrahub:
  global:
    infrahubTag: 1.2.0rc0-ha
  infrahubServer:
    replicas: 3
    persistence:
      enabled: false
    infrahubServer:
      env:
        INFRAHUB_DB_ADDRESS: infrahub-headless
        INFRAHUB_DB_PROTOCOL: neo4j # required for client-side routing
        INFRAHUB_BROKER_ADDRESS: messagequeue-rabbitmq
        INFRAHUB_CACHE_ADDRESS: redis-sentinel-proxy
        INFRAHUB_CACHE_PORT: 6379
        INFRAHUB_WORKFLOW_ADDRESS: prefect-server
        INFRAHUB_WORKFLOW_PORT: 4200
        PREFECT_API_URL: "http://prefect-server:4200/api"

        INFRAHUB_STORAGE_DRIVER: s3
        AWS_ACCESS_KEY_ID: admin
        AWS_SECRET_ACCESS_KEY: password
        AWS_S3_BUCKET_NAME: infrahub-data
        AWS_S3_ENDPOINT_URL: objectstore-minio:9000
        AWS_S3_USE_SSL: "false"

        INFRAHUB_ALLOW_ANONYMOUS_ACCESS: "true"
        INFRAHUB_DB_TYPE: neo4j
        INFRAHUB_LOG_LEVEL: INFO
        INFRAHUB_PRODUCTION: "false"
        INFRAHUB_INITIAL_ADMIN_TOKEN: 06438eb2-8019-4776-878c-0941b1f1d1ec
        INFRAHUB_SECURITY_SECRET_KEY: 327f747f-efac-42be-9e73-999f08f86b92
        INFRAHUB_GIT_REPOSITORIES_DIRECTORY: "/opt/infrahub/git"
    affinity:
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                service: infrahub-server
            topologyKey: kubernetes.io/hostname
  infrahubTaskWorker:
    replicas: 3
    infrahubTaskWorker:
      env:
        INFRAHUB_DB_ADDRESS: infrahub-headless
        INFRAHUB_DB_PROTOCOL: neo4j # required for client-side routing
        INFRAHUB_BROKER_ADDRESS: messagequeue-rabbitmq
        INFRAHUB_CACHE_ADDRESS: redis-sentinel-proxy
        INFRAHUB_CACHE_PORT: 6379
        INFRAHUB_WORKFLOW_ADDRESS: prefect-server
        INFRAHUB_WORKFLOW_PORT: 4200
        PREFECT_API_URL: "http://prefect-server:4200/api"

        INFRAHUB_STORAGE_DRIVER: s3
        AWS_ACCESS_KEY_ID: admin
        AWS_SECRET_ACCESS_KEY: password
        AWS_S3_BUCKET_NAME: infrahub-data
        AWS_S3_ENDPOINT_URL: objectstore-minio:9000
        AWS_S3_USE_SSL: "false"

        INFRAHUB_DB_TYPE: neo4j
        INFRAHUB_LOG_LEVEL: DEBUG
        INFRAHUB_PRODUCTION: "false"
        INFRAHUB_API_TOKEN: 06438eb2-8019-4776-878c-0941b1f1d1ec
        INFRAHUB_TIMEOUT: "60"
        INFRAHUB_GIT_REPOSITORIES_DIRECTORY: "/opt/infrahub/git"
        PREFECT_WORKER_QUERY_SECONDS: 3
        PREFECT_AGENT_QUERY_INTERVAL: 3
    affinity:
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                service: infrahub-task-worker
            topologyKey: kubernetes.io/hostname
  redis:
    enabled: false
  neo4j:
    enabled: false
  rabbitmq:
    enabled: false
  prefect-server:
    enabled: false
EOT
  ]
}

#### Infrahub dependencies

resource "helm_release" "database_ha_service" {
  depends_on = [helm_release.database_ha]

  name       = "database-service"
  chart      = "neo4j-headless-service"
  repository = "https://helm.neo4j.com/neo4j/"
  version    = "5.20.0"

  create_namespace = true
  namespace        = local.target_namespace

  values = [
    <<EOT
neo4j:
  name: "infrahub"
EOT
  ]
}

resource "helm_release" "database_ha" {
  count = 3

  name       = "database-${count.index}"
  chart      = "neo4j"
  repository = "https://helm.neo4j.com/neo4j/"
  version    = "5.20.0"

  create_namespace = true
  namespace        = local.target_namespace

  values = [
    <<EOT
neo4j:
  name: "infrahub"
  minimumClusterSize: 3
  resources:
    cpu: "4"
    memory: "8Gi"
  password: "admin"
  edition: "enterprise"
  acceptLicenseAgreement: "yes"
config:
  dbms.security.auth_minimum_password_length: "4"
  dbms.security.procedures.unrestricted: apoc.*
logInitialPassword: false
volumes:
  data:
    mode: "dynamic"
    dynamic:
      storageClassName: standard
services:
  neo4j:
    enabled: false
EOT
  ]
}

resource "helm_release" "messagequeue_ha" {
  name    = "messagequeue"
  chart   = "oci://registry-1.docker.io/bitnamicharts/rabbitmq"
  version = "14.4.1"

  create_namespace = true
  namespace        = local.target_namespace

  values = [
    <<EOT
replicaCount: 3
auth:
  username: infrahub
  password: infrahub
metrics:
  enabled: true
startupProbe:
  enabled: true
podAntiAffinityPreset: hard
EOT
  ]
}

resource "helm_release" "objectstore_ha" {
  name    = "objectstore"
  chart   = "oci://registry-1.docker.io/bitnamicharts/minio"
  version = "15.0.5"

  create_namespace = true
  namespace        = local.target_namespace

  values = [
    <<EOT
mode: distributed
statefulset:
  replicaCount: 3
  drivesPerNode: 2
auth:
  rootUser: admin
  rootPassword: password
provisioning:
  enabled: true
  buckets:
    - name: infrahub-data
podAntiAffinityPreset: hard
EOT
  ]
}

#### Task manager

# Workaround since Prefect Helm chart does not use StatefulSets and multiple pod initialization causes issue with concurrent DB init
# StatefulSet would solve this issue because it creates pods sequentially
# Workaround by installing the Helm chart with one replica, and then scale up to 3 replicas
resource "null_resource" "scale_up_taskmanager" {
  depends_on = [helm_release.infrahub_ha]
  provisioner "local-exec" {
    command = "kubectl scale -n ${local.target_namespace} deployment/prefect-server --replicas=3"
  }
}

resource "helm_release" "taskmanager_ha" {
  depends_on = [helm_release.cache_ha, kubectl_manifest.taskmanagerdb_ha]

  name       = "taskmanager"
  chart      = "prefect-server"
  repository = "https://prefecthq.github.io/prefect-helm"
  version    = "2025.2.21193831"

  create_namespace = true
  namespace        = local.target_namespace

  values = [
    <<EOT
global:
  prefect:
    image:
      repository: registry.opsmill.io/opsmill/infrahub-enterprise
      prefectTag: 1.2.0rc0-ha
server:
  replicaCount: 1
  command:
    - /usr/bin/tini
    - -g
    - --
  args:
    - uvicorn
    - --host
    - "0.0.0.0"
    - --port
    - "4200"
    - --factory
    - infrahub.prefect_server.app:create_infrahub_prefect
  env:
    - name: PREFECT_UI_SERVE_BASE
      value: /
    - name: PREFECT_MESSAGING_BROKER
      value: prefect_redis.messaging
    - name: PREFECT_MESSAGING_CACHE
      value: prefect_redis.messaging
    - name: PREFECT_REDIS_MESSAGING_HOST
      value: redis-sentinel-proxy
    - name: PREFECT_REDIS_MESSAGING_DB
      value: "1"
secret:
  create: true
  name: ""
  username: "prefect"
  password: "prefect"
  host: "taskmanagerdb-rw"
  port: "5432"
  database: "prefect"
serviceAccount:
  create: false
postgresql:
  enabled: false
EOT
  ]
}

resource "kubernetes_service_v1" "redis_sentinel_proxy_svc" {
  depends_on = [helm_release.cache_ha]

  metadata {
    name      = "redis-sentinel-proxy"
    namespace = local.target_namespace
    labels = {
      "app.kubernetes.io/name" = "redis-sentinel-proxy"
    }
  }

  spec {
    type = "ClusterIP"
    port {
      port        = 6379
      target_port = "redis"
      name        = "cache-ha"
    }
    selector = {
      "app.kubernetes.io/name" = "redis-sentinel-proxy"
    }
  }
}

resource "kubernetes_deployment_v1" "redis_sentinel_proxy_deployment" {
  depends_on = [helm_release.cache_ha]

  metadata {
    name      = "redis-sentinel-proxy"
    namespace = local.target_namespace
    labels = {
      "app.kubernetes.io/name" = "redis-sentinel-proxy"
    }
  }

  spec {
    replicas = 2
    selector {
      match_labels = {
        "app.kubernetes.io/name" = "redis-sentinel-proxy"
      }
    }
    template {
      metadata {
        labels = {
          "app.kubernetes.io/name" = "redis-sentinel-proxy"
        }
      }
      spec {
        affinity {
          pod_anti_affinity {
            required_during_scheduling_ignored_during_execution {
              label_selector {
                match_labels = {
                  "app.kubernetes.io/name" = "redis-sentinel-proxy"
                }
              }
              topology_key = "kubernetes.io/hostname"
            }
          }
        }
        container {
          name  = "redis-sentinel-proxy"
          image = "patrickdk/redis-sentinel-proxy:v1.2"
          args = [
            "-master",
            "mymaster",
            "-listen",
            ":6379",
            "-sentinel",
            "cache:26379",
          ]
          port {
            container_port = 6379
            name           = "redis"
          }
        }
      }
    }
  }
}

resource "helm_release" "cache_ha" {
  name       = "cache"
  chart      = "redis"
  repository = "https://charts.bitnami.com/bitnami"
  version    = "19.5.2"

  create_namespace = true
  namespace        = local.target_namespace

  values = [
    <<EOT
nameOverride: cache
architecture: replication
auth:
  enabled: false
master:
  podAntiAffinityPreset: hard
  persistence:
    enabled: true
  service:
    ports:
      redis: 6379
replicas:
  replicaCount: 3
  podAntiAffinityPreset: hard
sentinel:
  enabled: true
EOT
  ]
}

resource "kubectl_manifest" "taskmanagerdb_ha" {
  depends_on = [helm_release.taskmanagerdb_ha_operator, kubernetes_secret_v1.db_secret]

  yaml_body = <<EOT
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: taskmanagerdb
  namespace: ${local.target_namespace}
spec:
  instances: 3
  storage:
    size: 10Gi
  postgresql:
    pg_hba:
      - host all all 10.0.0.0/8 md5
  bootstrap:
    initdb:
      database: prefect
      owner: prefect
      secret:
        name: prefect-user
EOT
}

resource "kubernetes_secret_v1" "db_secret" {
  depends_on = [helm_release.taskmanagerdb_ha_operator]

  metadata {
    name      = "prefect-user"
    namespace = local.target_namespace
  }
  data = {
    username = "prefect"
    password = "prefect"
  }
}

resource "helm_release" "taskmanagerdb_ha_operator" {
  name       = "cnpg"
  chart      = "cloudnative-pg"
  repository = "https://cloudnative-pg.github.io/charts"
  version    = "0.23.2"

  create_namespace = true
  namespace        = local.target_namespace

  values = [
    <<EOT
EOT
  ]
}

Infrahub components​

API server​

Task manager​

Task worker​

Frontend​

External systems​

Graph database​

Message bus​

Cache​

Git server (GitHub/GitLab)​

TLS-secured connections​

Graph database​

Infrahub High Availability deployment​

Infrahub components

API server

Task manager

Task worker

Frontend

External systems

Graph database

Message bus

Cache

Git server (GitHub/GitLab)

TLS-secured connections

Graph database

Infrahub High Availability deployment