# Infrahub by OpsMill - Documentation > Infrahub by OpsMill - the modern infrastructure data management platform. Infrahub is the open-source product built by OpsMill (https://opsmill.com). - [Infrahub by OpsMill - Documentation](/index.md) ## labs Hands-on Infrahub labs on Instruqt: guided learning tracks, standalone OpsMill labs, and labs from partners and the community. - [Labs](/labs.md): Hands-on Infrahub labs on Instruqt: guided learning tracks, standalone OpsMill labs, and labs from partners and the community. ## search - [Search the documentation](/search.md) ## ansible Collection version 1.8.3 - [opsmill.infrahub Ansible collection](/ansible.md): Collection version 1.8.3 ### guides - [Manage branches in Infrahub](/ansible/guides/branch.md): Infrahub lets you manage branches directly via the opsmill.infrahub.branch module. With this module you can create, confirm, and delete branches—using the state parameter to control the desired operation. - [Use Infrahub for Ansible dynamic inventory](/ansible/guides/dynamic-inventory.md): Overview - [Install opsmill.infrahub Ansible collection](/ansible/guides/installation.md): This guide assumes you have basic knowledge of Ansible and its ecosystem. For more information on using Ansible, please refer to the official Ansible documentation. - [Create nodes in Infrahub](/ansible/guides/node.md): Overview - [Retrieve data from Infrahub](/ansible/guides/query-and-lookup.md): Overview ### references - [Modules modules](/ansible/references/plugins/artifact_fetch_module.md): Fetch the content of an artifact from Infrahub - [Modules modules](/ansible/references/plugins/artifact_generate_module.md): Trigger artifact regeneration in Infrahub - [Modules modules](/ansible/references/plugins/branch_module.md): Creates, Updates or Deletes a branch in Infrahub - [Inventory inventory](/ansible/references/plugins/inventory_inventory.md): Infrahub inventory source (using GraphQL) - [Lookup lookup](/ansible/references/plugins/lookup_lookup.md): Queries and returns elements from Infrahub (using GraphQL) - [Modules modules](/ansible/references/plugins/node_module.md): Creates, Updates or Deletes a node in Infrahub - [Modules modules](/ansible/references/plugins/object_file_fetch_module.md): Fetch file content from a CoreFileObject node in Infrahub - [Modules modules](/ansible/references/plugins/query_graphql_module.md): Queries and returns elements from Infrahub GraphQL API - [Modules modules](/ansible/references/plugins/schema_module.md): Load, check, or export schemas in Infrahub - [Overview](/ansible/references/roles/install.md): The Infrahub Installation Role in the OpsMill Infrahub Ansible Collection allows you to install and configure Infrahub using Docker and systemd. ## arista-avd A reference design that models Arista datacenter fabrics in Infrahub and generates EOS configurations via the AVD pipeline. - [Arista AVD Reference Design](/arista-avd.md): A reference design that models Arista datacenter fabrics in Infrahub and generates EOS configurations via the AVD pipeline. ### cloudvision The repository validates generated EOS configurations in CloudVision during - [CloudVision validation](/arista-avd/cloudvision.md): The repository validates generated EOS configurations in CloudVision during ### containerlab The ContainerLab Topology artifact — how a fabric becomes a containerlab topology file, and how to deploy it. - [ContainerLab](/arista-avd/containerlab.md): The ContainerLab Topology artifact — how a fabric becomes a containerlab topology file, and how to deploy it. ### developer-guide How the Infrahub Arista AVD solution is built and how to extend it. - [Developer guide](/arista-avd/developer-guide.md): How the Infrahub Arista AVD solution is built and how to extend it. - [Architecture overview](/arista-avd/developer-guide/architecture.md): System architecture and data flow for the Infrahub Arista AVD solution. - [AvdArtifact & file storage](/arista-avd/developer-guide/avd/artifacts.md): The AvdArtifact node, its child AvdHostvarFile and AvdStructuredConfigFile nodes, and the end-to-end artifact definitions. - [Debugging the pipeline](/arista-avd/developer-guide/avd/debugging.md): Inspect intermediate files, force regeneration, and isolate a single generator or transform. - [Extending the pipeline](/arista-avd/developer-guide/avd/extending.md): Worked examples for adding a device role, adding a transform output, or adding a hostvar field. - [Hostvars reference](/arista-avd/developer-guide/avd/hostvars.md): The PyAVD-compatible hostvars structure built per device role by Phase 1 of the pipeline. - [AVD pipeline overview](/arista-avd/developer-guide/avd/overview.md): The two-phase AVD generator pipeline, the PyAVD version pinned, and the shape of the data flow. - [Role mapping](/arista-avd/developer-guide/avd/role-mapping.md): Infrahub device roles mapped to PyAVD device types. - [AVD transforms](/arista-avd/developer-guide/avd/transforms.md): The three Python transforms that render EOS configs and AVD documentation from stored data. - [Checks](/arista-avd/developer-guide/checks.md): Python checks that run in the proposed-change pipeline — CloudVision configuration validation and its workspace lifecycle, and fabric pool validation. - [Concepts](/arista-avd/developer-guide/concepts.md): The core Infrahub terms this reference design relies on, for readers new to the platform. - [Generators](/arista-avd/developer-guide/generators.md): The infrastructure generators that create devices, interfaces, cabling, and AVD inputs. - [Schemas](/arista-avd/developer-guide/schemas.md): All Infrahub schema definitions in this solution. - [Transforms](/arista-avd/developer-guide/transforms.md): Data transforms and artifact generation — how Infrahub data becomes configs, docs, and CSVs. ### how-to - [Add a network segment](/arista-avd/how-to/add-network-segment.md): Create a VRF, VLAN, and SVI on a fabric using the service portal. - [Add a server](/arista-avd/how-to/add-server.md): Provision a physical server into a compute rack and trigger the AVD cascade. - [Create a tenant](/arista-avd/how-to/create-tenant.md): Create an EVPN tenant with a MAC VRF VNI base allocation for one or more fabrics. - [Regenerate a fabric](/arista-avd/how-to/regenerate-fabric.md): Use the Fabric Design page to inspect and re-run the generator chain for a fabric. - [Upgrade AVD version](/arista-avd/how-to/upgrade-avd-version.md): Move to a newer PyAVD version and validate the result on a branch before production. ### provision-first-fabric Run the generator chain end-to-end on Fabric-L3LS-MultiPod-A and reach rendered AVD artifacts. - [Provision your first fabric](/arista-avd/provision-first-fabric.md): Run the generator chain end-to-end on Fabric-L3LS-MultiPod-A and reach rendered AVD artifacts. ### quick-start Install dependencies, bring the stack up, and load seed data. - [Quick start](/arista-avd/quick-start.md): Install dependencies, bring the stack up, and load seed data. ### supported-capabilities What this Arista AVD reference design supports today, what is partial, and what is not yet covered. - [Supported capabilities](/arista-avd/supported-capabilities.md): What this Arista AVD reference design supports today, what is partial, and what is not yet covered. ### troubleshooting Diagnose and fix the most frequent failures — stack health, generator order, seed data, artifacts. - [Common issues](/arista-avd/troubleshooting.md): Diagnose and fix the most frequent failures — stack health, generator order, seed data, artifacts. ### viewing-artifacts Find, preview, and download the AVD EOS configs and fabric/device documentation. - [Viewing artifacts](/arista-avd/viewing-artifacts.md): Find, preview, and download the AVD EOS configs and fabric/device documentation. ## backup The Infrahub Ops CLI provides command-line tools for operating and maintaining an Infrahub deployment: - [Infrahub Backup](/backup.md): The Infrahub Ops CLI provides command-line tools for operating and maintaining an Infrahub deployment: ### backup Use Infrahub Backup to create and restore verifiable recovery points for an Infrahub deployment. Run it as a CLI against Docker Compose, or as a Helm chart on Kubernetes, to capture the graph database and the task-manager database as a single archive you can restore from later. Artifact storage lives outside those databases, so you snapshot it as a separate, coordinated step to keep a recovery complete. - [Infrahub Backup](/backup/backup.md): Use Infrahub Backup to create and restore verifiable recovery points for an Infrahub deployment. Run it as a CLI against Docker Compose, or as a Helm chart on Kubernetes, to capture the graph database and the task-manager database as a single archive you can restore from later. Artifact storage lives outside those databases, so you snapshot it as a separate, coordinated step to keep a recovery complete. - [Back up your Infrahub instance](/backup/backup/create.md): Create a comprehensive backup of an Infrahub instance on Docker Compose. Follow these steps to protect your data before an upgrade or to establish a disaster-recovery routine. - [Install Infrahub Backup](/backup/backup/install.md): Install the infrahub-backup CLI on a Docker Compose or Kubernetes deployment, or build it from source. - [Back up Infrahub on Kubernetes](/backup/backup/kubernetes-backup.md): Back up an Infrahub instance running on Kubernetes with the infrahub-backup Helm chart. If you deploy Infrahub through GitOps tools like ArgoCD or Flux without direct kubectl access, manage backups declaratively through Helm values. - [Restore Infrahub on Kubernetes](/backup/backup/kubernetes-restore.md): Restore an Infrahub instance on Kubernetes from a backup stored in S3-compatible storage. The restore runs as a Kubernetes Job through the same infrahub-backup Helm chart. - [Refresh an environment on a schedule](/backup/backup/kubernetes-scheduled-restore.md): Run the restore CronJob in the infrahub-backup Helm chart to keep a staging or demo deployment on the newest backup another environment produces. Each run restores whichever archive is newest under the source bucket and prefix, so the target release never names a filename and the schedule keeps working as new backups land. - [Restore from a backup](/backup/backup/restore.md): Restore an Infrahub instance from a backup file on Docker Compose. Follow these steps to recover from data loss, roll back a change, or move to new infrastructure. - [Manage backup retention](/backup/backup/retention.md): A backup schedule that never deletes anything eventually fills its disk or its bucket, and the failure usually surfaces as a failed backup rather than as a full volume. A retention policy bounds that growth: you declare how many recovery points to keep, and infrahub-backup deletes the rest. ### collect Use Infrahub Collect to gather everything OpsMill support needs to investigate an issue — service logs, diagnostic status, configuration, and metrics — into a single local archive. Run it when support asks for logs, or to attach diagnostic data to a support request. The same command produces the same bundle on Docker Compose and Kubernetes. - [Infrahub Collect](/backup/collect.md): Use Infrahub Collect to gather everything OpsMill support needs to investigate an issue — service logs, diagnostic status, configuration, and metrics — into a single local archive. Run it when support asks for logs, or to attach diagnostic data to a support request. The same command produces the same bundle on Docker Compose and Kubernetes. - [Collect a troubleshooting bundle](/backup/collect/create.md): Collect a troubleshooting bundle from an Infrahub deployment and share it with OpsMill support. Collection is read-only and safe to run at any time, including while the instance is degraded. See Infrahub Collect for what a bundle contains and how secrets are masked. The same command produces the same bundle on Docker Compose and Kubernetes. - [Install Infrahub Collect](/backup/collect/install.md): Install the infrahub-collect CLI. One binary collects from both Docker Compose and Kubernetes deployments; on Kubernetes it runs from your workstation or a CI runner and reaches the cluster through kubectl. ### reference - [CLI command reference](/backup/reference/commands.md): Complete reference for the Infrahub Ops CLI tools (infrahub-backup, infrahub-collect), their commands, flags, and options. - [Configuration reference](/backup/reference/configuration.md): Complete reference for configuring the Infrahub Ops CLI tools through environment variables and runtime flags. ### self-update Update the infrahub-backup and infrahub-collect binaries to a newer release with the built-in update command. The command downloads the matching release, verifies its checksum, and replaces the binary in place. - [Update the CLI](/backup/self-update.md): Update the infrahub-backup and infrahub-collect binaries to a newer release with the built-in update command. The command downloads the matching release, verifies its checksum, and replaces the binary in place. ### tutorials - [Get started with Infrahub Backup](/backup/tutorials/getting-started.md): By the end of this tutorial you will have installed infrahub-backup, detected your deployment environment, created and verified your first backup — including a snapshot of artifact storage — and restored from it. That is the core maintenance workflow for an Infrahub instance on Docker Compose. ## demo-dc Welcome to the Infrahub demo-dc example. This repository showcases how Infrahub serves as an infrastructure data management platform for managing modern network infrastructure with design-driven automation. It demonstrates Infrahub's core capabilities including: - [Infrahub demo-dc example](/demo-dc.md): Welcome to the Infrahub demo-dc example. This repository showcases how Infrahub serves as an infrastructure data management platform for managing modern network infrastructure with design-driven automation. It demonstrates Infrahub's core capabilities including: ### cloud-management This tutorial shows an example of cloud resource management schema, which provides a vendor-agnostic way to model cloud infrastructure across AWS, GCP, and Azure. You'll load sample cloud data and explore how Infrahub can serve as a unified inventory for multi-cloud environments. - [Cloud resource management](/demo-dc/cloud-management.md): This tutorial shows an example of cloud resource management schema, which provides a vendor-agnostic way to model cloud infrastructure across AWS, GCP, and Azure. You'll load sample cloud data and explore how Infrahub can serve as a unified inventory for multi-cloud environments. ### concepts This document explains the architectural patterns, design decisions, and core Infrahub concepts demonstrated in this demo. Read this to understand the "why" behind design-driven automation, composable topologies, generators, and the overall approach to infrastructure management. - [Understanding the concepts](/demo-dc/concepts.md): This document explains the architectural patterns, design decisions, and core Infrahub concepts demonstrated in this demo. Read this to understand the "why" behind design-driven automation, composable topologies, generators, and the overall approach to infrastructure management. ### containerlab-deployment This tutorial walks you through deploying a virtual data center topology using Containerlab. You'll take the configurations generated by Infrahub and spin up a fully functional network lab with Arista cEOS virtual switches. - [Deploy a virtual lab with Containerlab](/demo-dc/containerlab-deployment.md): This tutorial walks you through deploying a virtual data center topology using Containerlab. You'll take the configurations generated by Infrahub and spin up a fully functional network lab with Arista cEOS virtual switches. ### developer-guide This guide provides a technical deep-dive into how the Infrahub demo works under the hood. Use this when you want to extend functionality, troubleshoot issues, customize the demo, or understand implementation details. - [Developer guide](/demo-dc/developer-guide.md): This guide provides a technical deep-dive into how the Infrahub demo works under the hood. Use this when you want to extend functionality, troubleshoot issues, customize the demo, or understand implementation details. ### enterprise This guide explains how to configure the demo environment to use Infrahub Enterprise edition instead of the Community edition. Enterprise edition provides additional features including increased performance, enhanced security, and enterprise support. - [Using Infrahub Enterprise](/demo-dc/enterprise.md): This guide explains how to configure the demo environment to use Infrahub Enterprise edition instead of the Community edition. Enterprise edition provides additional features including increased performance, enhanced security, and enterprise support. ### install This guide walks you through setting up the Infrahub demo environment on your local system. By the end, you'll have a fully functional Infrahub instance with pre-loaded schemas, data, and generators ready to explore. - [Installation guide](/demo-dc/install.md): This guide walks you through setting up the Infrahub demo environment on your local system. By the end, you'll have a fully functional Infrahub instance with pre-loaded schemas, data, and generators ready to explore. ### security-management This tutorial walks you through Infrahub's security management capabilities. You'll explore how security policies are modeled as structured data, how they relate to firewalls, and how Infrahub transforms security policy objects into vendor-specific firewall configurations. - [Working with security management](/demo-dc/security-management.md): This tutorial walks you through Infrahub's security management capabilities. You'll explore how security policies are modeled as structured data, how they relate to firewalls, and how Infrahub transforms security policy objects into vendor-specific firewall configurations. ### service-catalog This tutorial walks you through using Infrahub's Service Catalog web interface to manage data center and colocation center deployments. The Service Catalog provides a user-friendly web application built with Streamlit that simplifies infrastructure provisioning by automating branch creation, data loading, and generator execution. - [Using the service catalog](/demo-dc/service-catalog.md): This tutorial walks you through using Infrahub's Service Catalog web interface to manage data center and colocation center deployments. The Service Catalog provides a user-friendly web application built with Streamlit that simplifies infrastructure provisioning by automating branch creation, data loading, and generator execution. ### user-walkthrough This tutorial guides you through the complete workflow of creating a data center topology using Infrahub's design-driven automation. You'll create a new branch, load a topology design, run a generator to create infrastructure, review the changes, and merge them to the main branch. - [User walkthrough](/demo-dc/user-walkthrough.md): This tutorial guides you through the complete workflow of creating a data center topology using Infrahub's design-driven automation. You'll create a new branch, load a topology design, run a generator to create infrastructure, review the changes, and merge them to the main branch. ### virtualization This tutorial shows how the demo models a virtualization layer on top of the data center fabric: hypervisor clusters, the physical hosts that form them, and the virtual machines running on those hosts. You'll load the demo data, follow what the generators do with it, and create a VM of your own through the Service Catalog. - [Virtual machine management](/demo-dc/virtualization.md): This tutorial shows how the demo models a virtualization layer on top of the data center fabric: hypervisor clusters, the physical hosts that form them, and the virtual machines running on those hosts. You'll load the demo data, follow what the generators do with it, and create a VM of your own through the Service Catalog. ## demo-otn This demo models a European research-network optical core in Infrahub, down to - [Infrahub OTN demo](/demo-otn.md): This demo models a European research-network optical core in Infrahub, down to ### ai-payloads Distributed training traffic runs on this network in operation, so the model - [AI and HPC payloads](/demo-otn/ai-payloads.md): Distributed training traffic runs on this network in operation, so the model ### client-mapping A customer hands over an Ethernet port, an SDH circuit, a 2 Mbps tributary or an - [Client mapping](/demo-otn/client-mapping.md): A customer hands over an Ethernet port, an SDH circuit, a 2 Mbps tributary or an ### concepts The plant layer is four kinds: a fiber span, an optical multiplex section, a - [Optical plant concepts](/demo-otn/concepts.md): The plant layer is four kinds: a fiber span, an optical multiplex section, a ### demo-guide Nine scenarios against the loaded GÉANT dataset. Every number on the three - [Demo guide](/demo-otn/demo-guide.md): Nine scenarios against the loaded GÉANT dataset. Every number on the three ### developer-guide How this repository is laid out, how to change it, and how to check that you did - [Developer guide](/demo-otn/developer-guide.md): How this repository is laid out, how to change it, and how to check that you did ### installation-setup Prerequisites - [Install and load the demo](/demo-otn/installation-setup.md): Prerequisites ### link-budget A wavelength either closes or it does not. The link budget is the arithmetic - [Link budget](/demo-otn/link-budget.md): A wavelength either closes or it does not. The link budget is the arithmetic ### loadable-scenarios Three pairs of scenarios and six singles, one task each, then the runbook that - [Loadable scenarios and the runbook](/demo-otn/loadable-scenarios.md): Three pairs of scenarios and six singles, one task each, then the runbook that ### network-map The fourteen PoPs of the modelled European optical core, with the 21 multiplex sections between them coloured by OSNR margin. Paris to Madrid is red, Frankfurt to Milan carries a 4,134 GHz occupancy chip, and Vienna to Milan is marked as Raman pumped and asymmetric. - [Network map](/demo-otn/network-map.md): The fourteen PoPs of the modelled European optical core, with the 21 multiplex sections between them coloured by OSNR margin. Paris to Madrid is red, Frankfurt to Milan carries a 4,134 GHz occupancy chip, and Vienna to Milan is marked as Raman pumped and asymmetric. ### odu-map Which of the two maps you are looking at - [ODU map](/demo-otn/odu-map.md): Which of the two maps you are looking at ### provisioning-scenarios The two scenarios that write. One provisions a service and shows what the route - [Provisioning scenarios](/demo-otn/provisioning-scenarios.md): The two scenarios that write. One provisions a service and shows what the route ### quickstart Go from a fresh clone to a proposed change Infrahub will not merge. You - [Quick start](/demo-otn/quickstart.md): Go from a fresh clone to a proposed change Infrahub will not merge. You ### reporting-scenarios The six scenarios that read rather than write, and the two maps every PoP - [Reporting scenarios](/demo-otn/reporting-scenarios.md): The six scenarios that read rather than write, and the two maps every PoP ### schema-reference The OTN schema is 50 kinds across eight files in schemas/. Eight - [Schema reference](/demo-otn/schema-reference.md): The OTN schema is 50 kinds across eight files in schemas/. Eight ### spectral-model A wavelength occupies a width of spectrum, not a channel number. It is centred on - [Spectral model](/demo-otn/spectral-model.md): A wavelength occupies a width of spectrum, not a channel number. It is centred on ### what-this-shows Two readers land on this page. One is evaluating Infrahub and has no interest - [What this demo shows](/demo-otn/what-this-shows.md): Two readers land on this page. One is evaluating Infrahub and has no interest ## demo-service-catalog A proof of concept for a service catalog using Infrahub and Streamlit. - [Home](/demo-service-catalog.md): A proof of concept for a service catalog using Infrahub and Streamlit. ### getting-started - [Developer Walkthrough](/demo-service-catalog/getting-started/developer-walkthrough.md): A step-by-step guide to running the demo from a developer perspective. - [How to Install the Service Catalog Demo](/demo-service-catalog/getting-started/installation.md): Set up the Service Catalog demo environment with Infrahub and Streamlit to enable self-service network provisioning. - [How to Use the Service Catalog Demo](/demo-service-catalog/getting-started/user-walkthrough.md): Experience the complete service delivery workflow from request to provisioning using both the Service Catalog portal and Infrahub interface. ## emma Emma is an experimental AI-powered assistant designed to help you interact with Infrahub, OpsMill's next-generation infrastructure management platform. Emma helps you manage infrastructure schemas and data through an intuitive web interface. - [Welcome to Emma](/emma.md): Emma is an experimental AI-powered assistant designed to help you interact with Infrahub, OpsMill's next-generation infrastructure management platform. Emma helps you manage infrastructure schemas and data through an intuitive web interface. ### features - [Data Import and Export](/emma/features/data-import-export.md): Emma provides powerful tools for importing data into Infrahub from CSV files and exporting Infrahub data back to CSV format. This enables efficient migration from existing systems and data sharing between tools. - [AI Schema Builder](/emma/features/schema-builder.md): Emma's Schema Builder is an AI-powered tool that helps you create Infrahub schemas using natural language descriptions. Instead of manually writing YAML schema definitions, you can describe what you need and let Emma's AI assistant generate the appropriate schema structure. - [Schema Management](/emma/features/schema-management.md): Emma provides comprehensive tools for managing Infrahub schemas throughout their lifecycle. From loading and visualizing schemas to organizing them in libraries, Emma makes schema management accessible and efficient. ### getting-started - [Configuration](/emma/getting-started/configuration.md): Emma connects to Infrahub using standard environment variables and configuration options. This guide covers all the ways to configure Emma for your environment. - [First Steps](/emma/getting-started/first-steps.md): Welcome to Emma! This guide will walk you through your first experience using Emma to interact with Infrahub. - [Installation](/emma/getting-started/installation.md): Emma is an experimental AI-powered assistant for Infrahub that helps you manage infrastructure schemas and data through an intuitive web interface. ### guides - [Building Your First Schema](/emma/guides/building-your-first-schema.md): This guide walks you through creating your first Infrahub schema using Emma's AI Schema Builder. We'll create a practical schema for modeling network devices. ### reference - [Configuration Reference](/emma/reference/configuration.md): This reference covers all configuration options available in Emma, including environment variables, feature flags, and runtime settings. - [Feature Flags](/emma/reference/feature-flags.md): Emma uses feature flags to enable experimental and beta functionality. This allows you to try new features while maintaining stability in production environments. - [Troubleshooting](/emma/reference/troubleshooting.md): This guide covers common issues you might encounter while using Emma and how to resolve them. ## exporter Infrahub Exporter is a service that exports metrics and service discovery information from Infrahub to monitoring systems like Prometheus and OpenTelemetry. - [Infrahub Exporter](/exporter.md): Infrahub Exporter is a service that exports metrics and service discovery information from Infrahub to monitoring systems like Prometheus and OpenTelemetry. ### guides - [Configure Infrahub Exporter](/exporter/guides/configuration.md): Configuration - [Install and run Infrahub Exporter](/exporter/guides/installation.md): This guide walks you through installing and running the Infrahub Exporter on your system. ## infrahub-demo-sp Welcome to the Infrahub demo-sp example — a service-provider bundle that - [Infrahub demo-sp example](/infrahub-demo-sp.md): Welcome to the Infrahub demo-sp example — a service-provider bundle that ### architecture This page summarises the data flow and the role of each directory. - [Architecture](/infrahub-demo-sp/architecture.md): This page summarises the data flow and the role of each directory. ### lab - [Containerlab](/infrahub-demo-sp/lab/containerlab.md): This page covers the optional containerlab integration: what runs, which PEs ### quickstart This page gets you from clone to a running demo in about 10 minutes. For an - [Quick start](/infrahub-demo-sp/quickstart.md): This page gets you from clone to a running demo in about 10 minutes. For an ### schema-reference This page covers the SP-demo-specific schema nodes defined in schemas/sp/. - [Schema reference](/infrahub-demo-sp/schema-reference.md): This page covers the SP-demo-specific schema nodes defined in schemas/sp/. ### services - [L3VPN service](/infrahub-demo-sp/services/l3vpn.md): This page documents the L3VPN service end-to-end: how an operator creates - [SD-WAN service](/infrahub-demo-sp/services/sdwan.md): The SD-WAN service models a customer's SD-WAN overlay across one or more ### troubleshooting Common failure modes and their fixes. If your problem is not here, check - [Troubleshooting](/infrahub-demo-sp/troubleshooting.md): Common failure modes and their fixes. If your problem is not here, check ### validation - [Batfish validation](/infrahub-demo-sp/validation/batfish.md): BatfishBackboneCheck runs the Batfish query ## infrahub-solution-ai-dc ### solution-ai-dc - [Demo guide](/infrahub-solution-ai-dc/solution-ai-dc/demo-guide.md): Run the AI/DC solution end to end — from loading design data to generating a complete data center fabric. - [Design-driven automation](/infrahub-solution-ai-dc/solution-ai-dc/design-driven-automation.md): How the AI/DC solution separates design intent from infrastructure implementation using Infrahub Generators. - [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md): Multi-tenant EVPN/VXLAN overlay services layered on top of the AI/DC solution's OSPF-underlay Clos fabric. - [Generator patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md): Implementation patterns used in the AI/DC solution Generators — queries, device creation, IP allocation, cabling, and checksum propagation. - [Installation & Setup](/infrahub-solution-ai-dc/solution-ai-dc/installation-setup.md): Prerequisites, environment setup, and repository structure for the AI/DC solution. - [Modular Generator architecture](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md): How the AI/DC solution chains Generators through checksum triggers, validates parent completion, and allocates interfaces deterministically. - [Multivendor configuration](/infrahub-solution-ai-dc/solution-ai-dc/multivendor-config.md): How the AI/DC solution renders per-vendor (Cisco, Arista, Dell, Juniper) device configuration from a single design model using vendor device groups. - [AI/DC solution](/infrahub-solution-ai-dc/solution-ai-dc/overview.md): Design-driven automation for large-scale AI data center fabrics using Infrahub Generators. ## infrahubctl ### infrahubctl infrahubctl is a command line utility designed to help with the day to day management of an Infrahub installation. - [infrahubctl](/infrahubctl/infrahubctl.md): infrahubctl is a command line utility designed to help with the day to day management of an Infrahub installation. ### infrahubctl-branch Manage the branches in a remote Infrahub instance. - [infrahubctl branch](/infrahubctl/infrahubctl-branch.md): Manage the branches in a remote Infrahub instance. ### infrahubctl-check Execute user-defined checks. - [infrahubctl check](/infrahubctl/infrahubctl-check.md): Execute user-defined checks. ### infrahubctl-dump Export nodes and their relationships out of the database. - [infrahubctl dump](/infrahubctl/infrahubctl-dump.md): Export nodes and their relationships out of the database. ### infrahubctl-generator Run a generator script. - [infrahubctl generator](/infrahubctl/infrahubctl-generator.md): Run a generator script. ### infrahubctl-graphql Various GraphQL related commands. - [infrahubctl graphql](/infrahubctl/infrahubctl-graphql.md): Various GraphQL related commands. ### infrahubctl-info Display the status of the Python SDK. - [infrahubctl info](/infrahubctl/infrahubctl-info.md): Display the status of the Python SDK. ### infrahubctl-load Import nodes and their relationships into the database. - [infrahubctl load](/infrahubctl/infrahubctl-load.md): Import nodes and their relationships into the database. ### infrahubctl-marketplace Browse and download schemas from the Infrahub Marketplace. - [infrahubctl marketplace](/infrahubctl/infrahubctl-marketplace.md): Browse and download schemas from the Infrahub Marketplace. ### infrahubctl-menu Manage the menu in a remote Infrahub instance. - [infrahubctl menu](/infrahubctl/infrahubctl-menu.md): Manage the menu in a remote Infrahub instance. ### infrahubctl-object Manage objects in a remote Infrahub instance. - [infrahubctl object](/infrahubctl/infrahubctl-object.md): Manage objects in a remote Infrahub instance. ### infrahubctl-protocols Export Python protocols corresponding to a schema. - [infrahubctl protocols](/infrahubctl/infrahubctl-protocols.md): Export Python protocols corresponding to a schema. ### infrahubctl-render Render a local Jinja2 Transform for debugging purpose. - [infrahubctl render](/infrahubctl/infrahubctl-render.md): Render a local Jinja2 Transform for debugging purpose. ### infrahubctl-repository Manage the repositories in a remote Infrahub instance. - [infrahubctl repository](/infrahubctl/infrahubctl-repository.md): Manage the repositories in a remote Infrahub instance. ### infrahubctl-run Execute a script. - [infrahubctl run](/infrahubctl/infrahubctl-run.md): Execute a script. ### infrahubctl-schema Manage the schema in a remote Infrahub instance. - [infrahubctl schema](/infrahubctl/infrahubctl-schema.md): Manage the schema in a remote Infrahub instance. ### infrahubctl-task Manage Infrahub tasks. - [infrahubctl task](/infrahubctl/infrahubctl-task.md): Manage Infrahub tasks. ### infrahubctl-telemetry Usage: - [infrahubctl telemetry](/infrahubctl/infrahubctl-telemetry.md): Usage: ### infrahubctl-transform Render a local transform (TransformPython) for debugging purpose. - [infrahubctl transform](/infrahubctl/infrahubctl-transform.md): Render a local transform (TransformPython) for debugging purpose. ### infrahubctl-validate Helper to validate the format of various files. - [infrahubctl validate](/infrahubctl/infrahubctl-validate.md): Helper to validate the format of various files. ### infrahubctl-version Display the version of Python and the version of the Python SDK in use. - [infrahubctl version](/infrahubctl/infrahubctl-version.md): Display the version of Python and the version of the Python SDK in use. ## integrations Infrahub integrates with a wide range of infrastructure management and automation tools to help you synchronize data, automate workflows, and manage your infrastructure. This page provides an overview of all available integrations and their capabilities. - [Infrahub Integrations](/integrations.md): Infrahub integrates with a wide range of infrastructure management and automation tools to help you synchronize data, automate workflows, and manage your infrastructure. This page provides an overview of all available integrations and their capabilities. ## mcp Infrahub MCP Server connects AI assistants and IDE agents to Infrahub using the open Model Context Protocol standard — so agents can query, create, update, and propose changes to your infrastructure data through a consistent, audited interface. It works with any MCP-compatible client (Claude Desktop, VS Code, Cursor, CLI agents, and more) with no custom glue code required. - [Infrahub MCP Server](/mcp.md): Infrahub MCP Server connects AI assistants and IDE agents to Infrahub using the open Model Context Protocol standard — so agents can query, create, update, and propose changes to your infrastructure data through a consistent, audited interface. It works with any MCP-compatible client (Claude Desktop, VS Code, Cursor, CLI agents, and more) with no custom glue code required. ### getting-started - [Set up authentication](/mcp/getting-started/authentication.md): This guide walks through configuring each authentication mode. For a deeper explanation of how the auth architecture works — sequence diagrams, the two-layer model, audit logs, and branch-placeholder resolution — see Authentication architecture. - [Your first agent run](/mcp/getting-started/first-agent-run.md): This walkthrough connects an LLM client to your running Infrahub MCP server and asks it a read-only question. By the end you'll have confirmed the server is reachable, the agent can discover your schema, and the built-in infrahub_agent system prompt is guiding the conversation correctly. - [Install the Infrahub MCP server](/mcp/getting-started/installation.md): This guide gets you from a fresh checkout to a running Infrahub MCP server in a few minutes. Once it's running, continue with Authentication to pick a credential model, then Your first agent run to connect an LLM. - [Make a change through an agent](/mcp/getting-started/make-a-change.md): This walkthrough takes you through the full write path: ask an agent to update something in Infrahub, watch it auto-create an isolated session branch, and open a Proposed Change for human review. No data on your default branch is modified until a human approves the merge. ### guides - [Running with Docker](/mcp/guides/docker.md): This guide covers deploying the Infrahub MCP server as a container, including running it alongside Infrahub in the same Docker Compose stack. - [Docker Compose deployment](/mcp/guides/docker-compose.md): This guide covers running the Infrahub MCP server as a container — standalone for development, or as a sidecar alongside Infrahub in the same Docker Compose stack for production. - [Installing Infrahub MCP](/mcp/guides/installation.md): This guide provides step-by-step instructions for installing and configuring different MCP clients to connect to the Infrahub MCP server. ### integrations - [Claude Agent SDK](/mcp/integrations/claude-agent-sdk.md): The Claude Agent SDK is Anthropic's framework for building custom agents on top of the Claude API. It has native MCP support, so connecting to the Infrahub MCP server is a short wiring exercise. - [Claude Code](/mcp/integrations/claude-code.md): Claude Code is Anthropic's CLI/IDE agent. It auto-detects .mcp.json files at project roots and loads the servers on startup — a perfect fit for the Infrahub MCP server when you're working inside a specific Infrahub consumer repository (IaC, Ansible, Python clients, etc.). - [Claude Desktop](/mcp/integrations/claude-desktop.md): Claude Desktop is the default starting point for most users — it speaks MCP over stdio and can spawn the Infrahub MCP server locally without any infrastructure. - [Cursor](/mcp/integrations/cursor.md): Cursor supports MCP servers via its Tools & Integrations panel. This page shows the Infrahub MCP server configuration and notes specific to Cursor's tool-approval UX. - [OpenAI Agents SDK](/mcp/integrations/openai-agents-sdk.md): The OpenAI Agents SDK has first-class support for MCP servers. This page shows how to build a custom infrastructure-automation agent that talks to the Infrahub MCP server over Streamable HTTP. - [VS Code](/mcp/integrations/vscode.md): VS Code supports MCP through its extension API — the Infrahub MCP server shows up as a tool provider alongside Copilot. This page covers the code --add-mcp setup and the VS Code Copilot compatibility flag. ### references - [Authentication architecture](/mcp/references/authentication.md): For the step-by-step configuration recipes, see Set up authentication. This page explains how each mode works end-to-end. - [Configuration reference](/mcp/references/configuration.md): The Infrahub MCP server is configured entirely through environment variables. All middleware features are opt-in — the server works out of the box with only the required connection variables set. - [Infrahub MCP methods](/mcp/references/methods.md): Response formats ### release-notes - [Release Notes](/mcp/release-notes.md) - [Release 1.0.0](/mcp/release-notes/release-1_0_0.md): The first stable release — query Infrahub and propose changes for review from any MCP-compatible client. - [Release 1.0.1](/mcp/release-notes/release-1_0_1.md): Error messages now include schema hints so AI agents can correct invalid queries and retry. - [Release 1.1.1](/mcp/release-notes/release-1_1_1.md): Run the MCP server as a shared service with pass-through authentication, read-only mode, and observability. - [Release 1.1.2](/mcp/release-notes/release-1_1_2.md): Dependency and internal tooling maintenance, with no changes to tools or runtime behavior. - [Release 1.1.3](/mcp/release-notes/release-1_1_3.md): Fix startup for non-HTTP transports, plus pipeline and tooling maintenance. - [Release 1.1.4](/mcp/release-notes/release-1_1_4.md): Align with current Infrahub SDK releases, plus routine dependency maintenance. - [Release 1.1.5](/mcp/release-notes/release-1_1_5.md): Validate a session branch before reuse so an unusable branch fails with a clear reason. - [Release 1.1.6](/mcp/release-notes/release-1_1_6.md): Keep an AI conversation going after its session branch is merged or deleted, with automatic recovery and a reset tool to start the next change set. - [Release 1.1.7](/mcp/release-notes/release-1_1_7.md): Trace how two infrastructure objects are connected and find what a change could reach, with two read-only graph-traversal tools backed by Infrahub 1.10. ### use-cases - [Brownfield network onboarding](/mcp/use-cases/brownfield-onboarding.md): Onboarding an existing ("brownfield") network into Infrahub is a long tail of small writes: one device per rack, one interface per device, one cable per interface. Agents shine here — they can read source-of-truth data from CSV exports, SNMP, LLDP neighbours, or NetBox, and turn it into idempotent Infrahub upserts on a session branch you can review all at once. - [Compliance analysis](/mcp/use-cases/compliance-analysis.md): Agents are a natural fit for ad-hoc compliance questions: "do all edge routers have dual uplinks to different spines?", "are all production VLANs documented?", "which devices are missing an owner?". This recipe shows how to run those checks as read-only agent workflows against Infrahub. - [Cross-system correlation](/mcp/use-cases/cross-system-correlation.md): Infrastructure data doesn't live in one place. A ticket in Jira mentions a device, a Git commit touches a configuration file, an Ansible run fails against a hostname. Connecting those dots — "which device is this about, what depends on it, what changed recently?" — is a natural fit for an agent that can read Infrahub through MCP and your other systems through their own tools. - [Natural language to GraphQL](/mcp/use-cases/natural-language-graphql.md): getnodes and searchnodes handle most everyday queries, but some questions need the full expressive power of GraphQL: aggregates, deep joins, multi-kind filters, and computed fields. This recipe shows how an agent goes from a free-form English question to a working query_graphql call, using the server's own schema as the grounding. - [Safe changes via branch isolation](/mcp/use-cases/safe-changes-branch-isolation.md): Branch isolation is the single most important safety property of the Infrahub MCP server: an agent can never write directly to your default branch. Every mutation lives on an ephemeral session branch that only becomes part of the default branch once a human reviews and merges a Proposed Change. This page explains the model end-to-end and shows how to tune it. - [Troubleshooting data queries](/mcp/use-cases/troubleshooting-queries.md): When an agent answers "I couldn't find that device" — it's almost always a data-query problem, not a data problem. This recipe walks through the diagnostic flow for "why is my query returning nothing?" ## nornir This integration bridges Nornir's powerful automation framework with Infrahub's next-generation infrastructure data management platform. - [Nornir Integration with Infrahub](/nornir.md): This integration bridges Nornir's powerful automation framework with Infrahub's next-generation infrastructure data management platform. ### getting-started This guide helps you set up your first integration between Nornir and Infrahub. By the end, you'll have a working automation setup that demonstrates the power of dynamic inventory management. - [Getting Started](/nornir/getting-started.md): This guide helps you set up your first integration between Nornir and Infrahub. By the end, you'll have a working automation setup that demonstrates the power of dynamic inventory management. ### guides - [How to Configure Schema Mappings](/nornir/guides/configuring-schema-mappings.md): This guide shows you how to configure schema mappings to control how Infrahub data translates into Nornir inventory properties. Schema mappings are the bridge between your Infrahub data model and Nornir's expected host attributes. - [How to Create Your First Automation Workflow](/nornir/guides/your-first-automation-workflow.md): This guide walks you through building a complete network automation workflow using Infrahub as your inventory source and artifact management system. You'll retrieve device configurations managed by Infrahub. ### references - [Artifact management plugin](/nornir/references/plugins/artifact_tasks.md): regeneratehostartifact - [CoreFileObject management tasks](/nornir/references/plugins/file_object_tasks.md): uploadfileobject - [Inventory plugin](/nornir/references/plugins/infrahub_inventory.md): InfrahubInventory ### topics - [Understanding Artifact Lifecycle Management](/nornir/topics/artifact-lifecycle-management.md): This topic explores how Infrahub's artifact system works with Nornir to enable sophisticated configuration management workflows. Understanding artifact lifecycle management is crucial for implementing robust, auditable network automation. - [Infrahub Inventory Concepts](/nornir/topics/infrahub-inventory-concepts.md): This topic provides a deep understanding of how Infrahub's graph database model maps to Nornir's inventory concepts. Understanding these mappings is essential for designing effective automation workflows. - [Understanding the Nornir-Infrahub Integration](/nornir/topics/understanding-nornir-infrahub-integration.md): This topic explains the core concepts behind the Nornir-Infrahub integration, how it works, and why it represents a paradigm shift in network automation. ## python-sdk ### guides - [Executing queries in a batch](/python-sdk/guides/batch.md): The Python SDK client allows you to group the execution of multiple queries in a batch. - [Branch management](/python-sdk/guides/branches.md): The Python SDK provides multiple methods to manage the branches in an Infrahub instance. - [How to create and configure an Infrahub client](/python-sdk/guides/client.md): This guide shows you how to create and configure an Infrahub client using the Python SDK. You'll learn how to set up authentication, configure proxy settings, and customize client behavior for your infrastructure automation workflows. - [Create, update and deleting nodes](/python-sdk/guides/create_update_delete.md): We will be using the following schema in this guide: - [Traversing the graph](/python-sdk/guides/graph_traversal.md): Graph traversal lets you discover how nodes are connected without knowing the relationship path in advance. The SDK exposes two client methods: - [Installing infrahub-sdk](/python-sdk/guides/installation.md): The Infrahub SDK for Python is available on PyPI and can be installed using the pip package installer. It is recommended to install the SDK into a virtual environment. - [Using the object-storage](/python-sdk/guides/object-storage.md): The Python SDK can be used to interface with Infrahub's object-storageartifact-file-storage/overview). - [Overview](/python-sdk/guides/python-typing.md): Use Python's type system with the Infrahub SDK to catch schema mismatches while you write code, not when it runs. This guide shows how to type your SDK calls with Protocols, generate typed classes from your own schema, and generate Pydantic models from your GraphQL queries. - [Querying data in Infrahub](/python-sdk/guides/query_data.md): We can query data in 3 ways using the SDK: - [Using Resource Managers](/python-sdk/guides/resource-manager.md): The goal of this guide is to show you how to create a resource pool using the Python SDK, and how you can allocated resources with them. - [Using the client store](/python-sdk/guides/store.md): The client in the SDK contains a store that is used to store objects in a local cache. - [Using the client tracking mode](/python-sdk/guides/tracking.md): The Python SDK provides a feature known as Tracking Mode. This mode allows for the aggregation and tracking of operations performed during a session, enhancing efficiency and data management. ### introduction The Infrahub Python SDK is a client library for interacting with Infrahub programmatically. It handles authentication, query construction, and data serialization so you can work with infrastructure data using native Python objects instead of raw API calls. - [Infrahub Python SDK](/python-sdk/introduction.md): The Infrahub Python SDK is a client library for interacting with Infrahub programmatically. It handles authentication, query construction, and data serialization so you can work with infrastructure data using native Python objects instead of raw API calls. ### reference - [Compatibility matrix](/python-sdk/reference/compatibility.md): This page documents which versions of the Infrahub Python SDK are compatible with each version of Infrahub. - [Python SDK Configuration](/python-sdk/reference/config.md): The Python SDK (Async or Sync) client can be configured using an instance of the Config class. - [Python SDK Templating](/python-sdk/reference/templating.md): Filters can be used when defining computed attributes or Jinja2 Transforms within Infrahub. ### sdk_ref - [infrahub_sdk.client](/python-sdk/sdk_ref/infrahub_sdk/client.md): Classes - [infrahub_sdk.graph_traversal.models](/python-sdk/sdk_ref/infrahub_sdk/graph_traversal/models.md): Pydantic models for the Infrahub graph-traversal queries (Infrahub 1.10+). - [infrahub_sdk.graph_traversal.query](/python-sdk/sdk_ref/infrahub_sdk/graph_traversal/query.md): GraphQL query strings and variable builders for graph traversal (Infrahub 1.10+). - [infrahub_sdk.node.node](/python-sdk/sdk_ref/infrahub_sdk/node.md): Classes - [infrahub_sdk.node.attribute](/python-sdk/sdk_ref/infrahub_sdk/node/attribute.md): Classes - [infrahub_sdk.node.constants](/python-sdk/sdk_ref/infrahub_sdk/node/constants.md): This module is empty or contains only private/internal implementations. - [infrahub_sdk.node.metadata](/python-sdk/sdk_ref/infrahub_sdk/node/metadata.md): Classes - [infrahub_sdk.node.parsers](/python-sdk/sdk_ref/infrahub_sdk/node/parsers.md): Functions - [infrahub_sdk.node.property](/python-sdk/sdk_ref/infrahub_sdk/node/property.md): Classes - [infrahub_sdk.node.related_node](/python-sdk/sdk_ref/infrahub_sdk/node/related_node.md): Classes - [infrahub_sdk.node.relationship](/python-sdk/sdk_ref/infrahub_sdk/node/relationship.md): Classes ### topics - [Manage data with Object files](/python-sdk/topics/object_file.md): Introduction - [Understanding tracking in the Python SDK](/python-sdk/topics/tracking.md): Introduction ## schema-library Welcome to the Schema Library for Infrahub! This repository offers a collection of schemas designed to streamline and standardize infrastructure-related data structures. - [Schema library for Infrahub](/schema-library.md): Welcome to the Schema Library for Infrahub! This repository offers a collection of schemas designed to streamline and standardize infrastructure-related data structures. ### contributing We welcome contributions and feedback! Please open an issue or submit a pull request to suggest additions, improvements, or to report bugs. - [Contributing](/schema-library/contributing.md): We welcome contributions and feedback! Please open an issue or submit a pull request to suggest additions, improvements, or to report bugs. ### reference - [Azure](/schema-library/reference/azure.md): This schema extension introduces cloud support for Microsoft Azure. - [Cable](/schema-library/reference/cable.md): This schema extension contains a basic Cable model allowing you to connect two endpoints. - [Circuit](/schema-library/reference/circuit.md): This schema extension adds Circuit and Circuit Endpoint nodes and links them to your infrastructure. A circuit could be a fiber connecting two sites, with one endpoint on each site. - [Circuit Contract](/schema-library/reference/circuit_contract.md): This schema extension adds a Circuit Contract node to capture the service agreement behind a circuit with its provider. - [Circuit Service](/schema-library/reference/circuit_service.md): This schema extension contains model coming on top of circuit to capture a single service shared across multiple circuits. - [Cluster](/schema-library/reference/cluster.md): This schema extension contains the foundations to capture clusters. With this one in place you can unlock various cluster flavours (a hosting cluster able to host VMs, a firewall cluster built with specific appliances ...). - [Compute](/schema-library/reference/compute.md): This schema extension captures physical servers and virtual machines, with a generic compute unit both inherit from and a generic for anything able to host VMs. Pair it with extensions/hosting_cluster to group compute units into clusters. - [Cross-Connect](/schema-library/reference/cross_connect.md): This schema extension adds a Cross-Connect node. Think of it as a cable operated by a provider: attach it to a location, then connect endpoints to it (e.g. the rear interface of a patch panel or a circuit endpoint). - [DCIM](/schema-library/reference/dcim.md): Base DCIM schema: device types, platforms, devices, and physical and virtual interfaces, plus the endpoint and connector generics that cables, circuits and cross-connects build on. - [Device Module](/schema-library/reference/device_module.md): This schema extension lets you capture module bays on a DcimPhysicalDevice and the modules installed into them (e.g. fan trays, line cards). A device can have any number of module bays, and each bay can optionally hold one module. - [PSU Module](/schema-library/reference/device_module_psu.md): This schema extension adds a PSU (Power Supply Unit) flavour on top of the generic Module and Module Type from extensions/device_module, so you can track power supplies installed in a device's module bays with PSU-specific attributes such as wattage and hot-swap capability. - [Firewall Policer](/schema-library/reference/firewall_policer.md): This schema extension contains models for traffic policers and the classes and rules that reference them, so you can describe rate limiting alongside the interfaces it is applied to. - [Hosting Cluster](/schema-library/reference/hosting_cluster.md): A generic cluster built from compute units (e.g. servers) that can host virtual machines. - [InfiniBand Fabric Hardware](/schema-library/reference/infiniband.md): This schema extension models the hardware of an InfiniBand fabric: switches with their configuration profile (AAA, role, split-ready, SSH and CLI settings), switch and management interfaces with lane width, split and speed options, rail-optimized scalable units (RSU) and the fabric that groups them. It does not model the InfiniBand protocol itself (subnet manager, partitions, routing). - [Interface Breakout](/schema-library/reference/interface_breakout.md): This schema extension introduces relationships to support breakout interfaces, enabling you to document the breakout of a physical interface into smaller physical interfaces. - [IPAM](/schema-library/reference/ipam.md): Base IPAM schema: IP prefixes and IP addresses, plus the scope generics a prefix or a VLAN group can be attached to. - [IPAM Aggregate](/schema-library/reference/ipam_aggregate.md): This schema extension captures the top-level IPv4 and IPv6 blocks assigned by a Regional Internet Registry (RIR). It adds an Aggregate node for the block and an RIR node for the registry or private entity that manages the address space. - [LAG](/schema-library/reference/lag.md): This schema extension includes models for Link Aggregation Groups (LAGs), enabling you to link physical interfaces as building blocks of your LAG interface. It can be used in standard networking environments as well as in compute scenarios, such as capturing bond interfaces. - [Location](/schema-library/reference/location.md): Base Location schema: the generic every location tier inherits from, and the hosting generic that lets devices and other objects be placed at a location. Concrete tiers such as Site come from the location extensions. - [Location Extended](/schema-library/reference/location_extended.md): This schema extension is the most detailed when it comes to location, you'll find all the layers you can think of. It defines its own hierarchical Location.Rack, incompatible with the flat one extensions/rack defines, so load one or the other, not both. - [Location Minimal](/schema-library/reference/location_minimal.md): This schema extension provides a self-contained Region -> Country -> Metro -> Site hierarchy for storing location data, with the Site carrying facility, physical address, timezone and status. Its Site node is the same as the one in extensions/location_site, so the two can be loaded together. A location name is unique across every tier, so a single-country deployment should enter the hierarchy at Country, with Region as the national node, rather than repeating a country under several regions. - [Location Site](/schema-library/reference/location_site.md): This schema extension introduces a Site node with facility, physical address, timezone and status, for deployments that want a flat list of sites without a hierarchy. It is the same Site node as in extensions/location_minimal, which adds Region, Country and Metro tiers above it. - [MLAG](/schema-library/reference/mlag.md): This schema extension contains the foundations to capture Multi-Chassis Link Aggregation Groups (MLAG). It comes on top of the LAG extension. - [Module Port](/schema-library/reference/module_port.md): This schema extension adds module ports: the ports a module provides, as declared by its module type - what NetBox module-type definitions list under interfaces, console-ports and power-ports. - [Modules Linecards](/schema-library/reference/modules_linecards.md): This schema extension allows you to capture Linecard related information like the version. You can insert the Linecard into a Dcim Physical Device and leverage the Linecard type model. The Linecard can accept PIC to help configure PORT information like breakout-capabilities and configurations. - [Modules Routing Engine](/schema-library/reference/modules_routing_engine.md): This schema extension allows you to capture Routing Engine related information like the version. You can insert the Routing Engine into a Dcim Physical Device and leverage the Routing Engine type model. - [Optical Multiplexer](/schema-library/reference/optical_multiplexer.md): This schema extension models optical add-drop multiplexers (OADM) and the wavelength division multiplexing (WDM) channels they carry, for both CWDM and DWDM. It adds an Optical Multiplexer device with front and rear interfaces, a WDM Channel node holding channel number, wavelength and frequency, and a WDM Transceiver flavour of the transceiver model tuned to a channel. - [Optical Transport](/schema-library/reference/optical_transport.md): Comprehensive optical transport network schemas for DWDM/WDM systems (ADVA FSP 3000 and similar platforms). Covers four layers: wavelength (ITU-T G.694.1 grid, optical bands, DWDM channels), topology (logical optical nodes, passive multiplexers, fiber links), equipment (transponder/amplifier/ROADM modules, ROADM degrees, WSS cross-connects), and service (end-to-end optical services, optical paths, path segments). Not designed to be loaded together with extensions/optical_multiplexer. - [Organization](/schema-library/reference/organization.md): Base Organization schema: the generic organization plus Manufacturer and Provider nodes. - [Patch Panel](/schema-library/reference/patch_panel.md): This schema extension allows you to capture patch panel related information like rear and front interfaces and the mapping between them. You can insert the patch panel into a rack and leverage the device type model. Cassettes and other inserts are tracked as regular device modules in module bays, through extensions/device_module. - [Internet Exchange](/schema-library/reference/peering_ixp.md): This schema extension adds an Internet Exchange (IXP) node and an IXP Connection node linking your devices to an exchange, and ties IXP connections to the BGP sessions established over them. - [Physical Disk](/schema-library/reference/physical_disk.md): Simple schema allowing you to capture physical disk information for inventory and lifecycle management. - [QinQ](/schema-library/reference/qinq.md): This schema extension adds the Service VLAN and Customer VLAN nodes needed to model QinQ on top of the VLAN extension. - [QoS](/schema-library/reference/qos.md): This schema extension contains models for Quality of Service (QoS) - [Rack](/schema-library/reference/rack.md): This schema extension introduces a Rack node representing a physical equipment rack installed at a site. Racks are placed at a LocationSite, which comes from extensions/locationsite or extensions/locationminimal, and can host devices and other equipment. experimental/location_extended defines its own Rack node, so do not load both. - [Routing](/schema-library/reference/routing.md): This schema extension contains the Routing Protocol generic that every protocol extension builds on. The idea is one protocol instance per IpamVRF and DcimDevice pair. - [Routing Aggregate](/schema-library/reference/routing_aggregate.md): This schema extension adds an Aggregate Route node: a summary route, with its destination prefix and discard flag, announced by a routing protocol instance. - [Routing BGP](/schema-library/reference/routing_bgp.md): This schema extension adds Autonomous System, BGP Peer Group and BGP Session nodes, and links autonomous systems to devices and organizations. - [Routing BGP Community](/schema-library/reference/routing_bgp_community.md): This schema extension adds a BGP Community node to reference from routing policies. - [Routing BGP Route Reflector](/schema-library/reference/routing_bgp_rr.md): This schema extension adds a Route Reflector Cluster node on top of the BGP extension and links it to BGP peer groups. - [Routing OSPF](/schema-library/reference/routing_ospf.md): This schema extension adds OSPF and OSPF Interface nodes to model an OSPF instance per device and VRF and the interfaces it runs on. - [Routing PIM](/schema-library/reference/routing_pim.md): This schema extension adds PIM (Protocol Independent Multicast) and PIM Interface nodes to model a PIM instance per device and VRF and the interfaces it runs on. - [Routing Policies](/schema-library/reference/routing_policies.md): This schema extension contains the Routing Policy generic. Each routing protocol extension subclasses it with its own policy flavour. - [Routing Policies Aggregate](/schema-library/reference/routing_policies_aggregate.md): This schema extension adds an Aggregate Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on Aggregate Route with relationships to it. - [Routing Policies BGP](/schema-library/reference/routing_policies_bgp.md): This schema extension adds a BGP Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on BGP Peer Group and BGP Session with relationships to it. - [Routing Policies OSPF](/schema-library/reference/routing_policies_ospf.md): This schema extension adds an OSPF Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on OSPF with relationships to it. - [Routing Policies PIM](/schema-library/reference/routing_policies_pim.md): This schema extension adds a PIM Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on PIM with relationships to it. - [Security](/schema-library/reference/security.md): This schema extension contains models for implementing detailed security. - [SNMP](/schema-library/reference/snmp.md): This schema extension contains models for SNMP Communities and SNMP Clients. A community links to the devices it is configured on; any further wiring, to a tenant for instance, is left to you. - [Tenancy](/schema-library/reference/tenancy.md): This schema extension introduces a Tenant node that can own devices, IP prefixes and addresses, and hosting locations. See the comment at the bottom of tenancy.yml for an example of how to extend tenancy onto optional extension nodes such as DcimCircuit. - [Topology](/schema-library/reference/topology.md): A schema for defining and managing network topology, strategies, and services. - [Transceiver](/schema-library/reference/transceiver.md): This schema extension models pluggable transceivers across form factors (SFP, SFP+, QSFP, QSFP28, QSFP-DD, OSFP, CFP, XFP and more), with a Standard Transceiver and a Bidirectional Transceiver flavour built on a shared generic. A transceiver is either plugged into a physical interface or stored as a spare at a location, and can be linked to its manufacturer. - [VLAN](/schema-library/reference/vlan.md): This schema extension adds VLAN and VLAN Group nodes, with a generic VLAN to build on, and links VLANs to prefixes and layer 2 interfaces. - [VLAN Translation](/schema-library/reference/vlan-translation.md): This schema extension is based on Juniper VLAN MAP, and not yet test out for other vendors. - [VRF](/schema-library/reference/vrf.md): This schema extension adds VRF and Route Target nodes and links prefixes and IP addresses to a VRF. - [VRRP](/schema-library/reference/vrrp.md): This schema extension adds VRRP Group and VRRP Interface nodes, linking a virtual IP address to the layer 3 interfaces that take part in the group. ## skills Infrahub Skills is an open-source AI skills package that gives your AI coding assistant built-in knowledge of Infrahub's data model, conventions, and workflow patterns. Instead of learning the platform from documentation alone, describe what you want to build in plain language and the skills produce valid, best-practice Infrahub resources — schemas, object data, generators, transforms, checks, and menus — ready to load into a running instance. - [Infrahub Skills](/skills.md): Infrahub Skills is an open-source AI skills package that gives your AI coding assistant built-in knowledge of Infrahub's data model, conventions, and workflow patterns. Instead of learning the platform from documentation alone, describe what you want to build in plain language and the skills produce valid, best-practice Infrahub resources — schemas, object data, generators, transforms, checks, and menus — ready to load into a running instance. ### how-it-works Infrahub Skills gives an AI assistant domain-specific knowledge about the Infrahub platform. Each skill is a structured set of Markdown files — rules, examples, and references — that the AI reads at the point it's needed. The assistant matches each user request to the relevant skill automatically, loads only the context required for that task, and applies the rules embedded in the skill. - [How It Works](/skills/how-it-works.md): Infrahub Skills gives an AI assistant domain-specific knowledge about the Infrahub platform. Each skill is a structured set of Markdown files — rules, examples, and references — that the AI reads at the point it's needed. The assistant matches each user request to the relevant skill automatically, loads only the context required for that task, and applies the rules embedded in the skill. ### installation-setup Infrahub Skills can be installed into any AI tool that supports skills or custom context files. Installation takes one command for most setups — install into a specific Infrahub repository to work in that project, or install globally to use the skills across all your projects. - [Installation & Setup](/skills/installation-setup.md): Infrahub Skills can be installed into any AI tool that supports skills or custom context files. Installation takes one command for most setups — install into a specific Infrahub repository to work in that project, or install globally to use the skills across all your projects. ### release-notes What changed in each release of Infrahub Skills, and what is different about your workflow after upgrading. Newest first. - [Release Notes](/skills/release-notes.md): What changed in each release of Infrahub Skills, and what is different about your workflow after upgrading. Newest first. - [Release 0.0.1](/skills/release-notes/release-0_0_1.md): The first release of Infrahub Skills — eight skills that build, query, and audit Infrahub resources from natural language. - [Release 1.2.0](/skills/release-notes/release-1_2_0.md): Version realignment from 0.0.1 to 1.2.0 ahead of the automated release tooling. No skill changes. - [Release 1.2.1](/skills/release-notes/release-1_2_1.md): A documentation site for the skills package — installation, how-it-works, and a per-skill reference. - [Release 1.2.2](/skills/release-notes/release-1_2_2.md): New managing-schemas rules for production patterns, plus relationship and check-registration fixes. - [Release 1.2.3](/skills/release-notes/release-1_2_3.md): Every managing-* skill appears again after a fresh install; a frontmatter field had hidden six of them. - [Release 1.2.4](/skills/release-notes/release-1_2_4.md): analyzing-data now answers the question you asked instead of a built-in example when none is passed. - [Release 1.2.5](/skills/release-notes/release-1_2_5.md): Report Infrahub bugs and feature requests with the reporting-issues skill, sanitized and routed automatically. - [Release 1.2.6](/skills/release-notes/release-1_2_6.md): A diagnostics skill, CoreFileObject schema support, and a branch-first default for every data change. - [Release 1.2.7](/skills/release-notes/release-1_2_7.md): A CSV/TSV import skill, Profiles and Object Templates support, YAGNI audit rules, and diagnostics collection built on the infrahub-collect tool. - [Release 1.2.8](/skills/release-notes/release-1_2_8.md): Four new skills — NetBox device-type conversion, concept tutoring, diagnostic-bundle triage, and skill-gap reporting — plus verified generic and cardinality semantics and a corrected infrahubctl command surface. ### skills-reference - [Data Analyzer](/skills/skills-reference/analyzing-data.md): The Data Analyzer skill queries and analyzes data from a live Infrahub instance using the Infrahub MCP server. It answers operational questions that span multiple node types — correlating data, detecting drift, tracing service impact, auditing data quality — without requiring the user to write GraphQL queries manually. - [Diagnostics Analyzer](/skills/skills-reference/analyzing-diagnostics.md): The Diagnostics Analyzer reads a bundle the Diagnostics Collector already produced and turns its logs into a triage report. It reads the manifest before any log, sweeps every service for tracebacks and other error signals, correlates related errors into incidents with root causes separated from cascades, and searches existing opsmill/infrahub issues so you learn whether your crash is already known — without touching the running deployment, applying a fix, or filing anything on your behalf. - [Repo Auditor](/skills/skills-reference/auditing-repo.md): The Repo Auditor performs a comprehensive audit of an Infrahub repository against all Infrahub best practices. It checks schema files, object files, Python components (checks, generators, transforms), .infrahub.yml registration, and project structure — then generates a report identifying issues and explaining what to fix and why. - [Diagnostics Collector](/skills/skills-reference/collecting-diagnostics.md): The Diagnostics Collector guides you through producing a support bundle with OpsMill's infrahub-collect tool when Infrahub is misbehaving. It installs and verifies the binary, detects your Docker Compose or Kubernetes deployment, runs infrahub-collect create with the flags that match your symptom, walks you through reviewing the bundle before you share it, and hands off — without diagnosing root cause or filing anything on your behalf. - [NetBox Device Type Converter](/skills/skills-reference/converting-netbox-device-types.md): The NetBox Device Type Converter turns NetBox device-type and module-type YAML into Infrahub Object Templates, so you create a device with its ports already in place from one reusable definition. The input format is netbox-community/devicetype-library, which you can browse at the NetBox Data Exchange. A bundled Python script does the conversion, driven by a mapping profile that describes your target schema. Point it at the right schema and read what it reports back — without editing your schema for you or guessing a field name the profile does not define. - [Data Importer](/skills/skills-reference/importing-data.md): The Data Importer turns CSV or TSV inputs into Infrahub object YAML and loads it onto a fresh branch. It introspects the live schema, maps columns to attributes by heuristic plus an up-front interview, splits denormalized inputs across the right kinds with the correct load order, and fails closed when a column has no schema home — without ever proposing a schema edit. - [Check Manager](/skills/skills-reference/managing-checks.md): The Check Manager produces Python validation checks that run in Infrahub's proposed change pipeline. Each check consists of three components: a GraphQL query file, a Python class inheriting from InfrahubCheck, and a .infrahub.yml registration entry. The skill generates all three and ensures they are correctly wired together. - [Generator Manager](/skills/skills-reference/managing-generators.md): The Generator Manager produces Python generators that query Infrahub for design objects and automatically create or update the infrastructure objects derived from them. Generators are idempotent: re-running them updates existing objects rather than creating duplicates. The skill generates the Python class, GraphQL query, and .infrahub.yml registration. - [Menu Manager](/skills/skills-reference/managing-menus.md): The Menu Manager produces YAML menu definition files that customize the Infrahub web UI sidebar. Menus can be flat or deeply nested, with group headers, icons, and ordering. The skill generates syntactically correct menu YAML and handles the nesting structure that is easy to get wrong by hand. - [Object Manager](/skills/skills-reference/managing-objects.md): The Object Manager produces YAML data files for Infrahub infrastructure objects — devices, locations, organizations, interfaces, modules, and any custom node type. It applies the correct file structure, value mappings, and load order so files are ready to load into a running Infrahub instance. - [Schema Manager](/skills/skills-reference/managing-schemas.md): The Schema Manager produces valid Infrahub schema YAML from natural language descriptions. It applies Infrahub's naming conventions, relationship rules, attribute type requirements, and display property patterns automatically — without requiring manual study of the schema format. - [Transform Manager](/skills/skills-reference/managing-transforms.md): The Transform Manager produces data transforms that convert Infrahub data into other formats — device configurations, JSON exports, CSV reports, and more. Transforms can be implemented as Python classes, Jinja2 templates, or a hybrid of both. The skill generates the transform code, GraphQL query, optional Jinja2 templates, and .infrahub.yml registration. - [Issue Reporter](/skills/skills-reference/reporting-issues.md): The Issue Reporter files a bug report or feature request against the correct repository in the Infrahub ecosystem. There are 11 candidate repos and most people do not know which one owns their problem, so the skill reads your working directory for cues, proposes a target, searches for duplicates, and drafts an issue that matches that repo's intake form. It never auto-submits: it stops at a review gate and submits only after you approve both the content and the method. - [Skill Gap Reporter](/skills/skills-reference/reporting-skill-gaps.md): The Skill Gap Reporter turns friction with an Infrahub skill into a reviewed GitHub issue against opsmill/infrahub-skills. Infrahub skills fail quietly it hands the redacted draft to the Issue Reporter, which shows it to you and gets explicit approval before any submission. - [Concept Tutor](/skills/skills-reference/teaching-concepts.md): The Concept Tutor teaches Infrahub concepts starting from wherever the learner is, using their own repo files and live instance as lesson material instead of generic examples. It probes what the learner already knows, explains each concept anchored to their own schema and data, sets a hands-on exercise verified solvable before it's shown, and records progress so a later session resumes instead of restarting. ### spec-driven-development Spec-Driven Development (SDD) is a structured planning mode for complex or multi-part Infrahub builds. Instead of generating files immediately, the AI reasons through requirements with you first — capturing what needs to be built, validating the approach against Infrahub conventions, breaking the work into discrete tasks, and only generating once the plan is approved. - [Spec-Driven Development](/skills/spec-driven-development.md): Spec-Driven Development (SDD) is a structured planning mode for complex or multi-part Infrahub builds. Instead of generating files immediately, the AI reasons through requirements with you first — capturing what needs to be built, validating the approach against Infrahub conventions, breaking the work into discrete tasks, and only generating once the plan is approved. ## sync Infrahub Sync synchronizes infrastructure data between Infrahub and external systems — NetBox, Nautobot, IP Fabric, Slurp’it, Cisco ACI, Peering Manager, ServiceNow-style CMDBs, and any system with a REST/JSON API. Use it to migrate from a legacy system of record, run two systems side-by-side, or keep Infrahub in sync with another source of truth. Define a sync project declaratively in YAML, and the CLI generates the adapter code, calculates the diff, and applies only the deltas on each run. - [Infrahub Sync](/sync.md): Infrahub Sync synchronizes infrastructure data between Infrahub and external systems — NetBox, Nautobot, IP Fabric, Slurp’it, Cisco ACI, Peering Manager, ServiceNow-style CMDBs, and any system with a REST/JSON API. Use it to migrate from a legacy system of record, run two systems side-by-side, or keep Infrahub in sync with another source of truth. Define a sync project declaratively in YAML, and the CLI generates the adapter code, calculates the diff, and applies only the deltas on each run. ### adapters - [Cisco ACI adapter](/sync/adapters/aci.md): What is Cisco ACI? - [Choose an adapter](/sync/adapters/choosing-an-adapter.md): Infrahub Sync includes adapters for the most common infrastructure systems and a Generic REST API adapter for everything else. For systems with non-standard APIs, build a custom adapter. - [Device42 adapter](/sync/adapters/device42.md): What is Device42? - [GenericRestAPI adapter](/sync/adapters/genericrestapi.md): The GenericRestAPI adapter is a flexible, configurable adapter that can connect to any REST API following common patterns. It's designed to reduce code duplication and provide a foundation for creating adapters for new systems without having to build them from scratch. - [Infrahub adapter](/sync/adapters/infrahub.md): What is Infrahub? - [IP Fabric adapter](/sync/adapters/ipfabric.md): What is IP Fabric? - [LibreNMS adapter](/sync/adapters/librenms.md): What is LibreNMS? - [Local Adapters](/sync/adapters/local-adapters.md): Using local adapters - [Nautobot adapter](/sync/adapters/nautobot.md): What is Nautobot? - [NetBox adapter](/sync/adapters/netbox.md): What is NetBox? - [Observium adapter](/sync/adapters/observium.md): What is Observium? - [Peering Manager adapter](/sync/adapters/peering-manager.md): What is Peering Manager? - [PeeringDB adapter](/sync/adapters/peeringdb.md): What is PeeringDB? - [Prometheus adapter](/sync/adapters/prometheus.md): What is Prometheus? - [Slurp’it adapter](/sync/adapters/slurpit.md): What is Slurp’it? ### contributing This guide covers how to set up a development environment for infrahub-sync and contribute to the project. For the release runbook, see RELEASING.md at the repository root — that's maintainer-only. - [Contributing](/sync/contributing.md): This guide covers how to set up a development environment for infrahub-sync and contribute to the project. For the release runbook, see RELEASING.md at the repository root — that's maintainer-only. ### creating-a-sync-project A sync project is a directory containing a config.yml file that defines one synchronization between two systems. Configure four parts: source and destination, sync order, schema mapping, and sync behavior. A Nautobot → Infrahub example runs through each. - [Create a sync project](/sync/creating-a-sync-project.md): A sync project is a directory containing a config.yml file that defines one synchronization between two systems. Configure four parts: source and destination, sync order, schema mapping, and sync behavior. A Nautobot → Infrahub example runs through each. ### custom-certificates For sync sources or destinations that use TLS certificates signed by an internal CA — common in enterprise networks — install the CA in the system trust store, then point Python's HTTP client at that store. - [Use custom CA certificates](/sync/custom-certificates.md): For sync sources or destinations that use TLS certificates signed by an internal CA — common in enterprise networks — install the CA in the system trust store, then point Python's HTTP client at that store. ### installation Infrahub Sync is available on PyPI and can be installed using the pip package installer. It is recommended to install the Sync into a virtual environment. - [Install Infrahub Sync](/sync/installation.md): Infrahub Sync is available on PyPI and can be installed using the pip package installer. It is recommended to install the Sync into a virtual environment. ### migrating-from-netbox-or-nautobot Migrate data from NetBox or Nautobot into Infrahub one model at a time. The existing system keeps running throughout, and you decide when — or whether — to retire it. - [Migrate from NetBox or Nautobot](/sync/migrating-from-netbox-or-nautobot.md): Migrate data from NetBox or Nautobot into Infrahub one model at a time. The existing system keeps running throughout, and you decide when — or whether — to retire it. ### orchestration Infrahub Sync runs as a single CLI command. Schedule it with whatever tooling already runs scheduled jobs in your environment — cron, CI, Prefect, Dagster, or a homegrown runner. - [Schedule sync runs](/sync/orchestration.md): Infrahub Sync runs as a single CLI command. Schedule it with whatever tooling already runs scheduled jobs in your environment — cron, CI, Prefect, Dagster, or a homegrown runner. ### reference - [Cache layout](/sync/reference/cache-layout.md): infrahub-sync diff and infrahub-sync apply persist run state under: - [infrahub-sync](/sync/reference/cli.md): Infrahub-sync: synchronize data between infrastructure sources and destinations. - [Sync instance configuration](/sync/reference/config.md): The configuration file allows you to define the resources needs for the sync. - [Incremental Extraction](/sync/reference/incremental-extraction.md): infrahub-sync can skip re-extracting unchanged data on warm runs by - [Schema mapping reference](/sync/reference/schema-mapping.md): The schema mapping section of a sync project's config.yml defines how data translates between the source system and the destination system. Worked examples for NetBox → Infrahub and Nautobot → Infrahub appear at the end. ### release-notes - [Release Notes](/sync/release-notes.md) - [Infrahub Sync](/sync/release-notes/infrahub-sync.md) - [Release 1.5.6](/sync/release-notes/infrahub-sync/release-1_5_6.md): Release Number - [Release 1.6.0](/sync/release-notes/infrahub-sync/release-1_6_0.md): Release Number - [Release 2.0.0](/sync/release-notes/infrahub-sync/release-2_0_0.md): Release Number - [Release 2.0.1](/sync/release-notes/infrahub-sync/release-2_0_1.md): Release Number ### running-a-sync Learn how to use Infrahub Sync's commands to calculate differences, synchronize data, and apply previously cached plans against your destination. - [Run a sync](/sync/running-a-sync.md): Learn how to use Infrahub Sync's commands to calculate differences, synchronize data, and apply previously cached plans against your destination. ### tutorials - [Sync NetBox to Infrahub](/sync/tutorials/netbox-demo-to-infrahub.md): Set up a local Infrahub instance, load a data model, and synchronize infrastructure data from the public NetBox demo. ### using-netbox-or-nautobot-with-infrahub If you're adopting Infrahub but already have an established system of record in NetBox or Nautobot — with accumulated data, custom fields, and automation built around it — run Infrahub side-by-side with the existing tool and use Infrahub Sync to keep both systems holding the same data. - [Use NetBox or Nautobot with Infrahub](/sync/using-netbox-or-nautobot-with-infrahub.md): If you're adopting Infrahub but already have an established system of record in NetBox or Nautobot — with accumulated data, custom fields, and automation built around it — run Infrahub side-by-side with the existing tool and use Infrahub Sync to keep both systems holding the same data. ## vscode Infrahub VSCode extension - your development companion for infrastructure automation - [VSCode Extension](/vscode.md): Infrahub VSCode extension - your development companion for infrastructure automation ### guides - [How to Configure Multiple Infrahub Servers](/vscode/guides/configure-multiple-servers.md): Set up connections to development, staging, and production Infrahub servers for seamless environment switching - [How to Execute GraphQL Queries](/vscode/guides/execute-graphql-queries.md): Run GraphQL queries against Infrahub servers with variable support and branch selection - [How to Manage Branches](/vscode/guides/manage-branches.md): Create, delete, and work with Infrahub branches directly from VSCode - [How to Run Transforms and Artifacts](/vscode/guides/running-transforms.md): Execute Jinja2 and Python transforms directly from VSCode with automatic command selection and variable support - [How to use Infrahub snippets in VSCode](/vscode/guides/snippets.md): Step-by-step guide to inserting and customizing Infrahub YAML and automation snippets in Visual Studio Code - [How to Visualize Your Schema](/vscode/guides/visualize-schema.md): Explore and understand your Infrahub schema structure using the interactive graph visualizer ### reference - [Extension Commands and Settings Reference](/vscode/reference/commands-settings.md): Complete reference for all commands, settings, and configuration options in the Infrahub VSCode extension ### topics - [Understanding the Extension Architecture](/vscode/topics/extension-architecture.md): Deep dive into how the Infrahub VSCode extension is designed and how its components work together - [Schema Validation and YAML Intelligence](/vscode/topics/schema-validation.md): Understanding how the extension provides intelligent YAML editing and schema validation for Infrahub - [Security Configuration and Best Practices](/vscode/topics/security-configuration.md): Comprehensive guide to security settings, TLS configuration, and best practices for the Infrahub VSCode extension ### tutorials - [Getting Started with Infrahub VSCode Extension](/vscode/tutorials/getting-started.md): Learn how to install and configure the Infrahub VSCode extension for your first infrastructure automation project ## artifact-file-storage ### configure The storage API provides direct access to Infrahub's object storage layer for uploading and retrieving content by identifier. - [Configure storage](/artifact-file-storage/configure.md): The storage API provides direct access to Infrahub's object storage layer for uploading and retrieving content by identifier. ### overview Infrahub uses an object storage layer to persist binary and text content outside of the graph database. This layer stores the raw bytes of file objects and rendered artifacts. Separating file content from graph data allows Infrahub to keep the graph database focused on relationships, metadata, and version control while delegating bulk storage to a system optimized for that purpose. - [About storage](/artifact-file-storage/overview.md): Infrahub uses an object storage layer to persist binary and text content outside of the graph database. This layer stores the raw bytes of file objects and rendered artifacts. Separating file content from graph data allows Infrahub to keep the graph database focused on relationships, metadata, and version control while delegating bulk storage to a system optimized for that purpose. ## artifacts ### content-composition This guide shows you how to build a composite artifact whose Transformation pulls in content from other artifacts or file objects. This enables modular configuration pipelines where each artifact generates one section of a configuration, and a composite artifact assembles the final result. - [Composing artifact content](/artifacts/content-composition.md): This guide shows you how to build a composite artifact whose Transformation pulls in content from other artifacts or file objects. This enables modular configuration pipelines where each artifact generates one section of a configuration, and a composite artifact assembles the final result. ### overview An artifact is the result of a Transformation for a specific context and/or object. - [Artifacts](/artifacts/overview.md): An artifact is the result of a Transformation for a specific context and/or object. ### use Generate configuration files and other artifacts by combining Infrahub data with templates. This guide shows you how to create artifacts that automatically update when your infrastructure data changes. - [Use artifacts](/artifacts/use.md): Generate configuration files and other artifacts by combining Infrahub data with templates. This guide shows you how to create artifacts that automatically update when your infrastructure data changes. ## automation-and-outputs - [Design & Integrate](/automation-and-outputs.md) ## branches-and-change-control - [Branches & Change Control](/branches-and-change-control.md) ## branches ### branch-agnostic-data A branch-agnostic attribute or relationship on a branch-aware object stores its value once and shares that single value with every branch, instead of keeping a copy per branch. Deleting the object that holds such a field on one branch therefore does not release the value — the object may still be readable on another branch, and its branch-agnostic fields must remain readable with it. - [Branch-agnostic data](/branches/branch-agnostic-data.md): A branch-agnostic attribute or relationship on a branch-aware object stores its value once and shares that single value with every branch, instead of keeping a copy per branch. Deleting the object that holds such a field on one branch therefore does not release the value — the object may still be readable on another branch, and its branch-agnostic fields must remain readable with it. ### create Start a new branch from the default branch (typically main or master). This creates a snapshot of the current state that can be modified independently. Creating a new branch is almost instantaneous because Infrahub uses copy-on-write semantics — no data is duplicated, only a pointer to the base timestamp and the delta of subsequent changes. - [Create a branch](/branches/create.md): Start a new branch from the default branch (typically main or master). This creates a snapshot of the current state that can be modified independently. Creating a new branch is almost instantaneous because Infrahub uses copy-on-write semantics — no data is duplicated, only a pointer to the base timestamp and the delta of subsequent changes. ### delete After a branch has served its purpose — whether it's been merged, abandoned, or superseded — it can be deleted. Deleting a branch that has not been merged will permanently discard all changes accumulated on it. - [Delete a branch](/branches/delete.md): After a branch has served its purpose — whether it's been merged, abandoned, or superseded — it can be deleted. Deleting a branch that has not been merged will permanently discard all changes accumulated on it. ### merge Once changes are complete, they merge back into the parent branch through a controlled process — almost always via a Proposed Change. This ensures that all modifications are validated and conflicts are resolved before integration. - [Merge a branch](/branches/merge.md): Once changes are complete, they merge back into the parent branch through a controlled process — almost always via a Proposed Change. This ensures that all modifications are validated and conflicts are resolved before integration. ### overview Infrahub's branching model allows teams to work on features, fixes, or experiments in isolated environments. Each branch represents a separate line of development, enabling parallel work streams without interference. Unlike traditional version control systems, Infrahub's approach combines the familiar Git branching concepts with the unique data management capabilities of its graph database foundation. - [Branches](/branches/overview.md): Infrahub's branching model allows teams to work on features, fixes, or experiments in isolated environments. Each branch represents a separate line of development, enabling parallel work streams without interference. Unlike traditional version control systems, Infrahub's approach combines the familiar Git branching concepts with the unique data management capabilities of its graph database foundation. ### rebase Rebasing is a powerful operation that updates a branch with the latest changes from its parent branch. This is essential for maintaining branch health and ensuring smooth integration of changes. - [Rebase a branch](/branches/rebase.md): Rebasing is a powerful operation that updates a branch with the latest changes from its parent branch. This is essential for maintaining branch health and ensuring smooth integration of changes. ### resolve-conflicts Conflicts occur when the same data changes in both a feature branch and the main branch. Infrahub's conflict management system is designed to identify these conflicts with precision and provide tools for effective resolution. - [Resolve conflicts](/branches/resolve-conflicts.md): Conflicts occur when the same data changes in both a feature branch and the main branch. Infrahub's conflict management system is designed to identify these conflicts with precision and provide tools for effective resolution. ## category ### advanced-schema-features - [Advanced schema features](/category/advanced-schema-features.md) ### authentication - [Authentication](/category/authentication.md) ### configuration-files - [Configuration Files](/category/configuration-files.md) ### display--presentation - [Display & presentation](/category/display--presentation.md) ### install--configure - [Install & configure](/category/install--configure.md) ### maintain--upgrade - [Maintain & upgrade](/category/maintain--upgrade.md) ### permissions - [Permissions](/category/permissions.md) ### run--observe - [Run & observe](/category/run--observe.md) ### schema-operations - [Schema operations](/category/schema-operations.md) ### user-management--security - [User Management & Security](/category/user-management--security.md) ## change-approval ### change-approval-workflow This guide walks you through implementing a change approval workflow in Infrahub. By the end, you'll have a structured process ensuring all infrastructure changes are properly reviewed and approved before being merged into production, enhancing your governance and safety practices. - [How to implement a change approval workflow](/change-approval/change-approval-workflow.md): This guide walks you through implementing a change approval workflow in Infrahub. By the end, you'll have a structured process ensuring all infrastructure changes are properly reviewed and approved before being merged into production, enhancing your governance and safety practices. ## checks ### overview Checks are user-defined logic, stored in an external repository linked to Infrahub, that run as part of a proposed change. They let users perform any kind of data validation logic during a proposed change. If a check does not complete successfully, the proposed change cannot be merged. - [Checks & Validation](/checks/overview.md): Checks are user-defined logic, stored in an external repository linked to Infrahub, that run as part of a proposed change. They let users perform any kind of data validation logic during a proposed change. If a check does not complete successfully, the proposed change cannot be merged. ## computed-attributes ### overview What a computed attribute is, when to use it, and how Infrahub recalculates it as your data changes. - [Computed attributes](/computed-attributes/overview.md): What a computed attribute is, when to use it, and how Infrahub recalculates it as your data changes. ## deploy-manage ### install-configure - [How to configure Infrahub](/deploy-manage/install-configure/configure-infrahub.md): This guide explains how to configure your Infrahub instance by setting environment variables that control various aspects of the system, including timeouts, security settings, and integration parameters. - [Hardware requirements](/deploy-manage/install-configure/hardware-requirements.md): This page outlines the hardware requirements for running Infrahub, including minimum and recommended specifications, enterprise sizing, cloud provider machine types, task manager database storage, and a utility for benchmarking your system's performance. - [Install Infrahub Community](/deploy-manage/install-configure/install/community.md): Infrahub Community is deployed as a container-based architecture and can be installed using several methods. - [Install Infrahub Enterprise Enterprise Edition](/deploy-manage/install-configure/install/enterprise.md): Infrahub Enterprise is based on the Community version, with several enhancements for: - [Observability stack](/deploy-manage/install-configure/install/observability-stack.md): Infrahub ships an observability stack that runs alongside Infrahub: - [Installation](/deploy-manage/install-configure/install/overview.md): Infrahub Community and Enterprise are deployed as container-based architectures. The installation methods below are for non-resilient deployments suitable for development, testing, and single-node production environments. - [Tune performance](/deploy-manage/install-configure/performance-tuning.md): Infrahub's defaults target a general-purpose deployment. As an instance grows — more objects in the database, more open branches, more automation — some of those defaults stop matching the workload. The settings below are the ones worth revisiting, each with the symptom it addresses and what you give up in exchange. - [High availability](/deploy-manage/install-configure/production-deployment/high-availability.md): This guide covers how to deploy Infrahub in a high availability configuration that eliminates single points of failure. - [Production deployment](/deploy-manage/install-configure/production-deployment/overview.md): This guide walks you through deploying Infrahub in a production environment with enhanced security, reliability, and maintainability. By following these steps, you'll set up a production-ready Infrahub instance that follows industry best practices. - [Trust a private CA](/deploy-manage/install-configure/production-deployment/private-ca.md): This guide explains how to make Infrahub trust certificates issued by your organization's private certificate authority (CA). Once configured, every outbound TLS connection Infrahub opens, such as cloning a repository from an internal Git server, calling a webhook, reaching an SSO provider, or connecting to the database, the message broker and S3 object storage, trusts your CA without rebuilding the Infrahub Docker image. ### maintain-upgrade - [Backup and restore](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md): This guide shows you how to create comprehensive backups of your Infrahub deployment and restore them when needed. You'll learn to backup the Neo4j graph database, object storage, and task management data to ensure complete data recovery capabilities. - [Cluster backup and restore Enterprise Edition](/deploy-manage/maintain-upgrade/database-backup/cluster-backup-and-restore.md): If you're running Infrahub with a Neo4j cluster, follow these steps to backup from one node and restore to another while maintaining cluster integrity. - [Understanding database backup and restore](/deploy-manage/maintain-upgrade/database-backup/overview.md): This topic explains how Infrahub's database backup and restore system works, the architectural decisions behind it, and the various approaches available for protecting your data. Understanding these concepts helps you make informed decisions about your backup strategy and troubleshoot issues when they arise. - [Upgrade Community](/deploy-manage/maintain-upgrade/upgrade/community.md): The process to migrate your instance of Infrahub to the latest version will vary depending on your deployment of Infrahub. However, at a high-level, it will involve getting the latest version and performing any needed Database Migrations and Schema updates. - [Upgrade Enterprise Enterprise Edition](/deploy-manage/maintain-upgrade/upgrade/enterprise.md): The process to migrate your instance of Infrahub Enterprise to the latest version will vary depending on your deployment of Infrahub Enterprise. However, at a high-level, it will involve getting the latest version and performing any needed Database Migrations and Schema updates. - [Upgrade the observability stack](/deploy-manage/maintain-upgrade/upgrade/observability-stack.md): Docker Compose - [Upgrade](/deploy-manage/maintain-upgrade/upgrade/overview.md): Upgrading Infrahub involves pulling the latest container images, running database and schema migrations, and restarting services. The specific steps vary by edition (Community vs Enterprise) and deployment method (Docker Compose vs Helm). ### run-observe - [Activity log](/deploy-manage/run-observe/activity-log.md): Changes (events) in Infrahub are documented in the Activity log. It helps you see which objects were impacted, when a change was made, and who made it. It can be used to troubleshoot unforeseen changes, audit previous operations, and comprehend the order of updates across various branches. - [Configure log forwarding Enterprise Edition](/deploy-manage/run-observe/log-forwarding/configure-log-forwarding.md): This guide walks through configuring Infrahub to forward audit events and application logs to external SIEM or syslog systems. For conceptual background on how log forwarding works, see the log forwarding overview. - [Log forwarding Enterprise Edition](/deploy-manage/run-observe/log-forwarding/overview.md): Log forwarding enables Infrahub to stream audit events and application logs to external systems via the syslog protocol. This supports compliance, security monitoring, and operational visibility by integrating with SIEM platforms and centralized log management. - [Tasks](/deploy-manage/run-observe/tasks.md): The Tasks system in Infrahub is designed to manage and control various backend operations with robust error reporting, improved supervision, and enhanced logging capabilities. - [Local telemetry storage](/deploy-manage/run-observe/telemetry.md): Infrahub stores a daily telemetry snapshot locally in the Neo4j database, regardless of whether remote telemetry reporting is enabled. This ensures all deployments - including air-gapped and opted-out environments - retain usage data for support, auditing, and license compliance. ### user-management - [Authentication](/deploy-manage/user-management/authentication.md): Infrahub provides flexible authentication options to fit various organizational needs. - [Advanced LDAP configuration Enterprise Edition](/deploy-manage/user-management/ldap/advanced-ldap.md): These options extend a working LDAP setup with directory group resolution, nested-group handling, server failover, and stricter TLS. Configure the connection and user lookup first — see Configure LDAP authentication. - [Configure LDAP authentication Enterprise Edition](/deploy-manage/user-management/ldap/configure-ldap.md): This guide explains how to connect Infrahub to your LDAP directory so users can sign in with their directory credentials. - [LDAP authentication Enterprise Edition](/deploy-manage/user-management/ldap/overview.md): With LDAP authentication, users sign in to Infrahub with the credentials from your existing directory — Active Directory, OpenLDAP, or any RFC 4510-compliant LDAP server. Use it when your organization manages identities in a central directory and does not run an OIDC or OAuth2 identity provider. - [Managing API tokens](/deploy-manage/user-management/managing-api-tokens.md): API tokens can be used as an authentication mechanism for Infrahub's REST- and GraphQL API, the Python SDK and infrahubctl. - [Managing preferences](/deploy-manage/user-management/managing-preferences.md): Infrahub stores per-user preferences that control aspects of the web interface and how it displays data. Preferences follow your account to any session or device, and — with the manageglobalpreferences permission — an administrator can set organisation-wide defaults for users who haven't set their own. - [Manage accounts and permissions](/deploy-manage/user-management/permissions-roles/manage-accounts-and-permissions.md): In Infrahub, managing access and control starts with creating accounts, assigning them to groups, and managing their roles and permissions. - [Permissions and roles](/deploy-manage/user-management/permissions-roles/overview.md): Roles and permissions are essential for controlling user access and behavior in Infrahub. Within the platform, they provide precise control over what users can see, modify, or manage. - [Advanced SSO configuration](/deploy-manage/user-management/sso/advanced-sso.md): Multiple identity providers - [Configure SSO (single sign-on)](/deploy-manage/user-management/sso/configure-sso.md): This guide walks you through configuring single sign-on (SSO) in Infrahub using OpenID Connect (OIDC) or OAuth2 authentication protocols. - [Single sign-on (SSO)](/deploy-manage/user-management/sso/overview.md): Single sign-on (SSO) allows users to authenticate once with an external identity provider and gain access to Infrahub without needing separate credentials. Infrahub integrates with popular identity providers — Microsoft Entra ID, Okta, Google Workspace, and others — through industry-standard protocols. ## deployment-and-management - [Deployment & Management](/deployment-and-management.md) ## development - [Contributing](/development.md) ### backend In order start developing on Infrahub backend, it is recommended to have a decent knowledge about topics such as Docker, Python and generally UNIX systems. Tools such as Docker and Python virtual environment help us in isolating the development work without interfering with the system itself. In this guide, we will use: - [Backend guide](/development/backend.md): In order start developing on Infrahub backend, it is recommended to have a decent knowledge about topics such as Docker, Python and generally UNIX systems. Tools such as Docker and Python virtual environment help us in isolating the development work without interfering with the system itself. In this guide, we will use: ### changelog Infrahub utilizes a tool called towncrier for Changelog management and generation. - [Changelog guide](/development/changelog.md): Infrahub utilizes a tool called towncrier for Changelog management and generation. ### docs Welcome to the Infrahub documentation guide. This document aims to answer any questions that may come up when creating or updating documentation. - [Documentation guide](/development/docs.md): Welcome to the Infrahub documentation guide. This document aims to answer any questions that may come up when creating or updating documentation. ### editor More details coming soon - [Visual Studio Code](/development/editor.md): More details coming soon ### frontend Welcome to the Infrahub frontend guide! This guide details the technologies and steps required to contribute effectively to the Infrahub frontend. - [Frontend guide](/development/frontend.md): Welcome to the Infrahub frontend guide! This guide details the technologies and steps required to contribute effectively to the Infrahub frontend. - [Getting set up with frontend](/development/frontend/getting-set-up.md): Make sure Infrahub Backend is up and running. If not, in your terminal execute: - [Running & writing tests for frontend](/development/frontend/testing-guidelines.md): If you have never run Infrahub tests before, we highly suggest following the frontend guide. ### git-best-practices This guide explains Git workflows and best practices when contributing to Infrahub, covering submodule management, branching strategies, and pull request workflows. - [How to work with Git when developing Infrahub](/development/git-best-practices.md): This guide explains Git workflows and best practices when contributing to Infrahub, covering submodule management, branching strategies, and pull request workflows. ### style-guide This guide defines writing style, capitalization, and grammar rules for Infrahub documentation. As a general rule, prefer consistency and simplicity when possible. For anything not answered below, reference the Microsoft Style Guide. - [Documentation style guide](/development/style-guide.md): This guide defines writing style, capitalization, and grammar rules for Infrahub documentation. As a general rule, prefer consistency and simplicity when possible. For anything not answered below, reference the Microsoft Style Guide. ## development-resources - [Development Resources](/development-resources.md) ### developer-guide Infrahub support various form of extensibility that rely on users providing their own code that then will be executed by Infrahub. - [Developer guide](/development-resources/developer-guide.md): Infrahub support various form of extensibility that rely on users providing their own code that then will be executed by Infrahub. ### graphql-fragments GraphQL fragments are reusable field selections that can be shared across multiple stored queries. Instead of duplicating the same nested field selections in every query, you define them once as a fragment and reference them using the standard GraphQL spread syntax (...fragmentName). - [Using GraphQL fragments](/development-resources/graphql-fragments.md): GraphQL fragments are reusable field selections that can be shared across multiple stored queries. Instead of duplicating the same nested field selections in every query, you define them once as a fragment and reference them using the standard GraphQL spread syntax (...fragmentName). ### graphql - [Working with groups](/development-resources/graphql/groups.md): Groups are first-class objects in Infrahub that can be queried and manipulated through GraphQL. Groups provide powerful ways to organize and operate on collections of infrastructure objects. - [GraphQL](/development-resources/graphql/overview.md): The GraphQL interface is the main interface to interact with Infrahub. The GraphQL schema is automatically generated based on the core models and the user-defined schema models. - [Queries & mutations](/development-resources/graphql/queries-and-mutations.md): In GraphQL, a query is used to fetch data and mutations are used to create/update or delete data. In Infrahub, a GraphQL query and 4 mutations will be generated for each model you define in the schema. The name of the query or mutation is based on the namespace and name of the model. - [Single-target queries](/development-resources/graphql/single-target-queries.md): When writing GraphQL queries for transformations, generators, artifacts, and computed attributes, it's critical to use a single-target query pattern to ensure proper tracking by the system. - [Stored queries](/development-resources/graphql/stored-queries.md): Infrahub can store GraphQL queries in the database to simplify execution and associate them with other internal objects such as Transformations. ### local-demo-environment A local environment based on Docker Compose is available for demo and testing. - [Local demo environment](/development-resources/local-demo-environment.md): A local environment based on Docker Compose is available for demo and testing. ### sbom A software bill of materials (SBOM) is a complete inventory of the components, libraries, and dependencies contained in a piece of software. SBOMs are a core building block for supply-chain security: they let you audit what ships inside an image, match it against vulnerability databases, and satisfy compliance requirements such as those described in the US Executive Order on cybersecurity. - [Image signing and SBOM](/development-resources/sbom.md): A software bill of materials (SBOM) is a complete inventory of the components, libraries, and dependencies contained in a piece of software. SBOMs are a core building block for supply-chain security: they let you audit what ships inside an image, match it against vulnerability databases, and satisfy compliance requirements such as those described in the US Executive Order on cybersecurity. ### testcontainers The infrahub-testcontainers package starts a complete Infrahub stack in Docker during your test session. Use it to write integration tests that validate schemas, SDK usage, or GraphQL queries against a live Infrahub instance. - [Integration testing with Testcontainers](/development-resources/testcontainers.md): The infrahub-testcontainers package starts a complete Infrahub stack in Docker during your test session. Use it to write integration tests that validate schemas, SDK usage, or GraphQL queries against a live Infrahub instance. ## events ### event-actions An event action is a node you create in Infrahub that defines an outcome: add a device to a group, run a Generator definition. Actions on their own do nothing — they fire only when an event rule matches and points at them. - [Event actions](/events/event-actions.md): An event action is a node you create in Infrahub that defines an outcome: add a device to a group, run a Generator definition. Actions on their own do nothing — they fire only when an event rule matches and points at them. ### event-rules An event rule is a node you create in Infrahub that names a set of conditions and ties one or more actions to fire when those conditions match. Rules turn raw events into useful automation: instead of reacting to every infrahub.node.updated event, you describe the precise update you care about — "an InfraDevice whose status changed to active" — and Infrahub runs your action only when that pattern shows up. - [Event rules](/events/event-rules.md): An event rule is a node you create in Infrahub that names a set of conditions and ties one or more actions to fire when those conditions match. Rules turn raw events into useful automation: instead of reacting to every infrahub.node.updated event, you describe the precise update you care about — "an InfraDevice whose status changed to active" — and Infrahub runs your action only when that pattern shows up. ### event-system Infrahub emits a structured event every time a significant mutation happens in the system: a node is created, updated, or deleted; a branch is created, merged, rebased, or deleted; a group gains or loses a member; and many more. Events are the foundation of every event-driven feature — automation, webhook delivery, the activity log, and external SIEM forwarding all consume them. - [Event system](/events/event-system.md): Infrahub emits a structured event every time a significant mutation happens in the system: a node is created, updated, or deleted; a branch is created, merged, rebased, or deleted; a group gains or loses a member; and many more. Events are the foundation of every event-driven feature — automation, webhook delivery, the activity log, and external SIEM forwarding all consume them. ### overview Infrahub emits events on every significant mutation in the system — node creates, branch operations, group membership changes, and more. The event system lets you react to those events automatically: when a device is updated, when a circuit joins a provisioning group, when a branch merges. Common uses include keeping groups in sync as data changes, triggering Generators on attribute updates, and notifying external systems through webhooks. - [Events](/events/overview.md): Infrahub emits events on every significant mutation in the system — node creates, branch operations, group membership changes, and more. The event system lets you react to those events automatically: when a device is updated, when a circuit joins a provisioning group, when a branch merges. Common uses include keeping groups in sync as data changes, triggering Generators on attribute updates, and notifying external systems through webhooks. ## faq Here are the key details about Infrahub: - [Frequently asked questions](/faq.md): Here are the key details about Infrahub: ## generators ### build A Generator queries data and creates new nodes and relationships from the result. The steps below cover how to create one. - [Build a generator](/generators/build.md): A Generator queries data and creates new nodes and relationships from the result. The steps below cover how to create one. ### modular A Generator reads data from Infrahub and creates new objects based on the result. A single Generator works well when the scope is contained: one input kind, one set of outputs, no intermediate dependencies. - [Modular Generators](/generators/modular.md): A Generator reads data from Infrahub and creates new objects based on the result. A single Generator works well when the scope is contained: one input kind, one set of outputs, no intermediate dependencies. ### modular-best-practices The patterns below come from real-world experience building and operating modular Generator cascades in Infrahub. They address problems that are not obvious until you have built a multi-layer cascade and run it in production. - [Best practices for modular Generators](/generators/modular-best-practices.md): The patterns below come from real-world experience building and operating modular Generator cascades in Infrahub. They address problems that are not obvious until you have built a multi-layer cascade and run it in production. ### overview A Generator is a generic plugin that queries data and creates new nodes and relationships based on the result. - [Generators](/generators/overview.md): A Generator is a generic plugin that queries data and creates new nodes and relationships based on the result. ### remove Removing a Generator involves two separate things: the Generator definition registered from .infrahub.yml, and the objects the Generator created in Infrahub. Removing the definition stops the Generator from running, but it does not delete the objects it already created — those stay in Infrahub until you remove them. - [Remove a generator](/generators/remove.md): Removing a Generator involves two separate things: the Generator definition registered from .infrahub.yml, and the objects the Generator created in Infrahub. Removing the definition stops the Generator from running, but it does not delete the objects it already created — those stay in Infrahub until you remove them. ## git-integration ### branch-synchronization Infrahub automatically creates all branches from a connected Git repository by default. While this ensures full visibility, it can create unnecessary noise when many Git branches are unrelated to Infrahub data or workflows. Selective branch synchronization introduces an optional configuration that allows teams to control which Git branches are imported and synchronized into Infrahub based on custom naming patterns. - [Selective branch synchronization](/git-integration/branch-synchronization.md): Infrahub automatically creates all branches from a connected Git repository by default. While this ensures full visibility, it can create unnecessary noise when many Git branches are unrelated to Infrahub data or workflows. Selective branch synchronization introduces an optional configuration that allows teams to control which Git branches are imported and synchronized into Infrahub based on custom naming patterns. ### connect-repository Connecting an external Git repository will enable many features in Infrahub, such as Transformations, Generators, Checks ... that rely on the repository to store code files. - [How to connect external Git repositories](/git-integration/connect-repository.md): Connecting an external Git repository will enable many features in Infrahub, such as Transformations, Generators, Checks ... that rely on the repository to store code files. ### develop-changes Change the schemas, objects, Transformations, Generators, and other .infrahub.yml content an instance imports from a read-only repository, and land the change on the instance's main safely. This is the recommended workflow for a single instance; promoting a change across environments builds on it — see promote changes between environments. - [Develop changes from a Git repository](/git-integration/develop-changes.md): Change the schemas, objects, Transformations, Generators, and other .infrahub.yml content an instance imports from a read-only repository, and land the change on the instance's main safely. This is the recommended workflow for a single instance; promoting a change across environments builds on it — see promote changes between environments. ### infrahub-yml The .infrahub.yml file serves as the central manifest that defines how Infrahub integrates with external Git repositories. This topic explains the role of this configuration file and the design philosophy behind its structure. - [Understanding the .infrahub.yml configuration file](/git-integration/infrahub-yml.md): The .infrahub.yml file serves as the central manifest that defines how Infrahub integrates with external Git repositories. This topic explains the role of this configuration file and the design philosophy behind its structure. ### multi-environment Run a separate Infrahub instance for each environment — development, staging, production — from one Git repository. Each instance reads its schemas, objects, and automation from a read-only repository pinned to one long-lived branch. To move a change from one environment to the next, merge it into the next branch on your Git host, then trigger an import on the target instance. - [Multiple environments from a single repository](/git-integration/multi-environment.md): Run a separate Infrahub instance for each environment — development, staging, production — from one Git repository. Each instance reads its schemas, objects, and automation from a read-only repository pinned to one long-lived branch. To move a change from one environment to the next, merge it into the next branch on your Git host, then trigger an import on the target instance. ### overview Connect an external Git repository to keep the code that drives Infrahub — schemas, Transformations, Generators, checks, GraphQL queries, and artifact definitions — under version control, next to your infrastructure data. Infrahub imports that code and keeps it in sync as the repository changes. - [Git Integration](/git-integration/overview.md): Connect an external Git repository to keep the code that drives Infrahub — schemas, Transformations, Generators, checks, GraphQL queries, and artifact definitions — under version control, next to your infrastructure data. Infrahub imports that code and keeps it in sync as the repository changes. ### promote-between-environments Run a separate Infrahub instance for each environment from one Git repository, and move changes from one environment to the next. This guide covers the promotion — taking a change that is already developed on one environment and landing it on the next. To develop and land a change on a single instance, see develop changes from a Git repository; for the concepts, see multiple environments from a single repository. - [Promote changes between environments](/git-integration/promote-between-environments.md): Run a separate Infrahub instance for each environment from one Git repository, and move changes from one environment to the next. This guide covers the promotion — taking a change that is already developed on one environment and landing it on the next. To develop and land a change on a single instance, see develop changes from a Git repository; for the concepts, see multiple environments from a single repository. ## graph-traversal ### analyze-dependencies Use Dependency mode to find every object of a given kind that is reachable from a source object. - [Analyze dependencies](/graph-traversal/analyze-dependencies.md): Use Dependency mode to find every object of a given kind that is reachable from a source object. ### overview Use Graph Traversal to navigate the relationships in your infrastructure data — trace every path - [Graph Traversal](/graph-traversal/overview.md): Use Graph Traversal to navigate the relationships in your infrastructure data — trace every path ### query-with-graphql Both Graph Traversal modes are available programmatically — as GraphQL queries - [Query traversal with GraphQL](/graph-traversal/query-with-graphql.md): Both Graph Traversal modes are available programmatically — as GraphQL queries ### trace-a-path Use Path mode to find the paths between two objects in your infrastructure graph. - [Trace a path](/graph-traversal/trace-a-path.md): Use Path mode to find the paths between two objects in your infrastructure graph. ## groups ### add-members Attach existing objects to a Standard group. - [Add members to a group](/groups/add-members.md): Attach existing objects to a Standard group. ### create Create a Standard group when you want to hand-pick the objects it contains. - [Create a group](/groups/create.md): Create a Standard group when you want to hand-pick the objects it contains. ### delete Remove a group without affecting its member objects. - [Delete a group](/groups/delete.md): Remove a group without affecting its member objects. ### overview Groups are containers that establish relationships between objects in your infrastructure. They let you create logical collections that span different object types — you might group together devices, locations, and configurations that share a common purpose or characteristic. - [Groups](/groups/overview.md): Groups are containers that establish relationships between objects in your infrastructure. They let you create logical collections that span different object types — you might group together devices, locations, and configurations that share a common purpose or characteristic. ### query-members Read what's in a group, or find what groups an object belongs to. Use these patterns for auditing, verification, or feeding automation. - [Query group membership](/groups/query-members.md): Read what's in a group, or find what groups an object belongs to. Use these patterns for auditing, verification, or feeding automation. ### remove-members Detach objects from a Standard group without deleting the objects themselves. - [Remove members from a group](/groups/remove-members.md): Detach objects from a Standard group without deleting the objects themselves. ### use-in-automation Target a group from an artifact definition, Transformation, or Check so that the automation applies to every member without hard-coding a list. - [Use groups in automation](/groups/use-in-automation.md): Target a group from an artifact definition, Transformation, or Check so that the automation applies to every member without hard-coding a list. ## immutable-history ### overview Infrahub preserves previous values and relationships as your infrastructure data changes. Earlier versions remain available for queries, so you can inspect what existed at a specific time, compare changes across a period, and trace how an object or relationship changed. - [Immutable history](/immutable-history/overview.md): Infrahub preserves previous values and relationships as your infrastructure data changes. Earlier versions remain available for queries, so you can inspect what existed at a specific time, compare changes across a period, and trace how an object or relationship changed. ### query-historical-data You can read infrastructure data as it existed at previous points in time, and compare how it changed between them. A timestamped query answers what current data cannot: which devices and interfaces existed for a site during last night's incident, which interface attributes changed between the last known-good time and now, or which prefixes were assigned to a site on a given date. - [Query historical data](/immutable-history/query-historical-data.md): You can read infrastructure data as it existed at previous points in time, and compare how it changed between them. A timestamped query answers what current data cannot: which devices and interfaces existed for a site during last night's incident, which interface attributes changed between the last known-good time and now, or which prefixes were assigned to a site on a given date. ## integrations-overview - [Integrations](/integrations-overview.md) ## ipam ### allocate-ips-and-prefixes When to allocate from a pool - [Allocate IPs and prefixes](/ipam/allocate-ips-and-prefixes.md): When to allocate from a pool ### build-your-ipam-schema Infrahub includes the IpamNamespace node and the BuiltinIPNamespace, BuiltinIPPrefix, and BuiltinIPAddress generics by default — but no concrete IP prefix or address node until you load one. - [Build your IPAM schema](/ipam/build-your-ipam-schema.md): Infrahub includes the IpamNamespace node and the BuiltinIPNamespace, BuiltinIPPrefix, and BuiltinIPAddress generics by default — but no concrete IP prefix or address node until you load one. ### overview Use Infrahub IPAM to manage IP prefixes, IP addresses, and namespaces alongside the rest of your infrastructure data. Prefixes and addresses are ordinary Infrahub objects, so you can add relationships to devices, interfaces, and services the same way you would on any other node. Infrahub computes hierarchy automatically and calculates utilization on read. Every change is branch-aware. Both IPv4 and IPv6 are supported. - [IP Address Management](/ipam/overview.md): Use Infrahub IPAM to manage IP prefixes, IP addresses, and namespaces alongside the rest of your infrastructure data. Prefixes and addresses are ordinary Infrahub objects, so you can add relationships to devices, interfaces, and services the same way you would on any other node. Infrahub computes hierarchy automatically and calculates utilization on read. Every change is branch-aware. Both IPv4 and IPv6 are supported. ### plan-changes-on-a-branch Use a branch to draft IP prefix and address changes, review exactly what will change, and merge once you're confident — the same version-control workflow Infrahub applies to the rest of your infrastructure data. - [Plan changes on a branch](/ipam/plan-changes-on-a-branch.md): Use a branch to draft IP prefix and address changes, review exactly what will change, and merge once you're confident — the same version-control workflow Infrahub applies to the rest of your infrastructure data. ### query-ipam-data Read the computed values on IP prefixes and addresses, or follow the hierarchy relationships. Use these patterns for reporting, automation, or verification. - [Query IPAM data](/ipam/query-ipam-data.md): Read the computed values on IP prefixes and addresses, or follow the hierarchy relationships. Use these patterns for reporting, automation, or verification. ### use-namespaces Use IP namespaces to isolate sets of IP prefixes and addresses within the same Infrahub instance — for example, to give each customer or business unit its own IP space, even when their ranges overlap. - [Use namespaces](/ipam/use-namespaces.md): Use IP namespaces to isolate sets of IP prefixes and addresses within the same Infrahub instance — for example, to give each customer or business unit its own IP space, even when their ranges overlap. ## learn ### tutorials - [Build a check in Infrahub](/learn/tutorials/build-a-check.md): By the end of this tutorial you will have built, deployed, and validated a custom check that enforces a naming convention on tags. You will set up sample data, write a GraphQL query, implement the check logic in Python, configure it in .infrahub.yml, deploy it via a connected Git repository, and verify it runs against a proposed change. - [Build your first schema](/learn/tutorials/build-your-first-schema.md): By the end of this tutorial you will have a working schema for network devices and interfaces, covering nodes, attributes, relationships, and generic abstractions. You'll load each version into a branch and verify the results. - [Build chained generators](/learn/tutorials/generators/build-chained-generators.md): By the end of this tutorial you will have wired two layers of modular Generators together — fabric → pod → rack — using a checksum attribute and Infrahub's event framework. You will have added a GeneratorTarget generic with a checksum attribute to your downstream node kinds, implemented a GeneratorMixin that writes the checksum to downstream targets as the last step of generate(), built downstream Generators that validate upstream completeness before doing any work, and created CoreGeneratorAction and CoreNodeTriggerRule objects so the chain runs automatically when an upstream layer finishes. You will leave with a concrete model of how a checksum makes Generator chaining idempotent — re-runs that produce the same output do not re-trigger the next layer. - [Build your first generator](/learn/tutorials/generators/build-your-first-generator.md): By the end of this tutorial you will have built a working Generator end-to-end: modeled two object kinds, written a GraphQL query, implemented a Python Generator class, registered it in .infrahub.yml, run it locally with infrahubctl, and verified it runs automatically as part of a proposed change. You will leave with a Widget and Resource schema loaded into Infrahub and a Generator that creates Resource objects from each Widget's count attribute. - [Organize objects with groups](/learn/tutorials/groups.md): By the end of this tutorial you will have created a group, added two objects to it, and queried the result end-to-end. You'll leave with a concrete mental model of how groups work and where to go next. - [Tutorials](/learn/tutorials/overview.md): The most direct way to learn a specific Infrahub workflow is to run a tutorial. Each one walks you through a real task end to end — modeling a schema, writing a check, building a Generator — against your own Infrahub instance, using the same GraphQL, Python, and schema snippets you'd write in production. You follow the steps directly in these docs, at your own pace, with no external platform involved. - [Build a Jinja2 Transformation](/learn/tutorials/transformations/build-a-jinja2-transformation.md): By the end of this tutorial you will have built a working Jinja2 Transformation end-to-end: loaded a small network-device schema, created a few sample devices, written a GraphQL query that filters by device name, written a Jinja template that renders a configuration snippet from the result, registered it in .infrahub.yml, tested it locally with infrahubctl render, added the repository to Infrahub, and called the render API. You'll leave with a deviceconfigtransform you can call against any device by name. - [Build a Python Transformation](/learn/tutorials/transformations/build-a-python-transformation.md): By the end of this tutorial you will have built a working Python Transformation end-to-end: loaded a small network-device schema, created a few sample devices, written a GraphQL query that filters by device name, implemented a DeviceConfigTransform Python class that returns a JSON object derived from the response, registered it in .infrahub.yml, tested it locally with infrahubctl transform, and called the Transformation via the REST API. You'll leave with both the raw-dictionary and SDK-converted patterns side by side so you can pick the one that fits the next Transformation you write. ## menu ### overview Infrahub lets you control the menu on the left side of the web interface. - [Menu customization](/menu/overview.md): Infrahub lets you control the menu on the left side of the web interface. ## object-templates ### allocate-resources-from-pools Object templates support automatic resource allocation. This guide extends the device and interface example from Use object templates to show how to wire a CoreIPAddressPool to the interface template so that IP addresses are allocated automatically on object creation. - [Allocate resources from pools](/object-templates/allocate-resources-from-pools.md): Object templates support automatic resource allocation. This guide extends the device and interface example from Use object templates to show how to wire a CoreIPAddressPool to the interface template so that IP addresses are allocated automatically on object creation. ### overview In many infrastructures, multiple instances of objects share a common structure. Consider network devices: we know in advance the port setup for a given model. When documenting this in our source of truth, repeatedly entering the same port details is both inefficient and error-prone. The Object Template feature allows you to create a reusable blueprint for any object. This blueprint can be used to generate multiple instances that adhere to the predefined structure, ensuring uniformity while reducing manual effort. - [Object Templates](/object-templates/overview.md): In many infrastructures, multiple instances of objects share a common structure. Consider network devices: we know in advance the port setup for a given model. When documenting this in our source of truth, repeatedly entering the same port details is both inefficient and error-prone. The Object Template feature allows you to create a reusable blueprint for any object. This blueprint can be used to generate multiple instances that adhere to the predefined structure, ensuring uniformity while reducing manual effort. ### use This guide provides a structured approach to defining an object template and creating object instances based on that template. - [Use object templates](/object-templates/use.md): This guide provides a structured approach to defining an object template and creating object instances based on that template. ### with-profiles When both generatetemplate and generateprofile are configured on a schema node, you can assign Profiles to templates to enable bulk configuration updates. Objects created from templates automatically inherit the Profiles assigned to those templates, allowing you to update values in bulk by modifying the Profile. - [Assign Profiles to a template](/object-templates/with-profiles.md): When both generatetemplate and generateprofile are configured on a schema node, you can assign Profiles to templates to enable bulk configuration updates. Objects created from templates automatically inherit the Profiles assigned to those templates, allowing you to update values in bulk by modifying the Profile. ## objects ### convert-object-kind Object conversion in Infrahub provides a powerful mechanism to transform existing objects from one schema type to another without losing data or breaking relationships. This capability addresses the common infrastructure management challenge of evolving data models while preserving existing configurations and connections. - [Object conversion](/objects/convert-object-kind.md): Object conversion in Infrahub provides a powerful mechanism to transform existing objects from one schema type to another without losing data or breaking relationships. This capability addresses the common infrastructure management challenge of evolving data models while preserving existing configurations and connections. ### create-objects There are three ways to add objects to Infrahub. Which one you need depends on how many objects you are creating and where their definition comes from: - [Create objects](/objects/create-objects.md): There are three ways to add objects to Infrahub. Which one you need depends on how many objects you are creating and where their definition comes from: ### import-from-csv Prompt an AI coding assistant to translate a CSV or TSV export into Infrahub object YAML, then load it the same way as any other object file. - [Import data from CSV](/objects/import-from-csv.md): Prompt an AI coding assistant to translate a CSV or TSV export into Infrahub object YAML, then load it the same way as any other object file. ### load-from-yaml An Object file is a YAML file that allows you to manage data to be loaded in Infrahub based on your own custom schema. It provides a declarative way to define and manage resources in your Infrahub instance. - [Object files](/objects/load-from-yaml.md): An Object file is a YAML file that allows you to manage data to be loaded in Infrahub based on your own custom schema. It provides a declarative way to define and manage resources in your Infrahub instance. ### manage-from-cli Use infrahubctl to query, create, update, and delete objects directly from your terminal. The commands accept any schema kind in your instance and can display results as a table, JSON, CSV, or YAML. - [Manage objects with infrahubctl](/objects/manage-from-cli.md): Use infrahubctl to query, create, update, and delete objects directly from your terminal. The commands accept any schema kind in your instance and can display results as a table, JSON, CSV, or YAML. ### metadata Data lineage - [Data lineage and metadata](/objects/metadata.md): Data lineage ### overview An object is a record stored in Infrahub that conforms to a schema node definition. If schema nodes are the blueprints, objects are the actual instances of data — a specific router, an interface, a BGP session, a rack. - [Objects](/objects/overview.md): An object is a record stored in Infrahub that conforms to a schema node definition. If schema nodes are the blueprints, objects are the actual instances of data — a specific router, an interface, a BGP session, a rack. ## overview A graph-based infrastructure data management platform with built-in version control, CI workflows, peer review, and API access. - [What is Infrahub](/overview.md): A graph-based infrastructure data management platform with built-in version control, CI workflows, peer review, and API access. ### architecture Infrahub's architecture combines graph database technology with Git-like version control to create a powerful platform for infrastructure management. This topic explains the core components, design principles, and how they work together to deliver a unified infrastructure management solution. - [Understanding Infrahub's architecture](/overview/architecture.md): Infrahub's architecture combines graph database technology with Git-like version control to create a powerful platform for infrastructure management. This topic explains the core components, design principles, and how they work together to deliver a unified infrastructure management solution. ### build-with-ai Describe the network you want to model and have your assistant build the Infrahub schema, automation, and queries that represent it. - [Build with AI](/overview/build-with-ai.md): Describe the network you want to model and have your assistant build the Infrahub schema, automation, and queries that represent it. - [Set up your AI assistant](/overview/build-with-ai/setup.md): Install the Infrahub Skills, connect the MCP server, and choose the combination that fits how you work. - [Work with your assistant](/overview/build-with-ai/working-with-ai.md): Decide when to describe a change directly and when to plan it first, and what to check in what your assistant produces. ### community-vs-enterprise Infrahub is available in two distinct editions designed to meet different organizational needs: Community Edition and Enterprise Edition. This topic explains the differences between these editions to help you choose the right version for your infrastructure management needs. - [Understanding Infrahub community vs enterprise](/overview/community-vs-enterprise.md): Infrahub is available in two distinct editions designed to meet different organizational needs: Community Edition and Enterprise Edition. This topic explains the differences between these editions to help you choose the right version for your infrastructure management needs. ### concepts Core principles behind Infrahub — flexible schemas, version control, Transformations, Generators, and design-driven automation. - [Key Concepts](/overview/concepts.md): Core principles behind Infrahub — flexible schemas, version control, Transformations, Generators, and design-driven automation. ### explore See Infrahub in action through the sandbox, interactive labs, videos, and demos. - [Explore Infrahub](/overview/explore.md): See Infrahub in action through the sandbox, interactive labs, videos, and demos. ### next-steps Turn your local Infrahub setup into a proof of concept for your organization. - [Next Steps](/overview/next-steps.md): Turn your local Infrahub setup into a proof of concept for your organization. ### quickstart Set up a local Infrahub instance, load a data model, create infrastructure data, and explore branching. - [Quick Start](/overview/quickstart.md): Set up a local Infrahub instance, load a data model, create infrastructure data, and explore branching. ## profiles ### assign When creating or updating an object, reference a Profile by HFID. The object inherits the Profile's values for any attributes you don't set explicitly. - [Assign a Profile to an object](/profiles/assign.md): When creating or updating an object, reference a Profile by HFID. The object inherits the Profile's values for any attributes you don't set explicitly. ### create Define a reusable set of attribute values for a Profile-enabled node type. - [Create a Profile](/profiles/create.md): Define a reusable set of attribute values for a Profile-enabled node type. ### override-values When a Profile-assigned object should differ for a specific field, set the value explicitly when creating or updating the object. Explicit values take precedence over Profile values; other attributes still inherit from the Profile. - [Override specific Profile values](/profiles/override-values.md): When a Profile-assigned object should differ for a specific field, set the value explicitly when creating or updating the object. Explicit values take precedence over Profile values; other attributes still inherit from the Profile. ### overview This topic explains the concept of Profiles in Infrahub, their purpose in the system architecture, and how they enable configuration consistency across your infrastructure. You'll gain a deeper understanding of Profile inheritance, priority mechanisms, and the design decisions behind this feature. - [Profiles](/profiles/overview.md): This topic explains the concept of Profiles in Infrahub, their purpose in the system architecture, and how they enable configuration consistency across your infrastructure. You'll gain a deeper understanding of Profile inheritance, priority mechanisms, and the design decisions behind this feature. ### priority-and-inheritance When an object has one or more Profiles assigned, Infrahub uses a deterministic process to decide which value applies to each attribute or relationship. This page explains both the inheritance flow and the priority system that resolves conflicts when multiple Profiles apply. - [Priority and inheritance](/profiles/priority-and-inheritance.md): When an object has one or more Profiles assigned, Infrahub uses a deterministic process to decide which value applies to each attribute or relationship. This page explains both the inheritance flow and the priority system that resolves conflicts when multiple Profiles apply. ### update Change Profile values; all objects assigned to the Profile inherit the change automatically. Objects that have explicitly overridden the attribute keep their override — only objects relying on the Profile value see the change. - [Update a Profile](/profiles/update.md): Change Profile values; all objects assigned to the Profile inherit the change automatically. Objects that have explicitly overridden the attribute keep their override — only objects relying on the Profile value see the change. ### use-multiple A single object can be assigned multiple Profiles. When multiple Profiles define the same attribute, the profile_priority value (lower number = higher priority) determines which Profile's value wins. - [Use multiple Profiles](/profiles/use-multiple.md): A single object can be assigned multiple Profiles. When multiple Profiles define the same attribute, the profile_priority value (lower number = higher priority) determines which Profile's value wins. ## proposed-changes ### lifecycle Proposed changes follow a workflow with specific states that track progression from initial creation to final resolution. - [Lifecycle and state transitions](/proposed-changes/lifecycle.md): Proposed changes follow a workflow with specific states that track progression from initial creation to final resolution. ### overview A proposed change in Infrahub is a structured workflow mechanism that enables teams to review, discuss, and merge changes in a controlled and collaborative manner. It serves as the primary method for implementing infrastructure changes safely while maintaining proper oversight and governance. - [Proposed Changes](/proposed-changes/overview.md): A proposed change in Infrahub is a structured workflow mechanism that enables teams to review, discuss, and merge changes in a controlled and collaborative manner. It serves as the primary method for implementing infrastructure changes safely while maintaining proper oversight and governance. ### resolve-conflict When a proposed change has data conflicts between its source and target branches, Infrahub blocks the merge to protect the integrity of your infrastructure data. Conflicts are resolved during the review process, before the change can be approved and merged. - [Resolve a proposed-change conflict](/proposed-changes/resolve-conflict.md): When a proposed change has data conflicts between its source and target branches, Infrahub blocks the merge to protect the integrity of your infrastructure data. Conflicts are resolved during the review process, before the change can be approved and merged. ### review-and-stamp The review process gives team members structured ways to evaluate a proposed change, comment on specific elements, and ultimately stamp it as approved or rejected. - [Review and stamp](/proposed-changes/review-and-stamp.md): The review process gives team members structured ways to evaluate a proposed change, comment on specific elements, and ultimately stamp it as approved or rejected. ## reference - [Reference](/reference.md) ### configuration Configuration options for Infrahub - [Infrahub configuration](/reference/configuration.md): Configuration options for Infrahub ### dotinfrahub The repository configuration file allows you to define multiple resources that need to be imported into Infrahub. - [Repository configuration file](/reference/dotinfrahub.md): The repository configuration file allows you to define multiple resources that need to be imported into Infrahub. ### error-catalogue Catalogue version: 1 — 14 codes - [Error Catalogue](/reference/error-catalogue.md): Catalogue version: 1 — 14 codes ### graph-traversal Reference for the InfrahubPathTraversal and InfrahubReachableNodes GraphQL queries. - [Graph Traversal reference](/reference/graph-traversal.md): Reference for the InfrahubPathTraversal and InfrahubReachableNodes GraphQL queries. ### infrahub-cli - [CLI](/reference/infrahub-cli.md) - [infrahub db](/reference/infrahub-cli/infrahub-db.md): Manage the graph in the database. - [infrahub dev](/reference/infrahub-cli/infrahub-dev.md): Usage: - [infrahub recover](/reference/infrahub-cli/infrahub-recover.md): Recover from failed operations. - [infrahub server](/reference/infrahub-cli/infrahub-server.md): Control the API Server. - [infrahub upgrade](/reference/infrahub-cli/infrahub-upgrade.md): Upgrade Infrahub to the latest version. ### infrahub-events - [Account events](/reference/infrahub-events/account.md): Account Logged In Event - [Artifact events](/reference/infrahub-events/artifact.md): Artifact Created Event - [Branch events](/reference/infrahub-events/branch.md): Branch Created Event - [Commit events](/reference/infrahub-events/commit.md): Commit Updated Event - [Group events](/reference/infrahub-events/group.md): Group Auto Create Capped Event - [Node events](/reference/infrahub-events/node.md): Node Created Event - [Event reference](/reference/infrahub-events/overview.md): Complete reference for all events emitted by Infrahub. Each page lists every event in that category with its type identifier, description, and payload fields. - [Proposed events](/reference/infrahub-events/proposed.md): Proposed Change Approval Revoked Event - [Schema events](/reference/infrahub-events/schema.md): Schema Updated Event - [Validator events](/reference/infrahub-events/validator.md): Validator Failed Event ### infrahub-tests The tests configuration file allows you to define multiple tests for Infrahub resources such as Jinja2 Transformations, Python Transformations, checks and GraphQL queries. - [Tests configuration file](/reference/infrahub-tests.md): The tests configuration file allows you to define multiple tests for Infrahub resources such as Jinja2 Transformations, Python Transformations, checks and GraphQL queries. ### ldap This reference document describes the configuration options for LDAP authentication in Infrahub. - [LDAP](/reference/ldap.md): This reference document describes the configuration options for LDAP authentication in Infrahub. ### menu A menu definition file allows you to control the layout and structure of the menu on the left side of the Infrahub web interface. More information can be found in the Controlling the menu guide. - [Menu definition file](/reference/menu.md): A menu definition file allows you to control the layout and structure of the menu on the left side of the Infrahub web interface. More information can be found in the Controlling the menu guide. ### message-bus-events This document provides detailed documentation for all events used in the Infrahub message bus system. - [Message bus events](/reference/message-bus-events.md): This document provides detailed documentation for all events used in the Infrahub message bus system. ### permissions Reference documentation for Infrahub's global and object permissions - [Permissions](/reference/permissions.md): Reference documentation for Infrahub's global and object permissions ### schema - [Schema Specification](/reference/schema.md) - [Attribute](/reference/schema/attribute.md): In a schema file, an attribute can be defined inside a node, a generic or a node extension. - [Generic](/reference/schema/generic.md): All options to define a Generic in the schema - [Groups](/reference/schema/groups.md): Groups schema reference - [Node](/reference/schema/node.md): Node schema reference - [Node Extension](/reference/schema/node-extension.md): All options to define a node extension in the schema - [Relationship](/reference/schema/relationship.md): In a schema file, a relationship can be defined inside a node, a generic or a node extension. - [Schema Update](/reference/schema/validator-migration.md): In this context, an element represent either a Node, a Generic, an Attribute or a Relationship ### schema-validation Infrahub requires the user to define multiple YAML files. One or more files defining the schema in Infrahub and the external repository configuration file. - [Schema validation](/reference/schema-validation.md): Infrahub requires the user to define multiple YAML files. One or more files defining the schema in Infrahub and the external repository configuration file. ### sso This reference document describes the available SSO protocols, configuration options, and parameters in Infrahub. - [SSO (Single sign-on)](/reference/sso.md): This reference document describes the available SSO protocols, configuration options, and parameters in Infrahub. ## release-notes - [Release Notes](/release-notes.md) ### deprecation-guides - [Deprecation Guides](/release-notes/deprecation-guides.md) - [How to migrate from display_labels to display_label](/release-notes/deprecation-guides/display_labels.md): With Infrahub version 1.5, the displaylabels configuration was deprecated in favor of a Jinja2-based displaylabel configuration that provides more flexibility and consistency. - [Schema load validation and the write contract](/release-notes/deprecation-guides/schema-load-write-contract.md): Infrahub now validates every schema submitted to POST /api/schema/load against a - [How to migrate off the SSO account-name fallback](/release-notes/deprecation-guides/sso-account-name-fallback.md): Infrahub 1.8.7 and 1.9.7 introduce a feature flag, ssoaccountname_fallback, that ### infrahub Chronological feed of all Infrahub releases, with full detail for the newest release lines. - [Infrahub release notes](/release-notes/infrahub.md): Chronological feed of all Infrahub releases, with full detail for the newest release lines. - [Documentation restructure](/release-notes/infrahub/docs-restructure.md): Why we reorganized - [Release 0.10.0 - Alpha #4](/release-notes/infrahub/release-0_10.md): Hierarchical mode and the new pytest-based test framework. - [Release 0.11.0 - Alpha #5](/release-notes/infrahub/release-0_11.md): Check testing, read-only repositories, object search, and database backup/restore. - [Release 0.12.0 - Beta #1](/release-notes/infrahub/release-0_12.md): Unified storage improvements, CI pipeline, and schema updates. - [Release 0.13.0](/release-notes/infrahub/release-0_13.md): Built-in IPAM: IPv4/IPv6 prefixes and addresses with automatic prefix trees. - [Release 0.14.0](/release-notes/infrahub/release-0_14.md): Start of the open beta: documentation overhaul and the new Resource Manager. - [Release 0.15.0](/release-notes/infrahub/release-0_15_0.md): User-experience enhancements and groundwork for future features. - [Release 0.15.1](/release-notes/infrahub/release-0_15_1.md): Bug-fix release. - [Release 0.15.2](/release-notes/infrahub/release-0_15_2.md): Bug-fix release. - [Release 0.15.3](/release-notes/infrahub/release-0_15_3.md): Add usage of Towncrier to generate Changelog as part of the release process. Serve Swagger & Redoc files locally so that the REST-API docs work offline or when isolated from the internet. - [Release 0.16](/release-notes/infrahub/release-0_16_0.md): Number resource pools and object-management improvements for additional use cases. - [Release 0.16.1](/release-notes/infrahub/release-0_16_1.md): - In list views, always show relationships of type 'Parent.' In the details view of an object, hide the 'Parent' relationship if the parent is the current object itself. - [Release 0.16.2](/release-notes/infrahub/release-0_16_2.md): Loading a schema with an invalid order by field raises a proper error. Updates internal logic to improve performance when generating a diff. - [Release 0.16.3](/release-notes/infrahub/release-0_16_3.md): Save a diff in smaller pieces instead of all at once to prevent out-of-memory error. Fixes exception handling section in the Python SDK batch guide. - [Release 0.16.4](/release-notes/infrahub/release-0_16_4.md): Fixed an issue on the UI where a new relationship was being added to the main branch instead of the current branch. - [Release 0.6.0](/release-notes/infrahub/release-0_6.md): New logo and colors, user management and authentication, and the first version of Groups. - [Release 0.7.0](/release-notes/infrahub/release-0_7.md): Early beta release. - [Release 0.8.0 - Alpha #2](/release-notes/infrahub/release-0_8.md): Early beta release. - [Release 0.9.0 - Alpha #3](/release-notes/infrahub/release-0_9.md): Early beta release. - [Release 1.0](/release-notes/infrahub/release-1_0_0.md): General availability: improved diffing, rebasing, and branch merges for a production-ready source of truth. - [Release 1.0.1](/release-notes/infrahub/release-1_0_1.md): When a user is not logged in and the branch name is not found, hide the quick-create action and display the message: 'No branch found' (4801) Fix automation to trigger generation of artifacts after merging a branch (4804) - [Release 1.0.10](/release-notes/infrahub/release-1_0_10.md): Make URL fields clickable in the details view (5005) - [Release 1.0.2](/release-notes/infrahub/release-1_0_2.md): Update branch merge logic to use smaller queries outside of a transaction to allow merging a branch with many changes (4448) Ensure the GraphQL query InfrahubResourcePoolUtilization works properly when the schema is different in the branch (4761) - [Release 1.0.3](/release-notes/infrahub/release-1_0_3.md): Fix hierarchical schema update logic to correctly update peer on parent relationship of new node (4838) Fix hierarchical schema update logic to correctly update peer on parent of new child node (4839) - [Release 1.0.4](/release-notes/infrahub/release-1_0_4.md): Profiles now have a Human-Friendly Identifier (HFID) defined based on their Profile name (4758) Workers out of sync after deleting node from schema (4836) - [Release 1.0.5](/release-notes/infrahub/release-1_0_5.md): Node attribute name can now be type (4381) - [Release 1.0.6](/release-notes/infrahub/release-1_0_6.md): Forbid changing the 'optional' property of an inherited attribute to not break GraphQL schema generation (4936) The permission to edit the default branch is now enforced properly when loading a schema (4958) - [Release 1.0.7](/release-notes/infrahub/release-1_0_7.md): Bug-fix release. - [Release 1.0.8](/release-notes/infrahub/release-1_0_8.md): Add a security setting for the default SSO user group, giving the name of a group to which SSO users are assigned if the identity provider does not give a list of groups to use (4924) Added a configuration setting for Git repositories that appends '.git' to repositories defined with an HTTP URL for the configured domains (5077) - [Release 1.0.9](/release-notes/infrahub/release-1_0_9.md): Adding invoke tasks to assist with release process. Add pagination and backend search in new combobox for relationships. - [Release 1.1.0](/release-notes/infrahub/release-1_1_0.md): Computed attributes — a highly requested feature — plus many behind-the-scenes improvements. - [Release 1.1.1](/release-notes/infrahub/release-1_1_1.md): Raise a better error when trying to resolve an invalid HFID for a relationship (5360) Fix an issue with session management that could lead to the crash of the GraphQL resolver - [Release 1.1.2](/release-notes/infrahub/release-1_1_2.md): Added a configuration option for INFRAHUB_PUBLIC_URL, which could be required for SSO depending on how Infrahub is published and accessed within your organization. - [Release 1.1.3](/release-notes/infrahub/release-1_1_3.md): Add a new link in the object details button to redirect to the tasks list with a filter for the current object - [Release 1.1.4](/release-notes/infrahub/release-1_1_4.md): Artifacts can now be of type: YAML, XML, markdown, SVG and CSV. - [Release 1.1.5](/release-notes/infrahub/release-1_1_5.md): Allow OIDC providers to fall back to the ID token for group membership reports if they are not provided within the userinfo URL. - [Release 1.1.6](/release-notes/infrahub/release-1_1_6.md): Allow Default Address Type quick selection in the Resource Manager form (3489) Added code viewer for new content-types, preview of raw markdown content, one-click file download or copy, and redesign of artifact details view (5452) - [Release 1.1.7](/release-notes/infrahub/release-1_1_7.md): Data diffs are loaded in sequential batches for faster performance with large changes. The diff tree and diff list can now be scrolled independently. - [Release 1.1.8](/release-notes/infrahub/release-1_1_8.md): Fixed an error in the query to count the number of peers for a given cardinality-many relationship. Fixed the HFID format in the mutations IPAddressPoolGetResource and IPPrefixPoolGetResource. - [Release 1.1.9](/release-notes/infrahub/release-1_1_9.md): Improved the performance of the GraphQL cardinality-many relationship resolver by batching database calls together. - [Release 1.10.0](/release-notes/infrahub/release-1_10_0.md): Graph path traversal with a visual topology explorer, native LDAP authentication for Enterprise, auto-created account groups from identity-provider claims, database-level branch merges, and a structured GraphQL error catalogue. - [Release 1.10.1](/release-notes/infrahub/release-1_10_1.md): Adds a shortest-paths toggle to path traversal; restores webhook delivery and fixes default-branch Git sync, merge backfills, and unique-constraint validation on merge. - [Release 1.10.2](/release-notes/infrahub/release-1_10_2.md): Fixes mutation events being dropped for nodes with ~250+ relationship peers, and errors from display labels reading a deleted peer. - [Release 1.10.3](/release-notes/infrahub/release-1_10_3.md): Initialization locks now auto-expire so a dead worker can't block startup; fixes IPv6 search normalization, repository connectivity status, and branch-merge rollback. - [Release 1.10.4](/release-notes/infrahub/release-1_10_4.md): Adds Neo4j cluster addresses via INFRAHUB_DB_ADDRESS; fixes Git repository re-pointing, cascade-delete validation, and missing webhook account IDs. - [Release 1.10.5](/release-notes/infrahub/release-1_10_5.md): Fixes a diff-recalculation race after rebase, schema-diff failures when merging branches that removed schema nodes, and schema migrations being counted twice on rebased branches. - [Release 1.10.6](/release-notes/infrahub/release-1_10_6.md): Speeds up node creation by avoiding a redundant peer query for empty relationships, and fixes repository cascade deletion, IPAM namespace link navigation, git-sync import, and diff-tree rendering. - [Release 1.10.7](/release-notes/infrahub/release-1_10_7.md): Speeds up GraphQL queries that only request a relationship peer's id, and fixes artifact generation, number-pool computed attributes, non-main default branches, group mutation events, git-sync, and several object form issues. - [Release 1.10.8](/release-notes/infrahub/release-1_10_8.md): Fixes a crash in the proposed change Data diff tab on large diffs, number pool allocation stalling on pre-existing values, mutation events being refused by Prefect, and migrations failing on transient database errors. - [Release 1.10.9](/release-notes/infrahub/release-1_10_9.md): Fixes a crash on the Tasks page when a task references a related node whose kind can no longer be resolved, and an attribute rename on a branch leaving the old attribute open, which could duplicate paths to the renamed attribute after a rebase. - [Release 1.11.0](/release-notes/infrahub/release-1_11_0.md): Scale and performance: a merge regenerates only the artifacts and Generators it actually affected, computed attributes recompute only when their inputs change, and schema constraint validation is scoped to the kinds and fields a change can violate. Webhook deliveries become tasks you can inspect, retry and cancel, an interrupted merge is detectable and recoverable, and task priority lanes keep user-facing operations ahead of background work. - [Release 1.11.1](/release-notes/infrahub/release-1_11_1.md): Makes object creates and updates faster, most notably when creating objects from an object template, and stops API server memory growing without bound by evicting superseded GraphQL schemas. Also fixes read queries being pushed onto the async path, which made schema introspection take over ten seconds, and repairs nodes left without attribute rows when an existing kind started inheriting from a generic. - [Release 1.11.2](/release-notes/infrahub/release-1_11_2.md): Fixes account preferences to preview the date and timezone values that saving will produce, cleans up an SSO account's identities and tokens when it is deleted, purges a deleted branch's settled tasks, and keeps the branch registry cache in sync with the database. Artifacts and Generators are now regenerated when a display label or human-friendly ID changes. - [Release 1.2.0](/release-notes/infrahub/release-1_2_0.md): Highly anticipated new features alongside bug fixes and performance improvements. - [Release 1.2.1](/release-notes/infrahub/release-1_2_1.md): Added relationships changes details in the activities. Added an INFRAHUB_SCHEMA_STRICT_MODE environment variable. - [Release 1.2.10](/release-notes/infrahub/release-1_2_10.md): Added the ability to use alternative value types for all attribute types with computed attributes. Computed Attribute of kind Jinja will only be recalculated during a schema update if the template itself has been updated. - [Release 1.2.11](/release-notes/infrahub/release-1_2_11.md): Add the CoreWeightedPoolResource generic to better control which resource should be used when allocating from a pool. - [Release 1.2.12](/release-notes/infrahub/release-1_2_12.md): Remove uniqueness constraint on generic templates to support upsert mutations (6478) Add a migration to clean up duplicated data from improper merges of branches containing node schemas with an updated kind or inheritance (6502) - [Release 1.2.2](/release-notes/infrahub/release-1_2_2.md): Fixed generic schema updates to correctly propagate an updated order weight to a downstream attribute or relationship on an inheriting schema. - [Release 1.2.3](/release-notes/infrahub/release-1_2_3.md): Added support for Jinja2 filters from Netutils. - [Release 1.2.4](/release-notes/infrahub/release-1_2_4.md): Fixed the migration to remove an attribute from a schema to correctly ignore overridden attributes from a generic schema. - [Release 1.2.5](/release-notes/infrahub/release-1_2_5.md): Added support for computed attributes on generics. Added a new database selected-export command to run an anonymized export of selected objects that includes no actual data. - [Release 1.2.6](/release-notes/infrahub/release-1_2_6.md): Added generics to node selection in number pool form. Enabled node select in the webhook form to quickly choose the node kind. - [Release 1.2.7](/release-notes/infrahub/release-1_2_7.md): Security release: Update the h11 package to 0.16.0. - [Release 1.2.8](/release-notes/infrahub/release-1_2_8.md): Added support for automatic query response conversion in Python Transformations. Enabled HCL syntax highlighting for artifacts. - [Release 1.2.9](/release-notes/infrahub/release-1_2_9.md): Added the INFRAHUB_TESTING_SCHEMA_STRICT_MODE environment variable to allow users to control INFRAHUB_SCHEMA_STRICT_MODE when using infrahub-testcontainers. - [Release 1.3.0](/release-notes/infrahub/release-1_3_0.md): A new Action System that automates routine tasks from system events — group triggers and Generator runs. - [Release 1.3.1](/release-notes/infrahub/release-1_3_1.md): Fix bug that could prevent renaming a unique attribute on a schema (6147) Fix a bug where Number attribute min_value/max_value/excluded_values constraints were not enforced during node creation (6714) - [Release 1.3.2](/release-notes/infrahub/release-1_3_2.md): Improve performance of uniqueness constraint checks during create/update/upsert mutations by allowing ordering elements from more specific to less specific within a constraint group (6377) Fixed: min/max constraints no longer trigger on empty values when the field is optional. - [Release 1.3.3](/release-notes/infrahub/release-1_3_3.md): Add a command to run a single migration - [Release 1.3.5](/release-notes/infrahub/release-1_3_5.md): Add a new check for orphaned Relationship vertices to the database check command - [Release 1.3.6](/release-notes/infrahub/release-1_3_6.md): Add a database check-inheritance command to validate and fix any schemas that have had their inheritance updated and a failed migration. - [Release 1.3.7](/release-notes/infrahub/release-1_3_7.md): Ensure that only users with 'manage schema' permissions can add or remove dropdown and enum values (6410) Fix bug in branch delete cypher query that could leave behind orphaned branch-agnostic relationships. - [Release 1.4.0](/release-notes/infrahub/release-1_4_0.md): New features, performance improvements, and fixes across the platform. - [Release 1.4.1](/release-notes/infrahub/release-1_4_1.md): Fix bug in schema validation that would incorrectly flag Dropdown attributes of node schema that override a generic attribute as having illegal values (7086) - [Release 1.4.10](/release-notes/infrahub/release-1_4_10.md): Fix issue with template that would set the value/source of all attributes even for the attribute that are not defined in the template. - [Release 1.4.11](/release-notes/infrahub/release-1_4_11.md): The login form now automatically focuses on the first field. - [Release 1.4.12](/release-notes/infrahub/release-1_4_12.md): Schema Visualizer now displays on-delete settings for relationships. Fixed display of common parent settings in relationships. - [Release 1.4.13](/release-notes/infrahub/release-1_4_13.md): Added new error message for git connection error (7392) - [Release 1.4.2](/release-notes/infrahub/release-1_4_2.md): Fix a bug where a proposed change could be merged without approval even if some approvals were required (Enterprise) Removed incorrect log warning about 'Branch schema hash is not set, cannot update branch registry' due to including the '-global-' branch when processing branch updates. - [Release 1.4.3](/release-notes/infrahub/release-1_4_3.md): Force branches data to be reloaded when the hash doesn't look healthy. In the UI, clicking the artifact generation button now refreshes the token and retries if the access token has expired. - [Release 1.4.4](/release-notes/infrahub/release-1_4_4.md): Fix HTTP 403 when trying to fetch object metadata in changelog without being allowed to manage permissions (ifc1760) Fix HTTP 403 when trying to fetch nodes though a CoreNode query, this could prevent users to select nodes in various places with the user interface (6733) - [Release 1.4.5](/release-notes/infrahub/release-1_4_5.md): Security release: Fixes bug in authentication logic that allowed expired and/or deleted API tokens to authenticate successfully. - [Release 1.4.6](/release-notes/infrahub/release-1_4_6.md): Make related nodes clickable in task views (6420) Add an option to match trigger actions on any attribute value - [Release 1.4.7](/release-notes/infrahub/release-1_4_7.md): Added optional configuration to fetch and map groups when using Google as an identity provider for OAuth/OIDC. Added the name of the artifact definition to the payload of artifact webhook events. - [Release 1.4.8](/release-notes/infrahub/release-1_4_8.md): Report proper branch when read-only repositories fail to synchronize due to invalid branch (5713) Add an HFID for Attribute and Relationship matches for a Node Trigger Rule (6713) - [Release 1.4.9](/release-notes/infrahub/release-1_4_9.md): Fix prefix/IP Address creation when passing in the IP namespace HFID (7239) Fix bug in schema integrity checks of a proposed change that prevented resolved violations from being removed (7278) - [Main changes](/release-notes/infrahub/release-1_5_0.md): HFID and display_label refactored at write time, Profiles reworked, backup/restore tool added, webhook formats aligned. - [Release 1.5.1](/release-notes/infrahub/release-1_5_1.md): Security release: Updated FastAPI and a vulnerable version of Starlette - [Release 1.5.2](/release-notes/infrahub/release-1_5_2.md): Fix migration that backfills display labels and human-friendly IDs to account for schema that only exist on the branch being migrated. Prevent attempting diff update on a deleted branch. - [Release 1.5.3](/release-notes/infrahub/release-1_5_3.md): Fixed bug that prevented retrieving cardinality-one relationships on a branch that was already merged and included changes to the relationship. - [Main changes](/release-notes/infrahub/release-1_6_0.md): Improved Git integration and UI navigation, branch management, Profiles in templates, dashboard landing page. - [Release 1.6.1](/release-notes/infrahub/release-1_6_1.md): Add support for PKCE within Oauth2 and OIDC authentications. - [Release 1.6.2](/release-notes/infrahub/release-1_6_2.md): Fix Migration041 to determine edge uniqueness correctly and account for incoming Relationship edges. - [Release 1.6.3](/release-notes/infrahub/release-1_6_3.md): Stop showing warnings for deprecated attribute schema length and regex fields when they are not used (7995) - [Release 1.7.0](/release-notes/infrahub/release-1_7_0.md): Audit capabilities with immutable metadata tracking, Profile support for relationships, backend upgrades. - [Release 1.7.1](/release-notes/infrahub/release-1_7_1.md): Added option to use username/password authentication on Redis connections (7994) - [Release 1.7.2](/release-notes/infrahub/release-1_7_2.md): Added 'parallel mode' plugin to the GraphQL web sandbox. GraphQL query parsing and validation is now leveraging a cache to improve performance and CPU consumption. - [Release 1.7.3](/release-notes/infrahub/release-1_7_3.md): Security release: Update system dependencies: OpenSSL patch for CVE-2025-15467 - [Release 1.7.4](/release-notes/infrahub/release-1_7_4.md): Fix bug in computed attribute calculation that prevented having a computed attribute on a branch if there were no computed attributes on the default branch (8270) Add migration to allow rebasing branches created before (8221) was fixed so that the uniqueness constraint on SchemaNode.name and SchemaGeneric.name is removed - [Release 1.7.5](/release-notes/infrahub/release-1_7_5.md): Add migration to handle setting duplicated schemas on the default branch to be deleted, keeping the one with the latest update (8221) Handle deleted parent relationship schemas when combining diffs without crashing (8388) - [Release 1.7.6](/release-notes/infrahub/release-1_7_6.md): Pass order weights into Profile schema fields to prevent reordering causing a hash mismatch Update schema validation and processing to be completely idempotent to prevent incorrect hash errors when retrieving schemas - [Release 1.7.7](/release-notes/infrahub/release-1_7_7.md): Fixed display labels showing 'None' for relationship-based fields after upsert. - [Release 1.8.0](/release-notes/infrahub/release-1_8_0.md): File objects, automatic branch freeze-on-merge, resource pools in object templates, Kubernetes backup support. - [Release 1.8.1](/release-notes/infrahub/release-1_8_1.md): Added schema processing to schema integrity check in order to validate the schema (8355) - [Release 1.8.2](/release-notes/infrahub/release-1_8_2.md): Fix scheduled reconfiguration of webhooks targeting 'all' events (8694) Fixed branch rebase conflicts during 1.7 to 1.8 upgrades caused by migration 056 writing unrelated schema attributes to branch timelines - [Release 1.8.3](/release-notes/infrahub/release-1_8_3.md): Corrected data diff view within a proposed change or branch view so that changes to Profiles show up. Fixed hard failures on GraphQL queries that would exhaust Neo4j's server thread pool by implementing back off retry. - [Release 1.8.4](/release-notes/infrahub/release-1_8_4.md): Link a merging proposed change to its diff during the merge operation if it has not been linked yet (8769) Fixed an issue where webhook match statements (node kind filters) were lost during scheduled reconfiguration. - [Release 1.8.5](/release-notes/infrahub/release-1_8_5.md): Ensure that changes to a password attribute counts as a mutation event for a node. Add distributed lock around branch creation to prevent race condition (8368) - [Release 1.8.6](/release-notes/infrahub/release-1_8_6.md): Schema deletions are now rejected when a generic is still referenced by the inheritance of another node, and partial schema writes are rolled back if loading the updated schema fails. - [Release 1.9.0](/release-notes/infrahub/release-1_9_0.md): Two headline additions: an interactive schema visualizer that turns your schema into a navigable graph, and syslog log forwarding for Enterprise — streaming activity, login/logout, and permission-denied events to SIEM systems such as Splunk, Datadog, and ELK over TCP or UDP with optional TLS. Jinja2 computed attributes now recalculate inline on local changes instead of spawning a background task per node, and new Jinja2 filters let artifacts inline and parse content from other artifacts and file objects. - [Release 1.9.1](/release-notes/infrahub/release-1_9_1.md): Fixes stale schema cache when importing Git commits mixing schema and object files, duplicate hierarchy parents after merge, and branch owners unable to delete their own branch. - [Release 1.9.10](/release-notes/infrahub/release-1_9_10.md): Fixes the branch-merge rollback so a mid-merge database error no longer leaves partially merged data on the destination branch. - [Release 1.9.2](/release-notes/infrahub/release-1_9_2.md): Merged or deleted branches no longer reappear as needing a rebase after object-type conversions or an upgrade. - [Release 1.9.3](/release-notes/infrahub/release-1_9_3.md): Broader rollback protection against partially merged branches, and a fix for a race condition where concurrent workers shared the same Neo4j session. - [Release 1.9.4](/release-notes/infrahub/release-1_9_4.md): Adds a global permission for rebasing branches; diff calculation now scales with the branch's changes instead of database size, and IP/MAC attribute values are normalized. - [Release 1.9.5](/release-notes/infrahub/release-1_9_5.md): Fixes CoreNumberPool upserts, migrations when an attribute stops being unique, and cardinality constraints declared on subtypes of a generic peer. - [Release 1.9.6](/release-notes/infrahub/release-1_9_6.md): Fixes Copy ID/HFID/Token actions over plain HTTP and merges deleting objects whose kind changed on the default branch after forking. - [Release 1.9.7](/release-notes/infrahub/release-1_9_7.md): Enables OIDC id_token verification by default and blocks local password changes for directory-backed accounts. Schema names containing __ are now rejected. - [Release 1.9.8](/release-notes/infrahub/release-1_9_8.md): Faster transform-based computed attributes; fixes schema-constraint reconciliation on generics, prefix utilization on new branches, and Profile-sourced relationship overrides. - [Release 1.9.9](/release-notes/infrahub/release-1_9_9.md): Upgrades GitPython to address GHSA-mv93-w799-cj2w (configuration injection); also fixes Git imports with conflicting branches and rebase permission checks. ## resource-manager ### allocate-ip-address IP address pools (CoreIPAddressPool) dynamically allocate individual IP addresses from source prefixes. - [Allocate IP addresses](/resource-manager/allocate-ip-address.md): IP address pools (CoreIPAddressPool) dynamically allocate individual IP addresses from source prefixes. ### allocate-ip-prefix IP prefix pools (CoreIPPrefixPool) allocate IP subnets from larger prefixes. - [Allocate IP prefixes](/resource-manager/allocate-ip-prefix.md): IP prefix pools (CoreIPPrefixPool) allocate IP subnets from larger prefixes. ### allocate-number Number pools (CoreNumberPool) automatically assign sequential numbers to numeric attributes. - [Allocate numbers](/resource-manager/allocate-number.md): Number pools (CoreNumberPool) automatically assign sequential numbers to numeric attributes. ### overview Resource Manager in Infrahub automates the allocation of network resources from predefined pools, eliminating manual assignment and preventing conflicts across your infrastructure. This system ensures consistent resource allocation while maintaining complete visibility and control over your available resources. - [Resource Manager](/resource-manager/overview.md): Resource Manager in Infrahub automates the allocation of network resources from predefined pools, eliminating manual assignment and preventing conflicts across your infrastructure. This system ensures consistent resource allocation while maintaining complete visibility and control over your available resources. ### weighted-allocation When multiple resources can be allocated from a pool, you can control the allocation order using weights. Resources with higher weights are allocated first. - [Weighted allocation](/resource-manager/weighted-allocation.md): When multiple resources can be allocated from a pool, you can control the allocation order using weights. Resources with higher weights are allocated first. ## schema-and-data - [Schema & Data](/schema-and-data.md) ## schema ### analyze-your-data Ask an AI assistant questions about live Infrahub data through the MCP server, without writing GraphQL. - [Analyze your data](/schema/analyze-your-data.md): Ask an AI assistant questions about live Infrahub data through the MCP server, without writing GraphQL. ### branch-awareness Every schema element — nodes, attributes, and relationships — can be configured to control how changes to its data behave across branches. - [Branch awareness](/schema/branch-awareness.md): Every schema element — nodes, attributes, and relationships — can be configured to control how changes to its data behave across branches. ### build-your-schema Describe a schema change in plain language and have your assistant produce the schema YAML for it, validated before it reaches you. - [Build your schema with AI](/schema/build-your-schema.md): Describe a schema change in plain language and have your assistant produce the schema YAML for it, validated before it reaches you. ### create-and-load Schemas are defined in YAML format and follow a strict structure. Splitting a schema into multiple smaller files is recommended to keep it maintainable as it grows. - [Create and load schema](/schema/create-and-load.md): Schemas are defined in YAML format and follow a strict structure. Splitting a schema into multiple smaller files is recommended to keep it maintainable as it grows. ### default-ordering The order_by field on a node or generic sets the default sort order for every list of that kind's objects. Once declared, the order applies everywhere the objects are listed - the web interface, the GraphQL API, and the Python SDK - without each query specifying how to sort. - [Default ordering](/schema/default-ordering.md): The order_by field on a node or generic sets the default sort order for every list of that kind's objects. Once declared, the order applies everywhere the objects are listed - the web interface, the GraphQL API, and the Python SDK - without each query specifying how to sort. ### display_label Display labels in Infrahub provide human-friendly names and identifiers for schema elements and object instances. They serve as the primary way users interact with and identify objects throughout the system, from the user interface to GraphQL queries. - [Display labels](/schema/display_label.md): Display labels in Infrahub provide human-friendly names and identifiers for schema elements and object instances. They serve as the primary way users interact with and identify objects throughout the system, from the user interface to GraphQL queries. ### extensions Schema extensions provide a mechanism to add attributes and relationships to existing nodes without redefining them. This enables modular schema composition where features can be developed, reviewed, and managed as self-contained files. - [Schema extensions](/schema/extensions.md): Schema extensions provide a mechanism to add attributes and relationships to existing nodes without redefining them. This enables modular schema composition where features can be developed, reviewed, and managed as self-contained files. ### field-visibility Infrahub schemas can declare how attributes and relationships should appear in the frontend, independently of their presence in the data model. These declarations are purely UI hints — they do not affect data access via the API, GraphQL, or forms. - [Field visibility](/schema/field-visibility.md): Infrahub schemas can declare how attributes and relationships should appear in the frontend, independently of their presence in the data model. These declarations are purely UI hints — they do not affect data access via the API, GraphQL, or forms. ### file-object Infrastructure management often involves more than just structured data. Contracts, documents, and pictures are all essential assets that relate directly to the devices, circuits, and services they describe. Infrahub's file object feature lets you attach files to any node in the graph, making these files first-class citizens of your infrastructure data model with full version control, branch isolation, and permission enforcement. - [File objects](/schema/file-object.md): Infrastructure management often involves more than just structured data. Contracts, documents, and pictures are all essential assets that relate directly to the devices, circuits, and services they describe. Infrahub's file object feature lets you attach files to any node in the graph, making these files first-class citizens of your infrastructure data model with full version control, branch isolation, and permission enforcement. ### generics-and-inheritance A Generic in Infrahub is a reusable base that shares attributes and relationships across multiple node types. They work similarly to abstract base classes or interfaces in object-oriented programming — you define common structure once, and node types that inherit_from a generic receive that structure automatically. - [Generics & inheritance](/schema/generics-and-inheritance.md): A Generic in Infrahub is a reusable base that shares attributes and relationships across multiple node types. They work similarly to abstract base classes or interfaces in object-oriented programming — you define common structure once, and node types that inherit_from a generic receive that structure automatically. ### hierarchy Infrahub supports organizing nodes of similar types into a tree structure. Hierarchy mode enables ancestor and descendant queries, and allows filtering related data across multiple levels of the tree without direct relationships between every node. - [Hierarchy](/schema/hierarchy.md): Infrahub supports organizing nodes of similar types into a tree structure. Hierarchy mode enables ancestor and descendant queries, and allows filtering related data across multiple levels of the tree without direct relationships between every node. ### marketplace The Infrahub Marketplace is a public catalog of pre-built schemas and schema collections that you can fetch and load into any Infrahub instance. - [Infrahub Marketplace](/schema/marketplace.md): The Infrahub Marketplace is a public catalog of pre-built schemas and schema collections that you can fetch and load into any Infrahub instance. ### migration Unlike most databases that support a single schema at a time, in Infrahub it is possible to have a different schema per branch. - [Schema migration](/schema/migration.md): Unlike most databases that support a single schema at a time, in Infrahub it is possible to have a different schema per branch. ### namespaces Every model in your schema belongs to a namespace. A namespace groups related models together and makes each model's kind unique, so two teams can both define a Device model without a name collision. - [Namespaces](/schema/namespaces.md): Every model in your schema belongs to a namespace. A namespace groups related models together and makes each model's kind unique, so two teams can both define a Device model without a name collision. ### nodes-and-attributes A Node is the fundamental unit of the schema — it represents a concrete object in your infrastructure model such as a device, interface, or IP address. Nodes are composed of Attributes (direct values) and Relationships (links to other nodes). This page covers how to define nodes and attributes, what kinds are available, and how to configure identifiers, constraints, and display. - [Nodes & attributes](/schema/nodes-and-attributes.md): A Node is the fundamental unit of the schema — it represents a concrete object in your infrastructure model such as a device, interface, or IP address. Nodes are composed of Attributes (direct values) and Relationships (links to other nodes). This page covers how to define nodes and attributes, what kinds are available, and how to configure identifiers, constraints, and display. ### number-pool The NumberPool attribute kind enables the dynamic creation of a CoreNumberPool for the specific schema node and attribute rather than requiring users to define this after the schema is loaded. - [Number pools](/schema/number-pool.md): The NumberPool attribute kind enables the dynamic creation of a CoreNumberPool for the specific schema node and attribute rather than requiring users to define this after the schema is loaded. ### order-weight The orderweight property controls how attributes and relationships are ordered in the Infrahub frontend, including table views and detailed object views. **Lower orderweight values appear first. This means items with smaller numbers are displayed before those with larger numbers. Understanding how default values are assigned and how to work with them effectively is crucial for creating well-organized schemas. - [Order weight](/schema/order-weight.md): The orderweight property controls how attributes and relationships are ordered in the Infrahub frontend, including table views and detailed object views. **Lower orderweight values appear first. This means items with smaller numbers are displayed before those with larger numbers. Understanding how default values are assigned and how to work with them effectively is crucial for creating well-organized schemas. ### overview The schema is Infrahub's data model — a flexible, organization-specific definition of what objects exist in your infrastructure, design, and business logic, what properties they have, and how they relate to each other. It acts as an abstraction layer over the graph database, so no knowledge of database administration is needed for day-to-day use. - [About schema](/schema/overview.md): The schema is Infrahub's data model — a flexible, organization-specific definition of what objects exist in your infrastructure, design, and business logic, what properties they have, and how they relate to each other. It acts as an abstraction layer over the graph database, so no knowledge of database administration is needed for day-to-day use. ### relationships A Relationship in Infrahub represents a typed link between two nodes. Unlike a foreign key, relationships have a kind that determines how they appear in the UI and how they behave during deletion and hierarchy traversal. - [Relationships](/schema/relationships.md): A Relationship in Infrahub represents a typed link between two nodes. Unlike a foreign key, relationships have a kind that determines how they appear in the UI and how they behave during deletion and hierarchy traversal. ## snippets ### attribute-kind-params | Parameter | Default | - [attribute-kind-params](/snippets/attribute-kind-params.md): | Parameter | Default | ### pre-reqs-base | Prerequisite Tool(s) | Post Installation Steps | - [pre-reqs-base](/snippets/pre-reqs-base.md): | Prerequisite Tool(s) | Post Installation Steps | ### pre-reqs-frontend | Prerequisite Tool(s) | Post Installation Steps | - [pre-reqs-frontend](/snippets/pre-reqs-frontend.md): | Prerequisite Tool(s) | Post Installation Steps | ## testing-framework ### overview Summary - [Testing framework](/testing-framework/overview.md): Summary ## transformations ### jinja2 A Jinja2 Transformation renders Infrahub data through a Jinja template, producing plain text output (configurations, manifests, payloads). The steps below cover how to write one. - [Write a Jinja2 Transformation](/transformations/jinja2.md): A Jinja2 Transformation renders Infrahub data through a Jinja template, producing plain text output (configurations, manifests, payloads). The steps below cover how to write one. ### overview A Transformation is a generic plugin to transform a dataset into a different format to simplify it's ingestion by third-party systems. - [Transformations](/transformations/overview.md): A Transformation is a generic plugin to transform a dataset into a different format to simplify it's ingestion by third-party systems. ### python A Python Transformation processes Infrahub data through user-written Python code, producing JSON output (or any structured data you serialize). Use this when Jinja templating isn't enough — for example, conditional logic, external API calls, or complex aggregation. The steps below cover how to write one. - [Write a Python Transformation](/transformations/python.md): A Python Transformation processes Infrahub data through user-written Python code, producing JSON output (or any structured data you serialize). Use this when Jinja templating isn't enough — for example, conditional logic, external API calls, or complex aggregation. The steps below cover how to write one. ## webhooks ### create 1. Login to Infrahub's web interface as an administrator. - [Create a webhook](/webhooks/create.md): 1. Login to Infrahub's web interface as an administrator. ### custom-transformation Use a custom webhook to transform event data before Infrahub sends it to an external endpoint. This is useful when the receiving system expects a specific payload format, such as Slack, Microsoft Teams, GitHub Actions, or other third-party APIs. - [Webhook with custom Transformation](/webhooks/custom-transformation.md): Use a custom webhook to transform event data before Infrahub sends it to an external endpoint. This is useful when the receiving system expects a specific payload format, such as Slack, Microsoft Teams, GitHub Actions, or other third-party APIs. ### overview Webhooks push data to external systems each time a matching event occurs in Infrahub. Use them to: - [What are webhooks?](/webhooks/overview.md): Webhooks push data to external systems each time a matching event occurs in Infrahub. Use them to: --- # Full Documentation Content 1 ### Orientation Load a base schema, seed the OtterNet topology, and explore the data with GraphQL, infrahubctl and the Python SDK. 1 hBeginner [Start lab](https://play.instruqt.com/opsmill/invite/ndncxvvluews)Details --- [Skip to main content](#__docusaurus_skipToContent_fallback) 📚 New docs structure: content is now grouped by capability, not split across Topics and Guides. [**See what changed →**](/release-notes/infrahub/docs-restructure.md) [![Infrahub](/img/infrahub-hori.svg)![Infrahub](/img/infrahub-hori-dark.svg)](/index.md) [Infrahub Docs](/index.md)[Labs](/labs.md) [Tools & SDKs](#) * [Python SDK](/python-sdk/introduction.md) * [Infrahubctl](/infrahubctl/infrahubctl.md) * [Infrahub Assistant | Emma](/emma.md) * [VScode Extension for Infrahub](/vscode.md) * [MCP Server for Infrahub](/mcp.md) * [Infrahub AI Skills](/skills.md) * [Infrahub Backup](/backup.md) * [Infrahub Collect](/backup/collect.md) [Integrations](#) * [All Integrations](/integrations.md) * [Ansible](/ansible.md) * [Arista AVD](/arista-avd.md) * [Infrahub Exporter](/exporter.md) * [Infrahub Sync](/sync.md) * [Nornir](/nornir.md) [Demos](#) * [Data Center](/demo-dc.md) * [Service Provider](/infrahub-demo-sp.md) * [Optical Transport](/demo-otn.md) * [Service Catalog](/demo-service-catalog.md) [Reference Designs](#) * [AI Datacenter](/infrahub-solution-ai-dc/solution-ai-dc/overview.md) * [Arista AVD](/arista-avd.md) Search [](https://github.com/opsmill/infrahub)[](https://discord.gg/opsmill) # Search the documentation Type your search here Powered by[](https://www.algolia.com/) Copyright © 2026 - Infrahub by OpsMill. --- # opsmill.infrahub Ansible collection Collection version 1.8.3 ## Collection overview[​](#collection-overview "Direct link to Collection overview") The OpsMill Infrahub Ansible Collection provides is intend to help interact with Infrahub through Ansible. This Ansible collection consists of a set of modules and plugins designed to work seamlessly with your existing infrastructure, enabling you to define and enforce the desired state of your infrastructure with ease. ## Guides[​](#guides "Direct link to Guides") To begin using the OpsMill Infrahub Ansible Collection, please follow our step-by-step guides: * **[Installation Guide](/ansible/guides/installation.md)**: Learn how to install the collection, including Python module and Ansible setup, as well as alternative installation options. * **[Dynamic Inventory Guide](/ansible/guides/dynamic-inventory.md)**: Discover how to leverage the collection's dynamic inventory features to streamline your infrastructure management. * **[Query & Lookup Guide](/ansible/guides/query-and-lookup.md)**: Learn how to retrieve structured GraphQL data from Infrahub using Query and Lookup plugins. * **[Create, Update and Delete Nodes Guide](/ansible/guides/node.md)**: Learn how to create, update and delete nodes in Infrahub using GraphQL action plugin or the node module. * **[Manipulate Branch Guide](/ansible/guides/branch.md)**: Learn how to manipulate Branch in Infrahub. ## References[​](#references "Direct link to References") ### Plugins[​](#plugins "Direct link to Plugins") These are the plugins in the `opsmill.infrahub` collection: #### Modules[​](#modules "Direct link to Modules") * [artifact\_fetch](/ansible/references/plugins/artifact_fetch_module.md) – Fetch the content of an artifact from Infrahub * [node](/ansible/references/plugins/node_module.md) – Creates, Updates or Deletes a node in Infrahub * [branch](/ansible/references/plugins/branch_module.md) – Creates, Updates or Deletes a branch in Infrahub * [query\_graphql](/ansible/references/plugins/query_graphql_module.md) – Queries and returns elements from Infrahub GraphQL API * [object\_file\_fetch](/ansible/references/plugins/object_file_fetch_module.md) – Fetch file content from a CoreFileObject node in Infrahub * [artifact\_generate](/ansible/references/plugins/artifact_generate_module.md) – Trigger artifact regeneration in Infrahub * [schema](/ansible/references/plugins/schema_module.md) – Load, check, or export schemas in Infrahub #### Inventory[​](#inventory "Direct link to Inventory") * [inventory](/ansible/references/plugins/inventory_inventory.md) – Infrahub inventory source (using GraphQL) #### Lookup[​](#lookup "Direct link to Lookup") * [lookup](/ansible/references/plugins/lookup_lookup.md) – Queries and returns elements from Infrahub (using GraphQL) ### Roles[​](#roles "Direct link to Roles") These are the roles in the `opsmill.infrahub` collection: * [install](/ansible/references/roles/install.md) – Install Infrahub --- # Manage branches in Infrahub Infrahub lets you manage branches directly via the `opsmill.infrahub.branch` module. With this module you can create, confirm, and delete branches—using the `state` parameter to control the desired operation. The `opsmill.infrahub.branch` module uses the Infrahub SDK to perform branch operations through GraphQL. You can specify the branch name, description, and whether the branch should be synchronized with Git. * Use `state: present` to create or confirm that a branch exists. * Use `state: absent` to delete a branch. The `opsmill.infrahub.branch` module does not support Ansible check option. ## Usage[​](#usage "Direct link to Usage") ### Creating a branch or confirming its existence[​](#creating-a-branch-or-confirming-its-existence "Direct link to Creating a branch or confirming its existence") The following playbook creates a branch named `test` with a description. If the branch already exists, the module will report that it already exists without making any changes. create\_branch.yml ``` --- - name: Manage Branch in Infrahub hosts: localhost gather_facts: false tasks: - name: Create a Branch 'test' opsmill.infrahub.branch: name: "test" sync_with_git: false description: "This is a test branch" state: present - name: Confirm 'test' exists opsmill.infrahub.branch: name: "test" state: present ``` **Example Output from create\_branch.yml** ``` TASK [Create a Branch 'test'] ************************************************************************* changed: [localhost] => {"branch": "id='1823716d-eb2a-c26f-35ef-c5155b8f34e4' name='test' description='This is a test branch' sync_with_git=False is_default=False has_schema_changes=False origin_branch='main' branched_from='2025-02-12T11:09:58.005622Z'", "changed": true, "msg": "InfrahubBranch test created"} TASK [Confirm 'test' exists] ************************************************************************* ok: [localhost] => {"branch": "id='1823716d-eb2a-c26f-35ef-c5155b8f34e4' name='test' description='This is a test branch' sync_with_git=False is_default=False has_schema_changes=False origin_branch='main' branched_from='2025-02-12T11:09:58.005622Z'", "changed": false, "msg": "InfrahubBranch test already exists."} ``` ### Deleting a branch[​](#deleting-a-branch "Direct link to Deleting a branch") To delete a branch, set the `state` parameter to absent. The module will remove the branch from Infrahub. delete\_branch.yml ``` --- - name: Delete Branch 'test' hosts: localhost gather_facts: false tasks: - name: Delete a Branch 'test' opsmill.infrahub.branch: name: "test" state: absent ``` **Example Output from delete\_branch.yml** ``` TASK [Delete a Branch 'test'] ************************************************************************* changed: [localhost] => {"branch": "id='1823716d-eb2a-c26f-35ef-c5155b8f34e4' name='test' description='This is a test branch' sync_with_git=False is_default=False has_schema_changes=False origin_branch='main' branched_from='2025-02-12T11:09:58.005622Z'", "changed": true, "msg": "InfrahubBranch test deleted"} ``` --- # Use Infrahub for Ansible dynamic inventory ## Overview[​](#overview "Direct link to Overview") The [**Inventory Plugin**](/ansible/references/plugins/inventory_inventory.md) component of the OpsMill Infrahub Ansible Collection is used to dynamically generate the inventory from Infrahub to be used in Ansible playbooks. ## Configuration[​](#configuration "Direct link to Configuration") In your `ansible.cfg` file, specify the inventory source as the `infrahub_inv.yml` file: ansible.cfg ``` [defaults] inventory = ./infrahub_inv.yml ``` ## Infrahub inventory plugin configuration[​](#infrahub-inventory-plugin-configuration "Direct link to Infrahub inventory plugin configuration") Configure the Infrahub inventory plugin in the `infrahub_inv.yml` file to define the returned hosts and groupings. Here's a minimal example: * Retrieving attributes for the `InfraDevice` Node Kind in a specific branch * Forcing the inclusion of specific attributes for `InfraDevice` * Creating composed variables for `hostname` infrahub\_inv.yml ``` plugin: opsmill.infrahub.inventory api_endpoint: "http://localhost:8000" nodes: InfraDevice: include: - name - primary_address.address - platform.ansible_network_os - site.parent.name - site.name - role - tags compose: hostname: name ``` ## Advanced configuration example[​](#advanced-configuration-example "Direct link to Advanced configuration example") This example demonstrates a more comprehensive configuration, including: * Retrieving attributes for the `InfraDevice` Node Kind in a specific branch * Forcing the inclusion of specific attributes for `InfraDevice` * Creating composed variables for `hostname` and `platform` * Creating groups based on the `site` name and `country` name infrahub\_inv.yml (Advanced) ``` strict: true branch: "branch1" nodes: InfraDevice: include: - name - primary_address.address - platform.ansible_network_os - site.parent.name - site.name - role - tags compose: hostname: name platform: platform.ansible_network_os ansible_host: primary_address.address | ansible.utils.ipaddr('address') keyed_groups: - prefix: site key: site.name | lower - prefix: country key: site.parent.name | lower ``` ## Viewing the inventory graph[​](#viewing-the-inventory-graph "Direct link to Viewing the inventory graph") To visualize the inventory retrieved from Infrahub, run the `ansible-inventory` command with the `-i` flag, specifying the `infrahub_inv.yml` file as the source, followed by `--graph`: ``` ansible-inventory -i infrahub_inv.yml --graph ``` **Sample Output** ``` @all: |--@ungrouped: |--@site_atl1: | |--atl1-edge1 | |--atl1-edge2 | |--atl1-core1 | |--atl1-core2 | |--atl1-leaf1 | |--atl1-leaf2 |--@country_united_states_of_america: | |--atl1-edge1 | |--atl1-edge2 | |--atl1-core1 | |--atl1-core2 | |--atl1-leaf1 | |--atl1-leaf2 | |--ord1-edge1 | |--ord1-edge2 | |--ord1-core1 | |--ord1-core2 | |--ord1-leaf1 | |--ord1-leaf2 | |--jfk1-edge1 | |--jfk1-edge2 | |--jfk1-core1 | |--jfk1-core2 | |--jfk1-leaf1 |--@site_ord1: | |--ord1-edge1 | |--ord1-edge2 | |--ord1-core1 | |--ord1-core2 | |--ord1-leaf1 | |--ord1-leaf2 |--@site_jfk1: | |--jfk1-edge1 | |--jfk1-edge2 | |--jfk1-core1 | |--jfk1-core2 | |--jfk1-leaf1 ``` ## Viewing the inventory list[​](#viewing-the-inventory-list "Direct link to Viewing the inventory list") To view the inventory list, run: ``` ansible-inventory -i infrahub_inv.yml --list ``` **Sample Output** ``` { "_meta": { "hostvars": { "atl1-core1": { "id": "181fd07a-e43c-f377-35bf-c513b592513f", "site": { "id": "181fd075-b6d7-bd90-35b7-c51c888ef203", "name": "atl1" } }, "atl1-core2": { "id": "181fd07b-2615-5c76-35b3-c519087e0eb5", "site": { "id": "181fd075-b6d7-bd90-35b7-c51c888ef203", "name": "atl1" } }, ... } }, "all": { "children": [ "ungrouped", "site_atl1", "site_ord1", "site_jfk1", "site_den1", "site_dfw1" ] }, "site_atl1": { "hosts": [ "atl1-edge1", "atl1-edge2", "atl1-core1", "atl1-core2", "atl1-leaf1", "atl1-leaf2" ] }, ... } ``` ## Running playbooks with dynamic inventory[​](#running-playbooks-with-dynamic-inventory "Direct link to Running playbooks with dynamic inventory") To run a playbook that uses the dynamic inventory, specify the inventory file when executing the playbook: ``` ansible-playbook -i infrahub_inv.yml your_playbook.yml ``` Replace `infrahub_inv.yml` with the path to your dynamic inventory file, and `your_playbook.yml` with the name of your playbook file. ### Targeting groups in your playbook[​](#targeting-groups-in-your-playbook "Direct link to Targeting groups in your playbook") To target groups from the inventory in your playbook, reference the groups as normal. For example, to target the `site_atl1` group: playbook.yml ``` --- - name: Playbook using Infrahub Dynamic Inventory hosts: site_atl1 tasks: - name: Gather facts setup: - name: Print host facts debug: var: ansible_facts ``` --- # Install `opsmill.infrahub` Ansible collection info This guide assumes you have basic knowledge of Ansible and its ecosystem. For more information on using Ansible, please refer to the [official Ansible documentation](https://docs.ansible.com/ansible/latest/index.html). ## Requirements[​](#requirements "Direct link to Requirements") * You must be running one of the two most recent releases of Infrahub * A Infrahub API token when using modules (at least read-only for the inventory). * Python 3.11+ * Python modules: * infrahub-sdk * Ansible 2.19+ ## Installation[​](#installation "Direct link to Installation") ### Python modules and Ansible[​](#python-modules-and-ansible "Direct link to Python modules and Ansible") ``` pip install infrahub-sdk pip install ansible ``` ### Infrahub Ansible collection[​](#infrahub-ansible-collection "Direct link to Infrahub Ansible collection") Before using this collection, you need to install it with the Ansible Galaxy command-line tool: ``` ansible-galaxy collection install opsmill.infrahub ``` You can also include it in a `requirements.yml` file and install it with `ansible-galaxy collection install -r requirements.yml`, using the format: ``` collections: - name: opsmill.infrahub ``` To upgrade the collection to the latest available version, run the following command: ``` ansible-galaxy collection install opsmill.infrahub --upgrade ``` You can also install a specific version of the collection, for example, if you need to downgrade. Use the following syntax to install a specific version: ``` ansible-galaxy collection install opsmill.infrahub:==VERSION ``` See using [Ansible collections](https://docs.ansible.com/ansible/latest/collections_guide/collections_installing.html) for more Collapsible. ### Other installation options[​](#other-installation-options "Direct link to Other installation options") #### Build from source[​](#build-from-source "Direct link to Build from source") * Build from source * Build from source (Pull Request) 1. **Step 1**: `git clone git@github.com:opsmill/infrahub-ansible.git` 2. **Step 2**: `cd ansible_modules` 3. **Step 3**: `ansible-galaxy collection build .` 4. **Step 4**: `ansible-galaxy collection install opsmill-infrahub*.tar.gz` *For more Collapsible on building from source, please refer to \[our dedicated documentation]\(link if available).* *Useful for testing changes in Pull Requests.* 1. **Step 1**: `git clone git@github.com:opsmill/infrahub-ansible.git` 2. **Step 2**: `cd ansible_modules` 3. **Step 3**: `git fetch origin pull//head:` 4. **Step 4**: `git checkout ` 5. **Step 5**: `ansible-galaxy collection build .` 6. **Step 6**: `ansible-galaxy collection install opsmill-infrahub*.tar.gz` *For detailed information on checking out pull requests locally, please refer to [GitHub's Documentation](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally).* --- # Create nodes in Infrahub ## Overview[​](#overview "Direct link to Overview") Infrahub provides two methods for creating nodes using GraphQL: * [**Query Plugin**](/ansible/references/plugins/query_graphql_module.md): Use the `opsmill.infrahub.query_graphql` module with a mutation to create nodes. * [**Node Module**](/ansible/references/plugins/node_module.md): Use the dedicated `opsmill.infrahub.node` module to create, update and delete nodes directly. `opsmill.infrahub.node` allowed a better control in your playbook as the module support both Ansible diff and check options. ## Configuration[​](#configuration "Direct link to Configuration") Before using the Infrahub node modules, you need to configure the connection to your Infrahub instance. All node operations require: * **Infrahub API endpoint**: The URL of your Infrahub instance * **API token**: Authentication token to access the Infrahub API You can configure these settings using environment variables or by passing them directly in your playbook. ### Environment Variables (Recommended)[​](#environment-variables-recommended "Direct link to Environment Variables (Recommended)") Set these environment variables before running your playbooks: ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_API_TOKEN="your-api-token-here" ``` ### Playbook Configuration[​](#playbook-configuration "Direct link to Playbook Configuration") Alternatively, set them on the play with the `environment` keyword, which applies to every task in it: ``` - name: Work against Infrahub hosts: localhost gather_facts: false environment: INFRAHUB_ADDRESS: "http://localhost:8000" INFRAHUB_API_TOKEN: "your-api-token-here" tasks: - name: Create a node opsmill.infrahub.node: kind: "InfraDevice" data: name: "device1" ``` Or pass them directly to the modules: ``` - name: Create node with direct parameters opsmill.infrahub.node: api_endpoint: "http://localhost:8000" token: "{{ lookup('ansible.builtin.env', 'INFRAHUB_API_TOKEN') }}" kind: "InfraDevice" data: name: "device1" ``` ## Usage[​](#usage "Direct link to Usage") ### Creating a node[​](#creating-a-node "Direct link to Creating a node") * Node Plugin * Creating nodes with the query plugin The Create Node Plugin simplifies node creation and supports the `allow_upsert` parameter. When `allow_upsert` is set to true (default mode), the module will update an existing node if it already exists, preventing duplicate entries or errors. This example demonstrates how to create various nodes (an InfraDevice, a VLAN, and an InterfaceL2) using the dedicated Create Node Plugin: node.yml ``` --- - name: Create InfraDevice examples gather_facts: false hosts: localhost environment: INFRAHUB_ADDRESS: "http://localhost:8000" INFRAHUB_API_TOKEN: "{{ lookup('ansible.builtin.env', 'INFRAHUB_API_TOKEN') }}" tasks: - name: Create a InfraDevice 'atl1-sw1' opsmill.infrahub.node: kind: "InfraDevice" data: name: "atl1-sw1" type: "MX204" site: "atl1" platform: "Juniper JunOS" - name: Create a VLAN 'atl1-vlan1' opsmill.infrahub.node: kind: "InfraVLAN" data: name: "atl1-vlan1" vlan_id: 10 status: "active" role: "server" - name: Create a InterfaceL2 'xe-0/0/1' on 'atl1-sw1' and add VLAN 'atl1-vlan1' opsmill.infrahub.node: kind: "InfraInterfaceL2" data: name: "xe-0/0/1" device: "atl1-sw1" speed: 10 l2_mode: "Access" tagged_vlan: - ["atl1-vlan10", "10"] ``` **Example Output from node.yml** ``` PLAY [Create InfraDevice examples] ******************************************************************** TASK [Create a InfraDevice 'atl1-sw1'] **************************************************************** changed: [localhost] => {"InfraDevice": {"name": "atl1-sw1", "platform": "Juniper JunOS", "site": "atl1", "type": "MX204"}, "changed": true, "msg": "InfraDevice 18237235-3a70-85f6-2db7-c5139cfb8e9a created"} TASK [Create a VLAN 'atl1-vlan1'] ******************************************************************** changed: [localhost] => {"InfraVLAN": {"name": "atl1-vlan1", "role": "server", "status": "active", "vlan_id": 10}, "changed": true, "msg": "InfraVLAN 18237235-95f7-64ce-2dbc-c51f7c63c1d5 created"} TASK [Create a InterfaceL2 'xe-0/0/1' on 'atl1-sw1' and add VLAN 'atl1-vlan1'] *********************** changed: [localhost] => {"InfraInterfaceL2": {"device": "atl1-sw1", "l2_mode": "Access", "name": "xe-0/0/1", "speed": 10, "tagged_vlan": [["atl1-vlan10", "10"]]}, "changed": true, "msg": "InfraInterfaceL2 18237235-dd9a-4034-2dbb-c51a0c3505ce created"} PLAY RECAP ******************************************************************************************** localhost : ok=3 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 ``` In this example, we create BuiltinTags using a GraphQL mutation via the Query Plugin: node.yml ``` --- - name: Create BuiltinTag using Query module gather_facts: false hosts: localhost environment: INFRAHUB_ADDRESS: "http://localhost:8000" INFRAHUB_API_TOKEN: "{{ lookup('ansible.builtin.env', 'INFRAHUB_API_TOKEN') }}" vars: infrahub_tags: - name: my-tag10 description: "This is tag 10 description" - name: my-tag11 description: "This is tag 11 description" tasks: - name: Create tags opsmill.infrahub.query_graphql: query: | mutation CreateTag($name: String!, $description: String!) { BuiltinTagCreate( data: { name: {value: $name}, description: {value: $description} } ) { ok object { id } } } graph_variables: name: "{{ item.name }}" description: "{{ item.description }}" loop: "{{ infrahub_tags }}" register: created_tags ``` ### Deleting a node[​](#deleting-a-node "Direct link to Deleting a node") * Node Plugin To delete a node, set the `state` parameter to absent. The module will remove the node from Infrahub. delete\_tag.yml ``` --- - name: Delete tag 'tag1' hosts: localhost gather_facts: false environment: INFRAHUB_ADDRESS: "http://localhost:8000" INFRAHUB_API_TOKEN: "{{ lookup('ansible.builtin.env', 'INFRAHUB_API_TOKEN') }}" tasks: - name: Delete tag1 opsmill.infrahub.node: kind: "BuiltinTag" data: name: "tag1" state: absent ``` **Example Output from delete\_tag.yml** ``` TASK [Delete tag1] ********************************************************************* changed: [localhost] => {"BuiltinTag": {"__typename": "BuiltinTag", "description": {"value": null}, "display_label": "tag1", "hfid": ["tag1"], "id": "182371ab-d055-096d-2dbd-c51e34a81383", "name": {"value": "tag1"}}, "changed": true, "msg": "BuiltinTag 182371ab-d055-096d-2dbd-c51e34a81383 deleted"} ``` --- # Retrieve data from Infrahub ## Overview[​](#overview "Direct link to Overview") The **OpsMill Infrahub Ansible Collection** provides two ways to interact with Infrahub GraphQL API: * [**Query Plugin**](/ansible/references/plugins/query_graphql_module.md): The `opsmill.infrahub.query_graphql` module allows you to send a structured GraphQL query and dynamically process the returned results. This method is used as an action plugin within your playbook. * [**Lookup Plugin**](/ansible/references/plugins/lookup_lookup.md): The `opsmill.infrahub.lookup` plugin enables you to retrieve GraphQL data using a query and integrate the results into your playbook (using the lookup plugin syntax). ## Configuration[​](#configuration "Direct link to Configuration") Before using the Infrahub plugins, you need to configure the connection to your Infrahub instance. Both plugins require: * **Infrahub API endpoint**: The URL of your Infrahub instance * **API token**: Authentication token to access the Infrahub API You can configure these settings using environment variables or by passing them directly in your playbook. ### Environment Variables (Recommended)[​](#environment-variables-recommended "Direct link to Environment Variables (Recommended)") Set these environment variables before running your playbooks: ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_API_TOKEN="your-api-token-here" ``` ### Playbook Configuration[​](#playbook-configuration "Direct link to Playbook Configuration") Alternatively, set them on the play with the `environment` keyword, which applies to every task in it: ``` - name: Work against Infrahub hosts: localhost gather_facts: false environment: INFRAHUB_ADDRESS: "http://localhost:8000" INFRAHUB_API_TOKEN: "your-api-token-here" tasks: - name: Build the query ansible.builtin.set_fact: query_dict: InfraDevice: edges: node: name: value: null - name: Run the query opsmill.infrahub.query_graphql: query: "{{ query_dict }}" ``` Or pass them directly to the modules: ``` - name: Query with direct parameters opsmill.infrahub.query_graphql: api_endpoint: "http://localhost:8000" token: "{{ lookup('ansible.builtin.env', 'INFRAHUB_API_TOKEN') }}" query: "{{ query_dict }}" ``` ## Usage[​](#usage "Direct link to Usage") * Querying Infrahub with the Query Plugin * Querying Infrahub with the Lookup Plugin query\_infrahub.yml ``` - name: Infrahub action plugin gather_facts: false hosts: localhost environment: INFRAHUB_ADDRESS: "http://localhost:8000" INFRAHUB_API_TOKEN: "{{ lookup('ansible.builtin.env', 'INFRAHUB_API_TOKEN') }}" tasks: - name: SET FACTS TO SEND TO GRAPHQL ENDPOINT ansible.builtin.set_fact: variables: device_name: "atl1-edge1" enabled: true query_dict: InfraInterface: '@filters': {enabled__value: '$enabled', device__name__value: '$device_name'} edges: node: id: null name: value: null ... on InfraInterfaceL3: ip_addresses: edges: node: address: value: null ... on InfraInterfaceL2: tagged_vlan: edges: node: vlan_id: value: null name: value: null untagged_vlan: node: vlan_id: value: null name: value: null - name: Execute Query Plugin opsmill.infrahub.query_graphql: query: "{{ query_dict }}" graph_variables: "{{ variables }}" ``` * The `query_dict` defines a **GraphQL query structure**. * The query filters devices by `name` and interfaces by `enabled`. * The **`opsmill.infrahub.query_graphql`** plugin executes the query and returns the results. To run the query playbook, use: ``` ansible-playbook query_infrahub.yml ``` This executes the query and fetches data dynamically from **Infrahub**. **Example Output from Query plugin** ``` ok: [localhost] => { "msg": { "edges": [ { "node": { "id": "18219d8a-5417-acae-2dbc-c5156eefb194", "ip_addresses": { "edges": [ { "node": { "address": { "value": "10.1.0.0/31" } } } ] }, "name": { "value": "Ethernet1" } } }, { "node": { "id": "18219d8b-afcc-3d1b-2db4-c510fcc52b8a", "ip_addresses": { "edges": [] }, "name": { "value": "Ethernet10" } } }, { "node": { "id": "18219d8b-b7bc-dc82-2dba-c51741f8258e", "name": { "value": "Ethernet11" }, "tagged_vlan": { "edges": [] }, "untagged_vlan": { "node": { "name": { "value": "atl1_server" }, "vlan_id": { "value": 200 } } } } }, { "node": { "id": "18219d8b-d71d-e7d1-2dba-c515e35ae516", "name": { "value": "Ethernet12" }, "tagged_vlan": { "edges": [] }, "untagged_vlan": { "node": { "name": { "value": "atl1_server" }, "vlan_id": { "value": 200 } } } } }, { "node": { "id": "18219d8a-7e5a-df0a-2db2-c51e9967f805", "ip_addresses": { "edges": [ { "node": { "address": { "value": "10.1.0.2/31" } } } ] }, "name": { "value": "Ethernet2" } } }, { "node": { "id": "18219d8a-8673-961c-2db0-c5179da68d60", "ip_addresses": { "edges": [ { "node": { "address": { "value": "10.1.0.20/31" } } } ] }, "name": { "value": "Ethernet3" } } }, { "node": { "id": "18219d8a-8a01-a4b8-2db8-c51c7024f9dd", "ip_addresses": { "edges": [ { "node": { "address": { "value": "10.1.0.24/31" } } } ] }, "name": { "value": "Ethernet4" } } }, { "node": { "id": "18219d8a-8d42-143c-2dbf-c51a0cc428a9", "ip_addresses": { "edges": [ { "node": { "address": { "value": "203.111.0.1/29" } } } ] }, "name": { "value": "Ethernet5" } } }, { "node": { "id": "18219d8b-63d4-9267-2db4-c518a51761cf", "ip_addresses": { "edges": [ { "node": { "address": { "value": "203.111.0.9/29" } } } ] }, "name": { "value": "Ethernet6" } } }, { "node": { "id": "18219d8b-8e96-24cf-2db4-c51ba3de6b92", "ip_addresses": { "edges": [] }, "name": { "value": "Ethernet7" } } }, { "node": { "id": "18219d8b-921b-2dd9-2dbb-c519044a4a98", "ip_addresses": { "edges": [] }, "name": { "value": "Ethernet8" } } }, { "node": { "id": "18219d8b-95ce-829d-2dbd-c519355daf70", "ip_addresses": { "edges": [ { "node": { "address": { "value": "203.111.0.17/29" } } } ] }, "name": { "value": "Ethernet9" } } }, { "node": { "id": "18219d89-dbf4-92da-2dbd-c51e7009abd0", "ip_addresses": { "edges": [ { "node": { "address": { "value": "10.0.0.1/32" } } } ] }, "name": { "value": "Loopback0" } } }, { "node": { "id": "18219d8a-3900-2637-2db7-c511a30b601a", "ip_addresses": { "edges": [ { "node": { "address": { "value": "172.16.0.1/16" } } } ] }, "name": { "value": "Management0" } } }, { "node": { "id": "18219d8b-dfe6-4944-2db9-c511dc191dc6", "name": { "value": "port-channel1" } } } ] } } ``` This example retrieves a list of all locations (`LocationSite`) from Infrahub and prints them. lookup\_infrahub.yml ``` - name: Infrahub lookup gather_facts: false hosts: localhost environment: INFRAHUB_ADDRESS: "http://localhost:8000" INFRAHUB_API_TOKEN: "{{ lookup('ansible.builtin.env', 'INFRAHUB_API_TOKEN') }}" tasks: - name: SET FACT OF STRING ansible.builtin.set_fact: query_string: | query { LocationSite { edges { node { name { value } } } } } - name: Obtain list of sites from Infrahub ansible.builtin.set_fact: query_response: "{{ query('opsmill.infrahub.lookup', query=query_string) }}" - name: Print result ansible.builtin.debug: msg: "{{ query_response }}" ``` * The `query_string` defines a **GraphQL query** to retrieve **all locations**. * The **`opsmill.infrahub.lookup`** plugin executes the query. * The result is stored in `query_response` and printed. To run the lookup playbook, use: ``` ansible-playbook lookup_infrahub.yml ``` This executes the lookup and retrieves the requested data. **Example Output from lookup playbook** ``` PLAY [Infrahub lookup] ************************************************************************************************************************************* TASK [SET FACT OF STRING] ********************************************************************************************************************************** ok: [localhost] TASK [Obtain list of sites from Infrahub] ****************************************************************************************************************** ok: [localhost] TASK [Print result] **************************************************************************************************************************************** ok: [localhost] => { "msg": [ { "node": { "name": { "value": "atl1" } } }, { "node": { "name": { "value": "den1" } } }, { "node": { "name": { "value": "dfw1" } } }, { "node": { "name": { "value": "jfk1" } } }, { "node": { "name": { "value": "ord1" } } } ] } ``` ## Conclusion[​](#conclusion "Direct link to Conclusion") Both the Query Plugin and Lookup Plugin allow you to perform GraphQL queries against Infrahub, but they are tailored to different usage patterns in your Ansible playbooks. Use the Query Plugin when you want to execute complex, structured queries with variables as an action, and the Lookup Plugin when you prefer a simpler query string approach for integrating query results directly into variables. --- # Modules modules Fetch the content of an artifact from Infrahub through Infrahub SDK ## Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ---------------- | ------ | -------- | ------- | ------------------------------------------------------------------------- | | `api_endpoint` | `str` | No | | Endpoint of the Infrahub API, optional env=INFRAHUB\_ADDRESS | | `token` | `str` | No | | The API token created through Infrahub, optional env=INFRAHUB\_API\_TOKEN | | `timeout` | `int` | No | 10 | Timeout for Infrahub requests in seconds | | `artifact_name` | `str` | No | | Name of the artifact | | `artifact_id` | `str` | No | | UUID of the artifact | | `target_id` | `str` | No | | Id of the target for this artifact | | `branch` | `str` | No | main | Branch in which the request is made | | `validate_certs` | `bool` | No | True | Whether or not to validate SSL of the Infrahub instance | ## Examples[​](#examples "Direct link to Examples") ``` --- - name: Infrahub action plugin artifact_fetch gather_facts: false hosts: platform_eos vars: ansible_become: true tasks: - name: Query Startup Config for Edge Devices opsmill.infrahub.artifact_fetch: artifact_name: "Startup Config for Edge devices" target_id: "{{ id }}" register: startup_artifact - name: Save configs to localhost ansible.builtin.copy: content: "{{ startup_artifact.text }}" dest: "/tmp/{{ inventory_hostname }}-startup.conf" mode: '644' delegate_to: localhost ``` ## Return values[​](#return-values "Direct link to Return values") | Key | Type | Description | | ------ | ------ | --------------------------------------- | | `json` | `dict` | Content of the artifact in JSON format. | | `text` | `str` | Content of the artifact in TEXT format. | --- # Modules modules Triggers the regeneration of an artifact for a specified target node in Infrahub. The module looks up the artifact associated with the target node and triggers regeneration. ## Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ---------------- | ------ | -------- | ------- | ------------------------------------------------------------------------------ | | `api_endpoint` | `str` | No | | Endpoint of the Infrahub API, optional env=INFRAHUB\_ADDRESS | | `token` | `str` | No | | The API token created through Infrahub, optional env=INFRAHUB\_API\_TOKEN | | `timeout` | `int` | No | 10 | Timeout for Infrahub requests in seconds | | `artifact_name` | `str` | No | | Name of the artifact (mutually exclusive with artifact\_id) | | `artifact_id` | `str` | No | | UUID of the artifact (mutually exclusive with artifact\_name) | | `target_id` | `str` | Yes | | UUID of the target node (e.g., device ID) that the artifact is associated with | | `branch` | `str` | No | main | Branch in which the request is made | | `validate_certs` | `bool` | No | True | Whether to validate SSL of the Infrahub instance | | `state` | `str` | No | present | Use C(present) or C(absent) for adding or removing. | ## Examples[​](#examples "Direct link to Examples") ``` --- # Example 1: Regenerate artifact by name for a device - name: Regenerate artifact by name gather_facts: false hosts: localhost connection: local tasks: - name: Regenerate Startup Config for a device opsmill.infrahub.artifact_generate: artifact_name: "Startup Config" target_id: "{{ device_id }}" register: result - name: Display regeneration result ansible.builtin.debug: var: result --- # Example 2: Regenerate artifact by UUID - name: Regenerate artifact by ID gather_facts: false hosts: localhost tasks: - name: Regenerate specific artifact by ID opsmill.infrahub.artifact_generate: artifact_id: "12345678-1234-1234-1234-123456789abc" target_id: "{{ device_id }}" --- # Example 3: Using with Infrahub inventory plugin # Run with: ansible-playbook playbook.yml -i inventory.infrahub.yml -l "*edge*" - name: Regenerate artifacts using inventory host IDs gather_facts: false hosts: all connection: local tasks: - name: Regenerate Startup Config for each device opsmill.infrahub.artifact_generate: artifact_name: "Startup Config" target_id: "{{ id }}" register: result ``` ## Return values[​](#return-values "Direct link to Return values") | Key | Type | Description | | --------------- | ------ | ------------------------------------------------------------ | | `artifact_id` | `str` | UUID of the artifact that was regenerated | | `artifact_name` | `str` | Name of the artifact that was regenerated | | `definition_id` | `str` | UUID of the artifact definition | | `target_id` | `str` | UUID of the target node that the artifact is associated with | | `changed` | `bool` | Whether the artifact regeneration was triggered | | `msg` | `str` | Message indicating the result of the operation | --- # Modules modules Creates, Updates or Deletes a branch (InrahubBranch) in Infrahub through Infrahub SDK ## Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ---------------- | ------ | -------- | ------- | ------------------------------------------------------------------------- | | `api_endpoint` | `str` | No | | Endpoint of the Infrahub API, optional env=INFRAHUB\_ADDRESS | | `token` | `str` | No | | The API token created through Infrahub, optional env=INFRAHUB\_API\_TOKEN | | `timeout` | `int` | No | 10 | Timeout for Infrahub requests in seconds | | `name` | `str` | Yes | | Name of the branch to create | | `sync_with_git` | `bool` | No | False | Whether to sync the branch with git | | `description` | `str` | No | | Description of the branch | | `validate_certs` | `bool` | No | True | Whether or not to validate SSL of the Infrahub instance | | `state` | `str` | No | present | Use C(present) or C(absent) for adding or removing. | ## Examples[​](#examples "Direct link to Examples") ``` --- - name: Infrahub playbook for opsmill.infrahub.branch gather_facts: false hosts: localhost tasks: - name: Create a Branch 'test' opsmill.infrahub.branch: name: "test" sync_with_git: false description: "This is a test branch" state: present - name: Delete a Branch 'test' opsmill.infrahub.branch: name: "test" state: absent ``` ## Return values[​](#return-values "Direct link to Return values") | Key | Type | Description | | -------- | ------ | ----------------------------------------------------------------------- | | `object` | `dict` | Serialized Branch object as created or already existent within Infrahub | | `msg` | `str` | Message indicating failure or info about what has been achieved | --- # Inventory inventory Get inventory hosts from Infrahub. When strict is false, a compose, groups or keyed\_groups expression that fails to resolve emits one warning per distinct failure naming the affected hosts, instead of failing silently. ## Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ------------------------ | ------ | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `plugin` | \`\` | Yes | | token that ensures this is a source file for the 'opsmill.infrahub' plugin. | | `api_endpoint` | \`\` | Yes | | Endpoint of the Infrahub API | | `token` | \`\` | Yes | | Infrahub API token to be able to read against Infrahub. | | `timeout` | `int` | No | 60 | Timeout for Infrahub requests in seconds | | `prefetch_relationships` | `bool` | No | True | Prefetch relationships for Infrahub nodes | | `nodes` | `dict` | Yes | | Configuration for specific node types within Infrahub. Defines the attributes to include or exclude for each node. | | `branch` | `str` | No | main | Branch in which the request is made | | `compose` | `dict` | No | | List of custom Ansible host vars to create from the objects fetched from Infrahub | | `keyed_groups` | `list` | No | \[] | Create groups based on attributes or relationships. groups is created as `attribute__value` | | `groups` | `dict` | No | | Create groups based on jinja filter. | | `hostnames` | `list` | No | \[] | A list of attribute paths used to determine the inventory hostname. Each entry is a dotted path resolved against node attributes (e.g., "name", "primary\_address.address"). The special value "display\_label" resolves to the node's display label. First non-empty string value wins. Falls back to display\_label if none resolve. Referenced attributes must be present in the node's include list. | | `validate_certs` | \`\` | No | True | Whether or not to validate SSL of the Infrahub instance | ## Examples[​](#examples "Direct link to Examples") ``` # inventory.yml file in YAML format # Example command line: ansible-inventory -v --list -i .yml # Add -vvv to the command to also see the GraphQL query that gets sent in the debug output. # Add -vvvv to the command to also see the JSON response that comes back in the debug output. # Minimum required parameters plugin: opsmill.infrahub.inventory api_endpoint: http://localhost:8000 # Can be omitted if the INFRAHUB_ADDRESS environment variable is set token: 1234567890123456478901234567 # Can be omitted if the INFRAHUB_API_TOKEN environment variable is set # Complete Example # This will : # - Retrieve in the branch "branch1" attributes for the Node Kind "InfraDevice" # - The attributes wanted for "InfraDevice" are forced with the keyword "include" # - Create 2 compose variable "hostname" ad "platform" (platform will override the attribute platform retrieved) # - Create group based on the "site" name strict: true branch: "branch1" nodes: InfraDevice: include: - name - platform.ansible_network_os - primary_address.address - site.name - interfaces compose: hostname: name platform: platform.ansible_network_os keyed_groups: - prefix: site key: site.name # Using hostnames to set clean inventory hostnames plugin: opsmill.infrahub.inventory api_endpoint: http://localhost:8000 hostnames: - name - display_label nodes: InfraDevice: include: - name - primary_address.address ``` ## Return values[​](#return-values "Direct link to Return values") | Key | Type | Description | | ------- | ------ | ------------------------------------------------ | | `_list` | `list` | list of composed dictionaries with key and value | --- # Lookup lookup Get inventory hosts from Infrahub ## Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ----------------- | ------ | -------- | ------- | -------------------------------------------------------- | | `api_endpoint` | \`\` | Yes | | Endpoint of the Infrahub API | | `token` | \`\` | Yes | | Infrahub API token to be able to read against Infrahub. | | `timeout` | `int` | No | 10 | Timeout for Infrahub requests in seconds | | `query` | `str` | Yes | | GraphQL query to send to Infrahub to obtain desired data | | `graph_variables` | `dict` | No | | dictionary of keys/values to pass into the GraphQL query | | `branch` | `str` | No | main | Branch in which the request is made | | `validate_certs` | \`\` | No | True | Whether or not to validate SSL of the Infrahub instance | ## Examples[​](#examples "Direct link to Examples") ``` - name: Infrahub lookup gather_facts: false hosts: localhost tasks: - name: SET FACT OF STRING ansible.builtin.set_fact: query_string: | query { BuiltinLocation { edges { node { name { value } } } } } - name: Obtain list of sites from Infrahub ansible.builtin.set_fact: query_response: "{{ query('opsmill.infrahub.lookup', query=query_string) }}" - name: Print result ansible.builtin.debug: msg: "{{ query_response }}" ``` ## Return values[​](#return-values "Direct link to Return values") | Key | Type | Description | | ------ | ------ | ---------------------------------------------- | | `data` | `dict` | Data result from the Infrahub GraphQL endpoint | --- # Modules modules Creates, Updates or Deletes a node of a given Kind in Infrahub through Infrahub SDK ## Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ---------------- | ------ | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `api_endpoint` | `str` | No | | Endpoint of the Infrahub API, optional env=INFRAHUB\_ADDRESS | | `token` | `str` | No | | The API token created through Infrahub, optional env=INFRAHUB\_API\_TOKEN | | `timeout` | `int` | No | 10 | Timeout for Infrahub requests in seconds | | `kind` | `str` | Yes | | Kind of node to create | | `data` | `raw` | Yes | | Dictionary of node attributes | | `branch` | `str` | No | main | Branch in which the request is made | | `validate_certs` | `bool` | No | True | Whether or not to validate SSL of the Infrahub instance | | `state` | `str` | No | present | Use C(present) or C(absent) for adding or removing. | | `file_path` | `str` | No | None | Local filesystem path to the file to upload when creating or updating a CoreFileObject node. Required when the kind inherits from CoreFileObject (unless C(fetch\_file) is used instead). Mutually exclusive with C(fetch\_file). When provided, the module computes a SHA-1 checksum and skips the upload if it matches the server-side checksum (idempotent). Fails if the kind does not inherit from CoreFileObject. Ignored in check mode (no upload performed). | | `fetch_file` | `bool` | No | False | When C(true), download the file content from the CoreFileObject node and include it in the result as C(binary) (base64-encoded) and C(text) (UTF-8 decoded for text MIME types, null otherwise). Required when the kind inherits from CoreFileObject and C(file\_path) is not provided. Mutually exclusive with C(file\_path). Ignored in check mode (no download performed). Fails if the kind does not inherit from CoreFileObject. | ## Examples[​](#examples "Direct link to Examples") ``` --- - name: Infrahub playbook for opsmill.infrahub.node gather_facts: false hosts: localhost tasks: - name: Create tag1 opsmill.infrahub.node: kind: "BuiltinTag" data: name: "tag1" state: present - name: Delete tag1 opsmill.infrahub.node: kind: "BuiltinTag" data: name: "tag1" state: absent ``` ## Return values[​](#return-values "Direct link to Return values") | Key | Type | Description | | -------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `object` | `dict` | Serialized object as created or already existent within Infrahub | | `msg` | `str` | Message indicating failure or info about what has been achieved | | `binary` | `str` | Base64-encoded file content downloaded from the CoreFileObject node. Present only when I(fetch\_file=true) and not in check mode. | | `text` | `str` | UTF-8 decoded file content for text MIME types (text/plain, application/json, etc.). null for binary MIME types. Present only when I(fetch\_file=true) and not in check mode. | --- # Modules modules Downloads the binary file content stored on a CoreFileObject schema node. Identifies the node by UUID (node\_id) or HFID (hfid). Optionally saves the file to a local destination path. Returns base64-encoded binary content and metadata regardless of I(dest). ## Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ---------------- | ------ | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `api_endpoint` | `str` | No | | Endpoint of the Infrahub API, optional env=INFRAHUB\_ADDRESS | | `token` | `str` | No | | The API token created through Infrahub, optional env=INFRAHUB\_API\_TOKEN | | `timeout` | `int` | No | 10 | Timeout for Infrahub requests in seconds | | `validate_certs` | `bool` | No | True | Whether or not to validate SSL of the Infrahub instance | | `branch` | `str` | No | main | Branch in which the request is made | | `kind` | `str` | Yes | | Schema kind that inherits from CoreFileObject (e.g. C(NetworkCircuitContract)) | | `node_id` | `str` | No | | UUID of the CoreFileObject node to fetch. One of I(node\_id) or I(hfid) is required. | | `hfid` | `list` | No | | Human-friendly ID component values for the CoreFileObject node. One of I(node\_id) or I(hfid) is required. | | `dest` | `str` | No | None | Local path to save the file content. When a directory path is given (trailing slash or existing directory), the file is saved as `{dest}/{node.file_name}`. When a file path is given, the file is saved exactly at that path. When omitted, file content is returned as variables only. | ## Examples[​](#examples "Direct link to Examples") ``` --- - name: Fetch contract PDF by node UUID opsmill.infrahub.object_file_fetch: kind: NetworkCircuitContract node_id: "abc123-uuid" register: fetch_result - name: Write fetched file to disk ansible.builtin.copy: content: "{{ fetch_result.binary | b64decode }}" dest: /tmp/contract.pdf - name: Fetch and save to directory by HFID opsmill.infrahub.object_file_fetch: kind: NetworkCircuitContract hfid: - "contract.pdf" dest: /tmp/contracts/ register: fetch_result # File saved to /tmp/contracts/contract.pdf # fetch_result.dest == "/tmp/contracts/contract.pdf" - name: Fetch and save to explicit file path opsmill.infrahub.object_file_fetch: kind: NetworkCircuitContract node_id: "abc123-uuid" dest: /tmp/my-contract.pdf register: fetch_result # File saved to exactly /tmp/my-contract.pdf ``` ## Return values[​](#return-values "Direct link to Return values") | Key | Type | Description | | ----------- | ----- | ---------------------------------------------------------------------------------------------------------------- | | `binary` | `str` | Base64-encoded file content downloaded from the CoreFileObject node. | | `text` | `str` | UTF-8 decoded file content for text MIME types (text/plain, application/json, etc.). null for binary MIME types. | | `file_name` | `str` | Original filename as stored in Infrahub. | | `file_type` | `str` | MIME type of the file as detected by Infrahub. | | `file_size` | `int` | Size of the file in bytes. | | `checksum` | `str` | SHA-1 hex digest of the file content as stored in Infrahub. | | `node_id` | `str` | UUID of the fetched CoreFileObject node. | | `dest` | `str` | Resolved local path where the file was saved. null if I(dest) was not provided. | | `msg` | `str` | Status message. | --- # Modules modules Queries Infrahub via its GraphQL API through Infrahub SDK ## Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ----------------- | ------ | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `api_endpoint` | `str` | No | | Endpoint of the Infrahub API, optional env=INFRAHUB\_ADDRESS | | `token` | `str` | No | | The API token created through Infrahub, optional env=INFRAHUB\_API\_TOKEN | | `timeout` | `int` | No | 10 | Timeout for Infrahub requests in seconds | | `query` | `str` | Yes | | GraphQL query parameters or filters to send to Infrahub to obtain desired data | | `graph_variables` | `dict` | No | | Dictionary of keys/values to pass into the GraphQL query | | `branch` | `str` | No | main | Branch in which the request is made | | `validate_certs` | `bool` | No | True | Whether or not to validate SSL of the Infrahub instance | | `update_hostvars` | `bool` | No | False | Whether or not to populate data in the in the root (example hostvars\[inventory\_hostname]) or within the 'data' key (example `hostvars[inventory_hostname]['data']`). Beware, that the root keys provided by the query will overwrite any root keys already present, leverage the GraphQL alias feature to avoid issues. | ## Examples[​](#examples "Direct link to Examples") ``` --- - name: Infrahub action plugin query_graphql gather_facts: false hosts: localhost tasks: - name: SET FACTS TO SEND TO GRAPHQL ENDPOINT ansible.builtin.set_fact: variables: device_name: "atl1-edge1" enabled: true query_dict: InfraDevice: '@filters': {name__value: '$device_name'} edges: node: name: value: null interfaces: '@filters': {enabled__value: '$enabled'} edges: node: name: value: null - name: Action Plugin opsmill.infrahub.query_graphql: query: "{{ query_dict }}" graph_variables: "{{ variables }}" ``` ## Return values[​](#return-values "Direct link to Return values") | Key | Type | Description | | ------ | ------ | ---------------------------------------------- | | `data` | `dict` | Data result from the Infrahub GraphQL endpoint | --- # Modules modules Load, check, or export schemas in Infrahub through the Infrahub SDK. Use C(action=load) to load schemas into Infrahub. Use C(action=check) to validate schemas without applying them. Use C(action=export) to export existing schemas from Infrahub. ## Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ---------------------- | ------ | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `api_endpoint` | `str` | No | | Endpoint of the Infrahub API, optional env=INFRAHUB\_ADDRESS | | `token` | `str` | No | | The API token created through Infrahub, optional env=INFRAHUB\_API\_TOKEN | | `timeout` | `int` | No | 10 | Timeout for Infrahub requests in seconds | | `validate_certs` | `bool` | No | True | Whether or not to validate SSL of the Infrahub instance | | `action` | `str` | Yes | | The schema action to perform. C(load) loads schemas into Infrahub. C(check) validates schemas without applying them. C(export) exports existing schemas from Infrahub. | | `branch` | `str` | No | main | Branch in which the request is made | | `schemas` | `list` | No | | List of inline schema definitions (nodes and generics). For C(load) and C(check), at least one of C(schemas) or C(schema\_files) must be provided. | | `schema_files` | `list` | No | | List of YAML file paths containing schema definitions. Files are read on the Ansible controller. For C(load) and C(check), at least one of C(schemas) or C(schema\_files) must be provided. | | `namespaces` | `list` | No | | List of namespace names to filter the export. Only used with C(action=export). | | `wait_until_converged` | `bool` | No | False | Wait for schema to be synchronized across all workers. Only used with C(action=load). | ## Examples[​](#examples "Direct link to Examples") ``` --- - name: Check schema from inline definition opsmill.infrahub.schema: action: check schemas: - name: Site namespace: Location attributes: - name: name kind: Text unique: true - name: Load schema from file opsmill.infrahub.schema: action: load schema_files: - "schemas/my_schema.yml" - name: Load schema with convergence wait opsmill.infrahub.schema: action: load schema_files: - "schemas/my_schema.yml" wait_until_converged: true - name: Export all schemas opsmill.infrahub.schema: action: export register: result - name: Export schemas for specific namespaces opsmill.infrahub.schema: action: export namespaces: - Infra - Location register: result ``` ## Return values[​](#return-values "Direct link to Return values") | Key | Type | Description | | ---------------- | ------ | --------------------------------------------------------------------- | | `changed` | `bool` | Whether the schema was updated (load) or always false (check/export). | | `schema_updated` | `bool` | Whether the schema hash changed after loading. | | `hash` | `str` | New schema hash after loading. | | `previous_hash` | `str` | Previous schema hash before loading. | | `warnings` | `list` | Schema warnings returned during load. | | `valid` | `bool` | Whether the schema passed validation. | | `errors` | `dict` | Validation errors when schema check fails. | | `schemas` | `dict` | Exported schemas organized by namespace. | | `msg` | `str` | Message indicating the result of the operation. | --- The **Infrahub Installation Role** in the **OpsMill Infrahub Ansible Collection** allows you to install and configure Infrahub using `Docker` and `systemd`. # Overview This role automates the installation and configuration of Infrahub by: * Setting up the required directory structure * Configuring Docker Compose * Installing and configuring `systemd` services * Managing Infrahub configuration # Requirements This role requires: * Docker Engine with Docker Compose v2 to be installed * Root or `sudo` access for `systemd` service installation (on the target server) # Role variables | Variable | Type | Default | Description | | ---------------------------------------- | ------ | ----------------------------- | ---------------------------------------------------------------------------------------------------------- | | `install_infrahub_version` | `str` | | Version of Infrahub to install. Can be any Docker image tag name. | | `install_infrahub_url` | `str` | `https://infrahub.opsmill.io` | URL from where to fetch the Infrahub Docker Compose file. | | `install_infrahub_install_directory` | `str` | `/opt/infrahub` | Install directory for the Infrahub files (Docker Compose and configuration file). | | `install_infrahub_config` | `dict` | | Environment variables to pass as configuration for Infrahub. | | `install_infrahub_docker_project` | `str` | `infrahub` | Docker project name to use when starting Infrahub. | | `install_infrahub_docker_pull_images` | `bool` | `true` | Whether to pull the required Docker images. | | `install_infrahub_setup_systemd` | `bool` | `true` | Whether to install the `systemd` service for Infrahub. | | `install_infrahub_systemd_directory` | `str` | `/etc/systemd/system/` | Where to install the `systemd` service unit file. | | `install_infrahub_systemd_service_state` | `str` | `restarted` | Target state of the `systemd` service. Can be used to avoid starting Infrahub during the role's execution. | # Installation example * Quick start * Production setup For a basic installation with default settings: install\_infrahub\_minimal.yml ``` - name: Install Infrahub hosts: infrahub_servers become: true roles: - role: opsmill.infrahub.install vars: install_infrahub_version: "latest" ``` ## Production setup[​](#production-setup "Direct link to Production setup") For a production environment with custom configuration: install\_infrahub\_advanced.yml ``` - name: Install Infrahub with custom configuration hosts: infrahub_servers become: true roles: - role: opsmill.infrahub.install vars: install_infrahub_version: "2.0.0" install_infrahub_install_directory: "/usr/local/infrahub" install_infrahub_docker_project: "infrahub_prod" install_infrahub_docker_pull_images: true install_infrahub_setup_systemd: true install_infrahub_systemd_service_state: "started" infrahub_config: INFRAHUB_ADMIN_EMAIL: "admin@example.com" INFRAHUB_ADMIN_PASSWORD: "secure_password" INFRAHUB_DATABASE_URL: "postgresql://user:pass@db:5432/infrahub" ``` # Usage To install Infrahub, use: ``` ansible-playbook install_infrahub.yml -i inventory.yml ``` --- # Arista AVD Reference Design This reference design models Arista datacenter fabrics in Infrahub and generates EOS device configurations, per-device documentation, and ANTA test catalogs via the AVD pipeline. All changes run through Infrahub's branching and proposed-change workflow. If you are new to Infrahub, start with the [Infrahub documentation](https://docs.infrahub.app) to understand branches, proposed changes, generators, and artifacts before working through the guides here. Six stages take a high-level fabric design to versioned, deployable configuration. Infrahub owns the data, orchestration, and version control; PyAVD runs natively inside it to generate the structured configuration and documentation. ## What it's for[​](#what-its-for "Direct link to What it's for") * **Generate a complete fabric from a design** — define topology parameters and addressing pools; generators create all super-spines, spines, and leaves, allocate loopback, interconnect, and management addresses, BGP ASNs, and node IDs, and cable devices together automatically. * **Render EOS device configurations and documentation** — PyAVD runs inside Infrahub workers and produces EOS CLI configurations, per-device and fabric-level Markdown documentation, and a cabling plan CSV as downloadable artifacts. * **Make incremental day-two changes** — edit the design and regenerate; checksum-based idempotency applies changes only to affected objects; branch-aware pools prevent collisions across parallel work. * **Give other teams access to network data** — the fabric is queryable through the Infrahub Web UI, GraphQL API, and MCP interface; the Streamlit service portal provides guided workflows for stakeholders without API or CLI access. * **Track and review every change** — all changes run through Infrahub branches and proposed changes, with a full diff before any change reaches a device. ## How to use it[​](#how-to-use-it "Direct link to How to use it") ### Provision a fabric[​](#provision-a-fabric "Direct link to Provision a fabric") * Create a NetworkFabric with pods and racks; set device counts and assign addressing pool ranges. * Run FabricGenerator from the Infrahub UI — PodGenerator and RackGenerator trigger automatically from event rules. * The generator chain creates all devices, allocates addresses, ASNs, and node IDs, and cables them together. * Run the AVD generators to produce per-device host\_vars and structured configuration. * Render EOS artifacts through the transforms, or open a proposed change — the CI pipeline renders them for every device at once. ### Operate day-two through the service portal[​](#operate-day-two-through-the-service-portal "Direct link to Operate day-two through the service portal") * Add a network segment — create a VRF, VLAN, and SVI on a target fabric. * Provision a server into a compute rack. * Create an EVPN tenant with a VNI base allocation across one or more fabrics. * Each operation creates a branch and opens a proposed change for review before anything reaches production. ### Query and access through the API and MCP[​](#query-and-access-through-the-api-and-mcp "Direct link to Query and access through the API and MCP") * **GraphQL API** — query devices, addresses, configurations, and topology programmatically. * **MCP interface** — AI assistant access to all fabric data. * **Infrahub Web UI** — searchable, filterable views for stakeholders without Git or Python access. ## Who it's for[​](#who-its-for "Direct link to Who it's for") * **Network automation teams running AVD with static variable files** — add a source of truth, API and UI layer, and branch-based change control on top of an existing AVD workflow. → [Provision Your First Fabric](/arista-avd/provision-first-fabric.md) * **Teams evaluating how to operate AVD at scale** — the pipeline derives per-device host\_vars from the source of truth; no separate inventory files are required. → [Quick Start](/arista-avd/quick-start.md) * **Contributors extending the pipeline** — add new device roles, schema fields, or transform outputs; the developer guide covers the full chain, role mapping, and concrete examples. → [Developer Guide](/arista-avd/developer-guide.md) ## What's included[​](#whats-included "Direct link to What's included") * **Schemas** — the source-of-truth definition. Specifies what data Infrahub stores, how it relates, and what generators and transforms can read. * Topology: Fabric → Pod → Rack → Device hierarchy * IPAM: prefixes and addresses with role tagging (loopback, interconnect, management, server) * EVPN: VRFs, SVIs, L2 VLANs; MLAG: domain and peer pool definitions * AVD types: `AvdArtifact` for per-device host\_var and structured-config tracking with checksums * **Generators** — the automation layer. Define topology parameters and pool ranges; generators derive the full fabric from that design intent. All are checksum-based and idempotent. * FabricGenerator, PodGenerator, RackGenerator — create devices, allocate addresses, assign BGP ASNs and node IDs, cable devices together * GenerateAVDDeviceHostvar — assembles per-device PyAVD input from the source of truth * AvdDeviceStructuredConfigGenerator — runs PyAVD to produce structured configuration * GenerateServerCabling — handles server attachment * **Transforms** — the rendering layer. Reads structured data from generators and outputs downloadable artifacts. PyAVD runs inside Infrahub workers. * EOS device configurations * Per-device and fabric-level Markdown documentation * Cabling plan CSV * ANTA test catalogs (generation is included; test execution on the roadmap) * Computed interface descriptions * **Seed data** — a ready-to-run starting point. `invoke load` populates Infrahub immediately with manufacturers, device types, device profiles and templates, addressing and number pools, and two example fabrics with pods, racks, and seed VLANs. * **Service portal** — a Streamlit application for self-service day-2 operations. Every operation creates a branch and opens a proposed change for review. * Add a network segment (VRF, VLAN, SVI) * Provision a server into a rack * Create an EVPN tenant * Fabric Design visualization (topology, cabling, settings, EVPN tenants) * **Stack** — Docker Compose bundling everything needed to run locally: Infrahub with PyAVD, the service portal, a bundled Ansible runner for device deployment, and Neo4j. ## Best practices[​](#best-practices "Direct link to Best practices") * **Work on branches.** Create a named branch for each change set. The generator chain and service portal both operate on branches; proposed changes give you a diff before anything merges. * **Run `invoke load` in order.** The load sequence is ordered: schemas, then menu, then seed data, then repository registration, then triggers. Running steps out of order or skipping `uv sync` first is the most common cause of load failures. * **Re-run generators idempotently.** All generators use checksum-based change detection — re-running after a partial failure is safe and applies only what changed. * **Use the service portal for repeatable day-two operations.** The portal wraps generator calls and branch creation into guided workflows with validation. For one-off changes, the Infrahub UI and GraphQL API work directly; use the portal for provisioning workflows run by team members without API or CLI access. * **Scope to supported capabilities.** This reference design covers a defined set of AVD capabilities — uncommon or highly custom options may not be modeled. Review the [Supported Capabilities](/arista-avd/supported-capabilities.md) page before planning a deployment. ## Get started[​](#get-started "Direct link to Get started") **Prerequisites:** Docker and Docker Compose · uv · Python 3.11+ 1. Clone the repository and run `uv sync --all-packages` to install dependencies. 2. Build the custom Infrahub image: `uv run invoke build` (one-time). 3. Start the stack: `uv run invoke start`. 4. Load schemas, seed data, and the repository: `uv run invoke load`. 5. Follow [Provision Your First Fabric](/arista-avd/provision-first-fabric.md) to generate a fabric and reach rendered EOS artifacts. ## Additional resources[​](#additional-resources "Direct link to Additional resources") | Goal | Guide | | ------------------------------- | --------------------------------------------------------------------------------------------------------------- | | Get the stack running | [Quick Start](/arista-avd/quick-start.md) — prerequisites, install steps, and first load | | Generate your first fabric | [Provision Your First Fabric](/arista-avd/provision-first-fabric.md) — end-to-end walkthrough | | Check what's supported | [Supported Capabilities](/arista-avd/supported-capabilities.md) — capability matrix | | Understand how it's built | [Architecture Overview](/arista-avd/developer-guide/architecture.md) — system components and generator pipeline | | Extend the pipeline | [Extending the Pipeline](/arista-avd/developer-guide/avd/extending.md) — new roles, transforms, schema fields | | Find solutions to common issues | [Troubleshooting](/arista-avd/troubleshooting.md) | --- # CloudVision validation The repository validates generated EOS configurations in CloudVision during Infrahub proposed-change validation. It also records the CloudVision workspace URL in the proposed change and registers a placeholder CustomWebhook handoff for submitting the linked workspace when the proposed change is submitted. ## Runtime configuration[​](#runtime-configuration "Direct link to Runtime configuration") CloudVision credentials are read from task-worker environment variables: ``` CLOUDVISION_SERVERS=cv.example.com CLOUDVISION_TOKEN=... CLOUDVISION_VERIFY_CERTS=true ``` Username/password authentication is also supported with `CLOUDVISION_USERNAME` and `CLOUDVISION_PASSWORD`. Optional proxy settings use the `CLOUDVISION_PROXY_*` variables. `docker-compose.override.yml` passes these variables into the Infrahub task worker so proposed-change checks can access them. ## Proposed-change validation[​](#proposed-change-validation "Direct link to Proposed-change validation") The `cv-config-validation` check uses the `cv_config_check` GraphQL query to collect the target fabric and related devices. The fabric must have `cloudvision_managed` set to `true` before CloudVision validation runs. Unmanaged fabrics skip CloudVision credential setup, serial-number checks, inventory checks, and workspace validation. For a managed fabric, validation first authenticates to CloudVision, then requires every confirmed device in the fabric to have a serial number and to exist in CloudVision inventory. Devices outside the target fabric are ignored, and missing optional relationships are treated as absent membership rather than runtime failures. Inventory-confirmed devices must also be active in CloudVision; inactive targeted devices fail validation even if the workspace build itself succeeds. After eligibility passes, only devices with generated structured-config artifacts are deployed to the validation workspace. If no generated structured-config artifacts exist for an otherwise eligible managed fabric, the check records an informational skip and does not create or build a workspace. For each selected device, the check downloads the `AvdStructuredConfigFile`, renders EOS CLI with `pyavd.get_device_config()`, deploys the configs to a CloudVision workspace, and requests a workspace build. Download, JSON decode, or render failures block the proposed change with a device-specific error. CloudVision connection, deployment, or workspace build failures also block the proposed change and include the fabric and workspace context when available. ## Workspace tracking[​](#workspace-tracking "Direct link to Workspace tracking") Successful validation creates or updates a deterministic CloudVision workspace for the proposed change and target fabric. If the workspace already exists and is not pending, the check returns it to a pending state before deploying the latest configs and requesting a build. When the tracking schema is loaded, validation also creates or updates a `CloudvisionWorkspace` object in Infrahub. The object tracks: * `name`: workspace display name * `workspace_id`: deterministic CloudVision workspace ID * `proposed_change_id`: proposed change that created the workspace * `workspace_url`: exact CloudVision workspace URL shown to reviewers * `thread_id`: proposed-change overview thread used for workspace comments * `change_control_id` / `change_control_url`: CloudVision change control created during submission, when one exists * `last_submission_error` / `last_submission_attempt_at`: latest failed CustomWebhook submission attempt * `submitted_at`: successful submission timestamp * `status`: `pending`, `built`, `submitted`, `abandoned`, or `submit_failed` * `fabric`: fabric validated by the workspace The workspace ID is deterministic from proposed-change ID and fabric name, so a validation rerun updates the same CloudVision workspace instead of creating a new one. Separate proposed changes against the same fabric receive different workspace IDs. After a workspace is created or reused, validation creates or reuses one deterministic `CoreChangeThread` in the proposed-change Overview. The first comment contains the exact workspace URL. Rerunning validation for the same proposed change and workspace reuses the stored `thread_id` or deterministic thread label and does not duplicate the URL comment. The CloudVision workspace display name uses the proposed-change name and fabric name. Its description uses the proposed-change description, with a generic Infrahub validation description when the proposed change has no description. When the check context does not include full proposed-change metadata, the check looks up the open proposed change by source branch and also tries the short branch name for `feat/` branches. If the `CloudvisionWorkspace` schema is unavailable during rollout, tracking is skipped without masking CloudVision validation success or failure. ## CustomWebhook submission[​](#customwebhook-submission "Direct link to CustomWebhook submission") The repository loads exactly one placeholder `CoreCustomWebhook` named `cloudvision-workspace-submission`. It is associated with proposed-change merge for the `cv-config-validation` workflow, references the `cv-workspace-submission-webhook-payload` `CoreTransformPython`, and uses this explicitly non-production URL: ``` https://placeholder.invalid/cloudvision-workspace-submission ``` The placeholder is a repository-loadable handoff marker for this phase. No real external automation receiver is required, and the placeholder URL is not a production deployment endpoint. The CustomWebhook processing entry point is `submit_linked_workspace_for_custom_webhook()` in `checks/cv_workspace_lifecycle.py`. It extracts the proposed-change ID and branch from the submitted proposed-change event, ignores events that explicitly identify a check other than `cv-config-validation`, and calls the shared submission handler. The shared submission handler is `submit_linked_workspace_for_proposed_change()` in `checks/cv_workspace_lifecycle.py`. It resolves `CloudvisionWorkspace` objects by the submitted proposed-change ID on the selected branch and submits only when exactly one linked workspace exists and is in a submit-ready state. On success, the handler appends a comment to the existing workspace thread with the CloudVision workspace identity and URL when available. The thread is marked resolved only after the success or already-complete comment is saved. Already-submitted workspaces are treated as complete and are not submitted again. On failure, the handler stores `status=submit_failed`, records the latest failure reason, appends an unresolved failure comment when possible, and leaves the thread unresolved. If thread or comment writes fail, the handler logs the same proposed-change, workspace, fabric, and error context as the fallback. If no linked workspace exists or multiple workspaces are linked ambiguously, the handler creates a proposed-change submission outcome thread when possible and records the skip or ambiguity there. Manual retry uses the invoke task: ``` uv run invoke submit-cv-workspace --proposed-change-id --branch main ``` ## Operational notes[​](#operational-notes "Direct link to Operational notes") CloudVision validation depends on the AVD generator chain having already produced structured-config artifacts. Missing artifacts do not exempt devices from managed-fabric serial-number or inventory eligibility; they only remove the device from workspace config deployment after eligibility succeeds. CloudVision build or EOS validation failures should be handled as data fixes first. Add schema or generator code only when a required configuration family cannot be represented with the existing model. The validation check builds workspaces for review only. CloudVision submission is handled by CustomWebhook processing or the manual retry task, not by the pre-merge validation check. CloudVision change-control management and Semaphore Ansible playbooks are out of scope for this phase. The CustomWebhook is only the handoff point for future deployment automation after linked workspace submission. ## Related[​](#related "Direct link to Related") * [Checks](/arista-avd/developer-guide/checks.md) — how the check is wired, its query and supporting modules, and how to run or extend it. * [Schemas → CloudvisionWorkspace](/arista-avd/developer-guide/schemas.md#cloudvisionworkspace--cloudvisionworkspace) — the tracking node's attributes and relationships. --- # ContainerLab [ContainerLab](https://containerlab.dev) runs a fabric as containers — Arista cEOS nodes for the network devices, plain Linux containers for the servers. The repository renders a topology file for any modelled fabric as an Infrahub artifact, so a virtual replica of the fabric can be brought up from the same source of truth that produces the EOS configs. Two flows exist, and they are separate on purpose: * **The committed lab** in `lab/` — a two-DC topology checked into the repository, booting from AVD renders that are also checked in. This is the day-to-day lab; see `lab/README.md`. * **The generated topology** — the `ContainerLab Topology` artifact described here, rendered per fabric from Infrahub data. Use this for fabrics the committed lab does not cover. ## The ContainerLab topology artifact[​](#the-containerlab-topology-artifact "Direct link to The ContainerLab topology artifact") | Property | Value | | ------------- | --------------------------------------------- | | Artifact name | `ContainerLab Topology` | | Attached to | Each `NetworkFabric` (target group `fabrics`) | | Content type | `application/yaml` | | Transform | `containerlab_topology` | The artifact is **fabric-scoped**: one topology per fabric, containing every device the fabric owns through its pods and racks. The rendered `name:` is the fabric name, so the management network and container names are derived from it (`clab--mgmt`). Find it on the fabric's **Artifacts** tab in the Infrahub UI, the same way as the fabric documentation — see [Viewing Artifacts](/arista-avd/viewing-artifacts.md). To preview a render locally without going through a proposed change: ``` # COLUMNS is set because infrahubctl prints via Rich, which wraps long lines at the # terminal width — irrelevant to the server-rendered artifact, but not to a local capture. COLUMNS=500 uv run infrahubctl transform containerlab_topology name=Fabric-L3LS-Multi-Domain ``` ## What ends up in the topology[​](#what-ends-up-in-the-topology "Direct link to What ends up in the topology") ``` --- name: Fabric-L3LS-Multi-Domain mgmt: network: clab-Fabric-L3LS-Multi-Domain-mgmt ipv4-subnet: 10.0.6.0/24 topology: kinds: arista_ceos: image: arista/ceos:4.36.0.1F startup-config: configs/__clabNodeName__.cfg linux: image: lab-server nodes: spine-infrahub-dc1-1: kind: arista_ceos mgmt-ipv4: 10.0.6.11 binds: - configs/eos-intf-mapping/DCS-7050CX3-32S.json:/mnt/flash/EosIntfMapping.json:ro dc1-server: kind: linux mgmt-ipv4: 10.0.6.100 binds: - configs/servers/dc1-server-netplan.yaml:/etc/netplan/netplan.yaml links: - endpoints: ["leaf-infrahub-dc1-1:eth49_1", "spine-infrahub-dc1-1:eth1_1"] ``` Nodes and links are emitted in a stable sorted order, and endpoints are ordered within each link, so two renders of unchanged data are byte-identical. `startup-config` is set only on kinds whose nodes are network devices — the `linux` kind has none, because servers boot from their netplan bind. The path is `configs/__clabNodeName__.cfg`, the directory the deploy playbook writes fetched EOS configs into. ### Which devices are included[​](#which-devices-are-included "Direct link to Which devices are included") Devices are selected by `DcimDevice.role`. Included: `super_spine`, `spine`, `leaf`, `border_leaf`, `l2leaf`, `l2spine`, `l3spine`. `ComputePhysicalServer` members of the fabric are included as Linux nodes. The `p`, `pe`, and `rr` roles are deliberately **excluded**. They belong to the ISIS-LDP fabric, whose interface naming has not been validated against ContainerLab, so admitting them would be speculative. Each excluded device is logged as a warning during the render rather than dropped silently, so `Fabric-ISIS-LDP` renders without those devices and says so in the transform log. A link is only emitted when it resolves to exactly two endpoints and both endpoints belong to devices present in `nodes`. ## Kind, image, and interface mapping come from the schema[​](#kind-image-and-interface-mapping-come-from-the-schema "Direct link to Kind, image, and interface mapping come from the schema") Nothing about node identity is hardcoded in the transform. Three schema attributes drive it: | Attribute | Node | Drives | | ----------------------------------------------- | ------------------------------ | ------------------------------------- | | `DcimPlatform.containerlab_os` | `kind:` on each node | `arista_ceos`, `linux` | | `DcimPlatform.containerlab_image` | `image:` on each `kinds` entry | `arista/ceos:4.36.0.1F`, `lab-server` | | `DcimDeviceType.containerlab_interface_mapping` | the `EosIntfMapping.json` bind | `DCS-7050CX3-32S.json` | All three are optional `Text` attributes. A device whose platform has no `containerlab_os` cannot be rendered as a node; a device type with no `containerlab_interface_mapping` gets no mapping bind (the `binds` key is omitted entirely when a node has nothing to bind). `containerlab_interface_mapping` holds a **filename only**, not a path, and not the file contents. The file itself lives in `lab/configs/eos-intf-mapping/` and is resolved relative to the topology file at deploy time. Note the filenames intentionally differ from the device type's `part_number` (`DCS-7050CX3-32S.json` for part number `DCS-7050CX3-32C`) — the attribute exists precisely so the mapping filename does not have to be derived from anything else. Seed values live in `objects/03_device_type.yml`. ## Interface names and why the mapping bind matters[​](#interface-names-and-why-the-mapping-bind-matters "Direct link to Interface names and why the mapping bind matters") Link endpoints are translated from EOS names to the Linux interface names cEOS exposes to ContainerLab: | EOS name | ContainerLab name | | -------------- | ----------------- | | `Ethernet5` | `eth5` | | `Ethernet1/1` | `eth1_1` | | `Ethernet49/1` | `eth49_1` | The rule is `Ethernet[/]` → `eth[_]`: strip the `Ethernet` prefix and replace `/` with `_`. For plain `Ethernet` interfaces that is all cEOS needs — it maps `ethN` to `EthernetN` by default. Breakout names are the problem. A generated config that says `interface Ethernet1/1` does not attach to anything if cEOS has decided that `eth1_1` is `Ethernet1_1`, or has not created the interface at all. `EosIntfMapping.json` is the file that tells cEOS which container interface corresponds to which EOS interface name, per device type. It is mounted read-only: ``` binds: - configs/eos-intf-mapping/DCS-7050SX3-48YC8.json:/mnt/flash/EosIntfMapping.json:ro ``` On a fabric whose spines use `Ethernet/1` uplinks and whose leaves use `Ethernet49-50/1`, the mapping bind is what makes the AVD-rendered config match the interfaces that actually exist. To confirm it took effect after a deploy: ``` docker exec clab-- Cli -c "show interfaces status" | head -20 ``` Expect `Ethernet1/1`-style names. Seeing `eth1_1` instead means the bind is missing, or points at a file that is not on disk next to the topology. Server nodes carry a netplan bind instead, mounted at `/etc/netplan/netplan.yaml`. The source filename is derived by convention from the device name (`configs/servers/-netplan.yaml`); netplan contents are not generated from Infrahub. ## Deploying with Ansible[​](#deploying-with-ansible "Direct link to Deploying with Ansible") `ansible/deploy_clab.yml` fetches the topology artifact and each device's `AVD EOS Configuration` artifact from Infrahub, stages them plus every committed bind source onto a ContainerLab host, and runs `containerlab deploy` there. It lives in `ansible/` because that directory is also the Semaphore playbook repository. The playbook is two plays, because the machine that talks to Infrahub is not necessarily the machine that runs the lab: | Play | Hosts | Does | | ---- | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | `localhost` | Resolves the fabric to a node ID, fetches the topology and per-device config artifacts, asserts each returned a body | | 2 | `clab_hosts` | Stages the topology, configs, and bind sources onto the lab host, validates every bind and `startup-config` path exists **there**, then deploys | Nothing is written to the controller's filesystem — the artifacts are copied straight to the lab host, so no shared filesystem is assumed. Point `clab_hosts` at whichever host runs ContainerLab, in `ansible/inventory_clab.yml`. It defaults to `localhost` with `ansible_connection: local`, so a single-machine setup works unchanged. This inventory must be passed explicitly, because `ansible/ansible.cfg` pins `inventory` to the dynamic Infrahub plugin. Required environment: ``` export INFRAHUB_ADDRESS=http://localhost:8000 export INFRAHUB_API_TOKEN= ``` The playbook uses the `opsmill.infrahub` collection. Install it from the repository root: ``` ansible-galaxy collection install -r ansible/galaxy-requirements.yml ``` The collection's plugins run on the Ansible **controller** and import `infrahub-sdk` directly, so `infrahub-sdk` must be importable by the controller's Python — otherwise they fail with `infrahub_sdk must be installed to use this plugin`. `lab/pyproject.toml` provides it, which is why the wrapper below runs through `uv run` from `lab/`. Then: ``` cd lab make deploy-from-infrahub FABRIC=Fabric-L3LS-Multi-Domain ``` Or invoke it directly, noting the explicit inventory: ``` uv run ansible-playbook -i ../ansible/inventory_clab.yml ../ansible/deploy_clab.yml \ -e fabric=Fabric-L3LS-Multi-Domain ``` To stage and validate without touching the lab, skip the deployment tasks: ``` ... --skip-tags deploy ``` Useful variables: | Variable | Default | Purpose | | ------------------ | ---------------------------- | ------------------------------------------------------------ | | `fabric` | *required* | Which fabric to deploy | | `clab_host_group` | `clab_hosts` | Inventory group running ContainerLab | | `clab_staging_dir` | `/opt/containerlab/` | Where files are staged **on the lab host** | | `clab_dir` | `/lab` | Where committed bind sources are read from on the controller | Verify: ``` containerlab inspect --topo topology.clab.yml docker ps --format '{{.Names}}' | grep clab- | wc -l ``` ### Running it from Semaphore[​](#running-it-from-semaphore "Direct link to Running it from Semaphore") `invoke init-semaphore` registers a **Fetch ContainerLab Files** template. It runs the same playbook with `--skip-tags deploy`, so it fetches the artifacts, stages every file the topology references, and validates them — but does not deploy. That is deliberate: the Semaphore container has no `containerlab` binary and no Docker socket, so an unskipped run can only ever fail on the containerlab check. The pulled files land on the Docker host, in `lab/clab-staging`: ``` lab/clab-staging/ ├── topology.clab.yml └── configs/ ├── .cfg # one per device ├── eos-intf-mapping/ └── servers/ ``` Every run ends by printing both the in-container and on-host paths, so there is no need to work them out. The directory is created by `invoke start` with mode 0777, because the container writes as a different uid than the host user; files it writes are owned by that uid, so they are readable but not writable from the host. `fabric` and `clab_staging_dir` come from the template's **ContainerLab** environment, not from a survey prompt — a declared survey variable is recorded on the task but never reaches `ansible-playbook` in Semaphore v2.17. Override them per run in the task's Environment field. To deploy rather than only fetch, point `clab_hosts` in `ansible/inventory_clab.yml` at a ContainerLab host reachable over SSH and clear the template's `--skip-tags deploy` argument. ## Pinning the lab's data with `manual_objects/`[​](#pinning-the-labs-data-with-manual_objects "Direct link to pinning-the-labs-data-with-manual_objects") `invoke load` loads `objects/` only. A second, opt-in set in `manual_objects/` is loaded manually: ``` uv run infrahubctl object load manual_objects/ --branch ``` It exists to make the multi-domain fabric line up with the committed lab rather than with whatever the pools happen to allocate: | File | Sets | | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `00_lab_l3ls_multi_domain.yml` | Fixed management addresses `10.0.6.11`–`.16` and `.21`–`.26` with matching serials across the 12 switches; `Ethernet5`/`Ethernet6` as `peering` interfaces on the two DCI leaves in each DC; the four `role=dci` `NetworkLink` objects between them; and one `EvpnGatewayGroup` per DC | | `15a_servers_l3ls_multi_domain.yml` | The two `ComputePhysicalServer` nodes the topology renders as Linux nodes | Without it, the generator allocates management addresses from `Fabric-L3LS-Multi-Domain-Mgmt-Pool` in allocation order, so the rendered `mgmt-ipv4` values are valid but won't match the values in the committed topology. Load it before running the generator chain, on the same branch. ## How the generated lab differs from the committed lab[​](#how-the-generated-lab-differs-from-the-committed-lab "Direct link to How the generated lab differs from the committed lab") The generated topology is a structural replica, not a byte-identical copy of `lab/topology.clab.yml`. The differences are intentional: | | Committed lab | Generated | | -------------------------- | ------------------------ | --------------------------------------------------------------------------------------------- | | Node names | `ih-dc1-spine1` | `spine-infrahub-dc1-1` — the Infrahub device names, no renaming layer | | Topology name | `infrahub-avd` | the fabric name, so container and management-network names differ | | `startup-config` directory | `avd/intended/configs/` | `configs/` — where the playbook writes fetched configs | | `ceos-config` bind | present | absent — serial and system-MAC files are per-lab-device-name and are not modelled in Infrahub | | CVaaS token bind | available, commented out | absent | Node counts, kinds, images, management addresses, link counts, interface-name forms, and bind mount points do match. **Server-to-server reachability is not expected to work in the generated lab.** The committed netplan files encode VLANs 11/12/19 and their addresses, while the multi-domain fabric models VLANs 21/22/29, and netplan is not generated from Infrahub. The `make ping` checks belong to the committed lab flow, not this one. ## Related[​](#related "Direct link to Related") * `lab/README.md` — the committed lab, its Makefile targets, and the cEOS image import. * [Viewing Artifacts](/arista-avd/viewing-artifacts.md) — finding, previewing, and downloading artifacts. * [Transforms](/arista-avd/developer-guide/transforms.md) — how transforms and artifact definitions are wired. --- # Developer guide This guide is for **contributors** who want to extend, debug, or maintain the AVD pipeline — adding device roles, transform outputs, schema fields, or fixing pipeline issues. It assumes familiarity with Python, GraphQL, and Infrahub generators/transforms. ## Start here[​](#start-here "Direct link to Start here") 1. **[Architecture Overview](/arista-avd/developer-guide/architecture.md)** — system components, data model hierarchy, and the generator pipeline at a glance. 2. **[AVD Pipeline → Overview](/arista-avd/developer-guide/avd/overview.md)** — the two-phase pipeline (hostvars → structured config) and the PyAVD version target. ## Reference[​](#reference "Direct link to Reference") * **[Schemas](/arista-avd/developer-guide/schemas.md)** — every YAML schema file and the kinds it defines. * **[Generators](/arista-avd/developer-guide/generators.md)** — the generator framework, file structure, and per-generator behaviour. * **[Transforms](/arista-avd/developer-guide/transforms.md)** — Python and Jinja2 transforms, queries, and content types. * **[Checks](/arista-avd/developer-guide/checks.md)** — proposed-change validation checks, including CloudVision configuration validation. ## AVD pipeline[​](#avd-pipeline "Direct link to AVD pipeline") The AVD pipeline is the technically distinguishing piece of this solution and has its own sub-section: * **[Overview](/arista-avd/developer-guide/avd/overview.md)** — two-phase pipeline + PyAVD version pin. * **[Hostvars Reference](/arista-avd/developer-guide/avd/hostvars.md)** — the PyAVD-compatible structure produced per device role. * **[Transforms](/arista-avd/developer-guide/avd/transforms.md)** — `avd_eos_config`, `avd_fabric_doc`, `avd_device_doc`. * **[AvdArtifact & File Storage](/arista-avd/developer-guide/avd/artifacts.md)** — the `AvdArtifact` node, child file nodes, checksum-based change detection. * **[Role Mapping](/arista-avd/developer-guide/avd/role-mapping.md)** — Infrahub roles → AVD device types. * **[Extending the Pipeline](/arista-avd/developer-guide/avd/extending.md)** — worked examples for new roles, new transform outputs, new hostvar fields. * **[Debugging the Pipeline](/arista-avd/developer-guide/avd/debugging.md)** — intermediate-file inspection, single-generator re-runs, common failure modes. ## Looking for the operator guides?[​](#looking-for-the-operator-guides "Direct link to Looking for the operator guides?") If you want to *use* the system to provision fabrics and view configurations without modifying code, start with [Quick Start](/arista-avd/quick-start.md) and the [how-to guides](/arista-avd/how-to/add-network-segment.md). --- # Architecture overview Developer Guide Assumes familiarity with Infrahub and Python. If you only want to *use* the system, start with [Quick Start](/arista-avd/quick-start.md). The solution is a repository of schemas, generators, and transforms loaded on top of the Infrahub platform. The sections below cover its components, data model, and the generator and transform pipelines. ## System components[​](#system-components "Direct link to System components") ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ Infrahub Platform │ ├─────────────────────────────────────────────────────────────────────────┤ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────┐ │ │ │ Neo4j Graph │ │ PostgreSQL │ │ Object Store │ │ │ │ Database │ │ (Metadata) │ │ (Hostvars, Configs) │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────────────┘ │ ├─────────────────────────────────────────────────────────────────────────┤ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────┐ │ │ │ Redis Cache │ │ RabbitMQ │ │ Infrahub Server │ │ │ │ │ │ (Queue) │ │ (API + Git Backend) │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────┐ │ Repository Solution │ ├─────────────────────────────────────────────────────────────────────────┤ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────┐ │ │ │ Schemas │ │ Generators │ │ Transforms │ │ │ │ (YAML DSL) │ │ (Python) │ │ (Python + Jinja2) │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────────────┘ │ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────┐ │ │ │ Object Data │ │ Queries │ │ Core Library │ │ │ │ (Seed YAML) │ │ (GraphQL) │ │ (src/solution_arista_avd) │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────┘ ``` ## Data model hierarchy[​](#data-model-hierarchy "Direct link to Data model hierarchy") The system models a 3-tier datacenter network fabric: ``` NetworkFabric (e.g., "Fabric-L3LS-MultiPod-A") ├── NetworkPod (e.g., "Pod-A1", "Pod-A2") │ ├── LocationRack (e.g., "Rack-A1-01", "Rack-A1-02") │ │ └── DcimDevice [leaf] (e.g., "leaf-A1-01-1") │ │ └── InterfacePhysical (e.g., "Ethernet1") │ │ ├── NetworkLink → remote interface │ │ └── IpamIPAddress │ └── DcimDevice [spine] (e.g., "spine-A1-1") └── DcimDevice [super_spine] (e.g., "ss-A-1") ``` ## IP address management[​](#ip-address-management "Direct link to IP address management") Fabric-level pool allocation: ``` NetworkFabric ├── loopback_pool: CoreIPPrefixPool │ └── Internal CoreIPAddressPool wrapper: Loopback0 addresses ├── vtep_pool: CoreIPPrefixPool │ └── Internal CoreIPAddressPool wrapper: VTEP loopback addresses ├── uplink_pool: CoreIPPrefixPool │ └── Prefix allocations for point-to-point links ├── mgmt_pool: CoreIPAddressPool │ └── OOB management addresses ├── CoreNumberPool: ASN Pool (65000-65999) │ └── Tier-aware eBGP ASN allocation: shared super-spine ASN per fabric, shared spine ASN per pod, leaf ASNs per device or MLAG domain └── CoreNumberPool: Node ID Pool (1-65535) └── Per-device unique identifier ``` ## Generator pipeline[​](#generator-pipeline "Direct link to Generator pipeline") Generators run in sequence to build infrastructure: ``` ┌─────────────────────────┐ │ 1. FabricGenerator │ Triggered on: NetworkFabric │ - Resolve fabric pools │ Creates: Super-spine devices │ - Create super-spines │ └───────────┬─────────────┘ ▼ ┌─────────────────────────┐ │ 2. PodGenerator │ Triggered on: NetworkPod │ - Create spine devices │ Creates: Spine switches │ - Link to super-spines │ └───────────┬─────────────┘ ▼ ┌─────────────────────────┐ │ 3. RackGenerator │ Triggered on: LocationRack │ - Create leaf devices │ Creates: Leaf switches │ - Link to spines │ └───────────┬─────────────┘ ▼ ┌─────────────────────────┐ │ 4. AVD Generators │ Triggered on: NetworkFabric/Device │ - Build hostvars │ Creates: AVD configs │ - Generate struct cfg │ └─────────────────────────┘ ``` Two further generators sit outside this chain: * **`ServerCablingGenerator`** (on `ComputePhysicalServer`) cables a server to the leaves in its rack, then reconciles its LAGs and VLANs and re-triggers hostvar generation for those leaves. * **`BackfillStructuredConfigGenerator`** (on `AvdStructuredConfigFile`) runs in the opposite direction, reading AVD's structured-config output back into IPAM, interface, BGP, and routing objects. See [Generators](/arista-avd/developer-guide/generators.md). ## Transform pipeline[​](#transform-pipeline "Direct link to Transform pipeline") Transforms convert data to artifacts: ``` ┌──────────────────┐ ┌────────────────────┐ ┌─────────────────┐ │ GraphQL Query │ ──▶ │ Transform Logic │ ──▶ │ Output Artifact│ │ (Data Fetch) │ │ (Python/Jinja2) │ │ (Config/Doc) │ └──────────────────┘ └────────────────────┘ └─────────────────┘ Examples: - DcimInterface → ComputedInterfaceDescription → "→ device:interface" - NetworkFabric → CablingPlan → CSV cabling matrix - DcimDevice → AvdEosConfig → EOS CLI configuration - NetworkFabric → AvdFabricDoc → Markdown documentation - DcimDevice → AvdAntaCatalog → ANTA test catalog (YAML) - NetworkFabric → ContainerLabTopology → ContainerLab topology (YAML) ``` ## Validation pipeline[​](#validation-pipeline "Direct link to Validation pipeline") Alongside transforms, proposed-change validation runs **checks** — Python routines that report pass, information, or error rather than producing an artifact. The repository ships one, `cv-config-validation`, which deploys the rendered EOS configs into a CloudVision workspace and blocks the proposed change on a failed build. See [Checks](/arista-avd/developer-guide/checks.md). ## Checksum-based change detection[​](#checksum-based-change-detection "Direct link to Checksum-based change detection") Generators use checksums to avoid redundant regeneration: ``` class GeneratorMixin: def calculate_checksum(self, related_node_ids: list[str]) -> str: """Create deterministic hash from related node IDs""" return hashlib.sha256("".join(sorted(related_node_ids))).hexdigest() # Usage in generator: new_checksum = self.calculate_checksum([pod.id, device.id, ...]) if new_checksum != target.checksum: # Regenerate target.checksum = new_checksum ``` ## Configuration files[​](#configuration-files "Direct link to Configuration files") | File | Role | | -------------------- | --------------------------------------------------- | | `.infrahub.yml` | Register queries, generators, transforms, artifacts | | `repository.yml` | Define repository as CoreRepository in Infrahub | | `docker-compose.yml` | Orchestrate Infrahub services | | `pyproject.toml` | Python dependencies and tool configuration | ## Docker service stack[​](#docker-service-stack "Direct link to Docker service stack") ``` services: infrahub-server: # Main API server infrahub-git: # Git backend for repository infrahub-worker: # Async task execution neo4j: # Graph database postgres: # Relational metadata redis: # Cache layer rabbitmq: # Message queue ``` ## Development workflow[​](#development-workflow "Direct link to Development workflow") ``` 1. Edit schema (schemas/*.yml) ↓ 2. Load schema: inv load-schema ↓ 3. Edit generator/transform code ↓ 4. Test: pytest tests/ ↓ 5. Lint: inv lint ↓ 6. Reload: inv load ↓ 7. Run generators via UI ``` ## Pool role resolution[​](#pool-role-resolution "Direct link to Pool role resolution") Fabric and pod IP pool intent is role-driven. `NetworkFabric.fabric_ip_pools` is the authoritative fabric collection for Management, Loopback, Loopback VTEP, Fabric Point-to-Point, DCI, and Fabric Supernet roles. `NetworkPod.pod_ip_pools` is the authoritative pod collection for pod-scoped Loopback, Loopback VTEP, Fabric Point-to-Point, MLAG, and MLAG Peering roles. Validation resolves a pool's purpose from the roles on its backing `IpamPrefix` resources. The proposed-change check rejects duplicate authoritative roles, mixed-role pools, non-IP pool members, pod management pools, and pod prefixes that are not contained by the matching parent fabric pool. During migration, legacy fabric and pod pool relationships are still present and object data is dual-populated. Generators prefer collection relationships and fall back to legacy relationships only when needed. ## Source[​](#source "Direct link to Source") * Infrahub configuration: [`.infrahub.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/.infrahub.yml) — the queries, generators, transforms, and artifact definitions registered with Infrahub. * Schemas: [`schemas/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/schemas) — the data model. * Generators: [`generators/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/generators) — Python generator classes. * Transforms: [`transforms/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/transforms) — Python transform classes and templates. * Checks: [`checks/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/checks) — proposed-change validation, currently CloudVision. * Playbooks: [`ansible/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/ansible) — the tree Semaphore runs for EOS config deployment and ContainerLab staging. * Core library: [`src/solution_arista_avd/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/src/solution_arista_avd) — shared protocols, AVD utilities, sorting, addressing. * Service portal: [`service_catalog/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/service_catalog) — Streamlit UI that orchestrates the portal workflows. --- # AvdArtifact & file storage Developer Guide Documents the graph schema that links the two pipeline phases. The AVD pipeline stores its intermediate data — hostvars and structured configs — in Infrahub as **graph nodes**, not in an external object store. Three schema kinds are involved, all defined in [`schemas/objects/objects.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/objects/objects.yml). ## Schema: `AvdArtifact`[​](#schema-avdartifact "Direct link to schema-avdartifact") Per-device container that links the device to its hostvars and structured config files. ``` - name: Artifact namespace: Avd human_friendly_id: - name__value uniqueness_constraints: - [device, name__value] attributes: - name: name kind: Text unique: true relationships: - name: device peer: DcimDevice kind: Attribute cardinality: one optional: false - name: hostvar_file peer: AvdHostvarFile kind: Component cardinality: one optional: true identifier: "avdartifact__hostvar_file" - name: structured_config_file peer: AvdStructuredConfigFile kind: Component cardinality: one optional: true identifier: "avdartifact__structured_config_file" ``` Key points: * **One artifact per device** — the `(device, name)` uniqueness constraint enforces this. * **Component relationships** to the two file nodes — the file nodes are owned by the artifact and deleted when it is. * Both file relationships are **optional** — an artifact with a hostvar file but no structured config is a valid state (it means Phase 1 has run but Phase 2 hasn't, yet). ## Schema: `AvdHostvarFile`[​](#schema-avdhostvarfile "Direct link to schema-avdhostvarfile") Stores the per-device PyAVD hostvars as a JSON file. Inherits from `CoreFileObject`. ``` - name: HostvarFile namespace: Avd inherit_from: - CoreFileObject human_friendly_id: - "artifact__name__value" uniqueness_constraints: - ["artifact"] relationships: - name: artifact peer: AvdArtifact kind: Parent cardinality: one optional: false identifier: "avdartifact__hostvar_file" ``` `CoreFileObject` provides: * `content` — the raw file bytes. * `content_type` — MIME type (typically `application/json` for hostvars). * `checksum` — content hash (managed by Infrahub). * `file_name` — display name. The `Parent` kind on the `artifact` relationship ties this node's lifecycle to the artifact — the file is removed when the parent artifact is deleted. ## Schema: `AvdStructuredConfigFile`[​](#schema-avdstructuredconfigfile "Direct link to schema-avdstructuredconfigfile") Same shape as `AvdHostvarFile`, for the structured-config JSON: ``` - name: StructuredConfigFile namespace: Avd inherit_from: - CoreFileObject human_friendly_id: - "artifact__name__value" uniqueness_constraints: - ["artifact"] relationships: - name: artifact peer: AvdArtifact kind: Parent cardinality: one optional: false identifier: "avdartifact__structured_config_file" ``` ## How the two phases share data[​](#how-the-two-phases-share-data "Direct link to How the two phases share data") Phase 1 is the sole writer of `hostvar_file`; Phase 2 reads hostvars and is the sole writer of `structured_config_file`; transforms are read-only consumers. ## Change detection via checksums[​](#change-detection-via-checksums "Direct link to Change detection via checksums") Generators avoid re-writing unchanged files by comparing SHA256 checksums: 1. Serialise the new content to JSON. 2. Compute SHA256. 3. Fetch the existing file node's `checksum` attribute (from `CoreFileObject`). 4. If the checksums match, skip the write. 5. If they differ, replace the file. The checksum is computed **in-memory per run** — it is not a custom attribute on `AvdArtifact`. This keeps the schema minimal and lets Infrahub handle file-level checksumming via `CoreFileObject`. ## Artifact definitions[​](#artifact-definitions "Direct link to Artifact definitions") Four artifact definitions (in [`.infrahub.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/.infrahub.yml)) turn the stored data into user-visible artifacts: | Artifact | Target group | Transform | Content type | | -------------------------- | ------------- | ------------------ | ------------------ | | `avd_eos_configuration` | `avd_devices` | `avd_eos_config` | `text/plain` | | `avd_device_documentation` | `avd_devices` | `avd_device_doc` | `text/markdown` | | `avd_fabric_documentation` | `fabrics` | `avd_fabric_doc` | `text/markdown` | | `avd_anta_catalog` | `avd_devices` | `avd_anta_catalog` | `application/yaml` | ``` artifact_definitions: - name: avd_eos_configuration targets: avd_devices transformation: avd_eos_config - name: avd_fabric_documentation targets: fabrics transformation: avd_fabric_doc - name: avd_device_documentation targets: avd_devices transformation: avd_device_doc - name: avd_anta_catalog targets: avd_devices transformation: avd_anta_catalog ``` The repository defines two further artifacts from the same data that are not part of the AVD pipeline: `cabling_plan` and `containerlab_topology`, both fabric-scoped. See [Transforms](/arista-avd/developer-guide/transforms.md). When an operator opens one of these artifacts in the UI, Infrahub runs the transform against the target node, which fetches the relevant files from the `AvdArtifact` tree. ## Target groups[​](#target-groups "Direct link to Target groups") * `avd_devices` — all `DcimDevice` nodes that should participate in AVD. Populated by upstream generators (for example, `generate-rack` adds newly created leaves to the group). * `fabrics` — all `NetworkFabric` nodes. Group membership is set by the generators at creation time; there is no separate "add to group" step in the AVD pipeline itself. --- # Debugging the pipeline Developer Guide For operator-facing issues (stack health, generator order, "no structured config available"), see the [Troubleshooting page](/arista-avd/troubleshooting.md). ## Inspecting hostvars and structured configs[​](#inspecting-hostvars-and-structured-configs "Direct link to Inspecting hostvars and structured configs") Both files live on the `AvdArtifact` for each device (see [AvdArtifact & File Storage](/arista-avd/developer-guide/avd/artifacts.md)). To read them: ### Via the Infrahub UI[​](#via-the-infrahub-ui "Direct link to Via the Infrahub UI") 1. Navigate to the device's `AvdArtifact` (for example, search for the artifact named after the device). 2. Open the `hostvar_file` or `structured_config_file` relationship — the child node is an `AvdHostvarFile` / `AvdStructuredConfigFile`. 3. Download or view the `content` attribute (JSON). ### Via the SDK[​](#via-the-sdk "Direct link to Via the SDK") ``` from infrahub_sdk import InfrahubClient client = InfrahubClient(address="http://localhost:8000") await client.login() artifact = await client.get( kind="AvdArtifact", device__name__value="leaf-pod-A1-1", branch="main", prefetch_relationships=True, include=["hostvar_file", "structured_config_file"], ) hostvars_node = artifact.hostvar_file.peer hostvars = hostvars_node.content.value # raw JSON string ``` ## Checksum-based change detection[​](#checksum-based-change-detection "Direct link to Checksum-based change detection") Both generators skip writes when content is unchanged. The flow is: 1. Serialise the new content (hostvars dict or structured config dict) to JSON. 2. Compute `hashlib.sha256(json_bytes).hexdigest()`. 3. Compare against the existing file's `checksum` attribute (provided by `CoreFileObject`). 4. If equal → skip the write (log "unchanged, skipped"). 5. If different → replace the file. ### Forcing a regeneration[​](#forcing-a-regeneration "Direct link to Forcing a regeneration") If you need to force a fresh write (for example, you suspect the checksum is stale or want to test the generator path end-to-end), delete the child file node: ``` hostvars_node = artifact.hostvar_file.peer await hostvars_node.delete() ``` The next generator run writes a new `AvdHostvarFile` unconditionally. ## Re-running a single generator[​](#re-running-a-single-generator "Direct link to Re-running a single generator") ### From the UI[​](#from-the-ui "Direct link to From the UI") 1. On a branch, open **Actions → Generator definitions**. 2. Pick the generator (for example, `generate-avd-device-hostvar`). 3. Click **Run** and select the target device (or fabric for Phase 2). ### Via the SDK[​](#via-the-sdk-1 "Direct link to Via the SDK") Generators can be triggered programmatically: ``` await client.execute_graphql( query=""" mutation RunGenerator($group: String!, $generator: String!) { CoreGeneratorDefinitionRun( data: { generator: $generator, group: $group } ) { ok } } """, variables={"generator": "generate-avd-device-hostvar", "group": "avd_devices"}, branch_name="my-branch", ) ``` See the service portal implementation in [`service_catalog/utils/api.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/service_catalog/utils/api.py) (`run_avd_pipeline()` and related helpers) for a working example. ## Missing structured config[​](#missing-structured-config "Direct link to Missing structured config") **Symptoms**: `avd_eos_config` transform returns `No structured config available`. **Diagnostic flow**: 1. Fetch the device's `AvdArtifact`. Is there one? If not — the device isn't in the `avd_devices` group. 2. Does `AvdArtifact.hostvar_file` exist? If not — Phase 1 didn't run for this device. Run `generate-avd-device-hostvar` for it. 3. Does `AvdArtifact.structured_config_file` exist? If not — Phase 2 didn't run (or failed) for this device's fabric. Run `generate-avd-device-structured-config` for the fabric. 4. If `structured_config_file` exists but `content` is empty or malformed — the previous Phase 2 run had a partial failure. Delete the file and re-run Phase 2. ## PyAVD validation errors[​](#pyavd-validation-errors "Direct link to PyAVD validation errors") `pyavd.validate_inputs()` is called in Phase 2 across **all** devices in the fabric. If one device has invalid hostvars, the whole Phase 2 run fails. **Reading the error**: ``` pyavd.j2lint.utils.ValidationError: Invalid type for ... in ... ``` The error names a field and a device. Fetch that device's hostvars (above) and look for: * Missing required fields for the role (`id`, `bgp_as`, `loopback_ipv4_address` for L3 roles). * Mismatched list lengths in the uplink block (`uplink_interfaces` vs `uplink_switches`). * Type mismatches — PyAVD expects stringified ASNs (`"65101"`), CIDR-less loopbacks, etc. Cross-reference [Hostvars Reference](/arista-avd/developer-guide/avd/hostvars.md) for the expected types. ## Common failure modes[​](#common-failure-modes "Direct link to Common failure modes") | Symptom | Likely cause | Fix | | -------------------------------------------------------- | ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | Phase 2 fails "Missing hostvar\_file for device X" | Phase 1 didn't complete for device X | Re-run Phase 1 for that device | | `get_avd_type` raises `ValueError` | New role added to schema without adding to `ROLE_TO_AVD_TYPE` | Update [`src/solution_arista_avd/avd.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/src/solution_arista_avd/avd.py) | | Fabric documentation empty or partial | One or more devices missing hostvars | Complete Phase 1 for all devices | | Artifact regenerates every run even when nothing changed | Hostvars dict has a non-deterministic field (for example, iteration order of a set) | Sort lists/dicts before JSON-serialising | | Transform returns stale output | `CoreFileObject.content` cached somewhere; rare | Force-regenerate the artifact from the UI preview panel | ## Pre-seeded device reconciliation[​](#pre-seeded-device-reconciliation "Direct link to Pre-seeded device reconciliation") When a fabric already contains pods, racks, or devices, run `generate-fabric` for the fabric first. A standard run preserves non-empty operator-provided device values, including `serial` and `mgmt_ip`, and fills missing generator-owned relationships needed by AVD. The expected log stream for a reconciled device includes field-decision entries for populated, preserved, or skipped fields. If the cascade stops after fabric generation, check whether the downstream pod or rack checksum changed. Changed targets should be handled by the existing trigger rules; unchanged targets should be visible as direct `CoreGeneratorDefinitionRun` calls for `generate-pod` or `generate-rack` with explicit node IDs. If hostvars are still missing after rack generation, follow the missing structured config diagnostic flow above and confirm all racks in the fabric have `generation_complete=True`. ## Turning up log verbosity[​](#turning-up-log-verbosity "Direct link to Turning up log verbosity") The generators log via the Infrahub SDK's logging. To see more detail on a dev stack, bump the log level in the Infrahub server's environment: ``` # in docker-compose.override.yml for the infrahub service environment: INFRAHUB_LOG_LEVEL: DEBUG ``` Then restart: ``` uv run invoke restart --component=infrahub-server ``` ## Comparing a render against AVD's own examples[​](#comparing-a-render-against-avds-own-examples "Direct link to Comparing a render against AVD's own examples") `scripts/compare_avd_examples.py` answers whether this design's render produces the same EOS features as the AVD example it is based on — use it when a new fabric design is added and you want evidence beyond a successful render. ``` # one rendered config against one AVD example config uv run python scripts/compare_avd_examples.py rendered.cfg avd_example.cfg # two directories of *.cfg, matched by basename uv run python scripts/compare_avd_examples.py rendered_dir/ avd_examples_dir/ # no inputs needed — exercises the comparison logic itself uv run python scripts/compare_avd_examples.py --self-test ``` Byte-for-byte identity is deliberately not the goal. Infrahub allocates its own addressing, hostnames, ASNs, and node IDs, so the script masks IP, MAC, IPv6, and ASN tokens before comparing, then reports top-level EOS feature sections (`router bgp`, `vlan`, `mlag configuration`, `router isis`, `mpls ldp`, …) as present in both, only in the render, or only in the example. A section present in the example but missing from the render is the difference to investigate. The reference configs come from the AVD repository, under `ansible_collections/arista/avd/examples//intended/configs/*.cfg`. ## Related reading[​](#related-reading "Direct link to Related reading") * [Overview](/arista-avd/developer-guide/avd/overview.md) — the pipeline shape at a glance. * [AvdArtifact & File Storage](/arista-avd/developer-guide/avd/artifacts.md) — exactly which node holds which piece of data. * [Troubleshooting](/arista-avd/troubleshooting.md) — operator-level issues and fixes. --- # Extending the pipeline Developer Guide The touch-point lists below give you the exact files to edit for the three most common extensions. ## Native schema vs. the escape hatch[​](#native-schema-vs-the-escape-hatch "Direct link to Native schema vs. the escape hatch") When closing a capability gap (for example, to support a new AVD example scenario), decide up front whether to model it **natively** in the schema or pass it through the **`avd_custom_hostvars` escape hatch**: * **Prefer a native schema change** when the capability is reused across more than one scenario, is a first-class topology/role/protocol concept operators select in the UI, or needs validation, pool allocation, or deterministic generation. Examples: device roles and their `ROLE_TO_AVD_TYPE` mapping, underlay protocol choices, EVPN inputs such as `evpn_vlan_aware_bundles` and EVPN Gateway Groups. * **Use the `avd_custom_hostvars` escape hatch** when the capability is specific to a single scenario, is a pass-through of PyAVD keys that need no allocation or cross-device derivation, or would be premature to model before real demand. Examples: campus dot1x/PoE/port-profiles/in-band management and MPLS/VPN-IPv4 for ISIS-LDP IPVPN. `avd_custom_hostvars` is a JSON attribute available at fabric, pod, and device scope. Its content deep-merges with the generator-produced hostvars, and **generator-produced values win** on conflict. Keep escape-hatch content in committed seed data (not manual UI edits) so a design stays reproducible and idempotent, and confirm every key is accepted by the pinned PyAVD version. Escape-hatch use is a deliberate, documented choice per capability — not a default fallback to avoid modeling. ## Add a new device role[​](#add-a-new-device-role "Direct link to Add a new device role") Scenario: you want to support a new Infrahub role (for example, `border-leaf`) that maps to a PyAVD type. **Touch points:** 1. **Schema** — add the role value to the `DcimDevice` `role` dropdown in [`schemas/dcim_extensions.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/dcim_extensions.yml) (the single authoritative device-role list). 2. **Reload the schema and regenerate generated files** — none of these files should be hand-edited: ``` uv run invoke load-schema # push schema to Infrahub uv run infrahubctl graphql export-schema --destination schema.graphql # refresh the local GraphQL SDL uv run infrahubctl protocols --schemas schemas --out src/solution_arista_avd/protocols.py # refresh typed protocol classes ``` 3. **Role map** — add the mapping in [`src/solution_arista_avd/avd.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/src/solution_arista_avd/avd.py): ``` ROLE_TO_AVD_TYPE: dict[str, str] = { ..., "border_leaf": "l3leaf", # or whatever PyAVD type fits } ``` 4. **Hostvars generator** — add a branch for the new role in [`generators/generate_avd_device_hostvar.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_hostvar.py) for any role-specific fields (uplink role, MLAG, EVPN data). 5. **Upstream generator** — whichever generator creates devices of this role (fabric/pod/rack/custom) needs to set the `role` attribute correctly and add the device to the `avd_devices` group. 6. **Tests** — add a case in [`tests/unit/test_avd.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/tests/unit/test_avd.py) covering `get_avd_type("border_leaf")`. 7. **Docs** — update [Role Mapping](/arista-avd/developer-guide/avd/role-mapping.md) and, if the role implies new hostvar fields, [Hostvars Reference](/arista-avd/developer-guide/avd/hostvars.md). ## Add a new transform output[​](#add-a-new-transform-output "Direct link to Add a new transform output") Scenario: you want an additional artifact per device or per fabric (for example, a JSON summary, a CSV inventory). **Touch points:** 1. **GraphQL query** — write the `.gql` query under `transforms/`. Example: `transforms/avd_inventory.gql`. 2. **Pydantic query model** — **do not write this manually.** Generate it with: ``` uv run infrahubctl graphql generate-return-types transforms/avd_inventory.gql ``` This reads `schema.graphql` (checked in at the repo root) and emits `transforms/avd_inventory_query.py` alongside the query. Re-run whenever the query or the schema changes. If the schema is stale, regenerate it first with `uv run infrahubctl graphql export-schema` (requires a running Infrahub). 3. **Transform class** — implement the transform in `transforms/avd_inventory.py` as a subclass of the Infrahub Python transform base class. Typical structure: ``` class AvdInventoryTransform(InfrahubTransform): query = "avd_inventory" async def transform(self, data: dict) -> str: parsed = AvdInventoryQuery.model_validate(data) # ... your logic here return output ``` 4. **Register in `.infrahub.yml`**: ``` queries: - name: avd_inventory file_path: "./transforms/avd_inventory.gql" python_transforms: - name: avd_inventory class_name: AvdInventoryTransform file_path: "./transforms/avd_inventory.py" artifact_definitions: - name: avd_inventory_csv targets: fabrics # or avd_devices, depending on scope transformation: avd_inventory ``` 5. **Tests** — add a unit test under `tests/unit/` exercising `transform()` on a fixture, plus optionally an integration test that hits a running Infrahub. After merge, operators can open the new artifact from the target node's **Artifacts** tab. ## Add a new field to hostvars[​](#add-a-new-field-to-hostvars "Direct link to Add a new field to hostvars") Scenario: you want PyAVD to receive an additional input field (for example, a per-device SNMP location string) that currently isn't populated. **Touch points:** 1. **Schema** — if the field isn't already represented, add it to the relevant schema (`DcimDevice`, `NetworkFabric`, etc.) in [`schemas/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/schemas). 2. **Reload the schema and regenerate generated files**: ``` uv run invoke load-schema # push schema to Infrahub uv run infrahubctl graphql export-schema --destination schema.graphql # refresh the local GraphQL SDL uv run infrahubctl protocols --schemas schemas --out src/solution_arista_avd/protocols.py # refresh typed protocol classes ``` 3. **GraphQL query** — update [`generators/avd_device_hostvar.gql`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/avd_device_hostvar.gql) to pull the new field. 4. **Pydantic query model** — regenerate, don't hand-edit: ``` uv run infrahubctl graphql generate-return-types generators/avd_device_hostvar.gql ``` This rewrites `generators/generate_avd_device_inputs_query.py` from the query and the refreshed schema. 5. **Hostvars builder** — map the new attribute into the PyAVD hostvars dict in [`generators/generate_avd_device_hostvar.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_hostvar.py): * Device-level, role-independent field → add it in `_build_hostvars()` (where `type`, `fabric_name`, `bgp_as`, loopback/mgmt basics are assembled). * Role-specific or multi-attribute field → add the logic in the appropriate role branch of the same file. 6. **Validation** — PyAVD's `validate_inputs()` flags unknown fields as errors. Confirm the field is in the PyAVD input schema for the version pinned (see [overview](/arista-avd/developer-guide/avd/overview.md#pyavd-version)). If it isn't a standard PyAVD field, look at using `custom_structured_configuration_prefix` or `structured_config` pass-through instead. 7. **Tests** — add a case in [`tests/unit/test_hostvar_ordering.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/tests/unit/test_hostvar_ordering.py) for any hostvars logic added to the generator. (`tests/unit/test_avd.py` covers only the role→type mapping in `src/solution_arista_avd/avd.py`.) 8. **Docs** — update [Hostvars Reference](/arista-avd/developer-guide/avd/hostvars.md) with the new field and its Infrahub source. ## Checklist: what to run before opening a PR[​](#checklist-what-to-run-before-opening-a-pr "Direct link to Checklist: what to run before opening a PR") * `uv run invoke lint` — ruff, mypy, yamllint must all pass. * `uv run pytest tests/unit` — all unit tests pass. * `uv run pytest tests/integration` — integration tests pass (requires a running Infrahub). * On a feature branch in a live Infrahub, trigger the affected generators twice and confirm idempotence — the second run should be a no-op per [Debugging the Pipeline → checksum-based skipping](/arista-avd/developer-guide/avd/debugging.md#checksum-based-change-detection). --- # Hostvars reference Developer Guide Hostvars structure is **PyAVD-version-sensitive** — see the [overview](/arista-avd/developer-guide/avd/overview.md#pyavd-version) for the pinned version. [`generate-avd-device-hostvar`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_hostvar.py) builds the PyAVD hostvars dict below for each `DcimDevice`. The dict is serialised to JSON and stored as an `AvdHostvarFile` attached to the device's `AvdArtifact` (see [AvdArtifact & File Storage](/arista-avd/developer-guide/avd/artifacts.md)). ## Top-level fields (all roles)[​](#top-level-fields-all-roles "Direct link to Top-level fields (all roles)") | Field | Type | Source | Notes | | ----------------------------- | ------ | -------------------------------------------------------------- | ----------------------------------------------------------------------------------- | | `type` | string | Role-mapped from `DcimDevice.role.value` | See [Role Mapping](/arista-avd/developer-guide/avd/role-mapping.md). | | `fabric_name` | string | `NetworkFabric.name.value` | | | `id` | int | `DcimDevice.node_id.value` | Fabric-unique device identifier. | | `bgp_as` | string | `DcimDevice.asn.node.asn.value` | Stringified; PyAVD expects a string. | | `loopback_ipv4_address` | string | `DcimDevice.loopback_ip` | Optional; stripped of CIDR. | | `loopback_ipv4_pool` | string | `DcimDevice.loopback_ip.node.ip_prefix.node.prefix.value` | Parent Infrahub prefix for the Loopback0 address. | | `vtep_loopback_ipv4_address` | string | `DcimDevice.vtep_loopback_ip` | Leaf and border-leaf only; stripped of CIDR. | | `vtep_loopback_ipv4_pool` | string | `DcimDevice.vtep_loopback_ip.node.ip_prefix.node.prefix.value` | Parent Infrahub prefix for the VTEP loopback address; emitted for VTEP leaf roles. | | `mgmt_ip` | string | `DcimDevice.mgmt_ip` | Optional; includes CIDR (for example, `10.255.0.11/24`). | | `mgmt_gateway` | string | Fabric-level setting | Optional. | | `spanning_tree_settings.mode` | string | `NetworkFabric.spanning_tree_mode.value` | Optional; PyAVD 6.3 fabric-wide STP mode (`mstp`, `rstp`, `rapid-pvst`, or `none`). | The builder for these basics lives in [`generators/generate_avd_device_hostvar.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_hostvar.py) as `_build_hostvars()`. (The role→AVD-type mapping it uses, `ROLE_TO_AVD_TYPE`, lives in [`src/solution_arista_avd/avd.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/src/solution_arista_avd/avd.py).) Role-specific STP priorities are modeled as `NetworkSpanningTreePriority` child objects on the fabric. When a child exists for the device role, the hostvars builder emits it under the matching AVD node type defaults, for example `l3leaf.defaults.spanning_tree_priority: 8192`. The legacy fabric-level `spanning_tree_priority` field is still present for non-destructive migration compatibility but is ignored by hostvar generation. ## Uplink fields — `spine`, `leaf`, `border_leaf`, `l2leaf`[​](#uplink-fields--spine-leaf-border_leaf-l2leaf "Direct link to uplink-fields--spine-leaf-border_leaf-l2leaf") Super-spines have no uplinks; all other roles do. | Field | Type | Notes | | -------------------------- | ------------- | ---------------------------------------------------------------- | | `uplink_interfaces` | list\[string] | Local interfaces, for example, `["Ethernet1", "Ethernet2"]`. | | `uplink_switches` | list\[string] | Upstream device hostnames, matched 1:1 with `uplink_interfaces`. | | `uplink_switch_interfaces` | list\[string] | Upstream interface names, matched 1:1 with `uplink_interfaces`. | These are derived from `DcimInterface` objects on the device that carry `role = "uplink"`, plus their connected remote interfaces via `NetworkLink`. ### Uplink role by device role[​](#uplink-role-by-device-role "Direct link to Uplink role by device role") Which *remote* role supplies the uplink depends on the local role: | Local role | Uplink remote role | | ------------- | -------------------- | | `super_spine` | none (top of fabric) | | `spine` | `super_spine` | | `leaf` | `spine` | | `border_leaf` | `spine` | | `l2leaf` | `leaf` | Enforced in [`generate_avd_device_hostvar.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_hostvar.py). ## Role-specific blocks[​](#role-specific-blocks "Direct link to Role-specific blocks") ### `super_spine`[​](#super_spine "Direct link to super_spine") No additional fields beyond top-level. Super-spines sit at the top of the fabric and receive uplinks from spines; they have no own uplinks. ### `spine`[​](#spine "Direct link to spine") * Uplink block (above) with upstream `super_spine` devices. * No leaf-level extensions (no MLAG, no virtual MAC). ### `leaf` and `border_leaf`[​](#leaf-and-border_leaf "Direct link to leaf-and-border_leaf") Leaves and Border Leafs map to PyAVD `l3leaf` and carry the richest hostvars: | Field | Notes | | ---------------------------- | --------------------------------------------------------------------------------------------- | | Uplink block | Upstream `spine` devices. | | `mlag_domain_id` | Derived from MLAG peer relationship if the leaf has a peer. | | `mlag_peer` | Hostname of the MLAG peer leaf. | | `mlag_peer_ipv4_address` | Peer link IP. | | `virtual_router_mac_address` | Per-fabric VMAC used for SVI gateways. | | `l3_interfaces` / SVIs | Emitted from `EvpnSvi` objects attached to VLANs on this leaf's L2 domain. | | `connected_endpoints` | Per interface with `role = "server"` (see below). | | EVPN tenants/VRFs/VLANs | Derived from `EvpnTenant` → `IpamVRF` → `EvpnSvi` → `IpamVLAN` chain filtered to this fabric. | Border Leafs additionally consume valid `NetworkLink` objects with `role=dci` and emit PyAVD `l3_edge.p2p_links` entries. Each DCI link must have exactly two inherited physical endpoints, both endpoint devices must use role `border_leaf`, and both endpoint interfaces must use role `peering`. When the fabric underlay routing protocol is **eBGP**, both endpoint devices must have a BGP ASN assigned and each end's `as` is taken from the endpoint device's own `asn`; with a non-BGP underlay (for example, OSPF) the link is still emitted for reachability, `as` is omitted, and no ASN is required. Point-to-point addresses are allocated as one `/31` per link. DCI pool resolution starts with the endpoint fabric's `fabric_ip_pools` member whose `IpamPrefix.role` is `dci`, falls back to the legacy `NetworkFabric.dci_pool`, and then uses a deterministic Fabric Supernet-derived fallback when the required DCI prefix-pool role is missing. For links between fabrics, the sorted-first endpoint chooses the shared allocation source so both border leafs allocate the same prefix. Endpoint IPs are not stored as DCI-specific link fields. Generated DCI entries are self-contained and do not use `l3_edge.p2p_links_profiles` or per-link `profile` references: ``` { "l3_edge": { "p2p_links": [ { "nodes": ["ih-dc1-leaf1a", "ih-dc2-leaf1a"], "interfaces": ["Ethernet5", "Ethernet5"], "as": [65101, 65201], "ip": ["172.16.0.0/31", "172.16.0.1/31"], "include_in_underlay_protocol": true, "speed": "100g" } ] } } ``` `speed` is emitted only when endpoint/interface data provides a resolvable speed. When it cannot be resolved, the key is omitted and PyAVD uses its normal behavior. Border Leafs are also the only role eligible for modeled EVPN Multi-Domain Gateway hostvars. A target Border Leaf that is a member of an `EvpnGatewayGroup` emits `l3leaf.nodes[].evpn_gateway`: ``` { "remote_peers": [{"hostname": "remote-border-leaf"}], "evpn_l2": {"enabled": true}, "evpn_l3": {"enabled": true, "inter_domain": true}, "d_path": { "enabled": true, "local_domain_id": "65100:1", "remote_domain_id": "65200:1" }, "all_active_multihoming": { "enabled": true, "evpn_ethernet_segment": { "identifier": "0000:0000:0000:0001:0001", "rt_import": "00:00:00:00:00:01" } } } ``` The generator derives the local D-PATH domain ID from `EvpnGatewayGroup.local_domain`, validates that the selected `EvpnGatewayGroup.pod.evpn_domain` matches that parent domain, derives the remote D-PATH domain ID from `EvpnGatewayGroup.remote_domain`, and derives `remote_peers[].hostname` from other valid Border Leaf members in gateway groups that share the same remote EVPN Domain. It does not emit deprecated PyAVD 6.3.0 keys under `all_active_multihoming` such as `enable_d_path`, `evpn_domain_id_local`, or `evpn_domain_id_remote`. Gateway group intent fails before writing the hostvar file when the target or any member is not a Border Leaf, the group has no `local_domain`, the selected Pod has no matching EVPN Domain, the remote domain is missing or conflicts with the local domain, a member is outside the group Pod, or required All-Active Ethernet Segment values are missing. Hostname-only remote peers depend on the structured-config generator aggregating every gateway member's stored hostvars before `pyavd.get_avd_facts()` runs. ### `l2leaf`[​](#l2leaf "Direct link to l2leaf") L2 leaves are BGP-less layer-2 extenders. The hostvars builder **skips**: * L3LS settings (no BGP peering section). * EVPN tenants, VRFs, SVIs. * MLAG (unless explicitly present). It keeps: * Top-level fields (id, role, loopback, mgmt). * Uplink block (upstream `leaf` devices). * `connected_endpoints` for `role = "server"` interfaces. ## `connected_endpoints` — server adapters[​](#connected_endpoints--server-adapters "Direct link to connected_endpoints--server-adapters") For every interface on the device whose `role.value == "server"`, an entry is emitted: ``` { "name": "server-1", "adapters": [ { "endpoint_ports": ["eth0"], "switch_ports": ["Ethernet10"], "switches": ["leaf-pod-A1-1"], "mode": "trunk", "vlans": "100-105" } ] } ``` * `mode: "trunk"` + `vlans: "100-105"` for interfaces with multiple tagged VLANs (formatted via `netutils`). * `mode: "access"` + a single `vlans: "100"` for access-only interfaces. * `native_vlan: 100` added if an untagged VLAN is configured alongside tagged VLANs. * For bonded servers, server `Bond1` is the primary VLAN source. Switch `Port-Channel` VLANs are used when the Bond has no VLAN relationships, and member Ethernet VLANs are only a compatibility fallback. * `spanning_tree_portfast` defaults to `edge` — the AVD convention for host-facing ports. Set `spanning_tree_portfast` on the **switch** interface (`edge` or `network`) to override it; the value is read from the leaf access port, not from the server side. In a Port-Channel the first member expressing an explicit intent wins, since a Port-Channel has one setting. The switchport VLAN itself comes from the server side: `generate-server-cabling` reconciles the server interface's `tagged_vlan` / `untagged_vlan` — including values inherited from a `ProfileDcimInterface` — onto the leaf port it cables. A host access profile that pins one untagged VLAN is what produces `mode: access` on that VLAN. PortFast is not propagated this way, because it is a property of the switch port. ## Pure Layer-2 tenants and tag-scoped VLANs[​](#pure-layer-2-tenants-and-tag-scoped-vlans "Direct link to Pure Layer-2 tenants and tag-scoped VLANs") An `Evpn.Tenant` whose `mac_vrf_vni_base` is unset emits **no** `mac_vrf_vni_base`, so PyAVD derives no VNI, no VXLAN, and no EVPN for it. That is what makes the standalone L2LS design pure Layer-2 (its `l2spine`/`l2leaf` devices are not VTEPs). Overlay tenants that do set a VNI base are unaffected. `Evpn.L2Vlan` has `rack_tags` (→ `LocationRack`) and `avd_tags` (→ `AvdTag`), mirroring the shape already on `Evpn.Svi`. Both are emitted as the VLAN's `tags` list — rack names first, then AVD tag names, deduplicated: ``` l2vlans: - id: 10 name: BLUE-NET tags: [bluezone] ``` AVD matches those against each node's `filter.tags`, which the generator emits on the leaf node-group from the rack's `avd_tags`: ``` l2leaf: node_groups: - group: L2LS_RACK1 filter: tags: [bluezone, greenzone] ``` The result is per-rack VLAN scoping without hand-listing VLANs per switch: tag a VLAN `bluezone`, tag the racks that should carry it, and only those leaf pairs render it. ## AVD custom hostvars escape hatch[​](#avd-custom-hostvars-escape-hatch "Direct link to AVD custom hostvars escape hatch") `avd_custom_hostvars` is an optional JSON attribute on `NetworkFabric`, `NetworkPod`, and `DcimDevice`. It is intended as an escape hatch for PyAVD hostvars that are not yet modeled by the Infrahub schemas and hostvar generator. Custom hostvars are merged in this order: 1. `NetworkFabric.avd_custom_hostvars` 2. `NetworkPod.avd_custom_hostvars` 3. `DcimDevice.avd_custom_hostvars` 4. Generated hostvars from Infrahub-modeled data That means device-level custom values override pod-level custom values, pod-level custom values override fabric-level custom values, and generated hostvars override all custom values. Custom hostvars are fill-only relative to modeled data: they can add keys the generator does not produce, but they cannot replace generated values such as `fabric_name`, role-specific `nodes`, generated tenant data, or generated connected endpoints. Dictionaries merge recursively. Lists and scalar values replace the lower-precedence value as a whole; there is no element-wise list merge. Missing, `null`, or empty custom values are ignored. Non-empty custom values must be mappings; a list or scalar raises `TypeError` before PyAVD validation runs. Example: ``` { "fabric_name": "ignored-custom-name", "custom_structured_configuration_prefix": ["custom"], "l3leaf": { "defaults": { "platform": "7280R3" }, "nodes": [ { "name": "ignored-custom-node" } ] } } ``` In the final hostvars, `custom_structured_configuration_prefix` and `l3leaf.defaults.platform` survive if the generator does not set them. The generated `fabric_name` and generated `l3leaf.nodes` still win. The escape hatch is the delivery mechanism for capabilities the AVD example scenarios need but that are not modeled natively — for example campus dot1x/PoE/port-profiles/in-band management and MPLS/VPN-IPv4 for ISIS-LDP IPVPN. See [Extending the Pipeline → Native schema vs. the escape hatch](/arista-avd/developer-guide/avd/extending.md#native-schema-vs-the-escape-hatch) for when to use it instead of a native schema change. ## Native inputs for the AVD example scenarios[​](#native-inputs-for-the-avd-example-scenarios "Direct link to Native inputs for the AVD example scenarios") The following native schema inputs anchor the AVD example scenarios. They are optional and default to backward-compatible values, so existing designs are unaffected: | Input | Node | Scenario | | ----------------------------------------------------- | --------------- | -------------------------------------------------------------------------------------------------- | | `evpn_vlan_aware_bundles` (Boolean) | `NetworkFabric` | Multi-Pod 5-stage Clos | | `underlay_routing_protocol` values `none`, `isis-ldp` | `NetworkFabric` | Standalone L2LS (`none`), ISIS-LDP IPVPN (`isis-ldp`) | | Roles `l2spine`, `l3spine`, `p`, `pe`, `rr` | `DcimDevice` | L2LS, campus, ISIS-LDP IPVPN — see [Role Mapping](/arista-avd/developer-guide/avd/role-mapping.md) | Generator consumption of these inputs (route-server derivation, standalone L2LS and campus topology generation) is delivered alongside the per-scenario seed designs. ## Validation[​](#validation "Direct link to Validation") Once the dict is built, Phase 1 calls `pyavd.validate_inputs()` on the whole hostvars object. Validation failures are non-recoverable — the generator returns a failure for that device and does **not** write the `AvdHostvarFile`. Common validation failures: * Missing required fields (`id`, `bgp_as`, `loopback_ipv4_address` for L3 roles). * Invalid role name — must be one of the four values in the [Role Mapping](/arista-avd/developer-guide/avd/role-mapping.md) table. * Uplink mismatches (for example, `uplink_interfaces` length ≠ `uplink_switches` length). ## Full leaf example[​](#full-leaf-example "Direct link to Full leaf example") ``` { "type": "l3leaf", "fabric_name": "Fabric-L3LS-MultiPod-A", "id": 1, "bgp_as": "65101", "loopback_ipv4_address": "10.255.1.1", "loopback_ipv4_pool": "10.255.1.0/24", "vtep_loopback_ipv4_address": "10.255.2.1", "vtep_loopback_ipv4_pool": "10.255.2.0/24", "mgmt_ip": "10.255.0.11/24", "mgmt_gateway": "10.255.0.1", "spanning_tree_settings": { "mode": "mstp" }, "l3leaf": { "defaults": { "spanning_tree_priority": 8192 } }, "uplink_interfaces": ["Ethernet1", "Ethernet2"], "uplink_switches": ["spine-A1-1", "spine-A1-2"], "uplink_switch_interfaces": ["Ethernet1", "Ethernet1"], "virtual_router_mac_address": "00:1C:73:00:00:11", "connected_endpoints": [ { "name": "server-1", "adapters": [ { "endpoint_ports": ["eth0"], "switch_ports": ["Ethernet10"], "switches": ["leaf-pod-A1-1"], "mode": "trunk", "vlans": "100-105" } ] } ] } ``` ## Tests[​](#tests "Direct link to Tests") Unit tests cover the hostvars builder and the role→type mapping: * [`tests/unit/test_hostvar_ordering.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/tests/unit/test_hostvar_ordering.py) — hostvars shape and deterministic ordering from `_build_hostvars()`. * [`tests/unit/test_avd.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/tests/unit/test_avd.py) — the `ROLE_TO_AVD_TYPE` / `get_avd_type()` mapping. Full hostvars generation is exercised by integration tests under `tests/integration/`. ## Pool Inputs[​](#pool-inputs "Direct link to Pool Inputs") Hostvars prefer role-driven pool collections. Fabric Point-to-Point uplinks resolve from `NetworkPod.pod_ip_pools` first, then `NetworkFabric.fabric_ip_pools`, then the legacy `NetworkFabric.uplink_pool` relationship. DCI point-to-point links resolve from `NetworkFabric.fabric_ip_pools` role `dci` first, then the legacy `NetworkFabric.dci_pool` relationship, then a deterministic Fabric Supernet fallback for missing required DCI prefix-pool roles. MLAG peer and MLAG L3 peering pools resolve from `NetworkPod.pod_ip_pools` roles `mlag` and `mlag_peering`, then legacy `mlag_peer_pool` and `mlag_l3_pool`. When a required MLAG pool is absent, the generator creates or reuses pod-scoped default pools named `-MLAG-Peer-Subnet` and `-MLAG-L3-Peering-Subnet`, each backed by its own child prefix carved from `169.254.0.0/16` and `192.0.0.0/24` respectively. The pools are per pod and wider than a `/31` on purpose: PyAVD carves a `/31` per MLAG pair, and pods sharing one L3 peering subnet would advertise the same addresses into the underlay from more than one pod. --- # AVD pipeline overview Developer Guide If you want to *use* the system to produce configs, start with [Quick Start](/arista-avd/quick-start.md). The Arista Validated Design (AVD) pipeline transforms Infrahub's network data model into PyAVD-compatible input data, then renders Arista EOS configurations and human-readable documentation from it. ## PyAVD version[​](#pyavd-version "Direct link to PyAVD version") Version-sensitive The integration targets **pyavd >= 6.3.0, < 6.4.0** (pinned in [`pyproject.toml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/pyproject.toml)). The following sections are version-sensitive — review them when upgrading PyAVD: * [Hostvars Reference](/arista-avd/developer-guide/avd/hostvars.md) — the PyAVD input schema. * [Role Mapping](/arista-avd/developer-guide/avd/role-mapping.md) — AVD device type names (for example, `l3leaf`, `super-spine`). * [Transforms](/arista-avd/developer-guide/avd/transforms.md) — the PyAVD functions the transforms call (`validate_inputs`, `get_avd_facts`, `get_device_structured_config`, `get_device_config`, `get_fabric_documentation`). ## The two-phase pipeline[​](#the-two-phase-pipeline "Direct link to The two-phase pipeline") ### Phase 1 — Hostvars[​](#phase-1--hostvars "Direct link to Phase 1 — Hostvars") **Generator**: [`generate-avd-device-hostvar`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_hostvar.py) **Target**: each `DcimDevice` in the `avd_devices` group (one task per device). For each device the generator: 1. Extracts device attributes — name, role, BGP ASN, node ID, loopback, management IP. 2. Determines the **uplink role** based on the device's role: `spine → super_spine` interfaces, `leaf` and `border_leaf → spine` interfaces, `l2leaf → leaf` interfaces, `super_spine →` no uplinks. 3. Extracts connected endpoints (servers) from interfaces with `role = "server"`, including tagged/untagged VLANs. 4. For leaves, extracts the MLAG peer information, then the virtual router MAC. 5. For leaves and spines, queries EVPN tenants, VRFs, SVIs, and L2 VLANs associated with the fabric (skipped entirely for `l2leaf`). 6. For Border Leafs, evaluates `NetworkLink` objects with `role=dci` in the fabric and emits valid links as profile-free PyAVD `l3_edge.p2p_links` entries. DCI addressing resolves from `NetworkFabric.fabric_ip_pools` role `dci` first, falls back to the legacy `NetworkFabric.dci_pool`, and then uses deterministic Fabric Supernet fallback when the required DCI prefix-pool role is missing. 7. Builds a complete PyAVD `hostvars` dict (see [Hostvars Reference](/arista-avd/developer-guide/avd/hostvars.md)). 8. Serialises to JSON, computes a SHA256 checksum, and compares against the previous content. If changed (or absent), writes a new `AvdHostvarFile` as a child of the device's `AvdArtifact` node. ### Phase 2 — structured config[​](#phase-2--structured-config "Direct link to Phase 2 — structured config") **Generator**: [`generate-avd-device-structured-config`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_structured_config.py) **Target**: each `NetworkFabric` in the `fabrics` group (one task per fabric). For the fabric the generator: 1. Walks the fabric hierarchy (`fabric → pods → devices`, `fabric → pods → racks → devices`) to collect every device. 2. Verifies each device has a hostvar artifact; fails fast if any is missing (meaning Phase 1 didn't complete for that device). 3. Fetches the hostvars JSON for every device. 4. Calls `pyavd.validate_inputs()` across all hostvars. 5. Calls `pyavd.get_avd_facts()` once for the fabric to derive shared facts (routed-uplink allocations, VLAN assignments, etc.). 6. For each device, calls `pyavd.get_device_structured_config(hostvars, facts)` and gets a dict of structured AVD config. 7. Serialises to JSON, computes a SHA256 checksum, and compares against the previous content. If changed (or absent), writes a new `AvdStructuredConfigFile` as a child of the device's `AvdArtifact`. ### Transforms — on demand[​](#transforms--on-demand "Direct link to Transforms — on demand") When an operator opens an AVD artifact in the Infrahub UI, the matching transform runs: * **`avd_eos_config`** — reads `structured_config_file`, calls `pyavd.get_device_config()`, returns `text/plain`. * **`avd_device_doc`** — reads `structured_config_file`, calls the PyAVD device documentation function, returns `text/markdown`. * **`avd_fabric_doc`** — reads hostvars and structured configs for all devices in the fabric, calls `pyavd.get_fabric_documentation()`, returns `text/markdown`. See [Transforms](/arista-avd/developer-guide/avd/transforms.md) for the full transform-by-transform reference. ## Components at a glance[​](#components-at-a-glance "Direct link to Components at a glance") | Generator / Transform | Target | File | | --------------------------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `generate-avd-device-hostvar` | per device | [`generators/generate_avd_device_hostvar.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_hostvar.py) | | `generate-avd-device-structured-config` | per fabric | [`generators/generate_avd_device_structured_config.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_structured_config.py) | | `avd_eos_config` | per device | [`transforms/avd_eos_config.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_eos_config.py) | | `avd_device_doc` | per device | [`transforms/avd_device_doc.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_device_doc.py) | | `avd_fabric_doc` | per fabric | [`transforms/avd_fabric_doc.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_fabric_doc.py) | ## Related pages[​](#related-pages "Direct link to Related pages") * [Hostvars Reference](/arista-avd/developer-guide/avd/hostvars.md) — the exact PyAVD input structure built per role. * [AvdArtifact & File Storage](/arista-avd/developer-guide/avd/artifacts.md) — schema, relationships, and how Phase 1 and Phase 2 share data. * [Role Mapping](/arista-avd/developer-guide/avd/role-mapping.md) — Infrahub roles → AVD device types. * [Transforms](/arista-avd/developer-guide/avd/transforms.md) — detailed transform-by-transform breakdown. * [Extending the Pipeline](/arista-avd/developer-guide/avd/extending.md) — worked examples for adding roles, transform outputs, or hostvar fields. * [Debugging the Pipeline](/arista-avd/developer-guide/avd/debugging.md) — object-store inspection, forced regeneration, single-generator re-runs. --- # Role mapping Developer Guide Role names are **PyAVD-version-sensitive** — see the [overview](/arista-avd/developer-guide/avd/overview.md#pyavd-version) for the pinned version. Infrahub's `DcimDevice.role.value` is a string enum that the hostvars generator maps to a PyAVD `type`. The mapping lives in [`src/solution_arista_avd/avd.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/src/solution_arista_avd/avd.py): ## Table[​](#table "Direct link to Table") | Infrahub role | PyAVD `type` | Primary scenario | | ------------- | ------------- | --------------------------------------- | | `super_spine` | `super-spine` | L3LS, 5-stage Clos | | `spine` | `spine` | L3LS | | `leaf` | `l3leaf` | L3LS | | `border_leaf` | `l3leaf` | Dual-DC (DCI) | | `l2leaf` | `l2leaf` | L3LS access, L2LS access, campus access | | `l2spine` | `l2spine` | Standalone L2LS fabric | | `l3spine` | `l3spine` | L2LS (L3 variant), campus core | | `p` | `p` | ISIS-LDP IPVPN (provider core) | | `pe` | `pe` | ISIS-LDP IPVPN (provider edge) | | `rr` | `rr` | ISIS-LDP IPVPN route reflector | All PyAVD `type` values above are valid entries in the pinned PyAVD default `node_type_keys`. Roles beyond `l3leaf`/`l2leaf` are the schema anchors for the AVD example scenarios; the scenario-specific rendering behaviour is delivered either natively or through the `avd_custom_hostvars` escape hatch — see [Extending the Pipeline](/arista-avd/developer-guide/avd/extending.md). ## The mapping in code[​](#the-mapping-in-code "Direct link to The mapping in code") ``` # src/solution_arista_avd/avd.py ROLE_TO_AVD_TYPE: dict[str, str] = { "super_spine": "super-spine", "spine": "spine", "leaf": "l3leaf", "border_leaf": "l3leaf", "l2leaf": "l2leaf", "l2spine": "l2spine", "l3spine": "l3spine", "p": "p", "pe": "pe", "rr": "rr", } def get_avd_type(role: str) -> str: if role not in ROLE_TO_AVD_TYPE: msg = f"Unknown device role: {role}" raise ValueError(msg) return ROLE_TO_AVD_TYPE[role] ``` An unrecognized role raises `ValueError` at generation time — Phase 1 fails for that device. ## Underlay-driven role selection[​](#underlay-driven-role-selection "Direct link to Underlay-driven role selection") The four non-L3LS example designs do not set spine/leaf roles manually. Instead the upstream generator derives them from the **fabric underlay**, so the same spine/leaf topology renders different device types per design: | Fabric underlay | Spine-tier role | Leaf-tier role | Example design | | --------------- | --------------- | -------------- | ------------------- | | `none` | `l2spine` | `l2leaf` | Standalone L2LS | | `ospf` | `l3spine` | `l2leaf` | Campus | | `isis-ldp` | `p` | `pe` | MPLS ISIS-LDP IPVPN | These come from `SPINE_ROLE_BY_UNDERLAY` and `LEAF_ROLE_BY_UNDERLAY` in [`avd.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/src/solution_arista_avd/avd.py). The selection is **gated to non-L3LS underlays only**: a routed L3LS fabric (underlay `ebgp`) is not in either map, so it falls back to the default `spine` / `leaf` roles. ## MLAG in non-L3LS designs[​](#mlag-in-non-l3ls-designs "Direct link to MLAG in non-L3LS designs") `MLAG_MAIN_TIER_ROLES` (`l2leaf`, `l2spine`, `l3spine`) is the main tier of the non-L3LS designs that forms MLAG pairs. When the fabric underlay is one of `SPINE_UPLINK_UNDERLAYS` (`none`, `ospf`, `isis-ldp`), devices in these roles render node-group / peer-link / MLAG-domain configuration — the same as the L3LS leaf family. The gate leaves the L3LS access-tier `l2leaf` (pure access under EVPN) unaffected. Which generator forms the pair depends on the tier: | Tier | Generator | Peer-link source | | ------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------- | | `l2leaf` (rack tier) | `generate-rack` | Highest-numbered free access ports — the `arista-7050sx3-48yc8c` l2leaf model ships no dedicated `mlag_peer` interfaces | | `l2spine` (pod tier, underlay `none`) | `generate-pod` | Highest-numbered free **super-spine-facing** ports, unused in a standalone L2LS fabric (it has no super-spines) | Both go through the shared `assign_mlag_peer_interfaces` helper on the generator mixin, so the choice is deterministic (ordered by the interface's computed `index`) and idempotent — a re-run converts nothing further. The l2spine pair has **no BGP ASN**: a pure Layer-2 tier runs no BGP. ## Per-tier spanning-tree priorities[​](#per-tier-spanning-tree-priorities "Direct link to Per-tier spanning-tree priorities") `Network.SpanningTreePriority` links a fabric to a per-role MSTP priority. Its `role` dropdown covers `super_spine`, `spine`, `leaf`, `l2leaf` and — for the non-L3LS designs — `l2spine` and `l3spine`. The L2LS example sets `l2spine: 4096` / `l2leaf: 16384`, which the hostvars generator emits as each tier's `spanning_tree_priority`. ## Role implications[​](#role-implications "Direct link to Role implications") The role governs these downstream behaviors in the hostvars generator and in PyAVD itself: | Role | Uplink source | Gets EVPN data? | MLAG? | | ------------- | ----------------- | ---------------------------------------------------------------------------------------------- | ----------------- | | `super_spine` | — (top of fabric) | No | No | | `spine` | `super_spine` | No | No | | `leaf` | `spine` | Yes | Yes (if peer set) | | `border_leaf` | `spine` | Yes, including DCI links and EVPN Gateway when the device is a member of an `EvpnGatewayGroup` | Yes (if peer set) | | `l2leaf` | `leaf` | No (skipped) | Yes (if peer set) | See [Hostvars Reference](/arista-avd/developer-guide/avd/hostvars.md) for exactly which fields each role emits. ## Tests[​](#tests "Direct link to Tests") The role mapping is exercised by: * [`tests/unit/test_avd.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/tests/unit/test_avd.py) — covers `get_avd_type()` for each role and the `ValueError` on unknown roles. ## Adding a new role[​](#adding-a-new-role "Direct link to Adding a new role") See [Extending the Pipeline → Add a new device role](/arista-avd/developer-guide/avd/extending.md#add-a-new-device-role). --- # AVD transforms Developer Guide Documents the transform implementations. To *view* artifacts as an operator, see [Viewing Artifacts](/arista-avd/viewing-artifacts.md). Four Python transforms turn the data produced by the [two-phase pipeline](/arista-avd/developer-guide/avd/overview.md) into user-facing artifacts. All four are registered in [`.infrahub.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/.infrahub.yml). Generated Pydantic models The `*_query.py` files referenced below are **generated** from their matching `.gql` and the checked-in `schema.graphql` via `infrahubctl graphql generate-return-types`. Do not hand-edit them. See [Transforms → Query Classes](/arista-avd/developer-guide/transforms.md#query-classes) for the regeneration command. | Transform | Target group | Content type | Wraps | | ------------------ | ------------- | ------------------ | ---------------------------------- | | `avd_eos_config` | `avd_devices` | `text/plain` | `pyavd.get_device_config()` | | `avd_device_doc` | `avd_devices` | `text/markdown` | PyAVD device documentation | | `avd_fabric_doc` | `fabrics` | `text/markdown` | `pyavd.get_fabric_documentation()` | | `avd_anta_catalog` | `avd_devices` | `application/yaml` | `pyavd.get_device_test_catalog()` | ## `avd_eos_config`[​](#avd_eos_config "Direct link to avd_eos_config") **Class**: `AvdEosConfigTransform` **Source**: [`transforms/avd_eos_config.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_eos_config.py) **Query**: [`transforms/avd_device_config.gql`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_device_config.gql) **Pydantic model**: `transforms/avd_device_config_query.py` Renders a single device's Arista EOS CLI configuration. Flow: 1. Query resolves the target device and navigates to `AvdArtifact.structured_config_file`. 2. Transform fetches the structured-config JSON from the `AvdStructuredConfigFile` (a `CoreFileObject`). 3. Calls `pyavd.get_device_config(structured_config)`. 4. Returns the EOS CLI text. If `structured_config_file` is missing or empty, the transform returns a user-readable "No structured config available" message rather than crashing — see [Debugging the Pipeline](/arista-avd/developer-guide/avd/debugging.md#missing-structured-config) for the diagnostic flow. ## `avd_device_doc`[​](#avd_device_doc "Direct link to avd_device_doc") **Class**: `AvdDeviceDocTransform` **Source**: [`transforms/avd_device_doc.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_device_doc.py) **Query**: `transforms/avd_device_config.gql` (reused) **Pydantic model**: `transforms/avd_device_config_query.py` Renders per-device markdown documentation. Flow: 1. Same query as `avd_eos_config` — resolves device and its structured config. 2. Calls the PyAVD device documentation function on the structured config. 3. Returns markdown. ## `avd_fabric_doc`[​](#avd_fabric_doc "Direct link to avd_fabric_doc") **Class**: `AvdFabricDocTransform` **Source**: [`transforms/avd_fabric_doc.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_fabric_doc.py) **Query**: [`transforms/avd_fabric_devices.gql`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_fabric_devices.gql) **Pydantic model**: `transforms/avd_fabric_devices_query.py` Renders fabric-wide markdown documentation covering the full topology. Flow: 1. Query resolves the fabric and walks to every device in its pods and racks. 2. Transform fetches **hostvars** and **structured config** files for all devices. 3. Calls `pyavd.get_avd_facts(all_hostvars)` to build the shared facts. 4. Calls `pyavd.get_fabric_documentation(avd_facts, structured_configs, fabric_name)`. 5. Returns markdown. Fabric documentation requires hostvars to be present for *every* device in the fabric. If any device has no hostvars, the transform fails the artifact generation with a message naming the missing devices. ## `avd_anta_catalog`[​](#avd_anta_catalog "Direct link to avd_anta_catalog") **Class**: `AvdAntaCatalogTransform` **Source**: [`transforms/avd_anta_catalog.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_anta_catalog.py) **Query**: [`transforms/avd_anta_catalog.gql`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_anta_catalog.gql) **Pydantic model**: `transforms/avd_anta_catalog_query.py` Renders a per-device [ANTA](https://anta.arista.com) test catalog as YAML. Flow: 1. Query resolves the `target` device *and* every device in the graph, so siblings can be filtered by fabric in the transform. The device's fabric is `pod.parent`, a discriminated union — only a `NetworkFabric` parent has the `name` and `anta_enabled` fields the gating needs. 2. If the fabric has `anta_enabled` unset or false, return a marker comment and stop. 3. Download each same-fabric device's structured config, passing it through `pyavd.validate_structured_config()`. 4. Build one `AVDFabricData` from all of them — catalog generation is fabric-wide, unlike EOS config rendering. 5. Call `pyavd.get_device_test_catalog(hostname, target_structured_config, fabric_data)` and dump it as YAML. Every "cannot render" path returns a comment rather than raising, so the artifact always renders and states the reason: ``` # ANTA disabled for fabric Fabric-L3LS-Multi-Domain # No structured config for leaf-infrahub-dc1-1 # ANTA catalog: no fabric for leaf-infrahub-dc1-1 ``` ## Registration in `.infrahub.yml`[​](#registration-in-infrahubyml "Direct link to registration-in-infrahubyml") ``` python_transforms: - name: avd_eos_config class_name: AvdEosConfigTransform file_path: "./transforms/avd_eos_config.py" - name: avd_fabric_doc class_name: AvdFabricDocTransform file_path: "./transforms/avd_fabric_doc.py" - name: avd_device_doc class_name: AvdDeviceDocTransform file_path: "./transforms/avd_device_doc.py" - name: avd_anta_catalog class_name: AvdAntaCatalogTransform file_path: "./transforms/avd_anta_catalog.py" artifact_definitions: - name: avd_eos_configuration targets: avd_devices transformation: avd_eos_config - name: avd_fabric_documentation targets: fabrics transformation: avd_fabric_doc - name: avd_device_documentation targets: avd_devices transformation: avd_device_doc - name: avd_anta_catalog targets: avd_devices transformation: avd_anta_catalog content_type: application/yaml ``` ## Other transforms[​](#other-transforms "Direct link to Other transforms") The repository also ships transforms outside the AVD pipeline — the cabling-plan CSV, computed interface descriptions, the ContainerLab topology, and the CloudVision webhook payload. They are documented in [Transforms](/arista-avd/developer-guide/transforms.md). ## Adding a new transform[​](#adding-a-new-transform "Direct link to Adding a new transform") See [Extending the Pipeline → Adding a new transform output](/arista-avd/developer-guide/avd/extending.md#add-a-new-transform-output). --- # Checks Developer Guide Documents the check implementations. For the operator view of CloudVision validation — credentials, workspace tracking, and submission — see [CloudVision Validation](/arista-avd/cloudvision.md). ## Overview[​](#overview "Direct link to Overview") A check is a Python routine that Infrahub runs during proposed-change validation. Unlike a transform, it produces no artifact: it reports success, informational messages, or errors, and an error blocks the proposed change. Checks live in `checks/` and are registered under `check_definitions:` in `.infrahub.yml`: ``` check_definitions: - name: cv-config-validation file_path: "./checks/cv_config_check.py" class_name: CVConfigValidationCheck targets: fabrics parameters: name: name__value ``` `targets` is a group, exactly as for generators and artifact definitions — `cv-config-validation` runs once per member of the `fabrics` group, with the fabric name passed as the `name` parameter. ## `cv-config-validation`[​](#cv-config-validation "Direct link to cv-config-validation") **Class**: `CVConfigValidationCheck` **Source**: [`checks/cv_config_check.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/checks/cv_config_check.py) **Query**: [`checks/cv_config_check.gql`](https://github.com/opsmill/infrahub-arista-avd/blob/main/checks/cv_config_check.gql) (registered as `cv_config_check`) **Target**: `NetworkFabric` (group `fabrics`) **Timeout**: 600 seconds The check deploys each device's rendered EOS configuration into a CloudVision workspace and asks CloudVision to build it, so a reviewer sees CloudVision's own verdict on the configuration before the branch merges. ``` class CVConfigValidationCheck(InfrahubCheck): query = "cv_config_check" timeout = 600 async def validate(self, data: dict[str, Any]) -> None: parsed = CVConfigCheckQuery(**_normalize_optional_relationships(data)) ... ``` The flow, in order: 1. Resolve the target fabric from the query response. A fabric with `cloudvision_managed = false` skips everything that follows. 2. Read CloudVision credentials from the task-worker environment (`get_cloudvision_config()`), and authenticate. 3. Require every confirmed device in the fabric to have a serial number and to exist — and be active — in CloudVision inventory. 4. Select the devices that have a stored `AvdStructuredConfigFile`, download each one, and render EOS CLI with `pyavd.get_device_config()`. 5. Deploy the configs to a deterministic workspace for this proposed change and fabric, and request a build. 6. Record the workspace as a `CloudvisionWorkspace` object and post its URL to a proposed-change thread. Download, JSON-decode, render, connection, deployment, and build failures are all reported as errors and block the proposed change. The behavioural detail — eligibility rules, workspace naming and reuse, thread comments, and what happens when the tracking schema is absent — is documented on the [CloudVision Validation](/arista-avd/cloudvision.md) page rather than duplicated here. ### Optional relationships and the generated query model[​](#optional-relationships-and-the-generated-query-model "Direct link to Optional relationships and the generated query model") `_normalize_optional_relationships()` runs before the generated Pydantic model parses the response. GraphQL omits nullable relationship selections entirely rather than returning `null`, so a device with no pod or no `avd_artifact` arrives with the key missing. The helper fills those keys with `{"node": None}` so an absent relationship parses as absent membership instead of failing validation. ### Supporting modules[​](#supporting-modules "Direct link to Supporting modules") | Module | Holds | | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | [`checks/cv_helpers.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/checks/cv_helpers.py) | Credential loading from the environment, deterministic workspace ID/name/description/URL derivation, proposed-change context lookup, workspace rollback | | [`checks/cv_workspace_lifecycle.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/checks/cv_workspace_lifecycle.py) | Workspace threads and comments, and the two submission entry points below | The two submission entry points in `cv_workspace_lifecycle.py`: * `submit_linked_workspace_for_custom_webhook()` — the CustomWebhook entry point. It extracts the proposed-change ID and branch from the event, ignores events that name a check other than `cv-config-validation`, and delegates to the shared handler. * `submit_linked_workspace_for_proposed_change()` — the shared handler. It resolves `CloudvisionWorkspace` objects by proposed-change ID and submits only when exactly one linked workspace exists in a submit-ready state (`built` or `submit_failed`). The manual retry path for the same handler is an invoke task: ``` uv run invoke submit-cv-workspace --proposed-change-id --branch main ``` ### The webhook payload transform[​](#the-webhook-payload-transform "Direct link to The webhook payload transform") `cv_workspace_submission_webhook_payload` (`CVWorkspaceSubmissionWebhookPayload`, in `transforms/cv_workspace_submission_webhook.py`) renders the CustomWebhook body. It is a transform rather than a check, but it belongs to this pipeline: it returns the check name, the proposed-change ID, and one entry per linked workspace with its ID, status, URL, and fabric name. See [Transforms](/arista-avd/developer-guide/transforms.md#cvworkspacesubmissionwebhookpayload). ## `fabric-pool-validation`[​](#fabric-pool-validation "Direct link to fabric-pool-validation") **Class**: `FabricPoolValidationCheck` **Source**: [`checks/fabric_pool_check.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/checks/fabric_pool_check.py) **Query**: [`checks/fabric_pool_check.gql`](https://github.com/opsmill/infrahub-arista-avd/blob/main/checks/fabric_pool_check.gql) (registered as `fabric_pool_check`) **Target**: `NetworkFabric` (group `fabrics`) The check validates the role-driven pool collections the generators read — `NetworkFabric.fabric_ip_pools` and `NetworkPod.pod_ip_pools` — so a pool mistake fails the proposed change instead of producing wrong addressing at generation time. See [Pool role resolution](/arista-avd/developer-guide/generators.md#pool-resolution) for how the generators consume the same collections. Per fabric: 1. Every member of `fabric_ip_pools` must be a `CoreIPAddressPool` or `CoreIPPrefixPool`. 2. Each pool's purpose is resolved from the `IpamPrefix.role` values on its resources; a pool with mixed authoritative roles is an error, and two pools claiming the same role in one fabric is an error. 3. The required role set depends on fabric intent — the underlay and overlay routing protocols, and whether the fabric has DCI links. A required role satisfied only through a legacy relationship (for example `NetworkFabric.uplink_pool`) is reported as information, not an error, so migration can proceed incrementally. 4. A required role with no pool and no Fabric Supernet to carve one from is an error. When a Fabric Supernet does exist, its remaining free space is checked against the prefixes the missing roles would need. Per pod: 1. Members of `pod_ip_pools` are type- and role-checked as above, and the `mlag` and `mlag_peering` roles must be address pools rather than prefix pools. 2. MLAG roles required by the pod — driven by the parent fabric's underlay protocol and whether any rack enables MLAG — must be present through `pod_ip_pools` or a legacy pod relationship. 3. Pod Loopback, Loopback VTEP, and Fabric Point-to-Point prefixes must be contained by the matching fabric pool. A role the fabric leaves to its Fabric Supernet to carve on demand has nothing to contain against yet and is skipped rather than reported. Unit coverage is in [`tests/unit/test_fabric_pool_check.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/tests/unit/test_fabric_pool_check.py). ## Schema[​](#schema "Direct link to Schema") `schemas/cv/cv.yml` defines `CloudvisionWorkspace` — `Cloudvision.Workspace` — the node the check writes its workspace tracking to. See [Schemas](/arista-avd/developer-guide/schemas.md#cloudvisionworkspace--cloudvisionworkspace). ## Running and testing a check[​](#running-and-testing-a-check "Direct link to Running and testing a check") Run it against a fabric from the CLI: ``` # Variables are passed as key=value; this check takes the fabric name. uv run infrahubctl check cv-config-validation name=Fabric-L3LS-Multi-Domain --branch # List the checks the repository defines uv run infrahubctl check --list ``` In the UI, a check runs automatically as part of proposed-change validation; its result appears under the proposed change's **Checks** tab. Unit and integration coverage is in [`tests/unit/test_cv_integration.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/tests/unit/test_cv_integration.py). Because the check reaches CloudVision through PyAVD's `CVClient`, tests exercise it with that client stubbed rather than against a live CloudVision instance. ## Adding a check[​](#adding-a-check "Direct link to Adding a check") 1. Write the GraphQL query under `checks/` and register it in the `queries:` block of `.infrahub.yml`. 2. Regenerate the matching `*_query.py` — do not hand-write it: ``` uv run infrahubctl graphql generate-return-types checks/my_check.gql ``` 3. Implement a class deriving from `InfrahubCheck`, setting `query` and implementing `async def validate(self, data)`. Report through `self.log_info()` and `self.log_error()`; an error fails the check. 4. Register it under `check_definitions:` with its `targets` group and `parameters`. 5. Add tests under `tests/unit/`. ## Source[​](#source "Direct link to Source") * Checks: [`checks/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/checks) * Registration: [`.infrahub.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/.infrahub.yml) — `check_definitions:` block. * Operator documentation: [CloudVision Validation](/arista-avd/cloudvision.md). --- # Concepts This reference design is built on Infrahub. If you come from an AVD-and-files background, these are the platform terms used throughout the docs. Each links to the upstream [Infrahub documentation](https://docs.infrahub.app) for full detail. ## Branch[​](#branch "Direct link to Branch") A named, isolated line of change over the whole data model — like a Git branch, but for the graph. Every change set in this reference design happens on a branch: the generator chain and the service portal both create and operate on branches, so parallel work does not interfere. See [Infrahub branches](https://docs.infrahub.app/branches/overview). ## Proposed change[​](#proposed-change "Direct link to Proposed change") A request to merge a branch into `main`, with a full diff of the data and the rendered artifacts it affects. Reviewers see exactly what a change does — new devices, reallocated addresses, changed EOS config — before it reaches production. This is the review gate for every fabric change. See [proposed changes](https://docs.infrahub.app/proposed-changes/overview). ## Generator[​](#generator "Direct link to Generator") A Python routine that reads high-level design intent and creates the many detailed objects it implies — devices, interfaces, links, and address/number allocations. In this reference design, `FabricGenerator`, `PodGenerator`, and `RackGenerator` expand a fabric design into the full technical model, and the AVD generators assemble per-device host\_vars and structured configuration. Generators are idempotent (see below). See the [Generators](/arista-avd/developer-guide/generators.md) reference and [Infrahub generators](https://docs.infrahub.app/generators/overview). ## Transform[​](#transform "Direct link to Transform") A routine that converts stored data into an output document, saved as an artifact. Here, transforms render EOS CLI configuration, fabric and per-device Markdown documentation, the cabling-plan CSV, the ANTA test catalog, and the ContainerLab topology — with PyAVD running inside Infrahub's workers. See the [Transforms](/arista-avd/developer-guide/transforms.md) reference. ## Check[​](#check "Direct link to Check") A Python routine that Infrahub runs during proposed-change validation. It writes no artifact — it reports pass, information, or error, and an error blocks the merge. This reference design ships one: `cv-config-validation`, which deploys the rendered EOS configs into a CloudVision workspace and reports CloudVision's build result back to the proposed change. See the [Checks](/arista-avd/developer-guide/checks.md) reference and [CloudVision Validation](/arista-avd/cloudvision.md). ## Artifact[​](#artifact "Direct link to Artifact") A generated, downloadable output stored in the object store and versioned with the data that produced it — a rendered EOS config, a documentation file, a cabling CSV. An artifact regenerates when its underlying data or its transform changes, so it never drifts from the source of truth. The `AvdArtifact` node tracks each device's host\_vars and structured config with checksums. See [AvdArtifact & File Storage](/arista-avd/developer-guide/avd/artifacts.md). ## Resource pool[​](#resource-pool "Direct link to Resource pool") A managed range — IP prefixes, IP addresses, BGP ASNs, or node IDs — that hands out unique values on demand. Pools in this reference design are **branch-aware**, so two engineers working on different branches never allocate the same prefix, ASN, or node ID. See [Infrahub resource manager](https://docs.infrahub.app/resource-manager/overview). ## Checksum-based idempotency[​](#checksum-based-idempotency "Direct link to Checksum-based idempotency") Each generator records a checksum of the objects it depends on. On re-run, if nothing relevant changed, the generator skips its work; if only part of the design changed, it regenerates only the affected objects. This makes re-running after a partial failure safe, and limits each day-two change to the objects it actually affects. See [Regenerate a Fabric](/arista-avd/how-to/regenerate-fabric.md). --- # Generators Developer Guide Explains how the generators are structured. To *run* generators as an operator, start with [Quick Start](/arista-avd/quick-start.md). ## Overview[​](#overview "Direct link to Overview") Generators create infrastructure objects based on templates and target objects. They run via the Infrahub UI or API and use checksums for idempotent execution. ## Generator architecture[​](#generator-architecture "Direct link to Generator architecture") Each generator consists of: 1. **Generator Class** (`generate_*.py`) - Python class extending `InfrahubGenerator` 2. **Query Class** (`*_query.py`) - Pydantic models for GraphQL response parsing 3. **GraphQL Query** (`*.gql`) - Query to fetch target data ``` ┌──────────────────┐ ┌────────────────────┐ ┌─────────────────┐ │ GraphQL Query │ ──▶ │ Pydantic Parser │ ──▶ │ Generator │ │ (*.gql) │ │ (*_query.py) │ │ (generate_*.py)│ └──────────────────┘ └────────────────────┘ └─────────────────┘ ``` ## Device-design-driven generation[​](#device-design-driven-generation "Direct link to Device-design-driven generation") The fabric, pod, and rack generators take device counts and templates from their container's `device_designs` relationship — not from per-role fields on the container. Each design has a `role`, a `device_quantity`, and a `device_template`; see [Schemas](/arista-avd/developer-guide/schemas.md#device-design-entities--networkdevicedesign-generic) for the entity itself. Every generator resolves designs through the same `GeneratorMixin` helper: ``` # Which super-spines should this fabric have? template_id, quantity = self.device_design_for(fabric_node.device_designs, "super_spine") ``` `device_design_for` returns `(template_id, quantity)`, or `(None, 0)` when the container has no design for that role. **Absence means none**: a rack with no `l2leaf` design gets no L2 leaves, and the generator does not error. This replaces the older "set the count to `0`" idiom. Which role each tier reads: | Generator | Container | Design roles read | | ----------------- | --------------- | ----------------- | | `FabricGenerator` | `NetworkFabric` | `super_spine` | | `PodGenerator` | `NetworkPod` | `spine` | | `RackGenerator` | `LocationRack` | `leaf`, `l2leaf` | ### Cross-tier completeness reads[​](#cross-tier-completeness-reads "Direct link to Cross-tier completeness reads") A generator also reads the *upstream* container's designs to decide whether its prerequisites exist yet, so a partially generated fabric defers instead of producing a half-cabled topology: * `PodGenerator` reads the fabric's `super_spine` design. If the fabric expects super-spines but they do not all exist yet, the pod generator waits rather than cabling spines to an incomplete super-spine layer. A fabric with no `super_spine` design skips super-spine uplinks entirely. * `RackGenerator` reads the pod's `spine` design and compares it to the spines that exist, applying the same rule before cabling leaves upward. These reads are why the generator `.gql` queries select `device_designs` on the parent as well as on the target. ## Generators[​](#generators-1 "Direct link to Generators") ### FabricGenerator[​](#fabricgenerator "Direct link to FabricGenerator") **File**: `generators/generate_fabric.py` **Target**: `NetworkFabric` **Purpose**: Initialize fabric infrastructure **Actions**: 1. Resolve fabric-scoped pools * `loopback_pool` for device Loopback0 addresses * `vtep_pool` for VTEP loopback addresses * `mgmt_pool` for management addresses * `asn_pool` for BGP autonomous systems * `node_id_pool` for unique device identifiers 2. Create super-spine devices from the fabric's `super_spine` device design 3. Assign loopback IPs to super-spines **Query**: `generate_fabric.gql` ``` query FabricGenerator($fabric_id: String!) { NetworkFabric(ids: [$fabric_id]) { edges { node { id name { value } supernet_pool { value } # ... pool and template data } } } } ``` ### PodGenerator[​](#podgenerator "Direct link to PodGenerator") **File**: `generators/generate_pod.py` **Target**: `NetworkPod` **Purpose**: Create pod infrastructure **Actions**: 1. Create spine devices from the pod's `spine` device design 2. Link spines to super-spines 3. Allocate loopback IPs from pod pools 4. Set BGP ASN and node IDs **Query**: `generate_pod.gql` ``` query PodGenerator($pod_id: String!) { NetworkPod(ids: [$pod_id]) { edges { node { id name { value } fabric { node { ... } } # ... template and pool data } } } } ``` ### RackGenerator[​](#rackgenerator "Direct link to RackGenerator") **File**: `generators/generate_rack.py` **Target**: `LocationRack` **Purpose**: Create rack infrastructure **Actions**: 1. Create leaf and L2-leaf devices from the rack's `leaf` / `l2leaf` device designs 2. Link leaves to pod spines 3. Allocate loopback IPs 4. Set BGP ASN and node IDs **Query**: `generate_rack.gql` ``` query RackGenerator($rack_id: String!) { LocationRack(ids: [$rack_id]) { edges { node { id name { value } pod { node { ... } } # ... device and link data } } } } ``` ### GenerateAVDDeviceHostvar[​](#generateavddevicehostvar "Direct link to GenerateAVDDeviceHostvar") **File**: `generators/generate_avd_device_hostvar.py` **Target**: `DcimDevice` **Purpose**: Generate PyAVD hostvars for each device **Actions**: 1. Extract device attributes (hostname, role, ASN, node ID) 2. Extract IP addresses (loopback, management) 3. Determine uplink topology by device role 4. Extract connected endpoints (servers with VLANs) 5. Validate optional EVPN Gateway group intent for `border_leaf` devices 6. Build PyAVD-compatible hostvars structure 7. Upload hostvars JSON to object store 8. Create/update AvdArtifact with checksum **Query**: `avd_device_hostvar.gql` For EVPN Multi-Domain Gateway hostvars, the query fetches `EvpnGatewayGroup.local_domain`, the selected `pod` and its `evpn_domain`, `remote_domain`, members, and peer candidate groups from `remote_domain.remote_gateway_groups`. The generator emits `l3leaf.nodes[].evpn_gateway` only for valid grouped `border_leaf` devices, rejects Pod/local-domain mismatches and same local/remote domain intent, validates the final payload with `pyavd.validate_inputs()`, and derives hostname-only remote peers from valid groups that share the selected remote domain. ### AvdDeviceStructuredConfigGenerator[​](#avddevicestructuredconfiggenerator "Direct link to AvdDeviceStructuredConfigGenerator") **File**: `generators/generate_avd_device_structured_config.py` **Target**: `NetworkFabric` **Purpose**: Generate AVD structured configs for all fabric devices **Actions**: 1. Traverse fabric hierarchy (pods → devices, racks → devices) 2. Fetch hostvars from object store for each device 3. Validate inputs with `pyavd.validate_inputs()` 4. Generate AVD facts with `pyavd.get_avd_facts()` 5. Generate structured config per device 6. Upload configs to object store 7. Update AvdArtifact with config identifier **Query**: `generate_avd.gql` ### ServerCablingGenerator[​](#servercablinggenerator "Direct link to ServerCablingGenerator") **File**: `generators/generate_server_cabling.py` **Target**: `ComputePhysicalServer` (group `servers`) **Purpose**: Cable a server to the leaf switches in its rack, then reconcile its LAGs and VLANs **Actions**: 1. Resolve the server's rack and find the `leaf` / `l2leaf` switches in it 2. Build sorted interface maps for the server and for the leaves' `role=server` interfaces 3. On first run, pick the next free port index across those leaves and create the links; on a re-run, rebuild the existing cabling plan instead of cabling again 4. For a dual-homed server, create the server-side `Bond1` and the switch-side `Port-Channel` LAGs, assign members, and set `evpn_ethernet_segment` when the pair is not MLAG-backed 5. Assign VLANs from server intent — on a single-homed server they stay on the physical interface, on a dual-homed server they are assigned to the LAG 6. Trigger AVD hostvar regeneration for the leaves the server connects to **Query**: `generate_server_cabling.gql` Because steps 3–5 run on every invocation, the generator is the reconciliation path as well as the creation path: re-running it after a VLAN or LAG change updates an already-cabled server without producing duplicate links. The operator-facing walkthrough is [Add a Server](/arista-avd/how-to/add-server.md). ### BackfillStructuredConfigGenerator[​](#backfillstructuredconfiggenerator "Direct link to BackfillStructuredConfigGenerator") **File**: `generators/backfill_structured_config.py` **Target**: `AvdStructuredConfigFile` (group `avd_structured_configs`) **Purpose**: Read AVD's structured-config output back into the Infrahub data model **Actions**: Parse each device's stored structured config and upsert the objects it implies — `IpamPrefix` and `IpamIPAddress` entries, `DcimInterface.mtu`, BGP peer groups and neighbors, prefix lists, route maps, and static routes. **Query**: `backfill_structured_config.gql` This generator runs in the opposite direction to the rest of the chain: everything else turns intent into AVD inputs, while the backfill turns AVD's derived output into queryable objects. Those objects are reconciled *from* AVD, not authored as inputs — see [Supported Capabilities](/arista-avd/supported-capabilities.md). ## Generator execution order[​](#generator-execution-order "Direct link to Generator execution order") Run generators in this order for a new fabric: ``` 1. FabricGenerator (on Fabric) ↓ 2. PodGenerator (on each Pod) ↓ 3. RackGenerator (on each Rack) ↓ 4. AVD Hostvars (on each Device) ↓ 5. AVD Structured Cfg (on Fabric) ``` For an existing fabric, `generate-fabric` is also the reconciliation entry point. Checksum changes still drive the existing trigger rules: a changed pod checksum fires `generate-pod`, and a changed rack checksum fires `generate-rack`. When a pod or rack checksum is already current, the upstream generator explicitly continues the cascade with `CoreGeneratorDefinitionRun` targeted to the unchanged node IDs. This keeps repeated fabric runs from faking checksum churn while still reaching pod, rack, hostvar, and structured-config generation. The fabric generator skips direct continuation for the fabric-role pod because that pod is owned by `FabricGenerator` for super-spine creation. Pod generation uses the same pattern for racks: changed racks rely on checksum-trigger saves; unchanged racks are scheduled directly. Device reconciliation is fill-only by default. `GeneratorMixin.create_avd_device()` fetches any existing device by name before building the upsert payload, then populates missing generator-owned values such as status, role, object template, pod, rack, index, AVD group membership, node ID, management IP, loopback IP, VTEP loopback IP, and ASN. Existing non-empty operator values, including `serial` and `mgmt_ip`, are preserved during standard generation. ## Running generators[​](#running-generators "Direct link to Running generators") ### Via Infrahub UI[​](#via-infrahub-ui "Direct link to Via Infrahub UI") 1. Navigate to target object (Fabric, Pod, Rack, or Device) 2. Click **Actions** → **Generator definitions** 3. Select the generator 4. Click **Run** ### Via infrahubctl CLI[​](#via-infrahubctl-cli "Direct link to Via infrahubctl CLI") The CLI takes the generator name followed by `key=value` variables — the parameters declared for that generator in `.infrahub.yml`. Every generator here is parameterised by `name`, so the value is the target object's name, not its ID: ``` uv run infrahubctl generator generate-fabric name=Fabric-L3LS-MultiPod-A --branch uv run infrahubctl generator generate-pod name=Pod-A2 --branch uv run infrahubctl generator generate-rack name=Rack-A2-1 --branch uv run infrahubctl generator generate-avd-device-hostvar name=leaf-pod-a2-1-1 --branch # List the generators the repository defines uv run infrahubctl generator --list ``` `backfill-structured-config` is the exception: its parameter is the artifact name (`artifact__name__value`), still passed as `name=`. ## GeneratorMixin[​](#generatormixin "Direct link to GeneratorMixin") All generators use `GeneratorMixin` from `src/solution_arista_avd/generator.py`: ``` class GeneratorMixin: def calculate_checksum(self, related_node_ids: list[str]) -> str: """ Calculate deterministic checksum from related node IDs. Used to detect when regeneration is needed. """ sorted_ids = sorted(related_node_ids) combined = "".join(sorted_ids) return hashlib.sha256(combined.encode()).hexdigest() @classmethod def device_design_for(cls, device_designs, role) -> tuple[str | None, int]: """ Return (template_id, quantity) for one role's device design, or (None, 0) when the container has no design for that role. """ async def assign_mlag_peer_interfaces( self, device, count=2, carvable_roles=frozenset({"server", "mlag_peer"}) ) -> None: """ Repurpose a device's highest-numbered carvable ports as its MLAG peer-link, for switch models that ship no dedicated mlag_peer interfaces. Deterministic (ordered by the interface's computed `index`) and idempotent, so a re-run converts nothing further. Used by the rack generator for l2leaf pairs and the pod generator for the l2spine pair. """ ``` Usage in generator: ``` class FabricGenerator(GeneratorMixin, InfrahubGenerator): async def generate(self, data): # Calculate checksum from related nodes new_checksum = self.calculate_checksum([ pod.id for pod in data.pods ]) # Skip if unchanged if new_checksum == data.checksum: return # ... generate infrastructure ... # Update checksum data.checksum = new_checksum await data.save() ``` ## Query classes (Pydantic)[​](#query-classes-pydantic "Direct link to Query classes (Pydantic)") Each generator has a corresponding query class for type-safe parsing. **These `*_query.py` files are generated, not hand-written** — regenerate them whenever the `.gql` query or the schema changes: ``` uv run infrahubctl graphql generate-return-types generators/generate_fabric.gql ``` This reads `schema.graphql` at the repo root (refresh with `uv run infrahubctl graphql export-schema --destination schema.graphql` when needed) and emits the matching `*_query.py` next to the query file. Shape of a typical generated class: ``` # generators/fabric_generator_query.py (generated) from pydantic import BaseModel class FabricNode(BaseModel): id: str name: ValueWrapper[str] supernet_pool: ValueWrapper[str] pods: EdgesWrapper[PodNode] class FabricGeneratorQuery(BaseModel): NetworkFabric: EdgesWrapper[FabricNode] ``` ## Configuration[​](#configuration "Direct link to Configuration") Generators are registered in `.infrahub.yml`: ``` generator_definitions: - name: generate-fabric file_path: "./generators/generate_fabric.py" class_name: FabricGenerator targets: fabrics query: generate_fabric - name: generate-pod file_path: "./generators/generate_pod.py" class_name: PodGenerator targets: pods query: generate_pod - name: generate-rack file_path: "./generators/generate_rack.py" class_name: RackGenerator targets: racks query: generate_rack - name: generate-server-cabling file_path: "./generators/generate_server_cabling.py" class_name: ServerCablingGenerator targets: servers query: generate_server_cabling - name: backfill-structured-config file_path: "./generators/backfill_structured_config.py" class_name: BackfillStructuredConfigGenerator targets: avd_structured_configs query: backfill_structured_config ``` The AVD generators are registered in the same block; the file is the authoritative list of all seven. ## Pool resolution[​](#pool-resolution "Direct link to Pool resolution") The fabric, pod, rack, and hostvars generators consume role-driven pool collections first. `NetworkFabric.fabric_ip_pools` supplies fabric Management, Loopback, Loopback VTEP, Fabric Point-to-Point, DCI, and Fabric Supernet roles. `NetworkPod.pod_ip_pools` can override pod-specific Loopback, Loopback VTEP, and Fabric Point-to-Point pools. If a required fabric prefix pool is missing and a Fabric Supernet pool exists, `GeneratorMixin` creates deterministic fallback prefix pools with stable names such as `-Loopback-Pool`, then wraps Loopback and VTEP prefix pools in address pools for device allocation. Repeated runs upsert the same names. The hostvars generator resolves MLAG and MLAG Peering from `pod_ip_pools`, then legacy pod relationships, then pod-scoped default pools named `-MLAG-Peer-Subnet` and `-MLAG-L3-Peering-Subnet`. Each pod is allocated its own child prefix — a `/24` from `169.254.0.0/16` for the peer-link, a `/28` from `192.0.0.0/24` for L3 peering — because PyAVD carves a `/31` per MLAG pair out of the pool, and MLAG L3 peering addresses are advertised into the underlay. Treat the L3 peering default as a safety net, not a design: define an explicit `mlag_peering` pool. ## File structure[​](#file-structure "Direct link to File structure") ``` generators/ ├── generate_fabric.py # Fabric generator class ├── generate_fabric.gql # Fabric GraphQL query ├── fabric_generator_query.py # Fabric Pydantic models ├── generate_pod.py # Pod generator class ├── generate_pod.gql # Pod GraphQL query ├── pod_generator_query.py # Pod Pydantic models ├── generate_rack.py # Rack generator class ├── generate_rack.gql # Rack GraphQL query ├── rack_generator_query.py # Rack Pydantic models ├── generate_avd_device_hostvar.py # AVD hostvars generator ├── avd_device_hostvar.gql # AVD device query ├── generate_avd_device_structured_config.py # AVD structured config ├── generate_avd.gql # AVD fabric query ├── generate_avd_inputs_query.py # AVD fabric Pydantic models ├── generate_avd_device_inputs_query.py # AVD device Pydantic models ├── generate_server_cabling.py # Server cabling generator ├── generate_server_cabling.gql # Server cabling query ├── server_cabling_query.py # Server cabling Pydantic models ├── backfill_structured_config.py # Structured-config backfill generator ├── backfill_structured_config.gql # Backfill query └── backfill_structured_config_query.py # Backfill Pydantic models ``` ## Source[​](#source "Direct link to Source") * Generator framework: [`src/solution_arista_avd/generator.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/src/solution_arista_avd/generator.py) — `GeneratorMixin` with checksum-based change detection. * Infrastructure generators: * [`generators/generate_fabric.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_fabric.py) — `FabricGenerator`. * [`generators/generate_pod.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_pod.py) — `PodGenerator`. * [`generators/generate_rack.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_rack.py) — `RackGenerator`. * [`generators/generate_server_cabling.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_server_cabling.py) — `ServerCablingGenerator`. * [`generators/backfill_structured_config.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/backfill_structured_config.py) — `BackfillStructuredConfigGenerator`. * AVD generators (documented in detail in the [AVD Pipeline sub-section](/arista-avd/developer-guide/avd/overview.md)): * [`generators/generate_avd_device_hostvar.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_hostvar.py) — `GenerateAVDDeviceHostvar`. * [`generators/generate_avd_device_structured_config.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/generators/generate_avd_device_structured_config.py) — `AvdDeviceStructuredConfigGenerator`. * Registration: [`.infrahub.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/.infrahub.yml) — `generator_definitions:` block. * Tests: [`tests/unit/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/tests/unit) and [`tests/integration/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/tests/integration). --- # Schemas Developer Guide Documents the YAML schema files that define the data model. Every kind in the data model is defined in a YAML file under `schemas/` and loaded with `infrahubctl schema load schemas` (`inv load-schema`). The GraphQL kind is the schema `namespace` joined to its `name` — `Dcim` + `Device` becomes `DcimDevice`. Generics load as GraphQL interfaces; nodes load as GraphQL object types. Regenerate the typed protocol classes after any schema change (see [the command below](#protocols)). ## Schema files[​](#schema-files "Direct link to Schema files") | File | Defines | | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `base/dcim.yml` | `Dcim.GenericDevice`, `Dcim.PhysicalDevice`, `Dcim.Device`, interface generics/nodes, `Dcim.DeviceType` (incl. `containerlab_interface_mapping`), `Dcim.Platform` (incl. `containerlab_os`, `containerlab_image`) | | `base/ipam.yml` | `Ipam.IPAddress`, `Ipam.Prefix` base definitions | | `base/location.yml` | `Location.Generic`, `Location.Hosting` base definitions | | `base/organization.yml` | `Organization.Generic`, `Organization.Manufacturer`, `Organization.Provider` | | `logical_design.yml` | `Network.Fabric` (incl. `cloudvision_managed`), `Network.Pod`, `Network.BuildingBlock` | | `device_design.yml` | `Network.DeviceDesign` generic plus the fabric/pod/rack device-design nodes | | `dcim_extensions.yml` | `Network.Link`, including `role=dci` and DCI link fields, plus device extensions (`role`, BGP ASN relationship, `node_id`, loopback/mgmt, pod/rack relations) and the interface `role`/`description`/`ip_address` extensions | | `dci.yml` | `NetworkFabric.dci_pool` DCI addressing source | | `l3ls_extensions.yml` | L3LS fabric attributes (routing protocols, MTU, spanning-tree, EVPN overlay) and pod/rack/VRF/MLAG extensions | | `location_extensions.yml` | `Location.Hall`, `Location.Rack` (`rack_type`, leaf counts, `generation_complete`) | | `ipam_extensions.yml` | `Ipam.Prefix` `role` and `status` dropdowns | | `management.yml` | `Network.DnsServer`, `Network.NtpServer`, `Network.LocalUser` | | `generator.yml` | `Generator.Target` generic (`checksum` tracking) | | `vlan/vlan.yml` | `Ipam.VLAN`, `Ipam.L2Domain` | | `vrf/vrf.yml` | `Ipam.VRF`, `Ipam.RouteTarget` | | `evpn/evpn_services.yml` | `Evpn.Tenant`, `Evpn.Svi`, `Evpn.L2Vlan` | | `evpn/evpn_gateway.yml` | `Evpn.Domain`, `Evpn.GatewayGroup`, plus fabric/pod/device EVPN Gateway relationship extensions | | `lag/lag.yml` | `Interface.Lag`, `Generic.InterfaceBundle` | | `mlag/mlag.yml` | `Generic.MlagDomain`, `Mlag.Domain`, `Mlag.Interface` | | `routing/routing.yml` | `Routing.BGPPeerGroup`, `Routing.BGPNeighbor`, prefix lists, route maps, static routes | | `compute/compute.yml` | `Compute.GenericUnit`, `Compute.PhysicalServer`, virtualization hosts | | `avd/avd.yml` | `Avd.Evpn` | | `cv/cv.yml` | `Cloudvision.Workspace` — CloudVision workspace tracking for proposed-change validation | | `objects/objects.yml` | `Avd.Artifact`, `Avd.HostvarFile`, `Avd.StructuredConfigFile` | The device and interface `role` dropdowns that the fabric uses are defined in `dcim_extensions.yml`, not in the base `dcim.yml` — the extension redefines the base lists. ## Network fabric hierarchy[​](#network-fabric-hierarchy "Direct link to Network fabric hierarchy") ### `NetworkFabric` — `Network.Fabric`[​](#networkfabric--networkfabric "Direct link to networkfabric--networkfabric") Top-level container for a datacenter fabric. Inherits `Network.BuildingBlock` and `CoreArtifactTarget`; parents `NetworkPod`. * **Attributes**: `name` (unique), `index`, interface-sorting methods, `mgmt_gateway`, `avd_hostvars_ready`. L3LS attributes (via `l3ls_extensions.yml`): `underlay_routing_protocol` (`ebgp`/`ospf`), `overlay_routing_protocol` (`ebgp`/`ibgp`), `p2p_uplinks_mtu`, `spanning_tree_mode`, `virtual_router_mac`, EVPN/underlay/MLAG passwords, `anta_enabled`. * **Relationships**: `device_designs` -> `NetworkFabricDeviceDesign` (super-spine sizing), `fabric_ip_pools` -> `CoreResourcePool`, `uplink_pool` / `vtep_pool` / `loopback_pool` / `dci_pool` -> `CoreIPPrefixPool`, `asn_pool` / `node_id_pool` -> `CoreNumberPool`, `mgmt_pool` -> `CoreIPAddressPool`, `avd_evpn` -> `AvdEvpn`, `dns_servers` / `ntp_servers` / `local_users` -> management kinds. `fabric_ip_pools` is the preferred source for Management, Loopback, Loopback VTEP, Fabric Point-to-Point, DCI, and Fabric Supernet pools. Legacy fabric pool relationships remain optional fallback inputs during migration. ### `NetworkPod` — `Network.Pod`[​](#networkpod--networkpod "Direct link to networkpod--networkpod") A pod within a fabric. Inherits `Network.BuildingBlock` and `Generator.Target`; parented by `NetworkFabric`. * **Attributes**: `name` (unique), `index`, `role` (`fabric`, `cpu`, `storage`), interface-sorting methods, `checksum` (from `Generator.Target`). * **Relationships**: `device_designs` → `NetworkPodDeviceDesign` (spine sizing), `racks` → `LocationRack`, `devices` → `DcimDevice` (the pod's spines), `mlag_peer_pool` / `mlag_l3_pool` → `CoreIPAddressPool`. ### `NetworkBuildingBlock` — `Network.BuildingBlock` (generic)[​](#networkbuildingblock--networkbuildingblock-generic "Direct link to networkbuildingblock--networkbuildingblock-generic") Hierarchical base for `NetworkFabric` and `NetworkPod`. Attributes: `name` (unique), `index`. ### Device design entities — `Network.DeviceDesign` (generic)[​](#device-design-entities--networkdevicedesign-generic "Direct link to device-design-entities--networkdevicedesign-generic") Normalized description of the devices a container should produce, defined in `device_design.yml`. Instead of a fixed `_switch_template` relationship plus an `amount_of_s` attribute per role, each container relates to *many* device design entities — one per device role — through a `device_designs` relationship. * **`NetworkDeviceDesign`** (generic): `role` (`super_spine`, `spine`, `leaf`, `l2leaf`), `device_quantity` (Number ≥ 1), and `device_template` → `CoreObjectTemplate` (cardinality one; `on_delete: no-action`, so the shared template survives a design deletion). `role` is authoritative for generation. * **Concrete nodes**, each inheriting the generic and parented by one container: * `NetworkFabricDeviceDesign` → parent `NetworkFabric` (super-spine designs) * `NetworkPodDeviceDesign` → parent `NetworkPod` (spine designs) * `NetworkRackDeviceDesign` → parent `LocationRack` (leaf / l2leaf designs) * **Ownership**: each container's `device_designs` is a `Component` (many, `on_delete: cascade`) — deleting the container deletes its designs; the templates are untouched. * **Identity**: a design is unique per `(container, role)`; `human_friendly_id` is `"__"`. "None of a role" is the **absence** of a design (replacing `amount_of_*: 0`). In seed data, designs are nested under their container. A rack with an MLAG leaf pair and a single L2 leaf looks like this (from `objects/10a_l3ls_multipod_rack.yml`): ``` - name: "Rack-A2-1" index: 1 rack_type: compute pod: Pod-A2 parent: "Hall-A1" device_designs: data: - role: leaf device_quantity: 2 device_template: leaf-switch-compute - role: l2leaf device_quantity: 1 device_template: l2leaf-switch member_of_groups: ["racks"] ``` Omit a role's entry to get none of that device type — a rack with no `l2leaf` design gets no L2 leaves. Fabric and pod designs follow the same shape with `role: super_spine` and `role: spine` respectively. Adding a new device design for a supported role is data, not a schema change. Device designs are the only source of device sizing: the fabric, pod, and rack generators read `device_designs` exclusively, and the legacy paired fields they replaced (`amount_of_super_spines` / `super_spine_switch_template`, `amount_of_spines` / `spine_switch_template`, `amount_of_leafs` / `leaf_switch_template`, `amount_of_l2leafs` / `l2leaf_switch_template`) no longer exist in the schema. ### `NetworkLink` — `Network.Link`[​](#networklink--networklink "Direct link to networklink--networklink") A cabled connection between interfaces. Inherits `Dcim.Connector`, so it has `name` and `medium` (`mmf`, `smf`, `copper`) and relates to `connected_endpoints` → `DcimEndpoint`. A DCI connection is a normal `NetworkLink` with `role=dci`, not a separate schema node. * **DCI attributes**: `role` (`dci`) and `include_in_underlay_protocol` (Boolean, default `true`). BGP ASNs are taken from each endpoint device's own `asn`, not stored on the link. * **Relationships**: inherited `connected_endpoints`; no DCI-specific endpoint, pool, subnet, endpoint IP, speed, BFD, MTU, external-network, or EVPN Gateway fields are added. * **Addressing source**: the hostvars generator allocates one `/31` per valid DCI-role link from `NetworkFabric.fabric_ip_pools` role `dci`, then the legacy `NetworkFabric.dci_pool` fallback, then a deterministic Fabric Supernet-derived fallback when the required DCI prefix-pool role is missing. ## Devices and interfaces[​](#devices-and-interfaces "Direct link to Devices and interfaces") ### `DcimDevice` — `Dcim.Device`[​](#dcimdevice--dcimdevice "Direct link to dcimdevice--dcimdevice") The concrete network device (switch). Inherits `Dcim.GenericDevice`, `Dcim.PhysicalDevice`, and `CoreArtifactTarget`. * **Attributes**: `name` (unique), `description`, `os_version`, `status` (`active`, `provisioning`, `maintenance`, `drained`). Fabric extensions (via `dcim_extensions.yml`): `role` (`super_spine`, `spine`, `leaf`, `border_leaf`, `l2leaf`), `index`, `node_id`. * **Relationships**: `interfaces` → `DcimInterface`, `device_type` → `DcimDeviceType`, `platform` → `DcimPlatform`, `primary_address` / `loopback_ip` / `mgmt_ip` → `IpamIPAddress`, `pod` → `NetworkPod`, `rack` → `LocationRack`, `asn` → `RoutingAsn` (device BGP ASN), `avd_artifact` → `AvdArtifact`, `mlag_domain` → `MlagDomain`, plus routing relations (`bgp_peer_groups`, `bgp_neighbors`, `prefix_lists`, `route_maps`, `static_routes`). ### Interface kinds[​](#interface-kinds "Direct link to Interface kinds") `DcimInterface` (`Dcim.Interface`) is the interface generic; the concrete nodes are `InterfacePhysical` (`Interface.Physical`), `InterfaceVirtual` (`Interface.Virtual`), and `InterfaceLag` (`Interface.Lag`). GraphQL queries that select any interface root on `DcimInterface`. * **`DcimInterface` attributes**: `name`, `description`, `mtu`, `status`, `role`. The fabric `role` list (via `dcim_extensions.yml`) is `uplink`, `access`, `spine`, `super_spine`, `leaf`, `loopback`, `server`, `peering`, `storage`, `mlag_peer`. * **`DcimInterface` relationships**: `device` → `DcimGenericDevice` (parent), `ip_address` → `IpamIPAddress`, `untagged_vlan` / `tagged_vlan` → `IpamVLAN`. * Layer-2/3 behaviour comes from the `Interface.Layer2` (`l2_mode`) and `Interface.Layer3` (`ip_addresses`, `dot1q_id`, `mac_address`) generics. ### `DcimDeviceType` — `Dcim.DeviceType`[​](#dcimdevicetype--dcimdevicetype "Direct link to dcimdevicetype--dcimdevicetype") A device model. Attributes: `name` (unique), `part_number`, `height`, `full_depth`, `weight`. Relationships: `manufacturer` → `OrganizationManufacturer`, `platform` → `DcimPlatform`. ### `OrganizationManufacturer` — `Organization.Manufacturer`[​](#organizationmanufacturer--organizationmanufacturer "Direct link to organizationmanufacturer--organizationmanufacturer") A device manufacturer. Inherits `Organization.Generic`; attributes `name` (unique), `description`; relates to `device_type` → `DcimDeviceType`. ## Locations[​](#locations "Direct link to Locations") ### `LocationHall` — `Location.Hall`[​](#locationhall--locationhall "Direct link to locationhall--locationhall") A datacenter hall. Inherits `Location.Generic`; parents `LocationRack`. Attributes: `name`, `shortname`, `description`, `index`. ### `LocationRack` — `Location.Rack`[​](#locationrack--locationrack "Direct link to locationrack--locationrack") A physical rack. Inherits `Location.Generic`, `Location.Hosting`, and `Generator.Target`; parented by `LocationHall`. * **Attributes**: `name`, `index`, `rack_type` (`compute`, `storage`), `mlag`, `generation_complete`, `checksum`. * **Relationships**: `device_designs` → `NetworkRackDeviceDesign` (leaf and l2leaf sizing), `pod` → `NetworkPod`, `devices` → `DcimPhysicalDevice`. ## IPAM[​](#ipam "Direct link to IPAM") ### `IpamIPAddress` — `Ipam.IPAddress`[​](#ipamipaddress--ipamipaddress "Direct link to ipamipaddress--ipamipaddress") An IP address. Inherits `BuiltinIPAddress`. Relationships: `interface` → `Interface.Layer3`, `vrf` → `IpamVRF`. ### `IpamPrefix` — `Ipam.Prefix`[​](#ipamprefix--ipamprefix "Direct link to ipamprefix--ipamprefix") An IP prefix. Inherits `BuiltinIPPrefix`. * **`role`** (required, via `ipam_extensions.yml`): `supernet`, `pod_super_spine_spine`, `pod_leaf_spine`, `loopback`, `loopback-vtep`, `technical`, `management`, `backfill`. * **`status`** (via `ipam_extensions.yml`): `active`, `deprecated`, `reserved`. * **Relationships**: `gateway` → `IpamIPAddress`, `vlan` → `IpamVLAN`, `vrf` → `IpamVRF`, `location` → `Location.Hosting`. ### `IpamVLAN` — `Ipam.VLAN`[​](#ipamvlan--ipamvlan "Direct link to ipamvlan--ipamvlan") A VLAN. Attributes: `name`, `vlan_id`, `status`, `role` (`server`, `management`, `user`). Relationships: `l2domain` → `IpamL2Domain` (required), `prefixes` → `IpamPrefix`. ### `IpamL2Domain` — `Ipam.L2Domain`[​](#ipaml2domain--ipaml2domain "Direct link to ipaml2domain--ipaml2domain") A layer-2 domain grouping VLANs. Attributes: `name`. Relationships: `vlans` → `IpamVLAN`. ### `IpamVRF` — `Ipam.VRF`[​](#ipamvrf--ipamvrf "Direct link to ipamvrf--ipamvrf") A VRF. Attributes: `name` (unique), `vrf_rd`, `vrf_vni`, `vtep_diagnostic_loopback`. Relationships: `namespace` → `BuiltinIPNamespace`, `import_rt` / `export_rt` → `IpamRouteTarget`, `tenant` → `EvpnTenant`, `svis` → `EvpnSvi`. ### `IpamRouteTarget` — `Ipam.RouteTarget`[​](#ipamroutetarget--ipamroutetarget "Direct link to ipamroutetarget--ipamroutetarget") A route target. Attributes: `name` (unique), `description`. Relationships: `vrf` → `IpamVRF`. ## EVPN services[​](#evpn-services "Direct link to EVPN services") ### `EvpnTenant` — `Evpn.Tenant`[​](#evpntenant--evpntenant "Direct link to evpntenant--evpntenant") An EVPN tenant. Attributes: `name` (unique), `mac_vrf_vni_base`, `description`. Relationships: `fabrics` → `NetworkFabric`, `vrfs` → `IpamVRF`, `l2vlans` → `EvpnL2Vlan` (component). ### `EvpnSvi` — `Evpn.Svi`[​](#evpnsvi--evpnsvi "Direct link to evpnsvi--evpnsvi") An SVI. Attributes: `name`, `svi_id`, `ip_address_virtual`, `enabled`. Relationships: `vrf` → `IpamVRF` (parent), `vlan` → `IpamVLAN`, `rack_tags` → `LocationRack`, `avd_tags` → `AvdTag`. ### `EvpnL2Vlan` — `Evpn.L2Vlan`[​](#evpnl2vlan--evpnl2vlan "Direct link to evpnl2vlan--evpnl2vlan") An L2-only VLAN attached to a tenant. Attributes: `name`, `vlan_id`, `vni_override`. Relationships: `tenant` → `EvpnTenant` (parent), `vlan` → `IpamVLAN`. ### `EvpnDomain` — `Evpn.Domain`[​](#evpndomain--evpndomain "Direct link to evpndomain--evpndomain") An EVPN domain owned by one `NetworkFabric`. Attributes: `name`, `domain_id`, and optional `description`. Relationships: `fabric` -> `NetworkFabric` (parent), `pods` -> `NetworkPod`, `local_gateway_groups` -> `EvpnGatewayGroup` (component children), and `remote_gateway_groups` -> `EvpnGatewayGroup`. `domain_id` and `name` are unique per fabric. The hostvar generator uses `EvpnGatewayGroup.local_domain.domain_id` as the local EVPN Gateway D-PATH domain ID and `EvpnGatewayGroup.remote_domain.domain_id` as the remote D-PATH domain ID. ### `EvpnGatewayGroup` — `Evpn.GatewayGroup`[​](#evpngatewaygroup--evpngatewaygroup "Direct link to evpngatewaygroup--evpngatewaygroup") EVPN Multi-Domain Gateway intent shared by one or more Border Leaf devices in a selected Pod. Attributes include `resiliency_model` (only `all_active_multihoming`), EVPN L2/L3 enablement flags, D-PATH enablement, All-Active Multihoming enablement, and Ethernet Segment identifier/RT import values. Relationships: `local_domain` -> `EvpnDomain` (parent), `pod` -> `NetworkPod` (required non-owning context), `remote_domain` -> `EvpnDomain`, and `members` -> `DcimDevice`. The selected Pod must have `evpn_domain` set to the same object as `local_domain`, `remote_domain` must differ from `local_domain`, and group names are unique by `[local_domain, pod, name__value]`. Its schema-valid HFID uses the selected Pod and group name, while the display label and ordering include native `local_domain`, `pod`, `remote_domain`, and `name` fields. Reviewers distinguish the parent local domain from the EVPN Domain relationship view through `EvpnDomain.local_gateway_groups`; no computed or denormalized helper attribute is added solely for local-domain display. `NetworkFabric.evpn_domains`, `NetworkPod.evpn_domain`, `NetworkPod.evpn_gateway_groups`, and `DcimDevice.evpn_gateway_group` are additive relationships from `evpn/evpn_gateway.yml`. Both `EvpnDomain` and `EvpnGatewayGroup` set `include_in_menu: false` because the custom EVPN Services menu exposes one Domains item for `EvpnDomain`; gateway groups are reached from EVPN Domain relationship views. ## Compute[​](#compute "Direct link to Compute") ### `ComputePhysicalServer` — `Compute.PhysicalServer`[​](#computephysicalserver--computephysicalserver "Direct link to computephysicalserver--computephysicalserver") A physical server. Inherits `Compute.GenericUnit`, `Dcim.GenericDevice`, and `Generator.Target`. Attributes: `name`, `role` (`compute`, `gpu`), `status`. Relationships: `rack` → `LocationRack`, `interfaces` → `DcimInterface`. ## AVD[​](#avd "Direct link to AVD") ### `AvdArtifact` — `Avd.Artifact`[​](#avdartifact--avdartifact "Direct link to avdartifact--avdartifact") Per-device container linking a device to its stored hostvars and structured config. Attributes: `name` (unique). Relationships: `device` → `DcimDevice` (required), `hostvar_file` → `AvdHostvarFile` (component), `structured_config_file` → `AvdStructuredConfigFile` (component). See [AvdArtifact & File Storage](/arista-avd/developer-guide/avd/artifacts.md). ### `AvdHostvarFile` — `Avd.HostvarFile` · `AvdStructuredConfigFile` — `Avd.StructuredConfigFile`[​](#avdhostvarfile--avdhostvarfile--avdstructuredconfigfile--avdstructuredconfigfile "Direct link to avdhostvarfile--avdhostvarfile--avdstructuredconfigfile--avdstructuredconfigfile") Child file nodes holding the per-device hostvars and structured-config JSON. Both inherit `CoreFileObject` (providing `content`, `content_type`, `checksum`) and are parented by `AvdArtifact`. ### `AvdEvpn` — `Avd.Evpn`[​](#avdevpn--avdevpn "Direct link to avdevpn--avdevpn") AVD EVPN fabric-wide settings. Attributes include `ebgp_multihop` and `overlay_bgp_rtc`. Relationships: `fabric` → `NetworkFabric`. ### `AvdTag` — `Avd.Tag`[​](#avdtag--avdtag "Direct link to avdtag--avdtag") AVD-specific fabric tag object. Attributes: `name`, `description`. Relationships: `racks` → `LocationRack`; reciprocal rack assignments emit PyAVD node-group `filter.tags`, and SVI `avd_tags` emit PyAVD SVI `tags`. ## CloudVision[​](#cloudvision "Direct link to CloudVision") ### `CloudvisionWorkspace` — `Cloudvision.Workspace`[​](#cloudvisionworkspace--cloudvisionworkspace "Direct link to cloudvisionworkspace--cloudvisionworkspace") Tracks one CloudVision workspace created by the `cv-config-validation` check for a proposed change and fabric, defined in `cv/cv.yml`. Excluded from the UI menu (`include_in_menu: false`); identified by `workspace_id`. * **Attributes**: `name` (display name), `workspace_id` (unique — the CloudVision workspace UUID), `proposed_change_id`, `workspace_url`, `thread_id` (the `CoreChangeThread` used for lifecycle comments), `change_control_id` and `change_control_url` (set when a change control exists), `last_submission_error`, `last_submission_attempt_at`, `submitted_at`, and `status`. * **Relationships**: `fabric` → `NetworkFabric` (cardinality one). The workspace ID is derived deterministically from the proposed-change ID and the fabric name, so re-running validation updates the same workspace rather than creating another. See [Checks](/arista-avd/developer-guide/checks.md) and [CloudVision Validation](/arista-avd/cloudvision.md). Fabrics opt in through `NetworkFabric.cloudvision_managed` (Boolean, default `false`) in `logical_design.yml`; the check skips everything else when it is false. ## Generator target[​](#generator-target "Direct link to Generator target") ### `GeneratorTarget` — `Generator.Target` (generic)[​](#generatortarget--generatortarget-generic "Direct link to generatortarget--generatortarget-generic") Mixed into kinds that can be generator targets (`NetworkPod`, `LocationRack`, `ComputePhysicalServer`). Provides `checksum` (optional), which stores a hash of related node IDs for idempotent regeneration. ## Dropdown reference[​](#dropdown-reference "Direct link to Dropdown reference") **Device role** (`DcimDevice.role`): `super_spine`, `spine`, `leaf`, `border_leaf`, `l2leaf`, `l2spine`, `l3spine`, `p`, `pe`, `rr`. **Interface role** (`DcimInterface.role`): `uplink`, `access`, `spine`, `super_spine`, `leaf`, `loopback`, `vtep_loopback`, `server`, `peering`, `storage`, `mlag_peer`. **Pod role** (`NetworkPod.role`): `fabric`, `cpu`, `storage`. **Rack type** (`LocationRack.rack_type`): `compute`, `storage`. **Underlay routing protocol** (`NetworkFabric.underlay_routing_protocol`): `ebgp`, `ospf`, `none`, `isis-ldp`. **Overlay routing protocol** (`NetworkFabric.overlay_routing_protocol`): `ebgp`, `ibgp`. **Spanning-tree mode** (`NetworkFabric.spanning_tree_mode`): `mstp`, `rstp`, `rapid-pvst`, `none`. **CloudVision workspace status** (`CloudvisionWorkspace.status`): `pending`, `built`, `submitted`, `abandoned`, `submit_failed`. **Prefix role** (`IpamPrefix.role`): `fabric_supernet`, `fabric_point_to_point`, `dci`, `mlag`, `mlag_peering`, `supernet`, `pod_super_spine_spine`, `pod_leaf_spine`, `loopback`, `loopback-vtep`, `technical`, `management`, `backfill`. **Prefix status** (`IpamPrefix.status`): `active`, `deprecated`, `reserved`. ## Role-driven pool collections[​](#role-driven-pool-collections "Direct link to Role-driven pool collections") `NetworkFabric.fabric_ip_pools` is the preferred fabric-scope IP pool collection. It accepts `CoreResourcePool` members so Management address pools and Loopback, Loopback VTEP, Fabric Point-to-Point, DCI, and Fabric Supernet prefix pools can be managed through one relationship. Legacy fabric relationships remain optional during migration and are used only as fallback inputs. `NetworkPod.pod_ip_pools` is the preferred pod-scope IP pool collection. It accepts pod-specific Loopback, Loopback VTEP, Fabric Point-to-Point, MLAG, and MLAG Peering pools. Management remains fabric-scoped. Pool purpose is resolved from the `IpamPrefix.role` values on each pool's resources. A pool with mixed authoritative roles, duplicate role coverage in one fabric or pod, a non-IP pool in these collections, or a pod prefix outside the matching fabric prefix is invalid. ## Source[​](#protocols "Direct link to Source") * [`schemas/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/schemas) — all schema definitions. * [`schemas/base/dcim.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/base/dcim.yml) — base `Dcim.GenericDevice` / `Dcim.PhysicalDevice` / `Dcim.Device`, interfaces, `DcimDeviceType`; project device extensions (`role`, BGP ASN relationship, relations) and `Network.Link` live in [`schemas/dcim_extensions.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/dcim_extensions.yml). * [`schemas/logical_design.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/logical_design.yml) — `Network.Fabric`, `Network.Pod`. * [`schemas/base/location.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/base/location.yml) + [`schemas/location_extensions.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/location_extensions.yml) — `Location.Hall`, `Location.Rack`. * [`schemas/base/ipam.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/base/ipam.yml) + [`schemas/ipam_extensions.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/ipam_extensions.yml) — IPAM nodes (the `Prefix` `role`/`status` dropdowns live in the extension). * [`schemas/avd/avd.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/avd/avd.yml) — `Avd.Evpn`, `Avd.Tag`. * [`schemas/objects/objects.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/objects/objects.yml) — `Avd.Artifact`, `Avd.HostvarFile`, `Avd.StructuredConfigFile` (see [AvdArtifact & File Storage](/arista-avd/developer-guide/avd/artifacts.md) for the full reference). * [`schemas/cv/cv.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/cv/cv.yml) — `Cloudvision.Workspace`. * [`schemas/device_design.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/schemas/device_design.yml) — `Network.DeviceDesign` and the per-container design nodes. * Generated protocols: [`src/solution_arista_avd/protocols.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/src/solution_arista_avd/protocols.py) — regenerate after any schema change with: ``` uv run infrahubctl protocols --schemas schemas --out src/solution_arista_avd/protocols.py ``` Note the `--out` flag (not `--output`) and the explicit path — the default would drop `schema_protocols.py` in the current directory instead of overwriting the checked-in file. --- # Transforms Developer Guide Documents the transform implementations. To *view* artifacts as an operator, see [Viewing Artifacts](/arista-avd/viewing-artifacts.md). ## Overview[​](#overview "Direct link to Overview") Transforms convert Infrahub data into usable outputs (configs, documentation, computed attributes). They run on-demand when artifacts are accessed. ## Transform types[​](#transform-types "Direct link to Transform types") 1. **Python Transforms** - Complex logic, external library calls (PyAVD) 2. **Jinja2 Transforms** - Template-based text generation Every transform this repository registers is a Python transform. One of them, `containerlab_topology`, renders its output through a Jinja2 template it loads itself; the `jinja2_transforms:` block of `.infrahub.yml` is unused here. ## Transform architecture[​](#transform-architecture "Direct link to Transform architecture") ``` ┌──────────────────┐ ┌────────────────────┐ ┌─────────────────┐ │ GraphQL Query │ ──▶ │ Transform Class │ ──▶ │ Artifact │ │ (*.gql) │ │ (*Transform) │ │ (config/doc) │ └──────────────────┘ └────────────────────┘ └─────────────────┘ ``` ## Python transforms[​](#python-transforms "Direct link to Python transforms") ### ComputedInterfaceDescription[​](#computedinterfacedescription "Direct link to ComputedInterfaceDescription") **File**: `transforms/computed_interface_description.py` **Purpose**: Generate human-readable interface descriptions **Input**: DcimInterface **Output**: String like "→ remote-device:Ethernet1" ``` class ComputedInterfaceDescription(InfrahubTransform): async def transform(self, data): interface = data["DcimInterface"]["edges"][0]["node"] link = interface.get("link") if not link: return "" # Find the remote end remote = link["interface_a"] if link["interface_b"]["id"] == interface["id"] else link["interface_b"] return f"→ {remote['device']['name']['value']}:{remote['name']['value']}" ``` ### CablingPlan[​](#cablingplan "Direct link to CablingPlan") **File**: `transforms/cabling_plan.py` **Purpose**: Generate CSV cabling documentation for a fabric **Input**: NetworkFabric **Output**: CSV file with all connections ``` Source Device,Source Interface,Destination Device,Destination Interface,Link Type spine-A1-1,Ethernet1,super-spine-A-1,Ethernet1,uplink leaf-A1-01-1,Ethernet49,spine-A1-1,Ethernet1,uplink ``` ### AvdEosConfigTransform[​](#avdeosconfigtransform "Direct link to AvdEosConfigTransform") **File**: `transforms/avd_eos_config.py` **Purpose**: Convert AVD structured config to EOS CLI **Input**: DcimDevice (with AvdArtifact) **Output**: EOS CLI configuration ``` class AvdEosConfigTransform(InfrahubTransform): async def transform(self, data): device = data["DcimDevice"]["edges"][0]["node"] artifact = device["avd_artifact"]["node"] if not artifact["structured_config_identifier"]["value"]: return "! No structured config available" config = await self.client.object_store.get( identifier=artifact["structured_config_identifier"]["value"] ) return pyavd.get_device_config(json.loads(config)) ``` ### AvdFabricDocTransform[​](#avdfabricdoctransform "Direct link to AvdFabricDocTransform") **File**: `transforms/avd_fabric_doc.py` **Purpose**: Generate fabric-wide documentation **Input**: NetworkFabric **Output**: Markdown documentation ``` class AvdFabricDocTransform(InfrahubTransform): async def transform(self, data): # Collect all device hostvars and structured configs all_hostvars = {} all_structured = {} for device in devices: hostvars = await self.client.object_store.get(...) structured = await self.client.object_store.get(...) all_hostvars[device.name] = hostvars all_structured[device.name] = structured avd_facts = pyavd.get_avd_facts(all_hostvars) return pyavd.get_fabric_documentation( avd_facts, all_structured, fabric_name ) ``` ### AvdDeviceDocTransform[​](#avddevicedoctransform "Direct link to AvdDeviceDocTransform") **File**: `transforms/avd_device_doc.py` **Purpose**: Generate per-device documentation **Input**: DcimDevice **Output**: Markdown documentation for single device ### AvdAntaCatalogTransform[​](#avdantacatalogtransform "Direct link to AvdAntaCatalogTransform") **File**: `transforms/avd_anta_catalog.py` **Purpose**: Render a per-device [ANTA](https://anta.arista.com) test catalog from the stored structured config **Input**: DcimDevice **Output**: YAML catalog, or a one-line marker comment Unlike EOS config rendering, catalog generation needs fabric-wide data, so the transform gathers every sibling device's structured config in the same fabric into one `AVDFabricData` before calling `pyavd.get_device_test_catalog()`. It is gated by the fabric's `anta_enabled` flag. When ANTA is disabled — or the device has no fabric, or no structured config — the transform returns a marker comment instead of a catalog, so the artifact renders successfully and says why it is empty: ``` # ANTA disabled for fabric Fabric-L3LS-Multi-Domain ``` ### ContainerLabTopology[​](#containerlabtopology "Direct link to ContainerLabTopology") **File**: `transforms/containerlab_topology.py` **Purpose**: Render a [ContainerLab](https://containerlab.dev) topology file for a whole fabric **Input**: NetworkFabric **Output**: YAML topology (`topology.clab.yml` shape) The transform uses two queries — `containerlab_topology` for the fabric's devices and `containerlab_link_endpoints` to resolve link endpoints in batches — and renders through the `transforms/templates/containerlab_topology.j2` template. Node kinds, container images, and the interface-mapping bind come from schema attributes (`DcimPlatform.containerlab_os`, `DcimPlatform.containerlab_image`, `DcimDeviceType.containerlab_interface_mapping`), so changing the cEOS version is a data change rather than a code change. Nodes and links are emitted in a stable sorted order, so two renders of unchanged data are byte-identical. See the [ContainerLab page](/arista-avd/containerlab.md) for the full artifact shape, the role-selection rules, and how to deploy the topology. ### CVWorkspaceSubmissionWebhookPayload[​](#cvworkspacesubmissionwebhookpayload "Direct link to CVWorkspaceSubmissionWebhookPayload") **File**: `transforms/cv_workspace_submission_webhook.py` **Purpose**: Build the JSON body for the CloudVision workspace-submission `CoreCustomWebhook` **Input**: CloudvisionWorkspace **Output**: JSON object — check name, proposed-change ID, and one entry per linked workspace with its ID, status, URL, and fabric name This transform has no artifact definition: it is referenced by the webhook rather than rendered to the object store. See [Checks](/arista-avd/developer-guide/checks.md) for how it fits the CloudVision validation pipeline. ## Query classes[​](#query-classes "Direct link to Query classes") Each transform has Pydantic models for type-safe query parsing. **These `*_query.py` files are generated, not hand-written** — regenerate them whenever the `.gql` query or the schema changes: ``` uv run infrahubctl graphql generate-return-types transforms/computed_interface_description.gql ``` This reads `schema.graphql` at the repo root (refresh with `uv run infrahubctl graphql export-schema --destination schema.graphql` when needed) and emits the matching `*_query.py` next to the query file. Shape of a typical generated class: ``` # transforms/computed_interface_description_query.py (generated) class InterfaceLink(BaseModel): interface_a: InterfaceNode interface_b: InterfaceNode class InterfaceNode(BaseModel): id: str name: ValueWrapper[str] device: DeviceRef link: InterfaceLink | None ``` ## Artifacts[​](#artifacts "Direct link to Artifacts") Artifacts are the output files generated by transforms. ### Artifact definitions[​](#artifact-definitions "Direct link to Artifact definitions") Defined in `.infrahub.yml`: ``` artifact_definitions: - name: cabling_plan targets: fabrics transformation: cabling_plan content_type: text/csv - name: avd_eos_configuration targets: avd_devices transformation: avd_eos_config content_type: text/plain - name: avd_fabric_documentation targets: fabrics transformation: avd_fabric_doc content_type: text/markdown - name: avd_device_documentation targets: avd_devices transformation: avd_device_doc content_type: text/markdown - name: avd_anta_catalog targets: avd_devices transformation: avd_anta_catalog content_type: application/yaml - name: containerlab_topology targets: fabrics transformation: containerlab_topology content_type: application/yaml ``` `cv_workspace_submission_webhook_payload` is deliberately absent from this block — it renders a webhook body, not an artifact. ### Viewing artifacts[​](#viewing-artifacts "Direct link to Viewing artifacts") 1. Navigate to target object in Infrahub UI 2. Click **Artifacts** tab 3. Select artifact to view/download ### Regenerating artifacts[​](#regenerating-artifacts "Direct link to Regenerating artifacts") Artifacts regenerate automatically when: * Underlying data changes * Transform code changes * Manually triggered via UI ## Configuration[​](#configuration "Direct link to Configuration") Transforms are registered in `.infrahub.yml`: ``` python_transforms: - name: computed_interface_description class_name: ComputedInterfaceDescription file_path: "./transforms/computed_interface_description.py" - name: cabling_plan class_name: CablingPlan file_path: "./transforms/cabling_plan.py" - name: avd_eos_config class_name: AvdEosConfigTransform file_path: "./transforms/avd_eos_config.py" - name: avd_fabric_doc class_name: AvdFabricDocTransform file_path: "./transforms/avd_fabric_doc.py" - name: avd_device_doc class_name: AvdDeviceDocTransform file_path: "./transforms/avd_device_doc.py" - name: avd_anta_catalog class_name: AvdAntaCatalogTransform file_path: "./transforms/avd_anta_catalog.py" - name: containerlab_topology class_name: ContainerLabTopology file_path: "./transforms/containerlab_topology.py" - name: cv_workspace_submission_webhook_payload class_name: CVWorkspaceSubmissionWebhookPayload file_path: "./transforms/cv_workspace_submission_webhook.py" convert_query_response: false ``` ## File structure[​](#file-structure "Direct link to File structure") ``` transforms/ ├── computed_interface_description.py # Interface description ├── computed_interface_description.gql # Interface query ├── computed_interface_description_query.py # Pydantic models ├── cabling_plan.py # Cabling plan CSV ├── fabric_cabling_plan.gql # Fabric cabling query ├── fabric_cabling_plan_query.py # Pydantic models ├── avd_eos_config.py # EOS config transform ├── avd_device_config.gql # Device config query ├── avd_device_config_query.py # Pydantic models ├── avd_fabric_doc.py # Fabric documentation ├── avd_device_doc.py # Device documentation ├── avd_fabric_devices.gql # Fabric devices query ├── avd_fabric_devices_query.py # Pydantic models ├── avd_anta_catalog.py # ANTA catalog transform ├── avd_anta_catalog.gql # ANTA catalog query ├── avd_anta_catalog_query.py # Pydantic models ├── containerlab_topology.py # ContainerLab topology transform ├── containerlab_topology.gql # Fabric device/link query ├── containerlab_topology_query.py # Pydantic models ├── containerlab_link_endpoints.gql # Batched link-endpoint query ├── containerlab_link_endpoints_query.py # Pydantic models ├── cv_workspace_submission_webhook.py # CloudVision webhook payload ├── cv_workspace_submission_webhook.gql # Workspace query ├── cv_workspace_submission_webhook_query.py # Pydantic models └── templates/ └── containerlab_topology.j2 # ContainerLab topology template ``` ## Creating new transforms[​](#creating-new-transforms "Direct link to Creating new transforms") ### Python transform[​](#python-transform "Direct link to Python transform") 1. Create transform class: ``` # transforms/my_transform.py from infrahub_sdk.transforms import InfrahubTransform class MyTransform(InfrahubTransform): query = "my_query" async def transform(self, data): # Process data return "output" ``` 2. Create GraphQL query: ``` # transforms/my_query.gql query MyQuery($device_id: String!) { DcimDevice(ids: [$device_id]) { edges { node { name { value } } } } } ``` 3. Register in `.infrahub.yml`: ``` queries: - name: my_query file_path: "./transforms/my_query.gql" python_transforms: - name: my_transform class_name: MyTransform file_path: "./transforms/my_transform.py" artifact_definitions: - name: my_artifact targets: devices transformation: my_transform ``` ### Jinja2 transform[​](#jinja2-transform "Direct link to Jinja2 transform") 1. Create template: ``` {# transforms/templates/my_template.j2 #} Output for {{ node.name.value }} ``` 2. Register in `.infrahub.yml`: ``` jinja2_transforms: - name: my_jinja_transform template_path: "./transforms/templates/my_template.j2" query: my_query ``` ## Source[​](#source "Direct link to Source") * Python transforms: * [`transforms/avd_eos_config.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_eos_config.py) — `AvdEosConfigTransform`. * [`transforms/avd_fabric_doc.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_fabric_doc.py) — `AvdFabricDocTransform`. * [`transforms/avd_device_doc.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_device_doc.py) — `AvdDeviceDocTransform`. * [`transforms/computed_interface_description.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/computed_interface_description.py) — `ComputedInterfaceDescription`. * [`transforms/cabling_plan.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/cabling_plan.py) — `CablingPlan`. * [`transforms/avd_anta_catalog.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/avd_anta_catalog.py) — `AvdAntaCatalogTransform`. * [`transforms/containerlab_topology.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/containerlab_topology.py) — `ContainerLabTopology`. * [`transforms/cv_workspace_submission_webhook.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/transforms/cv_workspace_submission_webhook.py) — `CVWorkspaceSubmissionWebhookPayload`. * Templates: [`transforms/templates/`](https://github.com/opsmill/infrahub-arista-avd/tree/main/transforms/templates). * Registration: [`.infrahub.yml`](https://github.com/opsmill/infrahub-arista-avd/blob/main/.infrahub.yml) — `python_transforms:` and `artifact_definitions:` blocks. * The AVD transforms are documented in detail on the [AVD Transforms](/arista-avd/developer-guide/avd/transforms.md) page. * The CloudVision webhook payload transform is documented alongside the [Checks](/arista-avd/developer-guide/checks.md) it serves. --- # Add a network segment Creates a new EVPN network segment — a VRF, a VLAN, and an SVI — on a target fabric. The workflow runs on its own branch, regenerates hostvars and structured configs, and opens a proposed change for review. Prerequisites: * A running stack with seed data loaded ([Quick Start](/arista-avd/quick-start.md)). * A provisioned fabric with devices and artifacts ([Provision Your First Fabric](/arista-avd/provision-first-fabric.md)). * At least one EVPN tenant. If none exist, create one using [Create a Tenant](/arista-avd/how-to/create-tenant.md) first. ## Open the service portal[​](#open-the-service-portal "Direct link to Open the service portal") Navigate to **`http://localhost:8501`**. From the sidebar, open **Add Network Segment**. ## Fill the form[​](#fill-the-form "Direct link to Fill the form") The form has two columns: | Left column | Right column | | --------------------------------------------------------- | -------------------------------------------------------------- | | **Segment Name** — free text, for example, `web-services` | **VRF Name** — free text; leave blank to reuse an existing VRF | | **Tenant** — dropdown of existing EVPN tenants | **VRF VNI** — number (1–16777215), default `100` | | **VLAN ID** — number (1–4094), default `100` | **L2 Domain** — dropdown of available L2 domains | | **Gateway IP (CIDR)** — for example, `10.10.100.1/24` | **Target Fabric** — dropdown of fabrics | All fields are required except VRF Name (blank = use existing VRF, see below). ## Submit[​](#submit "Direct link to Submit") Click **Create Network Segment**. The portal performs these steps in order, showing progress: 1. **Create branch** named `add-segment-`. 2. **Create the VLAN** (`IpamVLAN`) on the chosen L2 domain. 3. **Create the VRF** (`IpamVRF`) under the chosen tenant — only if you filled in VRF Name. 4. **Create the SVI** (`EvpnSvi`) linking the VLAN to the VRF with the specified gateway. 5. **Run the AVD pipeline** — hostvars and structured configs regenerate for affected devices. This can take a few minutes. 6. **Create a proposed change** summarising the segment. When complete, you'll see a **View Proposed Change** button — click it to open the proposed change in the Infrahub UI. note If you left **VRF Name** blank, the SVI step is skipped and a warning is shown. You'll need to link the VLAN to an existing VRF manually in the Infrahub UI before the segment is usable. ## Review and merge[​](#review-and-merge "Direct link to Review and merge") In the proposed change: 1. Inspect the **Data** tab — see the three new objects (VLAN, VRF, SVI). 2. Inspect the **Artifacts** tab — the AVD EOS configurations for devices on the target fabric should have updated to include the new VLAN, VRF, and SVI. 3. Approve and **Merge** when the updated configs look correct. Once merged, the segment exists on `main`, and Ansible inventories built from Infrahub include the new config on the next deployment. ## If the service portal is unavailable[​](#if-the-service-portal-is-unavailable "Direct link to If the service portal is unavailable") You can do the same workflow in the Infrahub UI by creating the objects manually on a branch: 1. Create a branch. 2. Create an `IpamVLAN` on the target L2 domain. 3. Create an `IpamVRF` under the target tenant. 4. Create an `EvpnSvi` linking the two, with the gateway IP. 5. Run **`generate-avd-device-hostvar`** (per device) and **`generate-avd-device-structured-config`** (per fabric) from **Actions → Generator definitions**. 6. Create a proposed change from the branch. See also [Common Issues](/arista-avd/troubleshooting.md) if a step fails. ## Source[​](#source "Direct link to Source") Service-portal implementation: [`service_catalog/pages/1_Create_Segment.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/service_catalog/pages/1_Create_Segment.py). --- # Add a server Provisions a new physical server into a compute rack. The workflow runs on a branch, cables the server to the adjacent leaf switches, cascades through the AVD pipeline to update affected device configs, and opens a proposed change. Prerequisites: * A running stack with seed data loaded ([Quick Start](/arista-avd/quick-start.md)). * A provisioned fabric with at least one **compute** rack ([Provision Your First Fabric](/arista-avd/provision-first-fabric.md)). * At least one `TemplateComputePhysicalServer` template defined (seeded with the project). ## Open the service portal[​](#open-the-service-portal "Direct link to Open the service portal") Navigate to **`http://localhost:8501`**. From the sidebar, open **Add Server**. ## Fill the form[​](#fill-the-form "Direct link to Fill the form") | Field | Description | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | **Server Name** | Hostname for the new server, for example, `compute-pod-a2-3-1`. | | **Rack** | Dropdown of compute racks only (racks with `rack_type = "compute"`). Shown as ` ()`. | | **Server Template** | Dropdown of available `TemplateComputePhysicalServer` templates. Determines the server's interface count, role, and other defaults. | The branch used for the change is shown in the sidebar under **Select Branch**. By default it's `main`; use the **Create branch** link in the sidebar to create and select a new branch first if you want the change isolated. ## Submit[​](#submit "Direct link to Submit") Click **Add Server**. The portal performs: 1. **Create branch** named `add-server-` (if you weren't already on a non-default branch). 2. **Create the server** (`ComputePhysicalServer`) in the chosen rack, using the template's interface layout and marking status `provisioning`. 3. **Create the server as a member of the `servers` group** — this triggers the server-cabling generator and the AVD cascade. 4. **Wait \~60 seconds** for the generator chain to run: server cabling, hostvar regeneration, and structured config updates for the affected leaf switches. 5. **Create a proposed change** summarising the new server. You'll see two buttons: * **View Server** — opens the new `ComputePhysicalServer` object in Infrahub. * **View Proposed Change** — opens the proposed change. ## Review and merge[​](#review-and-merge "Direct link to Review and merge") In the proposed change: 1. **Data tab** — confirm the new server exists with its interfaces and cabling. 2. **Artifacts tab** — the leaves this server cables into should have updated EOS configurations reflecting the new access/trunk ports. 3. Approve and **Merge**. ## Automated cabling flow[​](#automated-cabling-flow "Direct link to Automated cabling flow") The server-cabling generator runs when a server is added to the `servers` group. The portal commits the new `ComputePhysicalServer` and its `servers` group membership in the same upsert, so the generator sees the target as a group member as soon as it runs. For a newly added server, the generator: 1. Reads the server rack and server interfaces from the generator query. 2. Finds leaf switches in the same rack and their `server` role physical interfaces. 3. Selects the first available leaf port index that is free on all target leaves. 4. Creates `NetworkLink` objects between the server interfaces and selected leaf physical interfaces. 5. For dual-homed, multi-leaf servers, creates server-side `Bond1`. 6. Creates one switch-side `InterfaceLag` named `Port-Channel` on each attached leaf. The `channel_id` is derived from the selected leaf port number and is owned in Infrahub. 7. Attaches each leaf physical interface to its local switch-side LAG. 8. Applies tagged and untagged VLANs. Single-homed servers use the paired leaf physical interface; dual-homed servers merge VLAN intent onto server `Bond1` and the switch `Port-Channel` objects instead of the member Ethernet interfaces. 9. For idempotence, reconciles already-cabled servers by rebuilding the cabling plan from existing `NetworkLink` objects, then reapplying VLANs and LAG state. 10. Sets EVPN Ethernet Segment on switch-side LAGs when the server is attached across non-MLAG leaves. 11. Marks AVD hostvars as not ready for the fabric and triggers hostvar generation, which cascades into structured config generation. The resulting hostvars include `port_channel.channel_id` for switch-side server LAGs, so PyAVD receives the explicit Port-Channel ID from Infrahub. For bonded servers, VLAN mode and VLAN lists are derived from the logical Bond or Port-Channel VLAN relationships, not from member Ethernet ports. ## If the wait times out[​](#if-the-wait-times-out "Direct link to If the wait times out") The portal waits a fixed 60 seconds for the generator chain to finish. On a busy machine the chain can take longer. If the proposed change shows no updated device configs: 1. Open the proposed change's branch in the Infrahub UI. 2. Navigate to **Actions → Tasks** and check the status of the hostvar and structured-config generator runs. 3. If they haven't started, trigger **`generate-avd-device-hostvar`** on the affected leaves manually, then **`generate-avd-device-structured-config`** on the fabric. See [Common Issues](/arista-avd/troubleshooting.md) for more. ## Source[​](#source "Direct link to Source") Service-portal implementation: [`service_catalog/pages/2_Add_Server.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/service_catalog/pages/2_Add_Server.py). --- # Create a tenant Creates a new EVPN tenant. A tenant is a logical container that network segments (VLANs, VRFs, SVIs) are attached to. Each tenant has a **MAC VRF VNI base** — VLAN VNIs are then computed as `base + VLAN ID`, giving every segment a unique VNI without manual allocation. Prerequisites: * A running stack with seed data loaded ([Quick Start](/arista-avd/quick-start.md)). * At least one provisioned fabric ([Provision Your First Fabric](/arista-avd/provision-first-fabric.md)). ## Open the service portal[​](#open-the-service-portal "Direct link to Open the service portal") Navigate to **`http://localhost:8501`**. From the sidebar, open **Create Tenant**. ## Fill the form[​](#fill-the-form "Direct link to Fill the form") | Field | Description | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Tenant Name** | Free text, for example, `ACME-Corp`. | | **MAC VRF VNI Base** | Number (1–16,777,000), default `20000`. VLAN VNI = base + VLAN ID, so pick a base that leaves enough headroom. `20000` supports VLANs 1–4094 without overlapping another base of `25000`, for example. | | **Target Fabrics** | Multi-select of existing fabrics. The tenant is associated with every fabric you select; network segments can then be created on any of them. Defaults to the first fabric. | ## Submit[​](#submit "Direct link to Submit") Click **Create Tenant**. The portal performs: 1. **Create branch** named `add-tenant-`. 2. **Create the tenant** (`EvpnTenant`) with the chosen VNI base and linked fabrics. 3. **Run the AVD pipeline** — hostvars and structured configs regenerate for devices on the selected fabrics. This can take a few minutes on larger fabrics. 4. **Create a proposed change** summarising the tenant. Click **View Proposed Change** when done. ## Review and merge[​](#review-and-merge "Direct link to Review and merge") The tenant itself doesn't add device-level configuration (no VRFs or VLANs have been created yet), so the updated AVD artifacts may be near-identical to before. The tenant becomes useful once you add network segments under it (see [Add a Network Segment](/arista-avd/how-to/add-network-segment.md)). Merge the proposed change to promote the tenant to `main`. ## Picking a VNI base[​](#picking-a-vni-base "Direct link to Picking a VNI base") If you'll only have one tenant ever, `20000` is fine. For multiple tenants, reserve non-overlapping ranges: | Tenant | Base | Effective range (assuming VLANs 1–4094) | | -------- | ----- | --------------------------------------- | | tenant-a | 20000 | 20001 – 24094 | | tenant-b | 25000 | 25001 – 29094 | | tenant-c | 30000 | 30001 – 34094 | This avoids VNI collisions across tenants on the same fabric. ## If the service portal is unavailable[​](#if-the-service-portal-is-unavailable "Direct link to If the service portal is unavailable") In the Infrahub UI: 1. Create a branch. 2. Create an `EvpnTenant` object. Set `name`, `mac_vrf_vni_base`, and link to the target fabrics. 3. Run **`generate-avd-device-hostvar`** (per device on each target fabric) and **`generate-avd-device-structured-config`** (per fabric). 4. Open a proposed change from the branch. ## Source[​](#source "Direct link to Source") Service-portal implementation: [`service_catalog/pages/3_Create_Tenant.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/service_catalog/pages/3_Create_Tenant.py). --- # Regenerate a fabric The Fabric Design page in the service portal is the interactive view of a fabric. From it you can inspect the topology and cabling, see fabric settings and EVPN tenants, and trigger a full regeneration of the fabric — devices, cabling, hostvars, and structured configs — from a single button. ## Open the fabric design page[​](#open-the-fabric-design-page "Direct link to Open the fabric design page") Navigate to **`http://localhost:8501`**. From the sidebar, open **Fabric View** (the page title is **Fabric Design View**). ## Pick branch and fabric[​](#pick-branch-and-fabric "Direct link to Pick branch and fabric") * **Select Branch** (sidebar dropdown) — choose which Infrahub branch to view. For inspection, any branch is fine. For regeneration, use a non-default branch so the change is isolated and reviewable. * **Select Fabric** (main area) — choose the fabric to view, for example, `Fabric-L3LS-MultiPod-A`. ## Tabs[​](#tabs "Direct link to Tabs") The page has four tabs: | Tab | What it shows | | -------------------- | ------------------------------------------------------------------------------------------------------------------- | | **Design Topology** | Hierarchical view of the fabric: pods, racks, devices. Useful to confirm the shape of the fabric before generating. | | **Cabling Topology** | Physical cabling map — every link between devices. Useful to spot missing or misrouted cabling. | | **Fabric Settings** | Underlay/overlay protocols, MTU, spanning-tree configuration. | | **EVPN Tenants** | Tenants associated with the fabric, their VRFs, SVIs, and L2 VLANs. | ## Regenerate the fabric[​](#regenerate-the-fabric "Direct link to Regenerate the fabric") In the **Generate Fabric** section (typically above the tabs or in a side panel depending on your window size): 1. Accept the auto-generated **Branch name** (format `generate--`) or edit it. 2. Click the **Generate** button (primary-styled). The portal triggers the full generator chain — `generate-fabric` → `generate-pod` → `generate-rack` → `generate-avd-device-hostvar` → `generate-avd-device-structured-config` — on the named branch. While the chain runs, the portal shows progress. The full run typically takes a few minutes for a small fabric; longer for fabrics with many pods and racks. ## What you get[​](#what-you-get "Direct link to What you get") When the run finishes, the portal creates a proposed change. Click **View Proposed Change** to review: * New or updated devices (if the fabric had none, or if pods/racks have been added since the last generation). * Updated cabling (if cabling changed). * Updated AVD artifacts for every device and the fabric itself. Review and merge as usual. ## When to regenerate a fabric[​](#when-to-regenerate-a-fabric "Direct link to When to regenerate a fabric") * After manually editing IP pools, fabric settings, or device templates that affect code paths in the generators. * After a failed partial run where some generators didn't complete. Re-running is idempotent — existing devices and objects are upserted, not duplicated. * After upgrading the PyAVD version, if the structured config output format has changed. ## Inspecting without regenerating[​](#inspecting-without-regenerating "Direct link to Inspecting without regenerating") You don't need to regenerate to browse. Pick a branch and fabric and switch between the four tabs to answer questions like: * "Is `Fabric-L3LS-MultiPod-A` cabled consistently across pods?" → **Cabling Topology**. * "Which tenants are on `Fabric-L3LS-MultiPod-B`?" → **EVPN Tenants**. * "What MTU is configured for the underlay?" → **Fabric Settings**. ## If the service portal is unavailable[​](#if-the-service-portal-is-unavailable "Direct link to If the service portal is unavailable") You can trigger the generator chain manually in the Infrahub UI: 1. Create a branch. 2. Open **Actions → Generator definitions**. 3. Run **`generate-fabric`** and select the fabric. 4. The chain cascades automatically via event triggers. 5. Create a proposed change from the branch. See [Provision Your First Fabric](/arista-avd/provision-first-fabric.md) for a step-by-step walkthrough of the same chain. ## Source[​](#source "Direct link to Source") Service-portal implementation: [`service_catalog/pages/4_Fabric_View.py`](https://github.com/opsmill/infrahub-arista-avd/blob/main/service_catalog/pages/4_Fabric_View.py). --- # Upgrade AVD version PyAVD (Arista's Python engine) is what renders your device configurations. When a new PyAVD release is available, you move to it and **validate the result on a branch** before anything reaches production. Because both Infrahub and AVD evolve, you confirm compatibility on both sides rather than upgrading in place. The branch-first upgrade flow is a maintainer/operator task — you rebuild the custom image, so it goes beyond the service portal. ## Before you start[​](#before-you-start "Direct link to Before you start") * A running stack (`invoke start`) with at least one fabric already generated, so you have a config baseline to diff against. * Know your current PyAVD version — it is pinned in `pyproject.toml` (`pyavd>=...`). * Check the [AVD documentation](https://avd.arista.com/) for breaking changes in the target version. ## Steps[​](#steps "Direct link to Steps") 1. **Create a branch for the upgrade.** Create a named Infrahub branch (for example `upgrade-avd`) so the change is isolated and reviewable. 2. **Bump the PyAVD pin.** Update the `pyavd` version in `pyproject.toml`, then re-sync and rebuild the custom image: ``` uv sync --all-packages uv run invoke build uv run invoke restart ``` 3. **Regenerate on the branch.** Re-run the AVD generators so host\_vars and structured configuration are rebuilt with the new PyAVD version. Regeneration is idempotent (checksum-based), so it only reprocesses what the new version changes — see [Regenerate a Fabric](/arista-avd/how-to/regenerate-fabric.md). 4. **Review the rendered-config diff.** Open a proposed change from your branch and inspect the diff of the rendered EOS configurations and structured config. This is where a PyAVD version bump shows its effect — look for unexpected changes to interfaces, BGP, or EVPN stanzas. 5. **Validate.** Confirm the rendered artifacts build cleanly and the diff matches the release notes' expected changes. If ANTA catalog generation is enabled, regenerate the catalogs and review them too. 6. **Merge through the proposed change.** Once the diff is understood and approved, merge the proposed change. Only then does the new PyAVD version reach production. ## If something looks wrong[​](#if-something-looks-wrong "Direct link to If something looks wrong") * A large or surprising diff points to a PyAVD default or schema change between versions — cross-check the release notes. * Roll back by discarding the branch (nothing merged, nothing deployed) and pinning the previous PyAVD version. * For pipeline-level failures, see [Debugging the Pipeline](/arista-avd/developer-guide/avd/debugging.md). --- # Provision your first fabric Prerequisites: [Quick Start](/arista-avd/quick-start.md) complete — Infrahub is running at `http://localhost:8000`, and seed data (fabrics, pods, racks, device types, IP pools) is loaded. At this point you have fabrics defined but **no devices**. The steps below generate the devices, cabling, hostvars, and configurations for `Fabric-L3LS-MultiPod-A`. ## The generator chain[​](#the-generator-chain "Direct link to The generator chain") The project ships four generators that must run in a specific sequence. You trigger the first one; each subsequent generator is triggered automatically by the previous one finishing. | Step | Generator | What it creates | | ---- | ----------------------------------------- | ---------------------------------------------------------------------------------------- | | 1 | **generate-fabric** | Super-spine switches using the fabric loopback, VTEP, management, ASN, and node ID pools | | 2 | **generate-pod** | Spine switches for each pod | | 3 | **generate-rack** | Leaf switches for each rack | | 4 | **generate-avd-device-hostvar** | Per-device PyAVD hostvars (stored in the graph as an `AvdHostvarFile`) | | 5 | **generate-avd-device-structured-config** | Per-device structured AVD config (stored as `AvdStructuredConfigFile`) | ## Step 1 — Create a branch[​](#step-1--create-a-branch "Direct link to Step 1 — Create a branch") Do this work on a branch so the changes stay isolated and you can review them as a proposed change before bringing them into `main`. In the Infrahub UI: click the branch selector in the top bar, then **+ Create branch**, and name it something like `generate-fabric-l3ls-multipod-a`. You can also create a branch from the CLI: ``` uv run infrahubctl branch create generate-fabric-l3ls-multipod-a ``` The CLI route needs credentials in your shell — either `source .envrc` first or set `INFRAHUB_USERNAME`/`INFRAHUB_PASSWORD` (or `INFRAHUB_API_TOKEN`). If you take the CLI route, also switch the UI's branch selector to the new branch — subsequent UI actions need to be scoped there. ## Step 2 — Run the fabric generator[​](#step-2--run-the-fabric-generator "Direct link to Step 2 — Run the fabric generator") 1. In the Infrahub UI, open **Actions → Generator definitions** from the main menu. 2. Find **`generate-fabric`** in the list and click it. 3. In the generator page, click the **Run** button. 4. Select the target fabric (`Fabric-L3LS-MultiPod-A`) from the dropdown. 5. Click **Run** to start. Infrahub queues the generator and shows progress. The fabric generator itself takes under a minute. ## Step 3 — Watch the chain run[​](#step-3--watch-the-chain-run "Direct link to Step 3 — Watch the chain run") You don't need to manually trigger the pod, rack, and AVD generators — they are chained via event triggers. In the UI: 1. Open **Actions → Tasks** (or watch the running-task indicator in the navbar). 2. Tasks appear in this order: * `generate-fabric` (1 task, per fabric) * `generate-pod` (one per pod in the fabric) * `generate-rack` (one per rack in the fabric) * `generate-avd-device-hostvar` (one per device created — super-spines, spines, leaves) * `generate-avd-device-structured-config` (one task for the whole fabric, runs after all hostvars are ready) The full chain typically takes a few minutes depending on fabric size. ## Step 4 — Verify devices exist[​](#step-4--verify-devices-exist "Direct link to Step 4 — Verify devices exist") Once all tasks complete, open **Devices → All Devices** in the menu. You should see devices with roles: * `super_spine` — top of the fabric * `spine` — one per pod * `leaf` — one or more per rack Each device has a BGP ASN, a node ID, a loopback IP, a management IP, and interfaces with IP addresses assigned from the fabric's pools. ## Step 5 — Render the AVD artifacts[​](#step-5--render-the-avd-artifacts "Direct link to Step 5 — Render the AVD artifacts") The per-device AVD artifacts (EOS configs, device documentation) can be rendered two ways: * **Manually on the branch**: open any device, switch to the **Artifacts** tab, and click **Regenerate** on each artifact. Fine for spot-checking one device. * **In a proposed change** (recommended for a full review): open a proposed change from the branch and the pipeline renders artifacts for every device in one step as part of the review. To open a proposed change: 1. Switch to **Branches** in the menu and select your branch. 2. Click **Create Proposed Change**. 3. Fill in a name and description and submit. ## Step 6 — Verify AVD artifacts are rendered[​](#step-6--verify-avd-artifacts-are-rendered "Direct link to Step 6 — Verify AVD artifacts are rendered") Open the **Artifacts** tab — either on the proposed change for the full set, or on an individual device — to see: * **AVD EOS Configuration** — Arista EOS CLI config for the device. * **AVD Device Documentation** — Markdown documentation for the device. * **AVD Fabric Documentation** — full fabric markdown documentation. See [Viewing Artifacts](/arista-avd/viewing-artifacts.md) for how to open and download each artifact. ## Step 7 — Merge the branch (optional)[​](#step-7--merge-the-branch-optional "Direct link to Step 7 — Merge the branch (optional)") Once you're happy with the results, bring the branch into `main`. Two options: * **Merge the proposed change** after review — the standard Git-style flow with diff inspection. * **Merge the branch directly** — from **Branches → your branch → Merge**, no proposed change needed. Faster but skips the review surface. You can now move on to day-2 workflows: [Add a Network Segment](/arista-avd/how-to/add-network-segment.md), [Add a Server](/arista-avd/how-to/add-server.md), [Create a Tenant](/arista-avd/how-to/create-tenant.md), or [Regenerate a Fabric](/arista-avd/how-to/regenerate-fabric.md). ## If something goes wrong[​](#if-something-goes-wrong "Direct link to If something goes wrong") The most common failures are documented in [Common Issues](/arista-avd/troubleshooting.md): * The fabric generator completes but no spines or leaves appear. * A task hangs in "running" state. * An artifact shows `no structured config available`. --- # Quick start The steps below take you from a fresh clone to a running Infrahub instance with seed data loaded. After this, see [Provision Your First Fabric](/arista-avd/provision-first-fabric.md) to generate devices, configurations, and AVD artifacts. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * [Docker](https://docs.docker.com/get-docker/) and Docker Compose. * [`uv`](https://docs.astral.sh/uv/getting-started/installation/) — the Python package manager this project uses. * Python 3.11 or newer. Everything else is installed by `uv sync` inside the project. ## 1. Install dependencies[​](#1-install-dependencies "Direct link to 1. Install dependencies") From the repository root: ``` uv sync --all-packages ``` This creates a virtualenv under `.venv/` and installs the project and its dependencies, including `pyavd` and the Infrahub SDK. ## 2. Build the custom Infrahub image[​](#2-build-the-custom-infrahub-image "Direct link to 2. Build the custom Infrahub image") The project extends the base Infrahub image with `pyavd` and project code. Build the image once: ``` uv run invoke build ``` To build against a different Infrahub release, set `INFRAHUB_BASE_VERSION` first — the compose files default to `1.10.10`: ``` export INFRAHUB_BASE_VERSION= uv run invoke build ``` Re-run this only after changes to `Dockerfile` or the Python dependencies. `invoke build --no-cache` forces a clean rebuild. ## 3. Start the stack[​](#3-start-the-stack "Direct link to 3. Start the stack") ``` uv run invoke start ``` This brings up, in the background: | Service | URL | Purpose | | -------------- | ----------------------- | -------------------------------------------------------------------- | | Infrahub UI | `http://localhost:8000` | Main web interface | | Service Portal | `http://localhost:8501` | Streamlit self-service portal | | Semaphore | `http://localhost:3000` | Ansible automation runner | | Neo4j Browser | `http://localhost:7474` | Graph database browser | | Prefect | `http://localhost:4200` | Task-manager UI — where generator, transform, and check runs show up | `invoke start` also creates `lab/clab-staging/` before compose runs, so the Semaphore container has a writable bind-mount source for [ContainerLab](/arista-avd/containerlab.md) files. Wait for services to become healthy. You can check with: ``` docker compose -f docker-compose.yml -f docker-compose.override.yml ps ``` All services should show `healthy` or `running`. Infrahub is ready once `http://localhost:8000` responds. ## 4. Load schemas, menus, objects, and repository[​](#4-load-schemas-menus-objects-and-repository "Direct link to 4. Load schemas, menus, objects, and repository") Once Infrahub is healthy, load everything in one command: ``` uv run invoke load ``` This runs, in order: 1. Initialise Semaphore (idempotent — safe to re-run). 2. Load schemas from `schemas/`. 3. Load the UI menu from `menus/`. 4. Load seed data from `objects/` — manufacturers, device types, IP pools, profiles, device templates, fabrics, racks, VLANs. 5. Register this repository with Infrahub and wait for it to reach `in-sync`. 6. Load the check queries from `repository_checks.yml`, which depend on the repository being synced. 7. Load event triggers and rules from `triggers.yml`. Seed data loads in filename order, and the numeric prefixes encode that order: shared data first (`00`–`06` — groups, manufacturers, device types, IPAM, management, profiles, device templates), then the example fabrics (`10`–`15`), each with its own fabric, rack, service, and server files. ## 5. Confirm everything loaded[​](#5-confirm-everything-loaded "Direct link to 5. Confirm everything loaded") Open the Infrahub UI at **`http://localhost:8000`** and log in. You should see: * **Devices → Types & Models → Manufacturers**: Arista, Dell, and other manufacturers. * **Fabric Design → Fabrics**: `Fabric-L3LS-MultiPod-A` and `Fabric-L3LS-MultiPod-B` with their pods. * **Locations → Racks**: pre-defined racks per pod. * **IPAM → Prefixes**: the fabric supernet and per-fabric prefix pools. If you don't see these, re-run `uv run invoke load` or see [Common Issues](/arista-avd/troubleshooting.md). ## Next: provision a fabric[​](#next-provision-a-fabric "Direct link to Next: provision a fabric") The stack is up but no devices exist yet — fabrics, pods, and racks are defined but leaves, spines, and super-spines need to be generated. Follow [Provision Your First Fabric](/arista-avd/provision-first-fabric.md) next. ## Common commands[​](#common-commands "Direct link to Common commands") | Command | What it does | | --------------------------------------------------- | ------------------------------------------------------------------ | | `uv run invoke start` | Start all services | | `uv run invoke stop` | Stop containers, keep volumes | | `uv run invoke destroy` | Stop and **remove** containers, networks, and volumes (wipes data) | | `uv run invoke restart` | Restart all services | | `uv run invoke restart --component=infrahub-server` | Restart a specific service | | `uv run invoke load` | Re-run the full load sequence | | `uv run invoke load-schema` | Reload schemas only | | `uv run invoke load-menu` | Reload UI menus only | | `uv run invoke init-semaphore` | Re-register the Semaphore project and templates (idempotent) | | `uv run invoke test` | Run the test suite, then Ruff and mypy | | `uv run invoke lint` | Ruff, yamllint, and mypy | | `uv run invoke format` | Apply Ruff formatting | `uv run invoke --list` shows the full set. --- # Supported capabilities This is a **reference design** that covers a defined set of AVD capabilities on Infrahub — it is not a full replacement for every AVD feature. Uncommon or highly customized AVD options may not be modeled. Use the matrix below to check the status of a capability before planning a deployment. **Status key:** ✅ Supported today · 🟡 Partial / confirm scope · ⬜ Not yet note Some boundaries below are marked *confirm scope* and are being finalized with the maintainers. Where a row says `confirm`, treat the exact edge as undecided rather than guaranteed. ## AVD example scenario coverage[​](#avd-example-scenario-coverage "Direct link to AVD example scenario coverage") Status of the official [AVD example designs](https://avd.arista.com/6.2/ansible_collections/arista/avd/examples/index.html) this reference design covers. Gaps are closed with **native schema** where a capability is reusable and first-class, and with the **`avd_custom_hostvars` escape hatch** where a full native model would be disproportionate (niche, single-scenario, pass-through). Each scenario has its own loadable fabric design; every device renders valid PyAVD EOS configuration (0 validation violations) on a fresh instance. | AVD example scenario | Status | Fabric | Design | | --------------------------------------- | ------ | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Single-DC L3LS | ✅ | `Fabric-L3LS` | eBGP underlay, EVPN/VXLAN L3LS. | | Single-DC Multi-Pod L3LS (5-stage Clos) | ✅ | `Fabric-L3LS-MultiPod-A` | 6 super-spines + 3 pods; super-spines as EVPN route servers; tenants as vlan-aware bundles (`evpn_vlan_aware_bundles`). | | Dual-DC L3LS | ✅ | `Fabric-L3LS-Multi-Domain` | EVPN DC Gateway (next-hop-self) + DCI `l3_edge` p2p links, via `avd_custom_hostvars`. | | L2LS fabric (standalone) | ✅ | `Fabric-L2LS` | underlay `none` → `l2spine` + `l2leaf`, pure Layer-2 (no VNI/VXLAN/EVPN), MLAG both tiers, two MLAG rack pairs; overlay-free `Evpn.Tenant` (`MY_FABRIC`) with tag-scoped VLANs `BLUE-NET`/`GREEN-NET`/`ORANGE-NET` and per-tier MSTP priorities (l2spine 4096 / l2leaf 16384), mirroring the AVD `l2ls-fabric` example. Host endpoints sit on leaf access ports via per-color access profiles (one untagged VLAN each, PortFast `edge`). The example's dual-homed `FIREWALL` (trunk Port-Channel to both spines) is **not modeled** — deliberately deferred, see below. | | Campus fabric | ✅ | `Fabric-Campus` | underlay `ospf` → `l3spine` core with anycast SVIs (`Evpn.Svi`) + `l2leaf` access; dot1x/PoE via escape hatch. | | ISIS-LDP IPVPN | ✅ | `Fabric-ISIS-LDP` | underlay `isis-ldp` → `p` core + `pe` edge; per-customer L3VPN VRFs (`Evpn.Tenant`/`Ipam.VRF`). | The non-L3LS designs are driven by the fabric `underlay_routing_protocol`, which the pod/rack generators map to the correct device roles (gated so eBGP L3LS fabrics are unaffected). **Deferred — L2LS spine-attached firewall.** The upstream `l2ls-fabric` example dual-homes a firewall to both `l2spine` switches as a trunk Port-Channel. Modeling it needs a connected endpoint attachable to spine-tier devices, which the current endpoint/cabling path does not cover (it cables endpoints to a rack's leaves). It was scoped out rather than forced through `avd_custom_hostvars`, so the L2LS example is feature-complete for the fabric, services and host endpoints but includes no firewall. Everything else in that design is modeled natively. Services are modeled **schema-first**: L2 VLANs (L2LS), anycast SVIs on the campus l3spine core, and per-customer L3VPN VRFs on the ISIS-LDP PE are all `Ipam.VLAN` / `Evpn.Tenant` / `Ipam.VRF` / `Evpn.Svi` **objects** rendered by the generator — the same service model as the L3LS fabrics. The `avd_custom_hostvars` escape hatch is reserved for capabilities the schema does not yet model — EVPN DC Gateway remote-peers and campus dot1x/PoE — a deliberate, documented niche exception (native modeling of those is future schema work). Native-vs-escape-hatch guidance is in the [developer guide](/arista-avd/developer-guide/avd/extending.md). ## Fabric generation[​](#fabric-generation "Direct link to Fabric generation") | Capability | Status | Notes | | ------------------------------------------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------- | | Generate a full fabric (Fabric → Pod → Rack → Device) from a design | ✅ | Super-spines, spines, and leaves are created from device templates — no per-device host\_vars authored manually. | | Cable devices together automatically | ✅ | Uplinks and device-to-device links created by the generators. | | Regenerate idempotently | ✅ | Checksum-based change detection skips work when nothing changed; re-running is safe. | ## Addressing & numbering[​](#addressing--numbering "Direct link to Addressing & numbering") | Capability | Status | Notes | | ----------------------------------------------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Allocate loopback, interconnect, and management prefixes/IPs from pools | ✅ | Drawn from branch-aware pools so parallel work does not collide. | | Allocate DCI point-to-point /31 prefixes from fabric DCI pool roles | ✅ | Generated DCI `l3_edge` addressing resolves from `NetworkFabric.fabric_ip_pools` role `dci` first, legacy `NetworkFabric.dci_pool` second, and deterministic Fabric Supernet fallback when the required DCI prefix-pool role is missing. | | Allocate BGP ASNs and node IDs from pools | ✅ | Assigned automatically during generation. | ## Services (VLAN / EVPN / VRF / MLAG / LAG / routing)[​](#services-vlan--evpn--vrf--mlag--lag--routing "Direct link to Services (VLAN / EVPN / VRF / MLAG / LAG / routing)") | Capability | Status | Notes | | ----------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Model VLANs and L2 domains | ✅ | Defined in the source of truth and rendered into config. | | Fabric-level EVPN settings | ✅ | Fabric EVPN overlay configuration. Exact EVPN depth is being confirmed. | | EVPN Multi-Domain Gateway on Border Leafs | 🟡 | Models Fabric-owned `EvpnDomain` objects with domain-owned local `EvpnGatewayGroup` children for `border_leaf` devices, then emits PyAVD EVPN Gateway hostvars for All-Active Multihoming only. Pods remain selected context and must point at the group's local domain. MLAG, Anycast IP, route-server, and route-reflector gateway models are not included. | | EVPN L3 VRFs | 🟡 | Wired into the PyAVD hostvar generator and produce config. The maintainers flagged `we don't do VRF and route targets` — **confirm** whether the exclusion is VRF-lite, route-leaking, or explicit route targets. | | MLAG (domain + peer) | 🟡 | Modeled and wired into hostvars; **confirm** supported scope. | | Server LAG | 🟡 | Modeled and wired into hostvars; **confirm** supported scope. | | BGP peer groups | 🟡 | Wired into hostvars and produce config; **confirm** supported scope. | | DCI links between Border Leafs | ✅ | `NetworkLink` objects with `role=dci` reuse shared physical endpoints and generate PyAVD `l3_edge.p2p_links`; external networks and EVPN Gateway are out of scope for this phase. | | Route targets | 🟡 | Modeled but AVD-derived (not fed as input). | | Prefix lists, route maps, static routes | 🟡 | Reconciled *from* AVD output via the backfill generator, not authored as inputs. | ## Rendering & artifacts[​](#rendering--artifacts "Direct link to Rendering & artifacts") | Capability | Status | Notes | | ----------------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------ | | Render Arista EOS device configurations (PyAVD) | ✅ | Deploy-ready per-device EOS CLI, as downloadable artifacts. | | Fabric and per-device documentation (Markdown) | ✅ | Generated from the same source of truth as the config. | | Cabling plan (CSV) | ✅ | One row per connection for the field/cabling team. | | Computed interface descriptions | ✅ | Consistent, auto-maintained interface descriptions. | | ANTA test catalog (per device, YAML) | ✅ | Catalog **generation** is included (gated by the fabric `anta_enabled` flag). Execution is not yet included — see below. | ## Validation (ANTA)[​](#validation-anta "Direct link to Validation (ANTA)") | Capability | Status | Notes | | --------------------------------------- | ------ | ------------------------------------------------------------------- | | ANTA test-catalog generation | ✅ | The `avd_anta_catalog` transform, gated by `anta_enabled`. | | ANTA execution / block-merge-on-failure | ⬜ | Running the tests and blocking merges on failure is on the roadmap. | ## Validation (CloudVision)[​](#validation-cloudvision "Direct link to Validation (CloudVision)") | Capability | Status | Notes | | -------------------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | CloudVision config validation in a proposed change | ✅ | The `cv-config-validation` check deploys the rendered configs to a CloudVision workspace and blocks the proposed change on a failed build. Opt in per fabric with `cloudvision_managed`. See [CloudVision Validation](/arista-avd/cloudvision.md). | | Workspace tracking and review link | ✅ | Each workspace is recorded as a `CloudvisionWorkspace` object and its URL posted to the proposed change. | | Workspace submission | 🟡 | Submission runs from a `CoreCustomWebhook` on proposed-change merge, or manually via `invoke submit-cv-workspace`. The shipped webhook target is a placeholder URL, not a production receiver. | | CloudVision change-control management | ⬜ | Out of scope for this phase. | ## Lab[​](#lab "Direct link to Lab") | Capability | Status | Notes | | ------------------------------------------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ContainerLab topology per fabric | ✅ | The `containerlab_topology` artifact renders every device and link the fabric owns; kinds, images, and interface mappings come from schema attributes. See [ContainerLab](/arista-avd/containerlab.md). | | Deploying the generated topology | ✅ | `ansible/deploy_clab.yml` stages the topology, EOS configs, and bind sources on a ContainerLab host and deploys them. The Semaphore template fetches and stages only — it does not deploy. | | ISIS-LDP devices in the generated topology | ⬜ | The `p`, `pe`, and `rr` roles are excluded; their interface naming is not validated against ContainerLab. | ## Deployment[​](#deployment "Direct link to Deployment") | Capability | Status | Notes | | -------------------------------- | ------ | -------------------------------------------------------------- | | Deploy configurations to devices | ✅ | Through the bundled Ansible runner or CloudVision (CVP/CVaaS). | ## Interfaces & change management[​](#interfaces--change-management "Direct link to Interfaces & change management") | Capability | Status | Notes | | ------------------------------------------------------- | ------ | ----------------------------------------------------------------------------- | | Self-service portal (Streamlit) for guided provisioning | ✅ | Alongside the Infrahub Web UI, GraphQL API, and MCP. | | Branches, proposed changes, approvals, full lineage | ✅ | Standard Infrahub platform change management. | | Approval rules that vary by service type | ⬜ | You can require approvals, but per-service approval rules are on the roadmap. | ## Brownfield & coverage[​](#brownfield--coverage "Direct link to Brownfield & coverage") | Capability | Status | Notes | | ---------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | Self-serve brownfield import | 🟡 | Modeling an existing fabric and importing configs via Infrahub Sync is done today in a **guided engagement**, not as a download-and-try path. | | Every AVD feature | ⬜ | This reference design covers a defined set of AVD inputs and scenarios, implemented per customer; uncommon or highly custom options may not be modeled. | ## Fabric pool management[​](#fabric-pool-management "Direct link to Fabric pool management") | Capability | Status | Notes | | ----------------------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | | Role-driven fabric pool collection | ✅ | `NetworkFabric.fabric_ip_pools` covers Management, Loopback, Loopback VTEP, Fabric Point-to-Point, DCI, and Fabric Supernet roles. | | Pod-scoped pool collection and containment validation | ✅ | `NetworkPod.pod_ip_pools` supports pod Loopback, VTEP, Fabric Point-to-Point, MLAG, and MLAG Peering roles with parent-fabric containment checks. | | Legacy pool migration compatibility | ✅ | Legacy fabric and pod pool relationships remain optional and seed data is dual-populated during migration. | | Deterministic fallback/default pools | ✅ | Fabric Supernet fallback creates stable prefix pools; MLAG defaults use stable `/31` pool objects. | --- # Common issues The failure modes below are the ones you'll hit most often. If your problem isn't here, the [developer guide](/arista-avd/developer-guide.md) has deeper debugging material for contributors. ## Stack is not healthy[​](#stack-is-not-healthy "Direct link to Stack is not healthy") ### Symptoms[​](#symptoms "Direct link to Symptoms") * `http://localhost:8000` doesn't respond. * `uv run invoke load` fails with connection errors. * Service portal at `http://localhost:8501` loads but shows `Unable to fetch data from Infrahub`. ### Diagnose[​](#diagnose "Direct link to Diagnose") ``` docker compose -f docker-compose.yml -f docker-compose.override.yml ps ``` Look for any service that is **not** `healthy` or `running`. The usual culprits are: * **Neo4j** — first-time startup takes 30–60 seconds. Re-check after a minute. * **PostgreSQL** — check the volume mount has write permission (`ls -l` on the docker volume location). * **RabbitMQ / Redis** — restart with `uv run invoke restart --component=`. Check logs for a failing service: ``` docker compose -f docker-compose.yml -f docker-compose.override.yml logs --tail 100 ``` ### Fix[​](#fix "Direct link to Fix") If restarting doesn't help, tear down and rebuild: ``` uv run invoke destroy # removes all volumes — your data is wiped uv run invoke start uv run invoke load ``` warning `invoke destroy` removes all volumes, including your graph database. Use this only if you're okay losing local data. ## Generators ran out of order[​](#generators-ran-out-of-order "Direct link to Generators ran out of order") ### Symptoms[​](#symptoms-1 "Direct link to Symptoms") * `generate-fabric` finishes but no pod, rack, or device tasks appear afterwards. * A leaf device is created but has no BGP ASN, node ID, or IP address. * Hostvars generation fails with `parent not found`. ### Cause[​](#cause "Direct link to Cause") Triggers connect the generators into a chain: `generate-fabric` → `generate-pod` → `generate-rack` → `generate-avd-device-hostvar` → `generate-avd-device-structured-config`. If triggers didn't load — the last step of `invoke load` — the chain is broken. ### Diagnose[​](#diagnose-1 "Direct link to Diagnose") In the Infrahub UI, open **Governance → Triggers**. You should see trigger entries covering each generator. If the Triggers page is empty, the trigger load step didn't run. ### Fix[​](#fix-1 "Direct link to Fix") Re-run the trigger load step: ``` uv run invoke load ``` This is idempotent and re-registers triggers. If the generators already ran out of order and left partial data, the cleanest fix is to discard the branch (Infrahub UI → Branches → your branch → Delete) and re-run from a fresh branch. ## Missing seed data[​](#missing-seed-data "Direct link to Missing seed data") ### Symptoms[​](#symptoms-2 "Direct link to Symptoms") * Running the fabric generator returns an error about missing IP pools, ASN pools, or device templates. * The fabrics list is empty. * Manufacturer or device-type lookups fail. ### Diagnose[​](#diagnose-2 "Direct link to Diagnose") Open the Infrahub UI and check each of these lists is populated: | Menu | Expected | | ---------------------------------------- | ----------------------------------------------------- | | Devices → Types & Models → Manufacturers | Arista, Dell, and others | | Devices → Types & Models → Device Types | Arista models (7050-CX3, etc.) | | Fabric Design → Fabrics | `Fabric-L3LS-MultiPod-A` and `Fabric-L3LS-MultiPod-B` | | IPAM → Prefixes | supernet and per-fabric pools | If any are empty, seed data did not load. ### Fix[​](#fix-2 "Direct link to Fix") Re-run seed data load. From the repository root: ``` uv run invoke load ``` This re-loads schemas, menus, objects, repository, and triggers. It is idempotent — existing objects are upserted, not duplicated. If `invoke load` fails on the `infrahubctl object load` step, the most common cause is Infrahub not being ready yet. Wait 10–20 seconds and re-run. ## "No structured config available" when viewing an artifact[​](#no-structured-config-available-when-viewing-an-artifact "Direct link to \"No structured config available\" when viewing an artifact") ### Symptoms[​](#symptoms-3 "Direct link to Symptoms") * The device's **AVD EOS Configuration** artifact opens but shows "No structured config available" or similar. * The device has hostvars (you can see `AvdHostvarFile` listed for it) but no `AvdStructuredConfigFile`. ### Cause[​](#cause-1 "Direct link to Cause") The structured-config generator (Phase 2 of the AVD pipeline) runs per fabric, not per device. It reads all device hostvars and generates structured configs for all devices at once. If that generator hasn't run, or ran before the hostvars for this device were ready, the structured config is missing. ### Fix[​](#fix-3 "Direct link to Fix") Re-run the structured-config generator for the fabric: 1. In the Infrahub UI, on the correct branch, open **Actions → Generator definitions**. 2. Click **`generate-avd-device-structured-config`**. 3. Click **Run** and select the fabric (for example, `Fabric-L3LS-MultiPod-A`). Once the task completes, the artifact renders on the next open. Alternatively, in the artifact preview panel, click **Regenerate**. ## The ANTA catalog artifact contains only a comment[​](#the-anta-catalog-artifact-contains-only-a-comment "Direct link to The ANTA catalog artifact contains only a comment") ### Symptoms[​](#symptoms-4 "Direct link to Symptoms") The **AVD ANTA Catalog** artifact renders successfully but holds a single line: ``` # ANTA disabled for fabric Fabric-L3LS-MultiPod-A ``` ### Cause[​](#cause-2 "Direct link to Cause") Catalog generation is opt-in per fabric. The transform checks `NetworkFabric.anta_enabled` and returns a marker comment rather than failing, so the artifact always renders and states why it is empty. ### Fix[​](#fix-4 "Direct link to Fix") Set `anta_enabled` on the fabric — on a branch, as with any data change — and regenerate. Two other markers point elsewhere: | Marker | Meaning | | ---------------------------------------- | ------------------------------------------------------------------------------------------------ | | `# No structured config for ` | The structured-config generator hasn't produced a config for that device — see the section above | | `# ANTA catalog: no fabric for ` | The device has no pod, or its pod has no parent fabric | ## CloudVision validation is skipped or fails[​](#cloudvision-validation-is-skipped-or-fails "Direct link to CloudVision validation is skipped or fails") ### Symptoms[​](#symptoms-5 "Direct link to Symptoms") * The `cv-config-validation` check reports an informational skip on a proposed change. * The check fails with a CloudVision connection or authentication error. ### Diagnose and fix[​](#diagnose-and-fix "Direct link to Diagnose and fix") | What you see | Cause | Fix | | --------------------------------------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Check skips everything | The fabric has `cloudvision_managed` set to `false` | Set it on the fabric; unmanaged fabrics skip credentials, serial, inventory, and workspace checks by design | | Connection or auth failure | `CLOUDVISION_SERVERS` / `CLOUDVISION_TOKEN` not reaching the task worker | `docker-compose.override.yml` forwards them from your shell (or a `.env` file) into the Infrahub containers, so set them and re-run `uv run invoke start` — `restart` reuses the existing containers and their old environment | | "device has no serial number" or an inventory error | A confirmed device in the fabric is missing a serial, or isn't in CloudVision inventory | Fix the device data, or remove it from the fabric before validating | | Informational skip, workspace never created | No device in the fabric has a generated structured config yet | Run the structured-config generator first | See [CloudVision Validation](/arista-avd/cloudvision.md) for the full behaviour and [Checks](/arista-avd/developer-guide/checks.md) for how to run the check directly. ## Service portal is down but Infrahub is up[​](#service-portal-is-down-but-infrahub-is-up "Direct link to Service portal is down but Infrahub is up") ### Symptoms[​](#symptoms-6 "Direct link to Symptoms") * `http://localhost:8000` works. * `http://localhost:8501` returns connection refused or a blank page. ### Fix[​](#fix-5 "Direct link to Fix") Restart the service portal container: ``` uv run invoke restart --component=service-catalog ``` The compose service is named `service-catalog`; `--component` takes a compose service name, which you can list with: ``` docker compose -f docker-compose.yml -f docker-compose.override.yml config --services ``` If the portal is unavailable and you need to complete a workflow, you can do most operations directly in the Infrahub UI: * **Add Network Segment** — create `IpamVLAN`, `IpamVRF`, and `EvpnSvi` objects manually in the UI on a branch, then run the hostvars and structured-config generators. * **Create Tenant** — create an `EvpnTenant` object in the UI. * **Add Server** — create a `ComputePhysicalServer` linked to a compute rack, then re-run the generator chain. ## Generators take longer than expected[​](#generators-take-longer-than-expected "Direct link to Generators take longer than expected") Normal durations on a typical laptop: | Generator | Target | Duration | | --------------------------------------- | ---------- | --------------------------------------------------- | | `generate-fabric` | one fabric | 10–30 s | | `generate-pod` | one pod | 10–20 s | | `generate-rack` | one rack | 10–20 s | | `generate-avd-device-hostvar` | one device | 20–45 s | | `generate-avd-device-structured-config` | one fabric | 1–3 min for a small fabric; longer for many devices | If a task has been in **Running** state for more than 10 minutes, check the task's log in **Actions → Tasks → \[task]** and look for errors. Most long-running tasks are waiting on a missing dependency (an IP pool, a parent object). The log names the missing item. ## Starting over[​](#starting-over-completely "Direct link to Starting over") ``` uv run invoke destroy # wipes everything uv run invoke build uv run invoke start uv run invoke load ``` This is the nuclear option — rebuild the image, restart the stack with empty volumes, and reload everything. --- # Viewing artifacts Once generators have run on a branch and you've opened a proposed change (see [Provision Your First Fabric](/arista-avd/provision-first-fabric.md) or any of the day-2 how-to pages), the proposed-change CI pipeline renders these artifact types: | Artifact | Attached to | Content type | Purpose | | ---------------------------- | -------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **AVD EOS Configuration** | Each `DcimDevice` | `text/plain` | The Arista EOS CLI configuration for that device. | | **AVD Device Documentation** | Each `DcimDevice` | `text/markdown` | Human-readable documentation describing the device. | | **AVD Fabric Documentation** | Each `NetworkFabric` | `text/markdown` | Fabric-wide topology and design documentation. | | **ContainerLab Topology** | Each `NetworkFabric` | `application/yaml` | A [ContainerLab](https://containerlab.dev) topology file for running the fabric as containers. | | **Cabling Plan** | Each `NetworkFabric` | `text/csv` | One row per connection for the field and cabling teams. | | **AVD ANTA Catalog** | Each `DcimDevice` | `application/yaml` | The device's [ANTA](https://anta.arista.com) test catalog. Rendered only when the fabric has `anta_enabled` set; otherwise the artifact holds a one-line comment saying so. | Per-device artifacts (`AVD EOS Configuration`, `AVD Device Documentation`) are rendered as part of the proposed-change CI. If you want to view them outside a proposed change, open them on a device's **Artifacts** tab and click **Regenerate**. ## Finding a device artifact[​](#finding-a-device-artifact "Direct link to Finding a device artifact") 1. In the Infrahub UI, open **Devices → All Devices**. 2. Click a device (for example `leaf-pod-A1-1`). 3. Click the **Artifacts** tab on the device's detail page. 4. You'll see rows for **AVD EOS Configuration** and **AVD Device Documentation**. ## Previewing an artifact[​](#previewing-an-artifact "Direct link to Previewing an artifact") Click the artifact row to open a preview panel. The preview shows: * The rendered content inline. * Metadata: content type, last rendered timestamp, size. * A **Download** button. * A **Regenerate** button (forces a fresh render even if nothing has changed). ### EOS configuration preview[​](#eos-configuration-preview "Direct link to EOS configuration preview") The EOS config is plain text — paste-ready for a lab switch or a virtual Arista instance. Example excerpt: ``` ! hostname leaf-pod-A1-1 ! router bgp 65101 router-id 10.255.1.1 … ``` ### Markdown documentation preview[​](#markdown-documentation-preview "Direct link to Markdown documentation preview") The fabric and device markdown documents include tables, topology descriptions, and interface lists. They render directly in the Infrahub preview. ## Finding a fabric artifact[​](#finding-a-fabric-artifact "Direct link to Finding a fabric artifact") 1. Open **Fabric Design → Fabrics**. 2. Click the fabric (`Fabric-L3LS-MultiPod-A`). 3. Click the **Artifacts** tab. 4. Open **AVD Fabric Documentation** or **ContainerLab Topology**. ### ContainerLab topology preview[​](#containerlab-topology-preview "Direct link to ContainerLab topology preview") The topology is YAML. It names every device the fabric owns as a ContainerLab node — cEOS nodes for the switches, Linux nodes for the servers — plus every fabric link as a `endpoints` pair: ``` topology: nodes: spine-infrahub-dc1-1: kind: arista_ceos mgmt-ipv4: 10.0.6.11 links: - endpoints: ["leaf-infrahub-dc1-1:eth49_1", "spine-infrahub-dc1-1:eth1_1"] ``` Node kinds, container images, and interface-mapping binds come from schema attributes rather than the transform, so changing the cEOS version is a data change. See the [ContainerLab page](/arista-avd/containerlab.md) for the full shape and for how to deploy the topology. ## Downloading artifacts[​](#downloading-artifacts "Direct link to Downloading artifacts") In the preview panel, click **Download**. Content is served with the correct `Content-Type`: * EOS configs save as `.txt`. * Markdown docs save as `.md`. * The ContainerLab topology saves as `.yml`, ready to pass to `containerlab deploy --topo`. ## Regenerating an artifact[​](#regenerating-an-artifact "Direct link to Regenerating an artifact") Artifacts regenerate automatically when the underlying data changes, but you can force a regeneration from the preview panel's **Regenerate** button. Typical reasons to force a regenerate: * You edited a device attribute directly in the UI and want to see the config update. * A previous generator run was interrupted and the artifact is stale. ## What if an artifact is empty or says `no structured config available`?[​](#what-if-an-artifact-is-empty-or-says-no-structured-config-available "Direct link to what-if-an-artifact-is-empty-or-says-no-structured-config-available") This means the structured-config generator hasn't run for the fabric yet. See the [troubleshooting page](/arista-avd/troubleshooting.md) for the fix. ## Downstream consumption[​](#downstream-consumption "Direct link to Downstream consumption") The artifacts are also accessible via the Infrahub API and through Ansible playbooks orchestrated by Semaphore at `http://localhost:3000`. Two playbook trees exist, and they consume different artifacts: * `ansible/` at the repository root — the tree Semaphore runs. `inventory.yml` builds the inventory from Infrahub and `deploy.yml` fetches each device's **AVD EOS Configuration** with `opsmill.infrahub.artifact_fetch`. `deploy_clab.yml` fetches the **ContainerLab Topology** artifact plus every device's EOS config, stages them on a ContainerLab host, and deploys the lab; see the [ContainerLab page](/arista-avd/containerlab.md). * `lab/playbooks/` — the AVD-toolchain playbooks for the committed lab (`build.yml`, `deploy.yml`, `deploy-eapi.yml`, `test.yml`), driven from `lab/Makefile`. Both need the `opsmill.infrahub` collection: `ansible-galaxy collection install -r ansible/galaxy-requirements.yml`. --- # Infrahub Backup The Infrahub Ops CLI provides command-line tools for operating and maintaining an Infrahub deployment: * **[Infrahub Backup](/backup/backup.md)** — create and restore complete, verifiable recovery points on Docker Compose or Kubernetes. * **[Infrahub Collect](/backup/collect.md)** — gather logs, diagnostics, and metrics into a single archive for OpsMill support. Both binaries share the same environment detection, configuration, and self-update mechanism. ## Where to start[​](#where-to-start "Direct link to Where to start") * New to the tools? Start with the [Get started tutorial](/backup/tutorials/getting-started.md) for a full backup and restore cycle. * Ready to install? See [Install Infrahub Backup](/backup/backup/install.md) or [Install Infrahub Collect](/backup/collect/install.md). * Looking for a flag or environment variable? See the [CLI command reference](/backup/reference/commands.md) and [Configuration reference](/backup/reference/configuration.md). * Keeping binaries up to date? See [Update the CLI](/backup/self-update.md). --- # Infrahub Backup Use Infrahub Backup to create and restore verifiable recovery points for an Infrahub deployment. Run it as a CLI against Docker Compose, or as a Helm chart on Kubernetes, to capture the graph database and the task-manager database as a single archive you can restore from later. Artifact storage lives outside those databases, so you snapshot it as a separate, coordinated step to keep a recovery complete. ## Why reliable backups matter[​](#why-reliable-backups-matter "Direct link to Why reliable backups matter") Infrahub keeps critical infrastructure state across several systems: the Neo4j graph database, the PostgreSQL database behind the task manager, and external artifact storage. If any piece is lost or corrupted, you can end up with drifted configuration, orphaned tasks, or a deployment that no longer starts. Regulatory and business-continuity requirements often call for auditable recovery procedures, so ad-hoc database dumps are rarely enough — you need consistent, verifiable recovery points. ## How it works[​](#how-it-works "Direct link to How it works") Infrahub Backup coordinates backup and restore across deployment targets. During a backup it captures each database with the correct credentials and packages everything so a restore is deterministic. Enterprise deployments are backed up online, with no downtime; Community Edition is dumped offline, so the tool first stops the Infrahub services and restarts them once the snapshot is complete. During a restore it brings services back in dependency order, reapplies the data, and records logs so you can confirm the operation succeeded. The CLI runs the provider-specific Docker or Kubernetes commands for you and sequences each step. On Kubernetes, the `infrahub-backup` Helm chart manages backups declaratively through your existing GitOps pipeline. Schedule backups with a CronJob, push archives to S3-compatible storage, and run restores — all without direct kubectl access. ## High availability[​](#high-availability "Direct link to High availability") Infrahub Backup supports high availability (HA) deployments on both Kubernetes and Docker Compose: * **Kubernetes**: HA PostgreSQL is supported through [CloudNativePG](https://cloudnative-pg.io/) only. The tool detects the primary pod and targets it during backup and restore — no extra configuration needed. * **Docker Compose**: HA setups require stopping the `task-manager` and `task-manager-background-svc` containers before a restore, to keep them from accessing the database while data is replaced. See [Restore from a backup](/backup/backup/restore.md#docker-high-availability-deployments). ## Community edition support[​](#community-edition-support "Direct link to Community edition support") Infrahub Backup works with Infrahub Community Edition as well as Enterprise deployments. A backup against Community Edition stops the Infrahub service while the snapshot is taken, so plan for a short interruption. A restore must target the same edition that produced the backup — Enterprise backups cannot be restored to Community Edition. ## Best practices[​](#best-practices "Direct link to Best practices") ### Schedule regular backups[​](#schedule-regular-backups "Direct link to Schedule regular backups") * On Kubernetes, enable the `infrahub-backup` Helm chart in CronJob mode for automated, declarative scheduling that fits your GitOps workflow. * On Docker Compose, automate backups with cron, systemd timers, or CI workflows; daily is a good baseline, with more frequent snapshots for high-change environments. * Configure S3-compatible storage to push backups off-cluster, and apply bucket lifecycle policies for retention. * Rotate archives with a [retention policy](/backup/backup/retention.md) that matches your recovery point objectives, and replicate critical backups to an off-site location. * Monitor backup jobs and alert on failure — silent backup gaps are often noticed only when it is too late. ### Run regular restore tests[​](#run-regular-restore-tests "Direct link to Run regular restore tests") * Restore into a disposable environment at least monthly to confirm that credentials, network access, and storage quotas are still correct. * Track how long a restore takes and compare it to your recovery time objectives; tune the schedule or resource sizing if the gap widens. * Document each drill, including any manual steps you find, and fold the lessons back into your runbooks. ### Validate backups after major Infrahub upgrades[​](#validate-backups-after-major-infrahub-upgrades "Direct link to Validate backups after major Infrahub upgrades") * After every major Infrahub version upgrade, capture a fresh backup and run a full restore rehearsal. * Watch for schema migrations or new services introduced by the upgrade that need additional credentials, storage buckets, or automation hooks. * Keep the validation artifacts — logs, checksums, and timelines — so you can demonstrate post-upgrade recoverability during audits. ## Learn by doing[​](#learn-by-doing "Direct link to Learn by doing") New to the tool? The [Get started tutorial](/backup/tutorials/getting-started.md) covers a full backup and restore cycle on a Docker Compose deployment. --- # Back up your Infrahub instance Create a comprehensive backup of an Infrahub instance on Docker Compose. Follow these steps to protect your data before an upgrade or to establish a disaster-recovery routine. ## Community edition support[​](#community-edition-support "Direct link to Community edition support") Infrahub Community Edition deployments are fully supported by `infrahub-backup`, but there are a few operational differences to plan for compared to Enterprise environments. warning Backing up Community Edition stops the Infrahub application container while the snapshot is taken. Schedule the command during a maintenance window to avoid user-facing downtime. Restoring a backup that was taken from an Enterprise cluster to a Community Edition instance is not supported. Create and maintain backups from the same edition you intend to restore to. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before creating a backup: * Ensure infrahub-backup is installed and configured * Verify you have sufficient disk space (at least 2x your database size) * Have write permissions to the backup directory * Confirm your Infrahub instance is accessible ## Step 1: Check for running tasks[​](#step-1-check-for-running-tasks "Direct link to Step 1: Check for running tasks") Before initiating a backup, check if there are active tasks that might be interrupted by checking the Infrahub web interface or waiting for natural completion. If tasks are running, you have two options: * Wait for Completion * Force Backup Wait for tasks to complete naturally by monitoring the Infrahub UI, then proceed with backup: ``` infrahub-backup create ``` Force backup despite running tasks: ``` # Force backup (running tasks will be ignored) infrahub-backup create --force ``` ## Step 2: Create the backup[​](#step-2-create-the-backup "Direct link to Step 2: Create the backup") Execute the backup command with your desired options: ### Basic backup[​](#basic-backup "Direct link to Basic backup") For a standard backup with all components: ``` infrahub-backup create ``` ### Selective metadata backup[​](#selective-metadata-backup "Direct link to Selective metadata backup") Control which Neo4j metadata to include: ``` # Backup without metadata (smallest size) infrahub-backup create --neo4jmetadata=none # Backup with only user accounts infrahub-backup create --neo4jmetadata=users # Backup with only roles infrahub-backup create --neo4jmetadata=roles # Backup with everything (default) infrahub-backup create --neo4jmetadata=all ``` ### Custom backup location[​](#custom-backup-location "Direct link to Custom backup location") Specify an alternative backup directory: ``` infrahub-backup create --backup-dir=/mnt/backups/infrahub ``` ### Target specific project[​](#target-specific-project "Direct link to Target specific project") If running multiple Infrahub instances: ``` infrahub-backup create --project=infrahub-production ``` ## Step 3: Monitor backup progress[​](#step-3-monitor-backup-progress "Direct link to Step 3: Monitor backup progress") The backup process provides detailed progress information: ``` INFO[0000] Starting backup process... INFO[0000] Checking for running tasks... INFO[0001] No running tasks found INFO[0001] Creating backup ID: 20250929_143022 INFO[0002] Stopping Infrahub application containers... INFO[0005] Application containers stopped INFO[0005] Backing up Neo4j database... INFO[0015] Neo4j backup completed (1.2GB) INFO[0015] Backing up PostgreSQL database... INFO[0018] PostgreSQL backup completed (256MB) INFO[0020] Creating compressed archive... INFO[0025] Archive created: infrahub_backup_20250929_143022.tar.gz INFO[0025] Starting application containers... INFO[0030] All containers started successfully INFO[0030] Backup completed successfully ``` ## Step 4: Verify backup integrity[​](#step-4-verify-backup-integrity "Direct link to Step 4: Verify backup integrity") After backup completion, verify the backup file: ### Check backup file[​](#check-backup-file "Direct link to Check backup file") ``` # List backup files with sizes ls -lh infrahub_backups/ # Verify archive integrity tar -tzf infrahub_backups/infrahub_backup_20250929_143022.tar.gz > /dev/null && echo "Archive is valid" # View backup metadata tar -xzOf infrahub_backups/infrahub_backup_20250929_143022.tar.gz backup_information.json | jq '.' ``` ### Validate backup contents[​](#validate-backup-contents "Direct link to Validate backup contents") The backup should contain: * `backup_information.json` - Backup information and checksums * `database/` - Neo4j database files * `prefect.dump` - PostgreSQL dump Example metadata structure: ``` { "metadata_version": 2025092500, "backup_id": "20250929_143022", "created_at": "2025-09-29T14:30:22Z", "tool_version": "1.0.0", "infrahub_version": "0.15.0", "components": ["database", "task-manager-db"], "checksums": { "database": "sha256:abc123...", "prefect.dump": "sha256:def456..." }, "redacted": false } ``` ## Step 5: Backup artifact storage[​](#step-5-backup-artifact-storage "Direct link to Step 5: Backup artifact storage") Capture any artifact storage (object stores, shared volumes, artifact registries) that Infrahub references during task execution. Align the snapshot timing with the database backup so the two stay consistent. * Identify the storage location configured for your task manager (bucket, container, or filesystem path). * Copy or synchronize the artifacts into a dated folder or snapshot using the native tooling for your storage provider. * Record the snapshot path or version ID alongside your database backup metadata so operators can retrieve the artifacts during a restore. If your storage tier already offers automatic versioning or replication, confirm the retention window meets your recovery objectives and document how to retrieve the relevant versions during a disaster scenario. ## Step 6: Test restore capability[​](#step-6-test-restore-capability "Direct link to Step 6: Test restore capability") Periodically test restoration to ensure backups are valid: ``` # Review what would be restored infrahub-backup restore infrahub_backups/infrahub_backup_20250929_143022.tar.gz ``` ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Automated backups with cron[​](#automated-backups-with-cron "Direct link to Automated backups with cron") You can either use a [systemd timer](/backup/backup/install.md#system-service-installation) or regular cron. Create a cron job for regular backups: ``` # Edit crontab crontab -e # Add daily backup at 2 AM 0 2 * * * /usr/local/bin/infrahub-backup create >> /var/log/infrahub-backup.log 2>&1 # With retention management (keep 7 days) 0 2 * * * /usr/local/bin/infrahub-backup create --retention-days 7 >> /var/log/infrahub-backup.log 2>&1 ``` ### Bound the backup directory with a retention policy[​](#bound-the-backup-directory-with-a-retention-policy "Direct link to Bound the backup directory with a retention policy") Pass `--retention-days` or `--retention-count` to delete out-of-policy archives after a successful backup, so a schedule cannot fill its disk: ``` # Keep 7 days of archives, and never fewer than the 14 most recent infrahub-backup create --retention-days 7 --retention-count 14 ``` The rules combine as a union, the most recent archive at each location always survives, and the local directory and the S3 prefix are evaluated separately. Preview any policy with `infrahub-backup prune --dry-run` before relying on it. See [Manage backup retention](/backup/backup/retention.md) for the full semantics, the permissions S3 pruning requires, and the standalone `prune` command. ### Backup with notification[​](#backup-with-notification "Direct link to Backup with notification") Create a wrapper script for notifications: ``` #!/bin/bash BACKUP_RESULT=$(infrahub-backup create 2>&1) if [ $? -eq 0 ]; then echo "Backup successful" | mail -s "Infrahub Backup Success" ops@example.com else echo "$BACKUP_RESULT" | mail -s "Infrahub Backup Failed" ops@example.com exit 1 fi ``` ### S3 backup storage[​](#s3-backup-storage "Direct link to S3 backup storage") Upload backups directly to S3 or S3-compatible storage (like MinIO): * AWS S3 * MinIO * Environment Variables ``` # Upload to S3 (deletes local file by default) infrahub-backup create --s3-upload --s3-bucket my-backups --s3-prefix infrahub/prod # Upload to S3 but keep local copy infrahub-backup create --s3-upload --s3-bucket my-backups --s3-keep-local ``` ``` # Upload to MinIO or S3-compatible storage infrahub-backup create --s3-upload \ --s3-bucket my-backups \ --s3-endpoint http://minio.local:9000 \ --s3-region us-east-1 ``` ``` # Configure S3 via environment variables export INFRAHUB_S3_BUCKET=my-backups export INFRAHUB_S3_PREFIX=infrahub/prod export AWS_ACCESS_KEY_ID=your-access-key export AWS_SECRET_ACCESS_KEY=your-secret-key infrahub-backup create --s3-upload ``` info AWS credentials are loaded from the standard AWS credential chain: environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`), shared credentials file (`~/.aws/credentials`), or IAM roles when running on AWS infrastructure. ### Redacted backups[​](#redacted-backups "Direct link to Redacted backups") If you need to share a backup for debugging or testing purposes without exposing sensitive data, use the `--redact` flag. This replaces all attribute values in the Neo4j database with random UUIDs before the backup is created. danger The `--redact` flag is a **destructive operation** that irreversibly modifies the live database. All attribute values will be replaced with random UUIDs. Only use this on a disposable copy of your instance, never on a production database you intend to keep using. Because this operation is destructive, `--force` is required as confirmation: ``` infrahub-backup create --redact --force ``` The resulting backup archive is marked as redacted in its metadata (`"redacted": true`), so consumers can identify that the data has been anonymized. A typical workflow for creating a redacted backup safely: 1. Create a normal backup of your production instance 2. Restore that backup to a temporary or staging instance 3. Run the redacted backup on the temporary instance 4. Share the redacted backup archive 5. Tear down the temporary instance ### Manual remote backup storage[​](#manual-remote-backup-storage "Direct link to Manual remote backup storage") If you prefer manual transfers, you can copy backups after creation: ``` # Backup locally first infrahub-backup create # Transfer to remote storage LATEST_BACKUP=$(ls -t infrahub_backups/*.tar.gz | head -1) # S3 example (manual) aws s3 cp "$LATEST_BACKUP" s3://my-bucket/infrahub-backups/ # SCP example scp "$LATEST_BACKUP" backup-server:/backups/infrahub/ # Rsync example rsync -avz "$LATEST_BACKUP" backup-server:/backups/infrahub/ ``` ## Validation[​](#validation "Direct link to Validation") Confirm your backup strategy works: * ✓ Backup completes without errors * ✓ Archive passes integrity check * ✓ Metadata contains expected components * ✓ File size is reasonable for your data volume * ✓ Test restore succeeds in non-production environment ## Related resources[​](#related-resources "Direct link to Related resources") * [Restore from a backup](/backup/backup/restore.md) * [Manage backup retention](/backup/backup/retention.md) * [CLI command reference](/backup/reference/commands.md) --- # Install Infrahub Backup Install the `infrahub-backup` CLI on a Docker Compose or Kubernetes deployment, or build it from source. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before installing Infrahub Backup, ensure you have: **For Docker deployments:** * Administrative or sudo access on your system * Docker with Docker Compose installed **For Kubernetes deployments (in-band via Helm):** * Permissions to install or upgrade the `infrahub-backup` Helm chart in your cluster **For Kubernetes deployments (out-of-band CLI):** * Administrative or sudo access on your system * kubectl configured with permissions to access the Infrahub namespace **If building from source:** * Git and network access to clone the repository * Go 1.25 or later installed ## Installation methods[​](#installation-methods "Direct link to Installation methods") * Docker * Kubernetes Choose one of the following methods to install the CLI for Docker deployments: * Direct Download * Manual Selection * Build from Source ### Download the pre-built binary[​](#download-the-pre-built-binary "Direct link to Download the pre-built binary") Download the pre-built binary directly: ``` # Download the appropriate binary for your system curl https://infrahub.opsmill.io/ops/$(uname -s)/$(uname -m)/infrahub-backup -o infrahub-backup # Make it executable chmod +x infrahub-backup # Move to a directory in your PATH (optional) sudo mv infrahub-backup /usr/local/bin/ ``` ### Select and download a specific version[​](#select-and-download-a-specific-version "Direct link to Select and download a specific version") If you need a specific version or architecture, manually select the appropriate binary: 1. Identify your system architecture: ``` uname -s # Operating system (Linux/Darwin) uname -m # Architecture (x86_64/aarch64) ``` 2. Download the matching binary: * **Linux AMD64**: `curl -L https://infrahub.opsmill.io/ops/Linux/x86_64/infrahub-backup -o infrahub-backup` * **Linux ARM64**: `curl -L https://infrahub.opsmill.io/ops/Linux/aarch64/infrahub-backup -o infrahub-backup` * **macOS AMD64**: `curl -L https://infrahub.opsmill.io/ops/Darwin/x86_64/infrahub-backup -o infrahub-backup` * **macOS ARM64**: `curl -L https://infrahub.opsmill.io/ops/Darwin/arm64/infrahub-backup -o infrahub-backup` 3. Make executable and optionally install system-wide: ``` chmod +x infrahub-backup sudo mv infrahub-backup /usr/local/bin/ # Optional ``` ### Build from source code[​](#build-from-source-code "Direct link to Build from source code") To install Infrahub Backup, build from source: 1. Install Go 1.25 or later: ``` # Check if Go is installed go version ``` 2. Clone the repository: ``` git clone https://github.com/opsmill/infrahub-ops-cli.git cd infrahub-ops-cli ``` 3. Build the binaries: ``` make build ``` 4. Install to your PATH: ``` sudo cp bin/infrahub-backup /usr/local/bin/ ``` * Helm (In-band) * CLI (Out-of-band) ### Enable via Infrahub Helm chart (Recommended)[​](#enable-via-infrahub-helm-chart-recommended "Direct link to Enable via Infrahub Helm chart (Recommended)") If you deploy Infrahub on Kubernetes using the Infrahub Helm chart, enable the `infrahub-backup` subchart. This in-band approach deploys backup functionality directly within your cluster—no separate CLI installation required. ``` # values.yaml for Infrahub Helm chart infrahub-backup: enabled: true backup: enabled: true mode: "cronjob" schedule: "0 2 * * *" storage: type: "s3" s3: bucket: "my-infrahub-backups" endpoint: "https://s3.amazonaws.com" region: "us-east-1" secretName: "backup-s3-credentials" ``` Apply the configuration: ``` helm upgrade infrahub oci://registry.opsmill.io/opsmill/chart/infrahub \ --namespace infrahub \ --values values.yaml ``` This approach integrates backups with your existing Infrahub deployment and is ideal for GitOps workflows with ArgoCD or Flux. For detailed configuration options, see: * [Back up Infrahub on Kubernetes](/backup/backup/kubernetes-backup.md) ### Install the CLI for out-of-band operations[​](#install-the-cli-for-out-of-band-operations "Direct link to Install the CLI for out-of-band operations") If you need to run backup operations from outside the cluster (for example, from a CI/CD pipeline or local workstation), install the CLI binary: ``` # Download the appropriate binary for your system curl https://infrahub.opsmill.io/ops/$(uname -s)/$(uname -m)/infrahub-backup -o infrahub-backup # Make it executable chmod +x infrahub-backup # Move to a directory in your PATH (optional) sudo mv infrahub-backup /usr/local/bin/ ``` Ensure kubectl is configured with access to the Infrahub namespace before running commands. ## Verify installation[​](#verify-installation "Direct link to Verify installation") note If you installed via Helm (in-band), skip this section—verification is handled by Kubernetes. The steps below apply to CLI installations only. After installing the CLI, verify it is working correctly: ``` # Check versions infrahub-backup version # Verify environment detection infrahub-backup environment detect # Display help infrahub-backup --help ``` Expected output: ``` Version: [git ref] ``` ## Configure environment[​](#configure-environment "Direct link to Configure environment") note If you installed via Helm (in-band), configuration is managed through Helm values. The environment variables below apply to CLI installations only. Infrahub Backup CLI uses environment variables for configuration. Set these based on your deployment: ### Docker Compose deployments[​](#docker-compose-deployments "Direct link to Docker Compose deployments") ``` # Optional: Target a specific project export INFRAHUB_COMPOSE_PROJECT=infrahub-production # Optional: Custom backup location export BACKUP_DIR=/data/backups/infrahub ``` ### Kubernetes deployments[​](#kubernetes-deployments "Direct link to Kubernetes deployments") ``` # Set the namespace export INFRAHUB_K8S_NAMESPACE=infrahub # Set kubeconfig if not default export KUBECONFIG=/path/to/kubeconfig ``` ### Database credentials[​](#database-credentials "Direct link to Database credentials") If your deployment uses non-default credentials and the tools cannot fetch them automatically: ``` # Neo4j settings export INFRAHUB_DB_USERNAME=neo4j export INFRAHUB_DB_PASSWORD=your-password export INFRAHUB_DB_DATABASE=neo4j # PostgreSQL settings (for task manager) export PREFECT_API_DATABASE_CONNECTION_URL=postgresql://user:pass@localhost/prefect ``` ## Validation[​](#validation "Direct link to Validation") note If you installed via Helm (in-band), validate your deployment by checking the CronJob status with `kubectl get cronjobs -n infrahub`. The steps below apply to CLI installations only. Confirm the CLI can connect to your Infrahub instance: ``` # List available projects (Docker) infrahub-backup environment list # Test environment detection infrahub-backup environment detect ``` If these commands succeed, installation is complete. ## Advanced usage (Docker deployments)[​](#advanced-usage-docker-deployments "Direct link to Advanced usage (Docker deployments)") ### System service installation[​](#system-service-installation "Direct link to System service installation") For Docker deployments, you can run scheduled backups using a systemd service: 1. Create service file: ``` sudo tee /etc/systemd/system/infrahub-backup.service > /dev/null < /dev/null < /dev/null && echo "Archive is valid" # View metadata tar -xzOf infrahub_backup_20250120_020000.tar.gz backup_information.json | jq '.' ``` ## Understanding permissions[​](#understanding-permissions "Direct link to Understanding permissions") The Helm chart creates a ServiceAccount with the required RBAC permissions to perform backups. ### Default behavior[​](#default-behavior "Direct link to Default behavior") By default, the chart creates: * A ServiceAccount named `infrahub-backup` * A Role with required permissions * A RoleBinding linking the ServiceAccount to the Role ### Required permissions[​](#required-permissions "Direct link to Required permissions") The backup process needs these Kubernetes permissions: | Resource | Verbs | Purpose | | -------------- | ---------- | ------------------------------- | | `pods` | list, get | Discover Infrahub pods | | `pods/exec` | create | Execute backup commands in pods | | `deployments` | get, patch | Scale down/up during backup | | `statefulsets` | get, patch | Scale down/up during backup | | `pods/log` | get | Monitor backup progress | ### Network access to the API server[​](#network-access-to-the-api-server "Direct link to Network access to the API server") RBAC grants *authorization*, but the backup Job also needs *network* connectivity to the Kubernetes API server. The tool discovers pods, scales services, and runs commands inside pods through the API server's `exec`, `cp`, and `log` subresources — these operations stream through the API server, not directly between pods. This applies to both the in-band Helm Job and any out-of-band `kubectl` run. If your cluster enforces default-deny egress with [NetworkPolicies](https://kubernetes.io/docs/concepts/services-networking/network-policies/), add an egress rule that lets the backup pod reach the API server, typically the `kubernetes.default.svc` ClusterIP on port 443 (or your control-plane endpoint): ``` apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: infrahub-backup-apiserver-egress namespace: infrahub spec: podSelector: matchLabels: app.kubernetes.io/name: infrahub-backup policyTypes: - Egress egress: # Allow the backup pod to reach the Kubernetes API server. # Adjust the port to match your cluster (443 for the in-cluster # service, 6443 for many control-plane endpoints). - ports: - protocol: TCP port: 443 ``` warning A default-deny egress NetworkPolicy is a common cause of backups that appear to start, then stall at the pod discovery, scale-down, or database steps. If RBAC is correct but `exec`/`cp` operations hang or time out, confirm the backup pod can reach the API server first. ### Using an existing ServiceAccount[​](#using-an-existing-serviceaccount "Direct link to Using an existing ServiceAccount") If your security policy requires using a pre-existing ServiceAccount: ``` infrahub-backup: serviceAccount: create: false name: "my-existing-serviceaccount" rbac: create: false # Assumes permissions already exist ``` ### Cloud provider integration[​](#cloud-provider-integration "Direct link to Cloud provider integration") Add annotations for cloud provider IAM integration: ``` infrahub-backup: serviceAccount: annotations: # AWS IRSA eks.amazonaws.com/role-arn: "arn:aws:iam::123456789012:role/infrahub-backup" # GCP Workload Identity iam.gke.io/gcp-service-account: "infrahub-backup@project.iam.gserviceaccount.com" ``` ## Validation[​](#validation "Direct link to Validation") Confirm your backup configuration works: * Backup Job/CronJob is created in the correct namespace * ServiceAccount has required RBAC permissions * S3 credentials secret exists and is correctly referenced * Backup completes without errors in logs * Backup file appears in S3 bucket (or can be retrieved from pod for local storage) * Backup archive passes integrity check * Backup metadata contains expected components ## Related resources[​](#related-resources "Direct link to Related resources") * [Restore Infrahub on Kubernetes](/backup/backup/kubernetes-restore.md) * [Refresh an environment on a schedule](/backup/backup/kubernetes-scheduled-restore.md) * [Back up your Infrahub instance (CLI)](/backup/backup/create.md) --- # Restore Infrahub on Kubernetes Restore an Infrahub instance on Kubernetes from a backup stored in S3-compatible storage. The restore runs as a Kubernetes Job through the same `infrahub-backup` Helm chart. To refresh a staging or demo deployment from another environment's backups on a recurring schedule instead, see [Refresh an environment on a schedule](/backup/backup/kubernetes-scheduled-restore.md). warning Restoring a backup will overwrite your current Infrahub data. Create a safety backup before proceeding if you need to preserve any recent changes. ## High availability (CloudNativePG)[​](#high-availability-cloudnativepg "Direct link to High availability (CloudNativePG)") If your deployment uses a CloudNativePG HA PostgreSQL cluster, the restore process automatically detects and targets the primary pod. No special configuration is needed. See the [Kubernetes backup guide](/backup/backup/kubernetes-backup.md#high-availability-cloudnativepg) for details on supported HA operators. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before restoring: * A backup file stored in S3-compatible storage * The backup was created from the same Infrahub edition (Community or Enterprise) * Access to modify Helm values for your Infrahub deployment * S3 credentials with read access to the backup bucket * Network egress from the restore pod to the Kubernetes API server (see [Network access to the API server](/backup/backup/kubernetes-backup.md#network-access-to-the-api-server)) ## Step 1: Identify the backup to restore[​](#step-1-identify-the-backup-to-restore "Direct link to Step 1: Identify the backup to restore") List available backups in your S3 bucket: ``` # AWS S3 aws s3 ls s3://my-infrahub-backups/ # MinIO mc ls myminio/my-infrahub-backups/ ``` Note the exact filename of the backup you want to restore, for example: `infrahub_backup_20250120_020000.tar.gz` ### Verify backup metadata[​](#verify-backup-metadata "Direct link to Verify backup metadata") Before restoring, verify the backup is compatible: ``` # Download and inspect metadata aws s3 cp s3://my-infrahub-backups/infrahub_backup_20250120_020000.tar.gz ./ tar -xzOf infrahub_backup_20250120_020000.tar.gz backup_information.json | jq '.' ``` Check that: * `neo4j_edition` matches your current deployment * `infrahub_version` is compatible with your target version * `components` includes the data you need to restore ## Step 2: Configure S3 source[​](#step-2-configure-s3-source "Direct link to Step 2: Configure S3 source") Create or verify the S3 credentials secret exists: ``` kubectl create secret generic backup-s3-credentials \ --namespace infrahub \ --from-literal=AWS_ACCESS_KEY_ID=your-access-key \ --from-literal=AWS_SECRET_ACCESS_KEY=your-secret-key ``` ## Step 3: Configure the restore Job[​](#step-3-configure-the-restore-job "Direct link to Step 3: Configure the restore Job") * Via Infrahub Helm Chart (Recommended) * Standalone Chart Add the restore configuration to your Infrahub Helm values: ``` # values.yaml for Infrahub Helm chart infrahub-backup: enabled: true # Disable backup during restore backup: enabled: false # Enable restore restore: enabled: true mode: "job" storage: type: "s3" s3: bucket: "my-infrahub-backups" key: "infrahub_backup_20250120_020000.tar.gz" endpoint: "https://s3.amazonaws.com" region: "us-east-1" secretName: "backup-s3-credentials" ``` Create a values file for the restore operation: ``` # restore-values.yaml backup: enabled: false restore: enabled: true mode: "job" storage: type: "s3" s3: bucket: "my-infrahub-backups" key: "infrahub_backup_20250120_020000.tar.gz" endpoint: "https://s3.amazonaws.com" region: "us-east-1" secretName: "backup-s3-credentials" ``` ## Step 4: Deploy the restore Job[​](#step-4-deploy-the-restore-job "Direct link to Step 4: Deploy the restore Job") warning The restore process will stop Infrahub services temporarily. Plan for downtime during the restore operation. * Via Infrahub Helm Chart * Standalone Chart Update your Infrahub Helm release: ``` helm upgrade infrahub oci://registry.opsmill.io/opsmill/chart/infrahub \ --namespace infrahub \ --values values.yaml ``` Install or upgrade the chart with restore enabled: ``` helm upgrade --install infrahub-backup oci://registry.opsmill.io/opsmill/chart/infrahub-backup \ --namespace infrahub \ --values restore-values.yaml ``` ## Step 5: Monitor restore progress[​](#step-5-monitor-restore-progress "Direct link to Step 5: Monitor restore progress") ### Watch the restore Job[​](#watch-the-restore-job "Direct link to Watch the restore Job") ``` # Check Job status kubectl get job -n infrahub -l app.kubernetes.io/name=infrahub-backup # Watch pod status kubectl get pods -n infrahub -l app.kubernetes.io/name=infrahub-backup -w ``` ### View restore logs[​](#view-restore-logs "Direct link to View restore logs") ``` # Stream logs from the restore pod kubectl logs -n infrahub -l app.kubernetes.io/name=infrahub-backup -f ``` Expected output for a successful restore: ``` INFO[0000] Starting restore process... INFO[0001] Downloading backup from S3: s3://my-infrahub-backups/infrahub_backup_20250120_020000.tar.gz INFO[0010] Download completed (1.5GB) INFO[0010] Extracting backup archive... INFO[0012] Validating backup metadata... INFO[0012] Backup ID: 20250120_020000 INFO[0012] Infrahub version: 0.15.0 INFO[0012] Components: database, task-manager-db INFO[0013] Validating checksums... INFO[0015] All checksums valid INFO[0015] Stopping Infrahub services... INFO[0020] Wiping transient data (cache, message-queue)... INFO[0022] Restoring PostgreSQL database... INFO[0030] PostgreSQL restore completed INFO[0030] Restarting support services... INFO[0035] Restoring Neo4j database... INFO[0060] Neo4j restore completed INFO[0060] Starting Infrahub services... INFO[0070] All services started INFO[0070] Restore completed successfully ``` ## Step 6: Verify restored instance[​](#step-6-verify-restored-instance "Direct link to Step 6: Verify restored instance") ### Check service health[​](#check-service-health "Direct link to Check service health") ``` # Verify all pods are running kubectl get pods -n infrahub # Check Infrahub server logs kubectl logs -n infrahub -l app.kubernetes.io/component=infrahub-server --tail=50 ``` ### Validate data integrity[​](#validate-data-integrity "Direct link to Validate data integrity") 1. **Access the Infrahub UI** - Log in and verify your data is present 2. **Check the GraphQL API** - Query a known object to confirm data restoration 3. **Review task manager** - Verify historical task runs are visible ### Test a sample query[​](#test-a-sample-query "Direct link to Test a sample query") ``` # Port-forward to the Infrahub server kubectl port-forward -n infrahub svc/infrahub-server 8000:8000 # Query the API curl -X POST http://localhost:8000/graphql \ -H "Content-Type: application/json" \ -d '{"query": "{ InfrahubStatus { summary { schema_hash } } }"}' ``` ## Step 7: Disable restore and re-enable backups[​](#step-7-disable-restore-and-re-enable-backups "Direct link to Step 7: Disable restore and re-enable backups") After a successful restore, update your values to disable the restore Job and re-enable scheduled backups: ``` infrahub-backup: enabled: true backup: enabled: true mode: "cronjob" schedule: "0 2 * * *" storage: type: "s3" s3: bucket: "my-infrahub-backups" endpoint: "https://s3.amazonaws.com" region: "us-east-1" secretName: "backup-s3-credentials" restore: enabled: false # Disable restore ``` Apply the updated configuration: ``` helm upgrade infrahub oci://registry.opsmill.io/opsmill/chart/infrahub \ --namespace infrahub \ --values values.yaml ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Restore Job fails to start[​](#restore-job-fails-to-start "Direct link to Restore Job fails to start") Check if the S3 credentials secret exists: ``` kubectl get secret backup-s3-credentials -n infrahub ``` Verify the secret contains the expected keys: ``` kubectl get secret backup-s3-credentials -n infrahub -o jsonpath='{.data}' | jq 'keys' ``` ### Download fails[​](#download-fails "Direct link to Download fails") Check network connectivity and S3 endpoint configuration: ``` # View detailed error in logs kubectl logs -n infrahub -l app.kubernetes.io/name=infrahub-backup # Common issues: # - Incorrect bucket name # - Wrong S3 endpoint URL # - Invalid credentials # - Network policy blocking egress ``` ### Restore hangs or times out[​](#restore-hangs-or-times-out "Direct link to Restore hangs or times out") If the restore pod starts but stalls at the service-stop, database, or service-start steps, and the logs show no error, the pod may be unable to reach the Kubernetes API server. The restore relies on the API server's `exec` and `cp` subresources, which are blocked by default-deny egress NetworkPolicies even when RBAC is correct. ``` # Check whether NetworkPolicies restrict egress in the namespace kubectl get networkpolicy -n infrahub ``` If a default-deny egress policy is present, add an egress rule allowing the pod to reach the API server. See [Network access to the API server](/backup/backup/kubernetes-backup.md#network-access-to-the-api-server). ### Checksum validation fails[​](#checksum-validation-fails "Direct link to Checksum validation fails") The backup file may be corrupted. Try downloading a fresh copy: ``` # Verify the backup locally aws s3 cp s3://my-infrahub-backups/infrahub_backup_20250120_020000.tar.gz ./ tar -tzf infrahub_backup_20250120_020000.tar.gz > /dev/null ``` ### Services fail to restart[​](#services-fail-to-restart "Direct link to Services fail to restart") Check for resource constraints or scheduling issues: ``` # Check pod events kubectl describe pods -n infrahub -l app.kubernetes.io/name=infrahub # Check node resources kubectl top nodes ``` ## Validation[​](#validation "Direct link to Validation") Confirm your restore completed successfully: * Restore Job completed without errors * All Infrahub pods are running * Infrahub UI is accessible * Data is present and correct * Task manager shows historical runs * Scheduled backups are re-enabled * Restore Job is disabled to prevent accidental re-runs ## Related resources[​](#related-resources "Direct link to Related resources") * [Back up Infrahub on Kubernetes](/backup/backup/kubernetes-backup.md) * [Refresh an environment on a schedule](/backup/backup/kubernetes-scheduled-restore.md) * [Restore from a backup (CLI)](/backup/backup/restore.md) --- # Refresh an environment on a schedule Run the restore CronJob in the `infrahub-backup` Helm chart to keep a staging or demo deployment on the newest backup another environment produces. Each run restores whichever archive is newest under the source bucket and prefix, so the target release never names a filename and the schedule keeps working as new backups land. Follow this guide to point a target release at a source bucket, enable the restore CronJob, and trigger a run without waiting for the next scheduled time. danger A scheduled restore replaces all data in the deployment it targets, on every run. Enable it only on a deployment you are willing to overwrite repeatedly, and keep the release that produces the backups out of reach of the schedule. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before enabling a scheduled restore: * The target release uses `infrahub-backup` chart 1.3.0 or later with image tag `2.3.0` or later. Newest-archive selection was added in `2.3.0`. * The target is a separate release from the one producing the backups, in its own namespace. Each run restores into the namespace the pod runs in. * The source environment already holds at least one backup under the bucket and prefix. A run that finds no archive exits non-zero. * S3 credentials with `s3:ListBucket` and `s3:GetObject` on the bucket and prefix. List access is required because each run chooses its archive by listing the prefix. * Network egress from the restore pod to the Kubernetes API server. See [Network access to the API server](/backup/backup/kubernetes-backup.md#network-access-to-the-api-server). * Enough ephemeral storage on the node for one archive. The pod mounts an `emptyDir` at `restore.storage.path`, and each run downloads the selected object there before extracting it. * The backups were created from the same Infrahub edition, Community or Enterprise, as the target. ## How the schedule chooses an archive[​](#how-the-schedule-chooses-an-archive "Direct link to How the schedule chooses an archive") Setting `restore.storage.s3.latest: true` renders `restore --latest --s3`, and the chart passes the bucket, prefix, endpoint, and region to the pod as `INFRAHUB_S3_*` variables. Every run repeats the same selection against the live contents of the prefix: * Only objects named `infrahub_backup_.tar.gz`, optionally with an `.enc` suffix, take part. Anything else under the prefix — unrelated files, partial uploads, foreign names — is ignored. * Archives are ordered by the timestamp embedded in the name, newest first, with ties broken by name in descending order. Each run reports its selection before it touches the deployment: ``` INFO[0000] Restoring latest backup infrahub_backup_20250120_020000.tar.gz from s3://my-infrahub-backups/infrahub/prod ``` That line records which source backup the target is currently running, and it is readable from the job logs alone. The selection never falls back to an older archive. A run whose prefix holds no matching archive, or whose newest archive is encrypted with no decryption key available, exits non-zero without stopping a single container. See [Restore the most recent backup](/backup/backup/restore.md#restore-the-most-recent-backup) for the full selection behavior. info The chart has no values key for a decryption key and no way to mount one into the restore pod, so a scheduled restore reads only archives the source uploaded without encryption. If the source encrypts its backups, every run exits non-zero naming the archive it selected. ## Step 1: Point the release at the source bucket[​](#step-1-point-the-release-at-the-source-bucket "Direct link to Step 1: Point the release at the source bucket") Create the credentials secret in the target namespace, holding a key pair with read access to the source bucket: ``` kubectl create secret generic backup-s3-credentials \ --namespace infrahub-staging \ --from-literal=AWS_ACCESS_KEY_ID=your-access-key \ --from-literal=AWS_SECRET_ACCESS_KEY=your-secret-key ``` Note the bucket and prefix the source release uploads to — its `backup.storage.s3.bucket` and `backup.storage.s3.prefix`. The target's `restore.storage.s3.prefix` has to match, or each run lists an empty prefix and exits. List the prefix to confirm the archives are where you expect: ``` aws s3 ls s3://my-infrahub-backups/infrahub/prod/ ``` ## Step 2: Enable the restore CronJob[​](#step-2-enable-the-restore-cronjob "Direct link to Step 2: Enable the restore CronJob") * Via Infrahub Helm Chart (Recommended) * Standalone Chart Add the scheduled restore to the target deployment's Helm values: ``` # values.yaml for the target Infrahub Helm chart infrahub-backup: enabled: true # The target release must not also produce backups backup: enabled: false restore: enabled: true mode: "cronjob" schedule: "0 4 * * *" storage: type: "s3" s3: bucket: "my-infrahub-backups" prefix: "infrahub/prod" latest: true endpoint: "https://s3.amazonaws.com" region: "us-east-1" secretName: "backup-s3-credentials" ``` Create a values file for the target namespace: ``` # scheduled-restore-values.yaml backup: enabled: false restore: enabled: true mode: "cronjob" schedule: "0 4 * * *" storage: type: "s3" s3: bucket: "my-infrahub-backups" prefix: "infrahub/prod" latest: true endpoint: "https://s3.amazonaws.com" region: "us-east-1" secretName: "backup-s3-credentials" ``` The values that distinguish a scheduled restore from the one-shot restore Job: | Value | Purpose | | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `restore.mode` | `"cronjob"` renders a CronJob. `"job"`, the default, renders the one-shot [restore Job](/backup/backup/kubernetes-restore.md). | | `restore.schedule` | Cron expression for the refresh. Defaults to `0 4 * * *`, two hours after the backup chart's default `0 2 * * *`, so a nightly backup lands before the refresh reads the prefix. | | `restore.storage.s3.latest` | Selects the newest archive under the bucket and prefix on every run, instead of naming one. | | `restore.storage.s3.prefix` | The prefix the archives live under. Match the source release's `backup.storage.s3.prefix`. | Pick a schedule that leaves room for the source backup to finish uploading. A refresh that fires while the source is still uploading selects the previous archive instead — a valid restore, one cycle behind the data you expected. ## Step 3: Deploy and verify the schedule[​](#step-3-deploy-and-verify-the-schedule "Direct link to Step 3: Deploy and verify the schedule") With `restore.mode: "cronjob"` the chart creates an ordinary CronJob rather than a Helm hook, so `helm upgrade` returns as soon as the object is applied and does not wait for a restore. The one-shot restore Job behaves differently: it runs as a `post-install,post-upgrade` hook and holds the release open until it finishes. * Via Infrahub Helm Chart * Standalone Chart ``` helm upgrade infrahub oci://registry.opsmill.io/opsmill/chart/infrahub \ --namespace infrahub-staging \ --values values.yaml ``` ``` helm upgrade --install infrahub-backup oci://registry.opsmill.io/opsmill/chart/infrahub-backup \ --namespace infrahub-staging \ --values scheduled-restore-values.yaml ``` Confirm the CronJob exists and is not suspended: ``` kubectl get cronjobs -n infrahub-staging ``` ``` NAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGE infrahub-backup-restore 0 4 * * * False 0 30s ``` Nothing restores until the first scheduled time. ## Step 4: Trigger a run without waiting[​](#step-4-trigger-a-run-without-waiting "Direct link to Step 4: Trigger a run without waiting") Create a one-off Job from the CronJob to exercise the configuration before you rely on the schedule: ``` kubectl create job --from=cronjob/infrahub-backup-restore restore-check \ --namespace infrahub-staging ``` Read the CronJob name from `kubectl get cronjobs` rather than copying it. It is the release's full name with a `-restore` suffix, which differs between a standalone install and a subchart install. Follow the logs: ``` kubectl logs -n infrahub-staging -l app.kubernetes.io/name=infrahub-backup -f ``` Check the selection line names the archive you expect, then confirm the refreshed data: ``` # All Infrahub pods came back up kubectl get pods -n infrahub-staging # The restored data is present kubectl port-forward -n infrahub-staging svc/infrahub-server 8000:8000 curl -X POST http://localhost:8000/graphql \ -H "Content-Type: application/json" \ -d '{"query": "{ InfrahubStatus { summary { schema_hash } } }"}' ``` Remove the one-off Job so it does not sit in the namespace alongside the scheduled runs: ``` kubectl delete job restore-check -n infrahub-staging ``` ## Configuration guardrails[​](#configuration-guardrails "Direct link to Configuration guardrails") Four combinations fail while the chart renders, so `helm template` and `helm upgrade` both stop before anything reaches the cluster: | Configuration | Why it is rejected | | ---------------------------------------------------------------- | ------------------------------------------------------------------------- | | `restore.mode: "cronjob"` with `restore.storage.type: "local"` | The pod mounts an `emptyDir`, which is empty on every scheduled run. | | `restore.mode: "cronjob"` with `backup.enabled: true` | A release that produces backups must never be a scheduled-restore target. | | Both `restore.storage.s3.key` and `restore.storage.s3.latest` | Name one exact archive or take the newest, not both. | | Neither `restore.storage.s3.key` nor `restore.storage.s3.latest` | Without either, the restore has no archive to read. | Each rejection names the reason: ``` Error: execution error at (infrahub-backup/templates/job-restore.yaml:1:4): restore.mode=cronjob requires restore.storage.type=s3: local storage mounts an emptyDir, which is empty on every scheduled run ``` Both restore templates run the same check, so the path in the error reads `job-restore.yaml` even when the values set `restore.mode: "cronjob"`. Read the message, not the filename. The `backup.enabled` guardrail is scoped to one release: it stops a single release from both producing backups and restoring on a schedule. It cannot tell whether the namespace the CronJob lands in is the one you meant, because each run restores into its own namespace. Confirm the target namespace on every deploy. ## Failure posture[​](#failure-posture "Direct link to Failure posture") A scheduled restore is built for targets where the next run is an acceptable repair: * **A failed run is not retried.** `job.backoffLimit` is `0` and the chart performs no rollback. A run that fails after the Infrahub services were stopped leaves the target down until the next scheduled run completes. That suits a staging or demo environment; it is not a posture for production. * **Overlapping runs are skipped.** `cronJob.concurrencyPolicy` defaults to `Forbid`, so a run still going when the next one fires does not start. * **The target is unavailable during each run.** Every restore stops the Infrahub services, restores both databases, and starts the services again. * **Recent runs stay available for inspection.** `cronJob.successfulJobsHistoryLimit` and `cronJob.failedJobsHistoryLimit` both default to `3`, so the last three runs of each kind keep their pods and logs. Pause the schedule without removing it — during a maintenance window on the source, for example: ``` cronJob: suspend: true ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### The deploy fails with a restore validation error[​](#the-deploy-fails-with-a-restore-validation-error "Direct link to The deploy fails with a restore validation error") The values hit a render-time guardrail. Match the message against [Configuration guardrails](#configuration-guardrails); the error text names which combination was rejected. Nothing was applied to the cluster, so correcting the values and running `helm upgrade` again is enough. ### The run exits reporting no backups[​](#the-run-exits-reporting-no-backups "Direct link to The run exits reporting no backups") The prefix the target lists holds no matching archive. Compare the two values directly: ``` # What the target lists helm get values -n infrahub-staging # What the source wrote aws s3 ls s3://my-infrahub-backups/infrahub/prod/ ``` A trailing-slash or path mismatch between the target's `restore.storage.s3.prefix` and the source's `backup.storage.s3.prefix` is the common cause. This is also the expected state before the source takes its first backup. ### Access denied while listing the bucket[​](#access-denied-while-listing-the-bucket "Direct link to Access denied while listing the bucket") Selection lists the prefix before it downloads anything, so a credential with `s3:GetObject` but no `s3:ListBucket` fails at selection rather than at download. Grant both rights on the bucket and prefix. ### The CronJob exists but never runs[​](#the-cronjob-exists-but-never-runs "Direct link to The CronJob exists but never runs") Check whether the schedule is suspended: ``` kubectl get cronjob infrahub-backup-restore -n infrahub-staging \ -o jsonpath='{.spec.suspend}{"\n"}' ``` With `cronJob.suspend` set to `true`, nothing is scheduled. If `cronJob.startingDeadlineSeconds` is set and the controller misses that window — during node pressure or a cluster upgrade — the run is skipped rather than delayed. ### The run stops partway with no error[​](#the-run-stops-partway-with-no-error "Direct link to The run stops partway with no error") If the pod starts but stalls at the service-stop, database, or service-start steps and the logs show no error, the pod may be unable to reach the Kubernetes API server. Restore relies on the API server's `exec` and `cp` subresources, which default-deny egress NetworkPolicies block even when RBAC is correct. See [Network access to the API server](/backup/backup/kubernetes-backup.md#network-access-to-the-api-server). Because a failed run is not retried, the target stays in whatever state the run left it until the next scheduled time. Fix the egress rule, then trigger a run with `kubectl create job --from=cronjob/...` rather than waiting. ## Validation[​](#validation "Direct link to Validation") Confirm the schedule is working end to end: * The CronJob exists in the target namespace and is not suspended * A manually triggered run completed without errors * The job log names the archive and source location it restored from * All Infrahub pods in the target namespace are running * The restored data matches the source backup * [ ] `backup.enabled` is `false` on the target release ## Related resources[​](#related-resources "Direct link to Related resources") * [Restore Infrahub on Kubernetes](/backup/backup/kubernetes-restore.md) — one-shot restore from a named archive * [Back up Infrahub on Kubernetes](/backup/backup/kubernetes-backup.md) — configuring the source release that produces the backups * [Restore the most recent backup](/backup/backup/restore.md#restore-the-most-recent-backup) — how `--latest` selects an archive and when it refuses to run * [Manage backup retention](/backup/backup/retention.md) — bounding what accumulates under the source prefix --- # Restore from a backup Restore an Infrahub instance from a backup file on Docker Compose. Follow these steps to recover from data loss, roll back a change, or move to new infrastructure. ## Community edition compatibility[​](#community-edition-compatibility "Direct link to Community edition compatibility") Restores are supported for Infrahub Community Edition when the backup was created from the same edition. Enterprise backups include components that are not available in Community Edition and cannot be restored. danger Restoring an Enterprise backup to a Community Edition deployment is not supported. Always use backups that were captured from the same edition you plan to restore. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before restoring from backup: * Have a valid backup file created by infrahub-backup * Ensure sufficient disk space for extraction (3x backup size) * Stop all write operations to the current instance * Verify you have necessary permissions * **Create a current backup** before overwriting existing data danger Restoration replaces all existing data. Always verify you have a current backup before proceeding with a restore operation. ## Docker high availability deployments[​](#docker-high-availability-deployments "Direct link to Docker high availability deployments") If your Docker Compose deployment runs multiple task manager replicas (HA setup), you must stop the `task-manager` and `task-manager-background-svc` containers before running a restore. This prevents those services from accessing the database while data is being replaced. ``` # Stop task manager services before restore docker compose stop task-manager task-manager-background-svc ``` After the restore completes, restart the stopped services: ``` # Restart task manager services after restore docker compose start task-manager task-manager-background-svc ``` ## Step 1: Prepare for restoration[​](#step-1-prepare-for-restoration "Direct link to Step 1: Prepare for restoration") ### Create safety backup[​](#create-safety-backup "Direct link to Create safety backup") Before overwriting current data, create a safety backup: ``` # Backup current state infrahub-backup create --backup-dir=/tmp/safety-backup # Note the filename for potential recovery SAFETY_BACKUP=$(ls -t /tmp/safety-backup/*.tar.gz | head -1) echo "Safety backup: $SAFETY_BACKUP" ``` ## Step 2: Execute the restore[​](#step-2-execute-the-restore "Direct link to Step 2: Execute the restore") ### Basic restore[​](#basic-restore "Direct link to Basic restore") Restore from a backup file: ``` infrahub-backup restore infrahub_backups/infrahub_backup_20250929_143022.tar.gz ``` The restore process will: 1. Validate backup compatibility 2. Stop all Infrahub containers 3. Clear existing data 4. Restore Neo4j database 5. Restore PostgreSQL task manager 6. Restart all services ### Restore from S3[​](#restore-from-s3 "Direct link to Restore from S3") Restore directly from S3 or S3-compatible storage using an `s3://` URI: * AWS S3 * MinIO * Environment Variables ``` # Restore from S3 infrahub-backup restore s3://my-backups/infrahub/prod/infrahub_backup_20250929_143022.tar.gz ``` ``` # Restore from MinIO or S3-compatible storage infrahub-backup restore \ --s3-endpoint http://minio.local:9000 \ s3://my-backups/infrahub_backup_20250929_143022.tar.gz ``` ``` # Configure S3 via environment variables export AWS_ACCESS_KEY_ID=your-access-key export AWS_SECRET_ACCESS_KEY=your-secret-key infrahub-backup restore s3://my-backups/infrahub/prod/infrahub_backup_20250929_143022.tar.gz ``` info The object is downloaded to a temporary file inside the local backup directory before restoration, and removed again afterwards. Because the download never uses the archive's own name, a local archive of the same name — what `create --s3-upload --s3-keep-local` leaves behind — is never overwritten and never deleted. ### Restore the most recent backup[​](#restore-the-most-recent-backup "Direct link to Restore the most recent backup") If you do not want to name an archive — for example from a scheduled job, which cannot know a filename in advance — pass `--latest` instead of a file: ``` infrahub-backup restore --latest ``` `--latest` lists the configured backup directory, ranks the archives it finds, and restores the newest one. To rank archives in the configured S3 bucket and prefix instead, add `--s3`: ``` infrahub-backup restore --latest --s3 --s3-bucket my-backups --s3-prefix infrahub/prod ``` Exactly one location is consulted per run. With `--s3` the local backup directory is never read; without it the bucket is never read. The two are never merged, and there is no fallback from one to the other. Everything else about the restore is unchanged: `--latest` only decides which archive is used. Metadata validation, checksum verification, and the container stop and start sequence all behave exactly as they do when you name an archive, and every other `restore` flag composes with `--latest`. #### How the newest archive is chosen[​](#how-the-newest-archive-is-chosen "Direct link to How the newest archive is chosen") `--latest` ranks archives exactly as the retention policy ranks them: * Only names matching `infrahub_backup_.tar.gz`, optionally with an `.enc` suffix, take part. Anything else in the directory or under the prefix — unrelated files, partial uploads, foreign names — is ignored. * Archives are ordered by the timestamp embedded in the name, newest first, with ties broken by name in descending order. Because the ordering is shared with retention, the archive `--latest` restores is always one that retention's "the most recent backup at each location always survives" rule protects. See [Manage backup retention](/backup/backup/retention.md). Before the restore begins, the run reports what it selected and where it came from: ``` INFO[0000] Restoring latest backup infrahub_backup_20250929_143022.tar.gz from local:/opt/infrahub_backups ``` The location reads `local:` for the backup directory and `s3:///` for a bucket. That single line is what makes an unattended restore auditable from captured output alone, without access to the machine that ran it. #### When `--latest` refuses to run[​](#when---latest-refuses-to-run "Direct link to when---latest-refuses-to-run") `--latest` fails rather than guessing, and it never falls back to an older archive: | Situation | Result | | ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | No matching archives in the selected location | Non-zero exit naming the location that was listed. This is the expected state before the first backup runs. | | The newest archive is encrypted and no `--decrypt-key` was passed | Non-zero exit naming the archive. The check runs before the `--sleep` wait, before any download, and before any container is stopped. | | Both `--latest` and an archive name are passed | Non-zero exit. The two are mutually exclusive. | | `--s3` is passed without `--latest` | Non-zero exit. To restore one exact remote archive, pass its `s3://` URI as the argument instead. | Restoring whichever archive happens to be readable would leave a deployment holding stale data while reporting success. On a schedule nobody watches, that is worse than a job that visibly fails. info Because the selection happens before the `--sleep` wait, an archive transferred into place during the sleep is not considered by `--latest`. The wait exists so that a *named* file can be transferred in; with `--latest` the choice has already been made. info With `--latest --s3` the selected object is downloaded to a temporary file inside the backup directory and removed afterwards, exactly as an `s3://` URI restore downloads. A local archive sharing the selected object's name — what `create --s3-upload --s3-keep-local` leaves behind — is never overwritten and never deleted. #### Refresh staging from production on a schedule[​](#refresh-staging-from-production-on-a-schedule "Direct link to Refresh staging from production on a schedule") To keep a staging deployment on the newest production backup, run `restore --latest --s3` against the bucket production uploads to. Because no filename is involved, the job is stable across every backup production takes: ``` # /etc/cron.d/infrahub-staging-sync # Refresh staging from the newest production backup, nightly at 03:00. INFRAHUB_S3_BUCKET=my-backups INFRAHUB_S3_PREFIX=infrahub/prod 0 3 * * * root /usr/local/bin/infrahub-backup restore --latest --s3 --project infrahub-staging --reset-deployment-id >> /var/log/infrahub-staging-sync.log 2>&1 ``` Add `--reset-deployment-id` as shown, so the refreshed staging instance does not report the production deployment ID. Capture the job's output: the selection line above is the record of which production backup staging is currently running. danger A scheduled restore replaces all data in the deployment it targets. Confirm that `--project` — or the environment the job runs in — points at the deployment you intend to overwrite before enabling the schedule, and keep the production deployment out of reach of the job entirely. On Kubernetes, the `infrahub-backup` Helm chart renders the same refresh as a CronJob. See [Refresh an environment on a schedule](/backup/backup/kubernetes-scheduled-restore.md). ### Restore to specific project[​](#restore-to-specific-project "Direct link to Restore to specific project") If you have multiple Infrahub projects: ``` # Restore to a specific Docker Compose project infrahub-backup restore infrahub_backups/infrahub_backup_20250929_143022.tar.gz --project=infrahub-staging ``` ### Reset the deployment ID[​](#reset-the-deployment-id "Direct link to Reset the deployment ID") Every Infrahub instance carries a unique deployment ID stored on the Root node in the Neo4j database. When you restore a production backup into a non-production environment — for example, cloning prod into staging or spinning up a disaster-recovery replica — the restored instance inherits the source deployment ID, and both environments report the same identity. If you need the restored instance to appear as a distinct deployment, add the `--reset-deployment-id` flag: ``` infrahub-backup restore infrahub_backups/infrahub_backup_20250929_143022.tar.gz --reset-deployment-id ``` The flag generates a new UUID and writes it to the Root node after the database is restored and before the Infrahub application containers start, so the application never observes the source deployment ID. info Omit this flag when you are restoring into the same environment the backup came from (for example, recovering from data loss on the original instance). Resetting the deployment ID is only required when the restored instance is logically separate from the source. ## Step 3: Monitor restoration progress[​](#step-3-monitor-restoration-progress "Direct link to Step 3: Monitor restoration progress") Watch the detailed restoration output: ``` INFO[0000] Starting restore process... INFO[0000] Reading backup metadata... INFO[0001] Backup created: 2025-09-29T14:30:22Z INFO[0001] Backup version: 1.0.0 INFO[0001] Validating backup compatibility... INFO[0002] Validation successful INFO[0002] Stopping all Infrahub containers... INFO[0008] All containers stopped INFO[0008] Extracting backup archive... INFO[0012] Archive extracted successfully INFO[0012] Restoring Neo4j database... INFO[0025] Neo4j database restored INFO[0025] Restoring PostgreSQL database... INFO[0030] PostgreSQL database restored INFO[0032] Starting Infrahub containers... INFO[0040] All containers started INFO[0040] Restore completed successfully ``` ## Step 4: verify restoration[​](#step-4-verify-restoration "Direct link to Step 4: verify restoration") ### Check service health[​](#check-service-health "Direct link to Check service health") Verify all services are running: ``` # Check container status docker compose ps # All containers should show "Up" status # Example output: # NAME IMAGE STATUS # infrahub-server opsmill/infrahub:stable Up 2 minutes # database neo4j:5.13 Up 2 minutes # task-manager-db postgres:15 Up 2 minutes ``` ### Test application access[​](#test-application-access "Direct link to Test application access") Confirm Infrahub is accessible: ``` # Test HTTP endpoint curl -I http://localhost:8000 # Check GraphQL endpoint curl -X POST http://localhost:8000/graphql \ -H "Content-Type: application/json" \ -d '{"query": "{ InfrahubInfo { version } }"}' ``` ### Validate data integrity[​](#validate-data-integrity "Direct link to Validate data integrity") Verify your data was restored correctly: 1. Log into the Infrahub web interface 2. Check that your schemas are present 3. Verify critical data objects exist 4. Test a few key queries or operations 5. Review recent task history ## Step 5: post-restoration tasks[​](#step-5-post-restoration-tasks "Direct link to Step 5: post-restoration tasks") ### Verify integrations[​](#verify-integrations "Direct link to Verify integrations") Check external integrations are working: * Git repositories sync correctly * External authentication (LDAP/SSO) functions * Webhook endpoints are accessible * API integrations reconnect properly ## Validation[​](#validation "Direct link to Validation") Confirm restoration was successful: * ✓ All services are running * ✓ Web interface is accessible * ✓ Data appears complete and correct * ✓ Authentication works * ✓ Background tasks are processing * ✓ Integrations are functional ## Related resources[​](#related-resources "Direct link to Related resources") * [Back up your Infrahub instance](/backup/backup/create.md) * [CLI command reference](/backup/reference/commands.md) --- # Manage backup retention A backup schedule that never deletes anything eventually fills its disk or its bucket, and the failure usually surfaces as a failed backup rather than as a full volume. A retention policy bounds that growth: you declare how many recovery points to keep, and `infrahub-backup` deletes the rest. Follow this guide to preview a policy against your existing archives, apply it on demand, and attach it to a scheduled backup so the directory stays bounded without a wrapper script. You can also extend the same policy to S3-compatible storage. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before applying a retention policy: * Ensure `infrahub-backup` is installed and configured. See [Install Infrahub Backup](/backup/backup/install.md). * Know which directory holds your archives (`--backup-dir`, `./infrahub_backups` by default). * Confirm your recovery point objectives, so the policy you write matches what your organization requires. * For object storage, confirm your credentials can list and delete under the prefix. See [Grant list and delete permissions](#grant-list-and-delete-permissions). ## How retention decides what to keep[​](#how-retention-decides-what-to-keep "Direct link to How retention decides what to keep") Two independent rules define a policy, and you can set either one or both: | Flag | Rule | Environment variable | | --------------------- | ------------------------------ | -------------------------- | | `--retention-days N` | Keep backups newer than N days | `INFRAHUB_RETENTION_DAYS` | | `--retention-count N` | Keep the N most recent backups | `INFRAHUB_RETENTION_COUNT` | Flags and environment variables are the only two ways to configure retention. Where both supply the same rule, the flag on the command you ran wins. Both rules accept whole numbers of 1 or greater. To leave a rule inactive, omit its flag and leave its variable unset. Any other value either channel supplies — an explicit `0`, a negative number, `7.5`, `7d` — is a validation error that aborts the run before a single archive is examined, rather than being read as "disabled". A schedule configured wrongly fails loudly instead of quietly keeping everything or deleting more than you intended. A variable that is present but empty or whitespace-only counts as unset rather than as a bad value, because that is what Docker Compose and Kubernetes produce when a variable they were told to substitute has no value. Whitespace around a real value is tolerated. Four properties govern every deletion decision. **A backup survives if either rule claims it.** The rules combine as a union, not an intersection, so the more generous rule always wins. This is the property operators most often read backwards: ``` infrahub-backup prune --retention-days 7 --retention-count 14 --force ``` That policy keeps every archive from the last seven days *and* the 14 most recent archives, whichever set is larger. On a daily schedule it settles at 14 archives. On an hourly schedule the age rule dominates and roughly 168 archives survive. If you want a hard cap on the number of archives, set `--retention-count` alone. Pair the two rules when the schedule that produces the backups might stall. An age rule alone answers "how old may an archive be", not "how many recovery points must remain", so once every archive is older than the window the age rule claims nothing and only the keep-newest floor below applies — a scheduled `prune` erodes the location to a single archive. Adding a count rule raises that floor to the number you name, whether or not new backups keep arriving: ``` # Thirty days of history, and never fewer than 14 recovery points infrahub-backup prune --retention-days 30 --retention-count 14 --force ``` Retention attached to `create` is already protected from this: the policy is applied only after the backup is written and its checksum recorded, so a schedule that has started failing deletes nothing at all. The pairing matters for a standalone `prune` on its own schedule, which applies the policy whether or not a recent backup succeeded. **The most recent backup always survives.** Even a policy that matches every archive on disk keeps the newest one. There is no flag to override this, so a mistyped policy can never leave you with zero recovery points. **Each location is evaluated on its own.** The local directory and the S3 prefix are separate populations: each keeps its own newest archive, and each applies the rules to the archives it actually holds. Deleting a local copy never deletes the uploaded one. **Only recognized archives are considered.** Retention acts on files named `infrahub_backup_.tar.gz` or `infrahub_backup_.tar.gz.enc`, and it reads each backup's age from the timestamp embedded in that name rather than from the file modification time. Anything else in the directory — notes, unrelated archives, renamed backups — is invisible to retention and is never deleted. warning Because age comes from the filename, renaming an archive changes when retention considers it expired, and touching a file does not extend its life. Keep the generated names if you want the policy to behave predictably. That embedded timestamp carries no timezone. `infrahub-backup` writes it in the local time of the host taking the backup, reads it in the local time of the host applying retention, and measures a day as a fixed 24 hours. Archives written and pruned on the same host therefore always agree. If you prune from a container or scheduler host whose timezone differs, the age boundary moves by that offset and an archive can be deleted up to a day earlier or later than you expect. Give both hosts the same `TZ` if you split the work between them. ## Step 1: Preview what a policy would delete[​](#step-1-preview-what-a-policy-would-delete "Direct link to Step 1: Preview what a policy would delete") Run `prune` with `--dry-run` before trusting any new policy. It reports the exact set a real run would delete, deletes nothing, and never prompts: ``` infrahub-backup prune --backup-dir /data/backups/infrahub --retention-days 7 --dry-run ``` ``` INFO[0000] Applying retention policy (days: 7, count: 0) to 1 location(s) INFO[0000] Would prune backup infrahub_backup_20260715_073229.tar.gz from local:/data/backups/infrahub (dry run) INFO[0000] Would prune backup infrahub_backup_20260710_073229.tar.gz from local:/data/backups/infrahub (dry run) INFO[0000] Would prune backup infrahub_backup_20260705_073229.tar.gz from local:/data/backups/infrahub (dry run) INFO[0000] Retention at local:/data/backups/infrahub: 3 backup(s) kept, 3 candidate(s) to prune ``` Compare that list against what you expect to keep. A real run deletes exactly this set. ## Step 2: Apply the policy[​](#step-2-apply-the-policy "Direct link to Step 2: Apply the policy") Run the same command without `--dry-run`. On an interactive terminal, `prune` reports the candidates and asks once before deleting anything: ``` infrahub-backup prune --backup-dir /data/backups/infrahub --retention-days 7 ``` ``` INFO[0000] Applying retention policy (days: 7, count: 0) to 1 location(s) INFO[0000] Would prune backup infrahub_backup_20260715_073229.tar.gz from local:/data/backups/infrahub INFO[0000] Would prune backup infrahub_backup_20260710_073229.tar.gz from local:/data/backups/infrahub INFO[0000] Would prune backup infrahub_backup_20260705_073229.tar.gz from local:/data/backups/infrahub INFO[0000] Retention at local:/data/backups/infrahub: 3 backup(s) kept, 3 candidate(s) to prune Prune 3 backup(s) listed above? [y/N]: y INFO[0000] Pruned backup infrahub_backup_20260715_073229.tar.gz from local:/data/backups/infrahub INFO[0000] Pruned backup infrahub_backup_20260710_073229.tar.gz from local:/data/backups/infrahub INFO[0000] Pruned backup infrahub_backup_20260705_073229.tar.gz from local:/data/backups/infrahub INFO[0000] Pruned 3 of 3 confirmed backup(s) ``` The set you confirm is the set that is deleted: the candidates are chosen once, before the question, and nothing is listed again afterwards. A candidate another process removed in the meantime counts as already satisfied rather than as a failure, so the closing line can report fewer deletions than candidates without the run failing. Anything other than `y` or `yes` declines. A declined run deletes nothing and exits successfully, so an operator who changes their mind has not caused a failure. For scheduled and scripted runs, pass `--force` to skip both the prompt and the candidate listing. Every deletion is still logged as it happens, so a forced run remains auditable after the fact: ``` infrahub-backup prune --backup-dir /data/backups/infrahub --retention-days 7 --force ``` ``` INFO[0000] Applying retention policy (days: 7, count: 0) to 1 location(s) INFO[0000] Retention at local:/data/backups/infrahub: 3 backup(s) kept, 3 candidate(s) to prune INFO[0000] Pruned backup infrahub_backup_20260715_073229.tar.gz from local:/data/backups/infrahub INFO[0000] Pruned backup infrahub_backup_20260710_073229.tar.gz from local:/data/backups/infrahub INFO[0000] Pruned backup infrahub_backup_20260705_073229.tar.gz from local:/data/backups/infrahub ``` info Without `--force`, a run whose standard input is not a terminal refuses to delete anything and tells you to pass `--force`. This keeps a `prune` that was written for a terminal from deleting silently when it later runs under cron. `--dry-run` and `--force` together are rejected as contradictory: a dry run never deletes and never prompts, so there is nothing to force. warning Without `--force`, a run that cannot list one of its locations exits non-zero without prompting and without deleting anything at any location. Confirming a partial preview would mean answering for a set the tool does not fully know, and a bucket it cannot list is not the same as a bucket with nothing to prune. A mistyped `--backup-dir` and an unreachable S3 endpoint both land here. `--force` behaves differently by design: it prunes every location it could list and still exits non-zero naming the one that failed. ## Step 3: Attach retention to scheduled backups[​](#step-3-attach-retention-to-scheduled-backups "Direct link to Step 3: Attach retention to scheduled backups") Passing a retention flag to `create` applies the policy automatically after the backup finishes, with no prompt. Retention runs only once the archive is written and its checksum recorded, so a failed backup never triggers a deletion: ``` infrahub-backup create --retention-days 7 --retention-count 14 ``` Retention on `create` logs what a forced `prune` logs — the policy it applied, a `Retention at ` summary of how many archives it kept and how many it selected, and one line per deletion — so an unattended nightly run accounts for what it removed at the default log level. This replaces the `find`-based cleanup that backup scripts often carry, and it prunes only recognized archives instead of everything matching a glob: * Cron * Environment variables * Kubernetes (Helm) ``` # Daily backup at 2 AM, keeping 30 days of archives 0 2 * * * /usr/local/bin/infrahub-backup create --retention-days 30 >> /var/log/infrahub-backup.log 2>&1 ``` ``` # Both rules are also settable through the environment, for schedulers that # supply configuration that way rather than on the command line export INFRAHUB_BACKUP_DIR=/data/backups/infrahub export INFRAHUB_RETENTION_DAYS=30 export INFRAHUB_RETENTION_COUNT=60 infrahub-backup create ``` The `infrahub-backup` chart has no retention values key, so pass the rules to the backup CronJob through `backup.options.extraArgs`: ``` # values.yaml for the Infrahub Helm chart infrahub-backup: enabled: true backup: enabled: true mode: "cronjob" schedule: "0 2 * * *" storage: type: "s3" s3: bucket: "my-infrahub-backups" prefix: "infrahub/prod" region: "us-east-1" secretName: "backup-s3-credentials" options: extraArgs: - "--retention-days=30" - "--retention-count=60" ``` Each scheduled run then applies the policy after its backup succeeds, the same as a cron-driven `create`. Three differences matter on Kubernetes: * **The policy bounds the bucket, not the pod.** The backup directory is an `emptyDir` that starts empty on every run, and the default `backup.options.keepLocal: false` deletes the local copy as soon as the upload succeeds. The S3 prefix is the only location with archives left to prune. * **The credentials in `secretName` need delete rights.** Upload-only credentials make the S3 leg fail and the run exit non-zero. See [Grant list and delete permissions](#grant-list-and-delete-permissions). * **Retention rides along with a backup.** The chart always renders `create` as the command, so a prune-only CronJob is not expressible in values. Retention runs when a backup runs. Running `create` with no retention rule configured behaves exactly as it always has: nothing is pruned, and no archive is examined. note `--dry-run`, `--force`, and `--s3` belong to `prune` alone and are deliberately per-invocation switches, with no environment variable behind them. A persistent `force` setting would disarm the confirmation that makes pruning safe, and a persistent `s3` would delete objects an operator only configured for upload. If the backup succeeds but a deletion fails, the run exits non-zero with a message that states the backup succeeded and names the new archive, so you can tell a retention problem from a lost backup. Within each location, deletion is best-effort: one failure does not strand the remaining candidates. ## Step 4: Extend retention to object storage[​](#step-4-extend-retention-to-object-storage "Direct link to Step 4: Extend retention to object storage") Retention treats object storage as a second location, and it never touches that location unless you ask: * On `create`, the S3 leg runs exactly when that run uploaded an archive, which means `--s3-upload` was passed and the upload succeeded. * On `prune`, the S3 leg runs only when you pass `--s3`. Without it, a fully configured bucket is left alone. note `--s3-upload` without `--s3-keep-local` deletes the local copy of the new archive as soon as the upload succeeds, before retention runs. The keep-newest floor still protects each location's newest archive, but in the local directory that is the *previous* archive, not the one this run just made. Pass `--s3-keep-local` if you want the fresh archive in both places. ``` # Prune both the local directory and the configured bucket/prefix infrahub-backup prune --retention-days 30 --s3 --force \ --s3-bucket my-backups --s3-prefix infrahub/prod # Prune the local directory only; the bucket is untouched infrahub-backup prune --retention-days 30 --force ``` Only objects whose name matches the backup pattern are considered, and only under the configured prefix. Objects your team stores alongside the backups are never candidates. ### Grant list and delete permissions[​](#grant-list-and-delete-permissions "Direct link to Grant list and delete permissions") Uploading a backup needs write permission. Pruning one also needs to list the prefix and delete objects under it. Scope those rights to the backup prefix rather than the whole bucket: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "ListInfrahubBackupPrefix", "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::my-backups", "Condition": { "StringLike": { "s3:prefix": "infrahub/prod/*" } } }, { "Sid": "ManageInfrahubBackupObjects", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::my-backups/infrahub/prod/*" } ] } ``` With upload-only credentials the S3 leg fails and the run exits non-zero naming that location. What happens to the local directory depends on which right is missing and whether you passed `--force`: | Missing right | `--dry-run` | Interactive (no `--force`) | `--force` | | ---------------------- | --------------- | ------------------------------------------------------------------------------------------- | ---------------------- | | `s3:ListBucket` | Nothing deleted | Nothing deleted anywhere; you are not asked | Local directory pruned | | `s3:DeleteObject` only | Nothing deleted | Candidates listed for both locations; on `y`, local deletions succeed and S3 deletions fail | Local directory pruned | Without list permission the S3 leg cannot be previewed at all, and a preview the tool cannot complete stops the whole non-forced run before it deletes anything. Grant both rights together, and use `--dry-run` to confirm the bucket is being listed before you rely on a scheduled `--force` run. ### Combine with bucket lifecycle rules[​](#combine-with-bucket-lifecycle-rules "Direct link to Combine with bucket lifecycle rules") A bucket lifecycle rule and a retention policy solve overlapping problems from different sides, and running both is a reasonable belt-and-braces arrangement: * Retention keeps the two locations consistent with one policy you can preview with `--dry-run`, and it guarantees the newest archive survives. * A lifecycle rule enforces expiry inside the storage provider even if the tool never runs again, and it can also transition older archives to colder storage classes. If you use both, set the lifecycle expiry window wider than the retention window. A lifecycle rule has no notion of "keep the newest", so a window narrower than your backup interval can expire your last remaining archive. ## Validation[​](#validation "Direct link to Validation") Confirm your retention setup works: * ✓ `--dry-run` lists exactly the archives you expect to lose * ✓ A real run deletes that set and nothing else * ✓ Unrelated files in the backup directory are still present * ✓ At least one archive remains at every location * ✓ For object storage, the bucket reflects the same policy after a `--s3` run ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### An archive I expected to be deleted is still there[​](#an-archive-i-expected-to-be-deleted-is-still-there "Direct link to An archive I expected to be deleted is still there") Check whether it is the newest at its location — the floor keeps it unconditionally. Otherwise, confirm the name still matches `infrahub_backup_.tar.gz`; a renamed archive is invisible to retention. Remember also that the two rules combine as a union, so an archive older than `--retention-days` still survives while it sits within `--retention-count`. ### An archive I copied back from the bucket became a deletion candidate[​](#an-archive-i-copied-back-from-the-bucket-became-a-deletion-candidate "Direct link to An archive I copied back from the bucket became a deletion candidate") An archive you place in the backup directory yourself — copying one down from the bucket with `aws s3 cp` or `mc cp`, or restoring one from tape — keeps its original name and therefore its original timestamp. Retention reads it as an archive of that age, not as one that arrived today, so the next run may well delete it. Keep such a copy outside the backup directory, or rename it, if you need it to survive. `infrahub-backup restore s3://…` does not leave you in this situation: it downloads into the backup directory but removes its own copy once the restore finishes. ### Retention did nothing on the `plakar` backend[​](#retention-did-nothing-on-the-plakar-backend "Direct link to retention-did-nothing-on-the-plakar-backend") Retention is not yet supported for the `plakar` backend. A `create` run with `--backend plakar` and retention flags completes the backup, warns that retention is not yet supported, prunes nothing, and exits successfully. A `prune` run against that backend fails with a validation error rather than doing nothing quietly. Use the default tarball backend if you need retention today. ## Related resources[​](#related-resources "Direct link to Related resources") * [Back up your Infrahub instance](/backup/backup/create.md) * [Restore from a backup](/backup/backup/restore.md) * [CLI command reference](/backup/reference/commands.md) * [Configuration reference](/backup/reference/configuration.md) --- # Infrahub Collect Use Infrahub Collect to gather everything OpsMill support needs to investigate an issue — service logs, diagnostic status, configuration, and metrics — into a single local archive. Run it when support asks for logs, or to attach diagnostic data to a support request. The same command produces the same bundle on Docker Compose and Kubernetes. ## Safe to run at any time[​](#safe-to-run-at-any-time "Direct link to Safe to run at any time") Collection is strictly read-only. Unlike `infrahub-backup create`, the collect command never stops, restarts, or scales any container or pod, so it is safe to run against a production instance at any time — including while the instance is degraded. If a service is down, collection continues with the rest and records the failure in the manifest. A partial bundle is still a useful bundle — a degraded instance is precisely when you need one. The binary is self-contained. By default, collection works offline: it uses your existing Docker or kubectl access and performs no network access beyond the deployment itself. The one exception is the opt-in `--benchmark`, which downloads a benchmark image. ## Collect or back up?[​](#collect-or-back-up "Direct link to Collect or back up?") A troubleshooting bundle contains diagnostics for support to read; a [backup](/backup/backup/create.md) is a restorable snapshot of your data. They are separate operations with different safety profiles — collection is read-only, while a backup can stop containers while the snapshot is taken. When support asks for both, run `infrahub-collect create --include-backup` to produce them in one run. ## What a bundle contains[​](#what-a-bundle-contains "Direct link to What a bundle contains") Each bundle is a single `tar.gz` archive. Inside, every file lives under a top-level `bundle/` directory, alongside a `bundle_information.json` manifest that records an explicit outcome — success, failed, or skipped — for every collector, so support can see what was captured and what was not. | Collector | What it gathers | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | | Service logs | Container logs for every service and replica, plus previous-container logs for restarted pods | | Database | Neo4j server logs (full log directory with `--include-queries`) | | Message queue | RabbitMQ queues, exchanges, bindings, connections, and status | | Cache | Redis info, client list, configuration, and slow log | | Task worker | Prefect worker state, one directory per replica | | Task manager | Work pools, work queues, recent flow runs, the pending and running flow and task runs, events, and automations | | Server | Version, installed packages, API information, configuration, schema, and masked environment variables | | Metrics | Container resource usage | | Backup (opt-in) | A standalone backup produced by the standard backup behavior and referenced in the manifest, not embedded in the archive (`--include-backup`) | | Benchmark (opt-in) | Host resource benchmark results (`--benchmark`) | The layout is identical on Docker Compose and Kubernetes. ## Review before sharing[​](#review-before-sharing "Direct link to Review before sharing") Masking is based on key names: environment variables and configuration keys containing `password`, `secret`, `token`, or `key` are replaced with `********` before they are written to the bundle. Secrets stored under other key names are not detected. Service logs are collected as-is, and database query logs (`--include-queries`) can contain customer data. Review the bundle contents before sharing it. --- # Collect a troubleshooting bundle Collect a troubleshooting bundle from an Infrahub deployment and share it with OpsMill support. Collection is read-only and safe to run at any time, including while the instance is degraded. See [Infrahub Collect](/backup/collect.md) for what a bundle contains and how secrets are masked. The same command produces the same bundle on Docker Compose and Kubernetes. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before collecting a bundle: * Docker Compose * Kubernetes - The `infrahub-collect` binary is installed ([installation guide](/backup/collect/install.md)) - Docker and Docker Compose are available to your user - The Infrahub Compose project is running (fully or partially) * The `infrahub-collect` binary is installed ([installation guide](/backup/collect/install.md)) * `kubectl` is configured with access to the cluster running Infrahub * Your role can read pod logs and execute commands in pods (`pods/log` and `pods/exec`); no write or scale permissions are needed The binary is self-contained and collection works offline by default: it uses your existing Docker or kubectl access and performs no network access beyond your deployment itself. The one exception is the opt-in `--benchmark`, which downloads a benchmark image. ## Step 1: Verify environment detection[​](#step-1-verify-environment-detection "Direct link to Step 1: Verify environment detection") Confirm the tool can find your Infrahub deployment: ``` infrahub-collect environment detect ``` The tool auto-detects Docker Compose projects and Kubernetes namespaces containing Infrahub. To list every deployment it detects: ``` infrahub-collect environment list ``` ## Step 2: Create the bundle[​](#step-2-create-the-bundle "Direct link to Step 2: Create the bundle") * Docker Compose * Kubernetes For a single Infrahub project, run: ``` infrahub-collect create ``` If several Infrahub projects run on the same host, target one explicitly: ``` infrahub-collect create --project=infrahub-production ``` If a single namespace contains Infrahub, run: ``` infrahub-collect create ``` To target a specific namespace: ``` infrahub-collect create --k8s-namespace=infrahub-prod ``` On Kubernetes, the bundle includes logs from **every replica** of each service, and for pods that crashed and restarted it also captures the previous container's logs — often exactly where the error is. By default the bundle is written to `./infrahub_bundles`. Use `--output-dir` to choose another location: ``` infrahub-collect create --output-dir=/tmp/support ``` ## Step 3: Monitor collection progress[​](#step-3-monitor-collection-progress "Direct link to Step 3: Monitor collection progress") The command streams progress as each collector runs: ``` INFO[0000] Detecting deployment environment... INFO[0000] Kubernetes environment detected (namespace: infrahub-prod) INFO[0000] Creating collection ID: 20260702_141530 INFO[0001] Collecting logs: infrahub-server (2 replicas) INFO[0004] Collecting logs: task-worker (3 replicas, 1 restarted - including previous logs) INFO[0009] Collecting database logs... INFO[0012] Collecting message-queue status... INFO[0013] Collecting cache status... INFO[0014] Collecting task-manager state... INFO[0016] Collecting server info... INFO[0017] Collecting container metrics... INFO[0018] Creating compressed archive... INFO[0021] Bundle created: infrahub_bundles/support_bundle_20260702_141530.tar.gz ``` If a service is down or unreachable, collection continues with the remaining services and records the failure: ``` WARN[0013] Collector failed: cache status (container not running) - continuing ``` A partial bundle is still a useful bundle — a degraded instance is precisely when you need one. ## Step 4: Verify the bundle[​](#step-4-verify-the-bundle "Direct link to Step 4: Verify the bundle") Check the archive and review what was collected: ``` # Verify archive integrity tar -tzf infrahub_bundles/support_bundle_20260702_141530.tar.gz > /dev/null && echo "Archive is valid" # View the collection manifest tar -xzOf infrahub_bundles/support_bundle_20260702_141530.tar.gz bundle/bundle_information.json | jq '.' ``` The manifest records what was attempted and what succeeded: ``` { "manifest_version": 2026070200, "collect_id": "20260702_141530", "created_at": "2026-07-02T14:15:30Z", "tool_version": "1.2.0", "infrahub_version": "1.5.2", "environment": "kubernetes", "edition": "enterprise", "helm": { "release_name": "infrahub", "chart": "infrahub-enterprise", "chart_version": "1.2.3" }, "log_lines": 100000, "collectors": [ {"name": "logs/infrahub-server", "status": "success"}, {"name": "logs/task-worker", "status": "success"}, {"name": "cache-status", "status": "failed", "reason": "container not running"}, {"name": "benchmark", "status": "skipped", "reason": "not requested"} ] } ``` The `edition` field reports whether the deployment runs Infrahub Community or Enterprise. On Docker Compose it is classified from the `infrahub-server` container image, and on Kubernetes from the Helm chart name. It is omitted when the edition cannot be determined. The bundle layout is identical on Docker Compose and Kubernetes. Inside the archive, all files live under a top-level `bundle/` directory: * `bundle_information.json` - Collection manifest * `logs//` - Container logs, one file per replica (plus `*.previous.log` for restarted pods) * `database/` - Neo4j server logs * `message-queue/` - RabbitMQ queues, exchanges, connections, and status * `cache/` - Redis info, clients, configuration, and slow log * `task-worker/` - Task worker state, one directory per replica * `task-manager/` - Work pools, work queues, recent flow runs, the pending and running flow and task runs, events, and automations * `server/` - Version, installed packages, API information, configuration, schema, worker schema-sync status (`infrahub_status.json`), environment variables * `metrics/` - Container resource metrics Each file holds the output of the command that produced it and nothing else, so a `.json` file always parses. When a command fails, its file is absent and a sibling `.err.txt` records what ran, how it failed, and what the command reported — for example `task-manager/flow-runs.err.txt` in place of `flow-runs.json`. The manifest lists the same failure as a reason on the collector. Review before sharing Masking is based on key names: environment variables and configuration keys containing `password`, `secret`, `token`, or `key` are replaced with `********` before they are written to the bundle. Secrets stored under other key names are not detected. Service logs are collected as-is, and database query logs (`--include-queries`) can contain customer data. Review the bundle contents before sharing. ## Step 5: Share the bundle with support[​](#step-5-share-the-bundle-with-support "Direct link to Step 5: Share the bundle with support") Send the archive to OpsMill support through your usual support channel. The bundle is a plain local file, so environments with restricted egress can transfer it through whatever mechanism their security policy allows. ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Include a backup for issue reproduction[​](#include-a-backup-for-issue-reproduction "Direct link to Include a backup for issue reproduction") Support often asks for a backup alongside logs so they can reproduce your issue locally. Create both in one run: ``` infrahub-collect create --include-backup ``` This runs the standard `infrahub-backup` logic and records the backup in the bundle manifest. warning Unlike log collection, the backup step follows the normal backup behavior for your deployment, which can include stopping application containers while the snapshot is taken. Plan for a short interruption, or skip `--include-backup` and create the backup separately during a maintenance window. ### Include database query logs[​](#include-database-query-logs "Direct link to Include database query logs") Query logs can help diagnose slow or failing database operations, but they may contain customer data, so they are excluded by default: ``` infrahub-collect create --include-queries ``` ### Adjust the log volume limit[​](#adjust-the-log-volume-limit "Direct link to Adjust the log volume limit") By default, collection captures the most recent 100,000 log lines per container. Raise or lower the limit as needed: ``` # Collect more history for a long-running issue infrahub-collect create --log-lines=500000 # Or via environment variable export INFRAHUB_LOG_LINES=500000 infrahub-collect create ``` The effective limit is recorded in the manifest, so support knows whether logs were truncated. ### Run a performance benchmark[​](#run-a-performance-benchmark "Direct link to Run a performance benchmark") For performance investigations, support may ask you to include a benchmark run: ``` infrahub-collect create --benchmark ``` The benchmark requires downloading the OpsMill benchmark container image and measures whether the host meets Infrahub's resource requirements — disk IOPS, memory, and single-core CPU — rather than generating load against your instance. If the image cannot be pulled — for example in an air-gapped environment — the benchmark is skipped with a warning and the rest of the collection completes normally. ## Troubleshoot collection[​](#troubleshoot-collection "Direct link to Troubleshoot collection") ### No deployment detected[​](#no-deployment-detected "Direct link to No deployment detected") If `infrahub-collect create` reports that no environment was found, list what the tool can see: ``` infrahub-collect environment list ``` On Docker Compose, confirm the project is running with `docker compose ls`; the project name must contain `infrahub`, or you must name it with `--project`. On Kubernetes, the namespace is discovered through the `app.kubernetes.io/name=infrahub` label; for deployments with custom labels, name the namespace with `--k8s-namespace`. ### Permission denied on Kubernetes[​](#permission-denied-on-kubernetes "Direct link to Permission denied on Kubernetes") Log collection needs the `pods/log` permission and the status collectors need `pods/exec`. If your role lacks one of them, the affected collectors are recorded as failed in the manifest. Ask your cluster administrator for read access to both — collection needs no write or scale permissions. ### Docker or kubectl not found[​](#docker-or-kubectl-not-found "Direct link to Docker or kubectl not found") The tool depends on the Docker CLI or kubectl being installed and configured on the machine where it runs. When neither is usable, the command fails immediately and the error names the missing CLI. ### Individual collectors failed[​](#individual-collectors-failed "Direct link to Individual collectors failed") Collector failures on a degraded deployment are expected — a stopped cache container cannot answer status queries. The command still exits successfully, and the manifest records each failure with a reason. Send the partial bundle as-is; support reads the manifest first. ## Validation[​](#validation "Direct link to Validation") Confirm your collection works: * [ ] `infrahub-collect environment detect` finds your deployment * Bundle archive is created and passes the integrity check * Manifest lists an explicit outcome for every collector * Logs are present for every service replica * Masked environment output contains no plaintext secrets ## Related resources[​](#related-resources "Direct link to Related resources") * [Install Infrahub Collect](/backup/collect/install.md) * [Back up your Infrahub instance](/backup/backup/create.md) * [CLI command reference](/backup/reference/commands.md) * [Configuration reference](/backup/reference/configuration.md) --- # Install Infrahub Collect Install the `infrahub-collect` CLI. One binary collects from both Docker Compose and Kubernetes deployments; on Kubernetes it runs from your workstation or a CI runner and reaches the cluster through kubectl. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before installing Infrahub Collect, ensure you have: **For Docker deployments:** * Administrative or sudo access on your system * Docker with Docker Compose installed **For Kubernetes deployments:** * Administrative or sudo access on your system * kubectl configured with permissions to read pod logs and execute commands in pods (`pods/log` and `pods/exec`) in the Infrahub namespace **If building from source:** * Git and network access to clone the repository * Go 1.25 or later installed ## Installation methods[​](#installation-methods "Direct link to Installation methods") * Direct Download * Manual Selection * Build from Source ### Download the pre-built binary[​](#download-the-pre-built-binary "Direct link to Download the pre-built binary") Download the pre-built binary directly: ``` # Download the appropriate binary for your system curl https://infrahub.opsmill.io/ops/$(uname -s)/$(uname -m)/infrahub-collect -o infrahub-collect # Make it executable chmod +x infrahub-collect # Move to a directory in your PATH (optional) sudo mv infrahub-collect /usr/local/bin/ ``` ### Select and download a specific version[​](#select-and-download-a-specific-version "Direct link to Select and download a specific version") If you need a specific version or architecture, manually select the appropriate binary: 1. Identify your system architecture: ``` uname -s # Operating system (Linux/Darwin) uname -m # Architecture (x86_64/aarch64) ``` 2. Download the matching binary: * **Linux AMD64**: `curl -L https://infrahub.opsmill.io/ops/Linux/x86_64/infrahub-collect -o infrahub-collect` * **Linux ARM64**: `curl -L https://infrahub.opsmill.io/ops/Linux/aarch64/infrahub-collect -o infrahub-collect` * **macOS AMD64**: `curl -L https://infrahub.opsmill.io/ops/Darwin/x86_64/infrahub-collect -o infrahub-collect` * **macOS ARM64**: `curl -L https://infrahub.opsmill.io/ops/Darwin/arm64/infrahub-collect -o infrahub-collect` 3. Make executable and optionally install system-wide: ``` chmod +x infrahub-collect sudo mv infrahub-collect /usr/local/bin/ # Optional ``` ### Build from source code[​](#build-from-source-code "Direct link to Build from source code") To install Infrahub Collect, build from source: 1. Install Go 1.25 or later: ``` # Check if Go is installed go version ``` 2. Clone the repository: ``` git clone https://github.com/opsmill/infrahub-ops-cli.git cd infrahub-ops-cli ``` 3. Build the binaries: ``` make build ``` 4. Install to your PATH: ``` sudo cp bin/infrahub-collect /usr/local/bin/ ``` The same binary collects from Docker Compose and Kubernetes deployments — there is no separate in-cluster installation. On Kubernetes, the tool runs from your workstation (or a CI runner) and reaches the cluster through kubectl. ## Verify installation[​](#verify-installation "Direct link to Verify installation") After installing the CLI, verify it is working correctly: ``` # Check versions infrahub-collect version # Verify environment detection infrahub-collect environment detect # Display help infrahub-collect --help ``` Expected output: ``` Version: [git ref] ``` ## Configure environment[​](#configure-environment "Direct link to Configure environment") Infrahub Collect uses environment variables for configuration. Set these based on your deployment: ### Docker Compose deployments[​](#docker-compose-deployments "Direct link to Docker Compose deployments") ``` # Optional: Target a specific project export INFRAHUB_PROJECT=infrahub-production # Optional: Custom bundle location export INFRAHUB_OUTPUT_DIR=/data/bundles/infrahub ``` ### Kubernetes deployments[​](#kubernetes-deployments "Direct link to Kubernetes deployments") ``` # Set the namespace export INFRAHUB_K8S_NAMESPACE=infrahub # Set kubeconfig if not default export KUBECONFIG=/path/to/kubeconfig ``` ## Validation[​](#validation "Direct link to Validation") Confirm the CLI can connect to your Infrahub instance: ``` # List available deployments infrahub-collect environment list # Test environment detection infrahub-collect environment detect ``` If these commands succeed, installation is complete. ## Related resources[​](#related-resources "Direct link to Related resources") * [Collect a troubleshooting bundle](/backup/collect/create.md) * [CLI command reference](/backup/reference/commands.md) * [Configuration reference](/backup/reference/configuration.md) --- # CLI command reference Complete reference for the Infrahub Ops CLI tools (`infrahub-backup`, `infrahub-collect`), their commands, flags, and options. ## infrahub-backup[​](#infrahub-backup "Direct link to infrahub-backup") ### Command structure[​](#command-structure "Direct link to Command structure") ``` infrahub-backup [global-flags] [flags] [arguments] ``` ### Global flags[​](#global-flags "Direct link to Global flags") These flags apply to all infrahub-backup commands: | Flag | Description | Default | Environment Variable | | --------------------------- | -------------------------------------- | -------------------- | ---------------------- | | `--project ` | Target specific Docker Compose project | Auto-detect | `INFRAHUB_PROJECT` | | `--backup-dir ` | Directory for backup files | `./infrahub_backups` | `INFRAHUB_BACKUP_DIR` | | `--log-format ` | Output format for logs | `text` | `INFRAHUB_LOG_FORMAT` | | `--s3-bucket ` | S3 bucket name for backup storage | - | `INFRAHUB_S3_BUCKET` | | `--s3-prefix ` | S3 key prefix (path within bucket) | - | `INFRAHUB_S3_PREFIX` | | `--s3-endpoint ` | Custom S3 endpoint URL (for MinIO) | - | `INFRAHUB_S3_ENDPOINT` | | `--s3-region ` | AWS region for S3 bucket | `us-east-1` | `INFRAHUB_S3_REGION` | | `--help, -h` | Show help for any command | - | - | ### Backup commands[​](#backup-commands "Direct link to Backup commands") #### create[​](#create "Direct link to create") Creates a comprehensive backup of the Infrahub instance. **Syntax:** ``` infrahub-backup create [flags] ``` **Flags:** | Flag | Description | Default | Environment Variable | | ------------------------ | ---------------------------------------------------------------------------------------- | ------- | ------------------------------ | | `--force` | Force backup even if tasks are running | `false` | `INFRAHUB_FORCE` | | `--redact` | Redact all attribute values before backup (destructive, requires `--force`) | `false` | `INFRAHUB_REDACT` | | `--neo4jmetadata ` | Neo4j metadata to include | `all` | `INFRAHUB_NEO4JMETADATA` | | `--exclude-taskmanager` | Exclude the task manager (Prefect) database from the backup archive | `false` | `INFRAHUB_EXCLUDE_TASKMANAGER` | | `--s3-upload` | Upload backup to S3 after creation | `false` | `INFRAHUB_S3_UPLOAD` | | `--s3-keep-local` | Keep local backup file after S3 upload | `false` | `INFRAHUB_S3_KEEP_LOCAL` | | `--sleep` | Sleep duration after backup for manual file transfer | `0` | `INFRAHUB_SLEEP` | | `--retention-days ` | After a successful backup, prune backups older than N days (N >= 1, omit to disable) | unset | `INFRAHUB_RETENTION_DAYS` | | `--retention-count ` | After a successful backup, keep only the N most recent backups (N >= 1, omit to disable) | unset | `INFRAHUB_RETENTION_COUNT` | **Neo4j metadata options:** * `all` - Include all user and role metadata * `users` - Include only user accounts * `roles` - Include only role definitions * `none` - Exclude all metadata **Examples:** ``` # Basic backup infrahub-backup create # Force backup with running tasks infrahub-backup create --force # Backup without user metadata infrahub-backup create --neo4jmetadata=none # Backup and upload to S3 infrahub-backup create --s3-upload --s3-bucket my-backups --s3-prefix infrahub/prod # Backup and upload to S3, keeping local copy infrahub-backup create --s3-upload --s3-bucket my-backups --s3-keep-local # Create a redacted backup (replaces all attribute values with random UUIDs) infrahub-backup create --redact --force # Backup, then prune archives outside the retention policy infrahub-backup create --retention-days 30 --retention-count 60 ``` Retention runs only after the backup fully succeeds, and it never prompts. The local backup directory is always evaluated; the S3 prefix is evaluated when the same run uploaded to it. See [Manage backup retention](/backup/backup/retention.md). #### restore[​](#restore "Direct link to restore") Restores Infrahub from a backup file, an S3 URI, or the most recent backup in a location. **Syntax:** ``` infrahub-backup restore infrahub-backup restore --latest [--s3] ``` **Arguments:** * `` - Path to backup archive or S3 URI, required unless `--latest` is passed * Local file: `infrahub_backup_20250929_143022.tar.gz` * S3 URI: `s3://bucket/prefix/infrahub_backup_20250929_143022.tar.gz` **Flags:** | Flag | Description | Default | Environment Variable | | ----------------------- | ------------------------------------------------------------------------------------------------------ | ------- | ------------------------------ | | `--latest` | Restore the most recent backup instead of naming an archive; mutually exclusive with the argument | `false` | - | | `--s3` | With `--latest`: choose from the configured S3 bucket and prefix instead of the local backup directory | `false` | - | | `--exclude-taskmanager` | Skip restoring the task manager database even if the dump is present | `false` | - | | `--migrate-format` | Run Neo4j database format migration after restore | `false` | - | | `--sleep ` | Wait this long before the restore begins, for manual file transfer | `0` | - | | `--decrypt-key ` | Private key PEM file for decrypting an encrypted backup | - | `INFRAHUB_DECRYPT_KEY` | | `--reset-deployment-id` | Generate a new Root node UUID after restore to detach this instance from the source deployment ID | `false` | `INFRAHUB_RESET_DEPLOYMENT_ID` | `--decrypt-key` and `--reset-deployment-id` also read their variable, with the flag winning where both supply a value — which is what lets a scheduled restore configured entirely through the environment decrypt an archive. The remaining flags are command-line only: a persistent `--latest` would turn a mistyped `restore` into a data-overwriting default, a persistent `--s3` would silently move the location every restore reads, and `INFRAHUB_SLEEP` configures `create`'s `--sleep`, not this one. An encrypted archive is decrypted into a temporary file beside it, which is removed when the run ends. Both the archive and any other archive in the directory are left untouched, including a plain archive of the same timestamp. `--latest` ranks archives exactly as retention does — by the timestamp embedded in the name, newest first, ties broken by name descending — and only names matching `infrahub_backup_.tar.gz[.enc]` take part. Exactly one location is consulted per run, and the two are never merged. The run exits non-zero without touching the deployment when the selected location holds no archives, when the newest archive is encrypted and no `--decrypt-key` was passed, when `--latest` is combined with an archive name, or when `--s3` is passed without `--latest`. `--latest` never falls back to an older archive. See [Restore from a backup](/backup/backup/restore.md). **Examples:** ``` # Restore from local file infrahub-backup restore infrahub_backup_20250929_143022.tar.gz # Restore the newest archive in the backup directory infrahub-backup restore --latest # Restore the newest archive in the configured bucket and prefix infrahub-backup restore --latest --s3 --s3-bucket my-backups --s3-prefix infrahub/prod # Restore from S3 infrahub-backup restore s3://my-backups/infrahub/prod/infrahub_backup_20250929_143022.tar.gz # Restore from MinIO infrahub-backup restore --s3-endpoint http://minio.local:9000 s3://my-backups/infrahub_backup_20250929_143022.tar.gz # Restore when the task manager database was excluded from the backup infrahub-backup restore infrahub_backup_20251022_120000.tar.gz --exclude-taskmanager ``` #### prune[​](#prune "Direct link to prune") Applies a retention policy to existing backups without creating a new one. At least one retention rule is required. **Syntax:** ``` infrahub-backup prune (--retention-days | --retention-count ) [flags] ``` **Flags:** | Flag | Description | Default | Environment Variable | | ----------------------- | --------------------------------------------------------------------------- | ------- | -------------------------- | | `--retention-days ` | Delete backups older than N days (N >= 1) | unset | `INFRAHUB_RETENTION_DAYS` | | `--retention-count ` | Keep only the N most recent backups (N >= 1) | unset | `INFRAHUB_RETENTION_COUNT` | | `--dry-run` | List exactly what a real run would delete, delete nothing, and never prompt | `false` | - | | `--force` | Skip the confirmation prompt (for non-interactive and scripted use) | `false` | - | | `--s3` | Also prune backups under the configured S3 bucket/prefix | `false` | - | Flags and environment variables are the only channels that configure retention; the flag wins where both supply a rule. Any value either channel supplies that is not a whole number of at least 1 — including an explicit `0` — is a validation error that aborts the run. A variable that is present but empty counts as unset. `--dry-run`, `--force`, and `--s3` are per-invocation switches with no environment variable behind them. **Examples:** ``` # Preview what a 7-day policy would delete infrahub-backup prune --retention-days 7 --dry-run # Keep the 30 most recent backups, asking for confirmation first infrahub-backup prune --retention-count 30 # Non-interactive prune of both the local directory and the bucket infrahub-backup prune --retention-days 30 --s3 --force ``` **Behavior:** * The rules combine as a union: a backup survives if either rule claims it. * The most recent backup at each location always survives; there is no override. * The local directory and the S3 prefix are evaluated independently. S3 is never touched without `--s3`. * A backup's age comes from the timestamp in its filename, read in the local time of the host running the command, with a day measured as a fixed 24 hours. * The set previewed and confirmed is exactly the set deleted: candidates are selected once, and nothing is listed again after the confirmation. A candidate that had already vanished counts as satisfied, not as a failure. * Without `--force`, a run whose standard input is not a terminal refuses to delete anything. * Without `--force`, a run that cannot list one of its locations exits before the confirmation and deletes nothing at any location. With `--force`, every location that could be listed is still pruned. * `--dry-run` with `--force` is rejected as contradictory. **Exit codes:** | Outcome | Exit code | | --------------------------------------------------------------------------------------------------- | --------- | | Pruned successfully, dry run, nothing to prune, or prompt declined | `0` | | Validation error (no rule, value below 1, contradictory flags, `plakar` backend) | non-zero | | A location could not be listed, without `--force` — nothing deleted anywhere, no confirmation asked | non-zero | | A location failed, with `--force` — every other location is still pruned | non-zero | | A deletion failed after a complete preview — the remaining candidates are still attempted | non-zero | ### Environment commands[​](#environment-commands "Direct link to Environment commands") #### environment detect[​](#environment-detect "Direct link to environment detect") Detects and displays the current deployment environment. **Syntax:** ``` infrahub-backup environment detect ``` **Example output:** ``` INFO[0000] Detecting deployment environment... INFO[0000] Docker environment detected INFO[0000] Found Docker Compose project: infrahub-demo ``` #### environment list[​](#environment-list "Direct link to environment list") Lists all available Infrahub Docker Compose projects. **Syntax:** ``` infrahub-backup environment list ``` **Example output:** ``` infrahub-production Running 7/7 infrahub-staging Running 7/7 infrahub-dev Stopped 0/7 ``` ### Utility commands[​](#utility-commands "Direct link to Utility commands") #### version[​](#version "Direct link to version") Displays version information. **Syntax:** ``` infrahub-backup version ``` **Example output:** ``` Version: 1.0.0 ``` #### update[​](#update "Direct link to update") Updates the running binary to a newer release. Downloads the matching artifact from GitHub Releases, verifies its SHA-256 checksum, and replaces the binary atomically. Available on both `infrahub-backup` and `infrahub-collect`. **Syntax:** ``` infrahub-backup update [flags] ``` **Flags:** | Flag | Description | Default | | ----------------- | ---------------------------------------------------------------- | ------- | | `--check` | Report whether an update is available without installing it | `false` | | `--yes, -y` | Skip the confirmation prompt (required for non-interactive use) | `false` | | `--version ` | Install a specific release (e.g. `v1.7.2`) instead of the latest | latest | **Environment variables:** | Variable | Description | | --------------------------- | -------------------------------------------------------------------------------------------------- | | `GITHUB_TOKEN` / `GH_TOKEN` | Optional GitHub token to raise the API rate limit on CI/shared IPs. Never required for normal use. | Self-update is refused for binaries installed via Homebrew or running inside a container image, for development builds, and when the binary location is not writable. See [Update the CLI](/backup/self-update.md). ## infrahub-collect[​](#infrahub-collect "Direct link to infrahub-collect") ### Command structure[​](#command-structure-1 "Direct link to Command structure") ``` infrahub-collect [global-flags] [flags] ``` ### Global flags[​](#global-flags-1 "Direct link to Global flags") These flags apply to all infrahub-collect commands: | Flag | Description | Default | Environment Variable | | --------------------------- | -------------------------------------- | -------------------- | ------------------------ | | `--project ` | Target specific Docker Compose project | Auto-detect | `INFRAHUB_PROJECT` | | `--k8s-namespace ` | Target specific Kubernetes namespace | Auto-detect | `INFRAHUB_K8S_NAMESPACE` | | `--output-dir ` | Directory for bundle files | `./infrahub_bundles` | `INFRAHUB_OUTPUT_DIR` | | `--log-format ` | Output format for logs | `text` | `INFRAHUB_LOG_FORMAT` | | `--help, -h` | Show help for any command | - | - | ### Collect commands[​](#collect-commands "Direct link to Collect commands") #### create[​](#create-1 "Direct link to create") Collects a troubleshooting bundle from the Infrahub instance. Collection is read-only: no container or pod is stopped, restarted, or scaled. Individual collector failures are recorded in the bundle manifest and do not abort the run. **Syntax:** ``` infrahub-collect create [flags] ``` **Flags:** | Flag | Description | Default | Environment Variable | | ------------------- | ------------------------------------------------------------------------------------------------------------------ | -------- | -------------------------- | | `--log-lines ` | Maximum log lines collected per container | `100000` | `INFRAHUB_LOG_LINES` | | `--include-backup` | Also create a backup using the standard backup behavior | `false` | `INFRAHUB_INCLUDE_BACKUP` | | `--include-queries` | Include database query logs (may contain customer data) | `false` | `INFRAHUB_INCLUDE_QUERIES` | | `--benchmark` | Run the OpsMill benchmark and include its results (requires image download; skipped with a warning if unavailable) | `false` | `INFRAHUB_BENCHMARK` | **Examples:** ``` # Basic collection infrahub-collect create # Target a specific Kubernetes namespace infrahub-collect create --k8s-namespace=infrahub-prod # Collect with more log history infrahub-collect create --log-lines=500000 # Collect logs and a backup in one run infrahub-collect create --include-backup # Performance investigation with benchmark infrahub-collect create --benchmark ``` **Output:** A single archive named `support_bundle_.tar.gz` in the output directory. Inside the archive, all files live under a top-level `bundle/` directory, with a `bundle_information.json` manifest recording an explicit outcome for every collector. **Collectors:** | Collector | What it gathers | | -------------------- | -------------------------------------------------------------------------------------------------------- | | Service logs | Container logs for every Infrahub service, all replicas, plus previous-container logs for restarted pods | | Database logs | Neo4j server logs (`neo4j.log`, `debug.log`); full log directory with `--include-queries` | | Message-queue status | RabbitMQ queues, exchanges, bindings, connections, channels, and status | | Cache status | Redis info, client list, configuration, slow log, and database size | | Task-worker state | Prefect worker status, one directory per replica | | Task-manager state | Work pools, work queues, recent flow runs, events, and automations | | Server info | Versions, installed packages, API information/configuration/schema, masked environment variables | | Container metrics | Resource usage via `docker compose stats` or `kubectl top` | | Backup (opt-in) | Backup produced by the standard backup behavior (`--include-backup`) | | Benchmark (opt-in) | OpsMill benchmark results (`--benchmark`) | ### Environment and utility commands[​](#environment-and-utility-commands "Direct link to Environment and utility commands") `infrahub-collect` provides the same `environment detect`, `environment list`, `version`, and `update` commands as `infrahub-backup`, with identical behavior. See [`update`](#update) for the self-update flags and refusal cases. ## Configuration precedence[​](#configuration-precedence "Direct link to Configuration precedence") Configuration values are resolved in this order: 1. Command-line flags (highest priority) 2. Environment variables 3. Default values (lowest priority) ## Related documentation[​](#related-documentation "Direct link to Related documentation") * [Get started tutorial](/backup/tutorials/getting-started.md) * [Configuration reference](/backup/reference/configuration.md) * [Back up your Infrahub instance](/backup/backup/create.md) --- # Configuration reference Complete reference for configuring the Infrahub Ops CLI tools through environment variables and runtime flags. ## Configuration methods[​](#configuration-methods "Direct link to Configuration methods") Tools can be configured through these methods, applied in precedence order: 1. **Command-line flags** (highest priority) 2. **Environment variables** 3. **Default values** (lowest priority) ## Environment variables[​](#environment-variables "Direct link to Environment variables") ### Core configuration[​](#core-configuration "Direct link to Core configuration") | Variable | Description | Default | Example | | --------------------- | --------------------------------------------- | -------------------- | --------------- | | `INFRAHUB_BACKUP_DIR` | Directory for storing backup files | `./infrahub_backups` | `/data/backups` | | `INFRAHUB_OUTPUT_DIR` | Directory for storing troubleshooting bundles | `./infrahub_bundles` | `/data/bundles` | | `INFRAHUB_LOG_FORMAT` | Output format for logs | `text` | `json` | ### Retention configuration[​](#retention-configuration "Direct link to Retention configuration") | Variable | Description | Default | Example | | -------------------------- | --------------------------------------------------------- | ------- | ------- | | `INFRAHUB_RETENTION_DAYS` | Keep backups newer than N days (N >= 1, unset to disable) | unset | `30` | | `INFRAHUB_RETENTION_COUNT` | Keep the N most recent backups (N >= 1, unset to disable) | unset | `60` | Both rules combine as a union, and the most recent backup at each location always survives. A value that is not a whole number of at least 1 — including an explicit `0` — is a validation error that aborts the run rather than disabling the rule; a variable that is present but empty counts as unset. See [Manage backup retention](/backup/backup/retention.md). ### Docker compose configuration[​](#docker-compose-configuration "Direct link to Docker compose configuration") | Variable | Description | Default | Example | | ------------------ | --------------------------- | ----------- | --------------- | | `INFRAHUB_PROJECT` | Docker Compose project name | Auto-detect | `infrahub-prod` | ### Database configuration[​](#database-configuration "Direct link to Database configuration") #### Neo4j[​](#neo4j "Direct link to Neo4j") | Variable | Description | Default | Example | | ---------------------- | ------------------- | ------- | --------------- | | `INFRAHUB_DB_DATABASE` | Neo4j database name | `neo4j` | `infrahub` | | `INFRAHUB_DB_USERNAME` | Neo4j username | `neo4j` | `admin` | | `INFRAHUB_DB_PASSWORD` | Neo4j password | `admin` | `SecurePass123` | #### Task manager PostgreSQL[​](#task-manager-postgresql "Direct link to Task manager PostgreSQL") | Variable | Description | Default | Example | | ------------------------------------- | ---------------------------- | ----------- | ------------------------------------------ | | `PREFECT_API_DATABASE_CONNECTION_URL` | PostgreSQL connection string | Auto-detect | `postgresql://user:pass@localhost/prefect` | ### Collect configuration[​](#collect-configuration "Direct link to Collect configuration") | Variable | Description | Default | Example | | -------------------------- | ---------------------------------------------------- | ----------------------------------- | ------------------------------------ | | `INFRAHUB_BENCHMARK_IMAGE` | Container image used by the opt-in `--benchmark` run | `registry.opsmill.io/opsmill/bench` | `registry.example.com/opsmill/bench` | ## Command-line flag reference[​](#command-line-flag-reference "Direct link to Command-line flag reference") ### Global flags[​](#global-flags "Direct link to Global flags") | Flag | Environment Override | Description | | -------------- | --------------------- | -------------------------------------- | | `--backup-dir` | `INFRAHUB_BACKUP_DIR` | Set backup directory | | `--project` | `INFRAHUB_PROJECT` | Target specific Docker Compose project | | `--log-format` | `INFRAHUB_LOG_FORMAT` | Set log output format | ### Backup command flags[​](#backup-command-flags "Direct link to Backup command flags") | Flag | Description | | ------------------- | ------------------------------------------------------------- | | `--force` | Force backup creation even if tasks are running | | `--neo4jmetadata` | Neo4j metadata to include (all, none, users, roles) | | `--retention-days` | Prune backups older than N days after a successful backup | | `--retention-count` | Keep only the N most recent backups after a successful backup | ### Collect command flags[​](#collect-command-flags "Direct link to Collect command flags") | Flag | Environment Override | Description | | ------------------- | -------------------------- | ------------------------------------------------------------ | | `--log-lines` | `INFRAHUB_LOG_LINES` | Maximum log lines collected per container (default `100000`) | | `--include-backup` | `INFRAHUB_INCLUDE_BACKUP` | Also create a backup during collection | | `--include-queries` | `INFRAHUB_INCLUDE_QUERIES` | Include database query logs in the bundle | | `--benchmark` | `INFRAHUB_BENCHMARK` | Run the OpsMill benchmark and include its results | ## Auto-detection behavior[​](#auto-detection-behavior "Direct link to Auto-detection behavior") ### Docker Compose project detection[​](#docker-compose-project-detection "Direct link to Docker Compose project detection") Order of detection: 1. `--project` flag 2. `INFRAHUB_PROJECT` environment variable 3. Search for running Infrahub containers Detection command: ``` docker compose ls --filter "name=*infrahub*" ``` ### Database credential detection[​](#database-credential-detection "Direct link to Database credential detection") For Docker Compose deployments: ``` # Neo4j credentials from environment docker compose exec database printenv NEO4J_AUTH # PostgreSQL credentials from environment docker compose exec task-manager-db printenv POSTGRES_PASSWORD ``` ## Troubleshooting configuration[​](#troubleshooting-configuration "Direct link to Troubleshooting configuration") ### Debug configuration loading[​](#debug-configuration-loading "Direct link to Debug configuration loading") View effective configuration: ``` infrahub-backup --help ``` ### Test configuration[​](#test-configuration "Direct link to Test configuration") Validate configuration without running operations: ``` infrahub-backup environment detect ``` ### Common issues[​](#common-issues "Direct link to Common issues") #### Cannot detect environment[​](#cannot-detect-environment "Direct link to Cannot detect environment") ``` # Check Docker docker compose ls docker ps # Check for Infrahub containers docker ps --filter "name=infrahub" ``` #### Authentication failures[​](#authentication-failures "Direct link to Authentication failures") ``` # Verify credentials are set env | grep INFRAHUB env | grep PREFECT # Test database connectivity docker compose exec database cypher-shell -u neo4j ``` ## Examples[​](#examples "Direct link to Examples") ### Minimal configuration[​](#minimal-configuration "Direct link to Minimal configuration") For basic Docker Compose setup: ``` export INFRAHUB_BACKUP_DIR=/backups infrahub-backup create ``` ### Production configuration[​](#production-configuration "Direct link to Production configuration") ``` #!/bin/bash # production-backup.sh export INFRAHUB_LOG_FORMAT=json export INFRAHUB_PROJECT=infrahub-production export INFRAHUB_BACKUP_DIR=/data/backups/infrahub export INFRAHUB_DB_PASSWORD="${NEO4J_PASSWORD}" infrahub-backup create --neo4jmetadata=all ``` ## Related documentation[​](#related-documentation "Direct link to Related documentation") * [CLI command reference](/backup/reference/commands.md) * [Install Infrahub Backup](/backup/backup/install.md) * [Install Infrahub Collect](/backup/collect/install.md) --- # Update the CLI Update the `infrahub-backup` and `infrahub-collect` binaries to a newer release with the built-in `update` command. The command downloads the matching release, verifies its checksum, and replaces the binary in place. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * A binary you installed by direct download (not via Homebrew or a container image). * Network access to GitHub. * Permission to write the binary's location (use `sudo` if it lives in a system path such as `/usr/local/bin`). The same `update` command is available on both `infrahub-backup` and `infrahub-collect`; each updates the binary you invoke. The examples below use `infrahub-backup`. ## Check for an update[​](#check-for-an-update "Direct link to Check for an update") To see whether a newer version is available without changing anything on disk: ``` infrahub-backup update --check ``` You will see one of: ``` update available: v1.7.3 → v1.8.0 (https://github.com/opsmill/infrahub-backup/releases/tag/v1.8.0) # or already up to date (v1.8.0) ``` ## Update to the latest version[​](#update-to-the-latest-version "Direct link to Update to the latest version") ``` infrahub-backup update ``` The command shows the current and target versions and asks for confirmation before replacing the binary. On success it prints: ``` updated v1.7.3 → v1.8.0 ``` The download is verified against the release's published SHA-256 checksum and the binary is replaced atomically: if anything fails, your existing working binary is left untouched. ## Update without prompting (CI and scripts)[​](#update-without-prompting-ci-and-scripts "Direct link to Update without prompting (CI and scripts)") For unattended use, skip the confirmation prompt with `--yes`: ``` infrahub-backup update --yes ``` note In a non-interactive session (no terminal attached), the command refuses to run without `--yes` so it never blocks waiting for input. ## Install a specific version[​](#install-a-specific-version "Direct link to Install a specific version") To pin to or roll back to a specific release: ``` infrahub-backup update --version v1.7.2 --yes ``` ## Avoid GitHub rate limits[​](#avoid-github-rate-limits "Direct link to Avoid GitHub rate limits") The command uses the public GitHub API, which is rate-limited for unauthenticated requests (60 per hour per IP). On shared CI runners or over SSH you can raise the limit by exporting a token: ``` export GITHUB_TOKEN=ghp_... # or GH_TOKEN infrahub-backup update --yes ``` A token is never required for normal interactive use. ## When self-update is declined[​](#when-self-update-is-declined "Direct link to When self-update is declined") The command refuses to replace the binary in cases where doing so would be incorrect, and points you to the right path instead: | Message | What to do instead | | -------------------------- | --------------------------------------------------------- | | managed by Homebrew | `brew upgrade infrahub-backup` | | running inside a container | Pull a newer image tag | | development build | Install a released binary | | permission denied | Re-run with `sudo`, or as a user who can write the binary | note On Windows, a freshly downloaded binary may be flagged by Windows Defender. If this happens, allow the file in Defender and re-run the command. ## Validation[​](#validation "Direct link to Validation") Confirm the new version is in place: ``` infrahub-backup version ``` ## Related resources[​](#related-resources "Direct link to Related resources") * [Install Infrahub Backup](/backup/backup/install.md) * [CLI command reference](/backup/reference/commands.md) --- # Get started with Infrahub Backup By the end of this tutorial you will have installed `infrahub-backup`, detected your deployment environment, created and verified your first backup — including a snapshot of artifact storage — and restored from it. That is the core maintenance workflow for an Infrahub instance on Docker Compose. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this tutorial, ensure you have: * A running Infrahub instance (using Docker Compose) * Docker installed and running on your system * Terminal access with sudo privileges * At least 1GB of free disk space for backups ## Step 1: install infrahub-backup[​](#step-1-install-infrahub-backup "Direct link to Step 1: install infrahub-backup") First, install the infrahub-backup tool. The recommended method is to download binaries: ``` curl https://infrahub.opsmill.io/ops/$(uname -s)/$(uname -m)/infrahub-backup -o infrahub-backup chmod +x infrahub-backup # Verify installation ./infrahub-backup version ``` You should see output similar to: ``` Version: 1.0.0 ``` ## Step 2: detect your environment[​](#step-2-detect-your-environment "Direct link to Step 2: detect your environment") Before performing any operations, verify that infrahub-backup can detect your Infrahub deployment: ``` ./infrahub-backup environment detect ``` Expected output: ``` INFO[0000] Detecting deployment environment... INFO[0000] Docker environment detected INFO[0000] Found Docker Compose project: infrahub ``` If you have multiple Infrahub projects, list them all: ``` ./infrahub-backup environment list ``` This shows all available Docker Compose projects running Infrahub. ## Step 3: create your first backup[​](#step-3-create-your-first-backup "Direct link to Step 3: create your first backup") Now, let's create a backup of your Infrahub instance. This operation will: 1. Check for running tasks 2. Backup the Neo4j database 3. Backup the PostgreSQL task manager database 4. Create a compressed archive Run the backup command: ``` ./infrahub-backup create ``` You'll see detailed progress output: ``` INFO[0000] Starting backup process... INFO[0000] Checking for running tasks... INFO[0001] No running tasks found INFO[0004] Backing up Neo4j database... INFO[0010] Neo4j backup completed INFO[0010] Backing up PostgreSQL database... INFO[0012] PostgreSQL backup completed INFO[0012] Creating backup archive... INFO[0015] Backup created successfully: infrahub_backups/infrahub_backup_20250929_143022.tar.gz INFO[0015] Starting application containers... INFO[0020] Backup process completed ``` note The backup file is stored in the `infrahub_backups` directory by default. Note the filename for the restore step. ## Step 4: verify your backup[​](#step-4-verify-your-backup "Direct link to Step 4: verify your backup") Check that your backup was created successfully: ``` # List backup files ls -lh infrahub_backups/ # View backup contents (without extracting) tar -tzf infrahub_backups/infrahub_backup_*.tar.gz | head -10 ``` You should see: * A `.tar.gz` file with today's timestamp * Inside the archive: `backup_information.json`, database files, and other components ## Step 5: capture artifact storage[​](#step-5-capture-artifact-storage "Direct link to Step 5: capture artifact storage") If your Infrahub workflows produce artifacts stored outside the databases, take a moment to snapshot that storage so the backup remains consistent. Identify the bucket, container, or volume where artifacts live and copy the contents into a dated folder or version. Use your provider’s native tooling (for example, `aws s3 sync`, `az storage blob download`, or `rsync` for shared volumes) and note where you stored the snapshot so you can pair it with the database backup when restoring. If using standard Docker Compose deployment: ``` # Copy artifacts directory to backup location docker compose cp -a infrahub-server:/opt/infrahub/storage /backup/artifacts/ ``` ## Step 6: restore from backup[​](#step-6-restore-from-backup "Direct link to Step 6: restore from backup") To complete the learning cycle, let's restore from the backup you created. This simulates recovering from a disaster or rolling back changes. warning Restoring will replace all current data in your Infrahub instance. In a production environment, always verify you have a current backup before restoring an older one. ``` # Replace with your actual backup filename ./infrahub-backup restore infrahub_backups/infrahub_backup_20250929_143022.tar.gz ``` The restore process will: 1. Validate the backup file 2. Stop all containers 3. Restore the Neo4j database 4. Restore the PostgreSQL database 5. Restart all services Expected output: ``` INFO[0000] Starting restore process... INFO[0000] Validating backup file... INFO[0001] Backup validation successful INFO[0001] Stopping all containers... INFO[0005] Restoring Neo4j database... INFO[0015] Neo4j restore completed INFO[0015] Restoring PostgreSQL database... INFO[0018] PostgreSQL restore completed INFO[0018] Starting all containers... INFO[0025] Restore completed successfully ``` ## Step 7: verify the restore[​](#step-7-verify-the-restore "Direct link to Step 7: verify the restore") Confirm your Infrahub instance is running correctly: ``` # Check container status docker compose ps # Test Infrahub connectivity (adjust URL as needed) curl -I http://localhost:8000 ``` All containers should be in the "Up" state, and the HTTP request should return a successful response. ## What you've learned[​](#what-youve-learned "Direct link to What you've learned") You've completed a full backup and restore cycle. You now know how to: * Install and verify infrahub-backup * Detect your deployment environment * Create comprehensive backups of your Infrahub instance * Capture artifact storage snapshots to keep backups complete * Restore from a backup file * Verify successful operations ## Next steps[​](#next-steps "Direct link to Next steps") Now that you understand the basics, explore: * [Back up your Infrahub instance](/backup/backup/create.md) * [Restore from a backup](/backup/backup/restore.md) ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") If you encounter issues: **Permission denied when running commands:** * Ensure your user has Docker permissions: `sudo usermod -aG docker $USER` * Log out and back in for changes to take effect **Cannot detect environment:** * Verify Docker is running: `docker ps` * Check Docker Compose is installed: `docker compose version` **Backup fails with "running tasks detected":** * Wait for tasks to complete, or use `--force` flag to proceed anyway * Check task status in Infrahub UI --- # Infrahub demo-dc example Welcome to the Infrahub demo-dc example. This repository showcases how Infrahub serves as an infrastructure data management platform for managing modern network infrastructure with design-driven automation. It demonstrates Infrahub's core capabilities including: * Schema-driven data modeling * Composable topology generation * Version control for infrastructure data * Automated configuration generation * Validation workflows Whether you're a network engineer exploring automation, a developer building on Infrahub, or an architect evaluating infrastructure management platforms, this bundle provides hands-on experience with real-world patterns and workflows. ## Community contribution[​](#community-contribution "Direct link to Community contribution") note This demo repository is partially authored by the OpsMill community member [tomek](https://www.linkedin.com/in/tomekzajac/) from this example: [GitHub t0m3kz/infrahub-demo](https://github.com/t0m3kz/infrahub-demo) ## Documentation guide[​](#documentation-guide "Direct link to Documentation guide") This documentation is organized following the [Diataxis framework](https://diataxis.fr/) to help you find exactly what you need: ### Getting started[​](#getting-started "Direct link to Getting started") | Page | Purpose | | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | **[Installation guide](/demo-dc/install.md)** | Step-by-step instructions to install and set up the demo environment on your system. Start here if this is your first time running the demo. | ### Tutorials[​](#tutorials "Direct link to Tutorials") | Page | Purpose | | --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **[User walkthrough](/demo-dc/user-walkthrough.md)** | A complete hands-on tutorial that guides you through the end-user experience: creating topologies, managing branches, running generators, creating proposed changes, and validating configurations. Perfect for learning the workflow from start to finish. | | **[Deploy a virtual lab with Containerlab](/demo-dc/containerlab-deployment.md)** | Learn how to deploy generated configurations to a virtual network lab using Containerlab. Extract device configurations and topology files from Infrahub, spin up virtual Arista cEOS switches, and test your data center fabric before production deployment. | | **[Working with security management](/demo-dc/security-management.md)** | Explore Infrahub's security management capabilities by examining firewall policies, security zones, and address objects. Learn how structured security data transforms into vendor-specific firewall configurations (Juniper JunOS) and how to modify policies safely using branches. | | **[Virtual machine management](/demo-dc/virtualization.md)** | Load a virtualization layer on top of the data center fabric: hypervisor clusters, physical hosts, and virtual machines. See how generators place hosts in racks, cable them to leaf switches, assign VM addresses, and enforce an HTTPS-only firewall policy automatically. | | **[Cloud resource management](/demo-dc/cloud-management.md)** | Manage multi-cloud infrastructure (AWS, GCP, Azure) with a vendor-agnostic schema. Load demo cloud data including accounts, regions, virtual networks, instances, and security groups. Learn how Infrahub serves as a unified inventory for cloud resources. | | **[Using the service catalog](/demo-dc/service-catalog.md)** | Learn how to use the Service Catalog web interface for simplified infrastructure provisioning. Enable the Streamlit application, navigate between branches, view existing infrastructure, and create new data centers through a guided form-based workflow that automates branch creation and generator execution. | ### Guides[​](#guides "Direct link to Guides") | Page | Purpose | | ------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **[Developer guide](/demo-dc/developer-guide.md)** | Technical deep-dive into how the demo works under the hood. Covers schema architecture, bootstrap scripts, generators, transforms, checks, and testing. Use this when extending functionality, troubleshooting issues, or understanding implementation details. | | **[Using Infrahub Enterprise](/demo-dc/enterprise.md)** | Instructions for configuring the demo environment to use Infrahub Enterprise edition. Learn how to switch between Community and Enterprise editions, configure environment variables, and understand the differences between editions. | ### Topics[​](#topics "Direct link to Topics") | Page | Purpose | | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **[Understanding the concepts](/demo-dc/concepts.md)** | Explains the architectural patterns, design decisions, and core Infrahub concepts demonstrated in this project. Read this to understand the "why" behind design-driven automation, composable topologies, generators, and integration patterns. | ## Quick start[​](#quick-start "Direct link to Quick start") If you're ready to dive in: 1. Follow the **[installation guide](/demo-dc/install.md)** to set up your environment 2. Walk through the **[user tutorial](/demo-dc/user-walkthrough.md)** to create your first data center topology 3. Deploy to a **[virtual lab with Containerlab](/demo-dc/containerlab-deployment.md)** to test configurations 4. Try the **[Service Catalog](/demo-dc/service-catalog.md)** for simplified infrastructure provisioning 5. Explore the **[concepts](/demo-dc/concepts.md)** to deepen your understanding 6. Reference the **[developer guide](/demo-dc/developer-guide.md)** when you're ready to extend or customize ## What you'll learn[​](#what-youll-learn "Direct link to What you'll learn") Through this demo, you'll gain practical experience with: * **Schema-driven infrastructure modeling** - Defining devices, networks, and services with flexible, extensible schemas * **Design-driven automation** - Creating abstract topology designs that generators transform into concrete infrastructure * **Composable topologies** - Building complex data center and POP architectures from reusable components * **Version control for data** - Using branches and proposed changes to safely modify infrastructure * **Automated resource allocation** - Generating IP prefixes, VLANs, and addressing from resource pools * **Configuration generation** - Transforming structured data into device configurations via templates * **Data validation** - Enforcing consistency with custom checks * **GitOps patterns** - Managing infrastructure as code with full versioning and review workflows ## Architecture at a glance[​](#architecture-at-a-glance "Direct link to Architecture at a glance") The demo implements realistic network topologies including: * **Data center fabrics** with spine-leaf architecture and VxLAN/EVPN overlay * **Point of presence (POP) networks** with edge routers and peering connections * **Network segments** with load balancers and service endpoints * **Security zones and policies** with firewall rules and access control * **Virtualization layer** with hypervisor clusters, hosts cabled into the fabric, and policy-protected VMs * **Cloud infrastructure** with vendor-agnostic modeling for AWS, GCP, and Azure * **Resource pools** for IP address, VLAN, and ASN allocation * **Multi-vendor support** (Arista, Juniper, Cisco, and SONiC templates) * **Automated topology generation** from abstract design definitions ## Key features demonstrated[​](#key-features-demonstrated "Direct link to Key features demonstrated") ### Schema extensibility[​](#schema-extensibility "Direct link to Schema extensibility") The demo includes comprehensive schemas for DCIM, IPAM, topology, routing, security, and load balancing. These schemas define not just object types, but relationships, constraints, and lifecycle behaviors. ### Generators and automation[​](#generators-and-automation "Direct link to Generators and automation") When you create a topology design (for example, DC-3), generators automatically create all supporting objects including devices, interfaces, IP addresses, routing protocols, and BGP peer groups following best practices and business rules. ### Branch-based workflows[​](#branch-based-workflows "Direct link to Branch-based workflows") All changes happen in branches. Proposed changes provide diff views, run validation checks, and regenerate artifacts before merging to main. ### Artifact generation[​](#artifact-generation "Direct link to Artifact generation") Templates transform Infrahub data into deployable artifacts including device configurations, topology diagrams, and cabling matrices. ## Community and support[​](#community-and-support "Direct link to Community and support") * **Source code**: [GitHub repository](https://github.com/opsmill/infrahub-demo-dc) * **Infrahub documentation**: [docs.infrahub.app](https://docs.infrahub.app) * **Discord community**: [Discord](https://discord.gg/opsmill) * **OpsMill website**: [opsmill.com](https://opsmill.com) ## Next steps[​](#next-steps "Direct link to Next steps") Ready to get started? Head to the **[installation guide](/demo-dc/install.md)** to set up your environment. --- # Cloud resource management This tutorial shows an example of cloud resource management schema, which provides a vendor-agnostic way to model cloud infrastructure across AWS, GCP, and Azure. You'll load sample cloud data and explore how Infrahub can serve as a unified inventory for multi-cloud environments. ## Overview[​](#overview "Direct link to Overview") The cloud schema enables you to track: * **Cloud providers** - AWS, GCP, Azure, or other cloud platforms * **Cloud accounts** - AWS accounts, GCP projects, Azure subscriptions * **Regions and availability zones** - Geographic locations and fault domains * **Virtual networks** - VPCs (AWS), VPC networks (GCP), VNets (Azure) * **Subnets** - Network segments within virtual networks * **Security groups** - Network access control rules (Security Groups, Firewall Rules, NSGs) * **Compute instances** - Virtual machines across all providers * **Network infrastructure** - Internet gateways, NAT gateways, route tables, elastic IPs * **Network interfaces** - ENIs, NICs attached to instances This vendor-agnostic model allows you to manage multi-cloud infrastructure from a single source of truth, with consistent naming and relationships regardless of the underlying cloud provider. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this tutorial, ensure you have: * Completed the [installation guide](/demo-dc/install.md) and have Infrahub running * Loaded the bootstrap data and schemas * Access to the Infrahub web interface at `http://localhost:8000` ## Loading cloud demo data[​](#loading-cloud-demo-data "Direct link to Loading cloud demo data") The demo includes sample data for all three major cloud providers with realistic infrastructure examples. The easiest way to load the cloud demo is using the provided invoke task: ``` uv run invoke demo-cloud ``` This command: 1. Creates a new branch named `demo-cloud` 2. Loads all schemas (including the cloud schema) 3. Loads cloud object files with sample data for AWS, GCP, and Azure 4. Displays a URL to view the cloud resources ## Exploring cloud resources[​](#exploring-cloud-resources "Direct link to Exploring cloud resources") After loading the demo data, navigate to the cloud resources in the Infrahub web interface. ### Viewing all cloud resources[​](#viewing-all-cloud-resources "Direct link to Viewing all cloud resources") 1. Ensure you're on the correct branch (for example, `demo-cloud`) 2. Navigate to **Cloud Resource** in the left sidebar menu 3. You'll see a list of all cloud resource types Or access the cloud resources directly: ``` http://localhost:8000/objects/CloudResource?branch=demo-cloud ``` ### Sample data structure[​](#sample-data-structure "Direct link to Sample data structure") The demo includes a comprehensive multi-cloud environment: #### Cloud providers (3)[​](#cloud-providers-3 "Direct link to Cloud providers (3)") * Amazon Web Services (AWS) * Google Cloud Platform (GCP) * Microsoft Azure #### Cloud accounts (12)[​](#cloud-accounts-12 "Direct link to Cloud accounts (12)") Each provider has production, staging, and development accounts: * `opsmill-aws-production`, `opsmill-aws-staging`, `opsmill-aws-dev` * `opsmill-gcp-production`, `opsmill-gcp-staging`, `opsmill-gcp-dev` * `opsmill-azure-production`, `opsmill-azure-staging`, `opsmill-azure-dev` #### Regions and availability zones[​](#regions-and-availability-zones "Direct link to Regions and availability zones") * **AWS**: US East (N. Virginia), US West (Oregon), EU West (Ireland) * **GCP**: US Central (Iowa), US East (South Carolina), Europe West (Belgium) * **Azure**: East US, West US 2, West Europe Each region includes 3 availability zones. #### Virtual networks (12)[​](#virtual-networks-12 "Direct link to Virtual networks (12)") VPCs and VNets across all accounts with various configurations: * Production VPCs with public and private subnets * Staging and development networks * DNS support and hostname configuration #### Compute instances (19)[​](#compute-instances-19 "Direct link to Compute instances (19)") Various instance types across all providers: * Web servers, application servers, database servers * Linux and Windows instances * Different instance sizes (t3.large, m5.xlarge, n1-standard-2, Standard\_D2s\_v3, etc.) #### Network infrastructure[​](#network-infrastructure "Direct link to Network infrastructure") * Internet gateways for public connectivity * NAT gateways for private subnet outbound access * Route tables for traffic routing * Elastic/static IP addresses * Network interfaces with security group associations ## Schema architecture[​](#schema-architecture "Direct link to Schema architecture") The cloud schema uses a hierarchical structure with clear relationships: ``` CloudProvider └── CloudAccount └── CloudVirtualNetwork ├── CloudSubnet ├── CloudSecurityGroup ├── CloudInternetGateway └── CloudRouteTable CloudRegion └── CloudAvailabilityZone └── CloudInstance └── CloudNetworkInterface ``` ### Key relationships[​](#key-relationships "Direct link to Key relationships") * **CloudAccount** belongs to a **CloudProvider** (parent relationship) * **CloudRegion** is associated with a **CloudProvider** * **CloudAvailabilityZone** belongs to a **CloudRegion** (parent relationship) * **CloudVirtualNetwork** is associated with a **CloudAccount** and **CloudRegion** * **CloudSubnet** belongs to a **CloudVirtualNetwork** (parent relationship) * **CloudInstance** is associated with a **CloudAccount**, **CloudAvailabilityZone**, and **CloudSubnet** * **CloudSecurityGroup** can be attached to **CloudInstance** and **CloudNetworkInterface** ### Common attributes[​](#common-attributes "Direct link to Common attributes") All cloud resources inherit from the `CloudResource` generic, providing: * `name` - Resource name * `description` - Optional description * `cloud_id` - Provider-specific resource identifier (ARN, resource ID, etc.) * `status` - Operational status (active, stopped, provisioning, terminating, error) * `tags` - Optional tags for categorization ## Use cases[​](#use-cases "Direct link to Use cases") ### Multi-cloud inventory[​](#multi-cloud-inventory "Direct link to Multi-cloud inventory") Use Infrahub as a single source of truth for all cloud resources: * Track resources across AWS, GCP, and Azure in one place * Maintain consistent naming conventions * Link cloud resources to on-premises infrastructure ### Security auditing[​](#security-auditing "Direct link to Security auditing") Query security groups and their associations: * Identify instances with specific security group configurations * Audit network access rules across all clouds * Track public IP assignments ### Capacity planning[​](#capacity-planning "Direct link to Capacity planning") Analyze compute resources across your cloud footprint: * Count instances by type, region, or provider * Track resource utilization patterns * Plan for growth and optimization ### Network documentation[​](#network-documentation "Direct link to Network documentation") Document your cloud network architecture: * Map virtual networks, subnets, and routing * Track NAT and internet gateway configurations * Document network interface assignments ## Next steps[​](#next-steps "Direct link to Next steps") For more information on Infrahub concepts, see: * **[Understanding the concepts](/demo-dc/concepts.md)** - Core Infrahub patterns * **[Developer guide](/demo-dc/developer-guide.md)** - Extending schemas and creating transforms --- # Understanding the concepts This document explains the architectural patterns, design decisions, and core Infrahub concepts demonstrated in this demo. Read this to understand the "why" behind design-driven automation, composable topologies, generators, and the overall approach to infrastructure management. ## Design-driven automation[​](#design-driven-automation "Direct link to Design-driven automation") Traditional infrastructure automation often works at the device configuration level. You define configurations for individual devices, then deploy them. This approach has limitations when managing large-scale infrastructure. ### The traditional approach[​](#the-traditional-approach "Direct link to The traditional approach") In traditional automation: 1. You manually define each device and its configuration 2. Configurations are tightly coupled to specific devices 3. Scaling requires duplicating configuration patterns 4. Changing architecture means updating many individual configs 5. It's difficult to maintain consistency across the infrastructure ### The design-driven approach[​](#the-design-driven-approach "Direct link to The design-driven approach") Design-driven automation flips this model. Instead of defining devices, you define **intent** at a higher abstraction level: 1. You specify **what** you want (for example, "a 4-spine, 8-leaf data center") 2. Generators translate high-level designs into concrete infrastructure 3. Templates produce device-specific configurations from structured data 4. Scaling means adjusting design parameters, not editing configs 5. Architectural changes propagate automatically The demo implements this pattern. When you load `objects/dc/dc-arista-s.yml`, you're not defining 20 devices. You're defining a **design specification** that a generator transforms into complete infrastructure. ### Benefits[​](#benefits "Direct link to Benefits") * **Abstraction** - Work at the topology level, not device level * **Consistency** - Generators enforce best practices and standards * **Scalability** - Grow from 10 to 1000 devices by adjusting parameters * **Maintainability** - Change design templates, not individual configs * **Reusability** - Same generator can create DC-2, DC-3, DC-4 with different inputs ## Composable topologies[​](#composable-topologies "Direct link to Composable topologies") The demo uses composable building blocks to construct complex network architectures. ### Composition model[​](#composition-model "Direct link to Composition model") ``` Topology Design ├─ Device Templates (spine, leaf, border-leaf) ├─ Connectivity Patterns (fabric peering) ├─ Address Pools (management, loopback, fabric) ├─ Routing Design (eBGP, OSPF) └─ Service Overlays (EVPN, L2VPN, L3VPN) ``` Each layer builds on the previous one: 1. **Device templates** define interface layouts for each device role 2. **Connectivity patterns** define how devices interconnect 3. **Address pools** provide IP address and VLAN allocation 4. **Routing design** specifies underlay and overlay protocols 5. **Service overlays** enable tenant networks and services ### Example: DC-3 composition[​](#example-dc-3-composition "Direct link to Example: DC-3 composition") When you create DC-3, the generator: 1. Selects spine, leaf, and border-leaf **templates** 2. Applies fabric **connectivity pattern** (full mesh spine-to-leaf) 3. Allocates addresses from **pools** (loopback /32s, fabric /31s) 4. Configures **routing** (eBGP underlay, iBGP EVPN overlay) 5. Enables **EVPN service** framework This composition allows you to: * Swap eBGP for OSPF underlay by changing one parameter * Scale from 2 spines to 4 spines by updating device count * Change addressing scheme by modifying pool definitions * Add new device roles without rewriting generators ### Reusability across topologies[​](#reusability-across-topologies "Direct link to Reusability across topologies") The same composable patterns work for different topology types: * **Data centers** use spine-leaf composition with EVPN overlay * **POPs** use edge router composition with BGP peering * **Segments** use service endpoint composition with load balancers This reusability reduces code duplication and makes patterns portable. ## Generators as transformation engines[​](#generators-as-transformation-engines "Direct link to Generators as transformation engines") Generators transform abstract designs into concrete infrastructure. They embody business rules, network design patterns, and operational best practices. ### Generator responsibility[​](#generator-responsibility "Direct link to Generator responsibility") A generator's job is to: 1. **Read design input** - Parse high-level topology specifications 2. **Apply business rules** - Enforce naming conventions, numbering schemes 3. **Create infrastructure** - Generate devices, interfaces, IP addresses 4. **Establish relationships** - Connect devices, assign addresses, configure protocols 5. **Allocate resources** - Draw from IP pools, VLAN pools, ASN pools ### Example: create\_dc generator[​](#example-create_dc-generator "Direct link to Example: create_dc generator") The DC generator implements a spine-leaf fabric pattern: ``` async def generate(self, data: dict) -> None: # Read design parameters spine_count = data["spine_count"] leaf_count = data["leaf_count"] underlay_protocol = data["underlay_protocol"] # Create resource pools await self.create_address_pools() # Create devices from templates await self.create_devices() # Full mesh fabric peering await self.create_fabric_peering() # Configure underlay routing if underlay_protocol == "ospf": await self.create_ospf_underlay() else: await self.create_ebgp_underlay() # Configure overlay routing await self.create_ibgp_overlay() ``` ### Benefits of generators[​](#benefits-of-generators "Direct link to Benefits of generators") * **Encapsulation** - Complex logic in one place * **Testability** - Unit test generator logic separately * **Consistency** - Same generator always produces same output for same input * **Evolvability** - Update generator to change all future topologies * **Auditability** - Generator code documents design decisions ## Resource pools and allocation[​](#resource-pools-and-allocation "Direct link to Resource pools and allocation") Resource pools provide centralized management of scarce resources like IP addresses, VLANs, and ASNs. ### Pool concept[​](#pool-concept "Direct link to Pool concept") A resource pool is a defined range of resources that Infrahub allocates from as needed: ``` - kind: IpamIPPrefix data: - prefix: "10.0.0.0/16" pool_type: "management" description: "Management network pool" ``` When a generator needs an IP address for a management interface, it allocates from this pool. Infrahub tracks allocations and prevents conflicts. ### Pool types in the demo[​](#pool-types-in-the-demo "Direct link to Pool types in the demo") * **Management pools** - Out-of-band management addresses * **Loopback pools** - Router IDs and VTEP addresses * **Fabric pools** - Point-to-point links between switches * **Service pools** - Tenant network addressing * **VLAN pools** - Layer 2 segment identifiers * **ASN pools** - BGP autonomous system numbers ### Benefits[​](#benefits-1 "Direct link to Benefits") * **Automatic allocation** - No manual IP planning * **Conflict prevention** - Infrahub ensures uniqueness * **Visibility** - See which addresses are allocated vs available * **Reclamation** - Deleting devices returns addresses to pool * **Hierarchical allocation** - Pools can allocate from parent pools ## Schema-driven data modeling[​](#schema-driven-data-modeling "Direct link to Schema-driven data modeling") Schemas define the structure of your infrastructure data. They specify object types, attributes, relationships, and constraints. ### Why schemas matter[​](#why-schemas-matter "Direct link to Why schemas matter") Schemas provide: 1. **Type safety** - Attributes have defined types (text, number, IP address) 2. **Validation** - Constraints enforce data integrity 3. **Relationships** - Formalize connections between objects 4. **Extensibility** - Inheritance allows customization without modification 5. **API generation** - Infrahub auto-generates GraphQL API from schemas 6. **Computed attributes** - Automatically derive values from other attributes 7. **Range expansion** - Define multiple items with compact notation ### Schema evolution[​](#schema-evolution "Direct link to Schema evolution") Schemas can evolve over time through: * **Attribute addition** - Add new fields to existing nodes * **Relationship addition** - Create new connections between nodes * **Inheritance** - Extend base nodes with specialized types * **Constraints** - Add uniqueness or validation rules When you extend schemas, existing data remains compatible. This allows iterative refinement. ### Schema organization[​](#schema-organization "Direct link to Schema organization") The demo organizes schemas by domain: * **Base schemas** - Core models used across all topologies * **Extension schemas** - Domain-specific models (routing, security, load balancing) * **Namespaces** - Group related nodes (Dcim, Ipam, Service, Security) This organization supports: * **Modularity** - Enable/disable domains as needed * **Clarity** - Related nodes grouped together * **Maintainability** - Domain experts can own their schemas ### Advanced schema features[​](#advanced-schema-features "Direct link to Advanced schema features") The demo showcases several advanced Infrahub schema capabilities: **Computed attributes** automatically generate values based on other attributes. For example, Autonomous System names are computed from ASN values (AS65000 from ASN 65000) using Jinja2 templates. This ensures naming consistency and eliminates manual data entry. [Learn more about computed attributes](https://docs.infrahub.app/topics/computed-attributes). **Interface range expansion** allows compact definition of multiple interfaces. Instead of defining 30 interfaces individually, you can use `Ethernet1/[1-30]` which automatically expands to `Ethernet1/1` through `Ethernet1/30`. This dramatically reduces YAML verbosity in device templates and topology definitions. [Learn more about range expansion](https://docs.infrahub.app/python-sdk/reference/templating#netutils-filters). ## Branch-based infrastructure changes[​](#branch-based-infrastructure-changes "Direct link to Branch-based infrastructure changes") Infrahub uses Git-like branching for infrastructure data. This enables safe, reviewable changes with rollback capability. ### Branch workflow[​](#branch-workflow "Direct link to Branch workflow") 1. **Create branch** - Isolate changes from main branch 2. **Make modifications** - Add devices, change configs, update relationships 3. **Validate changes** - Run checks and generate artifacts 4. **Review changes** - Examine diffs and validation results 5. **Merge or discard** - Apply changes to main or abandon the branch ### Why branches matter[​](#why-branches-matter "Direct link to Why branches matter") Without branches, every change immediately affects the production infrastructure. This makes it risky to experiment or test ideas. Branches provide: * **Safety** - Test changes without affecting main * **Collaboration** - Multiple teams can work on different branches * **Review** - Examine changes before they take effect * **Rollback** - Discard unsuccessful experiments * **History** - Track what changed and when ### Proposed changes[​](#proposed-changes "Direct link to Proposed changes") Proposed changes are Infrahub's equivalent of pull requests. They: 1. Show diffs between source and destination branches 2. Run validation checks automatically 3. Regenerate affected artifacts 4. Provide a review interface for approvals 5. Merge atomically when approved This workflow brings software development practices (code review, CI/CD, version control) to infrastructure management. ## Artifacts and configuration generation[​](#artifacts-and-configuration-generation "Direct link to Artifacts and configuration generation") Artifacts are the final outputs that deploy to devices. The demo generates configurations, topology files, and documentation. ### Artifact generation flow[​](#artifact-generation-flow "Direct link to Artifact generation flow") ``` GraphQL Query → Transform → Template → Artifact ``` 1. **Query** - Fetch device data from Infrahub 2. **Transform** - Process data into template-friendly structure 3. **Template** - Render Jinja2 template with processed data 4. **Artifact** - Store generated configuration in Infrahub ### Why separate transforms and templates[​](#why-separate-transforms-and-templates "Direct link to Why separate transforms and templates") Separating Python transforms from Jinja2 templates provides: * **Reusability** - Same template works with different transforms * **Testability** - Test transform logic and template rendering independently * **Maintainability** - Network engineers can edit templates without Python knowledge * **Flexibility** - Swap templates for different vendor platforms ### Artifact regeneration[​](#artifact-regeneration "Direct link to Artifact regeneration") Artifacts automatically regenerate when: * Data changes (device attributes, relationships) * Templates change (update Jinja2 files) * Transforms change (update Python logic) * Proposed changes are created (for review) This ensures artifacts always reflect current state. ## Validation and checks[​](#validation-and-checks "Direct link to Validation and checks") Checks validate infrastructure before deployment. They catch errors, enforce policies, and ensure consistency. ### Check types[​](#check-types "Direct link to Check types") * **Configuration validation** - Verify device configs are valid * **Connectivity validation** - Ensure devices are properly connected * **Policy enforcement** - Check against organizational standards * **Best practice validation** - Verify design patterns are followed ### When checks run[​](#when-checks-run "Direct link to When checks run") Checks execute: 1. **On proposed change creation** - Before merge 2. **On branch commits** - During development 3. **On demand** - Manual execution 4. **On schedule** - Periodic validation (if configured) ### Check outcomes[​](#check-outcomes "Direct link to Check outcomes") Checks produce: * **Errors** - Must be fixed before merge * **Warnings** - Should be reviewed but don't block merge * **Info** - Informational messages for context This tiered approach balances strictness with flexibility. ## Integration patterns[​](#integration-patterns "Direct link to Integration patterns") The demo demonstrates integration with external tools and systems. ### Repository integration[​](#repository-integration "Direct link to Repository integration") Infrahub can sync with Git repositories to: * Load generators, transforms, and checks from code repos * Version control infrastructure-as-code components * Enable GitOps workflows * Collaborate using standard Git workflows ### CI/CD integration[​](#cicd-integration "Direct link to CI/CD integration") The demo includes GitHub Actions for: * Linting and type checking * Running unit tests * Running integration tests * Validating schemas and data This ensures code quality and catches issues before deployment. ### Containerlab integration[​](#containerlab-integration "Direct link to Containerlab integration") The demo can generate Containerlab topologies for: * Virtual lab deployment * Testing configurations in simulation * Training and demonstrations * Development and testing This provides a complete development-to-production workflow. ## Patterns for scale[​](#patterns-for-scale "Direct link to Patterns for scale") The demo implements patterns that scale from small to large deployments. ### Batching[​](#batching "Direct link to Batching") Generators use batching to efficiently create many objects: ``` batch = await client.create_batch() for device in devices: batch.add(kind="DcimDevice", data=device) await batch.execute() ``` Batching reduces API calls and improves performance. ### Async operations[​](#async-operations "Direct link to Async operations") All SDK operations use async/await for concurrency: ``` # Sequential (slow) device1 = await create_device("spine1") device2 = await create_device("spine2") # Concurrent (fast) results = await asyncio.gather( create_device("spine1"), create_device("spine2") ) ``` This allows generators to create infrastructure in parallel. ### Hierarchical allocation[​](#hierarchical-allocation "Direct link to Hierarchical allocation") Resource pools support hierarchy for delegation: ``` Root Pool (10.0.0.0/8) ├─ Region 1 (10.0.0.0/16) │ ├─ DC-1 (10.0.0.0/20) │ └─ DC-2 (10.0.16.0/20) └─ Region 2 (10.1.0.0/16) ``` This supports multi-region deployments with delegated address management. ## Mental models[​](#mental-models "Direct link to Mental models") ### Infrastructure as data[​](#infrastructure-as-data "Direct link to Infrastructure as data") Think of infrastructure not as static configurations, but as structured data that can be queried, transformed, and versioned like any other data. Traditional: "Here are 50 configuration files" Design-driven: "Here's data representing 50 devices, query what you need" ### Generators as factories[​](#generators-as-factories "Direct link to Generators as factories") Generators are factories that produce infrastructure from blueprints. The blueprint (design data) specifies what to build. The factory (generator) knows how to build it. Blueprint: "4 spines, 8 leaves, eBGP underlay" Factory: Creates 12 devices with 200+ interfaces and relationships ### Schemas as contracts[​](#schemas-as-contracts "Direct link to Schemas as contracts") Schemas are contracts between producers (generators, users) and consumers (transforms, checks). They define what data must be provided and what shape it takes. Contract: "A device must have a name, role, platform, and location" Producers: Ensure these fields are populated Consumers: Can rely on these fields existing ## Design philosophy[​](#design-philosophy "Direct link to Design philosophy") The demo embodies several design principles: ### Declarative over imperative[​](#declarative-over-imperative "Direct link to Declarative over imperative") Describe the desired state (declarative) rather than steps to achieve it (imperative). Imperative: "Create spine1, create eth1, assign IP 10.0.0.1, enable interface" Declarative: "Spine1 exists with eth1 having IP 10.0.0.1 in enabled state" ### Data-driven over code-driven[​](#data-driven-over-code-driven "Direct link to Data-driven over code-driven") Use data to drive behavior rather than hard-coding logic. Code-driven: `if dc == "DC-3": create_4_spines()` Data-driven: `for _ in range(design.spine_count): create_spine()` ### Separation of concerns[​](#separation-of-concerns "Direct link to Separation of concerns") Separate concerns into focused components: * Schemas define structure * Generators create objects * Transforms produce configs * Templates format output * Checks validate results Each component has a single responsibility. ### Convention over configuration[​](#convention-over-configuration "Direct link to Convention over configuration") Use sensible defaults and conventions to reduce configuration burden. Default: Spine interfaces are "Ethernet1-32" Convention: Spines connect to all leaves Configuration: Override when needed ## Next steps[​](#next-steps "Direct link to Next steps") Now that you understand the concepts, you can: * **Build new topologies** - Apply patterns to different use cases * **Extend generators** - Add new topology types or enhance existing ones * **Customize schemas** - Add organization-specific attributes * **Create custom checks** - Enforce your policies and standards For implementation details, see the [developer guide](/demo-dc/developer-guide.md). --- # Deploy a virtual lab with Containerlab This tutorial walks you through deploying a virtual data center topology using Containerlab. You'll take the configurations generated by Infrahub and spin up a fully functional network lab with Arista cEOS virtual switches. By the end of this tutorial, you'll have a running virtual data center that you can access, test, and validate before deploying to production hardware. ## What you'll accomplish[​](#what-youll-accomplish "Direct link to What you'll accomplish") In this tutorial, you will: 1. Generate a data center topology with Arista switches 2. Extract device configurations and topology files from Infrahub 3. Deploy the virtual lab using Containerlab 4. Verify the deployment and access virtual devices ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this tutorial, ensure you have: * Completed the [installation guide](/demo-dc/install.md) and have Infrahub running * Completed the [user walkthrough](/demo-dc/user-walkthrough.md) to understand the basic workflow * **Docker** installed and running with sufficient resources (at least 16GB RAM recommended to support a few cEOS containers) * **Containerlab** installed ([installation instructions](https://containerlab.dev/install/)) * **sudo access** for running Containerlab (required for network namespace operations) info This tutorial builds on the user walkthrough. You must complete the basic workflow (creating branches, loading designs, running generators) before deploying to Containerlab. ## Understanding the workflow[​](#understanding-the-workflow "Direct link to Understanding the workflow") The Containerlab deployment workflow connects Infrahub's generated artifacts to a virtual lab environment: ``` Infrahub Data → Generator → Artifacts → Containerlab Deployment ↓ Config Files + Topology File ↓ Virtual Network Lab (cEOS) ``` **Key components:** * **Generated configurations** - Device startup configs extracted from Infrahub artifacts * **Containerlab topology file** - YAML definition of virtual lab including device connections * **cEOS images** - Arista Container EOS virtual switches * **Containerlab** - Network lab orchestration tool that creates Docker containers with proper network connectivity ## Step 1: generate the Arista data center topology[​](#step-1-generate-the-arista-data-center-topology "Direct link to Step 1: generate the Arista data center topology") Use the automated demo command to create a complete Arista data center in Infrahub: ``` uv run invoke demo-dc-arista ``` This command: 1. Creates a new branch `add-dc3` 2. Loads the DC-3 Arista design (`objects/dc/dc-arista-s.yml`) 3. Returns immediately after loading (generator runs asynchronously) ### Wait for generator completion[​](#wait-for-generator-completion "Direct link to Wait for generator completion") The generator runs automatically in the background. Monitor its progress: 1. Navigate to `http://localhost:8000` 2. Switch to the `add-dc3` branch using the branch selector 3. Go to **System → Tasks** in the left sidebar 4. Find the `create_dc` generator task 5. Wait for status to change from "Pending" → "Running" → "Completed" Generator execution typically takes 2-3 minutes as it creates devices, interfaces, IP addresses, and routing configurations. tip While waiting, you can review the design parameters at **Network Topologies → Deployments → Data center → DC-3** to see what will be created. ## Step 2: Create a proposed change and merge[​](#step-2-create-a-proposed-change-and-merge "Direct link to Step 2: Create a proposed change and merge") Before extracting configurations, merge the topology to the main branch: 1. Navigate to **Proposed Changes** 2. Click **New Proposed Change** 3. Fill in the form: * **Name**: `Add DC-3 Arista data center` * **Source Branch**: `add-dc3` * **Destination Branch**: `main` * **Description**: `Deploy DC-3 with Arista switches for Containerlab testing` 4. Click **Create Proposed Change** 5. Review validations and artifacts 6. Click **Merge** once all checks pass info You can also extract configurations from the branch without merging, but merging ensures the topology is validated and ready for production use. ## Step 3: Extract configurations and topology files[​](#step-3-extract-configurations-and-topology-files "Direct link to Step 3: Extract configurations and topology files") Run the containerlab deployment command: ``` uv run invoke containerlab ``` This command performs two operations: ### Operation 1: Extract artifacts from Infrahub[​](#operation-1-extract-artifacts-from-infrahub "Direct link to Operation 1: Extract artifacts from Infrahub") The invoke task runs `uv run scripts/get_configs.py --branch add-dc3` which: * Queries Infrahub for all artifacts in the specified branch * Downloads device configurations (startup configs for each switch) * Downloads the Containerlab topology file * Saves everything to the `generated-configs/` directory **Directory structure created:** ``` generated-configs/ ├── clab/ │ └── DC-3.clab.yml # Containerlab topology definition └── devices/ ├── dc3-spine1.cfg # Spine switch configurations ├── dc3-spine2.cfg ├── dc3-leaf1.cfg # Leaf switch configurations ├── dc3-leaf2.cfg ├── dc3-leaf3.cfg ├── dc3-leaf4.cfg ├── dc3-border-leaf1.cfg # Border leaf configurations └── dc3-border-leaf2.cfg ``` ### Operation 2: Deploy with Containerlab[​](#operation-2-deploy-with-containerlab "Direct link to Operation 2: Deploy with Containerlab") The invoke task then runs `sudo -E containerlab deploy -t generated-configs/clab/DC-3.clab.yml` which: * Reads the topology definition file * Pulls the Arista cEOS Docker image from the OpsMill artifact repository * Creates Docker containers for each device * Configures virtual network connections between devices * Loads the generated startup configurations into each device * Starts all devices info The Containerlab topology file references a cEOS Docker image hosted in the OpsMill artifact repository. On first deployment, Docker will download this image (approximately 500MB). The generated configurations are automatically loaded into each cEOS container during instantiation via Containerlab's startup-configuration mechanism. ### Monitor deployment progress[​](#monitor-deployment-progress "Direct link to Monitor deployment progress") Watch the console output as Containerlab: 1. Creates the lab directory 2. Creates network links between devices 3. Creates containers for each device 4. Loads configurations into devices 5. Waits for devices to boot Deployment typically takes 2-4 minutes depending on your system and whether the Docker image is cached. ## Step 4: Verify the deployment[​](#step-4-verify-the-deployment "Direct link to Step 4: Verify the deployment") Once deployment completes, verify the virtual lab is running: ### Check container status[​](#check-container-status "Direct link to Check container status") ``` sudo containerlab inspect --name DC-3 ``` You should see output showing all devices with their management IP addresses and states. ### List running containers[​](#list-running-containers "Direct link to List running containers") ``` docker ps --filter "label=containerlab=DC-3" ``` All device containers should show as "Up" with healthy status. ### Check device connectivity[​](#check-device-connectivity "Direct link to Check device connectivity") ``` # List all Containerlab labs sudo containerlab inspect --all # Show detailed topology sudo containerlab graph --name DC-3 ``` ## Step 5: Access virtual devices[​](#step-5-access-virtual-devices "Direct link to Step 5: Access virtual devices") Connect to any device in your virtual lab using the Containerlab CLI: ### Connect to a spine switch[​](#connect-to-a-spine-switch "Direct link to Connect to a spine switch") ``` sudo containerlab exec --name DC-3 --node dc3-spine1 Cli ``` This drops you into the Arista EOS CLI. Try some commands: ``` # Check device hostname and version show version # Verify interfaces are up show ip interface brief # Check BGP neighbors show ip bgp summary # View running configuration show running-config ``` Type `exit` to return to your shell. ### Connect to a leaf switch[​](#connect-to-a-leaf-switch "Direct link to Connect to a leaf switch") ``` sudo containerlab exec --name DC-3 --node dc3-leaf1 Cli ``` Verify the leaf's connectivity to spines: ``` # Check BGP neighbors (should see both spines) show ip bgp summary # Check VXLAN configuration show vxlan vtep # Verify EVPN routes show bgp evpn summary ``` ### Connect to devices with SSH[​](#connect-to-devices-with-ssh "Direct link to Connect to devices with SSH") Containerlab also provides SSH access to devices. Find the management IPs: ``` sudo containerlab inspect --name DC-3 ``` Then SSH using the displayed IP addresses: ``` ssh admin@ ``` Default credentials are typically `admin` / `admin` for cEOS devices. ## Step 6: Destroy the lab (cleanup)[​](#step-6-destroy-the-lab-cleanup "Direct link to Step 6: Destroy the lab (cleanup)") When you're done testing, destroy the lab to free resources: ``` sudo containerlab destroy --name DC-3 ``` This command: * Stops all containers * Removes containers and networks * Cleans up Containerlab state * Preserves the configuration files in `generated-configs/` tip The generated configuration files remain in `generated-configs/` even after destroying the lab. This allows you to redeploy quickly or version control your configs. ## What you've learned[​](#what-youve-learned "Direct link to What you've learned") Congratulations! You've completed the Containerlab deployment workflow: * ✅ Generated a multi-vendor data center topology in Infrahub * ✅ Extracted device configurations and topology files * ✅ Deployed a virtual network lab with Containerlab * ✅ Verified device connectivity and routing protocols * ✅ Accessed and tested virtual devices * ✅ Cleaned up lab resources ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Using different branches[​](#using-different-branches "Direct link to Using different branches") Deploy configurations from any branch: ``` uv run invoke containerlab --branch my-branch --topology DC-3 ``` ### Testing configuration changes[​](#testing-configuration-changes "Direct link to Testing configuration changes") Use this workflow to test changes before production: 1. Create a branch in Infrahub 2. Modify device configurations or topology 3. Run generator 4. Extract configs and deploy to Containerlab 5. Test in virtual lab 6. If successful, merge to main and deploy to production ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Deployment fails with "permission denied"[​](#deployment-fails-with-permission-denied "Direct link to Deployment fails with \"permission denied\"") Ensure you're using `sudo` for Containerlab commands: ``` sudo -E containerlab deploy -t generated-configs/clab/DC-3.clab.yml ``` The `-E` flag preserves environment variables (important for Docker access). ### Docker image pull fails[​](#docker-image-pull-fails "Direct link to Docker image pull fails") If the cEOS image pull fails, verify: 1. Docker is running and you have internet connectivity 2. You have sufficient disk space (at least 2GB free) 3. You can access the OpsMill artifact repository ### Devices don't boot or configurations aren't loaded[​](#devices-dont-boot-or-configurations-arent-loaded "Direct link to Devices don't boot or configurations aren't loaded") Check container logs: ``` docker logs clab-DC-3-dc3-spine1 ``` Look for configuration syntax errors or boot failures. ### Out of memory errors[​](#out-of-memory-errors "Direct link to Out of memory errors") Containerlab labs can be resource-intensive. For DC-3 with 8 devices: * Minimum: 4GB RAM * Recommended: 8GB RAM * Optimal: 16GB RAM Reduce device count or increase Docker's memory allocation if needed. ## Next steps[​](#next-steps "Direct link to Next steps") For deeper understanding: * **[Understanding the concepts](/demo-dc/concepts.md)** - Learn about this design-driven automation demo * **[Developer guide](/demo-dc/developer-guide.md)** - Understand how generators and transformations work * **[Containerlab documentation](https://containerlab.dev/)** - Explore advanced Containerlab features --- # Developer guide This guide provides a technical deep-dive into how the Infrahub demo works under the hood. Use this when you want to extend functionality, troubleshoot issues, customize the demo, or understand implementation details. ## Project architecture[​](#project-architecture "Direct link to Project architecture") The demo follows Infrahub's SDK pattern with five core component types working together: ``` Schemas → Data → Generators → Transforms → Configurations ↓ Checks (Validation) ``` ### Component types[​](#component-types "Direct link to Component types") 1. **Schemas** (`schemas/`) - Define data models, relationships, and constraints 2. **Generators** (`generators/`) - Create infrastructure topology programmatically 3. **Transforms** (`transforms/`) - Convert Infrahub data to device configurations 4. **Checks** (`checks/`) - Validate configurations and connectivity 5. **Templates** (`templates/`) - Jinja2 templates for device configurations All components are registered in `.infrahub.yml`, which acts as the configuration hub. ## Project structure[​](#project-structure "Direct link to Project structure") ``` infrahub-demo-dc/ ├── .infrahub.yml # Component registration ├── checks/ # Validation checks │ ├── spine.py │ ├── leaf.py │ ├── edge.py │ ├── loadbalancer.py │ ├── virtualization_vm.py │ ├── virtualization_host.py │ └── virtualization_capacity.py ├── objects/ # Demo data │ ├── bootstrap/ # Initial data (21 files: groups, locations, platforms, hypervisor types, device types, templates, etc.) │ ├── cloud_security/ # Cloud security examples (services, devices, gateways) │ ├── dc/ # Data center design files │ │ ├── dc-arista-s.yml # DC-3 design data (Arista) │ │ ├── dc-cisco-s.yml # DC-2 design data (Cisco) │ │ ├── dc-cisco-s-border-leafs.yml # Cisco DC with border leafs │ │ ├── dc-juniper-s.yml # DC-5 design data (Juniper) │ │ └── dc-sonic-border-leafs.yml # DC-4 design data (SONiC with border leafs) │ ├── events/ # Event action definitions │ ├── lb/ # Load balancer configurations │ ├── pop/ # Point of presence design files │ │ ├── pop-1.yml # POP-1 design data │ │ └── pop-2.yml # POP-2 design data │ ├── security/ # Security zones, policies, rules (16 files) │ └── virtualization/ # Clusters, physical hosts, and VMs ├── generators/ # Topology generators │ ├── generate_dc.py # Data center generator │ ├── generate_pop.py # POP generator │ ├── generate_segment.py # Network segment generator │ ├── generate_virtualization_cabling.py # Host rack placement and leaf cabling │ ├── generate_virtualization_vm_security.py # VM addressing and HTTPS-only policy │ ├── generate_vm_artifact_groups.py # Per-hypervisor VM artifact group assignment │ ├── generate_dc_firewall_policy.py # Attach VM policy to DC firewalls │ ├── common.py # Shared utilities │ └── schema_protocols.py # Type protocols ├── menus/ # UI menu definitions │ └── menu-full.yml # Complete menu ├── queries/ # GraphQL queries │ ├── config/ # Configuration queries │ ├── topology/ # Topology queries │ ├── validation/ # Validation queries │ └── virtualization/ # Host cabling, VM security, and artifact group queries ├── schemas/ # Data model definitions │ ├── base/ # Core models │ │ ├── dcim.yml │ │ ├── ipam.yml │ │ ├── location.yml │ │ ├── topology.yml │ │ └── virtualization.yml │ └── extensions/ # Extended models │ ├── console/ │ ├── routing/ │ ├── security/ │ ├── service/ │ ├── topology/ │ └── virtualization/ ├── docs/ # Documentation (Docusaurus) │ ├── docs/ # Documentation content (.mdx files) │ ├── static/ # Static assets │ ├── docusaurus.config.ts # Docusaurus configuration │ └── package.json # Node.js dependencies ├── scripts/ # Automation scripts │ ├── bootstrap.py # Complete setup script │ ├── populate_security_relationships.py # Security data relationships │ ├── create_proposed_change.py # Create proposed changes │ └── get_configs.py # Retrieve device configurations ├── service_catalog/ # Streamlit Service Catalog application │ ├── pages/ # Streamlit pages │ │ ├── 1_Create_DC.py # DC creation UI │ │ └── 4_Create_VM.py # VM creation UI │ ├── utils/ # Utility modules │ │ ├── api.py # Infrahub API client │ │ ├── config.py # Configuration │ │ └── ui.py # UI helpers │ ├── Home.py # Main application page │ └── Dockerfile # Container definition ├── templates/ # Jinja2 config templates ├── tests/ # Test suite │ ├── integration/ # Integration tests │ ├── smoke/ # Smoke tests │ └── unit/ # Unit tests ├── transforms/ # Config transforms │ ├── edge.py │ ├── leaf.py │ ├── loadbalancer.py │ └── spine.py └── tasks.py # Invoke task definitions ``` ## Schemas[​](#schemas "Direct link to Schemas") Schemas define the data model using YAML. They specify nodes (object types), attributes, relationships, and constraints. ### Schema naming conventions[​](#schema-naming-conventions "Direct link to Schema naming conventions") * **Nodes**: PascalCase (for example, `DcimDevice`) * **Attributes**: snake\_case (for example, `device_type`) * **Relationships**: snake\_case (for example, `parent_location`) * **Namespaces**: PascalCase (for example, `Dcim`, `Ipam`, `Service`) ### Schema example[​](#schema-example "Direct link to Schema example") ``` nodes: - name: GenericDevice namespace: Dcim description: "A network device" inherit_from: - DcimDevice attributes: - name: hostname kind: Text optional: false unique: true - name: device_type kind: Text optional: true relationships: - name: location peer: LocationBuilding cardinality: one optional: false - name: interfaces peer: DcimInterface cardinality: many kind: Component ``` ### Computed attributes[​](#computed-attributes "Direct link to Computed attributes") Infrahub supports [computed attributes](https://docs.infrahub.app/topics/computed-attributes) that automatically generate values based on other attributes using Jinja2 templates. The demo uses this feature in the BGP schema for Autonomous System names. **Example from `schemas/extensions/routing/bgp.yml`:** ``` attributes: - name: name kind: Text computed_attribute: kind: Jinja2 jinja2_template: "AS{{asn__value}}" read_only: true optional: false - name: asn kind: Number description: "Autonomous System Number" ``` When you create an Autonomous System with ASN 65000, the `name` attribute is automatically computed as "AS65000". This ensures consistency and reduces manual data entry errors. **Benefits of computed attributes:** * **Consistency** - Standardized naming conventions enforced automatically * **Reduced errors** - No manual entry of derived values * **Dynamic updates** - Values recompute when dependencies change * **Read-only enforcement** - Prevents manual modification of computed values ### Schema types[​](#schema-types "Direct link to Schema types") The demo includes schemas for: * **DCIM** (Data Center Infrastructure Management) - Devices, interfaces, racks * **IPAM** (IP Address Management) - IP addresses, prefixes, VLANs * **Location** - Sites, buildings, rooms * **Topology** - Data centers, POPs, deployments * **Routing** - BGP, OSPF, routing policies * **Security** - Zones, policies, firewall rules * **Service** - Load balancers, segments, services ### Loading schemas[​](#loading-schemas "Direct link to Loading schemas") ``` uv run infrahubctl schema load schemas --branch main ``` Schemas are loaded into Infrahub and become the foundation for all data. ## Generators[​](#generators "Direct link to Generators") Generators create infrastructure topology programmatically from high-level design inputs. They inherit from `InfrahubGenerator` and implement the `generate()` method. ### Generator pattern[​](#generator-pattern "Direct link to Generator pattern") ``` from infrahub_sdk.generators import InfrahubGenerator from typing import Any class DCTopologyGenerator(InfrahubGenerator): async def generate(self, data: dict[str, Any]) -> None: """Generate data center topology based on design data.""" # 1. Query design data # 2. Create devices # 3. Create interfaces # 4. Create IP addresses # 5. Create routing configurations pass ``` ### DC generator workflow[​](#dc-generator-workflow "Direct link to DC generator workflow") The `create_dc` generator in `generators/generate_dc.py`: 1. **Queries the topology design** - Reads DC-3 parameters like spine count, leaf count, underlay protocol 2. **Creates resource pools** - Sets up IP prefix pools and VLAN pools 3. **Creates devices** - Generates spine, leaf, and border-leaf switches with correct roles and platforms 4. **Creates interfaces** - Adds physical interfaces, loopbacks, and sub-interfaces 5. **Creates connections** - Establishes fabric peering between spines and leaves 6. **Configures routing** - Sets up BGP or OSPF underlay and BGP EVPN overlay 7. **Assigns IP addresses** - Allocates addresses from pools for all interfaces ### A generator run is a desired state, not a script[​](#a-generator-run-is-a-desired-state-not-a-script "Direct link to A generator run is a desired state, not a script") This is the single most important thing to know before writing or editing a generator here, because getting it wrong deletes data rather than failing. Infrahub wraps `generate()` in a tracking context. Every object the run saves is recorded as belonging to that generator instance, and **anything the previous run saved that this run does not save is deleted on the next run**. Component children go with it: deleting a host deletes its interfaces and its virtual machines. The consequence is counter-intuitive. The natural way to make a generator idempotent is to skip work that is already done: ``` # Wrong: the skip drops the interface out of the tracking group, # so the next run deletes it. if interface.connector.peer: self.logger.info("already cabled, skipping") continue ``` That reads as a safe no-op and behaves as a delete one run later. The fix is to re-save the object even when nothing about it changed: ``` # Right: "still mine". A save with no changed attribute is cheap. if interface.connector.peer: self.logger.info("already cabled, keeping it") await interface.save(allow_upsert=True) continue ``` Practical rules for this repository: * Every `save()` uses `allow_upsert=True`. Without it the second run errors on the first existing object and aborts, leaving the tracking group half written * Re-save every object the generator owns on every run, including the target itself. `generators/generate_virtualization_cabling.py` does this through a `_keep_tracked()` helper, and re-saves the host, its cabled NICs, the leaf ports they land on, and the cables between them * Only nodes you `save()` join the tracking group. `add_relationships()` issues a relationship mutation and does not, which is why the generators that only add group members carry no deletion risk * An early `return` is a decision to delete: if the run bails out before saving the target, the target goes on the next run. Save it first, then return Reviewing a generator means asking "what did this run *not* save?", because that is the blast radius. ### Generator registration[​](#generator-registration "Direct link to Generator registration") Generators are registered in `.infrahub.yml`: ``` generator_definitions: - name: create_dc file_path: generators/generate_dc.py targets: topologies_dc query: topology_dc class_name: DCTopologyGenerator parameters: name: name__value ``` * **targets** - GraphQL query that selects which objects trigger the generator * **query** - GraphQL query providing input data * **parameters** - Parameters passed from triggering object ### Running generators[​](#running-generators "Direct link to Running generators") Generators can be triggered: 1. **Manually** via the web UI (Actions → Generator Definitions) 2. **Via API** using GraphQL mutations 3. **Automatically** via event actions (if configured) ## Transforms[​](#transforms "Direct link to Transforms") Transforms convert Infrahub data into device configurations. They inherit from `InfrahubTransform` and use Jinja2 templates. ### Transform pattern[​](#transform-pattern "Direct link to Transform pattern") ``` from infrahub_sdk.transforms import InfrahubTransform from typing import Any class SpineTransform(InfrahubTransform): query = "spine_config" # GraphQL query name async def transform(self, data: Any) -> Any: """Transform InfrahubHub data to spine configuration.""" device = data["DcimDevice"]["edges"][0]["node"] # Process data context = self.prepare_context(device) # Render template return self.render_template( template="spine.j2", data=context ) def prepare_context(self, device: Any) -> dict[str, Any]: """Prepare template context from device data.""" return { "hostname": device["name"]["value"], "interfaces": self.process_interfaces(device["interfaces"]), "bgp": self.process_bgp(device), } ``` ### Transform workflow[​](#transform-workflow "Direct link to Transform workflow") 1. **Query data** - Fetch device and related data via GraphQL 2. **Process data** - Transform into template-friendly structure 3. **Render template** - Use Jinja2 to generate configuration 4. **Return artifact** - Provide configuration as string ### Transform registration[​](#transform-registration "Direct link to Transform registration") Transforms are registered in `.infrahub.yml`: ``` python_transforms: - name: spine class_name: Spine file_path: transforms/spine.py artifact_definitions: - name: spine_config artifact_name: spine content_type: text/plain targets: spines # GraphQL query selecting devices transformation: spine # Transform name parameters: device: name__value ``` ## Templates[​](#templates "Direct link to Templates") Jinja2 templates generate device configurations from structured data. ### Template example[​](#template-example "Direct link to Template example") ``` hostname {{ hostname }} {% for interface in interfaces %} interface {{ interface.name }} {% if interface.description %} description {{ interface.description }} {% endif %} {% if interface.ip_address %} ip address {{ interface.ip_address }} {% endif %} {% if interface.enabled %} no shutdown {% endif %} {% endfor %} router bgp {{ bgp.asn }} {% for neighbor in bgp.neighbors %} neighbor {{ neighbor.ip }} remote-as {{ neighbor.asn }} {% endfor %} ``` Templates use standard Jinja2 syntax with filters and control structures. ## Checks[​](#checks "Direct link to Checks") Checks validate configurations and connectivity. They inherit from `InfrahubCheck` and implement the `check()` method. ### Check pattern[​](#check-pattern "Direct link to Check pattern") ``` from infrahub_sdk.checks import InfrahubCheck from typing import Any class CheckSpine(InfrahubCheck): query = "spine_validation" async def check(self, data: Any) -> None: """Validate spine device configuration.""" device = data["DcimDevice"]["edges"][0]["node"] # Validation logic if not self.has_required_interfaces(device): self.log_error( "Missing required interfaces", object_id=device["id"], object_type="DcimDevice" ) if not self.has_bgp_config(device): self.log_warning( "BGP not configured", object_id=device["id"] ) ``` ### Check registration[​](#check-registration "Direct link to Check registration") ``` check_definitions: - name: validate_spine class_name: CheckSpine file_path: checks/spine.py targets: spines parameters: device: name__value ``` ## GraphQL queries[​](#graphql-queries "Direct link to GraphQL queries") Queries are defined in `.gql` files and referenced by name in transforms and checks. ### Query example[​](#query-example "Direct link to Query example") ``` query GetSpineConfig($device_name: String!) { DcimDevice(name__value: $device_name) { edges { node { id name { value } role { value } platform { value } interfaces { edges { node { name { value } description { value } ip_addresses { edges { node { address { value } } } } } } } } } } } ``` ### Query registration[​](#query-registration "Direct link to Query registration") ``` queries: - name: spine_config file_path: queries/config/spine.gql ``` ## Bootstrap data[​](#bootstrap-data "Direct link to Bootstrap data") Bootstrap data provides initial objects like locations, platforms, and device types. ### Bootstrap structure[​](#bootstrap-structure "Direct link to Bootstrap structure") ``` objects/bootstrap/ ├── 01_organizations.yml # Organizations ├── 02_asn_pools.yml # BGP ASN pools ├── 03_locations.yml # Sites and buildings ├── 04_platforms.yml # Device platforms ├── 05_roles.yml # Device roles ├── 06_device_types.yml # Device models ├── 07_device_templates.yml # Interface templates └── ... ``` Files are numbered to ensure correct loading order due to dependencies. ### Interface range expansion[​](#interface-range-expansion "Direct link to Interface range expansion") The bootstrap data uses Infrahub's [interface range expansion feature](https://docs.infrahub.app/python-sdk/reference/templating#netutils-filters) to efficiently define multiple interfaces with compact syntax. This feature automatically expands range notation into individual interfaces. **Example from `objects/bootstrap/10_physical_device_templates.yml`:** ``` interfaces: kind: TemplateInterfacePhysical data: - template_name: N9K-C9336C-FX2_SPINE_Ethernet1/[1-30] name: Ethernet1/[1-30] role: leaf - template_name: N9K-C9336C-FX2_SPINE_Ethernet1/[31-36] name: Ethernet1/[31-36] role: uplink ``` When loaded, `Ethernet1/[1-30]` expands to 30 individual interfaces: `Ethernet1/1`, `Ethernet1/2`, ... `Ethernet1/30`. This dramatically reduces YAML verbosity when defining device templates with many interfaces. **Benefits of range expansion:** * **Compact notation** - Define dozens of interfaces in a single line * **Reduced errors** - Less repetitive typing means fewer mistakes * **Simplified maintenance** - Update interface ranges without editing individual entries * **Vendor compatibility** - Supports common interface naming patterns (Ethernet, GigabitEthernet, et-, ge-, etc.) This feature is used extensively throughout the bootstrap data for device templates, physical devices, and topology definitions. ### Loading bootstrap data[​](#loading-bootstrap-data "Direct link to Loading bootstrap data") ``` uv run infrahubctl object load objects/bootstrap --branch main ``` ## Testing[​](#testing "Direct link to Testing") The suite has three tiers. Only the integration tier needs Docker. ``` # Unit and specification tests: seconds, no containers uv run invoke test-unit # The integration tier every pull request runs uv run invoke test-integration --tier=core # Everything, including the extended workflows uv run invoke test-integration --tier=full ``` ### Unit and specification tests[​](#unit-and-specification-tests "Direct link to Unit and specification tests") `tests/unit/` covers individual functions with mocked clients, and `tests/smoke/` holds SDK plugin specifications: `test_graphql.yml` runs every GraphQL query registered in `.infrahub.yml` through the SDK's query checker, and `test_j2_transforms.yml` renders Jinja templates against recorded input. Neither tier starts a container, so both are fast enough to run on every save. ### Integration tests[​](#integration-tests "Direct link to Integration tests") The integration suite drives complete workflows against a real Infrahub deployment started by [infrahub-testcontainers](https://pypi.org/project/infrahub-testcontainers/). It does not use `invoke start`: it brings up its own throwaway stack, bootstraps it from this repository, and tears it down afterward. #### One deployment, shared by every module[​](#one-deployment-shared-by-every-module "Direct link to One deployment, shared by every module") `tests/integration/conftest.py` declares the deployment at **session** scope, so the whole suite starts one stack and bootstraps it once. Loading the schema and the bootstrap data takes several minutes, so a stack per module would put most of this coverage out of reach of any reasonable CI timeout. The consequence is that modules are **not independent**. They run in file-name order, and later modules build on what earlier ones merged into `main`: | Module | Workflow under test | | ------------------------- | ---------------------------------------------------------------------------- | | `test_00_bootstrap` | Schema, menu, objects, repository registration, event actions | | `test_10_dc_workflow` | Arista DC design, generator, proposed change, merge to `main` | | `test_20_artifacts` | Device configurations, Containerlab topology, cabling matrix, rack elevation | | `test_30_graphql` | Every query registered in `.infrahub.yml`, executed against the live schema | | `test_40_dc_vendors` | A second data center from a different vendor design, with border leafs | | `test_50_pop` | POP topology and its virtual devices | | `test_60_segment` | Network segment service over the merged fabric | | `test_70_day2` | Editing a device, then scaling a deployment out to a larger design | | `test_80_proposed_change` | Conflict detection and closing a proposed change without merging | Cross-module dependencies are declared with `pytest-dependency`, so a module whose prerequisite failed is skipped rather than reported as a second failure. #### Tiers and when they run[​](#tiers-and-when-they-run "Direct link to Tiers and when they run") Modules are marked `core` or `extended`: * `core` runs on every pull request. It must pass on its own, so it never depends on anything an extended module produces. * `extended` runs on a pull request whose branch bumps the Infrahub version (`update-infrahub-*`, but not `update-infrahub-sdk-*`), and on a manual run of the CI workflow with its `tier` input set to `full`. `.github/workflows/ci.yml` selects the tier. Dispatch the tier for a change to a generator or a transform, so those workflows are exercised before it merges. It does not block a merge, because an upstream Infrahub fault currently stops it passing reliably: Infrahub intermittently fails to resolve a member of its own internal generator group, which fails whichever generator is running at the time. Read a failure in that tier against this known fault before concluding the demo is broken. #### Testing a specific Infrahub version[​](#testing-a-specific-infrahub-version "Direct link to Testing a specific Infrahub version") By default the suite tests the Infrahub image matching the installed `infrahub-testcontainers` version, which is what makes a dependency-bump pull request meaningful. Override it to test any other version: ``` INFRAHUB_TESTING_IMAGE_VER=1.4.0 uv run invoke test-integration --tier=core ``` #### Writing an integration test[​](#writing-an-integration-test "Direct link to Writing an integration test") Use the helpers in `tests/integration/helpers.py` rather than raw GraphQL; they already carry the polling and the failure messages. ``` from . import constants as c from . import helpers as h pytestmark = pytest.mark.extended @pytest.mark.dependency(name="my_thing_loaded") async def test_01_load(async_client_main, infrahub_address, infrahub_bootstrap) -> None: """One sentence on the property under test.""" await h.ensure_branch(async_client_main, "my-branch") h.load_objects("objects/dc/dc-juniper-s.yml", address=infrahub_address, branch="my-branch") ``` Four conventions matter more here than in the unit suite: 1. **Derive expectations from the data model, not from literals.** `h.expected_role_counts` reads a design and returns the device counts it calls for. A hard-coded `assert len(devices) == 12` breaks whenever the design changes, and asserts nothing about the generator. 2. **Never treat "the first result appeared" as "the work finished".** Every asynchronous step here produces results progressively: a topology generator creates devices in its third of six phases, artifact generation produces one artifact per target-group member, and a proposed-change review opens more than twenty validators. Waiting for "everything that exists is done" is satisfied immediately in all three cases. Use `h.wait_for_quiescence`, pass an `expected` count to `h.wait_for_artifacts`, and let `h.wait_for_validations` require the validator count to settle. 3. **Say what failed in the assertion message.** These tests fail on a machine nobody can log in to. `h.merge_proposed_change` shows the shape: it quotes the merge task's own logs, the validators that did not conclude successfully, and what each of them objected to. 4. **Assert completion and conclusion separately.** A validator that never finishes is an Infrahub problem; a validator that finishes and reports a failure is a demo-data problem. #### Diagnosing a failure[​](#diagnosing-a-failure "Direct link to Diagnosing a failure") Container logs for `infrahub-server` and `task-worker` are attached to any run that had a failure, so the CI log already contains them. Locally, add `--log-cli-level=INFO` to watch each step as it happens, and `-k` to narrow the run: ``` uv run pytest tests/integration/test_10_dc_workflow.py --log-cli-level=INFO ``` Because the suite shares one deployment, running a single module still bootstraps the stack, and a module that depends on a merged fabric is skipped when run on its own. Two failure messages are worth recognizing: * `Unable to find the class CheckEdge` from a check or transform does **not** mean the class is missing. Infrahub reports any exception raised inside a check or transform under that message; the real cause is in the traceback above it, in the container log. * Before concluding that a failure is a defect in the demo or in Infrahub, check how the deployment was resourced. A stack trimmed down to share a busy machine produces database timeouts that look like workflow failures. ### Code quality[​](#code-quality "Direct link to Code quality") ``` # Type checking uv run mypy . # Linting uv run ruff check . # Formatting uv run ruff format . # All checks: rumdl, yamllint, ruff, mypy uv run invoke lint ``` ## Development workflow[​](#development-workflow "Direct link to Development workflow") ### Setting up for development[​](#setting-up-for-development "Direct link to Setting up for development") ``` # Clone repository git clone https://github.com/opsmill/infrahub-demo-dc.git cd infrahub-demo-dc # Install dependencies uv sync # Start Infrahub uv run invoke start # Optional: Enable Service Catalog in .env echo "INFRAHUB_SERVICE_CATALOG=true" >> .env uv run invoke restart-containers # Load bootstrap data uv run invoke bootstrap ``` ### Making changes[​](#making-changes "Direct link to Making changes") 1. **Create a feature branch** in Git 2. **Modify code** (generators, transforms, checks, schemas, Service Catalog) 3. **Add tests** for new functionality 4. **Run quality checks** (`uv run invoke lint`) 5. **Test locally** in Infrahub * For Service Catalog changes: use `uv run invoke start --rebuild` 6. **Commit changes** with descriptive messages 7. **Create pull request** for review ### Adding a new generator[​](#adding-a-new-generator "Direct link to Adding a new generator") 1. Create Python file in `generators/` 2. Implement `InfrahubGenerator` class 3. Register in `.infrahub.yml` under `generator_definitions` 4. Declare any helper module it imports under `watch.files`, so that editing the helper re-runs the generator. Infrahub reads the entry point, so a module imported at runtime is invisible to it 5. Create associated GraphQL query in `queries/` 6. Add unit tests 7. Test manually in Infrahub ### Adding a new transform[​](#adding-a-new-transform "Direct link to Adding a new transform") 1. Create Python file in `transforms/` 2. Implement `InfrahubTransform` class 3. Create Jinja2 template in `templates/` 4. Register in `.infrahub.yml` under `python_transforms` and `artifact_definitions` 5. Declare under `watch.files` any helper module the transform imports, and the template directory when the template is chosen from device data rather than named in the source. Both are invisible to Infrahub, and without them a template edit leaves every rendered artifact stale 6. Create GraphQL query in `queries/` 7. Add unit tests 8. Test artifact generation ### Adding a new check[​](#adding-a-new-check "Direct link to Adding a new check") 1. Create Python file in `checks/` 2. Implement `InfrahubCheck` class 3. Register in `.infrahub.yml` under `check_definitions` 4. Create GraphQL query in `queries/` 5. Add unit tests 6. Test in proposed change workflow ### Service catalog development[​](#service-catalog-development "Direct link to Service catalog development") The Service Catalog is a Streamlit application that runs in a Docker container. When making changes to the Service Catalog code, you need to rebuild the container image. #### Making changes to the service catalog[​](#making-changes-to-the-service-catalog "Direct link to Making changes to the service catalog") 1. **Edit Service Catalog code** in `service_catalog/`: * `Home.py` - Main landing page * `pages/1_Create_DC.py` - DC creation form * `utils/` - Utility modules (api.py, config.py, ui.py) 2. **Rebuild and restart** the Service Catalog container: ``` uv run invoke start --rebuild ``` The `--rebuild` flag forces Docker to rebuild the Service Catalog image with your code changes before starting the containers. #### When to use --rebuild[​](#when-to-use---rebuild "Direct link to When to use --rebuild") Use the `--rebuild` flag when you modify: * Streamlit page files (`Home.py`, `pages/*.py`) * Service Catalog utilities (`service_catalog/utils/`) * Service Catalog dependencies (if you modify `service_catalog/requirements.txt`) * Service Catalog Dockerfile #### Testing service catalog changes[​](#testing-service-catalog-changes "Direct link to Testing service catalog changes") 1. Make your code changes in `service_catalog/` 2. Rebuild and start with `uv run invoke start --rebuild` 3. Access the Service Catalog at `http://localhost:8501` 4. Test your changes in the web interface 5. Check logs for errors: ``` docker logs infrahub-demo-dc-service-catalog-1 ``` #### Service catalog environment variables[​](#service-catalog-environment-variables "Direct link to Service catalog environment variables") Configure the Service Catalog behavior via `.env`: ``` INFRAHUB_SERVICE_CATALOG=true # Enable the service catalog DEFAULT_BRANCH=main # Default branch to show GENERATOR_WAIT_TIME=60 # Seconds to wait for generator API_TIMEOUT=30 # API request timeout API_RETRY_COUNT=3 # Number of API retries ``` Changes to environment variables do not require `--rebuild`, just restart: ``` uv run invoke restart-containers ``` ## Extending schemas[​](#extending-schemas "Direct link to Extending schemas") ### Adding new attributes[​](#adding-new-attributes "Direct link to Adding new attributes") ``` nodes: - name: GenericDevice namespace: Dcim attributes: - name: serial_number # New attribute kind: Text optional: true unique: true ``` ### Adding new relationships[​](#adding-new-relationships "Direct link to Adding new relationships") ``` relationships: - name: backup_device # New relationship peer: DcimDevice cardinality: one optional: true description: "Backup device for redundancy" ``` ### Creating new node types[​](#creating-new-node-types "Direct link to Creating new node types") ``` nodes: - name: Router # New node type namespace: Dcim inherit_from: - DcimDevice attributes: - name: routing_instance kind: Text optional: false ``` After modifying schemas, reload them: ``` uv run infrahubctl schema load schemas --branch main ``` ## Common development tasks[​](#common-development-tasks "Direct link to Common development tasks") ### Debugging generators[​](#debugging-generators "Direct link to Debugging generators") Add logging to see execution flow: ``` import logging logger = logging.getLogger(__name__) class MyGenerator(InfrahubGenerator): async def generate(self, data: dict) -> None: logger.info(f"Processing topology: {data}") # ... generator logic ``` ### Testing transforms locally[​](#testing-transforms-locally "Direct link to Testing transforms locally") ``` # Create test data test_data = { "DcimDevice": { "edges": [{"node": {"name": {"value": "spine1"}}}] } } # Initialize transform transform = SpineTransform(client=client) # Run transform result = await transform.transform(test_data) print(result) ``` ### Validating templates[​](#validating-templates "Direct link to Validating templates") Use Jinja2 directly to test templates: ``` from jinja2 import Template template = Template(open("templates/spine.j2").read()) config = template.render(hostname="spine1", interfaces=[...]) print(config) ``` ## Additional resources[​](#additional-resources "Direct link to Additional resources") * **Infrahub documentation**: [docs.infrahub.app/](https://docs.infrahub.app/) --- # Using Infrahub Enterprise This guide explains how to configure the demo environment to use Infrahub Enterprise edition instead of the Community edition. Enterprise edition provides additional features including increased performance, enhanced security, and enterprise support. ## Understanding the editions[​](#understanding-the-editions "Direct link to Understanding the editions") Infrahub is available in two editions: * **Community edition** - The standard open-source version with core functionality * **Enterprise edition** - The commercial version with additional enterprise features The demo environment supports both editions and can be switched between them with a single configuration change. ## Configuration[​](#configuration "Direct link to Configuration") Edition selection is controlled by environment variables in the `.env` file at the root of the demo repository. The key variable is `INFRAHUB_ENTERPRISE`: ``` # Community edition (default) export INFRAHUB_ENTERPRISE="false" # Enterprise edition export INFRAHUB_ENTERPRISE="true" ``` ### Environment variables reference[​](#environment-variables-reference "Direct link to Environment variables reference") | Variable | Default | Description | | --------------------- | ----------------------- | ------------------------------------------------------------------- | | `INFRAHUB_ENTERPRISE` | `false` | Set to `true` to use Enterprise edition | | `INFRAHUB_VERSION` | `stable` | Infrahub version to use (for example, `1.5.1`, `stable`, `develop`) | | `INFRAHUB_ADDRESS` | `http://localhost:8000` | Infrahub API address | | `INFRAHUB_API_TOKEN` | (demo token) | API authentication token | ## Switching to Enterprise edition[​](#switching-to-enterprise-edition "Direct link to Switching to Enterprise edition") To switch from Community to Enterprise edition: ### Step 1: update the configuration[​](#step-1-update-the-configuration "Direct link to Step 1: update the configuration") Edit the `.env` file in the root of the demo repository: ``` # Change this line from "false" to "true" export INFRAHUB_ENTERPRISE="true" ``` ### Step 2: source the environment file[​](#step-2-source-the-environment-file "Direct link to Step 2: source the environment file") For the changes to take effect, source the environment file: ``` source .env ``` ### Step 3: verify the configuration[​](#step-3-verify-the-configuration "Direct link to Step 3: verify the configuration") Check that the Enterprise edition will be used: ``` uv run invoke info ``` You should see output similar to: ``` Infrahub Edition: Enterprise Version: 1.5.1 Command: curl -s https://infrahub.opsmill.io/enterprise/1.5.1 | ... ``` ### Step 4: restart Infrahub[​](#step-4-restart-infrahub "Direct link to Step 4: restart Infrahub") To apply the change, destroy the existing environment and start fresh: ``` # Stop and remove all containers and volumes uv run invoke destroy # Start with Enterprise edition uv run invoke start ``` tip The `destroy` command removes all containers and data volumes. If you have data you want to preserve, consider exporting it first using `infrahubctl`. ### Step 5: reload your data[​](#step-5-reload-your-data "Direct link to Step 5: reload your data") After switching editions, you'll need to reload your schemas and data: ``` uv run invoke bootstrap ``` ## Switching back to Community edition[​](#switching-back-to-community-edition "Direct link to Switching back to Community edition") To switch from Enterprise back to Community edition, follow the same process but set `INFRAHUB_ENTERPRISE="false"`: ``` # Edit .env and change to "false" source .env # Verify configuration uv run invoke info # Restart uv run invoke destroy uv run invoke start ``` ## How it works[​](#how-it-works "Direct link to How it works") When you run `uv run invoke start`, the system constructs the appropriate Docker Compose command based on your configuration: **Community edition URL pattern:** ``` https://infrahub.opsmill.io/{VERSION} ``` **Enterprise edition URL pattern:** ``` https://infrahub.opsmill.io/enterprise/{VERSION} ``` The invoke tasks in `tasks.py` automatically detect the `INFRAHUB_ENTERPRISE` setting and use the correct URL to download the appropriate Docker Compose configuration. ## Checking your current edition[​](#checking-your-current-edition "Direct link to Checking your current edition") At any time, you can check which edition you're configured to use: ``` source .env && uv run invoke info ``` This displays: * The edition (Community or Enterprise) * The version being used * The full Docker Compose command ## Enterprise features[​](#enterprise-features "Direct link to Enterprise features") Enterprise edition includes additional capabilities not available in Community edition. For details on specific enterprise features, refer to the [Infrahub documentation](https://docs.infrahub.app). info Enterprise edition requires valid licensing for production usage. Contact OpsMill for enterprise licensing information. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Changes not taking effect[​](#changes-not-taking-effect "Direct link to Changes not taking effect") If you change the `INFRAHUB_ENTERPRISE` setting but the wrong edition starts: 1. Ensure you sourced the `.env` file: `source .env` 2. Verify the setting with `uv run invoke info` 3. Make sure you destroyed the old environment before starting: `uv run invoke destroy` ### Environment file not found[​](#environment-file-not-found "Direct link to Environment file not found") If you get errors about missing environment variables: 1. Ensure the `.env` file exists in the root of the demo repository 2. Make sure you're running commands from the demo repository root 3. Source the file explicitly: `source .env` ## Best practices[​](#best-practices "Direct link to Best practices") * **Use explicit versions** - Instead of `stable`, specify exact versions like `1.5.1` for reproducible environments * **Document your choice** - Note which edition you're using in project documentation * **Test on Community first** - Validate your schemas and generators on Community edition before deploying to Enterprise * **Keep configurations in sync** - If working in a team, ensure everyone uses the same edition for consistency ## Additional resources[​](#additional-resources "Direct link to Additional resources") * [Infrahub documentation](https://docs.infrahub.app) * [OpsMill website](https://opsmill.com) * [Enterprise edition details](https://opsmill.com/pricing) --- # Installation guide This guide walks you through setting up the Infrahub demo environment on your local system. By the end, you'll have a fully functional Infrahub instance with pre-loaded schemas, data, and generators ready to explore. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before you begin, ensure you have the following installed on your system: ### Required software[​](#required-software "Direct link to Required software") * **Python 3.11 or 3.12** - The demo requires a recent Python version * **[uv](https://github.com/astral-sh/uv)** - Fast Python package manager for dependency management * **Docker** - For running Infrahub containers and optionally Containerlab (or equivalent such as OrbStack) * **Git** - For cloning the repository ### System requirements[​](#system-requirements "Direct link to System requirements") * **Operating system**: macOS, Linux, or Windows (with WSL2) * **Memory**: At least 12GB RAM (16GB recommended) * **Disk space**: At least 10GB free space * **Network**: Internet connection for downloading dependencies and Docker images ## Step 1: install uv[​](#step-1-install-uv "Direct link to Step 1: install uv") If you don't have `uv` installed, follow the installation instructions for your operating system: ``` # macOS and Linux curl -LsSf https://astral.sh/uv/install.sh | sh # Windows (PowerShell) powershell -c "irm https://astral.sh/uv/install.ps1 | iex" ``` Verify the installation: ``` uv --version ``` ## Step 2: clone the repository[​](#step-2-clone-the-repository "Direct link to Step 2: clone the repository") Clone the demo repository to your local machine: ``` git clone https://github.com/opsmill/infrahub-demo-dc.git cd infrahub-demo-dc ``` ## Step 3: install Python dependencies[​](#step-3-install-python-dependencies "Direct link to Step 3: install Python dependencies") Use `uv` to install all required Python packages: ``` uv sync ``` This creates a virtual environment and installs all necessary packages including the Infrahub SDK. ## Step 4: set up environment variables[​](#step-4-set-up-environment-variables "Direct link to Step 4: set up environment variables") Configure environment variables for connecting to Infrahub and customizing the demo environment. Copy the example file and customize as needed: ``` cp .env.example .env ``` Then edit the `.env` file or export the variables directly in your shell. ### Core Infrahub variables[​](#core-infrahub-variables "Direct link to Core Infrahub variables") These variables configure connectivity to the Infrahub API: ``` # Infrahub API server address # Used by CLI tools and local scripts to connect to Infrahub # Default: http://localhost:8000 INFRAHUB_ADDRESS=http://localhost:8000 # Infrahub API authentication token # Generate a token in the Infrahub UI under Settings > API Tokens # For demo purposes, use the provided token below INFRAHUB_API_TOKEN=06438eb2-8019-4776-878c-0941b1f1d1ec # Infrahub UI URL for browser links # Used by the Service Catalog to generate clickable links to Infrahub UI pages # Should be the URL accessible from your browser # Default: http://localhost:8000 INFRAHUB_UI_URL=http://localhost:8000 ``` info The API token `06438eb2-8019-4776-878c-0941b1f1d1ec` is a demo token for local development only. Do not use this in production environments. Generate a new token in the Infrahub UI under Settings > API Tokens for production use. ### Infrahub version and features[​](#infrahub-version-and-features "Direct link to Infrahub version and features") Control which version of Infrahub to run and which features to enable: ``` # Infrahub version to use # Specify the version of Infrahub to run (for example, 1.5.1, stable, latest) # Default: 1.5.1 INFRAHUB_VERSION=1.5.1 # Enable Infrahub Enterprise features # Set to "true" to enable enterprise features, "false" for community edition # Default: false INFRAHUB_ENTERPRISE=false ``` ### Service catalog configuration[​](#service-catalog-configuration "Direct link to Service catalog configuration") Enable and configure the optional Service Catalog Streamlit application: ``` # Enable the Service Catalog Streamlit application # Set to "true" to start the service catalog container with 'uv run invoke start' # Set to "false" (default) to run only the core Infrahub services # The service catalog provides a web UI for managing data center deployments INFRAHUB_SERVICE_CATALOG=false # Default branch to use when the service catalog application starts # Default: main DEFAULT_BRANCH=main # Time to wait (in seconds) for the Infrahub generator to complete after creating a DC # Increase this value if your generators take longer to complete # Default: 60 GENERATOR_WAIT_TIME=60 # API request timeout in seconds # Maximum time to wait for API requests to complete # Default: 30 API_TIMEOUT=30 # Number of retry attempts for failed API requests # Default: 3 API_RETRY_COUNT=3 ``` ### Managing environment variables with direnv[​](#managing-environment-variables-with-direnv "Direct link to Managing environment variables with direnv") [direnv](https://direnv.net/) is a helpful tool for automatically loading environment variables from `.env` files when you enter the project directory. While not required, it streamlines the development workflow. To use direnv: 1. Install direnv following the instructions at [direnv.net](https://direnv.net/docs/installation.html) 2. Create a `.envrc` file in the project root: ``` dotenv ``` 3. Allow direnv to load the file: ``` direnv allow ``` Now your environment variables will load automatically when you `cd` into the project directory. tip Other alternatives for managing environment variables include: * Using your shell's profile file (`.bashrc`, `.zshrc`) * Docker Compose environment files (`docker-compose.override.yml`) * Shell scripts that source the `.env` file * IDE/editor plugins for environment variable management ## Step 5: start Infrahub[​](#step-5-start-infrahub "Direct link to Step 5: start Infrahub") Start the Infrahub containers using the included invoke task: ``` uv run invoke start ``` This command downloads and starts all required Docker containers. The first run may take several minutes as Docker images are pulled. Wait for all containers to be healthy. You can check the status with: ``` docker ps ``` You should see containers for: * `infrahub-server` - Main Infrahub API server * `infrahub-database` - PostgreSQL database * `infrahub-cache` - Redis cache * `infrahub-message-queue` - RabbitMQ message broker * `infrahub-git` - Git repository server ## Step 6: access the Infrahub web interface[​](#step-6-access-the-infrahub-web-interface "Direct link to Step 6: access the Infrahub web interface") Once the containers are running, open your web browser and navigate to: ``` http://localhost:8000 ``` You should see the Infrahub login screen. Use the default credentials: * **Username**: `admin` * **Password**: `infrahub` ### Demo user accounts[​](#demo-user-accounts "Direct link to Demo user accounts") After running the bootstrap process (Step 7), additional user accounts are created to demonstrate role-based access control. These accounts provide different levels of access to Infrahub: #### Emma - read-only user[​](#emma---read-only-user "Direct link to Emma - read-only user") * **Username**: `emma` * **Password**: `emma123` * **Access level**: Read-only * **Permissions**: * View all objects across all branches * Cannot create, update, or delete any objects * Cannot manage schemas or merge proposed changes **Use this account to**: * Demonstrate read-only access for auditors or viewers * Show how Infrahub prevents unauthorized modifications * Test queries and views without risk of changing data #### Otto - schema reviewer[​](#otto---schema-reviewer "Direct link to Otto - schema reviewer") * **Username**: `otto` * **Password**: `otto123` * **Access level**: Schema management and full object access * **Permissions**: * Manage schemas (create, update, delete) * Review and approve proposed changes * Full CRUD (create, read, update, delete) operations on all objects **Use this account to**: * Demonstrate schema management workflows * Show the proposed change review process * Test advanced configurations with full object permissions info These demo accounts are created automatically when you run `uv run invoke bootstrap` (Step 7). The account creation script is located at `scripts/create_users_roles.py`. Security notice The passwords for these demo accounts are intentionally weak for demonstration purposes. In production environments, use strong passwords and integrate with your organization's authentication system (LDAP, SAML, or OIDC). ## Step 7: bootstrap the demo data[​](#step-7-bootstrap-the-demo-data "Direct link to Step 7: bootstrap the demo data") Now that Infrahub is running, load the demo schemas and data. You can use the automated bootstrap script or run the steps manually. * Automated (Recommended) * Manual Run the bootstrap command to load everything in one step: ``` uv run invoke bootstrap ``` This command performs all the steps in the Manual tab automatically. Run each step individually: ``` # 1. Load schemas uv run infrahubctl schema load schemas # 2. Load menu definitions uv run infrahubctl menu load menus/menu-full.yml # 3. Load bootstrap data (locations, device types, platforms) uv run infrahubctl object load objects/bootstrap # 4. Load security data (optional) uv run infrahubctl object load objects/security/ # 5. Populate security relationships (optional, required if you loaded security data) uv run python scripts/populate_security_relationships.py # 6. Add the demo repository uv run infrahubctl repository add DEMO https://github.com/opsmill/infrahub-demo-dc.git --read-only --ref main # 7. Load event actions (optional, enables automatic generator execution) uv run infrahubctl object load objects/events/ ``` tip Loading schemas and data can take several minutes. Watch the console output to monitor progress. ## Step 8: verify the installation[​](#step-8-verify-the-installation "Direct link to Step 8: verify the installation") To verify that everything is set up correctly: 1. **Check the web interface** - Navigate to `http://localhost:8000` and log in 2. **Browse the schema** - Go to **Schema** in the left sidebar and verify that you see schemas like `DcimDevice`, `IpamIPPrefix`, `TopologyDataCenter` 3. **View bootstrap data** - Navigate to **Infrastructure → Locations** and verify you see sample locations 4. **Check generators** - Go to **Actions → Generator Definitions** and verify you see `create_dc` and `create_pop` generators ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Port conflicts[​](#port-conflicts "Direct link to Port conflicts") If port 8000 is already in use, you'll see an error when starting Infrahub. To resolve this: 1. Stop the conflicting service using port 8000 2. Or modify the port mapping in your Docker Compose configuration ### Dependency issues[​](#dependency-issues "Direct link to Dependency issues") If you encounter Python dependency errors: ``` # Clear the virtual environment and reinstall rm -rf .venv uv sync ``` ### Docker issues[​](#docker-issues "Direct link to Docker issues") If Docker containers fail to start: ``` # Stop and remove all containers uv run invoke destroy # Restart uv run invoke start ``` ### Schema loading fails[​](#schema-loading-fails "Direct link to Schema loading fails") If schema loading fails with validation errors, ensure you're using a compatible version of Infrahub. This demo is tested with Infrahub 1.5.1 and later. ## Next steps[​](#next-steps "Direct link to Next steps") Now that you have Infrahub running with the demo data loaded, you're ready to: * Follow the **[user walkthrough](/demo-dc/user-walkthrough.md)** to create your first data center topology * Explore the **[concepts](/demo-dc/concepts.md)** to understand the architecture * Review the **[developer guide](/demo-dc/developer-guide.md)** to understand how the demo works under the hood ## Managing your environment[​](#managing-your-environment "Direct link to Managing your environment") ### Stopping Infrahub[​](#stopping-infrahub "Direct link to Stopping Infrahub") To stop the containers without removing data: ``` uv run invoke stop ``` ### Destroying the environment[​](#destroying-the-environment "Direct link to Destroying the environment") To completely remove containers and volumes (fresh start): ``` uv run invoke destroy ``` ### Restarting a specific component[​](#restarting-a-specific-component "Direct link to Restarting a specific component") To restart a single container: ``` uv run invoke restart ``` For example, to restart the API server: ``` uv run invoke restart infrahub-server ``` --- # Working with security management This tutorial walks you through Infrahub's security management capabilities. You'll explore how security policies are modeled as structured data, how they relate to firewalls, and how Infrahub transforms security policy objects into vendor-specific firewall configurations. By the end of this tutorial, you'll understand how to manage firewall policies using Infrahub's security schema and generate deployment-ready configurations. ## What you'll accomplish[​](#what-youll-accomplish "Direct link to What you'll accomplish") In this tutorial, you will: 1. Explore the security management schema structure 2. Review the corp-firewall device and its security policy 3. Understand how security objects relate to each other 4. Generate firewall configuration from security policy data 5. Modify security rules and regenerate configurations ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this tutorial, ensure you have: * Completed the [installation guide](/demo-dc/install.md) and have Infrahub running * Loaded the bootstrap data and security data (`uv run invoke bootstrap`) * Access to the Infrahub web interface at `http://localhost:8000` info The security data is loaded as part of the bootstrap process. If you haven't loaded it yet, run `uv run infrahubctl object load objects/security/` followed by `uv run python scripts/populate_security_relationships.py` to populate the many-to-many relationships. ## Understanding the demo security schema[​](#understanding-the-demo-security-schema "Direct link to Understanding the demo security schema") The demo security management schema in this repository provides an example data model for firewall policies, zones, services, and applications. The schema is organized into several categories of security objects that work together to define complete security policies. info As Infrahub provides the user a flexible schema, this specific security schema implementation serves as an example of what is possible, not a finished solution. The value of Infrahub is the ability to create a specific schema to the needs of your organization. ### Security schema architecture[​](#security-schema-architecture "Direct link to Security schema architecture") **Key relationships:** * **SecurityPolicy** contains multiple **SecurityPolicyRule** objects * **SecurityPolicyRule** references zones, address groups, service groups, applications, and profiles * **SecurityPolicy** is applied to one or more **SecurityFirewall** devices * **SecurityAddressGroup** aggregates IP addresses, prefixes, ranges, and FQDNs * **SecurityServiceGroup** aggregates services and service ranges This hierarchical model separates concerns (addresses, services, applications) and allows reuse of security objects across multiple policies and rules. ## Step 1: explore the corp-firewall device[​](#step-1-explore-the-corp-firewall-device "Direct link to Step 1: explore the corp-firewall device") Navigate to the corp-firewall device in the Infrahub web interface: 1. Go to `http://localhost:8000` 2. Navigate to **Device Management → Infrastructure → Firewalls** 3. Click on **corp-firewall** ### Device details[​](#device-details "Direct link to Device details") The corp-firewall is a Juniper SRX-1500 firewall with the following configuration: * **Device type**: SRX-1500 * **Platform**: Juniper JunOS * **Role**: edge\_firewall * **Location**: PAR-1 (Paris building) * **Status**: active ### Interfaces[​](#interfaces "Direct link to Interfaces") The firewall has several interfaces defined: * **fxp0** - Management interface * **ge-0/0/0** through **ge-0/0/3** - Leaf connection interfaces These interfaces connect the firewall to the network infrastructure and provide connectivity for security policy enforcement. ## Step 2: explore security management objects[​](#step-2-explore-security-management-objects "Direct link to Step 2: explore security management objects") The security policy for corp-firewall is built from objects organized under the **Security Management** menu in the Infrahub UI. ### Navigate security objects[​](#navigate-security-objects "Direct link to Navigate security objects") 1. In the left sidebar, find **Security Management** 2. Explore the following sections: #### Security zones[​](#security-zones "Direct link to Security zones") Go to **Security Management → Security Zones** Security zones represent trust boundaries: * **untrust** - External/untrusted networks (trust level: 0) * **trust** - Internal/trusted networks (trust level: 100) * **dmz** - Demilitarized zone (trust level: 50) Zones are used in policy rules to define traffic flow direction (for example, from untrust to dmz). #### Address objects[​](#address-objects "Direct link to Address objects") Go to **Security Management → Address Objects** You'll find several types of address objects: * **SecurityIPAddress** - Individual IP addresses (for example, web-server-01: 10.100.1.10) * **SecurityPrefix** - Network prefixes (for example, internal-network: 10.0.0.0/8) * **SecurityAddressGroup** - Groups of addresses (for example, web-servers, internet) Address groups simplify policy management by allowing you to reference multiple addresses with a single object. #### Service objects[​](#service-objects "Direct link to Service objects") Go to **Security Management → Service Objects** Service objects define network protocols and ports: * **SecurityService** - Individual services (for example, https: TCP/443, dns: UDP/53) * **SecurityServiceGroup** - Groups of services (for example, web-services containing http and https) Service groups make it straightforward to create rules that allow multiple related services. #### Applications[​](#applications "Direct link to Applications") Go to **Security Management → Applications** Application objects enable application-aware security: * **web-browsing** - HTTP/HTTPS web traffic * **ssl** - SSL/TLS encrypted traffic * **dns** - Domain name resolution Applications can be used in policy rules for more granular control than port-based services. #### Security policies[​](#security-policies "Direct link to Security policies") Go to **Security Management → Security Policies** You'll see **corporate-firewall-policy** which contains the rules applied to corp-firewall. Click on the policy to view its details including: * **Name**: corporate-firewall-policy * **Description**: Security policy for corporate edge firewall * **Firewalls**: corp-firewall (the policy is applied to this device) * **Rules**: One or more policy rules defining allowed/denied traffic #### Policy rules[​](#policy-rules "Direct link to Policy rules") Within the corporate-firewall-policy, you'll find policy rules like **allow-web-traffic**: * **Index**: 10 (lower numbers are processed first) * **Action**: permit * **Source zone**: untrust (internet) * **Destination zone**: dmz * **Source addresses**: internet (address group) * **Destination addresses**: web-servers (address group) * **Services**: web-services (service group containing http/https) * **Applications**: web-browsing * **Log**: true (log matching traffic) This rule allows internet users to access web servers in the DMZ using HTTP/HTTPS protocols. ## Step 3: understand the policy structure[​](#step-3-understand-the-policy-structure "Direct link to Step 3: understand the policy structure") The security policy demonstrates Infrahub's composable approach to policy management: ``` SecurityPolicy (corporate-firewall-policy) │ ├─ Applied to: corp-firewall │ └─ Contains Rules: │ └─ PolicyRule (allow-web-traffic) ├─ Source Zone: untrust ├─ Destination Zone: dmz ├─ Source Addresses: internet (AddressGroup) │ └─ Contains: 0.0.0.0/0 (SecurityPrefix) ├─ Destination Addresses: web-servers (AddressGroup) │ ├─ web-server-01 (SecurityIPAddress: 10.100.1.10) │ └─ web-server-02 (SecurityIPAddress: 10.100.1.20) ├─ Services: web-services (ServiceGroup) │ ├─ http (SecurityService: TCP/80) │ └─ https (SecurityService: TCP/443) ├─ Applications: web-browsing ├─ Action: permit └─ Log: true ``` **Key benefits of this structure:** * **Reusability** - Address groups, service groups, and applications can be reused across multiple rules * **Maintainability** - Update an address group once, and all rules using it automatically reflect the change * **Clarity** - Named objects make policies more readable than raw IP addresses and port numbers * **Validation** - Infrahub validates relationships ensuring policies reference valid objects ## Step 4: view the generated firewall configuration[​](#step-4-view-the-generated-firewall-configuration "Direct link to Step 4: view the generated firewall configuration") Infrahub transforms the security policy data into vendor-specific firewall configurations using templates and transforms. ### Access the firewall configuration artifact[​](#access-the-firewall-configuration-artifact "Direct link to Access the firewall configuration artifact") 1. While viewing **corp-firewall**, look for the **Artifacts** section or tab 2. You should see an artifact named **edge** or **firewall-configuration** 3. Click on the artifact to view the generated Juniper JunOS configuration ### Configuration sections[​](#configuration-sections "Direct link to Configuration sections") The generated configuration includes several sections: **Security zones:** ``` security { zones { security-zone untrust { interfaces { ge-0/0/0.0; } } security-zone dmz { interfaces { ge-0/0/1.0; } } security-zone trust { interfaces { ge-0/0/2.0; } } } } ``` **Address book entries:** ``` security { address-book { global { address web-server-01 10.100.1.10/32; address web-server-02 10.100.1.20/32; address-set web-servers { address web-server-01; address web-server-02; } } } } ``` **Security policies:** ``` security { policies { from-zone untrust to-zone dmz { policy allow-web-traffic { match { source-address internet; destination-address web-servers; application [ junos-http junos-https ]; } then { permit; log { session-init; session-close; } } } } } } ``` The configuration is generated from the structured security policy data using Jinja2 templates that understand Juniper JunOS syntax. ## Step 5: modify a security policy[​](#step-5-modify-a-security-policy "Direct link to Step 5: modify a security policy") Let's modify the security policy to add a new rule. ### Create a new policy rule[​](#create-a-new-policy-rule "Direct link to Create a new policy rule") 1. Navigate to **Security Management → Security Policy Rules** 2. Click **Add Security Policy Rule** 3. Fill in the form: * **Index**: 20 (processed after rule 10) * **Name**: allow-dns * **Policy**: corporate-firewall-policy * **Source Zone**: untrust * **Destination Zone**: dmz * **Source Addresses**: internet * **Destination Addresses**: web-servers * **Services**: (create or select a DNS service group) * **Action**: permit * **Log**: true 4. Click **Save** ### Regenerate the configuration[​](#regenerate-the-configuration "Direct link to Regenerate the configuration") After creating the rule: 1. Create a proposed change to review the impact 2. Navigate to **Proposed Changes** 3. Create a new proposed change comparing your branch to main 4. View the **Artifacts** tab to see the updated firewall configuration 5. The configuration should now include the new DNS rule tip Always use proposed changes when modifying security policies. This allows you to review the generated configuration changes before deploying to production firewalls. ## Step 6: understand the transformation process[​](#step-6-understand-the-transformation-process "Direct link to Step 6: understand the transformation process") The transformation from security policy objects to firewall configuration follows this flow: ``` Security Policy Data (Infrahub) ↓ GraphQL Query (fetch policy, rules, zones, addresses, services) ↓ Python Transform (transforms/edge.py) ↓ Jinja2 Template (templates/edge.j2) ↓ Juniper JunOS Configuration (artifact) ``` ### The transform[​](#the-transform "Direct link to The transform") The transform (`transforms/edge.py`) queries Infrahub for: * Device details * Security zones * Address objects and groups * Service objects and groups * Policy rules * Applications It then processes this data into a structure suitable for template rendering. ### The template[​](#the-template "Direct link to The template") The template (`templates/edge.j2`) contains Juniper JunOS-specific syntax and logic: * Iterates over zones to create zone configurations * Builds address books from address groups * Generates policy rules from SecurityPolicyRule objects * Maps Infrahub applications to Junos applications (for example, web-browsing → junos-http/junos-https) **Vendor flexibility:** The same security policy data could be transformed to different vendors by creating different templates: * Juniper JunOS (current implementation) * Cisco ASA * Palo Alto PAN-OS * Fortinet FortiGate This demonstrates Infrahub's vendor-neutral approach - the security policy is defined once and can be rendered for any supported platform. ## What you've learned[​](#what-youve-learned "Direct link to What you've learned") Congratulations! You've explored Infrahub's security management capabilities: * ✅ Understood the security schema architecture * ✅ Explored security zones, addresses, services, and applications * ✅ Reviewed the corp-firewall device and its security policy * ✅ Examined the relationship between policy objects * ✅ Viewed generated Juniper JunOS firewall configurations * ✅ Modified security policies and regenerated configurations * ✅ Understood the transformation process from data to configuration ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Creating complex policies[​](#creating-complex-policies "Direct link to Creating complex policies") Build more sophisticated security policies by: * **Combining multiple address groups** in a single rule * **Using application objects** for layer 7 filtering * **Adding URL categories** for web filtering * **Applying security profiles** for IPS/antivirus scanning * **Creating time-based rules** with schedules ### Multi-firewall deployments[​](#multi-firewall-deployments "Direct link to Multi-firewall deployments") Apply the same security policy to multiple firewalls: 1. Create additional SecurityFirewall objects 2. Associate them with the same SecurityPolicy 3. Generate configurations for each firewall 4. Maintain consistency across your firewall fleet ### Policy validation[​](#policy-validation "Direct link to Policy validation") Use Infrahub's validation capabilities to ensure: * No overlapping rules with conflicting actions * All referenced objects exist (addresses, services, applications) * Required fields are populated * Trust levels between zones are logical ### Integration with CI/CD[​](#integration-with-cicd "Direct link to Integration with CI/CD") Automate security policy management: 1. Store security objects in Git repositories 2. Use proposed changes for peer review 3. Run validation checks in CI pipelines 4. Deploy approved configurations automatically ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Security objects not appearing[​](#security-objects-not-appearing "Direct link to Security objects not appearing") If you don't see security objects in the UI: 1. Verify security schemas are loaded: `uv run infrahubctl schema load schemas` 2. Load security data: `uv run infrahubctl object load objects/security/` 3. Populate relationships: `uv run python scripts/populate_security_relationships.py` 4. Refresh your browser ### Configuration not generating[​](#configuration-not-generating "Direct link to Configuration not generating") If firewall configurations aren't being generated: 1. Check that the SecurityPolicy is associated with the SecurityFirewall 2. Verify the transform and template are registered in `.infrahub.yml` 3. Look for errors in the Infrahub task logs 4. Ensure all referenced objects (zones, addresses, services) exist ### Many-to-many relationships missing[​](#many-to-many-relationships-missing "Direct link to Many-to-many relationships missing") If address groups or service groups appear empty: 1. Run the populate\_security\_relationships script 2. This script manually creates the many-to-many relationships that can't be loaded via YAML 3. Verify in the UI that groups now show their members ## Next steps[​](#next-steps "Direct link to Next steps") Now that you understand security management in Infrahub, you can: * **Model your organization's security policies** using the security schema * **Create vendor-neutral firewall policies** that can be rendered for any platform * **Implement change management workflows** using branches and proposed changes * **Integrate with automation tools** to deploy configurations automatically * **Extend the schema** with custom security objects specific to your needs For deeper understanding: * **[Understanding the concepts](/demo-dc/concepts.md)** - Learn about schema-driven data modeling * **[Developer guide](/demo-dc/developer-guide.md)** - Understand how transformations and templates work * **[Infrahub security schema](https://github.com/opsmill/infrahub-demo-dc/blob/main/schemas/extensions/security/security.yml)** - Review the complete schema definition --- # Using the service catalog This tutorial walks you through using Infrahub's Service Catalog web interface to manage data center and colocation center deployments. The Service Catalog provides a user-friendly web application built with Streamlit that simplifies infrastructure provisioning by automating branch creation, data loading, and generator execution. By the end of this tutorial, you'll understand how to enable the Service Catalog, navigate the interface, view existing infrastructure, and create new data centers through a guided form-based workflow. ## What you'll accomplish[​](#what-youll-accomplish "Direct link to What you'll accomplish") In this tutorial, you will: 1. Enable the Service Catalog container 2. Access the web interface 3. Switch between Infrahub branches 4. View existing data centers and colocation centers 5. Visualize rack layouts with device placement 6. Create a new data center using the web form 7. Understand the automated workflow behind the scenes ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this tutorial, ensure you have: * Completed the [installation guide](/demo-dc/install.md) and have Infrahub running * Completed the [user walkthrough](/demo-dc/user-walkthrough.md) tutorial * Docker and Docker Compose installed and configured * Access to modify environment variables or Docker Compose configuration tip The Service Catalog is an optional component that simplifies infrastructure provisioning. It's particularly useful for teams who want a web-based interface for infrastructure requests. ## Step 1: enable the service catalog[​](#step-1-enable-the-service-catalog "Direct link to Step 1: enable the service catalog") The Service Catalog runs as a separate Docker container that can be enabled through environment variables or Docker Compose profiles. * Environment Variables (Recommended) * Docker Compose Profiles 1. Open your `.env` file (or create it from `.env.example`) 2. Set the Service Catalog variable to true: ``` INFRAHUB_SERVICE_CATALOG=true ``` 3. Restart Infrahub to apply the changes: ``` uv run invoke restart ``` Start Infrahub with the Service Catalog profile: ``` docker-compose --profile service-catalog up -d ``` ### Verify the container is running[​](#verify-the-container-is-running "Direct link to Verify the container is running") Check that the service-catalog container is running: ``` docker ps --filter "name=service-catalog" ``` You should see a container named `infrahub-demo-dc-service-catalog-1` or similar in the output. ## Step 2: access the service catalog[​](#step-2-access-the-service-catalog "Direct link to Step 2: access the service catalog") Once the container is running, access the Service Catalog web interface: 1. Open your web browser 2. Navigate to `http://localhost:8501` 3. The Service Catalog landing page should load The interface displays: * **Branch selector** - Dropdown to switch between Infrahub branches * **Data centers** - List of existing TopologyDataCenter objects * **Colocation centers** - List of existing TopologyColocationCenter objects * **Create new data center** - Button to access the DC creation form info If the page doesn't load, verify the container is running and check the logs with `docker logs infrahub-demo-dc-service-catalog-1`. ## Step 3: switch branches and view infrastructure[​](#step-3-switch-branches-and-view-infrastructure "Direct link to Step 3: switch branches and view infrastructure") The Service Catalog allows you to view infrastructure across different Infrahub branches. ### Switch branches[​](#switch-branches "Direct link to Switch branches") 1. Locate the **branch selector** dropdown at the top of the page 2. Click the dropdown to see all available branches 3. Select a branch (for example, main, add-dc3, etc.) 4. The page will refresh and display infrastructure from the selected branch ### View existing infrastructure[​](#view-existing-infrastructure "Direct link to View existing infrastructure") After selecting a branch, you'll see two lists: #### Data centers[​](#data-centers "Direct link to Data centers") Each data center card displays: * Data center name (for example, DC-1, DC-2, DC-3) * Location information * Link to view in Infrahub UI #### Colocation centers[​](#colocation-centers "Direct link to Colocation centers") Each colocation center card displays: * Colocation center name (for example, PAR-1, LON-1) * Location information * Link to view in Infrahub UI tip The lists automatically update when you switch branches, allowing you to see infrastructure in different states (main, proposed changes, feature branches). ## Step 4: visualize rack layouts[​](#step-4-visualize-rack-layouts "Direct link to Step 4: visualize rack layouts") The Service Catalog includes a rack visualization page that displays physical rack layouts with mounted devices, similar to NetBox's rack elevation diagrams. This feature helps you understand device placement, rack occupancy, and physical infrastructure organization. ### Access rack visualization[​](#access-rack-visualization "Direct link to Access rack visualization") 1. Navigate to the sidebar in the Service Catalog 2. Click **Rack Visualization** in the navigation menu 3. The rack visualization page will load ### Understanding the rack view[​](#understanding-the-rack-view "Direct link to Understanding the rack view") The rack visualization displays: * **Physical rack diagrams** - Visual representations of racks with numbered rack units (U1-U42) * **Mounted devices** - Colored rectangles showing device positions and heights * **Device information** - Names, types, and rack unit positions * **Role-based colors** - Visual differentiation by device role Network devices and hypervisor hosts both appear here. A host shows in the hypervisor colour and at its real height, so a 2U Dell PowerEdge R760 occupies two units while a 1U R650 occupies one. Hosts are placed by the `cable_virtualization_host` generator rather than by hand, so they appear only after the virtualization data has loaded and that generator has run. See the [virtual machine management tutorial](/demo-dc/virtualization.md) for how placement works. ### Select a location row[​](#select-a-location-row "Direct link to Select a location row") Before racks appear, you need to select a location row: 1. Use the **Select Location Row** dropdown at the top of the page 2. Choose a row (for example, Row-1 in Pod-1) 3. All racks in that row will display in a grid layout info Location rows are created automatically by the DC generator. Each data center has a Pod (for example, Pod-1) containing a Row (for example, Row-1) with multiple racks. ### Reading rack diagrams[​](#reading-rack-diagrams "Direct link to Reading rack diagrams") Each rack diagram shows: #### Rack units[​](#rack-units "Direct link to Rack units") * Numbered from bottom (U1) to top (U42) * Each unit represents 1.75 inches of vertical space * Devices mount at specific U positions #### Device rectangles[​](#device-rectangles "Direct link to Device rectangles") Devices appear as colored rectangles with: * **Height** - 1U, 2U, 4U devices shown proportionally * **Position** - Starting U position labeled (for example, U42, U41) * **Name or type** - Text showing device identifier * **Role color** - Background color indicating device function For 1U devices, only the device name or type is shown. For multi-unit devices (2U+), both the primary label and secondary information are displayed. #### Color coding by role[​](#color-coding-by-role "Direct link to Color coding by role") The rack visualization uses role-based colors to help identify device types at a glance: * **Green** - Leaf switches (access layer) * **Blue** - Spine switches (aggregation layer) * **Purple** - Border leaf switches (edge connectivity) * **Orange** - Console servers (out-of-band management) * **Brown** - OOB switches (management network) * **Cyan** - Edge routers (WAN connectivity) * **Red** - Firewalls (security devices) A legend at the bottom of the page explains the color mappings. ### Toggle device labels[​](#toggle-device-labels "Direct link to Toggle device labels") You can change what information displays on device rectangles: 1. Locate the **Display Options** section in the sidebar 2. Use the **Device Label** dropdown 3. Choose between: * **Hostname** (default) - Shows device name (for example, dc-3-leaf-01) * **Device Type** - Shows hardware model (for example, Arista DCS-7280SR-48C6) For multi-unit devices, the secondary label shows the alternate information. For instance, if you select "Device Type", the secondary label shows the hostname. ### Device placement logic[​](#device-placement-logic "Direct link to Device placement logic") The DC generator assigns devices to racks based on predefined rules: #### Leaf devices[​](#leaf-devices "Direct link to Leaf devices") * Assigned to rack numbers matching their device number * For example, dc-3-leaf-01 → Rack-1, dc-3-leaf-02 → Rack-2 * Positioned at the top of the rack (U42 for 1U devices) #### Infrastructure devices[​](#infrastructure-devices "Direct link to Infrastructure devices") Spine switches, border leafs, console servers, and OOB devices are assigned to middle racks: * **Spine switches** - Positioned below leaf devices * **Border leaf switches** - Positioned below spines * **Console servers** - Positioned below border leafs * **OOB switches** - Positioned at the bottom This placement strategy ensures infrastructure devices are centrally located for optimal cabling. ### Clickable device links[​](#clickable-device-links "Direct link to Clickable device links") Each device in the rack diagram is clickable: 1. Click any device rectangle 2. A new browser tab opens showing the device in Infrahub UI 3. View full device details, interfaces, and relationships The link preserves the current branch context, so you see the device as it exists in the selected branch. ### Example workflow[​](#example-workflow "Direct link to Example workflow") Here's how to use rack visualization after creating a data center: 1. Create a new DC using the Service Catalog form (for example, DC-3) 2. Wait for the generator to complete 3. Navigate to **Rack Visualization** 4. Switch to the branch containing your DC (for example, add-dc3) 5. Select the location row (for example, Row-1) 6. View the generated racks with mounted devices 7. Verify device placement matches your design 8. Click devices to inspect details in Infrahub tip Rack visualization is particularly useful during proposed change review. Switch to the branch, view rack layouts, and verify devices are positioned correctly before merging to main. ## Step 5: create a new data center[​](#step-5-create-a-new-data-center "Direct link to Step 5: create a new data center") The Service Catalog provides a guided form for creating new data centers. ### Access the creation form[​](#access-the-creation-form "Direct link to Access the creation form") 1. Click the **Create New Data Center** button 2. The form page will load with several input fields ### Fill in the form[​](#fill-in-the-form "Direct link to Fill in the form") Complete each field in the form: #### 1. Data center name[​](#1-data-center-name "Direct link to 1. Data center name") * Enter a unique name for your data center (for example, DC-6) * This will be used in the branch name and as the DC identifier #### 2. Location[​](#2-location "Direct link to 2. Location") * Select a location from the dropdown * Options include existing LocationBuilding objects (for example, PAR-1, LON-1) * The location determines where the data center will be deployed #### 3. Provider[​](#3-provider "Direct link to 3. Provider") * Select a provider from the dropdown * Options include existing ProviderNetworkProvider objects * The provider supplies connectivity and services #### 4. Design[​](#4-design "Direct link to 4. Design") * Select a design template from the dropdown * Options include predefined data center architectures * Different designs support different vendors and topologies #### 5. Management subnet[​](#5-management-subnet "Direct link to 5. Management subnet") * Select an IP prefix for management interfaces * Used for out-of-band device management * Choose an available /24 or appropriate size #### 6. Underlay subnet[​](#6-underlay-subnet "Direct link to 6. Underlay subnet") * Select an IP prefix for underlay routing * Used for spine-leaf physical connectivity * Typically a /24 or /16 depending on DC size #### 7. Overlay subnet[​](#7-overlay-subnet "Direct link to 7. Overlay subnet") * Select an IP prefix for overlay services * Used for VXLAN and EVPN addressing * Usually a /16 or larger for tenant networks ### Submit the request[​](#submit-the-request "Direct link to Submit the request") 1. Review all form fields for accuracy 2. Click the **Submit** button 3. The Service Catalog will display a progress indicator ## Step 6: understand the behind-the-scenes workflow[​](#step-6-understand-the-behind-the-scenes-workflow "Direct link to Step 6: understand the behind-the-scenes workflow") When you submit the form, the Service Catalog executes an automated workflow that: ### 1. Creates a branch[​](#1-creates-a-branch "Direct link to 1. Creates a branch") The Service Catalog creates a new branch in Infrahub with the naming pattern `add-{dc_name}`: ``` Branch name: add-dc6 (if you named your DC "DC-6") ``` This branch isolates your new infrastructure from the main branch, allowing review before merging. ### 2. Loads data to the branch[​](#2-loads-data-to-the-branch "Direct link to 2. Loads data to the branch") The Service Catalog constructs a DesignTopologyDesign YAML object with your form inputs and loads it to the new branch: ``` nodes: - kind: DesignTopologyDesign data: name: DC-6 location: PAR-1 provider: provider-1 design_template: dc-design-template-1 mgmt_prefix: 10.0.6.0/24 underlay_prefix: 192.168.6.0/24 overlay_prefix: 172.16.0.0/16 ``` ### 3. Waits for the generator[​](#3-waits-for-the-generator "Direct link to 3. Waits for the generator") After loading data, the Service Catalog waits for Infrahub's generator to execute: * Default wait time: 60 seconds (configurable via `GENERATOR_WAIT_TIME`) * The generator (for example, `create_dc`) processes the design data * Devices, interfaces, cables, and IP addresses are automatically created ### 4. Creates a proposed change[​](#4-creates-a-proposed-change "Direct link to 4. Creates a proposed change") Once the generator completes, the Service Catalog creates a Proposed Change in Infrahub: * Compares the new branch to the main branch * Shows all differences (new devices, interfaces, configurations) * Allows review before merging to production ### View the results[​](#view-the-results "Direct link to View the results") After the workflow completes: 1. Navigate to **Proposed Changes** in the Infrahub UI (`http://localhost:8000`) 2. Find the proposed change for your DC (for example, "Add DC-6") 3. Review the **Data** tab to see new objects 4. Review the **Artifacts** tab to see generated configurations 5. Approve and merge when ready tip This workflow demonstrates infrastructure-as-code principles: declarative design data is transformed into imperative implementation through automated generators. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Service catalog container not starting[​](#service-catalog-container-not-starting "Direct link to Service catalog container not starting") If the container fails to start: 1. Check Docker logs: ``` docker logs infrahub-demo-dc-service-catalog-1 ``` 2. Verify environment variables in `.env`: ``` INFRAHUB_SERVICE_CATALOG=true INFRAHUB_ADDRESS=http://infrahub-server:8000 ``` 3. Ensure Infrahub server is running and accessible: ``` docker ps --filter "name=infrahub-server" ``` ### Cannot access `http://localhost:8501`[​](#cannot-access-httplocalhost8501 "Direct link to cannot-access-httplocalhost8501") If the web interface doesn't load: 1. Verify port 8501 is not in use by another application 2. Check container port mapping: ``` docker ps --format "table {{.Names}}\t{{.Ports}}" --filter "name=service-catalog" ``` 3. Try accessing via container IP directly: ``` docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' infrahub-demo-dc-service-catalog-1 ``` ### Branch selector is empty[​](#branch-selector-is-empty "Direct link to Branch selector is empty") If no branches appear in the dropdown: 1. Verify Infrahub server connection from the container: ``` docker exec infrahub-demo-dc-service-catalog-1 curl http://infrahub-server:8000/api/schema ``` 2. Check that branches exist in Infrahub: ``` uv run infrahubctl branch list ``` 3. Review Service Catalog logs for API errors: ``` docker logs infrahub-demo-dc-service-catalog-1 | grep -i error ``` ### Generator timeout errors[​](#generator-timeout-errors "Direct link to Generator timeout errors") If DC creation fails with timeout errors: 1. Increase the wait time in `.env`: ``` GENERATOR_WAIT_TIME=120 # Increase to 120 seconds ``` 2. Check generator status in Infrahub UI under **Task Management** 3. Verify generator definition is registered in `.infrahub.yml` ### Form submission fails[​](#form-submission-fails "Direct link to Form submission fails") If the submit button doesn't work: 1. Check browser console for JavaScript errors (F12 → Console) 2. Verify all required fields are filled 3. Check that selected options (location, provider, design) exist in Infrahub 4. Review Service Catalog logs during submission: ``` docker logs -f infrahub-demo-dc-service-catalog-1 ``` ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Custom default branch[​](#custom-default-branch "Direct link to Custom default branch") Change the default branch shown in the branch selector: ``` # In .env DEFAULT_BRANCH=production ``` ### Adjust timeouts[​](#adjust-timeouts "Direct link to Adjust timeouts") Tune timeouts for your environment: ``` # In .env GENERATOR_WAIT_TIME=90 # Generator execution wait API_TIMEOUT=30 # API request timeout API_RETRY_COUNT=3 # API retry attempts ``` ### Integration with CI/CD[​](#integration-with-cicd "Direct link to Integration with CI/CD") The Service Catalog workflow can be automated via API: 1. POST form data to Service Catalog endpoint 2. Wait for branch creation 3. Run validation checks 4. Auto-approve proposed changes if checks pass This enables self-service infrastructure provisioning with approval gates. ## What you've learned[​](#what-youve-learned "Direct link to What you've learned") Congratulations! You've explored Infrahub's Service Catalog: * ✅ Enabled the Service Catalog container * ✅ Accessed the web interface at `http://localhost:8501` * ✅ Switched between branches to view different infrastructure states * ✅ Viewed existing data centers and colocation centers * ✅ Visualized rack layouts with color-coded device placement * ✅ Created a new data center using the guided form * ✅ Understood the automated workflow (branch → data → generator → proposed change) ## Next steps[​](#next-steps "Direct link to Next steps") Now that you understand the Service Catalog, you can: * **Customize the form** - Modify `service_catalog/pages/create_dc.py` to add fields * **Add validation** - Implement checks before submission * **Extend to other infrastructure types** - Create forms for POPs, segments, or services * **Integrate with approval workflows** - Connect to ticketing or change management systems * **Build custom dashboards** - Add Streamlit pages for monitoring and reporting For deeper understanding: * **[Developer guide](/demo-dc/developer-guide.md)** - Learn about generators and transforms * **[Containerlab deployment](/demo-dc/containerlab-deployment.md)** - Test generated configurations in virtual labs * **[Security management](/demo-dc/security-management.md)** - Manage firewall policies through Infrahub --- # User walkthrough This tutorial guides you through the complete workflow of creating a data center topology using Infrahub's design-driven automation. You'll create a new branch, load a topology design, run a generator to create infrastructure, review the changes, and merge them to the main branch. By the end of this walkthrough, you'll understand how to use Infrahub's branch-based workflow to safely design, validate, and deploy network infrastructure. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this tutorial, ensure you have: * Completed the [installation guide](/demo-dc/install.md) and have Infrahub running * Loaded the bootstrap data and schemas * Access to the Infrahub web interface at `http://localhost:8000` ## What you'll accomplish[​](#what-youll-accomplish "Direct link to What you'll accomplish") In this tutorial, you will: 1. Create a new branch for your changes 2. Load a data center design (DC-3) 3. Wait for the automatic generator to create devices, interfaces, and connections 4. Create a proposed change 5. Review the generated artifacts and validation results 6. Merge the changes to the main branch ## Step 1: Create a new branch[​](#step-1-create-a-new-branch "Direct link to Step 1: Create a new branch") All changes in Infrahub happen in branches. This allows you to work on infrastructure changes safely without affecting the main branch until you're ready. * Web Interface * CLI 1. Navigate to `http://localhost:8000` 2. Click on the branch selector in the top-right corner (shows "main" by default) 3. Click **Create Branch** 4. Enter branch name: `add-dc3` 5. Click **Create** You should see a success message confirming the branch was created. Create the branch from the command line: ``` uv run infrahubctl branch create add-dc3 ``` You should see a success message confirming the branch was created. tip Branch names should be descriptive and indicate what changes you're making, like `add-dc3`, `update-security-policies`, or `fix-bgp-config`. ## Step 2: Switch to your new branch[​](#step-2-switch-to-your-new-branch "Direct link to Step 2: Switch to your new branch") In the web interface: 1. Click the branch selector in the top-right 2. Select `add-dc3` from the list 3. The interface should now show `add-dc3` as the active branch info All subsequent actions will be performed on the `add-dc3` branch until you switch back to another branch. ## Step 3: load the DC-3 design[​](#step-3-load-the-dc-3-design "Direct link to Step 3: load the DC-3 design") The DC-3 design is a pre-configured data center topology specification that defines: * A data center named "DC-3" * Spine-leaf fabric architecture * Device roles and platform assignments * IP address pools and subnets * BGP autonomous system configuration Load the design data onto your branch: ``` uv run infrahubctl object load objects/dc/dc-arista-s.yml --branch add-dc3 ``` Watch the output for success messages. You should see confirmations that the topology data has been loaded. ### Verify the design loaded[​](#verify-the-design-loaded "Direct link to Verify the design loaded") In the web interface: 1. Ensure you're on the `add-dc3` branch 2. Navigate to **Network Topologies → Deployments → Data center** 3. You should see a new entry for **DC-3** 4. Click on DC-3 to view its properties The design specifies high-level parameters like: * Data center name and location * Number of spines and leaves * Underlay routing protocol (eBGP or OSPF) * Addressing schemes ### Multi-vendor support[​](#multi-vendor-support "Direct link to Multi-vendor support") This demo supports multiple network device vendors with pre-configured data center designs. The DC-3 topology you just loaded uses Arista EOS switches, but the demo can generate configurations for several other vendors. **Available vendor designs:** * **Arista EOS** - The default DC-3 design with Arista switches (`objects/dc/dc-arista-s.yml`) * **Cisco NX-OS** - The DC-2 design (`objects/dc/dc-cisco-s.yml`) * **Juniper JunOS** - DC-5 design with Juniper switches * **SONiC NOS** - DC-4 design with EdgeCore switches running SONiC **Using alternative vendor designs:** To explore Arista EOS configurations, you can use the automated demo command: ``` uv run invoke demo-dc-arista ``` This command creates a branch, loads the Arista design, waits for generator completion, loads the virtualization demo data (see the [virtual machine management tutorial](/demo-dc/virtualization.md)), then creates a proposed change. For other vendors, load the design files directly: ``` # Juniper design (DC-5) uv run infrahubctl object load objects/dc/dc-juniper-s.yml --branch add-dc5 # SONiC design (DC-4) uv run infrahubctl object load objects/dc/dc-sonic-border-leafs.yml --branch add-dc4 ``` Each vendor design generates appropriate configurations for that platform's CLI syntax and features. The same workflow (branch creation, design loading, generator execution, proposed change, merge) applies regardless of vendor. ## Step 4: Wait for automatic generator execution[​](#step-4-wait-for-automatic-generator-execution "Direct link to Step 4: Wait for automatic generator execution") When you loaded the DC-3 design in Step 3, Infrahub automatically triggered a generator action that transforms the abstract design into concrete infrastructure objects including devices, interfaces, IP addresses, and routing configurations. This automation is configured through an event action (defined in `objects/events/98_generator_action.yml`) that runs the `create_dc` generator whenever a data center topology is created or updated. ### Monitor generator progress[​](#monitor-generator-progress "Direct link to Monitor generator progress") Generator execution can take several minutes as it creates hundreds of objects. Monitor the task status: 1. Navigate to **System → Tasks** in the left sidebar 2. Find your generator task in the list (look for `create_dc`) 3. Watch the status change from "Pending" to "Running" to "Completed" info Generator actions are event-driven automations configured in Infrahub. When you load a data center topology object, the event system automatically triggers the `create_dc` generator. This eliminates the need for manual generator execution and ensures consistency across deployments. tip Generators create infrastructure following best practices. For DC-3, the create\_dc generator will create spine switches, leaf switches, border leafs, management connections, fabric peering, loopback interfaces, and BGP configurations. ## Step 5: Review generated infrastructure[​](#step-5-review-generated-infrastructure "Direct link to Step 5: Review generated infrastructure") Once the generator completes, explore what was created: ### View devices[​](#view-devices "Direct link to View devices") 1. Navigate to **Device Management → Infrastructure → All Devices** 2. You should see multiple devices created: * Spine switches (typically 2 or more) * Leaf switches (typically 4 or more) * Border leaf switches (typically 2) ### View interfaces[​](#view-interfaces "Direct link to View interfaces") Click on any device to see its details including: * Physical and loopback interfaces * IP addresses assigned to each interface * Interface descriptions and roles ### View IP prefixes[​](#view-ip-prefixes "Direct link to View IP prefixes") Navigate to **IP Address Management → IP Prefixes** to see: * Management network prefixes * Loopback prefixes * Underlay point-to-point link prefixes ### View routing[​](#view-routing "Direct link to View routing") Navigate to **Routing** sections to see: * BGP peer groups * BGP sessions between spines and leaves * Autonomous system assignments ## Step 6: Create a proposed change[​](#step-6-create-a-proposed-change "Direct link to Step 6: Create a proposed change") Now that you've made changes on your branch, create a proposed change to merge them into main. Proposed changes provide a review mechanism with diffs, validation checks, and artifact generation. ### Create the proposed change[​](#create-the-proposed-change "Direct link to Create the proposed change") 1. Navigate to **Proposed Changes** 2. Click **New Proposed Change** 3. Fill in the form: * **Name**: `Add DC-3 data center` * **Source Branch**: `add-dc3` * **Destination Branch**: `main` * **Description** (optional): `This change adds the DC-3 data center with spine-leaf fabric` 4. Click **Create Proposed Change** ### Understanding what happens next[​](#understanding-what-happens-next "Direct link to Understanding what happens next") When you create a proposed change, Infrahub automatically: * Creates a diff showing all objects added, modified, or deleted * Runs validation checks defined in your checks * Regenerates all artifacts affected by the changes * Runs repository checks if configured ## Step 7: Review validations and artifacts[​](#step-7-review-validations-and-artifacts "Direct link to Step 7: Review validations and artifacts") Navigate to your proposed change to review the results: ### View the diff[​](#view-the-diff "Direct link to View the diff") 1. Click on the **Files** or **Diff** tab 2. Review all the objects that will be created when you merge 3. You should see hundreds of additions including: * Devices * Interfaces * IP addresses * BGP configurations * Routing protocols ### Check validation results[​](#check-validation-results "Direct link to Check validation results") 1. Click on the **Checks** or **Validations** tab 2. Review the validation results 3. All checks should pass with a green checkmark 4. If any checks fail, review the error messages and fix issues before merging Common validations include: * **Spine connectivity checks** - Verify all spines are properly connected * **Leaf connectivity checks** - Verify all leaves are connected to all spines * **IP address uniqueness** - Ensure no duplicate IP assignments * **BGP configuration** - Validate BGP peer groups and sessions ### View generated artifacts[​](#view-generated-artifacts "Direct link to View generated artifacts") 1. Click on the **Artifacts** tab 2. You should see generated artifacts including: * **Device configurations** (one per device) * Spine, leaf, and border-leaf configurations in native vendor CLI format * For Arista and Juniper leaf switches, additional OpenConfig-formatted configurations demonstrating multi-format output * **Topology cabling matrix** (CSV format) - Complete cabling plan with source/destination interfaces * **Containerlab topology file** - Virtual lab topology definition for testing configurations Click on any artifact to view its content. For example, view a spine configuration to see the generated CLI commands. **Multi-format configuration rendering:** Infrahub's transformation system can generate multiple configuration formats from the same source data. For Arista EOS and Juniper JunOS leaf switches, you'll find both: * **Native CLI format** - Vendor-specific command syntax ready for deployment * **OpenConfig format** - Standardized, vendor-neutral YANG-based configuration This demonstrates how a single data model in Infrahub can produce multiple output formats, enabling flexibility in deployment methods (traditional CLI, NETCONF, gNMI, etc.) without duplicating source data. **Infrastructure artifacts:** Beyond device configurations, the generator creates supporting artifacts: * **CSV cabling matrix** - Documents physical connectivity between all devices and interfaces, useful for operations teams during installation and troubleshooting * **Containerlab topology file** - Enables spinning up virtual lab environments that mirror your production topology for testing and validation tip Artifacts are regenerated automatically when you create a proposed change. This ensures they always reflect the latest data. ## Step 8: Merge the proposed change[​](#step-8-merge-the-proposed-change "Direct link to Step 8: Merge the proposed change") Once you've reviewed everything and all validations pass, you're ready to merge: 1. In the proposed change view, click the **Merge** button 2. Confirm the merge in the dialog 3. Wait for the merge task to complete The merge process: * Applies all changes from the source branch to the destination branch * Runs any post-merge hooks or actions * Updates the main branch with your new infrastructure ### Verify the merge[​](#verify-the-merge "Direct link to Verify the merge") 1. Switch back to the **main** branch using the branch selector 2. Navigate to **Device Management → Infrastructure → All Devices** 3. You should now see all the DC-3 devices in the main branch 4. Navigate to **Services → Topology Deployments → Data center** 5. Verify that DC-3 appears in the main branch ## Step 9: View device configurations[​](#step-9-view-device-configurations "Direct link to Step 9: View device configurations") Now that DC-3 is in the main branch, view the generated device configurations: 1. Ensure you're on the **main** branch 2. Navigate to **Device Management → Infrastructure → All Devices** 3. Click on a spine device (for example, `dc3-spine1`) 4. Look for an **Artifacts** section or tab 5. Click on the spine configuration artifact You should see a complete device configuration including: * Interface configurations * IP addressing * BGP configuration * OSPF configuration (if using OSPF underlay) * VxLAN EVPN configuration ## What you've learned[​](#what-youve-learned "Direct link to What you've learned") Congratulations! You've completed the full workflow: * ✅ Created a branch for your changes * ✅ Loaded a topology design * ✅ Monitored automatic generator execution to create infrastructure * ✅ Created a proposed change with automatic validation * ✅ Reviewed diffs, checks, and artifacts * ✅ Merged changes to the main branch * ✅ Viewed generated device configurations ## Next steps[​](#next-steps "Direct link to Next steps") Now that you understand the basic workflow, you can: * **Explore variations** - Try creating DC-3 or a POP topology * **Modify designs** - Edit the objects/dc/dc-arista-s.yml file to change the topology * **Add network services** - Create Layer 2 or Layer 3 services on top of the fabric * **Customize configurations** - Modify templates in the `templates/` directory * **Add validation checks** - Create custom checks in the `checks/` directory For a deeper understanding, continue to: * **[Understanding the concepts](/demo-dc/concepts.md)** - Learn about design-driven automation and composable topologies * **[Developer guide](/demo-dc/developer-guide.md)** - Understand how generators, transforms, and checks work ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Generator fails or times out[​](#generator-fails-or-times-out "Direct link to Generator fails or times out") If the generator task fails: 1. Check the task logs in **System → Tasks** 2. Look for error messages indicating what went wrong 3. Common issues include missing data, incorrect schema relationships, or resource allocation failures ### Validation checks fail[​](#validation-checks-fail "Direct link to Validation checks fail") If validation checks fail: 1. Review the check output to understand what failed 2. Fix the underlying data issues 3. Re-run validations by updating the proposed change ### Merge conflicts[​](#merge-conflicts "Direct link to Merge conflicts") If you encounter merge conflicts: 1. Review the conflicting objects 2. Resolve conflicts manually if needed 3. You may need to rebase your branch or create a new branch --- # Virtual machine management This tutorial shows how the demo models a virtualization layer on top of the data center fabric: hypervisor clusters, the physical hosts that form them, and the virtual machines running on those hosts. You'll load the demo data, follow what the generators do with it, and create a VM of your own through the Service Catalog. ## Overview[​](#overview "Direct link to Overview") The virtualization schema tracks: * **Hypervisor types** - one row per hypervisor family holding what follows from it: the OS its hosts run, the artifact group its VMs join, the transform that renders their provisioning script, the template image prefix, and the overcommit ratios its capacity is judged against * **Clusters** - groups of hypervisor hosts, created from a per-family object template (VMware vSphere, KVM, Proxmox VE, Hyper-V) that carries the `hypervisor_type` relationship * **Physical hosts** - rack-mounted servers that run a hypervisor and appear in the DCIM model like any other device, created from a sized object template that carries the server model, its capacity and its ports * **Virtual machines** - VMs with compute sizing (vCPUs, memory, disk) supplied by a t-shirt sized profile, a hypervisor VM ID, and an optional customer link Everything around a VM is automated. When you load a host, a generator places it in a rack, cables it to the fabric, and gives it a management address out of the fabric's own hypervisor-management network. When you load a VM, another generator assigns it a primary IP address and puts it behind an HTTPS-only firewall policy. A validation check keeps the data consistent in every proposed change. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this tutorial: * Complete the [installation guide](/demo-dc/install.md) and have Infrahub running * Load the bootstrap data and schemas * Have access to the Infrahub web interface at `http://localhost:8000` ## Loading the virtualization demo[​](#loading-the-virtualization-demo "Direct link to Loading the virtualization demo") The virtualization data loads automatically as part of every DC demo task: ``` uv run invoke demo-dc-arista # or demo-dc-juniper, demo-dc-cisco ``` Each task runs the same pipeline: 1. Creates a branch and loads the DC topology design 2. Waits for the DC generator to build the fabric 3. Loads the hypervisor-management segment and the virtualization security policy, then the clusters, hosts, and VMs 4. Creates a proposed change for review The order matters, and the task handles it for you. Host cabling needs the leaf switches that the DC generator creates, so the virtualization objects load only after the generator finishes. The hypervisor-management segment loads before the first host, because host creation immediately triggers the cabling generator, which allocates the host's address from that network. The security policy loads before the first VM, for the same reason: VM creation triggers the security generator, which expects the policy to exist. ### Sample data[​](#sample-data "Direct link to Sample data") The demo ships one virtualization environment in the Frankfurt metro: | Kind | Objects | | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Clusters | `FRA1-PROXMOX-CLUSTER` (Proxmox VE), `FRA1-KVM-CLUSTER` (KVM), `FRA1-ESXI-CLUSTER` (VMware ESXi), `FRA1-HYPERV-CLUSTER` (Microsoft Hyper-V) | | Physical hosts | `fra1-pve-01`, `fra1-pve-02`, `fra1-kvm-01`, `fra1-kvm-02`, `fra1-esx-01`, `fra1-hv-01` | | Virtual machines | `fra1-vm-web01`, `fra1-vm-web02`, `fra1-vm-app01`, `fra1-vm-app02`, `fra1-vm-db01`, `fra1-vm-cache01` (VM IDs 100-105), plus the Windows VMs `fra1-vm-win01` (ESXi) and `fra1-vm-win02` (Hyper-V), which don't use a VM ID | Each customer's VMs form one coherent stack: OpsMill runs the Linux web and database tier, Customer 2 the two Windows servers, Customer 3 the application pair, and Customer 4 the cache. Every VM carries its owning customer's SSH public key, which cloud-init or cloudbase-init installs into the guest. ### Physical host hardware[​](#physical-host-hardware "Direct link to Physical host hardware") Hosts are created from a sized object template, so a host record only names what is specific to it: its name, description, location, hypervisor type and groups. Everything about the hardware comes from the template it clones. | Template | Server | Height | vCPUs | Memory | Disk | NICs | | ---------------------------- | ------------------------ | ------ | ----- | ------ | ------- | ----------------------- | | `VIRTUALIZATION_HOST_SMALL` | Dell PowerEdge R650 | 1U | 32 | 256 GB | 2000 GB | `eth0`, `eth1`, `idrac` | | `VIRTUALIZATION_HOST_MEDIUM` | HPE ProLiant DL380 Gen11 | 2U | 48 | 384 GB | 3000 GB | `eth0`-`eth3`, `ilo` | | `VIRTUALIZATION_HOST_LARGE` | Dell PowerEdge R760 | 2U | 64 | 512 GB | 4000 GB | `eth0`-`eth3`, `idrac` | | `VIRTUALIZATION_HOST_XLARGE` | HPE ProLiant DL385 Gen11 | 2U | 96 | 768 GB | 8000 GB | `eth0`-`eth5`, `ilo` | Only `eth0` and `eth1` are cabled to the fabric. Further NICs are spare capacity and stay free, and the `idrac` / `ilo` port is out-of-band management, which nothing cables either. The demo uses LARGE for the Proxmox hosts, SMALL for KVM, and MEDIUM for the ESXi and Hyper-V hosts; XLARGE ships unused, as a size to pick from. Hosts carry a `role` of `hypervisor`. The vocabulary also offers `compute`, for a server that runs its workload directly on the hardware without a hypervisor, and a `compute` host is rejected as a cluster member by the validation below. This is a separate attribute from the network device roles (leaf, spine, dc\_firewall and so on), which `DcimDevice` declares for itself: a switch cannot be a hypervisor, and a server cannot be a spine. Because each template names a real server model, and the model carries a height, hosts appear in rack elevations at the right size once the cabling generator has placed them. Look at the `rack_elevation` artifact on any `LocationRack`, or open **Rack Visualization** in the Service Catalog. ### Cluster templates and VM sizes[​](#cluster-templates-and-vm-sizes "Direct link to Cluster templates and VM sizes") Clusters and VMs get the same pick-instead-of-type treatment as hosts, from `objects/bootstrap/08_virtualization_templates.yml`. A cluster has no capacity of its own - that is the sum of its hosts - so its templates carry the hypervisor family rather than a size. Creating a cluster is a name plus a family pick: | Template | Hypervisor family | | -------------------------------- | ----------------- | | `VIRTUALIZATION_CLUSTER_PROXMOX` | Proxmox VE | | `VIRTUALIZATION_CLUSTER_KVM` | KVM | | `VIRTUALIZATION_CLUSTER_VMWARE` | VMware vSphere | | `VIRTUALIZATION_CLUSTER_HYPERV` | Microsoft Hyper-V | VM compute sizing comes from t-shirt sized profiles, at the 1:4 vCPU-to-memory ratio the demo workloads run at. A profile rather than a template because a VM's `host` is a mandatory parent relationship, which a template object may not leave unset - a size template would have to name a host. A size is a starting point, not a contract: an explicit value on the VM overrides the profile, which is how `fra1-vm-db01` (LARGE, 500 GB disk) and `fra1-vm-win02` (MEDIUM, 250 GB disk) get their bigger disks. | Profile | vCPUs | Memory | Disk | | -------------------------- | ----- | ------ | ------ | | `VIRTUALIZATION_VM_SMALL` | 2 | 8 GB | 40 GB | | `VIRTUALIZATION_VM_MEDIUM` | 4 | 16 GB | 100 GB | | `VIRTUALIZATION_VM_LARGE` | 8 | 32 GB | 250 GB | | `VIRTUALIZATION_VM_XLARGE` | 16 | 64 GB | 500 GB | The demo's web and cache VMs are SMALL, the application and Windows VMs are MEDIUM, and the database VM is LARGE. XLARGE ships unused, as a size to pick from - the same idea as the XLARGE host template. ## Adding a hypervisor[​](#adding-a-hypervisor "Direct link to Adding a hypervisor") Everything a hypervisor family implies lives in one row of `objects/bootstrap/07_hypervisor_types.yml`: the OS its hosts run, the artifact group its VMs join, the artifact definition that renders their provisioning script, the script language, and the template image prefix. Adding a fifth family is four steps: 1. Add the row, pointing `platform` at a `DcimPlatform` and `artifact_group` at a `CoreStandardGroup`. 2. Add that group to `objects/bootstrap/00_groups.yml`. 3. Write the provisioning template under `templates/configs/`, including the shared `partials/vm_userdata.j2`. Follow the conventions the four existing templates share: guard unset `vcpus`/`memory`/`disk`/`os_version` with an error block instead of doing arithmetic on them, and keep interpolations shell-safe (quoted heredoc delimiters, single-quoted free text) - see [the scripts fail closed](#the-scripts-fail-closed). 4. Register the transform and the artifact definition in `.infrahub.yml`. Nothing else changes. The generator that assigns artifact groups, the validation check, and the Service Catalog form all read the row rather than carrying their own copy of the mapping, and a cluster names its family through the `hypervisor_type` relationship, so the row is the only place it is written. ## What the generators do[​](#what-the-generators-do "Direct link to What the generators do") Four generators react to the loaded data. You don't run any of them by hand. ### Rack placement, fabric cabling and addressing[​](#rack-placement-fabric-cabling-and-addressing "Direct link to Rack placement, fabric cabling and addressing") The `cable_virtualization_host` generator runs whenever a physical host is created. For each host, scoped to the host's own metro: 1. **Placement** - if racks exist in the metro (from whichever DC design was generated there), the host moves into the emptiest one and takes a free rack unit, stacking downward from the top the same way the DC generator places switches. The rack unit matters beyond tidiness: the `rack_elevation` artifact skips any device without a position, so a host placed in a rack but given no rack unit never appears in the rack drawing. Occupancy is counted in rack units rather than devices, so a rack holding one tall chassis does not read as emptier than a rack of 1U switches 2. **Cabling** - the host's `eth0` and `eth1` interfaces are dual-homed to the two leaf switches with the most free `customer`-role ports, so cabling load-balances across leafs 3. **Addressing** - the host takes a management address from the `virtualization_host_pool` address pool, which sits on the prefix of the [hypervisor-management segment](#the-hypervisor-management-network) the leaves render a gateway for. The address belongs to this generator rather than one of its own because it is a property of the connection: it is only reachable through the ports cabled in the step above The host's NICs come from the sized `VIRTUALIZATION_HOST_SMALL` / `_MEDIUM` / `_LARGE` / `_XLARGE` object templates in the bootstrap data, not from the generator. Each template also carries the server model, its capacity, and an out-of-band management NIC; only `eth0` and `eth1` are cabled to the fabric. The generator is idempotent: a placed host stays in its rack, cabled interfaces are left alone, and a leaf port is never handed out twice. ### Virtual machine addressing and HTTPS-only security[​](#virtual-machine-addressing-and-https-only-security "Direct link to Virtual machine addressing and HTTPS-only security") The `secure_virtualization_vm` generator runs whenever a VM is created: 1. Allocates a primary IP address from the `virtualization_vm_pool` address pool (backed by the `100.64.0.0/16` prefix in the bootstrap data) 2. Registers that address in the shared `virtualization-vms` security address group The static rules in `virtualization-vms-policy` reference that group, so every new VM is covered without editing a rule: * `allow-https-to-virtualization-vms` permits HTTPS traffic to the group * `deny-all-other-to-virtualization-vms` denies everything else ### Firewall policy attachment[​](#firewall-policy-attachment "Direct link to Firewall policy attachment") A third generator attaches `virtualization-vms-policy` to every firewall with the `dc_firewall` role as soon as the firewall is created, regardless of which DC design created it. The HTTPS-only rules then render in that firewall's `juniper_firewall_config` artifact. See the [security management tutorial](/demo-dc/security-management.md) for how policy data becomes firewall configuration. ## The hypervisor management network[​](#the-hypervisor-management-network "Direct link to The hypervisor management network") A host address is only worth recording if a packet can reach it, so the hosts are addressed out of the fabric rather than out of a pool of their own. The management network is an ordinary `ServiceNetworkSegment`, the same kind the demo uses for tenant networks, defined in `objects/segments/hypervisor-mgmt-dc-arista.yml`: | Field | Value | | --------------- | ----------------------- | | `customer_name` | `Hypervisor-Management` | | `vlan_id` | 900 | | `segment_type` | `l3_gateway` | | `prefix` | `172.25.9.0/24` | | `deployment` | `dc-arista` | Loading it triggers the existing `create_segment` generator, which binds the segment to every `customer`-role interface on the deployment's leaf switches. Those are the ports the hosts are cabled to, so one segment covers every host in the DC without naming any of them. On the Arista DC that is 192 interfaces across four leaves. The leaf configuration then carries three things for the segment, in the `leaf_config` artifact of any leaf in the deployment: ``` vlan 900 name Hypervisor-Management interface Ethernet1 switchport access vlan 900 interface Vlan900 description Hypervisor-Management Gateway ip address virtual 172.25.9.1/24 vxlan vlan 900 vni 10900 ``` The gateway address comes from the segment's prefix. Before this the templates built it from the VLAN ID, as `192.168..1/24`, which is fine while VLAN IDs stay under 256 and produces an address that does not exist at all above that: VLAN 900 would have rendered `192.168.900.1/24`. `get_anycast_gateway` in `transforms/common.py` now derives the first usable address of the recorded prefix instead, and a segment with no prefix renders a comment rather than an invented address. ### Automatic address allocation[​](#automatic-address-allocation "Direct link to Automatic address allocation") No object file gives a host an address. The `cable_virtualization_host` generator does it, in the same run that racks and cables the host, and it allocates from the `virtualization_host_pool` address pool over the segment's prefix: ``` - Placed fra1-pve-01 in rack dc-arista-Rack-1 at U39 - Cabled fra1-pve-01:eth0 -> dc-arista-leaf-01:Ethernet7 - Cabled fra1-pve-01:eth1 -> dc-arista-leaf-02:Ethernet7 - fra1-pve-01 is served by segment dc-arista - OpsMill - production - Hypervisor-Management (VLAN 900) - Allocated 172.25.9.2/24 to fra1-pve-01 from virtualization_host_pool ``` The address lands on the host's `primary_address`, which is where this demo keeps a device's management address: the switches the DC generator creates hold theirs the same way, with interface-bound addresses reserved for loopbacks. So a hypervisor host shows its address on the host object, not on `eth0`. **Two conditions gate the allocation, and both are about reachability.** The host must have at least one cabled NIC. An address on a host with no path to the fabric would be a number in a database rather than something you can reach, so a host that could not be cabled is left unaddressed and says so. The deployment its leafs belong to must carry a routed segment over the pool's prefix. Checking only that the pool exists is not enough, and this is the subtle part: the pool is bootstrap data on the default branch, so it is visible from *every* branch, while segments are loaded per DC design. A branch with the pool but no segment would otherwise hand out addresses for a subnet no leaf routes. The generator therefore walks host → its leafs → their deployment → that deployment's segments, and allocates only when one of them carries the pool's prefix and renders a gateway (`l3_gateway` or `l3_vrf`). **The generator never reallocates.** A host that already has a primary address keeps it, and the allocation identifier is the host name, so even a host whose address was removed gets the same one back from the pool. Three further details are worth knowing if you extend this: **The gateway is a real object, not just rendered text.** `172.25.9.1/24` exists as an `IpamIPAddress` in the bootstrap data. Allocation takes the next free address in the prefix, and an address that exists is not free, so materialising the gateway is what stops the pool handing `.1` to a host. Hosts start at `172.25.9.2`. **One segment names one deployment.** `ServiceNetworkSegment.deployment` is mandatory and points at a single DC, so the segment lives in a per-design file (`objects/segments/hypervisor-mgmt-dc-arista.yml`). To address hosts in another design, copy that file, change `deployment`, and give it its own prefix and VLAN. A design with no segment file leaves its hosts cabled but unaddressed, which both the demo task and the generator say out loud rather than inventing a subnet the fabric does not route. **`topology` is what ties a host's fabric together.** The DC generator stamps that relationship on every switch it creates, which is how the generator gets from a leaf to its deployment and so to the right segment. Hosts inherit the same field from the generic device model but do not need it set: their fabric is found through the leafs they are cabled to. ## Exploring the virtualization objects[​](#exploring-the-virtualization-objects "Direct link to Exploring the virtualization objects") After the demo task finishes, switch to the demo branch (for example `add-dc3`) in the web interface: 1. Navigate to **Virtualization Cluster** to see the four Frankfurt clusters and their member hosts 2. Open a **Physical Host** such as `fra1-pve-01` and check its rack location, its cabled interfaces, and the management address the generator allocated from `172.25.9.0/24` 3. Open a **Virtual Machine** such as `fra1-vm-web01` and check its host, cluster, customer, and the primary IP address the generator assigned To confirm the security wiring, open the `virtualization-vms` address group and verify it contains one address per VM. ## Provisioning artifacts[​](#provisioning-artifacts "Direct link to Provisioning artifacts") Every VM gets two artifacts: a provisioning script that matches its hypervisor, and a cloud-init user-data file that's the same artifact everywhere but branches on the guest OS. ### Per-hypervisor provisioning scripts[​](#per-hypervisor-provisioning-scripts "Direct link to Per-hypervisor provisioning scripts") | Cluster type | Provisioning artifact | Target group | What it does | | ------------ | ------------------------------- | ------------- | ------------------------------------------------------------------------------------------ | | `proxmox` | `proxmox_vm_config` (bash) | `proxmox_vms` | `qm clone`s the template, attaches a cloud-init drive, injects user-data with `--cicustom` | | `kvm` | `kvm_vm_config` (bash) | `kvm_vms` | Builds a NoCloud seed ISO with `cloud-localds`, boots the clone with `virt-install` | | `hyperv` | `hyperv_vm_config` (PowerShell) | `hyperv_vms` | Builds a NoCloud ISO with `oscdimg`, creates a Generation 2 VM with `New-VM` | | `vmware` | `esxi_vm_config` (bash/govc) | `esxi_vms` | Clones the template with `govc`, passes user-data through `guestinfo.userdata` (base64) | The guest OS still changes a few settings inside each script: | Cluster type | Linux guest | Windows guest | | ------------ | ---------------------------------------------- | ----------------------------------------- | | Proxmox VE | `--ostype l26` | `--ostype win11` | | KVM/libvirt | `--os-variant ubuntu22.04` | `--os-variant win2k22` | | Hyper-V | Secure Boot disabled (template signing varies) | Secure Boot left on | | VMware ESXi | cloud-init's VMware guestinfo datasource | cloudbase-init's `VMwareGuestInfoService` | ### The scripts fail closed[​](#the-scripts-fail-closed "Direct link to The scripts fail closed") `vcpus`, `memory`, `disk` and `os_version` are optional in the schema - the size profiles normally set the first three, but nothing forces a VM written straight through the SDK or API to carry them. A script that did arithmetic on an unset value would either fail to render or, worse, render something that only breaks at deploy time (`-MemoryStartupBytes NoneGB`). So every provisioning template checks first and renders a short script that prints an error and exits non-zero instead. The same applies to a Proxmox VM without a VMID. The artifact always renders; what changes is whether running it provisions a VM or tells you what to fix. The scripts also run as root on the hypervisor, so nothing free-form may reach the shell unquoted. The schema holds device names to hostname-safe characters (`^[a-zA-Z0-9._-]+$`), metadata heredocs use quoted delimiters (`<<'METADATA'`), the Hyper-V metadata here-string is literal (`@'...'@`), and free text such as the Proxmox description is single-quoted with quote escaping. ### The `vm_userdata` artifact[​](#the-vm_userdata-artifact "Direct link to the-vm_userdata-artifact") `vm_userdata` (content type `application/yaml`) targets the `vm_userdata_targets` group - every VM, regardless of hypervisor. It renders a `#cloud-config` document: * Windows guests get a cloudbase-init `cloud-config`: hostname, an `Admin` user in the `Administrators` group, and the SSH key if one is set * Everything else gets a cloud-init `cloud-config`: hostname, an `admin` user with passwordless sudo, and the SSH key if one is set The SSH key comes from the VM's `ssh_public_key` attribute. No password is ever stored in Infrahub - set the first-boot password at deploy time instead (console access, or a cloudbase-init inject for Windows). Every per-hypervisor script includes `vm_userdata` directly, so the two stay in sync. ### Group membership[​](#group-membership "Direct link to Group membership") The `assign_vm_artifact_groups` generator runs whenever a VM is created, the same trigger as the security generator. It follows the VM's cluster to its hypervisor type and adds the VM to the artifact group that row names (`proxmox_vms`, `kvm_vms`, `hyperv_vms`, or `esxi_vms`) plus `vm_userdata_targets`. The demo data carries no static `member_of_groups` entry for the hypervisor groups, and the generator holds no hypervisor mapping of its own: which group a VM joins is data on `VirtualizationHypervisorType`, so a new hypervisor is a row in `objects/bootstrap/07_hypervisor_types.yml` rather than an edit to the generator. The generator only adds, and only on create, so moving a VM to a cluster of a different type leaves the old group membership behind; correct that by editing the groups directly. ## Image prerequisites[​](#image-prerequisites "Direct link to Image prerequisites") Each provisioning script expects a template image already on the target hypervisor, named ``: lowercase the VM's `os_version` and replace spaces with dashes. The prefix comes from the cluster's hypervisor type (`tpl-` for every family the demo ships), so changing the convention is a data edit rather than a change to four templates. `Ubuntu 22.04` becomes `tpl-ubuntu-22.04`; `Windows Server 2022` becomes `tpl-windows-server-2022`. | Hypervisor | Where the template lives | | ----------- | ---------------------------------------------------------- | | Proxmox VE | A template VM (`qm template`), found by name via `qm list` | | KVM/libvirt | A qcow2 image in `/var/lib/libvirt/images/` | | Hyper-V | A VHDX in `C:\HyperV\Templates\` | | VMware ESXi | A template VM, cloned with `govc vm.clone` | The template must have cloud-init (Linux) or cloudbase-init (Windows) preinstalled. The provisioning scripts only inject user-data and metadata - they never install the agent, so a template without it boots without applying the SSH key or user configuration. ## Creating a VM from the Service Catalog[​](#creating-a-vm-from-the-service-catalog "Direct link to Creating a VM from the Service Catalog") The Service Catalog includes a **Create VM** page for form-based provisioning. Enable the catalog first (see [using the service catalog](/demo-dc/service-catalog.md)), then open `http://localhost:8501` and select **Create VM**. The form asks for a name, the physical host, a Guest OS (Linux or Windows - it sets the OS version choices and picks cloud-init or cloudbase-init for the `vm_userdata` artifact), an optional description, customer, status, compute sizing (vCPUs, memory, disk), and an optional SSH public key. The cluster is derived from the host you pick. A VM ID field appears only for Proxmox and KVM clusters - required for Proxmox, optional for KVM - since Hyper-V and ESXi identify VMs by name instead. When shown, it's prefilled with the next unused ID and validated against the IDs already taken in the host's cluster. On submit, the page: 1. Creates a new branch 2. Creates the virtual machine on that branch 3. Waits for the security generator to assign the IP and update the address group 4. Creates a proposed change for review 5. Regenerates the VM's provisioning artifacts and displays the rendered script and user-data, each with a download button Merge the proposed change to bring the VM into `main`. ## Validation[​](#validation "Direct link to Validation") Two checks run automatically in every proposed change that touches this data. `validate_virtualization_vm` covers the VM: it names both its host and its cluster, and the host itself belongs to a cluster, so there are two paths to the same fact. The check fails when they disagree, for example a VM assigned to `FRA1-KVM-CLUSTER` while running on `fra1-pve-01`. `validate_virtualization_capacity` covers compute capacity: a VM's vCPUs, memory and disk are allocated on the VM, the hardware they come from is on the host, and nothing compares the sum of one against the other. The check does that at two levels and reports the numbers whether or not anything is wrong, so a proposed change shows current utilization: ``` info fra1-pve-01: 10/256 vCPU allocated (4% of the limit, 64 physical at 400%) info FRA1-PROXMOX-CLUSTER: 12/512 vCPU allocated across 2 hosts (2%), 5% of the 256 available with the largest host lost error fra1-pve-01 is oversubscribed after fra1-vm-toobig: 412 vCPU allocated against a limit of 256 (64 physical at 400% overcommit) ``` Three things about how it judges: **Overcommit is expected, so the limit is not the raw hardware number.** Each hypervisor family carries the ratio it is run at, on its `VirtualizationHypervisorType` row, as a percentage because `Number` is integer-only: 400 is 4:1, 100 is no overcommit. The demo ships 4:1 vCPU for Proxmox and KVM, 3:1 for VMware, 2:1 for Hyper-V, 1:1 memory everywhere, and thin provisioning of 1.5:1 on disk (2:1 for VMware). A family with no value set is treated as 1:1. **A powered-off VM counts for disk but not for vCPU or memory.** A stopped VM still occupies its disk image, while its cores and memory are only claimed while it runs. **Only the host level fails the change.** A VM that does not fit its own host cannot start, so that is an error. The cluster level errors only if the allocation exceeds the cluster's total capacity, since nothing could place those VMs. Losing N+1 headroom, meaning the cluster no longer fits if its largest host is lost, is reported as information: it is a capacity-planning signal rather than a reason to block a change. A single-host cluster has no host to lose, so no N+1 figure is reported for it. `validate_virtualization_host` covers the host: a host names the hypervisor family it runs, and so does the cluster it joins. The check fails when a host joins a cluster of a different family, for example the Hyper-V host `fra1-hv-01` placed in `FRA1-PROXMOX-CLUSTER`, and when a host with the `compute` role, which runs its workload directly on the hardware, is made a cluster member at all. Both are comparisons rather than compatibility tables, because host and cluster point at the same `VirtualizationHypervisorType` row. A relationship picker cannot be filtered by another field's value, so the host's display label carries its hypervisor family (`fra1-hv-01 (hyperv)`) and a mismatch is visible in the picker before it is saved. ## Schema architecture[​](#schema-architecture "Direct link to Schema architecture") ``` VirtualizationCluster ├── VirtualizationPhysicalHost (member host) │ └── VirtualizationVirtualMachine (parent: host) └── VirtualizationVirtualMachine (cluster membership) OrganizationCustomer └── VirtualizationVirtualMachine (customer link) ``` ### Key relationships[​](#key-relationships "Direct link to Key relationships") * **VirtualizationPhysicalHost** is a full DCIM device: it inherits the generic device and physical device models, so it has a rack position, interfaces, and can be an artifact target * **VirtualizationVirtualMachine** belongs to exactly one host (parent relationship) and one cluster, and optionally to a customer * Both hosts and VMs share the compute sizing attributes (`vcpus`, `memory`, `disk`) through the `ComputeGenericUnit` generic * **VirtualizationHypervisorType** is referenced by both the cluster and the host, which is what turns hypervisor compatibility into an identity comparison. It also holds the artifact group, artifact definition, script language and image prefix for that family, so those facts live in one row instead of being repeated across the generator, the Service Catalog and the templates * A host carries no `platform` of its own: its hypervisor type already names the OS, and a second copy of that fact is what would let a Windows host sit in a Proxmox cluster * The Create VM form validates that a VM ID is unique within its cluster; the schema itself does not enforce it, so two clusters can both use VM ID 100 ## Next steps[​](#next-steps "Direct link to Next steps") * **[Working with security management](/demo-dc/security-management.md)** - how the HTTPS-only policy renders into firewall configuration * **[Using the service catalog](/demo-dc/service-catalog.md)** - enable the catalog application used to create VMs * **[Developer guide](/demo-dc/developer-guide.md)** - generator, check, and schema implementation details --- # Infrahub OTN demo This demo models a European research-network optical core in Infrahub, down to the fiber span and the wavelength that crosses it. Plant, spectrum and services sit in one schema, so the answers come from the data rather than from four reconciled systems. A transport engineer asks three things: what runs over a given piece of glass, what a cut takes down, and where the next 400G wavelength fits. The network it models is shaped like GÉANT. GÉANT runs a EuroHPC HyperConnectivity programme that interconnects European supercomputing sites, so **AI and HPC transport** is one of the payloads the model treats as primary. If you are here to evaluate Infrahub rather than optical transport, start with [what this demo shows of Infrahub](/demo-otn/what-this-shows.md). It maps each capability to the scenario that exercises it, and names the capabilities this demo leaves alone. [Quick start](/demo-otn/quickstart.md) is the shortest path to a blocked merge. ![The fourteen PoPs of the modelled European optical core, with the 21 multiplex sections between them coloured by OSNR margin. Paris to Madrid is red, Frankfurt to Milan carries a 4,134 GHz occupancy chip, and Vienna to Milan is marked as Raman pumped and asymmetric.](/assets/images/network-map-8fcf74ec4e226fed9f8827de50441ce4.svg) Every PoP carries this map as an artifact, rendered from the graph. Colour is the OSNR margin a 400G DP-16QAM carrier has on each section. Paris to Madrid is the one it cannot cross. This copy is Berlin's, so Berlin and the routes terminating on it are drawn heavier. See [the network map](/demo-otn/network-map.md). A second map hangs off the same sites. `odu-map` draws the same fourteen PoPs and the same 21 sections, coloured by whether another circuit **fits** on a route rather than whether a wavelength **closes** on it. Where the network map shows OSNR margin against a reference mode, this one shows free tributary slots inside the wavelengths already lit. The first map is about the optical layer, the second about the digital layer carried inside it. See [the ODU map](/demo-otn/odu-map.md) for the five bands and why 13 of the 21 sections are grey on the default branch. ![The same fourteen PoPs and 21 multiplex sections, coloured instead by the largest ODU that still fits on the roomiest lit carrier of each section. Amsterdam to London and Vienna to Milan are red with nothing left, Paris to Frankfurt amber, Berlin to Frankfurt blue, five routes green, and the remaining twelve grey and dashed because no carrier on them holds a container.](/assets/images/odu-map-331b420986d1e2b3f2027c4ae1092553.svg) The same network, one layer down. This is Frankfurt's copy of `odu-map`, taken off a branch carrying `demo/05_odu_mixed_fill.yml` so that all five bands appear at once. Compare it with the map above. A route can have decibels of margin and still have nowhere to put a 10G circuit, which is the question no other artifact in this demo answers. Grey means no carrier on the route holds a container, which on this branch is the same twelve routes that carry no wavelength at all. ## The problem it models[​](#the-problem-it-models "Direct link to The problem it models") The intended state of an optical core is spread across a span inventory, an amplifier list, a channel plan and a service database, and none of them recomputes anything when another one changes. Two questions get asked against that, both under time pressure. **Does this path close?** A wavelength's margin is a sum over every span, splice, connector, ROADM and amplifier on the route, in each direction separately. Working that out manually for one candidate route takes an afternoon. Doing it for six, before choosing one, is why route selection often defaults to whatever worked last time. **What else does this change affect?** Retuning a span, adding a ROADM degree or moving traffic onto new glass changes the margin on every wavelength crossing it. It also changes which services share a duct with which. The affected set is a traversal of the graph, and a spreadsheet cannot traverse a graph. This demo holds both halves in one model. The intended state is objects in Infrahub. The arithmetic is a library that the reports and the pipeline checks both call, so a number on a page and a number in a check come from the same code. Every change lands on a branch, and the checks run against the proposed change. A section pushed below its OSNR target, or a channel booked twice, fails before the merge rather than after the cutover. ## Capacity planning[​](#capacity-planning "Direct link to Capacity planning") Capacity is what a section can still anchor, not how much spectrum is free. A carrier's centre may only sit on one of the 96 grid positions, and its whole width has to fit inside a single free block. That leaves megahertz spread across a section that no anchor can reach. Every plan that divides free spectrum by carrier width overstates what fits, and the overstatement grows as the plan fragments. Frankfurt to Milan is where that shows. 665,600 MHz free, 26 blocks, and one anchor that will take another 400G. Twenty-five of the 26 blocks are narrower than the narrowest mode in the catalog, so they fit nothing at all. That is a fragmentation finding rather than a spectrum finding, and the two have different fixes: one is a rewrite of the anchor plan, the other is more glass. A report that collapses them into "no capacity" sends an operator looking in the wrong place. So the capacity report answers in anchors and modes rather than in megahertz. It gives the occupied and free spectrum per section and per route, the free blocks with their edges, the anchors that can actually take another carrier, and the modes that fit nowhere. Naming the modes that fit nowhere is the part a feature list would drop, and it is the part a planner needs. On the shipped dataset all ten catalog modes still fit on that corridor, and eight of the ten can only anchor on channel 95. Load `demo/04_odu_ten_in_one.yml`, which spends the one wide block, and the widest block left is 38,000 MHz. All ten modes then fit nowhere. [The spectral model](/demo-otn/spectral-model.md) has the widths, the guard band and the arithmetic behind every figure in this section. ## Why AI and HPC traffic sharpens all of this[​](#why-ai-and-hpc-traffic-sharpens-all-of-this "Direct link to Why AI and HPC traffic sharpens all of this") Distributed training is latency-bound, not only bandwidth-bound. An all-reduce collective waits for its slowest peer, so round-trip time sets the pace of the whole job. Fiber propagation sets a lower bound on round-trip time of 4897 ns per kilometre in G.652 at 1550 nm. A route 300 km longer is a slower training run, and no amount of spare capacity on it changes that. Three consequences for planning, all of them visible in this demo. **The shortest route and the route with capacity are often not the same route.** Scenario two asks for a service the network refuses on latency. Free channels sit on the path it would have to take. A planner who ranks candidate routes on capacity alone books that service and misses its budget. **Latency has to be a gate, not a report.** A service states a `max_latency_ns` budget, and the routing engine takes it as a constraint on route selection. A service whose only reachable route is too slow is refused with the figure it missed by. The model decides that before anything is provisioned. **IP over DWDM moves the optical budget into the router.** 400ZR, OpenZR+ and 800ZR put coherent optics in the router's own pluggable. That removes the transponder and makes reach a property of the part you buy, so the model records the part: a catalog of five part numbers, and nine modules with serials on them that sit in ports. Three wavelengths in this network run router to router on `OpenZR+ 400G` pluggables and close with margin to spare. The demo also shows where that stops working, and the reason is not the one most people expect. The 120 km parts reach zero of the twenty-one sections, and on the shortest section a 400ZR still holds nearly 8 dB of OSNR margin. What refuses it is chromatic dispersion. The difference between the parts that work here and the parts that do not is the forward error correction, cFEC against oFEC, not the form factor. Both answers come from a query against the mode catalog rather than from an assumption. The [link budget](/demo-otn/link-budget.md) has the working, and [AI and HPC payloads](/demo-otn/ai-payloads.md) has the payload side. ## What it answers[​](#what-it-answers "Direct link to What it answers") | Question | How | | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | What runs over this piece of glass? | Graph traversal from the fiber span out to customers | | What breaks if it is cut? | The same traversal, reported as impact | | Where can the next 400G go? | Loss and OSNR budget over every candidate route | | How many more actually fit on this section? | Free blocks and the grid anchors inside them, not free megahertz | | Provision Berlin to Amsterdam at 400G | One service object, a generator, a proposed change | | What changes if I add a ROADM? | Branch diff before merge | | Can this AI training service meet its latency SLA? | Accumulated propagation against the service budget | | Which route is the weak one, and which is full? | A [rendered map](/demo-otn/network-map.md) on every PoP, coloured by OSNR margin | | Which routes can still take another 100G circuit? | A [second map](/demo-otn/odu-map.md) on every PoP, coloured by free tributary slots | | Does ten customers in one wavelength hold up? | Grooming packs them into one ODU4, and refuses the eleventh | | The route is too long for any wavelength. Now what? | Split it at an O-E-O regenerator, and budget each half on its own | | These two circuits were promised diverse. Are they? | A check on the declared group, silent about everything nobody promised | | Somebody asked for a service the network cannot build. Does it merge? | Not unless a person signed for the refusal. A check reads the `status` the generator wrote and fails the proposed change. | ## The modelled chain[​](#the-modelled-chain "Direct link to The modelled chain") Fourteen core sites, twenty-one optical multiplex sections, and 4,800,000 MHz of C-band per section with ninety-six 50 GHz grid positions a carrier may be anchored on. A fifteenth site is outside that chain. Amsterdam Science Park is a customer campus rather than a PoP, reached over an 18.4 km unamplified CWDM tail on the coarse eighteen-wavelength plan. It is there because a transport network has an edge, and the edge does not look like the core. ## Four properties of the model[​](#four-properties-of-the-model "Direct link to Four properties of the model") **Every optical value is a scaled integer.** Infrahub has no floating-point attribute kind, so loss, gain and OSNR are stored in millidecibels, lengths in metres, frequencies in megahertz, and latency in nanoseconds. The unit is part of the attribute name. **Ports and optical elements are separate abstractions.** Path traversal walks ports. The link budget sums elements. A fiber span is an element and not a device, so a single inheritance chain cannot express both. **Amplification is per direction.** Light crosses a section both ways and an amplifier restores power in one of them, so every section has two amplifier chains and produces two margins. The section holds one relationship per chain, so which way an amplifier faces is where it sits in the graph rather than a field on the amplifier that could disagree. A span with a Raman pump on it costs less one way and more the other, and the reports show both. **The reports return negative answers.** Reach and latency are modelled as data rather than assumed. The demo reports that 400ZR reaches nothing on this topology. The only route with usable spectrum left is too slow for the service asking for it. Eight of the ten catalog modes can anchor in exactly one place on the busiest corridor. Three regenerator sites in a row fail to make Madrid to Warsaw close at DP-16QAM. One section does not close at 400G on the most spectrally efficient mode. That last one leaves a failing check on the default branch, which is deliberate. The [link budget](/demo-otn/link-budget.md) page says which route and what the fix costs. The ODU map is the same discipline drawn rather than printed. Most of it is grey on the default branch, because 13 of the 21 sections carry no wavelength at all, and grey there means "not known" rather than "empty and available". A section where nothing fits and a section nobody measured are two different cells in its panel, and the map refuses to average them together. --- # AI and HPC payloads Distributed training traffic runs on this network in operation, so the model treats it as a primary payload alongside IP transit. Two of the four findings below are negative. Modelling reach and latency as data is what produces a negative answer at all: a model that assumes them can only report what works. ## What is real, and what is illustrative[​](#what-is-real-and-what-is-illustrative "Direct link to What is real, and what is illustrative") **Real.** GÉANT runs a EuroHPC HyperConnectivity programme that interconnects European supercomputing sites. The network modelled here is, in operation, an AI and HPC transport network. **Real.** 400ZR, OpenZR+ and 800ZR are published standards, not vendor marketing. 400ZR is an OIF Implementation Agreement; OpenZR+ is a multi-source agreement. All three exist specifically to carry high-bandwidth traffic between compute sites as IP over DWDM, with the coherent optics in the router's own pluggable and no separate transponder. **Real.** Distributed training is latency-bound and not only bandwidth-bound. All-reduce collectives wait for the slowest peer, so round-trip time sets the pace of the whole job. Fiber propagation sets the floor on round-trip time at **4897 ns per kilometre** in G.652 at 1550 nm. **Illustrative.** Which supercomputing facility attaches to which modelled point of presence, the specific span lengths, and the reach and OSNR figures in the mode catalog. They are representative of their classes. Every conclusion below survives a change to any of them, and each section says why. ## The compute sites in the model[​](#the-compute-sites-in-the-model "Direct link to The compute sites in the model") | Facility | Location | Modelled attachment | | ------------- | ----------------- | ------------------- | | JUPITER | Jülich, Germany | Frankfurt | | Leonardo | Bologna, Italy | Milan | | MareNostrum 5 | Barcelona, Spain | Madrid | | Karolina | Ostrava, Czechia | Prague | | MeluXina | Luxembourg | Brussels | | Vega | Maribor, Slovenia | Vienna | LUMI in Kajaani is deliberately absent. Finland is outside the fourteen PoPs, and inventing a point of presence to hold it would misrepresent the topology. ## Latency is computed, never declared[​](#latency-is-computed-never-declared "Direct link to Latency is computed, never declared") ``` propagation_delay_ns = length_m * group_index_milli * 1_000_000 / C_M_PER_S ``` `group_index_milli` is 1468 for G.652 at 1550 nm and lives on the fiber type, not in the code, because G.654 and G.655 differ and the difference is measurable over 1000 km. ROADM, amplifier and forward error correction latency add on top, and the running total lands on each path hop next to cumulative loss and OSNR. `OtnService.service_profile` is what decides whether a service has a latency budget at all. `ai-training-dci`, `ai-inference` and `hpc-research` carry `max_latency_ns`; `ip-transit` and `legacy-sdh` leave it null, and the latency report says how many services it excluded for that reason. ## The ZR catalog[​](#the-zr-catalog "Direct link to The ZR catalog") | Mode | Rate | Required OSNR | Nominal reach | FEC | FEC latency | | ------------ | ---- | ------------- | ------------- | ---- | ----------- | | 400ZR | 400G | 26.0 dB | 120 km | cFEC | 5 µs | | OpenZR+ 400G | 400G | 23.0 dB | 1000 km | oFEC | 9 µs | | OpenZR+ 300G | 300G | 20.5 dB | 2000 km | oFEC | 9 µs | | OpenZR+ 200G | 200G | 17.5 dB | 3000 km | oFEC | 9 µs | | 800ZR | 800G | 29.0 dB | 120 km | cFEC | 5 µs | Only transponder modes are provisionable in this model. A ZR wavelength originates in the router's own pluggable, and every router port in the loaded dataset is grey: no centre frequency, -2 dBm launch, LC connector. The catalog rows above exist to be reported on, and the selector filters them out as equipment rather than discovering them as a dispersion failure. ## Negative result one: 400ZR reaches nothing here[​](#negative-result-one-400zr-reaches-nothing-here "Direct link to Negative result one: 400ZR reaches nothing here") Both 120 km parts reach **zero of twenty-one** optical multiplex sections. The shortest section in the whole topology is Amsterdam to Brussels at 220 km. A 400ZR falls 100 km short of the easiest section in the network, and an 800ZR falls exactly as far short. The ordering holds if the reach figure moves by a few tens of kilometres, because the next shortest sections are 320 km and 330 km. ``` $ uv run invoke demo-reach --branch probe 400ZR, 800ZR reach nothing on this network. The shortest optical multiplex section is oms-ams-bru at 220.000 km, and every one of these parts is rated 120.000 km. ``` That is the answer a planner wants before ordering pluggables, and it is visible only because reach is modelled data rather than an assumption. The same report shows where the parts that do reach can go: `OpenZR+ 400G` covers twenty of twenty-one sections, missing only Paris to Madrid at 1250 km. **Reach is not the budget.** A mode whose catalog reach covers a section can still miss the OSNR margin over that section's amplifier chains. The reach report answers whether the part is worth ordering; `budget_report` computes whether the wavelength closes, once in each direction of travel. Every row in the reach report says so, because conflating the two would let the report claim something it did not compute. ## Negative result two: forward error correction does not drive route selection[​](#negative-result-two-forward-error-correction-does-not-drive-route-selection "Direct link to Negative result two: forward error correction does not drive route selection") Stronger error correction buys reach and costs latency, which suggests an optimal route that depends on which of the two a service needs more. On this network the arithmetic removes the choice. Frankfurt to Milan is 780 km of fiber, which is 3819.441 µs of propagation. Everything the electronics contribute, measured end to end on the provisioned service, is **5.300 µs**. Straight out of `ai_latency` on the demo branch: | Service | Length | Propagation | Total | Electronics | Share | | ------------------- | ------- | ----------- | ----------- | ----------- | ------ | | Frankfurt to Geneva | 590 km | 2889.065 µs | 2894.165 µs | 5.100 µs | 0.18 % | | Frankfurt to Milan | 780 km | 3819.441 µs | 3824.741 µs | 5.300 µs | 0.14 % | | Vienna to Milan | 800 km | 3917.380 µs | 3922.680 µs | 5.300 µs | 0.14 % | | Amsterdam to Milan | 1250 km | 6120.901 µs | 6127.051 µs | 6.150 µs | 0.10 % | The propagation column is summed span by span, each at its own fiber type's group index, rather than computed once over the total length. On this plant every span is G.652.D so the two agree to within a nanosecond of rounding, and the sum is what stays correct the day a G.654 span appears. The strongest correction in the catalog, oFEC at 9 µs, would still be 0.24 percent of the Frankfurt to Milan propagation. At continental distances propagation dominates by three orders of magnitude, so the OSNR-optimal route and the latency-optimal route are both the shortest route, and there is no trade-off to exploit. Modelling FEC latency was still worth doing. It is what let the model rule the trade-off out by measurement rather than by assumption. ## The tension that is real: capacity against latency[​](#the-tension-that-is-real-capacity-against-latency "Direct link to The tension that is real: capacity against latency") The shortest route fills up. The next route is longer, and distance is latency at 4897 ns per kilometre. Frankfurt to Milan direct is 780 km. Routed via Geneva it is 990 km, and the detour costs **1,028,312 ns** of extra propagation, 27 percent of the direct route's own delay from a single diversion. End to end, through the budget engine, the penalty is 1028.864 µs: 552 ns more than the propagation figure, from two extra ROADMs and four extra amplifiers. Against a four millisecond one-way budget: | Route | One way | Verdict | | ------------------ | ----------- | --------------------------- | | Direct, 780 km | 3824.741 µs | Passes, 175.259 µs to spare | | Via Geneva, 990 km | 4853.605 µs | Fails by 853.605 µs | Propagation delay is the same both ways, so one figure covers a route. Loss and OSNR are not: a section carries an amplifier chain per direction, and a span with a Raman pump on it costs less one way and more the other. Four milliseconds is the number that separates them, and the finding depends on it. At five milliseconds both routes pass and there is nothing to report, which is why the budget belongs on the service object as data a reader can check. The direct corridor holds 4,134,400 MHz of its 4,800,000 MHz C-band in the shipped dataset, so a new 400G fits today, but only once. The corridor has capacity and no alternative: > 665,600 MHz is free on the only latency-compatible route between these two compute sites, and exactly one of the 96 anchors, channel 95, can centre a 400G carrier inside a single free block. Dividing free spectrum by carrier width would have promised eight. When that anchor is gone there is no second route, because the only other one costs 1028 µs and puts the budget out of reach by 848. Capacity anywhere else in the network does not substitute for it. **When they are gone, the corridor is congested rather than closed.** A 400G service needs a whole `ODUC4`, all 320 tributary slots of a wavelength, so for this service the last usable block of spectrum and the last wavelength are the same thing. A smaller client is not in that position: a `100GBASE-LR4` needs 80 slots and grooms into a wavelength somebody has already lit, which consumes no spectrum at all. `demo/90_fra_mil_saturated.yml` runs both requests against the same exhausted corridor and gets opposite answers, and [the provisioning scenarios](/demo-otn/provisioning-scenarios.md#scenario-two-a-corridor-out-of-colour-and-what-still-fits) walks through them. The honest statement about a full corridor names the layer: no new wavelength can be lit on it, and whether a circuit fits depends on how big the circuit is. ## A regenerator is a latency term, and a large one[​](#a-regenerator-is-a-latency-term-and-a-large-one "Direct link to A regenerator is a latency term, and a large one") The FEC correction above is small enough to be ruled out by measurement. The framing delay at an O-E-O regenerator is not, because it comes with a second half of route attached to it. A circuit split at a regenerator pays for both segments plus what the junction charges for reframing, which `framing_latency_ns` holds on the device. Madrid to Warsaw takes **14,558,963 ns** end to end against **9,163,620 ns** for its Madrid to Frankfurt half alone. So a chain is the last thing a latency-bound payload should want, and the routing engine ranks a direct wavelength above one wherever both serve the route. Madrid to Warsaw takes the chain because it has no direct wavelength that closes on any of the ten modes. That is the only condition under which a chain wins, and it is a reach problem being paid for in latency. A service on that route with a tight `max_latency_ns` would be refused on the budget rather than quietly given the slow answer. ## Diversity is reported, and separately enforced when it was promised[​](#diversity-is-reported-and-separately-enforced-when-it-was-promised "Direct link to Diversity is reported, and separately enforced when it was promised") A training cluster split across two sites cannot tolerate a single fiber cut. Two services on two different routes look diverse on a map, and if their spans share a conduit, one backhoe cuts both. `srlg_exposure` answers that from the conduit model. On the demo dataset it finds five non-diverse pairs among five services, every one of them between two latency-sensitive services. The instructive one is Frankfurt to Geneva against Amsterdam to Milan: no shared section, no shared ROADM, no shared city pair, and they share `cd-fra-south` out of Frankfurt. Protection switching itself is out of scope. Reporting the exposure is cheap; building restoration is a different demo. **The declared case does block a merge.** Put two services in an `OtnDiversityGroup` and `checks/diversity.py` fails the proposed change when their routes share a duct. It stays silent about every pair above, because nobody promised anything about them and an operator may have accepted the exposure deliberately. Reporting and enforcing are two questions, and only the second one has an answer worth blocking on. ## Where the numbers come from[​](#where-the-numbers-come-from "Direct link to Where the numbers come from") Every figure above is produced by the loaded dataset and asserted by a test, so retuning the plant fails a test rather than quietly falsifying a document. | Claim | Test | | ------------------------------------------------------------ | ----------------------------------- | | 4897 ns per kilometre, and the two Frankfurt to Milan delays | `tests/unit/test_units.py` | | The 1,028,312 ns propagation penalty | `tests/unit/test_routing_claims.py` | | The four rows of the electronics-share table | `tests/unit/test_routing_claims.py` | | The end-to-end delays and the four millisecond verdicts | `tests/unit/test_routing_claims.py` | | 220 km is the unique shortest section | `tests/unit/test_impact_claims.py` | | Both 120 km parts reach zero sections | `tests/unit/test_budget_claims.py` | | Every span is G.652.D, so the propagation split is exact | `tests/unit/test_impact_claims.py` | --- # Client mapping A customer hands over an Ethernet port, an SDH circuit, a 2 Mbps tributary or an InfiniBand link out of a supercomputer. None of those travels on a wavelength as it arrives. Each is wrapped in a digital container first, and containers nest inside larger containers until one fills a wavelength. Two kinds cover that. `OtnClientSignal` is what arrives. `OtnContainer` is what it becomes. ## The client-signal catalog[​](#the-client-signal-catalog "Direct link to The client-signal catalog") Eleven signals across five layers, loaded from `objects/04_client_signals.yml`. The Mapping column names the ITU-T G.709 mapping procedure: **GMP** generic, **BMP** bit-synchronous, **AMP** asynchronous. | Signal | Alias | Layer | Bit rate | Renders as | First container | Mapping | | -------------- | ------ | ------------- | ---------------- | ------------ | --------------- | ------- | | `1000BASE-T` | | Ethernet | 1 250 000 kbps | 1.25 Gbps | `ODU0` | GMP | | `10GBASE-LR` | | Ethernet | 10 300 000 kbps | 10.3 Gbps | `ODU2e` | BMP | | `100GBASE-LR4` | | Ethernet | 103 100 000 kbps | 103.1 Gbps | `ODU4` | GMP | | `400GBASE-FR4` | | Ethernet | 412 500 000 kbps | 412.5 Gbps | `ODUC4` | GMP | | `STM-16` | OC-48 | SDH | 2 500 000 kbps | 2.5 Gbps | `ODU1` | AMP | | `STM-64` | OC-192 | SDH | 9 950 000 kbps | 9.95 Gbps | `ODU2` | AMP | | `STM-256` | OC-768 | SDH | 39 800 000 kbps | 39.8 Gbps | `ODU3` | AMP | | `E1` | | PDH | 2048 kbps | 2.048 Mbps | `VC-12` | AMP | | `FC-1200` | | Fibre Channel | 10 500 000 kbps | 10.5 Gbps | `ODU2e` | BMP | | `IB-EDR-4X` | | InfiniBand | 103 125 000 kbps | 103.125 Gbps | `ODU4` | GMP | | `IB-HDR-4X` | | InfiniBand | 212 500 000 kbps | 212.5 Gbps | `ODUflex` | GMP | Three details in that table are load-bearing. **The rate is stored in kbps.** E1 is 2.048 Mbps. As an integer number of megabits it rounds to 2, and so does T1 at 1.544 Mbps, which would make the two signals the model exists to distinguish indistinguishable. Kilobits are the smallest unit any signal here needs. **One template renders both ends of the table.** The catalog spans seven orders of magnitude, from 2048 kbps to 412 500 000. A single divisor makes one end unreadable: gigabits give E1 as `0.002048 Gbps`, megabits give 400GBASE-FR4 as `412500.0 Mbps`. So `bit_rate_display` switches at one gigabit: ``` {% if bit_rate_kbps__value is none %}{% elif bit_rate_kbps__value >= 1000000 %}{{ bit_rate_kbps__value / 1000000 }} Gbps{% else %}{{ bit_rate_kbps__value / 1000 }} Mbps{% endif %} ``` The same catalog therefore reports `2.048 Mbps` for E1 and `103.1 Gbps` for 100GBASE-LR4 without retyping either value. **`default_container_type` is the first step, not the last.** E1 maps into `VC-12`, not into `ODU1`. `ODU1` is where the chain ends up, two layers later. Writing the destination in the "first container" column would skip the SDH layer entirely and leave `OtnTributaryPort` with nothing to justify it. ## InfiniBand, and why its rates are not round numbers[​](#infiniband-and-why-its-rates-are-not-round-numbers "Direct link to InfiniBand, and why its rates are not round numbers") Two rows in the catalog are InfiniBand: EDR and HDR, both four lanes wide. Their bit rates are signalling rates rather than the round numbers the products are sold under. * EDR is four lanes at 25.78125 Gbps, so 103.125 Gbps. * HDR is four lanes at 53.125 Gbps, so 212.5 Gbps. That is the convention the rest of the catalog already follows. 400GBASE-FR4 is stored at 412.5 Gbps, not at 400, for the same reason. The transport network has to move every bit on the wire, and the line rate is what a container has to fit. Storing 400 for a signal that runs at 412.5 would make the container too small. The error would appear as an overbooked wavelength rather than as a wrong number on a page. Neither row has an alias. The `alias` attribute is described as the SONET name where one exists, and InfiniBand has none. HDR is the reason `ODUflex` is used by a catalog row rather than only listed in the enum. Every other row in the table maps into a fixed container sized for a standard client rate, and 212.5 Gbps is not one of those rates. A flexible container is what G.709 provides for exactly that case. **NDR is absent, and the ceiling is real.** InfiniBand NDR is four lanes at 106.25 Gbps, so 425 Gbps. The largest container the `default_container_type` enum offers is `ODUC4`, whose G.709 payload rate is roughly 421 Gbps. This repository does not compute that figure, and no test here derives it: it is cited from the standard. What this repository does show is the shape of the problem. The catalog's largest entry, 400GBASE-FR4 at 412.5 Gbps, maps into `ODUC4`, and `default_container_type` holds nothing above `ODUC4` for a 425 Gbps client to map into. `OtnContainer.odu_type` does hold `ODUC6` and `ODUC8`, but only as the line containers the 600G and 800G modes ride: no client signal maps directly into either. Adding NDR would mean extending `default_container_type` too, and writing a story about how a client faster than any container it may map into gets provisioned. The row is left out rather than added with an incorrect container. ## Choosing a signal, automatically or explicitly[​](#choosing-a-signal-automatically-or-explicitly "Direct link to Choosing a signal, automatically or explicitly") A service says how fast it is. Something has to decide what it hands over. **The automatic path is a rate rule with a flag on each catalog row.** When `OtnService.client_signal` is unset, the generator takes the smallest catalog signal that can carry the requested rate. It considers only rows with `OtnClientSignal.auto_selectable` set. Today that is every Ethernet, SDH, PDH and Fibre Channel row, and neither InfiniBand row. The rule is "smallest at or above", not "nearest at or below". A 400 Gbps service asking for the nearest rate at or below would get `100GBASE-LR4`, which cannot carry it. **The flag defaults to false on purpose.** A deny-list naming InfiniBand would fix the case in front of it and reopen the same hole the next time a specialised signal is added. The failure is silent, because nothing fails until some service happens to land in the new gap. A flag that starts false means a new row is unreachable until someone writes `true` in a diff. The schema refuses the write that omits it, which a Python allow-list could not do. **A service can state its signal, and a stated signal wins outright.** Set `client_signal` on the service and the generator uses that row, whatever the rate rule would have picked. A stated signal slower than the requested rate is refused with a named error rather than substituted. Substituting a faster row would make the relationship advisory: the service would provision, report success, and hand over a signal nobody asked for. **`IB-EDR-4X` can never be selected automatically, at any rate.** Not because of the flag, and setting the flag would not change it. `100GBASE-LR4` runs at 103 100 000 kbps and `IB-EDR-4X` at 103 125 000 kbps. Wherever both are candidates the Ethernet row is 25 000 kbps smaller, so the "smallest at or above" rule takes it. Every rate that reaches EDR reaches 100 Gigabit Ethernet first. An explicit `client_signal` is the only way to provision EDR. That is a property of the catalog worth knowing rather than a defect. Two signals 0.02 percent apart are interchangeable to a rate rule, and only the customer knows which one arrived on the fiber. `IB-HDR-4X` is unreachable for the other reason. Nothing in the Ethernet range sits between 103.1 and 412.5 Gbps, so HDR would be selectable in that gap if its row carried the flag. It does not, so HDR is stated too. `demo/03_infiniband_service.yml` is the worked example: Frankfurt to Prague at 212 Gbps with `client_signal: IB-HDR-4X`, which provisions into an `ODUflex`. Remove that one line and the same service provisions `400GBASE-FR4` instead, with no error and no warning. The request is 212 Gbps rather than 200 because HDR signals at 212.5 Gbps, which is 170 tributary slots, and a 200G wavelength carries an `ODUC2` offering 160. This client does not fit a 200G wavelength at all. At 212 the 200G mode drops out of the eligible set and the service lands on DP-16QAM 64GBd 400G, whose `ODUC4` offers 320: the `ODUflex` takes 170 and leaves 150 free. Run it after `demo-setup`. The task loads the file and provisions it: ``` uv run invoke demo-infiniband ``` The container it writes reads back as `odu_type: ODUflex` with `client_signal: IB-HDR-4X`, on `oms-prg-fra`. ## The container hierarchy[​](#the-container-hierarchy "Direct link to The container hierarchy") `OtnContainer` is one kind that nests inside itself. The `odu_type` vocabulary holds sixteen values: thirteen ODU types from G.709, plus `VC-12`, `VC-4` and `STM-N` from G.707. SDH containers are containers too, and the E1 story cannot be written without them. Nesting is one relationship, declared twice: ``` - name: parent_container peer: OtnContainer direction: outbound cardinality: one identifier: otn_container__children - name: child_containers peer: OtnContainer direction: inbound cardinality: many identifier: otn_container__children ``` The two `direction` keys are mandatory, not decorative. Both sides default to `bidirectional`, which collides on a shared identifier, and the load is rejected with `Identifier of relationships must be unique for a given direction`. Since `identifier` is immutable once loaded, getting this wrong means deleting the branch rather than editing the file. Declared this way it is one edge. Set the child's `parent_container`, read the parent, and the child is in `child_containers` with no second write. Each container has two slot numbers, and the distinction matters: * **`tributary_slots`** is what this container occupies in its parent. * **`tributary_slot_capacity`** is what it offers to its own children. An ODU4 offers 80 slots. An ODU2 inside it occupies 8. Ten ODU2s fill it exactly. The four stories below are the depths the model supports, and this is their shape: The last box is where this page stops and another one starts. Containers decide how much of a wavelength is used. The transponder mode decides how much spectrum that wavelength costs, and filling every slot in an ODU4 does not change it by a megahertz. Capacity on this network therefore has two answers, and [the spectral model](/demo-otn/spectral-model.md) holds the second. ## A regenerated circuit has containers on both sides[​](#a-regenerated-circuit-has-containers-on-both-sides "Direct link to A regenerated circuit has containers on both sides") A circuit that crosses an O-E-O regenerator is two wavelengths, so it is two sets of containers. `OtnContainer.segment_sequence` says which segment a container rides, and `OtnOpticalPath.segment_sequence` says the same thing about the wavelength. Both default to `1`, which is the correct reading and not merely a value that loads: a container written before regeneration existed rode the one and only segment. So a circuit is its containers ordered by `segment_sequence`, and its wavelengths are its paths ordered by the same number. Two readings of one sequence from two places, and a test asserts they agree. `OtnService.optical_path` is cardinality **many** for the same reason. One path per wavelength, each carrying its own budget, ordered by the sequence because an Infrahub relationship hands back a set rather than a list. **The schema refuses a duplicate, and a check catches the gap.** A `uniqueness_constraints` entry of `["service", "segment_sequence__value"]` on `OtnOpticalPath` means no service can hold two segment 2s. Infrahub refuses that write from every direction. It cannot notice a **missing** number, because a schema constrains what is written and says nothing about what is absent: `1, 2, 4` satisfies the constraint and is a broken circuit. That half is asserted in code, and it names the service and the absent number rather than naming a constraint. ## Four client stories[​](#four-client-stories "Direct link to Four client stories") ### Ethernet over OTN[​](#ethernet-over-otn "Direct link to Ethernet over OTN") A customer hands over 100 Gigabit Ethernet at a client port. It maps into one ODU4 with generic mapping, and that ODU4 fills a single 400G wavelength no more than a quarter full, so three more clients share the carrier. One container, one level, no nesting. This is the shortest chain the model supports, and it is the majority of modern traffic. ### SDH over OTN, ten customers in one ODU4[​](#sdh-over-otn-ten-customers-in-one-odu4 "Direct link to SDH over OTN, ten customers in one ODU4") Ten separate customers each hand over an STM-64 circuit. Each maps asynchronously into its own ODU2, which occupies 8 tributary slots. The ten ODU2s nest inside one ODU4, whose capacity is 80. Ten customers, one wavelength, one hierarchy that has to be read as a tree rather than a list. Reading the parent once returns all ten children with their slot counts, which is exactly the shape a capacity check needs. `demo/04_odu_ten_in_one.yml` is this story as a loadable scenario. It provisions the ten, reaches 80 of 80, and gets the eleventh refused. The [ODU map](/demo-otn/odu-map.md) page walks it. ### E1 over SDH over OTN[​](#e1-over-sdh-over-otn "Direct link to E1 over SDH over OTN") A 2 Mbps G.703 tributary arrives on a copper port. It maps into a VC-12, VC-12s are multiplexed into an STM-N, and the STM-N maps into an ODU1. Three levels of nesting, and the reason `VC-12`, `VC-4` and `STM-N` are in the `odu_type` enum at all. Drop them and this story cannot be written, and `OtnCopperPort` becomes a kind with no purpose. Legacy circuits are the whole reason a transport network still carries SDH. ### InfiniBand over OTN[​](#infiniband-over-otn "Direct link to InfiniBand over OTN") A supercomputer hands over an HDR link, four lanes at 53.125 Gbps. Nothing in the fixed container ladder is sized for 212.5 Gbps, so it maps into an `ODUflex`, which G.709 provides for that case. One container, one level, and the only one of the four stories a rate rule cannot pick on its own. The service states `client_signal: IB-HDR-4X` and the generator uses that row, for the reasons in [choosing a signal](#choosing-a-signal-automatically-or-explicitly) above. Sized from the client rather than from the table, the `ODUflex` takes 212 500 000 kbps over the 1.25 Gbit/s slot, which is 170 slots. That is the negative result behind the scenario's 400G wavelength. A 200G wavelength carries an `ODUC2` offering 160 slots, and 170 does not fit in 160. No InfiniBand HDR client fits a 200G wavelength anywhere on this network. On the 400G mode the `ODUC4` offers 320 and 150 are left free. ## The capacity rule: not in the schema, enforced twice around it[​](#the-capacity-rule-not-in-the-schema-enforced-twice-around-it "Direct link to The capacity rule: not in the schema, enforced twice around it") The rule is one sentence. For any container, the sum of `tributary_slots` across its children must not exceed its own `tributary_slot_capacity`. Every number the rule needs is in the schema, and reading a parent returns every child in one query. **The schema still cannot state it.** Infrahub's uniqueness and bounds constraints work on single attributes and single objects; a sum across a relationship is not something a schema constraint can express. Write twenty ODU2s under an ODU4 of capacity 80 and the write itself succeeds. **Two things then refuse it.** Both read the one slot table in `src/infrahub_demo_otn/containers.py`, so neither can accept what the other rejects. * `checks/container_capacity.py` runs in the proposed-change pipeline and walks every container on the branch, whatever route the data arrived by: a generator, a data file, or the UI. It fails and names the container with both figures. * The provisioning generator refuses rather than overfilling. A service with nowhere to groom and no wavelength left to light is left rejected with reason `no-slots`. The refusal names the tightest container it did not fit and both of its slot figures, and no container is created. The check on the twenty-ODU2 case, from a failing run: ``` odu-line-oc-overfill-probe is overfilled: its 20 children commit 160 tributary slots and it offers 80, so it is over by 80. ``` The message then lists the twenty children by name. The failure is logged against the container's own id, so the proposed change links it to the object rather than leaving a reader to find a name in a string. The repository ships fourteen checks: `channel_collision`, `osnr_margin`, `units_import`, `container_capacity`, `diversity`, `provisionable`, `channel_count_consistency`, `monitor_completeness`, `carrier_termination`, `mux_channel_binding`, `attenuator_range`, `transceiver_placement`, `transceiver_mode_support` and `connector_polish`. `container_capacity` is the one that walks the container tree. `provisionable` is what stops the refusal above from merging. A service left `rejected` with reason `no-slots` fails the proposed change, naming the service, the code and the detail. The one exception is a service somebody has set `refusal_accepted` on, which keeps the refusal on the record. **Unknown is reported as unknown.** Four of the sixteen container types have no G.709 slot figure, the flex container and the three SDH virtual container types. The check reports a container holding one of those as unknown rather than passing it as zero or failing it as overfull. Every figure derived from it on the [ODU map](/demo-otn/odu-map.md) reports unknown too. ## Negative result: cycles are still unguarded[​](#negative-result-cycles-are-still-unguarded "Direct link to Negative result: cycles are still unguarded") Nothing stops a container being made its own ancestor. The capacity check reads parents and children, not reachability, so a cycle is not what it looks for. This is a real gap rather than one the feature closed quietly, and the slot numbers on a cyclic tree mean nothing. --- # Optical plant concepts The plant layer is four kinds: a fiber span, an optical multiplex section, a conduit, and a fiber type. Together they answer "what is the light travelling through, and what does that cost it". What follows is the part of that answer the YAML cannot state for itself. ## A fiber span is an optical element, not a device[​](#a-fiber-span-is-an-optical-element-not-a-device "Direct link to A fiber span is an optical element, not a device") `OtnFiberSpan` inherits `OtnOpticalElement` and nothing else. It is the only kind in the model that does. Every other optical element is also a device. A transponder, a ROADM, an amplifier, a mux/demux, a patch panel and a Raman pump are all racked at a site, all have ports, and all attenuate light. A fiber span attenuates light and does none of the rest. It sits in a trench between two sites, it has no rack position, and light does not enter it through a port object. The span contributes the largest loss in the network. Over the shortest section in the loaded dataset, 220 km from Amsterdam to Brussels, the three spans contribute 53.001 dB and the two ROADMs contribute 14.000 dB. So the span has to be in the answer to "what does the budget sum", and it cannot be in the answer to "what is racked here". That is why the two generics stay apart instead of forming one inheritance chain: * `OtnGenericDevice` is anything racked at a site. * `OtnOpticalElement` is anything light passes through and loses power in. A single chain that made every optical element a device would have forced the span to grow a site relationship and a port list it can never fill. The split costs one extra `inherit_from` line on seven device kinds, and in return a lossy non-device is expressible. Loaded, `OtnOpticalElement` reports eight kinds using it. Seven are devices. One is the glass. The [schema reference](/demo-otn/schema-reference.md#two-generics-not-one-chain) draws which kind inherits which. ## A span's stored insertion loss is zero, and that is not its loss[​](#a-spans-stored-insertion-loss-is-zero-and-that-is-not-its-loss "Direct link to A span's stored insertion loss is zero, and that is not its loss") `OtnFiberSpan` inherits `insertion_loss_mdb` from `OtnOpticalElement`, and on a span the value stays at its default of `0`. Read a span through GraphQL and `insertion_loss_display` says `0.0 dB`. That number is the absence of a claim rather than a claim that the span is lossless. A span's loss is not a property anyone measures once and writes down. It is computed from four things the span does store: ``` loss = length_m x fiber_type.attenuation_coefficient_mdb_per_km / 1000 + splice_count x splice_loss_mdb + connector_count x connector_loss_mdb + aging_margin_mdb ``` Every term is on the span or one relationship away. There is deliberately no `total_loss_mdb` attribute: a stored total is a second copy of a derived fact, and nothing in the schema keeps the copy in sync with the parts. Change a span's length and a stored total becomes wrong with no signal that it has. The optical budget engine computes the sum at read time instead. The same rule removes four attributes from `OtnOpticalMultiplexSection`. It has no `total_length_m`, no `total_loss_mdb`, no `span_count` and no `hop_count`. All four are sums over its spans. ## A conduit is what makes route diversity honest[​](#a-conduit-is-what-makes-route-diversity-honest "Direct link to A conduit is what makes route diversity honest") `OtnConduit` has a name, an owner, a description, and a list of spans. No length, no route, no geometry. That looks thin until you ask what the kind is for. Two services provisioned on "diverse" paths through Amsterdam and Frankfurt are only diverse if their fiber is in different trenches. If both paths run through the same duct under the same motorway, one backhoe cuts both, and the routing engine that declared them diverse was wrong. A conduit is the shared-risk link group that makes that answerable. Spans in one conduit fail together. That is the entire fact, and it needs identity and membership, nothing more. Adding a route geometry would turn the conduit into a weak imitation of a GIS system while adding nothing to the diversity question. **An absent conduit means unknown, not diverse.** `OtnFiberSpan.conduit` is optional, because a span has to be creatable before anyone has surveyed which duct it runs in. A span with no conduit is a span whose shared risk has not been recorded. Treating that as proof of diversity is the single most expensive misreading available in this model, and no schema constraint can prevent it. The absence of data and the absence of risk look identical in the YAML. ## Declaring diversity, and what that makes enforceable[​](#declaring-diversity-and-what-that-makes-enforceable "Direct link to Declaring diversity, and what that makes enforceable") A conduit answers "do these two routes share a duct". It does not answer "were they supposed not to". Those are different questions and only the second one can block a merge. `OtnDiversityGroup` is where the second one is written down. It is a small kind: a unique name, a description, and the services in it. Two services pointing at one group are declaring that no single cut may take both. `checks/diversity.py` reads the group, walks each member's spans out to their conduits and fails the proposed change when two members share one. **It says nothing at all about services that declared nothing, and that silence is the feature.** `transforms/srlg_exposure.py` argues in its own docstring that shared risk should be reported and not enforced. An operator may have accepted an exposure deliberately, and a check has no way to know. That objection is correct against a check that flags every shared duct. This check answers it rather than overriding it: it only ever speaks about a promise somebody wrote down. An accepted exposure stays accepted. The question "who shares a duct with whom, declared or not" keeps its existing answer in the [SRLG report](/demo-otn/reporting-scenarios.md#scenario-seven-which-services-are-not-diverse), which is the right instrument for it because a report blocks nothing. The silence is a property of the query rather than of a condition in the Python. `queries/diversity.gql` is rooted on `OtnDiversityGroup.services`, so a service with no group is never fetched. A later change cannot invert an `if` that is not there. **A group is a node, not a text label, and the reason is worth keeping.** The first draft made it a string on the service. Two services would then have been in one group when their strings matched. `gold-pair` and `gold_pair` would be two groups of one, each of which the check passes in silence, and silence is already what this field means when it is absent. There would be no signal separating "no promise made" from "promise made and typed wrong". As a relationship, a mistyped group name is refused at write time with `Unable to find the node ... in the database`. **A member with no route yet is undetermined, not compliant.** Declaring the group before provisioning the circuits is the normal order of work. The check reports that case at INFO rather than passing it. A green check on an unprovisioned pair would be the same failure as the text label, arriving later. ## A refusal is an answer, and it does not merge unless somebody signed for it[​](#a-refusal-is-an-answer-and-it-does-not-merge-unless-somebody-signed-for-it "Direct link to A refusal is an answer, and it does not merge unless somebody signed for it") The provisioning generator either builds a service or refuses it, and a refusal is written onto the node: `status` goes to `rejected`, `rejection_code` takes one of six values, and `rejection_detail` holds the prose. `checks/provisionable.py` reads those three fields and fails the proposed change for every service that is refused and **not accepted**, naming the service, the code and the detail. The invariant is one sentence: **the default branch holds only services that can be provisioned, or refusals somebody signed for.** **The gate and the escape hatch are one mechanism.** On its own the gate reads as "refusals are forbidden", which is false. `refusal_accepted` is a `Boolean` that only a person ever sets. It records that an operator read the code and the detail and decided the record was worth keeping. Some refusals are the answer rather than a failure. Madrid to Warsaw at 400G refused for optical budget is a demo scenario whose entire point is the recorded refusal, and no amount of improving the network provisions it. Set the flag and the check goes quiet about that service, and the branch merges carrying a documented no. Refusals are not forbidden here. Unread ones are. This is the same shape as the diversity check above, and deliberately so. That one speaks only about promises somebody wrote down; this one speaks only about refusals nobody has signed. Widening either would delete the capability the silence exists to protect. The unconditional question, "which services are refused, signed for or not", has its own answer in the service trace report, which is the right instrument for it because a report blocks nothing. **The generator may clear the flag and may never set it**, and only at one boundary. While the service stays refused a rerun leaves the signature alone, or it would silently un-accept a decision somebody made. When the service provisions, the generator clears it, because the refusal that was accepted has ceased to exist. Without that second half the flag survives onto a working service, and the check then blocks the merge because the network improved. **Which of the three fields decides is worth drawing**, because the order the check reads them in is what the state table cannot show: `status` is read first, so a leftover code on a service that is no longer rejected blocks nothing. It is reported at INFO instead, because a stale code makes every filter by reason code wrong and a reader should hear that it was seen and deliberately not acted on. Acceptance is read **after** readability, which is the fail-closed rule and the one thing a straight reading of the table misses. A service marked `rejected` with no code is an error whether or not the flag is set. Accepting a refusal means having read it, and a refusal with no code cannot have been read. The two error paths on the right of the drawing are states the generator cannot produce. Reaching either means a hand edit or a write path nobody modelled, which is exactly what a check is for. **Why this cannot be a schema constraint.** The rule is not "this value may never be written", it is "this value may not be **merged** unaccepted". Merge is not a write path the schema sees. A constraint forbidding `status: rejected` would refuse the generator's own write, midway through this same pipeline, and recording refusals is the generator's job. The other two rules are about absence, and a constraint governs what is written and is blind to a gap. **Gating on a derived verdict is safe here for one reason.** A capacity verdict does go stale, and a three-week-old refusal blocking an unrelated change would be a real objection. It does not apply inside the pipeline. The generator is registered with `targets: optical_services` and runs **ahead** of the checks in the same proposed change. The verdict the check reads was written seconds earlier against this branch's own data. This check never sees a refusal that has been resting on the default branch. The [schema reference](/demo-otn/schema-reference.md#the-six-reason-codes-and-the-signature-beside-them) lists the six codes and what each colour means. ## Sections group spans, and do not own them[​](#sections-group-spans-and-do-not-own-them "Direct link to Sections group spans, and do not own them") An optical multiplex section runs ROADM to ROADM. It groups the ordered spans and the inline amplifiers between two degrees, which is the unit the capacity question is asked about. A section carries 4,800,000 MHz of C-band, not a span and not a path. Three things about the shape matter before loading data. **A span's position in its section is an integer on the span**, not an ordered list on the section. `oms_sequence` counts from the A end, and a span has no direction of its own, so one number covers it. The coarse tail has a sequence of 1 and no section, which is what keeps it out of the budget engine. **A section holds two amplifier chains, one per direction of travel.** Light crosses the section both ways and an amplifier restores power in one of them. A section with N spans holds N+1 amplifiers each way. The section holds each chain in its own relationship, `amplifiers_a2b` and `amplifiers_b2a`, and the two are budgeted separately. **No amplifier stores which chain it is in.** The relationship holding it already says so, and an attribute restating that would be a second copy that can disagree with the first. Read from the amplifier's own page the same fact is which of its two section relationships is set. An amplifier's `oms_sequence` counts along the direction it amplifies rather than from the A end, which is where it differs from a span's. See the [link budget](/demo-otn/link-budget.md) page for what that means for the arithmetic. **Nothing stops two spans claiming the same position.** A uniqueness constraint spanning `oms` and `oms_sequence` was tried and rejected by the platform: ``` uniqueness_constraints: cannot use grp relationship, relationship must be mandatory. ``` The constraint needs a mandatory relationship. `oms` has to stay optional so a span is creatable before the section it will join exists. Duplicate positions are therefore a data rule with no enforcement anywhere in this repository. Neither the schema nor any check rejects them, and `plant.py` sorts by `oms_sequence` without asking whether the values are distinct. It is stated here so the gap is visible rather than assumed covered. ## Two wavelength plans, and why they are two kinds[​](#two-wavelength-plans-and-why-they-are-two-kinds "Direct link to Two wavelength plans, and why they are two kinds") The core runs on the dense grid: ninety-six channels of 50 GHz each, between 191.35 and 196.10 THz, all inside the C band. That is ITU-T G.694.1, and `OtnFrequencyGrid` holds it. Those two frequencies are channel **centres**, 4,750,000 MHz apart. The band the capacity arithmetic runs against is the edge-to-edge extent, 191.325 to 196.125 THz, which is 4,800,000 MHz. The two figures differ by one channel width and using the wrong one silently loses a channel of capacity, so [the spectral model](/demo-otn/spectral-model.md) page keeps them apart and `units.py` names both. ITU-T G.694.2 is the other plan. It is coarse: eighteen wavelengths spaced 20 nm apart, from 1271 nm to 1611 nm. They are named by wavelength rather than by frequency, because 20 nm is far too wide for the frequency precision the dense grid needs. `OtnCwdmChannel` holds it, and the two kinds stay apart. | | G.694.1, dense | G.694.2, coarse | | ---------------- | -------------------------------------- | -------------------------- | | Kind | `OtnFrequencyGrid` | `OtnCwdmChannel` | | Channels | 96 | 18 | | Spacing | 50 GHz | 20 nm | | Range | 191.35 to 196.10 THz, centre to centre | 1271 to 1611 nm | | Named by | Channel number and frequency | Nominal central wavelength | | Can be amplified | Yes, all 96 | No | One kind holding both plans would be less code and a worse model. `OtnOpticalCarrier.channel` is mandatory and peers `OtnFrequencyGrid`, so the server rejects a carrier pointed at a coarse wavelength before the write lands, with `must be of type: ['OtnFrequencyGrid']`. Merge the plans and that rejection disappears, and the rule that replaces it is a check in a pipeline instead of a constraint at write time. A plan entry is not an optical element. `OtnCwdmChannel` inherits nothing. Light does not pass through a plan entry, it passes through the multiplexer that selects one, and the multiplexer is the element that attenuates it. ## Coarse optics are metro access, not core[​](#coarse-optics-are-metro-access-not-core "Direct link to Coarse optics are metro access, not core") The demo has one coarse link: Amsterdam Science Park to Amsterdam, 18.4 km of G.652.D, four lit wavelengths at 1471, 1491, 1511 and 1531 nm, a passive multiplexer at each end and no amplifier anywhere on it. That shape is the whole argument for coarse optics. Over 18.4 km a pair of passive multiplexers costs less than a pair of coherent transponders, and 20 nm of spacing means the lasers need no temperature stabilisation. Four wavelengths on one fiber pair is all the campus needs. The same shape is the argument against putting coarse optics on a core section. Amsterdam to Brussels is 220 km, the shortest section in the network, and Paris to Madrid is 1250 km. Every core section needs amplification, and coarse wavelengths cannot be amplified: * Sixteen of the eighteen sit outside the erbium window. An erbium-doped fiber amplifier has gain roughly between 1530 and 1565 nm. Five coarse wavelengths are in the O band, five in the E band, three in the S band and three in the L band. An amplifier fitted to a coarse link would do nothing for any of them. * The remaining two, 1531 and 1551 nm, are inside the erbium window and are still not usable as dense channels. Neither lands on the 50 GHz grid, so neither is writable as an `OtnFrequencyGrid` channel. The overlap between the two plans is real and of no use. Eighteen coarse wavelengths against ninety-six grid positions is the second half of the argument. A core section is a capacity question, and the coarse plan answers it with a fifth of the spectrum and no way to amplify what it does carry. ## The coarse tail is outside the budget engine, deliberately[​](#the-coarse-tail-is-outside-the-budget-engine-deliberately "Direct link to The coarse tail is outside the budget engine, deliberately") `OtnFiberType.attenuation_coefficient_mdb_per_km` is one number per fiber family, measured at 1550 nm. It is a coefficient rather than a loss, and the name says so because the attenuators are the things in this model that have an attenuation. The budget engine applies it to every span it is handed. On the dense grid that is accurate to a few thousandths of a decibel. On a coarse wavelength it is not. Silica attenuates more outside the C band, and the further out the wavelength sits the larger the gap. At the stored 200 mdB/km, the four-term formula above gives the 18.4 km tail 4.530 dB of fiber, splice and connector loss with the ageing allowance left out. The four lit wavelengths on that tail run at 1471, 1491, 1511 and 1531 nm, and none of them is the wavelength the coefficient was measured at. This repository holds no per-band attenuation figure, so it cannot state by how much the stored number understates them, and no test here derives one. For scale on why the gap matters at all, Paris to Madrid misses its OSNR target by 0.535 dB at `DP-16QAM 64GBd 400G`. A fraction of a decibel is the size of the margin that decides whether a section closes. So the tail belongs to no optical multiplex section. The budget engine reaches spans only through sections, so a span in no section is a span the engine never sees. `OtnFiberSpan.oms` is optional and writable: set it on the tail and the next run prices that span at the wrong coefficient and reports a margin that looks fine. Two tests enforce the boundary. `OtnSite.site_type` marks the campus `customer`, and the first test refuses an `oms` on any span touching such a site. The second asserts that the only multiplexers lighting a coarse wavelength are the two on the tail, so a core multiplexer cannot quietly acquire one. Modelling a coarse budget honestly needs an attenuation figure per wavelength band, which is a schema change rather than a note. ## Negative result: cFEC reaches no section here, and oFEC reaches almost all of them[​](#negative-result-cfec-reaches-no-section-here-and-ofec-reaches-almost-all-of-them "Direct link to Negative result: cFEC reaches no section here, and oFEC reaches almost all of them") The optical-mode catalog is data precisely so this kind of question is answered by a query rather than by an assumption. The answer is not "pluggables do not reach". It is a statement about forward error correction. | Mode | FEC | Reach | Required OSNR | Dispersion tolerance | Sections reached | | -------------- | ---- | ------- | ------------- | -------------------- | ---------------- | | `400ZR` | cFEC | 120 km | 26.0 dB | 2400 ps/nm | 0 of 21 | | `800ZR` | cFEC | 120 km | 29.0 dB | 1400 ps/nm | 0 of 21 | | `OpenZR+ 400G` | oFEC | 1000 km | 23.0 dB | 50,000 ps/nm | 20 of 21 | | `OpenZR+ 300G` | oFEC | 2000 km | 20.5 dB | 60,000 ps/nm | 21 of 21 | | `OpenZR+ 200G` | oFEC | 3000 km | 17.5 dB | 80,000 ps/nm | 21 of 21 | The shortest section in the loaded network is **220 km**, Amsterdam to Brussels, and the next two are 320 km and 330 km. Both cFEC parts are 120 km parts, so both reach zero, and the ordering does not change if the reach figure moves by a few tens of kilometres. `OpenZR+ 400G` misses only Paris to Madrid at 1250 km, and the 200G rung reaches as far as the 100G transponder does. **The mechanism is dispersion, not power, and that is the part worth knowing.** Put a 400ZR on Amsterdam to Brussels and the OSNR gate passes with **+7.890 dB** of margin. The dispersion gate is what refuses it: 220 km accumulates 3740 ps/nm against a 2400 ps/nm tolerance, over by 1340. On the two 330 km sections it is over by 3210. A DCI optic has power to spare on a regional section and no dispersion budget at all, because it was designed for a span with almost no accumulated dispersion and no compensation in front of it. That is what oFEC buys, and it is why OpenZR+ exists. Three wavelengths in this dataset run `OpenZR+ 400G` router to router, on those same three sections, and all three close at the top rung: **+10.890 dB** on Amsterdam to Brussels and **+7.747 dB** on each of Berlin to Prague and Hamburg to Berlin. Nothing had to drop to the 300G or 200G rung. What matters to a planner is when the answer arrives. A network built on the assumption "pluggables are cheaper, so use pluggables" would have found this out after buying them, and would have blamed the link budget rather than the FEC. ## Milan carries nineteen transponders and Brussels two[​](#milan-carries-nineteen-transponders-and-brussels-two "Direct link to Milan carries nineteen transponders and Brussels two") Each PoP gets `max(2, ceil(terminations / 2))` transponders, with two line ports on each. That gives Milan 19, Frankfurt 13, Amsterdam 4, Berlin 3 and every other PoP the floor of 2. Fifty-nine transponders, 118 line ports, of which 80 carry a wavelength. The dataset holds 126 line ports in all: six more sit on routers and carry the three OpenZR+ wavelengths, and two are the line side of the Frankfurt regenerator. The spread is not a modelling choice. It is what the carrier plan already said and the device count used to hide. Forty of the 43 wavelengths in the shipped plan ride the Frankfurt to Milan corridor, so Milan is an endpoint of 37 of them and Frankfurt of 25. When every site held the same two or four transponders, nothing in the inventory showed that. Now the transponder count is the traffic map. **The negative result is the other half of the same sentence.** Eight of the fourteen PoPs terminate no wavelength on a transponder: Brussels, Copenhagen, Geneva, Hamburg, London, Madrid, Prague and Warsaw. Each sits at the floor of two transponders, and all four of its line ports are dark. Thirty-eight of the 118 transponder line ports carry nothing, and 40 of the 126 line ports in the dataset do. Three of those eight are not idle any more, and the reason is a different piece of hardware. Brussels, Hamburg and Prague each terminate one of the router wavelengths on a pluggable in a router line port, which is the whole point of IP over DWDM: reach short enough that the transponder buys nothing. Five PoPs terminate no wavelength by any route: Copenhagen, Geneva, London, Madrid and Warsaw. They keep two transponders rather than none because five of those eight are endpoints of shipped demo scenarios, and the scenarios live in `demo/`, which is loaded by hand and never by git sync. Madrid and Warsaw are the quick start's headline refusal; London is the ten-into-one grooming runbook; Prague and Geneva are service endpoints. The floor is a modelling argument rather than a functional one, and the difference is worth stating. Provisioning would still run with no transponder at those sites, because `generators/optical_service.py` names no line port and binds none. What breaks is the network a reader is looking at: a PoP that can never terminate a wavelength is not a credible node, and someone who provisions a 400G service at Madrid and then finds no transponder in Madrid's inventory is reading a network that does not hold together. Only Copenhagen is genuinely idle now: Brussels and Hamburg terminate a router wavelength each. ## What the equipment reports, and what the model predicts[​](#what-the-equipment-reports-and-what-the-model-predicts "Direct link to What the equipment reports, and what the model predicts") Every optical figure elsewhere in this demo is computed. The budget engine predicts an OSNR cascade, and the check compares that prediction against what a modulation format needs. None of that is a measurement. A monitor is where the measurement goes. It is the interface an engineer looks at on real equipment, and each device family reports a different set, because each measures something different. | Monitor | What it reports | | ------------ | ------------------------------------------------------------------------------------------------ | | Amplifier | Input power, output power, gain, gain tilt | | ROADM degree | Total power, channel count present | | Mux or demux | Total power, channel count lit | | Raman | Pump power, gain achieved, back reflection | | Receiver | Rx power, OSNR, pre-FEC bit error rate, Q factor, chromatic dispersion, differential group delay | **Each row is its own kind.** Five kinds, not one kind with a field saying which family it belongs to. The readings barely overlap: of the fourteen in the model, eleven appear on exactly one family. The names suggest more sharing than there is, because input power, total power, pump power and received power are four different measurements that happen to have "power" in four different names. Three readings are shared, and only in two pairs: measured gain between an amplifier and a Raman pump, and total power and channel count between a ROADM degree and a multiplexer. At that separation the schema can express the whole thing itself. Every reading is mandatory on the kind that declares it, so a monitor missing one is refused. And a kind has no field at all for a reading its hardware cannot produce, so that is refused too. The two kinds that do report the same pair, a ROADM degree and a multiplexer, share those two readings through a generic. They stay separate kinds, because a degree and a multiplexer are different equipment. **OSNR sits on the receiver and nowhere else.** A coherent receiver's digital signal processor computes it as part of recovering the signal. An amplifier measures power, not noise ratio, so OSNR on an amplifier monitor would be a number its hardware has no way to take, and there is nowhere to put it. **A receiver with no wavelength reports loss of signal.** Every receiver monitor in the dataset used to carry the same block of plausible numbers, so a transponder with nothing on its line ports still claimed a healthy 25.1 dB of OSNR at -9.000 dBm. Sixteen of the fifty-nine monitors now say the opposite: received power at -40.000 dBm, which is the floor of the range, no OSNR, no Q factor, no dispersion and no differential group delay, and a pre-FEC bit error rate of 0.5, which is what pure noise gives. The other forty-three are derived from the route their carrier takes and no longer agree with each other, running from 24.786 dB on the 1580 km Frankfurt to Vienna hop to 28.025 dB on the 780 km Frankfurt to Milan one. The monitor still exists on a dark transponder rather than being left off. The completeness check gates on a transponder that has none, the six readings are mandatory on the kind, and reporting darkness is more use than removing the box: it separates dark from healthy instead of erasing the difference. There is one thing the schema still cannot say: nothing stops a receiver monitor being attached to an amplifier. A port's device relationship points at the generic device kind, and a concrete kind cannot narrow the peer of a relationship it inherits. The schema has no way to express the restriction, and recovering it in code would be the layer these five kinds exist to leave, so the gap is stated instead. ## A reading is a last known value, not a feed[​](#a-reading-is-a-last-known-value-not-a-feed "Direct link to A reading is a last known value, not a feed") Every monitor records the time its reading was taken, and that attribute is mandatory. A reading nobody can age is not evidence of anything. This is a source of truth for intended state. A live counter belongs in a time series database, and modelling one here would be the wrong tool. What belongs here is the most recent observation, because that is what makes drift computable: predicted against observed, with an age attached so a stale number cannot pass for a fresh one. One consequence worth stating. A real channel monitor reports power for every channel, ninety-six values on a loaded degree. That is a vector, and the attribute kinds able to hold one are not filterable, sortable or usable in a computed attribute. Ninety-six child objects per monitor would add forty thousand objects to a dataset of two thousand. Total power and channel count give the operational meaning without the vector, and the per-channel detail is left to the system built for it. ## What this layer deliberately does not have[​](#what-this-layer-deliberately-does-not-have "Direct link to What this layer deliberately does not have") | Absent | Why | | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `total_loss_mdb` on a span | Computed from length, attenuation, splices and connectors. | | `total_length_m`, `span_count` on a section | Sums over `spans`. | | A port or device relationship on a span | A span is not racked and light does not enter it through a port. | | An inverse for `site_a` and `site_b` on `OtnSite` | "Every span touching Amsterdam" is a native filter on the span side. Two lists on the site would present one fact twice. | | Route geometry on a conduit | The conduit answers shared risk, not where the trench goes. | | Channel occupancy on the frequency grid | Capacity is the C-band minus the union of the intervals in use, each interval centred on a carrier's anchor and sized by its mode. See [the spectral model](/demo-otn/spectral-model.md). | --- # Demo guide Nine scenarios against the loaded GÉANT dataset. Every number on the three pages below is produced by the engine and asserted by a unit test. | # | Scenario | Answers | | - | --------------------------------------------- | --------------------------------------------------------- | | 1 | Provision Berlin to Amsterdam at 400G | Where does the next service go, and what did it not pick? | | 2 | A corridor out of colour, and what still fits | Why does one service on it provision and the other not? | | 3 | What spectrum is left, and what can use it | Can another 400G fit? | | 4 | Cut the Frankfurt to Amsterdam fiber | What goes down, and whose is it? | | 5 | Trace a service end to end | What does this service run over? | | 6 | Where do the cheap pluggables reach? | The cFEC parts nowhere, the oFEC parts almost everywhere. | | 7 | Which services are not diverse? | Who shares a duct with whom? | | 8 | The AI services against their budgets | How much latency headroom is left? | | 9 | The InfiniBand handover | What happens when a service states its own client signal? | `uv run invoke demo` runs them in the order below, on one branch, and this is also the order the runbook follows. Everything left of the provisioning step reads a branch that holds the dataset and nothing else. Everything right of it reads services, so it needs the provisioning step to have run first. The scenarios live on three pages, and this page is the way in to all three. **[Provisioning scenarios](/demo-otn/provisioning-scenarios.md)** covers what to do before you start, then the two scenarios that write: provisioning Berlin to Amsterdam at 400G, and a corridor that has run out of colour where one service still fits and another does not. **[Reporting scenarios](/demo-otn/reporting-scenarios.md)** covers the six scenarios that read rather than write, from what spectrum is left to which services are not diverse, and the two maps every PoP carries as artifacts. **[Loadable scenarios and the runbook](/demo-otn/loadable-scenarios.md)** covers the three pairs loaded by hand rather than driven by a task, including [the ODU layer pair](/demo-otn/loadable-scenarios.md#two-loadable-scenarios-for-the-odu-layer), which fills one wavelength to capacity and then puts every band on the [ODU map](/demo-otn/odu-map.md) at once. The runbook and the clean-up steps are there too. If you are running this in front of people, skip to [the runbook](/demo-otn/loadable-scenarios.md#the-runbook). It opens on the [network map](/demo-otn/network-map.md) and then runs the same nine scenarios in narrative order, with the setup done once. --- # Developer guide How this repository is laid out, how to change it, and how to check that you did not break anything. ## Layout[​](#layout "Direct link to Layout") ``` schemas/ Infrahub schema YAML. Eight files, 50 kinds. objects/ The GÉANT dataset plus the generator target group. Loaded by .infrahub.yml. demo/ Scenario input for the walkthrough. NOT loaded by .infrahub.yml. src/ infrahub_demo_otn: the pure Python nobody needs a server to test. queries/ GraphQL, one file per artifact that reads it. checks/ Validation that blocks a merge. transforms/ Reports that answer a question. generators/ Automation that writes objects. menus/ The sidebar. scripts/ The dataset generator and the basemap extractor. tests/unit/ Everything above, offline. tests/integration/ The same thing against a live Infrahub in a container. docs/ This site. ``` ### `objects/` against `demo/`[​](#objects-against-demo "Direct link to objects-against-demo") The distinction matters more than it looks. `objects/` is the network, plus the one object the repository configuration depends on. Eleven of its eighteen files, the `objects/1*.yml` series, are produced by `scripts/generate_geant_dataset.py` from a fixed seed, guarded by `tests/unit/test_geant_dataset.py`, and loaded by `.infrahub.yml`. A manual edit to one of those fails the regenerate-and-diff test on the next run. The seven `objects/0*.yml` catalogs are input, not output: the generator reads `03_optical_modes.yml` rather than writing it. Fifteen sites, of which fourteen are PoPs, 21 optical multiplex sections, 133 fiber spans, 12 conduits, 445 devices, 1612 ports, 43 wavelengths, the 43 line containers riding them, and three O-E-O devices. Those three are last in the load order, because each names the wavelengths it terminates and a carrier has to exist before a device can point at it. The same reason put the Raman pumps in `15`. Transponders are the one device count the generator sizes per site instead of fixing. A PoP gets `max(2, ceil(terminations / 2))` of them and two line ports on each, so Milan holds 19, Frankfurt 13, and the eight PoPs that terminate no wavelength on a transponder hold two apiece. Fifty-nine in total, 118 line ports, of which 80 carry a wavelength and 38 are dark. The dataset holds 126 line ports, not 118. Six sit on routers and carry the three OpenZR+ wavelengths on pluggable optics, and two are the line side of the Frankfurt regenerator. Nine transceiver units against five part numbers go with them: six fitted and in service, two spares on a shelf and one returned to the vendor. Four attenuators, two fixed pads and two VOAs, are racked and on no path yet. [Concepts](/demo-otn/concepts.md) says why the spread is that wide. The per-kind counts live in `scripts/geant_manifest.json`, and the totals a reader sees are on [Installation and setup](/demo-otn/installation-setup.md), where `tests/unit/test_doc_claims.py` reads them back off the page. The exception is `objects/00_groups.yml`, which creates the generator target group that `.infrahub.yml` names twice. It is part of the configuration, not a fact about the network and not scenario data. A registration that does not create it produces a generator with no target, a pipeline that goes green, and no output. `demo/` is scenario input. It holds twenty-three service requests, six Raman pumps, three O-E-O regenerators and a fourth on the branch that closes. It also holds three extra wavelengths that exist to spend the last usable spectrum on one corridor, and twenty-two client containers that exist to make the same corridor full of slots. Three pairs of scenarios each demonstrate one thing twice. | Pair | What the two halves show | | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `04_odu_ten_in_one.yml`, `05_odu_mixed_fill.yml` | Ten circuits packed into one wavelength and the eleventh refused, with the refusal signed for so the branch merges; then all five [ODU map](/demo-otn/odu-map.md) bands on one branch. | | `06_mad_waw_16qam.yml`, `07_mad_waw_qpsk.yml` | Three regenerator sites all refused at DP-16QAM and nobody signs, so this is the one scenario whose proposed change is blocked; then the same Frankfurt split closing at DP-QPSK, which refuses nothing at all. Two branches, because the pair is a before and after. | | `08_diversity_mil_feeds.yml`, `09_diversity_fra_feeds.yml` | A declared diversity group that holds; then one that does not. One branch, so a single check run reports both. | None of it is a fact about the network, `.infrahub.yml` does not reference it, and `tests/unit/test_repository_config.py` asserts that it never will. Loading `demo/90_fra_mil_saturated.yml` with the dataset would make the published capacity claim of 4,134,400 MHz occupied and 665,600 MHz free false. It would also make every published occupancy figure for `oms-fra-mil` false at the ODU layer. If you are adding data, the question is which of the three it is. Facts go in `objects/` through the generator. Things the repository configuration names go in `objects/` manually. Things that exist to make a demo step happen go in `demo/`. ### `src/infrahub_demo_otn/`[​](#srcinfrahub_demo_otn "Direct link to srcinfrahub_demo_otn") | Module | Job | | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `units.py` | Every scale factor in the project. The only file allowed to hold one. | | `budget.py` | Loss, OSNR, dispersion and latency over an ordered chain, in both directions. | | `routing.py` | Route and mode selection. Pure functions over plain inputs. | | `plant.py` | GraphQL payloads into engine inputs. The adapter layer. | | `impact.py` | Occupancy, reach, exposure and latency verdicts. The reporting layer. | | `cartography.py` | Projection, framing and label placement. Geometry, no optics. | | `basemap.py` | Generated coastline data. Coordinates and nothing else. | | `containers.py` | The tributary slot table and the capacity rule. One implementation, called by the generator, the check and the ODU map. | | `chains.py` | The carrier cover over a section route: which wavelengths, joined at which device, make one circuit. Bounded and pure. | | `drift.py` | Configured gain against what each stage last reported delivering. | | `mapchrome.py` | What both maps draw the same way: frame, basemap, node glyphs, title block, panel rhythm. | | `mapengine.py` | The one drawing engine both maps run through: band classification, section ordering, layout, routes, title and panel. Generic over a section type, driven by a dialect, and unable to tell which map it is drawing. | | `mapdraw.py` | The network map's dialect plus `render_map`: the margin bands and their edges, the panel columns, the distance labels, the totals. | | `odudraw.py` | The ODU map's dialect plus `render_odu_map`: the headroom bands and their edges, the panel columns, the branch-named empty caption, and the totals rule that counts a wavelength once across the sections it crosses. | None of these imports `infrahub_sdk`. That is the rule that keeps the whole decision layer testable with no server running, and it is why `uv run invoke test-unit` finishes in seconds rather than minutes. The checks, transforms and generators are the layer that imports the SDK. They unwrap a payload, call into the modules above, and write or format the answer. The workers import this package from the image, not from the branch. There is no bind mount. The package is installed into the image's virtualenv, and the git worker that clones a branch to run a check against it still resolves `import infrahub_demo_otn` inside the container. A branch that edits `budget.py` runs the copy that was baked in. Run `uv run invoke build` and `uv run invoke start --rebuild` after any change under `src/`, or the check goes green against logic that is not on the branch. A **new module** under `src/infrahub_demo_otn/` is the sharper case, and it has caught this repository more than once. Until the image is rebuilt and the workers restart, that module does not exist for the process that has to import it. The repository syncs, the definition registers, the pipeline reports success, and every render raises `ImportError`. Rebuild first, then check that the import works inside the container, and treat a rendered output rather than a green pipeline as the evidence that it did. ### Regenerating the basemap[​](#regenerating-the-basemap "Direct link to Regenerating the basemap") `src/infrahub_demo_otn/basemap.py` is generated. The coastlines and borders come from Natural Earth 1:50m, which is public domain. The script clips them to the window the map draws and simplifies them into Python literals: ``` node scripts/extract_basemap.mjs countries-50m.json src/infrahub_demo_otn/basemap.py ``` The input is the `world-atlas` release named at the top of the script. This is the one step with no invoke task behind it: it needs Node and a file you downloaded, and it is run by hand when the window or the source release changes. Commit the result. It never runs in CI, and the renderer reaches the network at no point: the data is already in the image by the time a map is drawn. The clip has to contain the frame the map draws, which is wider than the box the sites sit in: `Frame.fit` grows the short axis to the canvas aspect. A clip that stops inside the drawn frame paints sea where land is, with a straight vertical edge, and every other basemap test still passes. `test_the_clip_contains_the_frame_the_map_actually_draws` compares the two. ### Two kinds of committed render, and only one kind may be refreshed[​](#two-kinds-of-committed-render-and-only-one-kind-may-be-refreshed "Direct link to Two kinds of committed render, and only one kind may be refreshed") Ten rendered SVG files are checked in. Two are illustrations and eight are test references. They look alike and they are opposites. Read this before touching any of them. | File | What it is | When it changes | | --------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------ | | `docs/docs/media/network-map.svg` | The illustration on the overview and network map pages | Deliberately, when the network, the palette or the basemap changes | | `docs/docs/media/odu-map.svg` | The illustration on the overview and ODU map pages | On the same terms | | `tests/unit/fixtures/network_map_*.svg` | Four test references, invariants | Effectively never | | `tests/unit/fixtures/odu_map_*.svg` | Four test references, invariants | Effectively never | #### The illustrations on the documentation pages[​](#the-illustrations-on-the-documentation-pages "Direct link to The illustrations on the documentation pages") `docs/docs/media/network-map.svg` is Berlin's rendered artifact, checked in so the pages have a picture without a running stack. `docs/docs/media/odu-map.svg` is the same idea for the ODU map. Both are snapshots, not generated files: refresh them when the network, the palette or the basemap changes, by fetching the artifact off a loaded stack. ``` uv run invoke load-repository # if the stack has no repository yet # then, from the Artifacts tab on Berlin, download network-map and odu-map and # save them as docs/docs/media/network-map.svg and docs/docs/media/odu-map.svg ``` #### The screenshots on the quick start[​](#the-screenshots-on-the-quick-start "Direct link to The screenshots on the quick start") `docs/docs/media/quickstart-*.png` are captures of the web interface against a loaded stack. They go stale when the thing they show changes: `quickstart-blocked.png` lists every validator the pipeline runs, so it moves whenever a check is added. There is no interactive step. Drive a headless browser over the DevTools protocol and write a lossless PNG: ``` # 1. put the scenario on the stack, then open a proposed change from its branch uv run invoke demo-regenerator # 2. capture at 1512 wide, the width the other figures use, with the viewport # tall enough to hold the whole list. The rows sit in an inner scroll # container, so a viewport-sized capture silently cuts the bottom off. ``` Two things to check before committing one. Log in, or the sidebar reads `anonymous` where every other figure reads `Admin`. And confirm the rows the alt text names are actually in the image, because the alt text on these figures enumerates them and is the part a screen reader gets. #### The golden renders under tests[​](#the-golden-renders-under-tests "Direct link to The golden renders under tests") There are eight, four per map. None is an illustration and none is fetched from anywhere. Each is rendered from the fixture sites and sections in `tests/unit/test_mapdraw.py` or `tests/unit/test_odudraw.py`, and the four cases per map are picked to hold down different parts of the drawing. | Fixture | What it holds down | | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `network_map_golden.svg` | The render that already shipped: focused on Paris, read from a named branch, so the focus ring and the provenance line in the footer are both in the bytes | | `network_map_no_focus.svg` | The same sections with no focus site, so the ring and its label are absent | | `network_map_focused_on_milan.svg` | A second focus site, so a focus wired to one node would show | | `network_map_unknown_margin.svg` | A section whose margin cannot be classified, gating `UNKNOWN_BAND`'s own colour and caption | | `odu_map_golden.svg` | The ODU render that already shipped, focused on Frankfurt | | `odu_map_no_focus.svg` | The same sections with no focus site | | `odu_map_focused_on_milan.svg` | A second focus site | | `odu_map_one_section_per_band.svg` | One section landing in each headroom band, `NO_ODU_BAND` included, so every band's colour and caption sit in one file | **All eight were captured before the two maps were merged onto one engine**, from unmodified `src/`, and watched passing against the unmodified code first. That ordering is the whole point: a reference taken after a move asserts only that the moved code matches itself. `network_map_golden.svg` was captured on the same terms one refactor earlier, before `mapchrome.py` was extracted from `mapdraw.py`. The ODU map used to have none of these. The reason given here was that the golden render is specifically about not disturbing an artifact that already shipped. The ODU map does ship, to every PoP, so the argument was followed to its conclusion rather than reversed. The seven missing references were captured from pre-merge code, before `mapengine.py` existed, and then the merge had something to fail against. **No fixture may be refreshed to make a failing test pass.** A failure means a pixel moved, which is the entire reason the files exist. So a failure here has exactly two readings, and you have to decide which: 1. **A refactor moved a pixel.** The refactor is wrong. Fix the code, not the file. This is the case the invariant is for, and it is what gated merging the two maps into `mapengine.py`. 2. **You changed a map on purpose.** Then refreshing is a decision, it belongs in its own commit, and the commit message says what moved and why. Regenerate from the fixtures rather than from a stack, one case at a time: ``` uv run invoke maps-regenerate --case network_map_golden uv run invoke maps-regenerate --case odu_map_golden ``` The case name is the fixture's file name without its suffix, and `maps-regenerate` with no `--case` re-renders all eight. The failure message prints the exact command for the case that failed, so nobody has to assemble it by hand. Neither kind of file is the other's substitute. The illustrations are real artifacts off real data and they move whenever the dataset does. The golden renders are fixture data and they are supposed to be boring. Determinism is asserted separately and still is: every case renders twice in one process and the two outputs are compared, and both maps also render their records in reverse order. That asks whether arrival order reaches the bytes, which is a different failure from a moved pixel and needs no committed file to catch. ## Adding an optical element[​](#adding-an-optical-element "Direct link to Adding an optical element") Say you want an `OtnDcm`, a dispersion-compensating module, that sits inline and adds loss. 1. **Ask whether it needs to exist, then whether it already does.** `OtnOpticalElement` may already have the attributes you need, in which case a new kind adds nothing. 2. **Add the node** to `schemas/otn_devices.yml`, inheriting `OtnGenericDevice` and `OtnOpticalElement`. Generics cannot inherit generics in this version, so composition happens on the concrete node. 3. **Every quantity is a scaled integer** with the unit in the attribute name, because Infrahub has no floating-point attribute kind. Loss in millidecibels, length in metres, frequency in megahertz, latency in nanoseconds. 4. **Add the scale factor to `units.py`** if it is new, with a docstring saying what it is for. Writing `* 1000` anywhere else fails review. A `_display` template that divides by the wrong constant fails `tests/unit/test_schema_contract.py`, which reads the divisor back out of every Jinja template and checks it against the attribute's unit suffix. 5. **Add the paired `_display` computed attribute**, then add its name to `EXPECTED_DISPLAY_ATTRIBUTES` in the same test. The guard is deliberately not automatic: a new display attribute should be a decision, not a side effect. 6. **Teach `plant.py` to build it** if the budget has to see it, and `budget.py` to sum it. Both have exhaustive offline tests; add yours there before touching the schema, not after. 7. **Decide whether it belongs in `menus/otn.yml`.** Most new kinds do not: the sidebar has thirteen leaves and everything else is reached from an object page. `tests/unit/test_menu.py` asserts that every kind outside the sidebar is the peer of a relationship on one inside it, so a kind with no route in fails there. If it does belong in the sidebar, set `include_in_menu: false` on the node too: custom menus and the auto-generated per-kind sidebar are additive, so a kind in both appears twice. 8. **Ship the documentation page in the same change.** A change is not done until its page exists and the site builds. ## Adding a report[​](#adding-a-report "Direct link to Adding a report") Ten transforms exist. An eleventh is four files: 1. `queries/.gql`, selecting `id` and `__typename` on the top-level nodes. 2. A derivation in `src/infrahub_demo_otn/impact.py`, pure, with tests in `tests/unit/test_impact.py`. 3. `transforms/.py` with `query = ""` on the class. The query name is bound there and never in the YAML: `python_transforms` forbids a top-level `query:` key and the repository configuration model rejects extras. 4. Registration in `.infrahub.yml` under `queries:` and `python_transforms:`, plus a payload in `tests/unit/test_transforms.py`. That last one is not optional: the test module reads the registrations and fails if a transform has no payload, precisely so a new report cannot ship untested. Give it its own query. Several of the existing ten overlap heavily and none of them shares, because a query two artifacts share is a query neither can change. A transform that renders rather than reports puts its pure layer somewhere other than `impact.py`. The two maps keep their geometry in `cartography.py`, their primitives in `mapchrome.py` and their drawing order in `mapengine.py`. Each one declares what is its own alone in `mapdraw.py` or `odudraw.py`: its bands, its panel columns, its wording and its totals. A third map is a third dialect, not a third renderer. The transform under `transforms/` stays a shim that holds neither a colour nor a formula. ## Generator tracking, and the object that has to opt out of it[​](#generator-tracking-and-the-object-that-has-to-opt-out-of-it "Direct link to Generator tracking, and the object that has to opt out of it") This one is not in the Infrahub documentation in a form that answers the question it raises here, it was measured against a live stack, and it decides how the provisioning generator is allowed to write a shared object. Read it before changing anything under `generators/`. **A tracking group is per target, not per definition.** The SDK names the group from the generator definition name plus a hash of the run's parameters. This repository's generator declares one parameter, `service`, so every service gets its own group. At the end of a run, the SDK deletes the members of *that* group which the run did not write again. **Every `save()` joins the group, including an upsert of an object that already existed.** That is the sharp edge. A line container written by two services is a member of both services' groups, and it survives a re-run of either one only as long as that run keeps writing it. **The dangerous run is not a re-provision, it is a run that stops writing.** A refusal, a de-provision or a re-route onto a different carrier writes no container, so the shared one becomes an unused member and is deleted. A sibling service is still holding a child under it. Measured on a live branch: the sibling's own container survived, because every `OtnContainer` relationship carries `on_delete: no-action`. What was left was a client container reporting a carrier with the line container it multiplexed into missing. That is worse than a cascade, because nothing in the sibling's history says it happened. **`save(..., update_group_context=False)` opts out completely.** The default only applies when the flag is left unset, and an explicit `False` skips the group membership write. A node that was never a member of any run's group appears in neither side of the delete set, so nothing can reclaim it. Two rules follow, and they are both load-bearing: * Every line container save passes `update_group_context=False`. That is what lets a generator create a wavelength's container at all. * A line container the current run did not create is never written, not even upserted. An upsert is what joins the second group. The cost is stated rather than hidden: nothing reclaims a line container when its last child goes away. A lit wavelength stays lit until someone turns it down, which is what happens in the plant too, so the demo is not lying. Untracked objects still accumulate on a branch that provisions and then de-provisions. One more divergence worth knowing before you treat a local run as a rehearsal. `infrahubctl generator service=` builds its parameters as `{"service": name}`, while the group-driven path the pipeline takes builds `{"name": name}`. Different dict, different hash, different group. A local run cannot reclaim what a pipeline run created, so a local re-run is not a faithful rehearsal of the pipeline and should not be quoted as one. ## Running it[​](#running-it "Direct link to Running it") ### The gates[​](#the-gates "Direct link to The gates") ``` uv run invoke lint # ruff format, ruff check, mypy, yamllint, rumdl, vale uv run invoke test-unit # no Docker, about ten seconds uv run invoke schema-check # schema formatting, offline uv run invoke docs # the documentation site ``` `lint` runs all six linters. All of them run in CI on every change, and none needs a running Infrahub. The prose linter needs `vale` on your PATH. Everything else needs only `uv sync`, so `lint` skips vale when it is absent and says loudly that it did. There is no schema-validate subcommand. `invoke schema-check` is the offline schema gate, and it checks formatting rather than semantics; the semantic guard is `tests/unit/test_schema_contract.py`. ### The sixth gate, which needs Docker[​](#the-sixth-gate-which-needs-docker "Direct link to The sixth gate, which needs Docker") ``` uv run invoke stop # first, and this is not optional uv run invoke test-integration --tier=core # 13 tests, about six minutes uv run invoke test-integration # all 37, about fifty ``` It loads this repository into a throwaway Infrahub through testcontainers and reads it back, which is the only layer that catches a load regression: a renamed attribute, an unresolvable reference, a bound violation, a relationship declared on both sides. None of those is visible offline. **Two tiers, and CI picks between them by what you changed.** `core` is the load, the repository sync and the proposed-change pipeline: what proves this repository still works against Infrahub. `full` adds the module that drives all 48 tasks and every scenario, which is forty of the fifty minutes and can only be broken by a change to the demo's own logic. `.github/file-filters.yml` lists the paths that mean full, and everything else runs core. Nineteen of this repository's twenty-four commits touched none of them. Run `full` yourself before changing anything under `src/`, `tasks.py` or the six Infrahub directories, or push and let CI decide. **Stop the demo stack first.** The two share no port and no database, but they do share the machine's memory. During feature 016 the pair exhausted the container runtime, the test database was killed, and twelve of thirteen tests failed for a reason that had nothing to do with the code. `invoke stop` keeps the volumes, so `invoke start` brings the demo back as it was. **Run the whole class or nothing.** The tests run in definition order and each depends on the state the previous one left. `pytest -k` deselects the methods that create the branch and load the data. Every survivor then fails with `Branch: geant-integration not found` at HTTP 404, which reads like a broken test rather than a broken selection. Feature 017 is why this section exists. `OtnOduSwitch` inherits `OtnOpticalElement`, the pinned list of that generic's implementers was not updated, and every offline gate stayed green: the schema loads, the queries pass, and nothing offline enumerated the live generic. Only a query against a loaded graph could tell. There is now a unit test pinning the same set from the YAML, so the next one costs a second rather than seven minutes. The integration assertion stays: it proves the server returns those kinds, which is a different claim from the schema declaring them. ### Against a running stack[​](#against-a-running-stack "Direct link to Against a running stack") ``` uv run invoke start uv run invoke branch-create --name probe uv run invoke load --branch probe uv run invoke info --branch probe ``` The command-line tool underneath resolves the address from `.env` but **not** the token, so a read succeeds and the first write fails with an authentication error. Every task passes both, through the subprocess environment rather than on the command line, so a token never reaches `ps` or your shell history. `uv run invoke list` prints the tasks a reader needs, grouped, and `--all` adds the rest. The demo scenarios are tasks too: `demo-setup` prepares the branch, `demo` runs the whole walkthrough, and `demo-capacity`, `demo-reach`, `demo-provision`, `demo-provision-all`, `demo-trace`, `demo-impact`, `demo-srlg`, `demo-latency`, `demo-infiniband`, `demo-refusal`, `demo-budget` and `demo-drift` run one step each. Five more load a scenario file onto a branch of their own rather than one of the nine numbered scenarios: `demo-raman`, `demo-odu`, `demo-regenerator`, `demo-diversity` and `demo-monitor-gap`. `demo-clean` deletes the branches they leave behind. ### A fail-closed default costs a second load[​](#a-fail-closed-default-costs-a-second-load "Direct link to A fail-closed default costs a second load") `OtnClientSignal.auto_selectable` decides whether the rate rule may pick a signal when a service names none. It defaults to **false**, and the direction is the point: a client signal added later is excluded from automatic selection until somebody decides otherwise in a diff. A default of true would admit it silently, which is the failure the flag replaced. **The consequence is an ordering rule.** Loading the schema onto a branch applies the default to every row that already exists, so every signal becomes non-selectable at that moment. The generator then refuses a service that states no client signal, correctly, until `objects/04_client_signals.yml` is loaded on top and restores the nine true values. ``` uv run invoke load-schema --branch uv run invoke load-objects --branch --file objects/04_client_signals.yml ``` A schema load alone is no longer enough on any branch holding client signals. This was measured during the migration rather than predicted: the refusal appeared, and it was the flag working rather than a fault. The same shape applies to any mandatory attribute given a fail-closed default on a kind that already has rows. The schema load is half the migration. ### Registering the repository[​](#registering-the-repository "Direct link to Registering the repository") Infrahub pulls a git repository and runs its checks, transforms, generators and artifacts inside the proposed-change pipeline. Until a repository is registered none of those definitions exist, so no check runs on a change and no artifact is attached to an object. `uv run invoke init` registers it as its last step, so a stack built from that one command already has the pipeline on. To do it against a stack you already have: ``` uv run invoke load-repository ``` It exports the committed tree into `.remote/`, which the containers see as `/remote`. It points a `CoreRepository` at that path and waits for the import. Then it counts what the sync created, which is the part worth watching: a repository can reach in-sync and create nothing, and the count is the only thing that says so. On this repository it creates fourteen check definitions, ten Python transforms, three artifact definitions, one generator definition and 25 GraphQL queries. Two details decide whether this works. **The location is resolved inside the container.** A path on your machine is a path the server cannot reach. That is why `docker-compose.override.yml` bind-mounts `./.remote` to `/remote` on the server and both workers. Containers that were running before that mount was added do not have it; `load-repository` checks and tells you to restart rather than leaving the import to fail. **The export is `git archive HEAD`, not a copy of your working tree.** What Infrahub imports is what you committed. An uncommitted change is not in the export, and a copy of the working tree would carry `.venv/` and `docs/node_modules/` into a directory the containers have to read. `tests/integration/test_infrahub.py` does the same thing a different way, with `GitRepo` from `infrahub_sdk.testing.repository` against the `/remote` directory the testcontainers stack mounts. That is what CI exercises. You can still run any artifact directly. That is faster than opening a change while you iterate on one: ``` uv run invoke check --name uv run invoke demo-capacity # and the other demo-* tasks uv run invoke demo-provision --service ``` **Order matters.** Infrahub creates a matching git branch in every registered repository at the moment an Infrahub branch is created. A branch that already existed when the repository was added has no branch on the repository side, and the proposed-change pipeline then has no commit to run that repository's checks from. Register the repository first, branch second. `init` does them in that order; if you add the repository to a stack that already has branches on it, recreate those branches. `tests/unit/test_repository_config.py` checks the configuration offline, with no server. It parses `.infrahub.yml` with the loader the SDK ships. It then checks that every registered path exists, every named class is defined, every query binding sits on the side of the fence its artifact type requires, and every artifact definition names a real transform and a content type from the eight-value allowlist. ## The rules that constrain a change[​](#the-rules-that-constrain-a-change "Direct link to The rules that constrain a change") **No floating point.** Infrahub's `Number` is an integer, and JSON attributes are not filterable, sortable or usable in computed attributes. Everything is scaled. **Generics cannot inherit generics.** Compose flat generics on the concrete node. **Every `Number` needs both bounds.** `min_value` and `max_value`, always. The schema contract test rejects one-sided bounds, including for the one attribute whose lower bound is negative. **No hand-written graph walk.** Path discovery is `client.traverse_paths()`, and `included_kinds` does not constrain the walk; `relationship_filter` does. **Absolute imports only.** `from infrahub_demo_otn.x import y`. Never relative, never through `sys.path`. **State the negative results.** If the model says 400ZR does not reach, or a route is too slow, the documentation says so. A demo that reports only its successes has not been tested against its own data. **A test earns its place if a change to a file in this repository can make it fail, and a change to Infrahub alone cannot.** Test the model, the data and the Python here. Do not test that Infrahub stores what it was told to store, that a pin is pinned, or that another test would fail if it were wrong. One test is a stated exception and says so in its own docstring. It is the integration test that asserts a rejected write arrives as HTTP 200 with a GraphQL `errors` array, because every check, generator and transform here reads its errors that way. --- # Install and load the demo ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Docker with Compose v2 * [uv](https://docs.astral.sh/uv/) * About 8 GB of free memory for the Infrahub stack ## Start the stack[​](#start-the-stack "Direct link to Start the stack") ``` git clone https://github.com/opsmill/infrahub-demo-otn cd infrahub-demo-otn cp .env.example .env uv sync uv run invoke init ``` `invoke init` destroys any existing stack, starts a new one, loads the schema, the menu and the dataset, and registers this repository. From nothing to a demo you can browse, in one command. Registration is the step that turns the proposed-change pipeline on. It creates the check definitions, the transforms, the artifact definitions and the generator definition from `.infrahub.yml`, so a change you open gets checked and every PoP carries a rendered map. Without it Infrahub holds the data and none of the automation. `uv run invoke load-repository` runs it on its own against a stack you already have. The first step builds a custom Infrahub image. The build installs the `infrahub_demo_otn` package into the image's virtualenv so checks, generators and transforms can all import it. `invoke start` on its own builds the image only when it is missing; pass `--rebuild` to force it. Infrahub is then available at . `uv run invoke list` prints the tasks you need, and `--all` adds the rest. `uv run invoke info` reports the address, the image, whether the stack answers and what is loaded on a branch. ## Verify the shared package reached the worker[​](#verify-the-shared-package-reached-the-worker "Direct link to Verify the shared package reached the worker") ``` uv run invoke check --name units_import ``` The check exists to prove the worker can import the shared package. It fails by name if the worker image loses it. An opaque `ModuleNotFoundError` in an unrelated generator is much harder to diagnose. ## Load the schema[​](#load-the-schema "Direct link to Load the schema") `invoke init` already did this. To load the schema on its own, onto a branch: ``` uv run invoke load-schema --branch main ``` Or check the schema files offline, with no running Infrahub: ``` uv run invoke schema-check ``` note `infrahubctl`, the tool these tasks call, reads the address from `.env` but **not** the API token, so an unauthenticated read succeeds while the first write fails with `Authentication failure`. Every invoke task passes both, through the subprocess environment rather than on the command line. That is the reason to run the task rather than the tool. ## Load the data[​](#load-the-data "Direct link to Load the data") The schema gives you 50 empty kinds. This step fills them with a European core network: 15 sites, of which 14 are PoPs, 21 ROADM-to-ROADM sections and 133 fiber spans. On that plant sit 445 devices, 1612 ports, 43 pre-provisioned wavelengths and one line container on each of them. Which branch to load onto depends on what is already there. * **A stack `invoke init` has only now created** holds nothing, so `main` is safe, and loading onto it makes the demo ready to browse the moment the load finishes. That is what `init` does. * **A stack that already holds objects** gets the load on a branch. A bad bulk load on the default branch means deleting objects one at a time; a bad load on a branch is one delete. For the second case: ``` uv run invoke branch-create --name geant uv run invoke load --branch geant ``` The object load takes a few minutes for 2490 objects. Files load in sorted order and that order matters, which is why they have numeric prefixes. The catalogs load first, then tags, sites, conduits, devices, ports, spans, sections, carriers and the line containers riding them. The O-E-O devices load last, because each names the wavelengths it terminates. A reference has to resolve against something already loaded. ### Check the load[​](#check-the-load "Direct link to Check the load") ``` uv run invoke inventory --branch geant ``` It counts 558 optical elements and 43 carriers, then ranks the sections by spectrum in use and lists every conduit two spans share. `oms-fra-mil` carries 40 of those carriers, holding 4,134,400 MHz of the 4,800,000 MHz the C-band gives that section, or 86.1 percent. Routers are absent from the element count on purpose. Light terminates at a router, so a router adds no insertion loss and does not inherit the optical element generic. The three ODU switches and the four attenuators are present, because they inherit the same generic. A pluggable transceiver is not an element either: it is a module fitted into a port, and the port is already on the path. The per-kind counts live in `scripts/geant_manifest.json`, written by the generator. The unit tests and the CI integration test read it directly. The figures on this page are copied by hand and then checked against it by `tests/unit/test_doc_claims.py`, which reads this page and fails when a number here stops matching the data. ### Regenerating the data[​](#regenerating-the-data "Direct link to Regenerating the data") The object files under `objects/1*.yml` are **output**. They come from a seed at the top of `scripts/generate_geant_dataset.py`: a site table, a section table, a conduit table and a carrier plan. Change a number there and re-run: ``` uv run invoke dataset-generate uv run invoke test-unit ``` To check the committed files still match the seed without writing anything: ``` uv run invoke dataset-check ``` Editing a file under `objects/` manually is not supported. Each generated file says so in its header and the next `--check` fails. ### What the data says, including the parts that are zero[​](#what-the-data-says-including-the-parts-that-are-zero "Direct link to What the data says, including the parts that are zero") Eight results in this dataset are negative, and the demo reports each one. | Finding | Why it matters | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **400ZR and 800ZR are unusable network-wide, and not because of power.** Both quote 120 km of reach against a shortest section of 220 km. Put 400ZR on that section anyway and it holds +7.890 dB of OSNR margin and fails on chromatic dispersion, 1340 ps/nm over its 2400 ps/nm tolerance. | The answer a planner wants before ordering pluggables. It is cFEC that does not reach here: the OpenZR+ parts are oFEC, and three of the loaded wavelengths run them router to router. See [the link budget](/demo-otn/link-budget.md). | | **Madrid has no diverse route.** It is single-homed on Paris. | MareNostrum 5 cannot survive a single fiber cut, and the diversity report says so rather than reporting nothing. | | **Thirteen of the twenty-one sections carry no wavelength.** | Deliberate. 40 of the 43 carriers are concentrated on one corridor, because one congested corridor is what the capacity-against-latency scenario needs. An evenly loaded network demonstrates nothing. | | **Twenty-five of the twenty-six free blocks on Frankfurt to Milan fit no mode at all.** The narrowest mode in the catalog, DP-QPSK 32GBd 100G, occupies 44,400 MHz. Twenty-one blocks are 20,400 MHz, two are 5,600, one is 38,000 and one is 35,200. | 665,600 MHz is free and almost none of it is usable. Fragmentation is a capacity finding in its own right and a free-megahertz total hides it. See [the spectral model](/demo-otn/spectral-model.md). | | **Channels 1 and 96 cannot anchor a 400G carrier.** A `64 GBd` carrier occupies 79,600 MHz and a `128 GBd` one 150,000 MHz, and centring either on the first or last grid position pushes it past the band edge. | The usable anchor range narrows as the mode widens: 96 anchors at `32 GBd`, 94 at every 400G mode. The grid is not the capacity. | | **The width model is optimistic at `64 GBd`.** It puts a `64 GBd` carrier at 79.6 GHz against the 87.5 GHz media channel a real deployment allocates, 7.9 GHz under. | The 9.2 GHz guard band is fitted to one published anchor and checked against a second it misses. Stated rather than hidden, because it is the demo's least defensible constant. | | **The chromatic dispersion gate does fire, once.** Madrid to Warsaw is 2970 km, which accumulates 50,490 ps/nm against a 50,000 ps/nm tolerance at 400G. | One site pair, at the highest modulation, one percent over. A second constraint that never fires proves only that the engine has two branches. | | **Paris to Madrid does not close at 400G on the most spectrally efficient mode.** It is short by 0.535 dB in both directions, so the OSNR check fails on the default branch. | Deliberate, and a true finding about a 1250 km route rather than a fault in the data. The [link budget](/demo-otn/link-budget.md) page has the answers an engineer would reach for, what each costs, and why regenerating this particular route is not among them. | ### The Frankfurt to Milan corridor[​](#the-frankfurt-to-milan-corridor "Direct link to The Frankfurt to Milan corridor") This is the pair the AI and HPC scenario runs on: JUPITER at Frankfurt to Leonardo at Milan. Every number below is reproducible from the branch you loaded above. | | Direct | Via Geneva | | ----------------------------------------- | -------------------------- | ------------------ | | Length | 780 km | 990 km | | One-way propagation | 3819 µs | 4848 µs | | One-way total, with node and FEC overhead | 3824.741 µs | 4853.605 µs | | Spectrum in use | 4,134,400 of 4,800,000 MHz | 0 of 4,800,000 MHz | | Anchors left for another 400G | 1 of 96 | 94 of 96 | The detour costs **1028 µs**, which is 27 percent of the direct route's own delay. The budget is checked against the total, not the propagation figure, so against a 4 ms one-way budget the direct route passes with 175.259 µs of margin and the detour misses by 853.605 µs. The finding is about the corridor's future, not about today. The service fits right now, but only just: 665,600 MHz is free and exactly one anchor, channel 95, can centre a 400G carrier inside a single free block. The corridor is 86.1 percent full and **has no latency-compatible alternative once it fills**, which is a capacity-exhaustion answer rather than a bandwidth one. Dividing free spectrum by a 79,600 MHz carrier width would have promised eight more services. Both candidate routes also leave Frankfurt through the same conduit, `cd-fra-south`, so a service split across the two is exposed to one backhoe outside the city. The shared-conduit table `invoke inventory` prints has the pair: `span-fra-gva-01` on `oms-fra-gva` and `span-fra-mil-01` on `oms-fra-mil`. ### Tear the branch down[​](#tear-the-branch-down "Direct link to Tear the branch down") ``` uv run invoke branch-delete --name geant ``` ## Run the unit tests[​](#run-the-unit-tests "Direct link to Run the unit tests") ``` uv run invoke test-unit ``` These need no running Infrahub. The link budget math is verified against hand-computed reference values, so it stays checkable even when the stack is down. ## Run the linters[​](#run-the-linters "Direct link to Run the linters") ``` uv run invoke lint ``` ## Tear down[​](#tear-down "Direct link to Tear down") ``` uv run invoke stop # keeps the data uv run invoke destroy # deletes the volumes and everything in them ``` --- # Link budget A wavelength either closes or it does not. The link budget is the arithmetic that decides, and in this demo it is a computation over loaded plant rather than a spreadsheet somebody maintains manually. Three independent gates, all evaluated, all reported: | Gate | Question | Passes when | | -------------------- | ------------------------------------------------------------------ | ------------------------------------------------- | | OSNR | Is there enough signal against the noise the amplifiers add? | `osnr_total - required_osnr - system_margin >= 0` | | Chromatic dispersion | Can the receiver still recover the pulse shape? | `accumulated <= mode.cd_tolerance` | | Gain | Can each amplifier restore the power the element ahead of it took? | `gain >= loss ahead, aging included` | The engine is `src/infrahub_demo_otn/budget.py`. It imports no Infrahub SDK, so its tests need no server, and every number crossing its boundary is a scaled integer. ## Two directions, and two chains of amplifiers[​](#two-directions-and-two-chains-of-amplifiers "Direct link to Two directions, and two chains of amplifiers") An optical multiplex section runs from one ROADM to the other, and light travels both ways over it. Amplification only works one way. An erbium-doped amplifier has an input and an output, and it restores the power of light entering its input. An amplifier hut therefore holds two amplifiers, one facing each way, sharing a building and nothing else. The section holds one relationship per chain, `amplifiers_a2b` and `amplifiers_b2a`, and which one holds an amplifier is which way it faces. Nothing on the amplifier repeats that. What the amplifier does have is an `oms_sequence` counting along the direction its own chain amplifies. A section with N spans has N+1 amplifiers each way, 2N+2 in total, and the two chains are budgeted separately. Here is `oms-ams-bru`, three spans between Amsterdam and Brussels: An amplifier name is a position and nothing else. Huts count from the A end and each holds two amplifiers. Hut 0 is the Amsterdam building with `01` and `02`, hut 1 is the first roadside hut with `03` and `04`, and so on. Read the two columns above side by side and `03` and `04` are the two amplifiers in one building, facing opposite ways. Their sequence numbers differ because the two chains reach that building from opposite ends. Sort either chain on its sequence and it is already in the order the light meets it. Nothing has to parse a name to get there. The chain comes from the relationship and the order comes from the sequence. Whether an amplifier is a booster, an inline or a pre-amplifier is the `role` on its own input and output ports. **Walking a section the other way swaps the chains. It does not reverse one.** Reversing a chain models the same amplifiers running backwards, which is not what is installed. The distinction is invisible on this dataset, where all 306 amplifiers have the same noise figure and the same gain. It stops being invisible the first time a section is built from mixed equipment. Two chains produce two margins for one section. Where a report shows a single margin, it shows the worse of the two, because that is the direction a service over the section is limited by. Both figures are listed underneath, so the choice is visible rather than assumed. ## A worked example over the shortest section[​](#a-worked-example-over-the-shortest-section "Direct link to A worked example over the shortest section") The shortest section in the network, 220 km over three spans, carrying a 400G wavelength on `DP-16QAM 64GBd 400G` from Amsterdam towards Brussels. Every number below comes out of `objects/`. **The inputs.** Three G.652.D spans of 73,334, 73,333 and 73,333 metres. Each has 18 fusion splices at 0.05 dB, two mated connector pairs at 0.3 dB, and a 1.5 dB aging allowance. A ROADM at each end at 7.0 dB. Four amplifiers in the Amsterdam-to-Brussels chain, each 4.0 dB noise figure and 22.0 dB gain, and four more facing the other way. **Per-span loss.** The four-term span formula, on the first span: ``` (73334 / 1000) * 200 = 14667 mdB attenuation 18 * 50 = 900 mdB splices 2 * 300 = 600 mdB connectors 1500 mdB aging allowance ------------ 17667 mdB = 17.667 dB ``` **Per-stage OSNR.** Launch power is +3.0 dBm per channel. The loss ahead of an amplifier is the loss of whatever sits immediately before it. The aging allowance is not charged here. It is a reserve against loss that has not happened yet, so it belongs to the power budget and to the gain requirement, not to the noise. That leaves 16.167 dB of real fiber loss on each span. ``` booster, behind the 7.0 dB ROADM: 3000 - 7000 - 4000 + 58000 = 50000 mdB = 50.000 dB each line amplifier, behind 16.167 dB of fiber: 3000 - 16167 - 4000 + 58000 = 40833 mdB = 40.833 dB ``` The 58.0 dB is the reference-bandwidth constant, `-10*log10(h * nu * delta_nu)` for 12.5 GHz, which is 0.1 nm at 1550 nm. It is a named constant in the engine, never a literal in the formula. **The hop table.** This is what the budget report transform renders. | # | Hop | Kind | Loss | Stage OSNR | Cumulative loss | Cumulative OSNR | Cumulative delay | | - | ----------------- | --------- | --------- | ---------- | --------------- | --------------- | ---------------- | | 1 | `roadm-ams-01` | node | 7.000 dB | | 7.000 dB | | 0.1 us | | 2 | `amp-ams-bru-01` | amplifier | | 50.000 dB | 7.000 dB | 49.500 dB | 0.2 us | | 3 | `span-ams-bru-01` | span | 17.667 dB | | 24.667 dB | 49.500 dB | 359.3 us | | 4 | `amp-ams-bru-03` | amplifier | | 40.833 dB | 24.667 dB | 39.836 dB | 359.4 us | | 5 | `span-ams-bru-02` | span | 17.667 dB | | 42.334 dB | 39.836 dB | 718.5 us | | 6 | `amp-ams-bru-05` | amplifier | | 40.833 dB | 42.334 dB | 37.067 dB | 718.6 us | | 7 | `span-ams-bru-03` | span | 17.667 dB | | 60.001 dB | 37.067 dB | 1077.7 us | | 8 | `amp-ams-bru-07` | amplifier | | 40.833 dB | 60.001 dB | 35.390 dB | 1077.8 us | | 9 | `roadm-bru-01` | node | 7.000 dB | | 67.001 dB | 34.890 dB | 1078.0 us | Two numbers in that table need explanation. The cumulative OSNR after the booster is 49.500 dB, not the 50.000 dB the stage delivers. Each ROADM traversed costs a 0.5 dB filtering penalty on top of its insertion loss, and the head ROADM has already been crossed. The cumulative OSNR drops 9.664 dB at the first line amplifier and only 2.769 dB at the second. Stages cascade in the linear domain, `1 / OSNR_total = sum(1 / OSNR_stage)`, so the first bad stage does most of the damage and each identical one after it does less. Two equal stages land exactly 3.010 dB below either of them. **The verdict.** | Quantity | Value | | --------------------------------- | ------------------ | | Total length | 220.000 km | | Total loss | 67.001 dB | | OSNR delivered | 34.890 dB | | Required by `DP-16QAM 64GBd 400G` | 24.500 dB | | System margin | 1.000 dB | | **OSNR margin** | **+9.390 dB** | | Accumulated dispersion | 3740.0 ps/nm | | Tolerated by the mode | 50000.0 ps/nm | | **Dispersion margin** | **+46260.0 ps/nm** | | Latency in this direction | 1082.0 us | Both gates pass, with room. On the shortest section in the network that is the expected answer. The interesting answers are on the long routes. Brussels to Amsterdam produces the same numbers. The two ROADMs have the same insertion loss, the two chains hold the same equipment, and no span here has a Raman pump, so nothing distinguishes the directions. That is the ordinary case. It is what makes the few sections where the two differ worth looking at. ## The model, and where it is tuned[​](#the-model-and-where-it-is-tuned "Direct link to The model, and where it is tuned") Five constants set the whole model. Two of them are tuned towards the favourable end of realistic, and the table marks which two. | Constant | Value | Note | | ------------------------ | --------------- | ---------------------------------------------------------------------------------------------------------- | | OSNR reference bandwidth | 58.0 dB | Physics. 12.5 GHz, which is 0.1 nm at 1550 nm. | | Per-channel launch power | +3.0 dBm | **Tuned.** Across 96 channels that is +22.8 dBm composite, a high-power booster rather than a typical one. | | Amplifier noise figure | 4.0 dB | **Tuned.** A good low-noise erbium-doped amplifier, not an average one. | | ROADM filtering penalty | 0.5 dB per node | Representative. | | System margin | 1.0 dB | Implementation penalty and end-of-life degradation. | The span lengths, the amplifier spacing, and the reach and OSNR figures in the optical mode catalog are illustrative in exactly the same sense. None of them is sourced from a vendor data sheet or from published plant records. The node and amplifier latency terms, 150 and 100 nanoseconds, are illustrative too. They sit three orders of magnitude below propagation and change no verdict; they are modelled because zero is a stronger claim than small. ## Beginning of life, and why the aging allowance is not in the OSNR[​](#beginning-of-life-and-why-the-aging-allowance-is-not-in-the-osnr "Direct link to Beginning of life, and why the aging allowance is not in the OSNR") Every span has an `aging_margin_mdb`, 1.5 dB by default. It is a reserve against repairs and degradation that have not happened yet. The engine keeps two loss numbers for every span: * `span_loss_mdb` includes the allowance. It determines the power budget and the amplifier gain requirement, because an amplifier has to still restore the launch power at end of life. * `span_fiber_loss_mdb` excludes it. It determines the OSNR, because that is the loss the light meets today. Charging the allowance to the OSNR as well would compute an end-of-life figure and then subtract a system margin on top of it, which is the same pessimism counted twice. Over the 17 spans of the longest route the allowance alone is 25.5 dB. So the OSNR this demo reports is a **beginning-of-life** figure, and the 1.0 dB system margin is what covers implementation penalty and aging on top. ## An attenuator costs two numbers, and the model keeps them apart[​](#an-attenuator-costs-two-numbers-and-the-model-keeps-them-apart "Direct link to An attenuator costs two numbers, and the model keeps them apart") Every other element on a path costs one figure, its `insertion_loss_mdb`. An attenuator costs two: ``` hop loss = insertion_loss_mdb + attenuation_mdb ``` `insertion_loss_mdb` is what the device costs by being in the path at all, and `attenuation_mdb` is what it is deliberately set to. A fixed pad is `0 + 5.0 dB`, because a pad is nothing but its value. A variable attenuator is `1.0 + 3.0 dB`, because the optics and the switching cost a decibel before anything is dialled in. Every other element leaves `attenuation_mdb` at zero, so the same expression covers a ROADM, a multiplexer and a patch panel unchanged. **Nothing stores the sum**, and that is the point of keeping the two apart. A stored total cannot answer "how much pad range is left", and that is the question an operator actually asks. On a variable attenuator the answer is `max_attenuation_mdb - attenuation_mdb`, which on the two shipped VOAs is 17.0 dB of range still available out of 20.0. `attenuator_range` is the check that holds that difference at zero or above, because Infrahub's `max_value` takes a constant rather than another attribute of the same object. **Negative result: no optical path in this dataset runs through an attenuator yet.** The four racked units, two pads and two VOAs, sit in the inventory and on no path, so no budget on this page moves because of them. The arithmetic is in the engine and the objects are loaded; the demo does not yet place one in a route. ## Raman, and what it does to a link budget[​](#raman-and-what-it-does-to-a-link-budget "Direct link to Raman, and what it does to a link budget") Raman amplification puts the gain inside the fiber. A pump laser injects light about a hundred nanometres below the signal wavelength. Stimulated Raman scattering transfers power from the pump to the signal while the two travel through the same glass. No gain block appears on the path. The signal reaches the far end of the span with more power in it than the loss of the fiber predicts. The quantity that describes this is **on-off gain**: the received signal power with the pump running, less the received signal power with the pump switched off. It is stored on `OtnRamanPump.on_off_gain_mdb`, and the engine credits it as reduced effective span loss rather than as another stage in the cascade: ``` effective fiber loss = fiber loss - on-off gain + combiner insertion loss ``` **The gain is credited one way and the combiner is charged both ways.** A pump amplifies one direction of travel and does nothing at all for light going the other way. The wavelength combiner that couples the pump onto the fiber sits in line on the glass, and light going either way passes through it and pays its insertion loss. A pumped span is therefore asymmetric: it costs less one way, by the gain minus the combiner, and more the other way, by the combiner alone. ### Which direction a pump amplifies[​](#which-direction-a-pump-amplifies "Direct link to Which direction a pump amplifies") The pump does not store the answer. It stores two facts about the hardware, and the direction follows from them: * **`injection_end`**, which end of the span the pump is spliced in at, named against the span's own `site_a` and `site_b`. * **`propagation`**, whether the laser fires with the signal or against it. A counter-propagating pump fires back up the fiber from the far end, so one at the B end amplifies the A to B signal. A co-propagating pump fires along with the signal from the near end, so one at the A end amplifies A to B as well. Those are two ways of reaching the same result, and all four cases are: | Injected at | Laser fires | Amplifies | | ----------- | ------------------ | --------- | | B end | Against the signal | A to B | | A end | With the signal | A to B | | A end | Against the signal | B to A | | B end | With the signal | B to A | Every pump in the loaded network is **counter-propagating**, so only the first and third rows occur here. The table shows all four because a reader who sees only the shipped cases will assume counter-propagating is the only kind there is. Firing against the signal means gain builds where the signal is weakest, which is the useful place to put it. Storing the conclusion alongside the two facts that fix it would be a third value that can contradict them. A counter-propagating pump recorded at the A end and marked as amplifying A to B is a self-contradictory object, and a model that accepts one will eventually hold one. ### What that is worth on this network[​](#what-that-is-worth-on-this-network "Direct link to What that is worth on this network") Nine spans have a pump on the default branch, all of them on Vienna to Milan, all injected at the Milan end and firing against the signal, which amplifies Vienna towards Milan. Each pump quotes 10.0 dB of on-off gain, and each combiner costs 0.5 dB. | Direction | Effective span loss against the bare fiber | | -------------------- | ------------------------------------------ | | Vienna towards Milan | 9.5 dB lower | | Milan towards Vienna | 0.5 dB higher | Nine spans are pumped, so the section total is nine times each figure. This is the one place in the network where the two directions of a section produce different answers, and it is why the reports show both. The budget report shows the arithmetic on the rows it applies to and nowhere else. A pumped span row shows its on-off gain, its combiner loss, and what the span would have cost unpumped, so the improvement is traceable to its source. The other 124 spans have no Raman columns at all, because a column of zeros across the rest of the network reads as a fault rather than as a correct answer. ### Where the treatment stops being predictive[​](#where-the-treatment-stops-being-predictive "Direct link to Where the treatment stops being predictive") Crediting the on-off gain as reduced loss is a first-order engineering approximation, and it is worth being exact about what it leaves out. A real distributed Raman amplifier contributes noise of its own: amplified spontaneous emission from the distributed gain, double Rayleigh backscattering, and pump relative intensity noise transferred onto the signal. **This model charges none of it.** It credits the gain, takes the combiner loss, and stops. So every OSNR figure on this page that involves a pumped span is **slightly optimistic** against a full distributed model. The direction of the error is known and its size is not modelled here. A decision that depends on a few tenths of a decibel over a pumped span needs a Raman simulation, not this arithmetic. The same statement sits in the engine module beside the code that produces the number, because someone reading the code will not have this page open. ## What the model does not include[​](#what-the-model-does-not-include "Direct link to What the model does not include") | Absent | Why | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Nonlinear penalties | Credible modelling needs a fiber effective-area and channel-count model this demo does not have. | | Polarisation-mode dispersion | Same reason. | | Regeneration and 3R placement | The network is transparent end to end by design. | | Flexgrid media channels | Anchors are quantised onto the 50 GHz grid. A carrier's occupied **width** is modelled and checked against the band. A `128 GBd` wavelength is 150,000 MHz wide rather than one channel, but its centre may only sit on a grid position. See [the spectral model](/demo-otn/spectral-model.md). | | Raman pump noise | The on-off gain is credited and the pump's own noise is not charged, so a pumped span reads slightly better than it would in a full distributed model. | | Any CWDM link | `OtnFiberType.attenuation_coefficient_mdb_per_km` is a 1550 nm figure, and applying it to a 1471 nm wavelength understates the loss. This repository holds no per-band attenuation figure, so it cannot say by how much. The coarse tail therefore belongs to no optical multiplex section, which is what keeps it out of the engine. See [optical plant concepts](/demo-otn/concepts.md). | ## Three findings the budget produced[​](#three-findings-the-budget-produced "Direct link to Three findings the budget produced") The demo reports negative results. These are the three the engine found, and each one is asserted by `tests/unit/test_budget_claims.py`, recomputed from `objects/` on every test run. ### Route length does not order signal quality[​](#route-length-does-not-order-signal-quality "Direct link to Route length does not order signal quality") Berlin to Amsterdam has four routes. Each is budgeted at `DP-16QAM 64GBd 400G` from Berlin, so the amplifier count is the chain that carries the light that way. | Route | km | Amplifiers | ROADMs | OSNR | Margin | | ----------------------------------------- | ---- | ---------- | ------ | --------- | ------------- | | Berlin - Hamburg - Amsterdam | 800 | 12 | 3 | 27.784 dB | **+2.284 dB** | | Berlin - Frankfurt - Amsterdam | 1010 | 14 | 3 | 26.007 dB | **+0.507 dB** | | Berlin - Prague - Frankfurt - Amsterdam | 1220 | 18 | 4 | 25.241 dB | **-0.259 dB** | | Berlin - Copenhagen - Hamburg - Amsterdam | 1330 | 20 | 4 | 25.378 dB | **-0.122 dB** | The last two rows are the finding. The Copenhagen route is 110 km longer than the Prague route and has 0.137 dB more margin, because its sections are built from shorter spans. Span loss enters the cascade exponentially; route length enters it linearly. A planner ranking routes by distance would pick the Prague route as the better fallback. The budget says otherwise, and no map shows it. All four routes carry 400G at `DP-QPSK 128GBd`, so the split above is a modulation choice rather than a corridor nobody can use. ### The dispersion gate fires once, on one site pair, at 400G only[​](#the-dispersion-gate-fires-once-on-one-site-pair-at-400g-only "Direct link to The dispersion gate fires once, on one site pair, at 400G only") Madrid to Warsaw, routed through Paris, Frankfurt and Prague, is 2970 km. At 17 ps/nm/km that accumulates **50,490 ps/nm** against the 50,000 ps/nm both 400G modes tolerate. It fails by one percent. At `DP-QPSK 128GBd 400G` that pair fails the OSNR gate as well, by 0.021 dB. Two independent constraints landing within a quarter of a decibel of each other on the same site pair is a coincidence, and it is worth naming as one. At 100G the pair passes both gates comfortably. A second constraint that never fires proves only that the engine has two branches. One that fires once, marginally, at the highest modulation and nowhere else gives a planner a real answer. Madrid to Warsaw at 400G needs dispersion compensation or a lower modulation, and no other pair does. ### The spectrally efficient 400G mode misses exactly one section[​](#the-spectrally-efficient-400g-mode-misses-exactly-one-section "Direct link to The spectrally efficient 400G mode misses exactly one section") `DP-16QAM 64GBd 400G` closes on 40 of the 42 section evaluations, which is 21 sections taken singly in both directions. The two it misses are the two directions of Paris to Madrid, 1250 km over 14 spans, each short by 0.535 dB. That is the longest single section in the network, and it is the only place the higher-order modulation does not fit. 16QAM 400G covers every section except the longest one, and QPSK 400G covers every route except the longest one. The coherent pluggables complete the picture, and the section below is about them. ## A DCI pluggable runs out of dispersion, not power[​](#a-dci-pluggable-runs-out-of-dispersion-not-power "Direct link to A DCI pluggable runs out of dispersion, not power") `400ZR` fails on the three shortest sections in the network, and it does not fail on OSNR. Amsterdam to Brussels at 220 km hands it **+7.890 dB** of margin against its 26.0 dB requirement, and the same section refuses it on chromatic dispersion: | Section | km | Accumulated CD | 400ZR tolerance | Over by | 400ZR OSNR margin | | ------------- | --- | -------------- | --------------- | ------- | ----------------- | | `oms-ams-bru` | 220 | 3740 ps/nm | 2400 ps/nm | 1340 | +7.890 dB | | `oms-ber-prg` | 330 | 5610 ps/nm | 2400 ps/nm | 3210 | +4.747 dB | | `oms-ham-ber` | 330 | 5610 ps/nm | 2400 ps/nm | 3210 | +4.747 dB | That is 2400 ps/nm, the lowest dispersion tolerance in the catalog, against a plant with 17 ps/nm/km of uncompensated G.652 fibre. 400ZR was specified for a data-centre interconnect: a short span with almost no accumulated dispersion and nothing in front of it to compensate. Put it on a regional section and it has power to spare and no dispersion budget left. **`OpenZR+ 400G` runs the same constellation, the same baud rate and the same cage, and closes on all three.** It tolerates 50,000 ps/nm rather than 2400, which is the oFEC doing the work, and the dataset lights three wavelengths on it router to router: `oc-ch003-ams-bru` at **+10.890 dB**, `oc-ch004-ber-prg` at **+7.747 dB** and `oc-ch006-ham-ber` at **+7.747 dB**. All three close at the top rung and none had to drop to 300G or 200G. So "the 120 km pluggables reach no section" is true and it is a statement about cFEC. Pluggable does not mean short reach. `OpenZR+ 200G` reaches 3000 km, the same as the long-haul 100G transponder. The distinction that matters when ordering optics is the FEC on the part, not the form factor it comes in. Every figure in this section is recomputed from `objects/` by `tests/unit/test_budget_claims.py` on each test run, the same way the three findings above are. ## Paris to Madrid does not close, and the check says so[​](#paris-to-madrid-does-not-close-and-the-check-says-so "Direct link to Paris to Madrid does not close, and the check says so") Run the OSNR check against the default branch and it fails. That is the state the network is in, not a fault in the demo, and the route it names is Paris to Madrid in both directions. 1250 km of fiber over 14 spans, with 15 amplifiers in each direction, is a demanding section, and `DP-16QAM 64GBd 400G` misses it by 0.535 dB. Madrid sits at the end of the longest single section here, and half a decibel is what a section that long costs at that modulation. The number is a property of the route, not of the dataset. An engineer looking at half a decibel reaches for three answers. Only two of them are available on this route. | Answer | What it costs | | ----------------------------- | ---------------------------------------------------------------------------------- | | Drop to `DP-QPSK 128GBd 400G` | The section closes at +4.965 dB, at twice the spectrum for every wavelength on it. | | Regenerate part way along | Not available here. See below. | | Put Raman on the section | The loss comes back without spending spectrum or a building. | **The middle answer is not available on this route, and that was measured rather than assumed.** Cutting a route in half at a regenerator needs a site in the middle to put one at. `oms-par-mad` is a **single** multiplex section carrying all 14 spans, and Madrid appears in no other section, so it hangs off Paris as a leaf. The boundaries between those 14 spans are amplifier huts, and this model gives every span its section's own endpoints as `site_a` and `site_b` rather than the huts it runs between. There is no object between Paris and Madrid to attach a device to. That sharpens the negative result instead of softening it. Regeneration is not the fix for Paris to Madrid, so the two answers that remain are the modulation and the Raman pumps, and the pumps are the cheaper one. Raman is the standard answer to a section that is a fraction of a decibel short, and this network already models the equipment for it. The fix is a branch: ``` uv run invoke demo-raman ``` That puts two pumps on each of the first three spans out of Paris, one at each end, and runs the same check. It passes, in both directions: `oms-par-mad` moves from -0.535 dB to +0.361 dB each way. Two pumped spans would also pass, at +0.041 dB, which is a pass on paper and nothing in the field. The pumps go in at both ends because a pump credits its on-off gain to the one direction it amplifies and its combiner charges both. Pumping one end only would fix one direction and make the other worse. The proposed change then holds six pump objects, three fiber spans whose only change is the pump list now pointing back at them, and no edited margin figure anywhere. That is the point. The margin moved because the plant changed. The default branch keeps the red check on purpose. A network whose every gate is green teaches nothing about what a gate is for. A shortfall hidden until someone tries to provision over it is a worse demonstration than one the pipeline reports on every run. ## A regenerated circuit, and the two things it costs[​](#a-regenerated-circuit-and-the-two-things-it-costs "Direct link to A regenerated circuit, and the two things it costs") A route with intermediate sites has an answer Paris to Madrid does not. Put an O-E-O regenerator at one of them, and the circuit becomes two wavelengths joined electrically: the light is received, the payload is reframed, and a fresh transmitter starts the second half. Each half carries its own OSNR budget, so a route too long for any single wavelength can still be built. `OtnOduSwitch` is the kind. Its `carriers` relationship names the wavelengths patched to the shelf, and `src/infrahub_demo_otn/chains.py` reads that to decide whether a junction is real. The two wavelengths must meet at a site, that site must host the device, and the device must be patched to both. **What terminates each half is a line port, on the regenerator itself.** The shelf carries two, one facing each segment, and each names its wavelength. So the Madrid to Paris half ends on a Madrid transponder at one end and on the regenerator at the other, and the Paris to Warsaw half starts on the same shelf and ends on a Warsaw transponder. Ask either wavelength what terminates it and it answers with hardware at both ends, the same way a direct circuit does. That is also what makes the split affordable. The inner ends need no transponder at the junction site, which matters because Paris and Frankfurt each hold one spare line port and a split needs two. ### Madrid to Warsaw, where one regenerator is not enough[​](#madrid-to-warsaw-where-one-regenerator-is-not-enough "Direct link to Madrid to Warsaw, where one regenerator is not enough") 2970 km over four sections, `oms-par-mad`, `oms-par-fra`, `oms-prg-fra` and `oms-prg-waw`. No single wavelength closes it on any of the ten modes in the catalog; the best is -0.021 dB at DP-QPSK 128GBd 400G. Three sites on that route can hold a regenerator, and `demo/06_mad_waw_16qam.yml` places one at each with the wavelength pair its split needs. Every figure below came back from a provisioning run. | Attempt | Segment 1 | Segment 2 | Verdict | | --------------------------------------------- | --------- | --------- | ---------- | | 400G DP-16QAM, regenerated at Paris | -0.535 dB | -2.439 dB | Refused | | 400G DP-16QAM, regenerated at Frankfurt | -2.755 dB | +0.240 dB | Refused | | 400G DP-16QAM, regenerated at Prague | -4.004 dB | +2.782 dB | Refused | | 400G DP-QPSK 128GBd, regenerated at Frankfurt | +2.745 dB | +5.740 dB | **Closes** | **Three of the four attempts fail, and that is the finding.** Cutting a route in half does not on its own make it close. The fix is a regenerator **and** a different modulation, and `demo/07_mad_waw_qpsk.yml` is the second half of that pair. Prague is worth reading as the runner-up. It closes its second half at 16QAM with 2.782 dB to spare and fails its first by 4 dB, which shows the split point matters independently of the mode. **A route's verdict is a conjunction over its segments, not an average.** One half short of OSNR refuses the whole circuit however much margin the other half has. The Paris split is refused on -0.535 dB while its second segment is worse still. `RouteBudget` offers no single margin figure for a regenerated route, deliberately. A scalar would have to be a minimum or a mean, and both read as a statement about a circuit that has no such property. ### Why a direct wavelength is always preferred[​](#why-a-direct-wavelength-is-always-preferred "Direct link to Why a direct wavelength is always preferred") Two costs, and the model states both rather than leaving them implied. **A regeneration.** The circuit needs a device at the junction, and that device is a transponder pair plus the framing hardware between them. It is a thing to buy, rack, power and maintain, at a site that has to be there already. **Latency.** The delay is the sum of both segments plus what the junction charges for reframing, which `framing_latency_ns` holds on the device. The Madrid to Warsaw circuit takes **14558.963 us** end to end at 3000 ns of framing delay, against 9163.620 us for the Madrid to Frankfurt half on its own. For the [AI and HPC payloads](/demo-otn/ai-payloads.md) that treat round-trip time as the budget, that is the term that matters. So the routing engine ranks a direct wavelength above a chain wherever both serve the route. Madrid to Warsaw takes the chain because it has no direct wavelength that closes on any mode, which is the only condition under which a chain wins. ## Re-validation on a proposed change[​](#re-validation-on-a-proposed-change "Direct link to Re-validation on a proposed change") The arithmetic above runs once. The check runs it again on every change. `checks/osnr_margin.py` runs on every proposed change. It reads the whole plant once and then runs two loops over it. **The carrier loop** budgets all 40 pre-provisioned wavelengths against their own modes and logs an error for each one that fails any of the three gates. Lengthen a span on a branch and every wavelength crossing it is re-evaluated before the change can merge. **The section sweep** budgets every section on its own, in both directions, against one reference mode, `DP-16QAM 64GBd 400G`. That is 42 evaluations over 21 sections, and it exists because the carrier loop cannot reach most of the network. Only eight sections carry a wavelength, so thirteen of the twenty-one were never evaluated by anything. Paris to Madrid was one of the thirteen. The sweep is also the only place the N+1-amplifiers-per-direction rule becomes a gate rather than a unit test, because nothing in the graph enforces how many amplifiers a section holds. A failure from either loop names the direction, not only the section. ``` uv run invoke check --name osnr_margin ``` On the default branch the sweep fails and the carrier loop passes: ``` ERROR osnr_margin::OsnrMarginCheck: FAILED ERROR Re-validated 40 carriers over 21 sections. Worst OSNR margin is +1.894 dB, on oc-ch047-fra-mil ERROR oms-par-mad a_to_b is short of OSNR by 0.535 dB on DP-16QAM 64GBd 400G: the section delivers 24.965 dB over 1250 km and 15 amplifiers, and the mode needs 24.500 dB plus 1.000 dB of system margin ERROR oms-par-mad b_to_a is short of OSNR by 0.535 dB on DP-16QAM 64GBd 400G: the section delivers 24.965 dB over 1250 km and 15 amplifiers, and the mode needs 24.500 dB plus 1.000 dB of system margin ERROR Swept 42 section evaluations over 21 sections in two directions against DP-16QAM 64GBd 400G. Worst standalone OSNR margin is -0.535 dB, on oms-par-mad a_to_b. Short of OSNR: oms-par-mad ``` That worst carrier is one of the 40 wavelengths on the congested Frankfurt to Milan corridor. It runs 780 km at 16QAM. It is the tightest margin of any wavelength on the network, and it is unaffected by Raman: the pumps are on Vienna to Milan and this carrier does not cross that section. Stretch one span of the Frankfurt to Milan corridor from 87 km to 200 km on a branch. The carrier loop says so, per wavelength and per gate: ``` oc-ch047-fra-mil on DP-16QAM 64GBd 400G is short of OSNR by 11.385 dB: the path delivers 14.115 dB over 893 km and 10 amplifiers, and the mode needs 24.500 dB plus 1.000 dB of system margin oc-ch047-fra-mil crosses amp-fra-mil-03, whose gain cannot recover the loss ahead of its input at end of life ``` A wavelength the check cannot evaluate is an error, never a skip. A check that passes silently over an object it could not read reports success for a network it never read. The sweep is the same principle applied to the plant: a section nobody has provisioned over is still a section that has to close. ## Reading the budget as a table[​](#reading-the-budget-as-a-table "Direct link to Reading the budget as a table") `transforms/budget_report.py` renders the same computation as structured output: one row per hop with running totals, plus a verdict block, for every wavelength on the branch. Every scaled integer is accompanied by its rendered unit, so no row asks the reader to divide by a thousand. ``` uv run invoke demo-budget ``` Each wavelength is budgeted in both directions. The headline margin is the worse of the two, and both are listed with the ROADM each walk started from, so a reader can tell an asymmetric section from a symmetric one. Any pumped span on the path is listed separately with what it costs each way. For a span pumped one way only, the worse direction is the unpumped one. A report that showed the reported walk alone would show a Raman span with no Raman on it. The wavelengths come back sorted by OSNR margin, worst first, because the worst one is the answer to "how much room is left". --- # Loadable scenarios and the runbook Three pairs of scenarios and six singles, one task each, then the runbook that walks the whole demo in narrative order, then how to put the default branch back. Each task makes its own branch, loads its own scenario input and runs its own check, so nothing here asks you for a branch name. Every number below is produced by the engine and asserted by a unit test. The [demo guide](/demo-otn/demo-guide.md) carries the run order and links the other scenario pages. ## Two loadable scenarios for the ODU layer[​](#two-loadable-scenarios-for-the-odu-layer "Direct link to Two loadable scenarios for the ODU layer") Both are scenario input under `demo/`. They share one branch because they touch no common wavelength, and one task owns it: ``` uv run invoke demo-odu ``` It makes `odu-demo` if it is not there, loads both files, provisions the eleven London to Milan circuits the first one asks for, and runs the capacity check. Run it again and it continues onto the same branch. **`demo/04_odu_ten_in_one.yml`, ten circuits in one wavelength.** Provision `svc-lon-mil-sdh-01` through `-10` and each maps into an `ODU2` of 8 slots that grooms into the same `ODU4` line container. Ten eights are eighty, so the tenth takes it to 80 of 80 and none of the ten lights a wavelength of its own. `svc-lon-mil-sdh-11` is then **refused**, reason `no-slots`, naming the container it did not fit with both slot figures and then saying that no wavelength could be lit either. Grooming is tried first and lighting second, so the file has to close both escapes. It spends the last usable block on `oms-fra-mil`, leaving 532,400 MHz in 29 blocks. The widest of those, 38,000 MHz, is narrower than the 44,400 MHz the narrowest mode in the catalog occupies. **This one ends in an accepted refusal too.** The file carries `refusal_accepted: true` on `svc-lon-mil-sdh-11` and on none of the ten that provisioned. `provisionable` reads the refusal, sees it signed for and passes. The scenario's point is that grooming is tried before lighting and a refusal has to close both escapes, and none of that needs a red pipeline. The flag also survives the generator: a rerun that refuses the service again leaves the signature alone, and only the path where the service actually provisions clears it. The band split on that branch is `odu4` 5, `full` 1, `no-odu` 15. Only `oms-ams-lon` turns red, and the two other sections the corridor crosses stay green because they still carry empty `ODUC4` wavelengths at 320 free. **`demo/05_odu_mixed_fill.yml`, every band at once.** No service and no generator run. It writes client containers directly, spread so that at least one section lands in each of the five bands. The measured split is `odu4` 5, `odu2` 1, `odu0` 1, `full` 2, `no-odu` 12, over 21 sections. Load it, fetch `odu-map` from any PoP, and the legend can be read against the picture. **Read the map, then check it.** Two minutes, and it is the step that says whether the drawing works without its author standing next to it. Give someone the rendered `odu-map` and nothing else, and ask for two lists: the sections that can still take a 100G circuit, and the sections where nothing fits. Both come off the colours, green for the first and red for the second. Then check the answer against the containers on the branch rather than against the map, which is the last thing `demo-odu` does. The check passes, and its summary line counts parents that fit, parents with no known figure and overfilled ones. Those counts come from the same slot table the map coloured itself with. A reader who names a green section the containers say is full has found a real disagreement. There is nowhere for one to hide: the map, the check and the generator all call one implementation of the capacity rule. ## Two loadable scenarios for the regenerator[​](#two-loadable-scenarios-for-the-regenerator "Direct link to Two loadable scenarios for the regenerator") Both are scenario input under `demo/`, and they are a before and after. They need **two branches**, because the point is the comparison: one branch keeps the refusal, and the other is the proposal a reviewer holds against it. ``` uv run invoke demo-regenerator ``` One command tells both halves. It makes `oeo-refused`, loads `06`, asks for `svc-mad-waw-400g` and runs `provisionable`, which fails. Then it makes `oeo-closed`, loads `06` and `07`, asks again, and runs the same check, which passes. **`demo/06_mad_waw_16qam.yml`, three regenerators and three refusals.** Madrid to Warsaw is 2970 km over four sections, and no single wavelength closes it on any of the ten modes. The best is -0.021 dB at DP-QPSK 128GBd 400G. The file puts an `OtnOduSwitch` at each of the three sites on that route where one could go, `oeo-par-01`, `oeo-fra-02` and `oeo-prg-01`, each holding the wavelength pair its split needs. All three splits are **refused**, reason `budget`: Paris returns -0.535 and -2.439 dB, Frankfurt -2.755, Prague -4.004. One run reports all three. The two later splits show one figure each because the generator stops budgeting a chain at its first failing segment. A verdict is a conjunction, so the second half cannot rescue the first and computing it would only produce a number nobody may act on. Both halves of all four attempts are in the [link budget](/demo-otn/link-budget.md#madrid-to-warsaw-where-one-regenerator-is-not-enough) table, measured segment by segment. That refusal is the scenario and not a fault in the file. A route's verdict is a conjunction over its segments, so one half short of OSNR refuses the whole circuit however much margin the other half has. **This scenario ends in a blocked merge, and it is the only one that does.** Nothing in `demo/06_mad_waw_16qam.yml` sets `refusal_accepted`, so a proposed change from `oeo-refused` goes red. The check the task runs on that branch says so: ``` svc-mad-waw-400g cannot be provisioned and was refused for budget: neither a direct wavelength nor a chain serves this route. Direct: oms-par-mad|oms-par-fra|oms-prg-fra|oms-prg-waw closes on no mode: best margin -0.021 dB on DP-QPSK 128GBd 400G. Chain: oc-ch070-mad-par|oeo-par-01|oc-ch070-par-waw does not close: segment 1 -0.535 dB, segment 2 -2.439 dB. It has not been accepted, so this branch does not merge. Fix the network or the request, or set refusal_accepted on the service to keep the refusal on the record ``` The service, the code and the detail are all in the one message, because a proposed change shows a reader this line and nothing else. The same run adds `Judged 1 service(s). 1 refused and unaccepted, 0 refused with no readable code, 0 accepting a refusal that does not exist, 0 refused and signed for, 0 carrying an optical path.` Every other refusal in this guide is signed for. This one is deliberately not, so the gate is seen firing at least once. `tests/unit/test_demo_scenarios.py::test_madrid_to_warsaw_signs_for_nothing_so_one_scenario_still_blocks` fails if somebody adds the flag here, because a feature every scenario accepts its way past is a feature nobody watching the demo ever sees. Two other checks are red on this branch for reasons that predate the gate. `osnr_margin` fails on a Paris to Madrid deficit that is on every branch including the default one. `channel_collision` fails because the three regenerator wavelength pairs this file loads overlap in spectrum on purpose. Neither is `provisionable`, and neither is this feature's doing. **`demo/07_mad_waw_qpsk.yml`, the mode that finishes the job.** One more wavelength pair on the same Frankfurt split, at DP-QPSK 128GBd 400G, on `oeo-fra-03`. No new site, no new section, no edited margin. The service then provisions as two segments: ``` no direct wavelength serves oms-par-mad|oms-par-fra|oms-prg-fra|oms-prg-waw, so it takes the chain oc-ch073-mad-fra|oeo-fra-03|oc-ch073-fra-waw in 2 segments regenerated at oeo-fra-03 at Frankfurt, 2970 km, segment 1 +2.745 dB, segment 2 +5.740 dB, 14558.963 us including the framing delay ``` Because `06` is loaded underneath, the same run still refuses all three 16QAM splits and says so. So one provisioning run tells the whole story: the fix for this route is a regenerator **and** a mode change, and three attempts at the regenerator alone were not enough. The [link budget](/demo-otn/link-budget.md#a-regenerated-circuit-and-the-two-things-it-costs) page has the table and what a chain costs. **This scenario refuses nothing, and it is the trap in the pair.** The run logs three discarded 16QAM splits. A gate reading the log would block a branch on which the service ended up fine. It reads the node instead: `svc-mad-waw-400g` on `oeo-closed` is `active`, carries two path segments, and holds no reason code, no detail and `refusal_accepted` false. `provisionable` passes, and it is measured rather than argued. `tests/unit/test_demo_scenarios.py::test_the_qpsk_fix_accepts_nothing_because_it_refuses_nothing` holds the file to it, because a signature here would be one on a refusal that does not exist, which is its own error. `channel_collision` is red on `oeo-closed` for the same reason it is red on `oeo-refused`: `06` is loaded underneath and its wavelength pairs overlap. That is the branch below, not the fix on top of it. **Why `oeo-fra-03` and not `oeo-fra-02`.** One device terminating all four Frankfurt wavelengths is a valid plant and it changes the answer, which was measured. With the 16QAM pair and the QPSK pair on one device, the generator takes the mixed cover and closes at +2.745 and +0.240 dB. Both halves close, so it is a real circuit, and it is a poorer demonstration. The second half survives on a quarter of a decibel, and a reader is left thinking the modulation mattered on one side only. A junction needs one device terminating both of its wavelengths, so keeping the QPSK pair on its own device makes that cover unavailable. ## Two loadable scenarios for declared diversity[​](#two-loadable-scenarios-for-declared-diversity "Direct link to Two loadable scenarios for declared diversity") These two share **one** branch, because the point is one run reporting a satisfied promise and a broken one side by side. ``` uv run invoke demo-diversity ``` It makes `diversity-demo`, loads both files, provisions all four members and then runs the check. **`demo/08_diversity_mil_feeds.yml`, a promise that holds.** Milan's two northern feeds arrive through different trenches. Vienna to Milan takes `oms-vie-mil`, whose ducted span sits in `cd-mil-northeast`, and Geneva to Milan takes `oms-gva-mil`, in `cd-mil-northwest`. No conduit is on both routes, so the check passes. It passes because the routes are disjoint and **not** because nobody asked, which is the distinction the file exists to draw. A satisfied requirement and an absent requirement both look like a green check, and only one of them is worth anything. **`demo/09_diversity_fra_feeds.yml`, a promise that does not.** Amsterdam to Frankfurt and Paris to Frankfurt are different cities on different sections, and both enter Frankfurt through `cd-fra-north`. The check **fails**, which is the expected result: ``` svc-fra-feed-ams-100g and svc-fra-feed-par-100g are both in diversity group dg-frankfurt-feeds and their routes share cd-fra-north. One cut in that duct takes both, so the diversity this group declares does not hold ``` With both files loaded the same run adds `Checked 2 diversity group(s) over 4 routed service(s). 1 pair(s) share a conduit against a declaration and 0 member(s) have no route yet. Services declaring no group are not judged here.` The Milan pair is judged and passes in the same breath as the Frankfurt pair is judged and fails. The last sentence is the check saying out loud what it did not look at. Run the same check against the `demo` branch and it reports `PASSED` in silence. Those five services share four ducts between them, which the exposure report lists in scenario seven. None of them declared a group, so this check has nothing to say about any of it. **Provision before checking.** A member with no route yet is reported as undetermined, and undetermined is not a pass. Running the check against one provisioned circuit proves nothing about the pair. ## One loadable scenario for the missing monitor[​](#one-loadable-scenario-for-the-missing-monitor "Direct link to One loadable scenario for the missing monitor") A single rather than a pair, because there is no second file worth reading beside it. The other scenarios on this page load something the network then refuses. This one loads a record an operator would sign off on, and the fault is that something else was never written. Deleting the file is the whole of the fix, so there is no "after" half to compare against. ``` uv run invoke demo-monitor-gap ``` **`demo/10_amplifier_without_monitor.yml`, a stage nobody can measure.** One `OtnAmplifier`, `amp-ham-ber-11`, written in the same fields as the 306 the dataset ships: every mandatory attribute set, every bound respected, no site because an amplifier hut is not a PoP. **The load succeeds**, and the success is the scenario. What is missing is an `OtnAmplifierMonitor` among its ports, and no constraint in `schemas/` can refuse that. A schema constrains what is written, so it cannot notice a gap. `ports` peers the generic `OtnGenericPort` as well, and Infrahub cannot filter a relationship to a generic by the kind of its peers, so the schema has no way to say "at least one of these ports is a monitor" even in principle. The check **fails**, with exactly one finding: ``` amp-ham-ber-11 is an OtnAmplifier and carries no OtnAmplifierMonitor, so nothing can compare its configured gain against what it is delivering. The drift report skips a stage with no monitor rather than reporting it ``` The message names what stops working and not only what is absent. A reader who meets "no monitor" has to work out whether that matters; a reader who meets this one already knows that the gain drift report is now quietly one stage short. The drift report is a task of its own, and this branch is the interesting place to run it: ``` uv run invoke demo-drift --branch monitor-gap ``` It compares the gain each amplifier and Raman pump is configured for against the gain its monitor last reported, and lists the stages outside tolerance. The dataset seeds a droop, so the report always has something to say. What it cannot say anything about is `amp-ham-ber-11`: a stage with no monitor is a row the report never prints, which is why the check above has to be the thing that catches it. The same run says what it counted: ``` Monitor coverage: 306/307 amplifiers, 9/9 Raman pumps, 59/59 transponders, 16/16 multiplexers, 42/42 ROADM degrees. Routers, patch panels and ODU switches carry no monitor and are not judged here ``` Per kind rather than as one total, because the totals differ by two orders of magnitude and 306 covered amplifiers would hide nine uncovered Raman pumps inside a single percentage. The last sentence is the check naming the boundary of what it judged, so a reader is not left inferring it from silence. Run the same check against the default branch and the first figure reads 306/306 and the opening word changes to `Monitor coverage complete`. `59/59 transponders` counts monitors, not light. Sixteen of those fifty-nine monitors sit on a transponder with no wavelength on either line port, and each reports loss of signal: received power at the floor of its range, and no OSNR, Q factor, dispersion or differential group delay. Before this dataset moved, every receiver reported the same healthy 25.1 dB whether or not anything was lit. This check has nothing to say about that either way. It asks whether a device carries a monitor, and a monitor honestly reporting darkness is covered. [Concepts](/demo-otn/concepts.md) says why eight PoPs terminate nothing. **The new amplifier is in no section chain, and that is deliberate.** `oms-ham-ber` has four spans and five amplifiers in each direction, because the budget engine pairs amplifier k with span k and puts a pre-amplifier after the last span. A sixth amplifier on a four-span section raises before a decibel is computed, so `osnr_margin` would turn red on a section this scenario has nothing to say about, and the branch would carry two failures with only one of them the point. The check does not need the relationship: it asks whether a device carries a monitor, and that question never goes through a section. **Negative result: no other check moves on this branch.** `channel_count_consistency` still passes, because an amplifier lights no wavelength and no channel count changes. `container_capacity`, `diversity` and `provisionable` see no service and no container. A check that fired on an unrelated change would be a check nobody reads twice. ## One loadable scenario for the multiplexer channel binding[​](#one-loadable-scenario-for-the-multiplexer-channel-binding "Direct link to One loadable scenario for the multiplexer channel binding") The second single, and the closest relative of the one above: both load records that every schema constraint accepts and that the model is nevertheless wrong about. ``` uv run invoke demo-mux-binding ``` **`demo/11_mux_channel_binding.yml`, two client ports on the wrong number of channels.** A multiplexer client port is one channel of the device, and it says which through one of two optional relationships: `dwdm_channel` to the dense 50 GHz grid, or `cwdm_channel` to the coarse plan. `CH094` on `mux-fra-01` binds neither. `CH1531-2` on `mux-ams-02` binds both, naming dense channel 94 and the coarse 1531 nm the device already lights. **The load succeeds for both**, and the reason is a limit worth naming. The two plans are separate kinds, and Infrahub has no cross-relationship constraint: the schema can make one relationship mandatory or leave both optional, and it cannot say "exactly one of these two". Making either mandatory would leave the other plan's ports with no way to be written at all. So the schema takes what it can, which is the peer kind and the cardinality of each edge, and the check takes the rest. The check **fails**, with exactly one finding per port: ``` mux-fra-01 carries client port CH094, which binds neither a dense channel nor a coarse wavelength. The port is one channel of the multiplexer and nothing says which, so no query can tell what light it passes and no service can be traced through it ``` ``` mux-ams-02 carries client port CH1531-2, which binds dense channel 94 and coarse wavelength 1531 nm at the same time. One filter slot passes one wavelength, so one of the two is wrong and the port's own record does not say which ``` Two findings and not three. The double binding names a coarse wavelength `mux-ams-02` does list, so the check has nothing to add about an unlisted one, and each port draws exactly one complaint. The same run says what it judged and what it did not: ``` 16 multiplexer(s) examined, 96 client port(s) judged for binding exactly one channel and 16 line port(s) skipped, because a line port carries every channel the device lights and binds none by design. 14 of the devices list no coarse wavelength, so their client ports are judged for the binding alone: the graph holds no dense equivalent of cwdm_channels to compare a dense binding against ``` **Negative result: half the question has no data behind it.** A coarse multiplexer lists its wavelengths on `cwdm_channels`, so its ports and that list are two records that can be held against each other, and the check does. A dense one lists nothing, so a client port on the wrong dense channel is a fault this check cannot see. Fourteen of the sixteen units are in that position and the summary says so, because a count of what was skipped is the only thing that keeps silence from reading as a pass. **Negative result: no other check moves on this branch.** A multiplexer port lights no wavelength on its own, carries no service and changes no channel count, so `channel_count_consistency`, `container_capacity`, `diversity`, `provisionable` and `carrier_termination` all read the branch exactly as they read the default one. ## One loadable scenario for the attenuator range[​](#one-loadable-scenario-for-the-attenuator-range "Direct link to One loadable scenario for the attenuator range") The third single, and the smallest question on this page: two numbers on one device, and the comparison between them that no schema constraint can make. ``` uv run invoke demo-attenuator-range ``` **`demo/12_attenuator_range.yml`, a VOA asked for more than it has.** `voa-mil-01` is a 20.0 dB variable attenuator on the add stage at Milan, and the file restates it at 24.0 dB. Every other field is the figure the dataset already ships, so the setting is the only difference between this record and the racked device. **The load succeeds**, and the split is the point. `attenuation_mdb` carries `min_value: 0` and `max_value: 30000`, so the schema refuses a physically impossible figure at write time, from an object file, the API, the UI or a hand edit during a demo. 24,000 is inside that. What the schema cannot say is "not more than this device's own maximum", because that bound is a sibling attribute's value and Infrahub has no cross-attribute constraint. The schema takes the half it can and the check takes the one comparison left over. The check **fails**, with exactly one finding: ``` voa-mil-01 is set to 24.000 dB and can produce 20.000 dB. The device will sit at its stop instead, so the plant delivers 4.000 dB more power than this record says it does and every budget computed from it is optimistic by that much ``` The same run says what it judged: ``` 2 variable attenuator(s) judged against their own maximum, 0 of them dialled to exactly that maximum and passing, because the bound is inclusive. Fixed pads are not judged here and carry no range to judge: the absolute 0 to 30 dB limit is a schema constraint on both kinds and is refused at write time ``` **The bound is inclusive.** A VOA sitting exactly at its maximum is at the end of its travel and is still a setting the device can hold, so equality passes. The demo dials past the stop rather than to it, because a scenario that turned on an off-by-one would be about the check's arithmetic rather than about the network. **Negative result: the fixed pads cannot be judged at all.** The two pads at the ends of the CWDM tail have no `max_attenuation_mdb`, and that is the whole reason the attenuator is two kinds rather than one: a pad with a range is not a value to validate, it is a field that should not exist. The check reports that it did not judge them rather than counting them as passes, because a green result over something nobody looked at reads as evidence. **Negative result: no other check moves on this branch.** An attenuator lights no wavelength, carries no service, holds no channel and has no ports, so `osnr_margin`, `channel_count_consistency`, `container_capacity`, `diversity`, `provisionable`, `carrier_termination` and `mux_channel_binding` all read the branch exactly as they read the default one. The one figure it would move is a link budget, and no optical path in this dataset runs through an attenuator yet. ## One loadable scenario for the transceiver placement[​](#one-loadable-scenario-for-the-transceiver-placement "Direct link to One loadable scenario for the transceiver placement") The fourth single, and the first of two about the pluggable optics the three router wavelengths run on. ``` uv run invoke demo-transceiver-placement ``` **`demo/13_transceiver_placement.yml`, a module in a port with no cage.** `ZRP-BRU-01` is the OpenZR+ QSFP-DD in the Brussels router's line port. The file restates it into `amp-ams-bru-08 OUT`, the booster port at the Brussels end of the same section. That is the port an operator points at when asked where the wavelength leaves the site, and it is also a fixed optical interface on an EDFA with nothing to plug a module into. **The load succeeds**, and the reason is the same limit the multiplexer scenario runs into from another direction. `OtnTransceiver.port` peers the `OtnOpticalPort` generic, and a relationship to a generic cannot be filtered by the kind of its peers. So the schema can offer the field or withhold it, and it cannot say "any of these three kinds and none of the other five". The port kind is the one thing this check is left holding. The check **fails**, with exactly one finding: ``` QDD-400G-ZRP ZRP-BRU-01 is fitted in amp-ams-bru-08 OUT, a port of kind OtnAmplifierPort. A pluggable optic goes in a line, a client or a router port, and no other port kind has a cage to take one, so either this unit is somewhere else or the port on its record is wrong ``` The same run says what it judged: ``` 9 transceiver(s) examined, 6 of them fitted in a port and judged, 3 on a shelf and not judged. An unfitted unit is a spare, an RMA or a decommissioned module, and holding one is why the port relationship is optional. Two modules in one port is not judged here: the schema refuses that write ``` **A port holding two optics has no branch, because the server refuses to build one.** `OtnLinePort`, `OtnClientPort` and `OtnRouterPort` each declare a cardinality-one `transceiver` on the same `otn_optical_port__transceiver` identifier `OtnTransceiver.port` uses, so both ends of the edge are cardinality one and the second module is rejected at write time: ``` Node 18d19871-0d7f-6758-306b-1188b66bb9eb has 2 peers for otn_optical_port__transceiver, maximum of 1 allowed ``` That rule used to live in this check. `uniqueness_constraints: [["port"]]` was tried first and Infrahub accepts it only where the relationship is mandatory, and a mandatory port leaves a spare, an RMA and a decommissioned unit with nowhere to sit, which is three of the nine units here. The reverse edge buys the same refusal at no such cost, and it is declared on the three concrete kinds with a cage rather than on the `OtnOpticalPort` generic, which would have handed the field to the five kinds that hold no module. **A second check speaks on this branch and blocks nothing.** Pulling the module out of `rtr-bru-01 1/2/1` leaves `oc-ch003-ams-bru` with a pluggable at one end and integrated optics at the other, and `transceiver_mode_support` reports that as INFO: ``` oc-ch003-ams-bru is terminated by a pluggable at rtr-ams-01 1/2/1 and by integrated optics at rtr-bru-01 1/2/1. The fitted end is judged against its own part and the other end carries no part number to judge, which is a mixed termination rather than a fault ``` **Negative result: nothing here says the cage fits.** A part carries a `form_factor` and a port does not, so "a QSFP-DD module in a QSFP-DD cage" is a comparison this model cannot make. The check refuses the port kind and says nothing about the cage, which is the honest half. ## One loadable scenario for the transceiver mode support[​](#one-loadable-scenario-for-the-transceiver-mode-support "Direct link to One loadable scenario for the transceiver mode support") The fifth single, and the one where two records that match on every visible field are still the wrong pair. ``` uv run invoke demo-transceiver-mode ``` **`demo/14_transceiver_mode_support.yml`, a 400ZR on an OpenZR+ wavelength.** The OpenZR+ module comes out of the Amsterdam router's line port and the 400ZR spare off the shelf goes in. Both parts are QSFP-DD. Both are DP-16QAM. Both are 400G. Both are tunable. What differs is forward error correction, cFEC against oFEC, and therefore reach: 120 km against 1000 km, on a 220 km section. **The load succeeds**, and no constraint could have refused it. `supported_modes` is a cardinality-many relationship on the part and the mode is a cardinality-one relationship on the carrier, two hops apart through a port. No attribute constraint reaches across two nodes and Infrahub has no cross-relationship constraint. The check **fails**, with exactly one finding: ``` oc-ch003-ams-bru runs OpenZR+ 400G and rtr-ams-01 1/2/1 holds QDD-400G-ZR ZR-SPARE-01, which supports 400ZR. The part does not list OpenZR+ 400G, so the wavelength is planned on an optic that cannot produce it. Cage, constellation and line rate say nothing here: the part number and the mode are the whole of the difference ``` The same run says what it judged: ``` 43 wavelength(s) examined, 3 judged against the parts fitted at their line ports and 40 skipped for holding no pluggable at either end. A skipped wavelength is terminated on integrated optics, which carry no part number and no mode list, so this check's silence about them is not a verdict ``` **Negative result: forty of the forty-three cannot be judged.** A transponder lights its wavelength from a laser on the line card, so there is no part number and no mode list to compare anything against. Only the three router wavelengths carry a pluggable, and a run reporting no findings over forty-three skips would look exactly like a run that judged them all and found nothing. The split is printed for that reason. **Negative result: this check says nothing about whether the mode closes.** Whether `OpenZR+ 400G` reaches over the plant is `osnr_margin`'s question, and it does: +10.890 dB on `oms-ams-bru`. The 400ZR fitted here would fail on chromatic dispersion rather than on power, over its 2400 ps/nm tolerance by 1340 ps/nm with 8 dB of OSNR still in hand. That is the argument for oFEC and it belongs on [the link budget page](/demo-otn/link-budget.md), not in this check. ## One loadable scenario for the connector polish[​](#one-loadable-scenario-for-the-connector-polish "Direct link to One loadable scenario for the connector polish") The sixth single, and the one where nothing about the object changed. What changed is the glass it faces. ``` uv run invoke demo-connector-polish ``` **`demo/15_connector_polish.yml`, a blue jumper on pumped fibre.** `span-vie-mil-01` is the first span of the one section this network pumps, and its Vienna end is the only end of that section anyone can reach: spans 2 to 9 terminate in huts. This file re-patches that end. `roadm-vie-01 DEG-MIL` comes off and `xpdr-vie-02 L2` goes on, a transponder patched straight onto the line past the ROADM. `L2` is the one dark line port at Vienna, so nothing carrying traffic moved. Its endface is UPC, as every line port in the dataset is. That is the right endface where a coloured line port normally sits, behind an add/drop stage inside a building, where return loss is a specification. On the far side of the ROADM it is a hazard: the nine pumps on this section put half a watt into the fibre and a flat endface sends about 50 dB of it back down the span. **The load succeeds**, and no constraint could have refused it. `polish` is an attribute on the port and `raman_pumps` is a relationship on the span, two hops apart, and Infrahub has no cross-relationship constraint. The check **fails**, with exactly one finding: ``` xpdr-vie-02 L2 terminates span-vie-mil-01, which is Raman-pumped, and its endface is UPC. A pump puts half a watt into this fibre and a UPC endface reflects that back down it, so this connector has to be APC before the pump is turned up ``` The same run says what it judged: ``` 133 span(s) examined, 9 of them Raman-pumped, 9 judged against their terminating ports and 0 unjudgeable for naming none. An unjudgeable span is not a passing one, and a run reporting no findings over a pumped section it could not reach has seen nothing ``` **That last zero is the figure to read.** A span reaches its ports through `OtnFiberSpan.terminating_ports`, and a pumped span naming none is reported as unjudgeable rather than clean, because an empty relationship and two correct APC ends look identical from inside the check. The dataset generator populates all 133 spans so the count reads zero, and a run where it does not is a run whose green result means nothing. **Negative result: the relationship had to be added before the check could be written.** No port kind had any edge to a fibre span or to an optical multiplex section in either direction. The nearest reachable ports were three hops away through the section and its ROADMs, and a ROADM carries one degree port per direction whose `connected_to` is null on every one of them, so the only thing naming the port facing a given span was the far site's shortname inside the port's name string. Parsing that is a naming convention doing a relationship's job, and the schema carries the edge instead. **Negative result: polish is judged on pumped glass and nowhere else.** The 124 unpumped spans terminate on APC degree ports and the check says nothing about them, and it says nothing about the 126 line ports either, because none of them faces pumped fibre on the default branch. A flat endface is a normal, correct choice almost everywhere in this network. ## The runbook[​](#the-runbook "Direct link to The runbook") About thirty minutes, one branch until step 12, in narrative order. **Setup, once.** The object load takes a few minutes; start it before the audience arrives. ``` uv run invoke start uv run invoke demo-setup ``` Every step below runs on the `demo` branch and none of them needs you to say so. Each one ends by naming the step that follows, and a step run out of order says which task to run first instead of failing inside the client. To watch the whole thing without typing between steps: ``` uv run invoke demo ``` **1. The network, on one page.** Two minutes, in the browser, before any command. Open **OTN Sites**, pick Frankfurt, open its **Artifacts** tab, and open `network-map`. Fourteen PoPs in their real positions, 21 routes coloured by how much OSNR margin a 400G `DP-16QAM` carrier has on each. Point at the red route into Madrid, then at the chip on Frankfurt to Milan. Several of the findings the next twelve steps compute are already on this drawing, which is what makes it worth two minutes. The [network map](/demo-otn/network-map.md) page has the rest of the legend. `odu-map` is on the same Artifacts tab and is mostly grey here. Say what it is, leave it closed, and come back to it at step 12. **2. What is in the network, and what is left.** Two minutes. ``` uv run invoke demo-capacity ``` Cover the 4,134,400 of 4,800,000 MHz first, then the eight-against-one gap between free spectrum and usable anchors. Then the sixteen empty sections, and the difference between a section's free spectrum and a route's. **3. Where the cheap parts reach.** Two minutes. Run it before anybody asks. ``` uv run invoke demo-reach ``` Zero of twenty-one. Pause here: this is the step where the model returns an answer nobody wanted, computed from data rather than assumed. **4. Ask for a service and watch it choose.** Four minutes. ``` uv run invoke demo-provision ``` Six candidate routes, Hamburg wins on fewest sections then margin. Read the discarded ones out loud: each one names the route, the mode, the margin and the reason it lost. **5. Provision the rest.** One minute, and it is setup for what follows. ``` uv run invoke demo-provision-all ``` Two groom and two light, in that order, every time. `svc-fra-mil-ai-400g` and `svc-ams-mil-ai-400g` take tributary slots in wavelengths that are already up, `oc-ch047-fra-mil` and `oc-ch002-ams-mil`. Then `svc-fra-gva-hpc-400g` and `svc-vie-mil-hpc-400g` find no line container with room and each light channel 2, on `oms-fra-gva` and `oms-vie-mil`. Channel 2 and not channel 1, because a 79,600 MHz carrier centred on channel 1 falls off the bottom of the band. Say the grooming out loud: it is the ODU layer choosing not to buy spectrum, and it is why the cut in step 7 drops seven wavelengths and not eight. **6. Trace one of them.** Two minutes. ``` uv run invoke demo-trace ``` Thirty-five hops, router to router. Point at the conduit column; it sets up step 8. **7. Cut the fiber.** Three minutes. Leave room for questions on this one. ``` uv run invoke demo-impact ``` Seven wavelengths, 2.8 Tbps, one AI service, one customer, and six wavelengths the model holds with no customer behind them. Then the duct breakdown, and the fact that `cd-fra-north` also carries Paris to Frankfurt. Then mistype it, deliberately: ``` uv run invoke demo-impact --section oms-ams-frs ``` It refuses. An impact tool that answers "no impact" to a typo is worse than one that answers nothing. **8. Who is not diverse.** Two minutes. ``` uv run invoke demo-srlg ``` Frankfurt to Geneva and Amsterdam to Milan, no shared section, one shared duct. **9. The latency budgets.** Two minutes. ``` uv run invoke demo-latency ``` Then the electronics column, and the sentence about there being no trade-off. **10. The one service that states its handover.** Two minutes. ``` uv run invoke demo-infiniband ``` Frankfurt to Prague at 212 Gbps with `client_signal: IB-HDR-4X`, and the container comes back `ODUflex`. Drop that one line from the file and the same request provisions `400GBASE-FR4`, with no error. The rate rule picks only from catalog rows marked `auto_selectable`, which the two InfiniBand rows are not. 212 Gbps and not 200: HDR signals at 212.5, which is 170 tributary slots, and a 200G wavelength offers 160. The request lands on DP-16QAM 64GBd 400G, whose `ODUC4` offers 320, so the `ODUflex` takes 170 and leaves 150 free. **11. Congestion in two layers.** Three minutes. Fill both layers of the Frankfurt to Milan corridor, then ask twice. ``` uv run invoke demo-refusal ``` The 400G is refused on slots: it needs a whole ODUC4 and the roomiest wavelength there has 240 of 320 free. The 100G then provisions on the same corridor without lighting anything. 532,400 MHz free in 29 blocks none of which reaches 44,400 says no new wavelength can be lit; it says nothing about room inside the ones already lit. **12. The other map, and one question off it.** Three minutes, in the browser. This one wants its own branch, which the task makes: ``` uv run invoke demo-odu ``` Open `odu-map` on any PoP on `odu-demo`. Same coastline, same fourteen discs, different question. Ask the room for two lists off the colours alone: which sections can still take a 100G circuit, and which can take nothing. Green is the first, red is the second, and the panel's FITS column spells both out. The task finishes by checking the answer against the containers instead of against the map. The two cannot disagree. The map, the check and the provisioning generator all call one implementation of the capacity rule. Finish on the grey: the sections carrying no wavelength at all, which the map says "not known" about rather than "available". **13. The branch.** One minute. Open the proposed change and show the diff and the checks. Everything in the last thirty minutes happened on a branch, the default branch is untouched, and the checks decide whether any of it merges. ## Clean up[​](#clean-up "Direct link to Clean up") ``` uv run invoke demo-clean ``` With no argument it deletes every branch a scenario task creates, naming each one as it goes and reporting how many of them it found. Nothing is removed without saying so, and `--branch` still takes a single name. The default branch is back to the state it shipped in: the fourteen PoPs and their plant, the customer campus on the end of its coarse tail, no services, and the OSNR check still failing on Paris to Madrid. Every service in this guide was provisioned on a branch, which is the point. --- # Network map ![The fourteen PoPs of the modelled European optical core, with the 21 multiplex sections between them coloured by OSNR margin. Paris to Madrid is red, Frankfurt to Milan carries a 4,134 GHz occupancy chip, and Vienna to Milan is marked as Raman pumped and asymmetric.](/assets/images/network-map-8fcf74ec4e226fed9f8827de50441ce4.svg) Berlin's copy. Every figure on it is read from the branch, so nothing here is typed in. Click it to open the SVG at full size, where the panel type is legible. Every PoP carries a drawing of the whole optical core. Open a site in Infrahub, then open its **Artifacts** tab. `network-map` is an SVG of the fourteen PoPs, the 21 multiplex sections between them and the 132 fiber spans inside those sections. Each route is coloured by the OSNR margin a fixed reference mode, `DP-16QAM 64GBd 400G`, would have on it. That is a margin evaluated on every section alike, not the margin of whatever wavelength happens to be lit there. The map is produced by the same code the checks and the reports call. Nothing on it is drawn by hand, and no figure on it is typed in. ## Where to find it[​](#where-to-find-it "Direct link to Where to find it") **OTN Sites** in the sidebar, then any PoP, then **Artifacts**. Infrahub renders the artifact from this repository, so the maps appear wherever the repository is registered, and a later pipeline run against a change re-renders them. An unchanged network renders byte for byte the same file, so a map that did change is a network that did. One map is rendered per PoP. Each copy draws the same network and emphasises a different site. The site the copy belongs to gets a dark ring, and the routes terminating on it are drawn heavier than the rest. A copy therefore reads as that site's reach into the core rather than as a general map with a pin stuck in it. Amsterdam Science Park carries no map. It is a customer campus on the end of a coarse metro tail, it belongs to no multiplex section, and the map does not draw it. A map attached to it would be a picture of a network it is not part of, so the absence is the correct result. ## Why the colour is a margin and not a loss[​](#why-the-colour-is-a-margin-and-not-a-loss "Direct link to Why the colour is a margin and not a loss") Route colour is the OSNR margin for `DP-16QAM 64GBd 400G`, which needs 24.500 dB, and the check adds 1.000 dB of system margin on top, so a section has to deliver 25.500 dB to pass. The mode is named on the map, next to the legend, because a margin means nothing without the mode it is a margin against. Loss was the obvious first choice and it is the wrong one. The plant here is uniform, so a section's loss tracks its length almost exactly, and a map coloured by loss draws how long each route is. The distance labels already say that. A margin answers the question a planner actually asks: does a carrier fit on this route, and by how much. | Colour | Margin | Sections on the shipped dataset | | ------ | -------------------------- | ------------------------------- | | Red | below 0 dB, does not close | 1 | | Amber | 0 to 2 dB | 2 | | Olive | 2 to 5 dB | 6 | | Green | 5 dB and above | 12 | | Grey | not computed | 0 | A section is budgeted once from each ROADM, and the two amplifier chains do not have to agree, so a route is painted with the worse of its two directions. Both figures are in the panel table. Grey is a fifth answer and not a missing one. A section whose margin could not be computed keeps its route and loses its colour, because an absent number is a different statement from a good one. Dropping the route would make the network look better connected than it is, and defaulting it into a passing band would make a broken section look healthy. ## Paris to Madrid is the red one[​](#paris-to-madrid-is-the-red-one "Direct link to Paris to Madrid is the red one") `DP-16QAM 64GBd 400G` does not close on Paris to Madrid. The margin is -0.535 dB, in both directions, and that is the section the red line is. The route is 1250 km, the longest single section in the network, and there is no intermediate PoP to split it into two shorter ones. Half a decibel is what a section that long costs at that modulation. This is why the OSNR check fails on the default branch, and it is a true finding about a long route rather than a fault in the data. The colour is a verdict about one mode, not about the fiber. Drop to `DP-QPSK 32GBd 100G` and the same section holds +9.965 dB, which is the tightest margin anywhere in the network at that mode. The [link budget](/demo-otn/link-budget.md) page has the three answers an engineer would reach for, and what each one costs. ## The one asymmetric route[​](#the-one-asymmetric-route "Direct link to The one asymmetric route") Vienna to Milan is the Raman-pumped section and the only route whose two directions land in different colour bands: +10.743 dB in the direction the pump serves, +0.955 dB against it. A pump credits its on-off gain to the one direction it amplifies, and its combiner charges insertion loss to both. The map marks the route with a dashed overlay in the better direction's colour and puts a circled **R** on it. A route that is good one way and thin the other is not read as uniformly thin. ## What each visual channel encodes[​](#what-each-visual-channel-encodes "Direct link to What each visual channel encodes") | Channel | What it says | | --------------------------------------- | -------------------------------------------------------------------------------------------------- | | Route colour | The worse of the section's two OSNR margins at the reference mode | | Dashed overlay on a route | The two directions sit in different bands. The dashes are the better direction's colour | | Circled **R** | A Raman pump on one of the section's spans, which is what makes it asymmetric | | Dots along a route | Span boundaries, one per amplifier site, placed at the fraction of route length where the hut sits | | Node radius and the number inside it | The site's optical degree, the number of sections terminating on it | | Dashed violet ring and a second caption | The site hosts a EuroHPC machine, named in the caption | | Dark ring and heavier route casing | The site this copy of the map belongs to | | Label riding a route | Route length in kilometres, rotated to follow its own line | | Chip on a route | Spectrum in use in gigahertz, shown only on a section holding a quarter of the C-band or more | Frankfurt to Milan is the only section wearing a chip: `4,134 GHz` of the 4,800 GHz the C-band gives it. The denominator is on the panel's totals line rather than in the chip, because the chip rides a rotated route and the box is narrow. Below a quarter of the band the figure is in the panel table and nowhere else, because a chip on all 21 routes would be 21 boxes competing with the distance labels. The map stays sparse on purpose. Exact figures live in the panel on the right, which holds the legend, then every section as a row, ordered worst margin first, with length, loss, both directional margins and occupancy. Under the table are four totals: route distance, fiber spans, spectrum in use, and the C-band extent per section. The footer names the branch the map was generated from, since a margin and an occupancy figure are true of the branch they were read from and of nothing else. ## A network with no services yet[​](#a-network-with-no-services-yet "Direct link to A network with no services yet") On a branch holding the plant and no carriers, the occupancy column reads zero on every row, no route wears a chip, and the spectrum in use total reads 0 GHz. That is the correct picture, not a broken render. Occupancy is derived from the carriers on the branch rather than stored on the section, so it answers for that branch and moves the moment a service is provisioned. Provision one and the next render of the map shows the gigahertz move by exactly the width of the mode that was chosen. ## Coastlines[​](#coastlines "Direct link to Coastlines") The land, the borders and the graticule are Natural Earth 1:50m outlines, public domain, embedded in the repository as generated Python data. The renderer makes no network request and no mapping library is installed. The [developer guide](/demo-otn/developer-guide.md#regenerating-the-basemap) has the command that regenerates the data. --- # ODU map ## Which of the two maps you are looking at[​](#which-of-the-two-maps-you-are-looking-at "Direct link to Which of the two maps you are looking at") Every PoP carries two SVG artifacts and both are a map of Europe, with the same coastline, the same fourteen discs and the same 21 routes between them. What they say about a route is not the same thing. | Artifact | Rendered title | Colour answers | | ------------- | ------------------------- | ------------------------------------------- | | `network-map` | European optical core | Does a wavelength **close** on this route? | | `odu-map` | ODU capacity and grooming | Does another **circuit fit** on this route? | The [network map](/demo-otn/network-map.md) colours a route by the OSNR margin a reference mode has on it, which is a statement about the optical layer: whether light survives the span, the amplifiers and the ROADMs end to end. This map colours a route by the free tributary slots inside the wavelengths already lit on it, which is a statement about the digital layer carried inside that light. One route can be green on one map and red on the other, and both are correct. A wavelength with 25 dB of margin and no free slot carries nothing more. Neither map derives from the other. The margin knows nothing about occupancy, and the slot count knows nothing about reach. There is no length, no loss, no margin and no span boundary on this drawing, because a multiplex section has no total length attribute for this map to state one from. ## Where to find it[​](#where-to-find-it "Direct link to Where to find it") **OTN Sites** in the sidebar, then any PoP, then **Artifacts**, then `odu-map`. Infrahub renders it from this repository on the branch you are reading, so the same site shows a different picture on `main` and on a branch with services provisioned. Nothing about the map is stored. One copy per PoP, fourteen in all. Each copy draws the same network and emphasises its own site with a dark ring and heavier route casings, exactly as the network map does. Amsterdam Science Park carries neither map: it is a customer campus on a coarse metro tail, it belongs to no multiplex section, and it appears on no route the map draws. Off a loaded stack you can render one without opening a proposed change: ``` uv run invoke load-repository # if the stack has no repository yet # then, from the Artifacts tab on any PoP, open or download odu-map ``` ![The fourteen PoPs of the modelled European optical core, with the 21 multiplex sections coloured by the largest ODU that still fits on the roomiest lit carrier. Amsterdam to London is red at 80 of 80 slots committed and Vienna to Milan red at 240 of 240, Paris to Frankfurt amber at 953 of 960, Berlin to Frankfurt blue at 1552 of 1600, five routes green, and the remaining twelve grey and dashed.](/assets/images/odu-map-331b420986d1e2b3f2027c4ae1092553.svg) Frankfurt's copy of `odu-map`, off a branch carrying `demo/05_odu_mixed_fill.yml`, so all five bands are on one picture. The default branch shows only two of them: `odu4` on eight sections and `no-odu` on thirteen. The measured split in this picture is `odu4` 5, `odu2` 1, `odu0` 1, `full` 2 and `no-odu` 12. Three of the five green routes are the router-to-router OpenZR+ wavelengths, each an empty ODUC4 offering all 320 of its slots. Every figure in the panel was checked against the containers on the branch rather than taken from the drawing. ## The five bands[​](#the-five-bands "Direct link to The five bands") A section is coloured by the largest container that still fits on its **roomiest** lit carrier. The edges are client sizes rather than round numbers, in tributary slots of 1.25 Gbit/s. | Colour | Free slots on the roomiest lit carrier | Reads as | | ------------ | -------------------------------------- | ------------------------------------------- | | Green | 80 or more | A 100G still fits | | Blue | 8 to 79 | A 10G fits, a 100G does not. A 40G needs 32 | | Amber | 1 to 7 | Only a 1G or 2.5G fits | | Red | 0 | Nothing fits | | Grey, dashed | no container on any carrier | Not known | One slot is the smallest client in the catalog, so below one nothing fits at all. Eight slots is an `ODU2`, the 10G tributary. Thirty-two is an `ODU3`, which is where a 40G client lands: STM-256 is 39.8 Gbit/s, so it needs 32 slots and not the eight an `ODU2` offers. Eighty is an `ODU4`, the 100G one. A band edge that was not a client size would colour a route by an arithmetic nobody provisions against. The red band is unbounded below rather than pinned at zero, so an overfilled carrier lands in it too. `free_slots` returns a negative figure for one instead of clamping, and a band that matched only zero would drop that case on the floor. ## Nothing fits and nobody knows are different cells[​](#nothing-fits-and-nobody-knows-are-different-cells "Direct link to Nothing fits and nobody knows are different cells") Red and grey are two different findings and the panel keeps them apart. The largest-fit column prints `none` for a section that was measured and has no room, and `no ODU` for one that was not measured at all. Two words rather than one blank, because a blank cell in that column reads as a value nobody got round to filling in and both of these are answers. Grey also gets a dash on the route, not only a colour. On the network map one route in twenty-one is grey and a reader has nineteen coloured ones to compare it against. Here the whole map can be grey, and then colour alone says nothing. ## Colour is the roomiest carrier, the panel is the tightest[​](#colour-is-the-roomiest-carrier-the-panel-is-the-tightest "Direct link to Colour is the roomiest carrier, the panel is the tightest") Headroom answers "can I still provision here", and provisioning goes onto the emptiest wavelength with room. So the colour follows the roomiest carrier on the section. The tightest carrier is reported separately, as a number and as a fill bar, in the same panel row. A section that looks roomy while one of its wavelengths is nearly full then shows both facts at once, and neither is averaged away. An aggregate percentage was rejected for exactly that reason. Thirty-eight per cent committed across a section whose `ODU4` sits at 76 of 80 tells a planner they have room where they have none. The bar is scaled to one `ODU4`, 80 slots, the same yardstick the top colour band uses, so the colour and the bar measure the section with one ruler at two ends. Frankfurt to Milan on the shipped dataset is the section that settles this. It carries 40 wavelengths, 37 `ODUC4` offering 320 slots and three `ODU4` offering 80, all of them empty. A per-carrier mean would draw its bar three quarters full and print the figure in red, when its tightest carrier is an untouched `ODU4`. ## A dark carrier is not free capacity[​](#a-dark-carrier-is-not-free-capacity "Direct link to A dark carrier is not free capacity") A wavelength with no container on it contributes to no numerator and no denominator. Slot capacity exists once a container is written on the wavelength, so counting an unlit channel as available slots would invent capacity no equipment is offering. Counting it as zero free would be worse: an unprovisioned section would read as full. The same rule runs upward through the tree. A child container of a type the slot table cannot size makes its parent's free-slot figure unknown rather than being computed from the children that are known. Four of the sixteen container types have no G.709 slot figure: the flex container and the three SDH virtual container types. A section holding one lands in the grey band even though it has a lit carrier. The panel says so: the row shows a lit-carrier count and `no ODU` beside it. ## `no-odu` does not mean empty and available[​](#no-odu-does-not-mean-empty-and-available "Direct link to no-odu-does-not-mean-empty-and-available") On a clean `main` the grey band covers **13 of the 21 sections**, and most of the map is grey. That is the honest picture and not a broken render. The 13 are sections with **no wavelength at all**, not sections whose wavelengths are unlit. Every pre-provisioned carrier in the dataset arrives carrying an empty line container, so a section with a carrier is lit. The 43 wavelengths in `objects/17_geant_carriers.yml` cover eight sections and nothing else: | Section | Lit carriers | Committed over offered | Band | | ------------- | ------------ | ---------------------- | ----- | | `oms-fra-mil` | 40 | 0 / 12080 | Green | | `oms-ams-fra` | 7 | 0 / 2240 | Green | | `oms-ber-fra` | 5 | 0 / 1600 | Green | | `oms-par-fra` | 3 | 0 / 960 | Green | | `oms-vie-mil` | 3 | 0 / 240 | Green | | `oms-ams-bru` | 1 | 0 / 320 | Green | | `oms-ber-prg` | 1 | 0 / 320 | Green | | `oms-ham-ber` | 1 | 0 / 320 | Green | The other thirteen have no carrier, so they have no slot figure, so they are grey. The band split on `main` is `odu4` 8 and `no-odu` 13, and `tests/unit/test_odudraw.py` asserts that pair against the shipped dataset. ## The panel[​](#the-panel "Direct link to The panel") The map carries no exact figure at all, which is why the panel is longer than the network map's. Every number a reader wants is there, one row per section, ordered so the section that runs out of room first is the first row. | Column | What it says | | ------------- | ---------------------------------------------------------------------------- | | Dot and route | The band colour, and the two site codes | | LIT | Lit carriers on the section, meaning carriers holding at least one container | | SLOTS | Committed slots over slots offered, across those lit carriers | | Bar | The tightest carrier on the section, against one `ODU4` of room | | TIGHT | The same figure as a number, in free slots | | FITS | The exact largest container type that fits, or `none`, or `no ODU` | Under the table are four totals: sections with a headroom figure, least headroom on a section, tightest carrier anywhere, and sections where nothing fits. The last two are the negative results, and they are counts rather than percentages because a percentage of twenty-one sections hides which ones. There is no network-wide capacity total, deliberately. A wavelength runs end to end over several sections and every one of them counts it, so summing the per-section offerings would report more capacity than the network has. The per-section figures are the honest ones and the totals are the extremes over them. The footer names the branch the figures were read from. A slot count is true of the branch it was read from and of nothing else. ## The scenarios that put something on the map[​](#the-scenarios-that-put-something-on-the-map "Direct link to The scenarios that put something on the map") Neither scenario file is loaded by `.infrahub.yml`. Both are scenario input under `demo/`, and they can share one branch. One task makes that branch, loads both files, provisions the eleven circuits the first one asks for and runs the capacity check over what is left: ``` uv run invoke demo-odu ``` **`demo/04_odu_ten_in_one.yml`, ten circuits in one wavelength.** Ten STM-64 services from London to Milan each map into an `ODU2` of 8 slots. All ten groom into the same `ODU4` line container, which offers 80. Ten eights are eighty, so after the tenth that container reads 80 of 80 and the wavelength is out of room. The eleventh service of the same size is **refused**, reason `no-slots`, naming the container it did not fit and both slot figures. Grooming is tried first and lighting second, so the refusal needs both to fail. The scenario also spends the last usable block of spectrum on `oms-fra-mil`, taking it to 4,267,600 MHz of 4,800,000 with 532,400 MHz left in 29 blocks whose widest is 38,000 MHz. That is narrower than the narrowest mode in the catalog, so nothing can be lit there and the lighting escape closes. The band split on that branch is `odu4` 5, `full` 1, `no-odu` 15. Only `oms-ams-lon` turns red, because the shared wavelength is the only lit carrier on it. `oms-ams-fra` and `oms-fra-mil` stay green, and that is the correct reading rather than a disappointment. A section is banded on its roomiest lit carrier, and both of those still carry empty `ODUC4` wavelengths offering 320 slots. **`demo/05_odu_mixed_fill.yml`, every band at once.** Five bands are five different sentences about a section, and a map showing one of them proves nothing about the other four. This file puts at least one section in each, on one branch, so the legend can be read against the picture. The measured split is `odu4` 5, `odu2` 1, `odu0` 1, `full` 2, `no-odu` 12, over 21 sections. Three of the five `odu4` sections are the router-to-router OpenZR+ wavelengths, which this file does not touch and which arrive empty. It writes client containers directly rather than running the generator twenty times, because the scenario is about what the map states and not about how the containers arrived. `oms-fra-mil` is left green on purpose: all 40 wavelengths cross it, the file fills eleven, and the remaining twenty-nine are still empty at 320 free. Filling it would take over a thousand more client containers. The model does not pretend otherwise. ## A branch with no ODU layer at all[​](#a-branch-with-no-odu-layer-at-all "Direct link to A branch with no ODU layer at all") Every route grey, dashed, and a caption in the title block naming the branch and saying no ODU layer is provisioned on it. The render succeeds; it is a caption rather than an error. The default branch is **not** that picture. Its pre-provisioned wavelengths ship carrying empty line containers, so they are lit and their five sections land in a real band. The all-grey render is reachable on a branch whose containers have been removed, and the caption names the branch so nobody reads it as the model being empty. ## Determinism[​](#determinism "Direct link to Determinism") The same records always produce the same bytes, whatever order they arrive in. That is what makes a changed artifact worth looking at: the map moved because the branch did. ## Coastlines[​](#coastlines "Direct link to Coastlines") The land, the borders and the graticule are the same Natural Earth 1:50m outlines the network map uses, embedded in the repository as generated Python data. Both maps draw them through `src/infrahub_demo_otn/mapchrome.py`, the shared chrome, and both run through `src/infrahub_demo_otn/mapengine.py`, the one drawing engine that lays out sections, routes, title and panel for either map. Between them they are the reason the two artifacts look like a pair. The [developer guide](/demo-otn/developer-guide.md#regenerating-the-basemap) has the command that regenerates the data, and [the golden renders](/demo-otn/developer-guide.md#the-golden-renders-under-tests) are what keep that sharing from moving either map. --- # Provisioning scenarios The two scenarios that write. One provisions a service and shows what the route selection did not pick. The other puts a second service on a corridor that has run out of colour, and shows why one closes and the other does not. Every number below is produced by the engine and asserted by a unit test. The [demo guide](/demo-otn/demo-guide.md) carries the run order and links the other scenario pages. ## Before you start[​](#before-you-start "Direct link to Before you start") ``` uv run invoke start uv run invoke list ``` Every numbered scenario below is an invoke task. The tasks read the address and the API token from `.env` and pass both to whatever they run, so nothing in the runbook asks you to export a credential first. That holds on every scenario page, including [loadable scenarios and the runbook](/demo-otn/loadable-scenarios.md). note The OSNR check fails on the default branch, and that is the shipped state. Paris to Madrid is 1250 km and falls 0.535 dB short at `DP-16QAM 64GBd 400G`, in both directions. It is a true finding about a long route, not a broken demo, and the [link budget](/demo-otn/link-budget.md) page covers what the fix costs. Expect to see it if you run the check before provisioning anything. ## Scenario one: Berlin to Amsterdam at 400G[​](#scenario-one-berlin-to-amsterdam-at-400g "Direct link to Scenario one: Berlin to Amsterdam at 400G") ### Create the branch and the request[​](#create-the-branch-and-the-request "Direct link to Create the branch and the request") ``` uv run invoke demo-setup --branch provision-ber-ams-400g ``` That creates the branch and loads the schema, the menu and the dataset. It then loads the five service requests. The generator target group comes with the dataset, in `objects/00_groups.yml`, and the numeric prefix puts it first. Each service declares `member_of_groups: [optical_services]`, and a service cannot join a group that does not exist yet. The task skips the dataset load when the branch already holds it, so a second run costs seconds rather than minutes. A schema load onto a branch already holding the 2490-object dataset runs longer than the client's 120 second timeout. The server finishes anyway. Poll `/api/schema?branch=provision-ber-ams-400g` rather than running the load again, and wait until `nodes` plus `generics` reaches the same count the endpoint reports for `main`. That total counts Infrahub's built-in kinds as well as the 44 this repository declares, so it moves with the Infrahub version and `main` is the only reliable thing to compare it against. The request itself is eight lines of intent. No route, no wavelength, no channel: ``` - name: svc-ber-ams-400g description: Berlin to Amsterdam, 400G IP transit. customer: NREN-DE rate_gbps: 400 sla: gold service_profile: ip-transit endpoint_a: rtr-ber-01 endpoint_z: rtr-ams-01 ``` ### Run the generator[​](#run-the-generator "Direct link to Run the generator") ``` uv run invoke demo-provision --branch provision-ber-ams-400g ``` ``` svc-ber-ams-400g: candidate 1 of 6 is oms-ham-ber|oms-ams-ham on DP-16QAM 64GBd 400G, 2 sections, 800 km, margin +2.284 dB, 3923.026 us, channel 2 svc-ber-ams-400g: candidate 2 of 6 is oms-ber-fra|oms-ams-fra on DP-16QAM 64GBd 400G, 2 sections, 1010 km, margin +0.507 dB, 4951.540 us, channel 38 svc-ber-ams-400g: candidate 3 of 6 is oms-ber-cph|oms-ham-cph|oms-ams-ham on DP-QPSK 128GBd 400G, 3 sections, 1330 km, margin +5.378 dB, 6519.243 us, channel 2 svc-ber-ams-400g: candidate 4 of 6 is oms-ber-prg|oms-prg-fra|oms-ams-fra on DP-QPSK 128GBd 400G, 3 sections, 1220 km, margin +5.241 dB, 5980.401 us, channel 23 svc-ber-ams-400g: candidate 5 of 6 is oms-ber-fra|oms-par-fra|oms-bru-par|oms-ams-bru on DP-QPSK 128GBd 400G, 4 sections, 1700 km, margin +2.608 dB, 8331.574 us, channel 2 svc-ber-ams-400g: candidate 6 of 6 is oms-ber-fra|oms-par-fra|oms-lon-par|oms-ams-lon on DP-QPSK 128GBd 400G, 4 sections, 2080 km, margin +2.003 dB, 10192.828 us, channel 2 svc-ber-ams-400g: chose oms-ham-ber|oms-ams-ham on DP-16QAM 64GBd 400G, channel 2, 800 km, margin +2.284 dB, 3923.026 us ``` **The winning route is empty and the anchor is still 2, not 1.** A `DP-16QAM 64GBd 400G` carrier occupies 79,600 MHz. Centring it on channel 1 would put its lower edge 14,800 MHz below the bottom of the C band. Channel 1 and channel 96 are unusable for every 400G mode in the catalog. The report says 94 usable anchors rather than 96 on a section with nothing on it at all. This is the smallest visible consequence of modelling width, and it shows up on the emptiest route in the demo. ### What the generator did[​](#what-the-generator-did "Direct link to What the generator did") The candidate routes came from `client.traverse_paths()`, filtered to the two relationships that mean "this section terminates on that ROADM". There is no graph walker in this repository. Each route was then budgeted at every transponder mode that reaches 400 Gbps, and the narrowest mode that closes won. That is why Hamburg and Frankfurt land on DP-16QAM and the two three-section routes land on DP-QPSK. Both three-section routes fail at DP-16QAM, by 0.259 dB through Prague and 0.122 dB through Copenhagen, and both close comfortably one modulation order down at twice the spectrum. Copenhagen is 110 km longer than Prague and has the better margin. Its sections are built from shorter spans, and span loss enters the OSNR cascade exponentially while route length enters it linearly. Route length alone does not order signal quality. The ranking is fewest sections, then highest margin, then lowest channel, then the route's own name. Hamburg and Frankfurt tie at two sections and Hamburg wins on 2.284 against 0.507. The order is total, so the same request produces the same answer on every run. note Ranking modes by margin instead of by spectrum would put every carrier in the demo on DP-QPSK, because dropping two bits per symbol gains about 5.5 dB. It would also double the spectrum each one occupies. The selector treats the lower-order fallback as the cost it is. ### Read the result[​](#read-the-result "Direct link to Read the result") The generator wrote 29 objects and changed one: | Kind | Count | What it holds | | ------------------- | --------- | -------------------------------------------------------------------------- | | `OtnOpticalCarrier` | 1 | Channel 2, DP-16QAM 64GBd 400G occupying 79,600 MHz, both Hamburg sections | | `OtnOpticalPath` | 1 | 800 km, 212.202 dB, 27.784 dB OSNR, +2.284 dB margin, 3923.026 us | | `OtnPathHop` | 25 | Three ROADMs, twelve spans, ten amplifiers, in order | | `OtnContainer` | 2 | The line container on the carrier, and the client container inside it | | `OtnService` | 1 changed | Status `active`, pointing at the path | No pre-provisioned wavelength crosses exactly `oms-ham-ber` and `oms-ams-ham`, so there was nothing to groom into and this run lit a wavelength. That is two containers rather than one. `odu-line-oc-svc-ber-ams-400g` is an `ODUC4` holding the carrier and offering 320 tributary slots. `odu-svc-ber-ams-400g` is the `400GBASE-FR4` client nested inside it with GMP mapping, occupying all 320 and holding no carrier of its own. A 400G client fills a 400G wavelength exactly, so that wavelength now reports 320 of 320 committed and nothing else can groom into it. A service on the Frankfurt to Milan corridor takes the other path and writes one container, because forty wavelengths are already lit there. Scenario two is that case. **The carrier it wrote has an empty `line_ports` list, and that is a limitation rather than a bug.** Every one of the 43 wavelengths in the shipped dataset names the two line ports that terminate it, one at each end, because the generator that writes `objects/` binds them. `generators/optical_service.py` names no line port and no transponder: it writes the carrier, the path, the hops and the containers, picks the route and the mode, and stops. So a wavelength provisioned by a demo scenario knows which sections it rides and not which hardware would light it. Binding it would mean choosing a free line port at each endpoint, which is an assignment problem this feature did not take on. The transponders are there to bind to, since every PoP carries at least two, so the gap is in the generator and not in the plant. The hop table is the interesting one. Every row shows the running totals at the output of that element: ``` 1 roadm-ber-01 len= 0.000 km loss= 7.000 dB osnr= delay=0.150 us 2 amp-ham-ber-10 len= 0.000 km loss= 7.000 dB osnr=49.500 dB delay=0.250 us 3 span-ham-ber-04 len= 82.500 km loss= 26.650 dB osnr=49.500 dB delay=404.229 us 4 amp-ham-ber-08 len= 82.500 km loss= 26.650 dB osnr=38.029 dB delay=404.329 us ... 11 roadm-ham-01 len=330.000 km loss= 92.600 dB osnr=31.747 dB delay=1616.716 us ... 25 roadm-ams-01 len=800.000 km loss=212.202 dB osnr=27.784 dB delay=3919.026 us ``` OSNR is empty on the first row and only on the first row. There is no noise to measure the signal against until an amplifier has added some. The amplifier names say nothing about direction, and nothing here needs them to. This service runs from Berlin towards Amsterdam, which is the B-to-A direction of both sections it crosses, so every amplifier on the walk comes out of those sections' `amplifiers_b2a` list. The amplifiers in the same huts facing the other way are in the `amplifiers_a2b` list and carry the traffic going back. An amplifier's own page shows the same thing: exactly one of its two section relationships is set. The path's 3923.026 us is four microseconds more than the last hop's 3919.026, which is the FEC encoder and decoder. Propagation is three orders of magnitude larger and dominates everything else on the route. ### Run it again[​](#run-it-again "Direct link to Run it again") ``` uv run invoke demo-provision --branch provision-ber-ams-400g ``` Same route, same mode, same channel, and the object counts do not move. Every object is named from the service name alone, so a second run updates rather than duplicates, and the selector excludes the service's own carrier when it counts occupancy. Without that, the run would find its own channel taken and step to the next one every time. ### Review and merge[​](#review-and-merge "Direct link to Review and merge") Open a proposed change from `provision-ber-ams-400g`. This needs the repository registered, which `invoke init` does and `uv run invoke load-repository` does on its own. The pipeline then runs the fourteen checks, the generator and all three artifact definitions against the change, and reports each one on it. It also needs the branch to exist in Git. The checks, the generator and the artifact definitions are files in this repository, so Infrahub reads them from a Git branch that matches the data branch. `infrahubctl branch create` defaults to `--no-sync-with-git` and `invoke demo-setup` does not override it, so a branch either of them made carries data only. A proposed change from one runs Data Integrity and Schema Integrity and stops there. Create the branch with `uv run infrahubctl branch create --sync-with-git` when you intend to open a proposed change from it, and the count goes from two validators to 22. The generator runs **before** the checks, which is what makes `provisionable` safe. It gates the merge on a verdict the generator derived. That verdict was written seconds earlier against this branch's own data, not read off a refusal that has been sitting on the default branch since some unrelated change put it there. Its diff: ``` OtnOpticalCarrier ADDED 1 oc-svc-ber-ams-400g OtnOpticalPath ADDED 1 path-svc-ber-ams-400g OtnPathHop ADDED 25 OtnContainer ADDED 2 odu-line-oc-svc-ber-ams-400g, odu-svc-ber-ams-400g OtnService ADDED 5 the five requests the setup loaded ``` The plant is not in the diff. `invoke init` loads the schema, the menu and the dataset onto the default branch. A branch cut from it inherits all of that, and the diff contains only what the branch added. That is the five service requests the setup step loaded, and the 29 objects the generator wrote for the one you provisioned. The line container is in the diff even though it joined no tracking group: tracking decides what a re-run may reclaim, not what a branch diff shows. The service it changed is one of the five, which is why the services appear here as added rather than modified: the default branch has none of them. A stack whose default branch holds no plant behaves differently: the setup step loads the dataset onto the branch instead, and the diff then contains the whole network as well. Four of the fourteen checks say something about this change: * `channel_collision` confirms no two carriers hold overlapping spectrum on the same section. It reports how much of the band the busiest one holds. * `osnr_margin` re-evaluates every wavelength on the branch, now 44 rather than 43, and separately sweeps all 21 sections in both directions against one reference mode. It reports the worst margin from each loop. * `container_capacity` walks the container tree and confirms no parent's children commit more tributary slots than it offers. The 400G client here fills its wavelength exactly, 320 of 320, which is the boundary case: exactly full passes, and one slot more would fail. * `channel_count_consistency` compares what each ROADM degree monitor reports against the carriers on the section that degree faces, and **it names four monitors without blocking the merge.** Four degrees sit along this route: `MON-DEG-HAM` at Berlin, `MON-DEG-BER` and `MON-DEG-AMS` at Hamburg, and `MON-DEG-HAM` at Amsterdam. All four still report 0 channels while the two sections they face now carry the wavelength the generator just lit. The paragraph below says why that is a report rather than a refusal. **The fourth one reports and does not refuse, and the clock is why.** Nothing in this repository writes a monitor reading when a wavelength is lit. The generator creates the carrier, the path, the hops and the containers, and it does not touch a port, so the reported count sits one behind the topology on every section the new wavelength crosses. A monitor reading is dated, the shipped ones at `2026-08-26T06:00:00Z`, and no reading can know about a wavelength designed after it was taken. So a monitor reporting **fewer** channels than its section holds is a wavelength designed and not yet turned up, and the check says so and lets the merge through. The direction it does refuse is a monitor reporting **more**: no reading of any age can invent light, so that one is either a stale count or a wavelength removed from the model while the equipment was left alone. What these four findings show is that provisioning is only half automated: the topology moved and the telemetry record has not caught up. The other ten say nothing, for different reasons. `units_import` only asserts that the worker can import the shared package. `monitor_completeness` is silent because this change adds no device, so nothing on the branch newly lacks a monitor. `carrier_termination` finds every one of the 43 shipped wavelengths still terminated at both ends, and it names the new one as skipped rather than judging it: the generator writes a carrier as `planned` and binds no line port, because provisioning allocates spectrum on a route and does not place hardware. The wavelength waits there until somebody turns it up. `diversity` is silent because nobody declared a diversity requirement for this service, and that silence is deliberate rather than an oversight: the check speaks only about promises an operator wrote down. `provisionable` passes and says what it looked at: every service on this branch provisioned, so there is no refusal to gate on. It prints its counts rather than passing quietly. A green check on a branch with no services and a green check on a provisioned network look identical from the proposed change. Only one of them is evidence of anything. The five hardware checks are quiet for one reason between them: **this change racks nothing.** `mux_channel_binding` reads every multiplexer against the client ports it carries, `attenuator_range` holds each variable attenuator to its own maximum, `transceiver_placement` asks whether each pluggable sits in a port with a cage, and `connector_polish` reads the endfaces at both ends of every Raman-pumped span. The generator writes a carrier, a path, its hops and two containers, and it fits no module and touches no span, so all four report on plant that did not move. `transceiver_mode_support` does see the new wavelength and skips it, for the reason `carrier_termination` skips it: the carrier binds no line port, so there is no fitted optic to hold its mode against. That collision check **is** the reservation. Nothing in this model stores occupancy as state, so nothing has to be released when a service is torn down. Two engineers provisioning on two branches each see the same free spectrum, because neither branch can see the other, and they collide at the proposed change. That is the correct place: it is where a human is already looking, and it is the only point at which both intentions exist at once. Two carriers no longer have to share an anchor to collide. A `128 GBd` carrier occupies 150,000 MHz and reaches three grid positions either side of its own centre, so the check compares intervals rather than channel numbers. To see it fire, put a second carrier on channel 3 over `oms-ham-ber` and run: ``` uv run invoke check --name channel_collision --branch provision-ber-ams-400g ``` ``` oc-collision-probe and oc-svc-ber-ams-400g share 29,600 MHz of spectrum on oms-ham-ber, from 191,410,200 to 191,439,800 MHz. oc-collision-probe on channel 3 running DP-16QAM 64GBd 400G (191,410,200 to 191,489,800 MHz, 79,600 MHz wide) against oc-svc-ber-ams-400g on channel 2 running DP-16QAM 64GBd 400G (191,360,200 to 191,439,800 MHz, 79,600 MHz wide). A wavelength holds its width for the whole length of every section it crosses, so only one of these can be provisioned ``` Neither carrier claims the other's channel number. The old check, which compared anchors, called this section clean. ## Scenario two: a corridor out of colour, and what still fits[​](#scenario-two-a-corridor-out-of-colour-and-what-still-fits "Direct link to Scenario two: a corridor out of colour, and what still fits") Frankfurt to Milan carries a distributed training workload. All-reduce collectives stall on round-trip time, so the request states a four millisecond one-way budget: ``` - name: svc-fra-mil-ai-400g description: Frankfurt to Milan, 400G distributed training interconnect. customer: EuroHPC-Leonardo rate_gbps: 400 sla: gold service_profile: ai-training-dci max_latency_ns: 4000000 endpoint_a: rtr-fra-01 endpoint_z: rtr-leonardo-mil-01 ``` The direct route is 780 km and takes 3824.741 us. It fits, with 175.259 us to spare, and every other route between these two sites misses the budget. It is also the busiest corridor in the network. Fill both of its layers: ``` uv run invoke demo-refusal --branch provision-ber-ams-400g ``` `demo/90_fra_mil_saturated.yml` loads three more carriers into the last usable block. That takes `oms-fra-mil` from 4,134,400 MHz occupied to 4,267,600 MHz, and leaves 532,400 MHz in 29 blocks whose widest is 38,000 MHz, narrower than any seeded mode. It then grooms one 100G tenant onto each of the twenty-two wavelengths that run Frankfurt to Milan and nowhere else. Each of those twenty-two is an `ODUC4` line container offering 320 tributary slots, so one 80-slot tenant leaves 240 free. Then it asks twice. ### The 400G is refused, and the reason is slots[​](#the-400g-is-refused-and-the-reason-is-slots "Direct link to The 400G is refused, and the reason is slots") ``` svc-fra-mil-ai-400g: discarded oms-fra-gva|oms-gva-mil [latency] closes on DP-16QAM 64GBd 400G over 990 km, and takes 4853.605 us against a budget of 4000.000 us, which it misses by 853.605 us svc-fra-mil-ai-400g: discarded oms-prg-fra|oms-prg-vie|oms-vie-mil [latency] closes on DP-QPSK 128GBd 400G over 1560 km, and takes 7645.589 us against a budget of 4000.000 us, which it misses by 3645.589 us svc-fra-mil-ai-400g: discarded oms-prg-fra|oms-prg-waw|oms-vie-waw|oms-vie-mil [latency] closes on DP-QPSK 128GBd 400G over 2600 km, and takes 12739.627 us against a budget of 4000.000 us, which it misses by 8739.627 us svc-fra-mil-ai-400g: candidate 1 of 1 is oms-fra-mil on DP-16QAM 64GBd 400G, 1 sections, 780 km, margin +1.894 dB, 3824.741 us, no anchor free, groom only svc-fra-mil-ai-400g: refused, no-slots: neither a direct wavelength nor a chain serves this route. Direct: odu-line-oc-ch047-fra-mil is the tightest of 40 line containers on oms-fra-mil and offers 320 slots with 240 free, and none of the 40 has room for the 320 slots ODUC4 takes, and no wavelength can be lit either, because no anchor puts a DP-16QAM 64GBd 400G carrier, which occupies 79,600 MHz, inside spectrum free on all 1 of its sections: the widest free block is 38,000 MHz. Chain: no pair of wavelengths covers a candidate route end to end and meets at one of the 3 O-E-O devices holding both ``` The service comes back `rejected`, with that sentence stored on it, and nothing else is written. The verdict is two fields rather than one: `rejection_code` is `no-slots`, one of six values the schema enforces, and `rejection_detail` holds the prose. Nothing parses the code back out of the sentence. **This scenario ends in an accepted refusal, so the branch merges.** `demo/90_fra_mil_saturated.yml` sets `refusal_accepted` on `svc-fra-mil-ai-400g`, and `provisionable` then says nothing about it. That is the file signing for the refusal it causes. It took the corridor away, the refusal is the answer it exists to demonstrate, and there is no fix to hold the merge open for. Without that signature the proposed change would go red, which is what [Madrid to Warsaw](/demo-otn/loadable-scenarios.md#two-loadable-scenarios-for-the-regenerator) is for. The signature is a nine-line restatement of a service defined in `demo/00_services.yml`, not a name-and-flag update, because `infrahubctl object load` answers a partial update with `customer is mandatory`. That duplicate can drift, so `tests/unit/test_demo_scenarios.py::test_the_saturated_scenario_restates_the_service_it_signs_for_unchanged` asserts the two blocks agree on every shared field and that the flag is the only thing `90` adds. Losing `max_latency_ns` in the copy would be the quiet failure. Four milliseconds is what rules out the Geneva detour at 4853.605 us, and leaves this corridor as the only route the service has. Note the fourth line. `oms-fra-mil` is not discarded for capacity any more. It comes back as the one surviving candidate, marked `no channel free, groom only`, because a service that grooms into a wavelength somebody already lit consumes no channel. The refusal then names the figure that actually blocks it. A `400GBASE-FR4` maps into an `ODUC4` and needs all 320 slots of a wavelength, and the roomiest wavelength on the corridor has 240. **The refusal closes three escapes, not two.** Grooming is tried first, lighting a new wavelength second, and a chain across an O-E-O device third. The message has to say why each one failed, or a reader is left wondering about the one it skipped. The `Chain:` clause names how many devices were considered. The dataset ships three, `oeo-fra-01` at Frankfurt and the two cross-connects, and none of them terminates a pair of wavelengths that covers this route. A chain would not have helped here anyway, because the route is one section long and the shortage is slots rather than reach. ### The 100G provisions on the same corridor[​](#the-100g-provisions-on-the-same-corridor "Direct link to The 100G provisions on the same corridor") ``` svc-fra-mil-transit-100g: candidate 1 of 1 is oms-fra-mil on DP-QPSK 32GBd 100G, 1 sections, 780 km, margin +12.394 dB, 3824.741 us, no channel free, groom only svc-fra-mil-transit-100g: chose oms-fra-mil on DP-QPSK 32GBd 100G, no anchor free, groom only, 780 km, margin +12.394 dB, 3824.741 us svc-fra-mil-transit-100g: grooms into odu-line-oc-ch047-fra-mil on oc-ch047-fra-mil, the tightest of 40 line containers on oms-fra-mil, offering 320 slots with 240 free, and the ODU4 takes 80 ``` Same corridor, same four millisecond budget, same spectrum that no mode can use, and this one provisions. Its client is a `100GBASE-LR4` in an `ODU4`, 80 slots, and 240 are free. It lights nothing: no `oc-svc-fra-mil-transit-100g` is created, the service's optical path points at `oc-ch047-fra-mil`, and the only object written on the wavelength is the client container nested under `odu-line-oc-ch047-fra-mil`. ### What this pair is for[​](#what-this-pair-is-for "Direct link to What this pair is for") Spectrum and slots run out independently, and a section reported as full is only full at one of the two layers. 532,400 MHz free in 29 blocks whose widest is 38,000 is a statement about colour. It says no new wavelength can be lit, because the narrowest mode in the catalog needs 44,400 MHz contiguous. It says nothing about whether the wavelengths already lit have room inside them. The capacity view answers the first question and the ODU map answers the second, and neither one substitutes for the other. This is also why the refusal has to be engineered rather than observed. A single empty `ODUC4` anywhere on the corridor offers 320 free slots and takes the 400G client. The scenario has to occupy all forty wavelengths before the refusal is reachable at all. Take one tenant out of `demo/90_fra_mil_saturated.yml` and the 400G provisions. ### The controls[​](#the-controls "Direct link to The controls") Before best-fit grooming, this scenario refused the 400G on latency: the direct corridor was rejected for capacity before the ODU layer was consulted, and the Geneva detour at 4853.605 us was the nearest surviving miss. That refusal was a true sentence about a question nobody asked, and it is why `routing.choose_route` now takes `require_free_channel` rather than gating on spectrum unconditionally. What that flag gates has widened with the model. A route has an anchor only when some grid position centres the mode's whole occupied width inside spectrum free on every section it crosses. Latency is still the constraint that decides which routes are candidates at all. Four milliseconds is still the number that separates them: | Budget | Candidate routes | | ------ | -------------------------------------------------------------------------------- | | 4 ms | The direct corridor only. The detour misses by 853.605 us. | | 5 ms | The direct corridor and the Geneva detour, and the direct one wins on hop count. | | None | Every route that closes optically, ranked on hop count then margin. | `tests/unit/test_routing_claims.py` asserts the three latency figures against the committed dataset, and `tests/unit/test_generator.py` asserts the two outcomes above from a forty-container payload, so retuning a span length or changing the packing rule fails a test rather than quietly making this page false. --- # Quick start Go from a fresh clone to a proposed change Infrahub will not merge. You provision a 400G service from Berlin to Amsterdam, then ask the same network for a circuit it cannot carry and watch a check hold the merge shut. Nine and a half minutes of command time, measured on a warm machine with the images already pulled. `invoke init` is eight of them, and the rest is seconds. Registration keeps running for a few minutes after `init` returns, so the proposed change at the end waits on that rather than on you. ## Before you start[​](#before-you-start "Direct link to Before you start") You need Docker with Compose v2, [uv](https://docs.astral.sh/uv/), and about 8 GB of free memory for the stack. [Install and load the demo](/demo-otn/installation-setup.md) carries the full prerequisites, what each step loads and how to verify it. This page runs a subset of those commands and explains less. The first command below runs `invoke init`, which takes the containers down with `-v` and deletes their volumes. Every object in the current stack goes with them, on every branch, and nothing is exported first. Run it on a machine whose Infrahub data you can afford to lose. ## When a step fails[​](#when-a-step-fails "Direct link to When a step fails") Every step here is an invoke task, and the tasks read the address and the API token from `.env` and pass both to whatever they run. [Install and load the demo](/demo-otn/installation-setup.md) is where the failures are covered: the check that says whether the shared package reached the worker, the `Authentication failure` that appears when a tool reads the address from `.env` but not the token, and how to tear the stack down and start over. ## What is deliberately red on the default branch[​](#what-is-deliberately-red-on-the-default-branch "Direct link to What is deliberately red on the default branch") The OSNR check fails on the default branch, and that is the shipped state. Paris to Madrid is 1250 km and falls 0.535 dB short at `DP-16QAM 64GBd 400G`, in both directions. It is a true finding about a long route rather than a broken demo, and you will meet it on every branch cut from the default one, including both branches below. [The link budget](/demo-otn/link-budget.md#paris-to-madrid-does-not-close-and-the-check-says-so) page has the answers an engineer would reach for and what each one costs. ## Start the stack[​](#start-the-stack "Direct link to Start the stack") ``` git clone https://github.com/opsmill/infrahub-demo-otn cd infrahub-demo-otn cp .env.example .env uv sync uv run invoke init ``` `invoke init` destroys any existing stack, starts a new one, loads the schema, the menu and the dataset, and registers this repository. Registration is the step that turns the proposed-change pipeline on, so the checks below have something to run in. Infrahub is then at . [![Berlin's Artifacts tab in Infrahub, listing two artifacts named network-map and odu-map, both with status Ready and content type image/svg+xml, each with a checksum, a storage id and the artifact definition that produced it.](/assets/images/quickstart-map-4ce61584082d86d7ba8d1e920d352d54.png)](/assets/images/quickstart-map-4ce61584082d86d7ba8d1e920d352d54.png) What registration bought you. Every PoP now carries two maps as artifacts, rendered from the graph rather than drawn by hand. This is Berlin's copy. ## Provision Berlin to Amsterdam at 400G[​](#provision-berlin-to-amsterdam-at-400g "Direct link to Provision Berlin to Amsterdam at 400G") ``` uv run invoke demo-provision ``` It does not ask you for a branch. It makes the `demo` branch if it is not there, in Git as well as in the graph, and that second half is what lets a proposed change from it read the checks, the generator and the artifact definitions out of this repository. A branch that exists only in the graph runs the two built-in validators and stops. Nor does it reload the network. An Infrahub branch inherits the data on the branch it forks from, and `invoke init` put the dataset on the default branch, so `demo` has the whole network the moment it exists. What it does load is the five service requests, which are scenario input rather than plant. It says so: ``` - branch demo does not exist yet, creating it in Git and in the graph - the dataset is already there, branch demo is ready - loading the 2 scenario file(s) branch demo needs ``` Each request is intent and nothing else: `svc-ber-ams-400g` asks for 400 Gbps between two routers and names no route, no wavelength and no channel. The generator then runs. It budgets six candidate routes at every transponder mode that reaches 400 Gbps and takes the narrowest mode that closes: ``` svc-ber-ams-400g: chose oms-ham-ber|oms-ams-ham on DP-16QAM 64GBd 400G, channel 2, 800 km, margin +2.284 dB, 3923.026 us ``` That run wrote 29 objects, among them one carrier, one optical path, 25 hops in order and two containers, and moved the service to `active`. [Scenario one](/demo-otn/provisioning-scenarios.md#scenario-one-berlin-to-amsterdam-at-400g) walks all six candidates, including why the winning route anchors on channel 2 rather than channel 1. [![The svc-ber-ams-400g service in Infrahub with status Active, an optical path named path-svc-ber-ams-400g, a container named odu-svc-ber-ams-400g, and both the rejection code and rejection detail fields empty.](/assets/images/quickstart-service-c2da9d2d9beb5024517357e4158a7cac.png)](/assets/images/quickstart-service-c2da9d2d9beb5024517357e4158a7cac.png) The service after the generator ran. It arrived naming no route and now has an optical path and a container, with both rejection fields empty. Open a proposed change from `demo` and the pipeline runs fourteen checks, the generator and three artifact definitions, reporting each on the change. No service on the branch is refused, so `provisionable` passes. `osnr_margin` is still red on the Paris to Madrid deficit, as it is on every branch. That red is the shipped finding above, not anything this branch did. ## The refusal that blocks the merge[​](#the-refusal-that-blocks-the-merge "Direct link to The refusal that blocks the merge") ``` uv run invoke demo-regenerator ``` Madrid to Warsaw is 2970 km over four sections, and no single wavelength closes it on any of the ten modes in the catalog. The task makes `oeo-refused`, loads a file that puts a regenerator at each of the three sites on that route where one could go, and asks for the service. All three splits are refused for optical budget, and one run reports all three. It then runs the check, which reads the refusal off the service: ``` svc-mad-waw-400g cannot be provisioned and was refused for budget: neither a direct wavelength nor a chain serves this route. Direct: oms-par-mad|oms-par-fra|oms-prg-fra|oms-prg-waw closes on no mode: best margin -0.021 dB on DP-QPSK 128GBd 400G. Chain: oc-ch070-mad-par|oeo-par-01|oc-ch070-par-waw does not close: segment 1 -0.535 dB, segment 2 -2.439 dB. It has not been accepted, so this branch does not merge. Fix the network or the request, or set refusal_accepted on the service to keep the refusal on the record ``` Open a proposed change from `oeo-refused` and that sentence is what a reviewer sees. A check that logs an error means the change cannot be merged. `log_error` blocks and `log_info` annotates, and there is no `log_warning` in between to soften a refusal with. The branch merges once somebody fixes the route or sets `refusal_accepted` on the service, which puts the decision on the record rather than passing quietly. `provisionable` is registered with no targets, which makes it global: it judges every service on the branch rather than the one you edited. A service nobody touched can be the one that blocks your merge. Two other checks are red on this branch, and neither of them is the point. `osnr_margin` fails on the Paris to Madrid deficit that sits on every branch, and `channel_collision` fails because the three regenerator wavelength pairs this file loads overlap in spectrum on purpose. The command does not stop there. It goes on to make a second branch, `oeo-closed`, adds one more wavelength pair at a lower modulation order and asks for the same service again. That one provisions as two segments and `provisionable` passes, so a single run shows you the refusal and the fix for it. [Loadable scenarios](/demo-otn/loadable-scenarios.md#two-loadable-scenarios-for-the-regenerator) has the figures for both halves. [![The svc-mad-waw-400g service in Infrahub with status Rejected, a rejection code of Optical budget, the full rejection detail naming both the direct route and the regenerator chain that failed, and Refusal Accepted set to false.](/assets/images/quickstart-refusal-0cfe8db9eb49394489724541a0339e17.png)](/assets/images/quickstart-refusal-0cfe8db9eb49394489724541a0339e17.png) The same fields as the provisioned service, filled in the other direction. The last row is the one that matters: nobody has accepted this refusal, so it blocks. [![The Checks tab of a proposed change in Infrahub, headed Admin wants to merge oeo-refused into main, listing 22 validators. Summary rings across the top read Artifact, Data, Generator, Repository and Schema in green and User in red. Nineteen rows carry a green tick; three carry a red warning triangle: Check: channel\_collision, Check: provisionable and Check: osnr\_margin.](/assets/images/quickstart-blocked-751987f31a32b467353cdaa46171259f.png)](/assets/images/quickstart-blocked-751987f31a32b467353cdaa46171259f.png) The merge held shut. Twenty-two validators ran and three of them said no. `provisionable` is the one this page is about. `osnr_margin` is the shipped Paris to Madrid deficit, and `channel_collision` fires because this scenario overlaps three regenerator pairs in spectrum on purpose. The five hardware checks are all green here: this branch changes a mode and a regenerator, and touches no optic, pad or multiplexer port. ## Clean up[​](#clean-up "Direct link to Clean up") ``` uv run invoke demo-clean ``` With no argument it deletes every branch a scenario task creates, naming each one as it goes and reporting how many of them it found, so nothing is removed without saying so. That is all three branches this page made. The default branch is back to the state it shipped in: the plant, no services, and the OSNR check still failing on Paris to Madrid. Every service you provisioned lived on a branch, which is the point. ## Where to go next[​](#where-to-go-next "Direct link to Where to go next") * [What this demo shows of Infrahub](/demo-otn/what-this-shows.md) lists each Infrahub capability the demo exercises, names the scenario that shows it, and says plainly which capabilities the demo does not touch. * [Provisioning scenarios](/demo-otn/provisioning-scenarios.md) walks the Berlin to Amsterdam run in full, then puts a second service on a corridor that has run out of spectrum and shows why one closes and the other does not. * [Loadable scenarios and the runbook](/demo-otn/loadable-scenarios.md#two-loadable-scenarios-for-the-regenerator) carries the fix for Madrid to Warsaw: the same route closes as two segments once one more wavelength pair arrives at a lower modulation order. --- # Reporting scenarios The six scenarios that read rather than write, and the two maps every PoP carries as artifacts. Every number below is produced by the engine and asserted by a unit test. ## Before you start[​](#before-you-start "Direct link to Before you start") Scenarios three to eight read rather than write, but they do not all read the same branch. Scenarios three and six ask about the plant, so a branch holding the dataset and nothing else is enough. Scenarios four, five, seven and eight ask about services, so they need the provisioning step to have run first: the output below names `svc-ams-mil-ai-400g` and counts five provisioned services. ``` uv run invoke branch-create --name probe uv run invoke load --branch probe # Scenarios four, five, seven and eight need these two as well. uv run invoke demo-setup --branch probe uv run invoke demo-provision-all --branch probe ``` The diagram on the [demo guide](/demo-otn/demo-guide.md) gives the run order and links the other scenario pages. ## Scenario three: what spectrum is left, and what can actually use it[​](#scenario-three-what-spectrum-is-left-and-what-can-actually-use-it "Direct link to Scenario three: what spectrum is left, and what can actually use it") **Run this before provisioning anything.** Occupancy is derived from the carriers on the branch, so the answer moves the moment a service is added, and the published figure belongs to the dataset. ``` uv run invoke demo-capacity --branch probe ``` ``` oms-fra-mil is the busiest section on probe: 4,134,400 of 4,800,000 MHz occupied, 665,600 MHz free in 26 blocks. 16 of 21 sections carry no wavelength at all. section occupied free blocks anchors for another 400G oms-fra-mil 4,134,400 665,600 26 1, channel 95 oms-ams-fra 1,050,000 3,750,000 1 73, channel 23 oms-ber-fra 750,000 4,050,000 2 77, channel 2 oms-par-fra 450,000 4,350,000 2 83, channel 2 oms-vie-mil 133,200 4,666,800 4 89, channel 2 ``` Nothing in the model stores any of that. Occupied is the union of the intervals the carriers crossing a section hold, each interval centred on a carrier's anchor and sized by its mode's symbol rate. Free is the band minus that union, reported as blocks. Both are computed at read time, so both are branch-aware at no extra cost. Thirteen of the twenty-one sections carry no wavelength at all. That is the real shape of this network: one congested corridor and twenty quiet ones. A report that listed only the busy sections would hide it. ### Free spectrum is not capacity[​](#free-spectrum-is-not-capacity "Direct link to Free spectrum is not capacity") The one figure to take away from this scenario is the gap between the two right-hand columns on the first row: ``` Yes, on channel 95. A DP-16QAM 64GBd 400G carrier occupies 79,600 MHz, 665,600 MHz is free on this section in 26 blocks, and 1 of the 96 anchors can take one. Free spectrum divided by width would have said 8. ``` **Eight against one.** Anchors are quantised onto the 50 GHz grid, so a carrier is usable only when its whole width fits inside a single free block with a grid position at its centre. Twenty-five of the twenty-six free blocks on `oms-fra-mil` are narrower than the narrowest mode in the catalog. Twenty-one are 20,400 MHz, two are 5,600, one is 38,000 and one is 35,200, against 44,400 MHz for `DP-QPSK 32GBd 100G`. Only the 152,800 MHz block at the top of the band can take anything at all. Dividing megahertz by width counts fragments no anchor reaches into. [The spectral model](/demo-otn/spectral-model.md) page has the arithmetic. ### Can another 400G fit?[​](#can-another-400g-fit "Direct link to Can another 400G fit?") Per section, yes, once: channel 95 on `oms-fra-mil`. You do not provision onto a section though, you provision onto a route, and a wavelength holds its width on every section it crosses. The same report answers the route-level question: ``` Amsterdam to Milan per-section free: ams-fra 3,750,000, fra-mil 665,600 route free: 665,600 in 26 blocks, 1 anchor, channel 95 Frankfurt to Milan via Geneva per-section free: fra-gva 4,800,000, gva-mil 4,800,000 route free: 4,800,000 in 1 block, 94 anchors, channel 2 ``` A route's free spectrum is the band minus the union of its sections' occupancy, never wider than the narrowest section and usually narrower. On this branch the Amsterdam to Milan figure happens to equal the Frankfurt to Milan one. The seven wavelengths already on `oms-ams-fra` sit between channels 2 and 20, and none of them reaches into the block the congested corridor still has free. **The empty route reports 94 anchors, not 96.** Channels 1 and 96 cannot centre a 79,600 MHz carrier without pushing it past a band edge. That is not a bug in the report. The usable anchor range narrows as the mode widens, and on an empty section a `32 GBd` carrier gets all 96 while every 400G mode gets 2 through 95. ### After provisioning[​](#after-provisioning "Direct link to After provisioning") Run the same command once the five demo services exist and it says: ``` oms-fra-mil is the busiest section on probe: 4,134,400 of 4,800,000 MHz occupied, 665,600 MHz free in 26 blocks. 13 of 21 sections carry no wavelength at all. ``` The congested corridor did not move at all, which is exactly what it reported before provisioning. The five services did change the picture: sixteen empty sections became thirteen, because `svc-ber-ams-400g` and `svc-fra-gva-hpc-400g` lit spectrum on three sections that were dark. The two services that cross the corridor, `svc-ams-mil-ai-400g` and `svc-fra-mil-ai-400g`, groomed into wavelengths that were already lit and took tributary slots inside them. Neither added a carrier to `oms-fra-mil`, and neither consumed a megahertz. That is what the ODU layer changes about this report. Provisioning a service no longer necessarily costs spectrum, and whether it does is decided one layer down. If a wavelength on the route has slot room for the container, the service rides it and the occupied figure stands still. So the occupied megahertz no longer tells the two runs apart on this corridor. The empty-section count does, and the capacity scenario still runs first so that you have the before figure to read the after one against. `invoke demo` runs `demo-capacity` before `demo-provision-all`, so a full demo run never reaches the state above on its own; only a manual re-run gets there. That is also why the acceptance walk over this page did not notice these two figures going stale when grooming arrived. ## Scenario four: cut the Frankfurt to Amsterdam fiber[​](#scenario-four-cut-the-frankfurt-to-amsterdam-fiber "Direct link to Scenario four: cut the Frankfurt to Amsterdam fiber") A digger goes through the duct between Amsterdam and Frankfurt. What is down? ``` uv run invoke demo-impact --branch probe ``` ``` Cutting oms-ams-fra drops 7 wavelength(s), 2.8 Tbps of transport, 1 recorded service(s) and 1 customer(s). plant roadm-ams-01 to roadm-fra-01, 470.000 km, 6 spans, 14 amplifiers (7 towards Frankfurt, 7 towards Amsterdam) customers EuroHPC-Leonardo routers rtr-ams-01, rtr-leonardo-mil-01 signals 400GBASE-FR4 x1 ``` When you triage an outage, you need the latency-sensitive rows before the transit ones. **Latency-sensitive services first:** ``` svc-ams-mil-ai-400g EuroHPC-Leonardo gold channel 2 oc-ch002-ams-mil 150,000 MHz, 191.32500 to 191.47500 THz 6127.051 us against a 7000.000 us budget rtr-ams-01 [1/1/1, 1/1/2] to rtr-leonardo-mil-01 [1/1/1, 1/1/2] ``` **Then the six wavelengths with nothing behind them:** ``` unattached: channels 5, 8, 11, 14, 17, 20, each 150,000 MHz wide These wavelengths carry no service object. They are real spectrum and a real loss; the model has no customer recorded behind them. ``` Six of the seven wavelengths on this section are spectrum the dataset ships with no `OtnService` behind them. A report that dropped them for having a null path would have said "one service affected" where the true answer is seven wavelengths, 1,050,000 MHz of C-band and 2.8 terabits. Every row carries the width and the two edges beside the anchor. A restoration plan needs to know how much spectrum to find somewhere else, and a channel number does not say. Seven carriers at 150,000 MHz each is 1,050,000 MHz to re-home, which is more than a fifth of the band on whatever section takes them. Provisioning grooms rather than lights. Two of the four services `demo-provision-all` places, `svc-ams-mil-ai-400g` and `svc-fra-mil-ai-400g`, find a wavelength that already has tributary slot room and take slots in it instead of lighting one of their own. So `svc-ams-mil-ai-400g` rides channel 2, a wavelength the dataset already ships, and channel 2 leaves the unattached list because it finally has a customer behind it. That is the intended behaviour: an operator fills the wavelengths already lit before paying for spectrum, so the model does too. ### The duct, not the section[​](#the-duct-not-the-section "Direct link to The duct, not the section") A cut in a duct is a cut in every fiber in it, and the section is only what an operator names: ``` cd-ams-southeast GEANT span-ams-fra-01 also carries: (nothing else) cd-fra-north GEANT span-ams-fra-06 also carries: oms-par-fra (no conduit) 4 spans of this section are outside any recorded conduit ``` `cd-fra-north` also carries Paris to Frankfurt. A digger in that trench takes both corridors, and the section-level view would never have said so. ### The typo that must not return green[​](#the-typo-that-must-not-return-green "Direct link to The typo that must not return green") ``` uv run invoke demo-impact --branch probe --section oms-ams-frs ``` ``` ValueError: no optical multiplex section matched the `section` variable on this branch. An empty result and a name that does not exist are different answers, and this is the second one ``` Thirteen sections in this dataset carry nothing, so "no impact" is a common and correct answer. An operator who mistypes a name and is told "no impact" stops looking. A section that exists and carries nothing reports zero: ``` $ uv run invoke demo-impact --branch probe --section oms-ams-bru Cutting oms-ams-bru drops 0 wavelength(s), 0 Gbps of transport, 0 recorded service(s) and 0 customer(s). ``` ## Scenario five: trace a service end to end[​](#scenario-five-trace-a-service-end-to-end "Direct link to Scenario five: trace a service end to end") ``` uv run invoke demo-trace --branch probe ``` ``` svc-ams-mil-ai-400g EuroHPC-Leonardo ai-training-dci active budget 7000.000 us endpoint_a rtr-ams-01 Amsterdam client ports 1/1/1, 1/1/2 endpoint_z rtr-leonardo-mil-01 Milan client ports 1/1/1, 1/1/2 path 1250.000 km, 318.399 dB, OSNR 25.287 dB, margin +5.287 dB 6127.051 us total, of which 6120.901 us is propagation and 6.150 us is everything else carrier oc-ch002-ams-mil, channel 2, 191.40000 THz, DP-QPSK 128GBd 400G occupying 150,000 MHz from 191.32500 to 191.47500 THz, SD-FEC at 4.000 us sections oms-ams-fra (roadm-ams-01 to roadm-fra-01) oms-fra-mil (roadm-fra-01 to roadm-mil-01) container odu-svc-ams-mil-ai-400g, ODUC4, GMP, 400GBASE-FR4 (ethernet) conduits cd-ams-southeast, cd-fra-north, cd-fra-south, cd-mil-northeast 35 hops: 1 OtnRoadm roadm-ams-01 0.000 km 7.000 dB 0.150 us 2 OtnAmplifier amp-ams-fra-01 0.000 km 7.000 dB 0.250 us 3 OtnFiberSpan span-ams-fra-01 78.334 km 25.767 dB 383.830 us cd-ams-southeast 4 OtnAmplifier amp-ams-fra-03 78.334 km 25.767 dB 383.930 us ... 34 OtnAmplifier amp-fra-mil-19 1250.000 km 311.399 dB 6122.901 us 35 OtnRoadm roadm-mil-01 1250.000 km 318.399 dB 6123.051 us ``` Router to glass to router, in order. The running totals appear at each element, with the duct each span is buried in. The last hop reads 6123.051 us and the path reads 6127.051. The four microsecond difference is the forward error correction encoder and decoder, which sit in the transponder rather than on the route. **What the trace will not tell you.** Which of `rtr-ams-01`'s two client ports carries this service. The dataset wires transponder line ports to ROADM add and drop ports and stops there; there is no recorded adjacency between a router port and a transponder client port. The trace lists every client-role port on each endpoint and says the assignment is not modelled, rather than picking one of two candidates and presenting the guess as a fact. ## Scenario six: where do the cheap pluggables reach?[​](#scenario-six-where-do-the-cheap-pluggables-reach "Direct link to Scenario six: where do the cheap pluggables reach?") 400ZR and 800ZR are cheap. A planner wants to know where they can replace a transponder. ``` uv run invoke demo-reach --branch probe ``` ``` 400ZR, 800ZR reach nothing on this network. The shortest optical multiplex section is oms-ams-bru at 220.000 km, and every one of these parts is rated 120.000 km. 400ZR pluggable 120.000 km 0/21 800ZR pluggable 120.000 km 0/21 DP-64QAM 64GBd 600G transponder 400.000 km 7/21 DP-16QAM 64GBd 400G transponder 1000.000 km 20/21 OpenZR+ 400G pluggable 1000.000 km 20/21 DP-16QAM 32GBd 200G transponder 1500.000 km 21/21 OpenZR+ 300G pluggable 2000.000 km 21/21 DP-QPSK 128GBd 400G transponder 2500.000 km 21/21 DP-QPSK 32GBd 100G transponder 3000.000 km 21/21 OpenZR+ 200G pluggable 3000.000 km 21/21 ``` Zero of twenty-one. The shortest section in the entire topology is 100 km beyond what either part reaches, and the next shortest are 320 km and 330 km. The finding does not depend on the exact catalog figure. That is the answer a planner wants before raising a purchase order, and it is only available because reach is modelled data rather than an assumption. The same table shows what does work: `OpenZR+ 400G` covers twenty of twenty-one sections and misses only Paris to Madrid at 1250 km, and three of the loaded wavelengths run it router to router. **Read the split as cFEC against oFEC rather than pluggable against transponder.** Every mode marked `pluggable` above is a coherent optic in a router cage. The two that reach nothing carry cFEC; the three that reach almost everything carry oFEC, and `OpenZR+ 200G` reaches as far as the 100G transponder does. [The link budget](/demo-otn/link-budget.md) has what happens when a 400ZR is put on a section it cannot have: the OSNR gate passes and the dispersion gate refuses it. One caveat applies to every row: reach is not the budget. A mode in reach of a section can still miss the OSNR margin over that section's amplifier chains, and `budget_report` is what computes that. A section has two chains and two margins, one per direction of travel, and the reports show the worse of the two. ## Scenario seven: which services are not diverse?[​](#scenario-seven-which-services-are-not-diverse "Direct link to Scenario seven: which services are not diverse?") ``` uv run invoke demo-srlg --branch probe ``` ``` 5 non-diverse pair(s) among 5 provisioned service(s) on probe, 5 of them between two latency-sensitive services. cd-fra-south 3 svc-ams-mil-ai-400g, svc-fra-gva-hpc-400g, svc-fra-mil-ai-400g cd-mil-northeast 3 svc-ams-mil-ai-400g, svc-fra-mil-ai-400g, svc-vie-mil-hpc-400g cd-ams-northeast 1 svc-ber-ams-400g cd-ams-southeast 1 svc-ams-mil-ai-400g cd-ber-north 1 svc-ber-ams-400g cd-fra-north 1 svc-ams-mil-ai-400g high svc-ams-mil-ai-400g and svc-fra-gva-hpc-400g share cd-fra-south. One cut in that duct takes both. high svc-fra-mil-ai-400g and svc-vie-mil-hpc-400g share cd-mil-northeast. One cut in that duct takes both. ... not exposed: svc-ber-ams-400g ``` The instructive pair is the first one. Frankfurt to Geneva and Amsterdam to Milan share no section, no ROADM and no city pair. On a route map they look diverse. They both leave Frankfurt through `cd-fra-south`, and one backhoe takes both. That is why the exposure is read off the spans and not off the sections. A section can cross several conduits and a conduit can hold spans from several sections, so intersecting section lists would miss this pair entirely. `svc-ber-ams-400g` is in no pair at all, which is the control: the report pairs services that share a duct rather than pairing everything it can. Diversity here is reported, not provisioned. Protection switching is out of scope, and an operator may have accepted this exposure deliberately, which is why this is a report and not a check that blocks a merge. **A declared requirement is the other half, and that one does block.** This report finds every pair sharing a duct, whether anybody asked for diversity or not. `checks/diversity.py` looks at the opposite set: only the pairs an operator put in an `OtnDiversityGroup`, where somebody wrote the promise down. It fails the proposed change when that promise does not hold, and says nothing at all about the pairs above, so the exposure this report lists stays accepted. The two loadable scenarios below show both outcomes side by side. ## Scenario eight: the AI services against their budgets[​](#scenario-eight-the-ai-services-against-their-budgets "Direct link to Scenario eight: the AI services against their budgets") ``` uv run invoke demo-latency --branch probe ``` ``` 4 latency-sensitive service(s) on probe. None over budget. The tightest is svc-fra-mil-ai-400g at +175.259 us. 1 provisioned service was excluded because its profile carries no latency budget. svc-fra-mil-ai-400g ai-training-dci 780 km 3824.741 us propagation 3819.441 electronics 5.300 (0.14%) budget 4000.000 +175.259 svc-ams-mil-ai-400g ai-training-dci 1250 km 6127.051 us propagation 6120.901 electronics 6.150 (0.10%) budget 7000.000 +872.949 svc-vie-mil-hpc-400g hpc-research 800 km 3922.680 us propagation 3917.380 electronics 5.300 (0.14%) budget 5000.000 +1077.320 svc-fra-gva-hpc-400g hpc-research 590 km 2894.165 us propagation 2889.065 electronics 5.100 (0.18%) budget 4000.000 +1105.835 ``` The electronics column is the negative result. ROADMs at 150 ns, amplifiers at 100 ns and forward error correction at 4 µs come to between 0.10 and 0.18 percent of the total on every one of these routes. Propagation at 4897 ns per kilometre is the rest. There is no FEC-against-reach trade-off to exploit on these four services: each one already sits on the route that is both OSNR-optimal and latency-optimal, so nothing was left on the table. That is not a general rule about continental distances. Span loss enters the OSNR cascade exponentially while route length enters latency linearly, which is how Copenhagen beats Prague on margin while being 110 km longer, as [scenario two](/demo-otn/provisioning-scenarios.md) shows. Modelling FEC latency was still worth doing, because it is what let the model rule the trade-off out here instead of assuming it. The excluded service is `svc-ber-ams-400g`, which is `ip-transit` and has no budget. The report says how many it dropped rather than silently showing four of five rows. See [AI and HPC payloads](/demo-otn/ai-payloads.md) for the arithmetic and the tension that is real, which is capacity against latency. ## The two maps every PoP carries[​](#the-two-maps-every-pop-carries "Direct link to The two maps every PoP carries") The scenarios above answer one question each. The maps answer the first question an audience asks, which is what the network looks like. Open **OTN Sites** in the Infrahub sidebar, pick a PoP, and open its **Artifacts** tab. Two maps hang there and both draw the fourteen PoPs and the 21 sections between them, from the graph on the branch you are reading. Each PoP gets its own copy of each, with that site and the routes terminating on it drawn heavier than the rest. A copy reads as that site's reach into the core. **`network-map`, titled "European optical core".** Colour is the OSNR margin a 400G DP-16QAM carrier has on a route: whether a wavelength closes on it. Three things on it are worth pointing at before anyone asks: * **The red route.** Paris to Madrid, the one section a 400G DP-16QAM carrier cannot cross. The same finding as the failing check in the note under [before you start](/demo-otn/provisioning-scenarios.md#before-you-start), drawn rather than printed. * **The chip on Frankfurt to Milan.** `4,134 GHz` of the 4,800 the C-band gives it, which is the corridor scenarios two and three are about. * **The dashes on Vienna to Milan.** The Raman-pumped section, good one way and thin the other. **`odu-map`, titled "ODU capacity and grooming".** Colour is the largest container that still fits on the roomiest lit carrier of a route: whether another circuit fits inside the wavelengths already there. The same geography, a different layer, and the two can disagree about the same route without either being wrong. On the default branch this map is mostly grey, and that is the picture rather than a broken render. Thirteen of the twenty-one sections carry no wavelength at all, so they have no slot figure, and grey says "not known" rather than "empty and available". The eight that do carry wavelengths are all green, because every pre-provisioned carrier ships with an empty line container on it. Infrahub renders these artifacts from this repository, so they exist wherever the repository is registered. The [network map](/demo-otn/network-map.md) and [ODU map](/demo-otn/odu-map.md) pages explain every colour, dot and ring on each drawing. --- # Schema reference The OTN schema is 50 kinds across eight files in `schemas/`. Eight of them are generics and 42 are concrete nodes. Four of those files define the equipment layer: the generics, the port kinds, the device kinds, and the site. They are the subject here. The plant, container, carrier and service layers have their own pages: [optical plant concepts](/demo-otn/concepts.md), [client mapping](/demo-otn/client-mapping.md) and [link budget](/demo-otn/link-budget.md). What follows is what the YAML cannot state for itself: why the generic layer is shaped the way it is, and why every physical quantity is an integer. ## The files[​](#the-files "Direct link to The files") | File | Holds | | ------------------------- | ------------------------------------------------------------------------------------- | | `schemas/otn_base.yml` | The five OTN generics | | `schemas/otn_ports.yml` | The port kinds, the monitors and the pluggable optics: two generics and sixteen nodes | | `schemas/otn_devices.yml` | The ten device kinds | | `schemas/location.yml` | `LocationGeneric`, vendored, `OtnSite` and `OtnFacility` | Infrahub loads every file in the directory into one schema, so the split has no runtime meaning. It is how the source reads and diffs. ## Two generics, not one chain[​](#two-generics-not-one-chain "Direct link to Two generics, not one chain") A device and an optical element are different things, and the schema keeps them apart on purpose. * **`OtnGenericDevice`** is anything racked at a site. It has a name, a status, a role, a site, and ports. * **`OtnOpticalElement`** is anything light passes through and loses power in. It has an insertion loss, a vendor, a model, and an element class. Ten device kinds inherit `OtnGenericDevice`. Nine of those ten also inherit `OtnOpticalElement`, and one kind inherits `OtnOpticalElement` alone. There is no edge between the two generics: A single inheritance chain, "optical element is a kind of device", would look tidier and would be wrong in both directions. **It would be wrong about the router.** Light terminates at a router. A router has no insertion loss, so a query for everything that attenuates light must return nine kinds, not ten. Put the router under a combined chain and the optical budget starts counting a loss that does not exist. **It would be wrong about the fiber span.** `OtnFiberSpan` contributes the largest loss in the network and is not a device. It is not racked, it has no ports, and it has no site, so it inherits `OtnOpticalElement` alone. A chain that made every optical element a device would force the span to grow a rack position and a port list it can never have. So the two questions get one query each: * "everything racked here" is a query against `OtnGenericDevice`. * "everything the budget must sum" is a query against `OtnOpticalElement`. `OtnOpticalElement` has no relationships and no identity keys, for the same reason. The span inherits it on its own. ## The generic layer[​](#the-generic-layer "Direct link to The generic layer") None of the five OTN generics inherits from another. Infrahub's generic schema has no `inherit_from` key, so a generic taxonomy cannot be stacked. It is composed instead: a concrete node lists the generics it needs. | Generic | Contributes | | ------------------- | ----------------------------------------------------------------------------------------------- | | `OtnGenericPort` | `name`, `role`, `enabled`, `admin_state`, `oper_state`, the parent `device`, and `connected_to` | | `OtnGenericDevice` | `name`, `status`, `role`, the `site`, and the `ports` list | | `OtnOpticalElement` | `insertion_loss_mdb`, `vendor`, `model`, `element_class` | | `OtnOpticalPort` | `center_frequency_mhz`, `tx_power_mdbm`, `rx_sensitivity_mdbm`, `connector_type`, `polish` | | `OtnCopperPort` | `speed_kbps`, `impedance_ohm`, `connector_type` | Composition is the whole design. `OtnLinePort` is `OtnGenericPort` plus `OtnOpticalPort`. `OtnTributaryPort` is `OtnGenericPort` plus `OtnCopperPort`. `OtnRouter` is `OtnGenericDevice` and nothing else. One inherited attribute is restated, and only to default it. `element_class` is determined by the kind: an amplifier is an `amplifier`, a span is a `fiber_span`. Each of the ten concrete kinds that inherit `OtnOpticalElement` declares it with the matching `default_value`, so no object file writes it. That restatement has to repeat the ten choices, because overriding an inherited Dropdown requires the full list, and the server only half enforces that. Omitting the `choices` key is rejected at load with `The property 'choices' is required for kind=Dropdown`. A `choices` key holding nine of the ten loads in silence, and the divergence appears only when an object tries to write the value the override left out. `tests/unit/test_schema_contract.py` asserts every override matches the generic's list, and it is the only thing that does. Adding a choice means editing eleven blocks, the generic and its ten kinds, and the test is what tells you one was missed. `connector_type` is restated the same way, and for a reason that is about history rather than defaults. It became a `Dropdown` on `OtnOpticalPort` after six kinds already inherited it as `Text`, and Infrahub refuses to change an attribute's kind underneath them: ``` OtnAmplifierPort.connector_type inherited from OtnOpticalPort must be the same kind ["Dropdown", "Text"] ``` So those six restate the full nine-choice list. The two mux port kinds arrived after the change and need no restatement. `OtnCopperPort.connector_type` stays `Text`, and no node inherits both port generics, so the two never meet. Nothing else is restated. Restating an attribute would let it drift from the generic, and the drift would break the guarantee that one query against `OtnGenericPort` returns every port with the same shape. ## Ports[​](#ports "Direct link to Ports") Nine kinds, eight optical and one copper. | Kind | Composes | What it is | | --------------------- | ----------------- | --------------------------------------------------- | | `OtnRouterPort` | generic + optical | Grey optics on an IP router | | `OtnClientPort` | generic + optical | Transponder client side | | `OtnLinePort` | generic + optical | DWDM line side, on a transponder or a router | | `OtnRoadmAddDropPort` | generic + optical | Local add and drop on a ROADM | | `OtnRoadmDegreePort` | generic + optical | Line-facing degree, one per direction | | `OtnAmplifierPort` | generic + optical | Amplifier input or output | | `OtnMuxClientPort` | generic + optical | One channel of a multiplexer, facing what lights it | | `OtnMuxLinePort` | generic + optical | The common side of a multiplexer | | `OtnTributaryPort` | generic + copper | E1 or T1 G.703 electrical tributary | There is no management or console port kind. This model covers the transmission path, and a management port is not on it. The two mux port kinds declare nothing but their channel binding. `OtnMuxClientPort` carries an optional `dwdm_channel` and an optional `cwdm_channel`, because the dense and coarse plans are two kinds and the schema has no way to say "exactly one of these two". Getting that right is `mux_channel_binding`'s job, and it owns both the neither case and the both case. `OtnMuxLinePort` binds nothing at all: the common side carries every channel the device lights, so naming one would be wrong. Neither mux port kind has an insertion loss of its own. The loss belongs to the multiplexer, through `OtnOpticalElement`, and that is where the budget engine reads it. A second per-port figure would give one loss two sources and invite double counting. Real AWG loss does vary across the passband, worst at the edge channels, and modelling that needs a passband this demo does not have. Two behaviours matter before you load data. **`connected_to` is one edge, not two.** It is declared once on `OtnGenericPort`, peers `OtnGenericPort`, and pins `identifier: otn_port__connected_to`. Set it from A to B and B reports A with no second write. The explicit identifier is what stops Infrahub deriving a different string per side and splitting the link into two one-way halves. The identifier is immutable once loaded. **Port names are unique per device across all nine kinds.** The constraint `["device", "name__value"]` is declared on the generic, and Infrahub enforces it over every kind that inherits it. Creating `OtnRouterPort` `dup/1` and then `OtnTributaryPort` `dup/1` on the same device fails on the second write. ### A line port names the wavelength it terminates[​](#a-line-port-names-the-wavelength-it-terminates "Direct link to A line port names the wavelength it terminates") `OtnLinePort.carrier` peers `OtnOpticalCarrier`, cardinality one and optional. `OtnOpticalCarrier.line_ports` peers back, cardinality many and optional. Both sides pin `identifier: otn_carrier__line_ports` by hand, for the same reason `connected_to` pins its own: the string is frozen the moment a load succeeds, and changing it afterwards costs a remove-and-re-add on both peers. Many on the carrier side, because a wavelength is terminated at each of its two ends by two ports on two devices at two sites. Optional on the port side, because a line port with no colour on it is a legal state: 40 of the 126 in the dataset are exactly that. **A line port is not only a transponder's.** Six of the 126 sit on routers, and each holds a pluggable coherent optic rather than a laser on a line card. The kind is the same either way, because the port does the same thing: it terminates one wavelength on the DWDM grid. What differs is where the optics come from, and that is a separate record. See [pluggable optics](#pluggable-optics-are-a-part-and-a-unit) below. Before this edge existed, the only thing tying a transponder to a wavelength was that both names carried the same site code. That is a naming convention doing a relationship's job, and it cannot say which of a site's transponders carries which of its wavelengths. Both sides are `kind: Attribute` with `on_delete: no-action`, written out rather than left to the default. **Deleting a transponder does not delete the wavelengths it carried.** It does delete its ports: `OtnGenericDevice.ports` is `kind: Component` with `on_delete: cascade`, because a port has no meaning apart from the device it sits in. The deletion stops at the port. A wavelength has two ends, so no single port owns it, and cascading from Milan would delete the wavelength out from under the Frankfurt transponder that is still installed and still patched to its ROADM. A carrier also legally exists with no line ports at all, which is what a freshly provisioned one is, and a peer whose existence does not depend on the relationship cannot be cascaded by it. The other direction is plainer still: cascading from the carrier would mean retiring a service deletes hardware from the inventory. What that leaves behind is the point. A lit wavelength nothing terminates is a real fault, and `no-action` leaves it sitting there, visible and open to a query, rather than making it vanish along with the transponder whose deletion caused it. **The objection, because it gets asked every time: light has direction, so should the edge?** No, and each half of the answer is separate. A line port is a transceiver, carrying the transmit and the receive side of the same wavelength, so one edge already covers both. A cardinality-one cascade would fire from either end, which gives a wavelength two owners rather than one. And `OtnOpticalCarrier` holds no direction at all: direction lives on the plant, in the per-direction amplifier lists on a section, and in the path traversal that walks them. The model already has a way to say the light stopped, which is `oper_state` on the port and `status` on the carrier, not deletion. ## Pluggable optics are a part and a unit[​](#pluggable-optics-are-a-part-and-a-unit "Direct link to Pluggable optics are a part and a unit") A coherent pluggable is two records. `OtnTransceiverType` is the part number and `OtnTransceiver` is the module with a serial on it. | Kind | Holds | | -------------------- | --------------------------------------------------------------------------------------- | | `OtnTransceiverType` | `part_number`, `vendor`, `description`, `form_factor`, `tunable`, and `supported_modes` | | `OtnTransceiver` | `serial`, `status`, the `type` it is, and the `port` it sits in | **What an optic can do is a property of the part, not of the unit.** A `QDD-400G-ZR` runs one mode and a `QDD-400G-ZRP` runs three, and that is true of every unit ever built to those numbers. Putting `supported_modes` on the unit would copy the same list onto every module of the same part and let two of them disagree. It also has to be a relationship rather than a list of strings: the modes are `OtnOpticalMode` objects with the OSNR, reach and dispersion figures the budget engine reads, and a string cannot be budgeted. The catalog ships five parts. One 400ZR, one 800ZR, and three that carry all three OpenZR+ rungs in a QSFP-DD, an OSFP and a CFP2-DCO cage. **Neither kind inherits anything.** A part number is not racked and light does not pass through it, so it is neither an `OtnGenericDevice` nor an `OtnOpticalElement`. The module's loss belongs to the port it lights, and the port already has `tx_power_mdbm` and `rx_sensitivity_mdbm`. The type says what optic is fitted; the port says what it is doing. **A port holds at most one module, and that is the schema saying so rather than a check.** `OtnLinePort`, `OtnClientPort` and `OtnRouterPort` each declare a cardinality-one `transceiver` on the same `otn_optical_port__transceiver` identifier `OtnTransceiver.port` uses, so both ends of the edge are cardinality one and the server refuses the second write: ``` Node 18d19871-0d7f-6758-306b-1188b66bb9eb has 2 peers for otn_optical_port__transceiver, maximum of 1 allowed ``` The reverse edge is declared on those three kinds rather than on `OtnOpticalPort`, because the generic would hand the field to the five port kinds that hold no module. A uniqueness constraint on `port` was tried first and Infrahub accepts one only where the relationship is mandatory, and a mandatory port leaves a spare, an RMA and a decommissioned unit with nowhere to sit. What is left for `transceiver_placement` is the port kind alone, and the reason is a limit worth naming: `OtnTransceiver.port` peers the `OtnOpticalPort` generic, and a relationship to a generic cannot be filtered by the kind of its peers. The schema can offer the field or withhold it. It cannot say "any of these three kinds and none of the other five". **Negative result: nothing here says the cage fits.** A part carries a `form_factor` and a port does not, so "a QSFP-DD module in a QSFP-DD cage" is a comparison this model cannot make. ## Amplification is per direction[​](#amplification-is-per-direction "Direct link to Amplification is per direction") Light crosses an optical multiplex section both ways, and an amplifier restores power in one of them. The section says which, by holding two lists: | Relationship | Kind | What it holds | | ---------------- | -------------- | ----------------------------------------------------- | | `amplifiers_a2b` | Many, optional | The chain running towards the section's own `roadm_b` | | `amplifiers_b2a` | Many, optional | The chain running towards its `roadm_a` | An amplifier has no direction attribute. Which chain it is in is the relationship holding it, and an attribute restating that would be a second copy that can disagree with the first. Read from the amplifier's own page, the same fact is `oms_a2b` or `oms_b2a`: exactly one of the two is set, and which one it is answers the question. That costs a relationship. One relationship cannot be the inverse of two identifiers, so the amplifier needs both. The alternative was giving the amplifier no section relationship at all and reaching the section by a server-side filter. That was rejected: an amplifier page is a page an operator lands on, and "which section and which way" is what it is for. Both lists stay optional. A section is creatable before its amplifiers exist, and that reason has not changed. | Attribute | Kind | What it holds | | -------------- | -------------------------- | --------------------------------------------------------------------------------- | | `oms_sequence` | Number, mandatory, 1 to 51 | Position in this amplifier's own chain, counting along the direction it amplifies | A section with N spans holds N+1 amplifiers per chain. `oms_sequence` does two jobs and both are its own. It orders a chain the relationship has already identified, so a chain sorted on it is in traversal order and nothing ever reverses one. And it fixes which member of that chain is which: position 1 is the booster, position N+1 is the pre-amplifier, and amplifier k feeds span k of its own walk. A span's `oms_sequence` does the other thing: a span has no direction, so it is numbered from the A end and stays there. **`oms_sequence` deliberately has no default.** A default of 1 would let a new amplifier take a silent position at the head of its chain. It would collide with the real first amplifier and sort stably into a plausible wrong answer instead of raising. It is not unique and cannot be. A uniqueness constraint cannot reference an optional relationship, and both section relationships have to stay optional. Rejecting duplicate positions is a test's job. Nothing on the server enforces how many amplifiers a section holds in either direction. A query that selects one list and forgets the other gets an empty chain rather than an error. The budget engine is what refuses it: its rule that a section of N spans has N+1 amplifiers each way fails for that direction and names the section. Without that rule the split would have traded a crash for a plausible wrong number. ## Amplifier names encode nothing a query needs[​](#amplifier-names-encode-nothing-a-query-needs "Direct link to Amplifier names encode nothing a query needs") An amplifier is `amp---`, where the two slugs are the section's own sites and `NN` is a position. A section of N spans has N+1 amplifier huts counting from the A end, each holding two amplifiers, and hut k takes ordinals 2k+1 and 2k+2. Frankfurt to Milan is nine spans, so `amp-fra-mil-01` is the booster at Frankfurt and `amp-fra-mil-20` is the booster at Milan. The name is an identifier: unique, stable, and encoding nothing a query needs. Everything it used to say is readable from the schema. Which chain an amplifier is in is the relationship holding it. Where it sits in that chain is `oms_sequence`. Whether it is a booster, an inline or a pre-amplifier is the `role` on its own IN and OUT ports, which reads `booster`, `line` and `preamp`. Any deterministic naming rule over an ordered set can be inverted. A reader who learns that ordinals run two per hut can work out which of `amp-fra-mil-01` and `amp-fra-mil-02` serves which chain. That is unavoidable and it is not the point. The point is that nothing reads it, so nobody has to know the rule to ask a question. ## Raman pumps[​](#raman-pumps "Direct link to Raman pumps") `OtnRamanPump` is a pump laser that injects Raman gain into one fiber span. It inherits `OtnGenericDevice` and `OtnOpticalElement`, like every other optical device kind, and adds four attributes of its own: | Attribute | Kind | What it holds | | --------------------- | ------------------- | ------------------------------------------------------------------------- | | `on_off_gain_mdb` | Number, 0 to 15000 | Received signal power with the pump on, less the same with it off | | `on_off_gain_display` | Text, read-only | The same figure in dB | | `injection_end` | Dropdown, mandatory | `site_a` or `site_b`, which end of its span the pump is spliced in at | | `propagation` | Dropdown, mandatory | `counter` if the laser fires against the signal, `co` if it fires with it | `span` points at the `OtnFiberSpan` the pump serves and is mandatory. Its inverse, `raman_pumps`, is declared on the span under the same identifier, and it is load-bearing twice. A budget walks a section, reaches its spans, and reads what is on them. Without that side of the edge nothing on the walk can see a pump. Every pumped span would sum to zero gain, and the check would pass while reporting a loss the network does not have. The inverse is also what makes the kind reachable in the interface, since the sidebar names no device kind and a kind nothing points at cannot be opened at all. The ceiling on `on_off_gain_mdb` is 15.0 dB rather than 20.0 dB. The smallest fiber loss on any shipped span is 16.167 dB, so one pump can never reduce a span to zero loss. It takes two on one span to reach the floor. ### `terminating_ports`, and why the span had to gain an edge[​](#terminating_ports-and-why-the-span-had-to-gain-an-edge "Direct link to terminating_ports-and-why-the-span-had-to-gain-an-edge") `OtnFiberSpan.terminating_ports` points at the ports at the two ends of a span. It peers `OtnGenericPort`, is optional and many, and the dataset generator fills it for every span. It exists because the polish check could not be written without it. Before it, no port kind had any relationship to a span or an optical multiplex section in either direction. The nearest reachable ports were three hops away, through the section and its ROADMs, and a ROADM carries one degree port per direction whose `connected_to` is unset. The only thing that named the right one was the far site shortname inside the port's own name string. Parsing that string would have put a naming convention back in a relationship's job, which this repository has already been bitten by once. The edge is declared on the span alone, for the same reason the transceiver's port edge is one-sided: the inverse would give the field to every port kind, and most ports terminate no span. A span whose `terminating_ports` is empty is reported as unjudgeable by `connector_polish`, never as clean. An empty relationship and a correct span look identical from outside, and a green result over an empty traversal would be worse than no check. A test over the generated data holds the other side of that trust: every port in a span's `terminating_ports` belongs to a device at that span's own `site_a` or `site_b`. ### The direction a pump amplifies is worked out, not stored[​](#the-direction-a-pump-amplifies-is-worked-out-not-stored "Direct link to The direction a pump amplifies is worked out, not stored") A pump holds two facts about the hardware: which end of the span it is spliced in at, and whether its laser fires with the signal or against it. There is no third attribute naming the direction of travel it amplifies. That is the answer, and the two facts above compute it: ``` amplifies A to B when (injection_end == site_a) == (propagation == co) ``` Read physically, a counter-propagating pump fires back up the fibre from the far end. One at the B end therefore amplifies the A to B signal. A co-propagating pump fires along with the signal from the near end, so one at the A end amplifies A to B too. Those are two ways of reaching the same answer. There are four cases in all. | `injection_end` | `propagation` | Amplifies | | --------------- | ------------- | --------- | | `site_b` | `counter` | A to B | | `site_a` | `co` | A to B | | `site_a` | `counter` | B to A | | `site_b` | `co` | B to A | Every pump in this network is counter-propagating, so only the first and third rows occur here. The table shows all four anyway, because a reader who sees only the shipped cases will assume counter-propagating is the only kind there is. Storing the conclusion beside its two premises would be a third value that can contradict them. A counter-propagating pump recorded at the A end and marked `a_to_b` is a self-contradictory object, and a schema that accepts one will eventually hold one. `injection_end` is mandatory and has a default of `site_b`. The default is a loading device, because a mandatory attribute needs one for the schema to load at all, and it is not a statement about this network. Every pump in the dataset writes the value explicitly, and a test says so. What the pump does to a budget, and where that treatment stops being predictive, is on the [link budget](/demo-otn/link-budget.md) page. ## Two attenuator kinds, not one with a type field[​](#two-attenuator-kinds-not-one-with-a-type-field "Direct link to Two attenuator kinds, not one with a type field") An attenuator takes power out of a link on purpose, to keep a receiver inside its own input range. There are two kinds of them here. | Kind | Holds | What it is | | ----------------------- | ---------------------------------------- | ----------------------------------------------------------- | | `OtnFixedAttenuator` | `attenuation_mdb` | A pad. A value set in a factory and readable off the label | | `OtnVariableAttenuator` | `attenuation_mdb`, `max_attenuation_mdb` | A VOA. A setting, and the top of the range it can be set to | Both inherit `OtnGenericDevice` and `OtnOpticalElement`, so both are racked at a site and both are on the budget's list. Neither declares any ports: they inherit the `ports` relationship and hold none, and they reach a path the way a fiber span does, through `OtnPathHop.element`. **The obvious alternative is one kind with a `variable` flag on it, and it is worse.** A pad has no range. On one kind with a flag, `max_attenuation_mdb` has to be optional, and then the schema accepts a VOA with no maximum and a pad that claims one. Both are impossible objects, and the only thing that would notice either is a check reporting after the fact. With two kinds the field is on the kind that has it, mandatory, and the server refuses the write. That is the whole argument, and it is the schema-first rule in one sentence: a wrong value is refused at write time rather than reported afterwards. `attenuator_range` is left holding what a schema genuinely cannot carry, which is a VOA whose setting is above its own maximum. That comparison is between two attributes of one object, and Infrahub's `max_value` takes a constant rather than another field. The cost is that `attenuation_display` is declared twice, once on each kind, from the same source and the same template. It has to be byte-identical, and `tests/unit/test_schema_contract.py` is what holds the two copies together. ## Regenerators and ODU cross-connects[​](#regenerators-and-odu-cross-connects "Direct link to Regenerators and ODU cross-connects") `OtnOduSwitch` is the eighth device kind. It receives one wavelength, reframes the payload and originates the next, which is what lets a circuit cross a route too long for any single wavelength. Two flat generics composed on the concrete node, the same shape `OtnMuxDemux` uses. | Attribute | What it says | | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `switching_mode` | `regenerator` carries the whole payload across without looking inside it. `cross_connect` demultiplexes to containers and regroups them, which is what lets the two segments carry different clients. | | `framing_latency_ns` | What the junction charges for reframing. Nanoseconds as an integer, the unit `fec_latency_ns` and `latency_ns` already use. | | `carriers` | The wavelengths patched to this shelf. Many, optional. Attachment, not termination: a regenerator does terminate the two it joins and a cross-connect grooms containers behind a transponder and terminates nothing, so one edge cannot mean both. `OtnLinePort.carrier` is what answers "what terminates this wavelength". | It inherits `OtnOpticalElement` because `OtnPathHop.element` peers that generic, and a segment's route has to be able to name the device it terminates on. **A regenerator has line-side optics, and they are where termination is read from.** Two `OtnLinePort` components on the shelf, one facing each segment it joins, each naming its wavelength through `OtnLinePort.carrier`. A regenerated circuit then answers the same question a direct one does: every wavelength has a port at each of its two ends, a transponder on the outside and the regenerator on the inside. `checks/carrier_termination.py` reads those ports and nothing else, which is why the answer is the same relationship in both cases. A cross-connect has none, and the difference is the reason the two modes are separate values rather than a flag. It sits at the electrical layer behind a transponder that has already taken the light off the fibre, so it has no line-side optics to give. `oxc-mil-01` is patched to 37 wavelengths that all terminate on Milan transponders; reading that edge as termination would make every one of them terminated three times. **Its insertion loss belongs to one side only.** `insertion_loss_mdb` arrives with the generic and applies to the **incoming** segment. The device terminates the light rather than passing it through, so the outgoing segment starts at a transmitter and not at an attenuated signal. That asymmetry is why the loss cannot be added into one total spanning both segments. It is the schema-level reason a regenerated circuit has a margin per segment rather than one figure. **`carriers` is not what finds the chain, and that was measured.** The relationship says which wavelengths a device terminates, and `src/infrahub_demo_otn/chains.py` reads it to evaluate a junction. The two wavelengths must meet at a site, that site must host the device, and the device must terminate both. A device-to-device traversal over this edge alone returns zero paths, because a ROADM has no edge to a carrier. Widening the filter until one is reachable brings the intended chain back among 65 paths. Of those, 48 are two carriers meeting on a shared section with no device between them at all, and removing this edge from the filter changes none of that. The graph does not constrain the junction, so the predicate is explicit in Python and the cover is unit tested against a pinned expectation. **A device with an empty `carriers` contributes no junction.** That is the correct reading of a racked but unpatched device, and it needs no special case. ### The three the dataset ships[​](#the-three-the-dataset-ships "Direct link to The three the dataset ships") | Device | Site | Mode | Framing delay | | ------------ | --------- | --------------- | ------------- | | `oeo-fra-01` | Frankfurt | `regenerator` | 3000 ns | | `oxc-fra-01` | Frankfurt | `cross_connect` | 5000 ns | | `oxc-mil-01` | Milan | `cross_connect` | 5000 ns | The two sites were measured rather than chosen. Every one of the 40 shipped wavelengths crosses `oms-fra-mil`, so 37 of them terminate at Milan and 25 at Frankfurt, and no third site reaches double figures. The regenerator is at Frankfurt because that is the only split of Madrid to Warsaw that closes, which the [link budget](/demo-otn/link-budget.md#madrid-to-warsaw-where-one-regenerator-is-not-enough) page has in full. Each of the three names at least one wavelength. An inert device would load cleanly, never be a junction, and read as a capability the demo does not have. `element_class` gained a tenth choice, `odu_switch`, when the first object forced the decision. None of the original nine names an O-E-O device. The alternative was defaulting to `transponder` or `roadm`, which puts a false value on the one attribute that says what a device is. Nothing in the repository reads `element_class`, so a wrong label costs nothing mechanically and is only untrue in the data, which is the whole argument for paying for the tenth choice. The choice cost nine blocks: the generic, the seven overrides, and the pinned list in `tests/unit/test_schema_contract.py`. A block missed is a dropdown that offers different options depending on the kind you look at, and the server accepts that in silence. ## Devices and sites[​](#devices-and-sites "Direct link to Devices and sites") `OtnGenericDevice.site` points at `OtnSite` and is optional. It is an `Attribute` relationship rather than a `Parent` one, because a `Parent` relationship is mandatory and a device has to be creatable before its site record exists. `OtnSite` inherits `LocationGeneric`, vendored from the published `opsmill/schema-library` location file. Only the generic is taken. The upstream file also defines a hosting node that peers a competing device hierarchy, which this repository does not use. `OtnSite` adds `latitude_microdeg` and `longitude_microdeg`. The inverse `devices` relationship is declared directly on the `Site` node in `schemas/location.yml`, peering `OtnGenericDevice` under the identifier `otn_site__devices`. The location layer therefore names the OTN device layer, and the two files load as one payload for that reason. ### `OtnFacility` is an edge, and used to be a prefix on a tag name[​](#otnfacility-is-an-edge-and-used-to-be-a-prefix-on-a-tag-name "Direct link to otnfacility-is-an-edge-and-used-to-be-a-prefix-on-a-tag-name") `OtnFacility` records a supercomputing facility hosted at a PoP. Six of the fourteen PoPs host one; the other eight host none, which is why both sides of `otn_site__facility` are optional. Cardinality is one on both sides, because no site in the modelled network hosts two. Widening that later is a migration a reader can see coming; narrowing it is not. Until it existed, the only record that a site hosted a facility was the text after `eurohpc-` in a `BuiltinTag` name, sliced back out by a helper duplicated in `transforms/network_map.py` and `transforms/odu_map.py`. That defence for it was half right: it read a tag, not a device, and nothing in this repository recovers an amplifier, a ROADM or a router from a name. The half that was wrong is what moved it. **The failure was silent in both directions**, and both were measured on a throwaway branch rather than argued. Renaming `eurohpc-vega` to `hpc-vega` dropped Vega from both maps and raised nothing anywhere. Attaching a mistyped `eurohpc_deucalion` to Geneva created a facility that no map draws and no check reports. `name` is the tag suffix and has to stay it: `mapchrome.py` upper-cases the value for the caption on the node disc, and the committed reference render holds the upper-cased suffix. So `marenostrum-5`, not `MareNostrum 5`. The readable form lives in `description`, which is the thing a tag could never carry. The six `eurohpc-` tags are still on their sites. They are data an operator wrote, and nothing reads them back. ### `site_type` keeps "the PoPs" a filter[​](#site_type-keeps-the-pops-a-filter "Direct link to site_type-keeps-the-pops-a-filter") `OtnSite.site_type` is a mandatory dropdown with two values, `pop` and `customer`, defaulting to `pop`. The loaded network has fifteen sites: fourteen PoPs and one customer campus, Amsterdam Science Park, reached over an 18.4 km coarse tail. The attribute exists so that "the PoPs" is a query rather than a number in a paragraph. Before it, every statement about the core said fourteen, and the first site that was not a PoP made all of them wrong at once. A report that means the core now filters on `site_type: pop` and stays correct through the next customer endpoint. It also does one job nothing else can. The guard in `tests/unit/test_geant_dataset.py` reads `customer` as its discriminator and refuses an `oms` on any span touching such a site. That refusal is what keeps a coarse tail out of the optical budget engine, which would price it at a 1550 nm attenuation coefficient. See [optical plant concepts](/demo-otn/concepts.md) for the size of the error. ### `OtnCwdmChannel` and `cwdm_channels`[​](#otncwdmchannel-and-cwdm_channels "Direct link to otncwdmchannel-and-cwdm_channels") `OtnCwdmChannel` is the ITU-T G.694.2 coarse wavelength plan: eighteen entries at 20 nm spacing from 1271 to 1611 nm, with two attributes and no more. The standard defines a coarse channel by its nominal central wavelength and gives it nothing else, so there is no name, no channel number and no frequency to store. Its `human_friendly_id` is the wavelength, and the display label renders `1471 nm`. `band` is stored rather than computed, for the same reason a port's centre frequency is stored. "Which coarse wavelengths could an erbium amplifier reach" is then a GraphQL filter instead of a Python loop over eighteen rows. The redundancy is covered by a guard test that recomputes all eighteen wavelengths and all eighteen bands from `units.py`, so the file cannot drift from the code. `OtnCwdmChannel` inherits nothing. Light does not pass through a plan entry, it passes through the multiplexer that selects one, so the multiplexer is the optical element and the plan entry is reference data. `OtnMuxDemux.cwdm_channels` is the edge between them, optional and many. It says which coarse wavelengths a multiplexer lights, which is a property of the device rather than of the plan. The network loads sixteen multiplexers and only the two on the coarse tail have a non-empty list. The relationship is also what makes `OtnCwdmChannel` reachable from another kind, which the sidebar reachability test in `tests/unit/test_menu.py` requires. ### One rule for the one-sided relationships[​](#one-rule-for-the-one-sided-relationships "Direct link to One rule for the one-sided relationships") Several relationships in the model point one way. One rule decides whether each gets an inverse, written once in the header of `schemas/otn_plant.yml`: > Declare the inverse when the peer is a page an operator lands on and the reverse list is what that page is for. Otherwise the reverse direction is a server-side filter on the forward side, which costs nothing and does not duplicate the fact. Six qualify and have one, each declared on the node it belongs to. A site reads the fiber spans terminating on it; a ROADM reads the sections it terminates; a channel reads the carriers holding it and a mode reads the carriers using it. `OtnService.endpoint_a` and `endpoint_z` do not, and the reason is structural. Their peer is a generic, so the inverse would have to sit on `OtnGenericDevice` in the base file and make the base layer depend on the service layer. Every inverse reuses the identifier its forward side already declares. That is what makes it read the edges already in the graph instead of an empty list, with no migration. An inverse under a new identifier reads nothing and looks broken. ## Every quantity is a scaled integer[​](#every-quantity-is-a-scaled-integer "Direct link to Every quantity is a scaled integer") Infrahub has no `Float` attribute kind. `Number` is an integer, and `JSON` is not filterable, not sortable, and not usable in a computed attribute. So every physical quantity is stored as an integer scaled by a fixed factor, with the unit in the attribute name. | Suffix | Unit stored | Scale | Example value | Means | | ----------- | --------------------- | ------------------- | ------------- | ----------------- | | `_mdb` | millidecibel | 1 dB = 1000 | `250` | 0.25 dB | | `_mdbm` | milli-dBm | 1 dBm = 1000 | `-28000` | -28.0 dBm | | `_mhz` | megahertz | 1 THz = 1000000 | `193700000` | 193.7 THz | | `_kbps` | kilobit per second | 1 Mbps = 1000 | `2048` | E1, 2.048 Mbps | | `_ohm` | ohm | none, already whole | `120` | 120 ohms | | `_microdeg` | millionth of a degree | 1 degree = 1000000 | `52379189` | 52.379189 degrees | The rate suffix is `_kbps` and not `_mbps` for a reason. E1 is 2.048 Mbps and T1 is 1.544 Mbps. Rounded to whole megabits both become 2, and the two signals the copper generic exists to carry stop being distinguishable. Every scaled attribute declares both a minimum and a maximum. A value outside the range is rejected at write time with an HTTP 422, so a bad number never reaches the budget engine: ``` 99000 is higher than the maximum allowed value 30000 at tx_power_mdbm 185000000 is lower than the minimum allowed value 191350000 at center_frequency_mhz ``` `src/infrahub_demo_otn/units.py` holds every scale factor and every conversion. No scale factor may be written inline anywhere else. ### Display attributes[​](#display-attributes "Direct link to Display attributes") A scaled attribute that appears in the user interface has a paired read-only `_display` attribute that renders engineering units. Four of them are on the generics, so every kind composing those generics gets them. | Attribute | Display attribute | Renders | | ---------------------- | -------------------------- | ----------- | | `insertion_loss_mdb` | `insertion_loss_display` | `0.25 dB` | | `center_frequency_mhz` | `center_frequency_display` | `193.7 THz` | | `tx_power_mdbm` | `tx_power_display` | `1.0 dBm` | | `rx_sensitivity_mdbm` | `rx_sensitivity_display` | `-28.0 dBm` | Five more names sit on the device kinds that need them: `noise_figure_display` and `gain_display` on `OtnAmplifier`, `on_off_gain_display` on `OtnRamanPump`, `attenuation_display` on both attenuator kinds, and `max_attenuation_display` on the variable one alone. A display name declared on more than one kind has to render the same way on each, so the source attribute and the Jinja2 template are byte-identical wherever the name repeats. `tests/unit/test_schema_contract.py` asserts that, and it is the only thing that would catch a template copied by hand and then edited on one side. Read-only here is stronger than greyed out. A `_display` attribute is absent from the generated `...UpdateInput` type. Writing to one is a GraphQL schema error rather than a permission error: ``` Field 'tx_power_display' is not defined by type 'OtnLinePortUpdateInput' ``` Scaled attributes that do not appear in the interface get no display attribute. ### The one inline divisor[​](#the-one-inline-divisor "Direct link to The one inline divisor") Each display attribute is a Jinja2 template, and the template engine cannot import Python. A template therefore has to include its own divisor: ``` {% if tx_power_mdbm__value is not none %}{{ tx_power_mdbm__value / 1000 }} dBm{% endif %} ``` That is the single exception to the "no inline scale factors" rule, and it is machine-checked. `tests/unit/test_schema_contract.py` extracts the divisor from every display template and asserts it equals the matching constant in `units.py`. A template that drifts from the module fails the unit suite, which is what bounds the exception. ## Monitors[​](#monitors "Direct link to Monitors") A monitor is the interface a device reports its own last-known readings on. There are five kinds, one per family of equipment, and each declares the readings its hardware can take: | Kind | Readings | | ----------------------- | ------------------------------------------------------------------------------------------------- | | `OtnAmplifierMonitor` | `input_power_mdbm`, `output_power_mdbm`, `measured_gain_mdb`, `tilt_mdb` | | `OtnRoadmDegreeMonitor` | `total_power_mdbm`, `channel_count` | | `OtnMuxDemuxMonitor` | `total_power_mdbm`, `channel_count` | | `OtnRamanMonitor` | `pump_power_mdbm`, `measured_gain_mdb`, `back_reflection_mdb` | | `OtnReceiverMonitor` | `rx_power_mdbm`, `measured_osnr_mdb`, `pre_fec_ber_ppb`, `q_factor_mdb`, `cd_fs_per_nm`, `dgd_fs` | Five kinds rather than one, because the readings barely overlap. Of the fourteen readings in the model, eleven appear on exactly one family. The names suggest more sharing than there is: `input_power_mdbm`, `total_power_mdbm`, `pump_power_mdbm` and `rx_power_mdbm` are four different measurements that happen to have "power" in four different names. Three readings are shared, and only in two pairs: `measured_gain_mdb` between an amplifier and a Raman pump, and `total_power_mdbm` and `channel_count` between a ROADM degree and a multiplexer. At that separation the schema can say the whole thing itself. Every reading is mandatory, so the server refuses a monitor that is missing one. And a kind has no field at all for a reading its hardware cannot produce. The server refuses that too: ``` measured_osnr_mdb is not a valid attribute or relationship for OtnAmplifierMonitor tilt_mdb is mandatory ``` Both halves, on every write path, rather than inside a proposed change. A coherent receiver's signal processor computes OSNR; an amplifier measures power. An amplifier monitor reporting OSNR would be a number the device cannot take, and now there is nowhere to put it. ### Two shared generics[​](#two-shared-generics "Direct link to Two shared generics") `OtnMonitor` declares `measured_at` and nothing else. It is the one fact every monitor shares, and it is mandatory, because a reading nobody can age is not evidence. This is a last known value, not a telemetry feed. `OtnChannelMonitor` declares `total_power_mdbm` and `channel_count`. A ROADM degree and a multiplexer report exactly those two, so they are declared once rather than twice. The kinds stay separate, because a degree and a multiplexer are different equipment and "show me the multiplexer monitors" should stay a question about a kind. It also leaves room for the two to diverge later, which is what a real degree monitor and a real multiplexer monitor eventually do, without a migration on loaded data. Neither generic inherits the other. Generics do not inherit generics in Infrahub, so both are flat and each concrete kind composes them alongside `OtnGenericPort`, which is where identity, state and the parent device come from. `OtnAmplifierPort` already uses that shape for two generics; the channel monitors use it for three. A bit error rate is held in parts per billion so it stays an integer: 2.1e-3 is 2100000. That follows the same rule as every other quantity here. ### What the schema still cannot say[​](#what-the-schema-still-cannot-say "Direct link to What the schema still cannot say") Nothing stops a receiver monitor from being attached to an amplifier. `OtnGenericPort.device` peers `OtnGenericDevice`, and a concrete kind cannot narrow the peer of a relationship it inherits. Infrahub 1.11.0 rejects the attempt from either end, before any data is loaded: ``` ProbeAmpPort's relationship device inherited from ProbeGenericPort must have the same peer (ProbeGenericDevice != ProbeAmp) ``` The remaining option would be a Python check, which is the layer these five kinds exist to leave. So the gap is accepted and stated rather than hidden. Nothing in the shipped network creates a mismatched pair, and the generator that writes the monitors is the only thing that ever creates one. ## The six reason codes and the signature beside them[​](#the-six-reason-codes-and-the-signature-beside-them "Direct link to The six reason codes and the signature beside them") A refused service carries its verdict in three attributes on `OtnService`, and none of them is a parsed string. `rejection_code` is a `Dropdown`, `rejection_detail` is `Text` capped at 512 characters, and `refusal_accepted` is a `Boolean`. Together they replace one free-text field that held `"{code}: {detail}"` and that two transforms had to split apart, `transforms/service_trace.py` and `transforms/ai_latency.py`. | Code | Label | Colour | What it means | | ---------- | ------------------ | --------------- | ----------------------------------------------------------------------- | | `no-route` | No route | grey `#9e9e9e` | No path exists between the two endpoints. | | `no-mode` | No mode | grey `#757575` | No transponder mode meets the requested rate. | | `budget` | Optical budget | red `#f44336` | A route exists and the OSNR margin is negative on all of them. | | `latency` | Latency budget | red `#b71c1c` | A route exists and every one of them is slower than the service allows. | | `capacity` | No spectrum | amber `#ff9800` | A route exists and no channel on it is free. | | `no-slots` | No tributary slots | amber `#ffb74d` | A wavelength was found and its containers have no room for the client. | **The colour grouping is information, not decoration.** The two greys mean the request was unanswerable, the two reds mean the physics refused a route that exists, and the two amber codes mean the network is full. An operator reading a list of refusals learns from the colour alone whether to re-plan the request, to regenerate the circuit, or to build more network. A string field carried none of that. **Negative result: only two of the six appear in practice today.** Every refusal the demo produces is `no-slots` or `budget`. The other four are reachable in principle, and no shipped scenario reaches them. `capacity` needs a route whose spectrum is full while its containers are not, and the saturated corridor closes both layers at once. `no-route` and `no-mode` need a request the topology and the mode catalog cannot answer at all, and both are answerable for every pair of sites in the dataset. They are in the schema because the generator writes them, not because a demo scenario shows them. **The choices are the Python constants, and a test says so.** All six live in `routing.py`, including `no-slots`, whose only writer is `generators/optical_service.py`. `tests/unit/test_schema_contract.py` asserts the six schema names equal the six constants, because a code added to one and not the other fails at write time on a live branch and nowhere earlier. Two neighbouring constants in `routing.py`, `CHANNEL_NO_SPECTRUM` and `CHANNEL_NO_BLOCK`, look like reason codes and are detail text. They are deliberately not choices. **`refusal_accepted` is `optional: false` with `default_value: false`**, and that pairing is what made it safe to add to a kind that already had rows. Mandatory with no default fails validation against every existing service and blocks the whole schema update. The default lands `false` on all of them, which is the correct reading of every service written before the flag existed. Only a person ever sets it to `true`. The generator may clear it, and only on the path where it provisions the service. See [the gate and the escape hatch](/demo-otn/concepts.md#a-refusal-is-an-answer-and-it-does-not-merge-unless-somebody-signed-for-it) for why both halves of that rule are load-bearing. The free-text `rejection_reason` these three replaced was deleted from the YAML outright rather than retired with `state: absent`. The dataset here is generated and reloads in one command, so the graph catches up on `uv run invoke init` and a retired declaration would be pure reading cost. `state: absent` is the right answer for a graph that has to survive the change, and this one does not. **So an instance loaded before this change needs `uv run invoke init` re-run, and that is not optional.** A deleted declaration is not a migration. The attribute stays on every service already in the graph until the schema is loaded again, and until then the three attributes above do not exist there at all. A check reading `status` and a transform reading `rejection_code` are both querying fields the running instance has never heard of. Reloading is the whole migration, and it is one command because the dataset is generated. ## Two deliberate absences[​](#two-deliberate-absences "Direct link to Two deliberate absences") Both are load-bearing. **A router contributes no insertion loss.** `OtnRouter` inherits `OtnGenericDevice` only. It has no `insertion_loss_mdb`, and it is not returned by a query against `OtnOpticalElement`. Light terminates at the router, so there is nothing to attenuate. Adding the generic to make the device list look uniform would break the query the optical budget depends on. **A grey router port has no centre frequency.** `center_frequency_mhz` is optional because a grey port is not on the C-band grid at all. When it is unset, `center_frequency_display` renders an empty string. It does not raise inside the template engine, and it does not block the save. An empty frequency on a router port is the correct reading, not missing data. ## Menu and navigation[​](#menu-and-navigation "Direct link to Menu and navigation") `menus/otn.yml` defines the sidebar. Five top-level entries and fifteen leaves: | Entry | What is under it | | ------------ | ---------------------------------------------------------------------------------------- | | OTN Sites | The fourteen PoPs and the customer campus. A leaf, because it is where you start | | OTN Services | Services, optical carriers | | OTN Plant | Devices, fiber spans, optical multiplex sections, conduits, transceivers | | OTN Catalog | Frequency grid, CWDM plan, optical modes, fiber types, transceiver types, client signals | | OTN Reports | Service traces | **Both wavelength plans are in the catalog.** The dense grid and the coarse plan are two entries side by side, so an engineer opening the catalog finds both. The alternative is reaching a wavelength plan by first opening a multiplexer, which is a route nobody guesses. Devices opens on the generic kind, so one click gives all ten device kinds. The sidebar names no port kind and no individual device kind, because the object pages already reach them. A site page lists the devices at the site; a device page lists its ports; an optical path lists its hops; a carrier lists its containers. An entry for "Patch panels" is a second route to a place you were already going to arrive at. **`OtnTransceiver` is the one exception, and it has to be.** A module is fitted into a port through a one-sided edge, so no port page lists it, and a spare on a shelf is fitted into nothing at all. Without its own entry the optics inventory is unreachable. `OtnTransceiverType` sits in the catalog beside the fiber types for a softer reason: a unit reaches its own part in one click, and the catalog is also read on its own, before any unit exists, to see which modes a part can run. **A site page lists the fiber terminating on it in two lists,** one for each end: "Fiber spans (A end)" and "Fiber spans (B end)". Frankfurt reads 16 and 24, and the 40 spans that touch it are the union. `site_a` and `site_b` are two relationships under two identifiers, so their inverses are two lists; merging them would lose the end each span terminates on. Every generic and node in this schema sets `include_in_menu: false`. Custom menus and the auto-generated per-kind sidebar are additive and do not deduplicate, so without it each kind would appear twice. A menu load is additive in the same way. `uv run invoke load-menu` deletes the existing OTN menu items before loading, so the file is the sidebar rather than an addition to it. Loading with `infrahubctl` directly gives a server the union of the file and whatever it had, which makes a shorter file produce a longer sidebar. ## Loading and checking the schema[​](#loading-and-checking-the-schema "Direct link to Loading and checking the schema") ``` uv run invoke schema-check ``` The offline gate. It checks schema formatting and needs no running Infrahub. There is no schema-validate subcommand. ``` uv run invoke load-schema --branch otn-schema ``` Loads `schemas/` onto a branch. Into a deployment that already holds objects, load onto a branch rather than onto `main`, so the change arrives through a proposed change with a diff to read. ``` uv run invoke test-unit ``` Runs the contract tests that guard the rules a formatter cannot see: no `Float`, no `JSON`, and both bounds on every scaled number. It also guards a unit suffix on every quantity, every display attribute present and read-only, and no generic inheriting from another generic. --- # Spectral model A wavelength occupies a width of spectrum, not a channel number. It is centred on the frequency its anchor names, and it holds that width for the whole length of every section it crosses. Everything on this page follows from that one sentence, including the parts of it that are uncomfortable. The demo used to count anchors. Under that model the Frankfurt to Milan corridor read "96 channels, 71 occupied, 25 free", and all three numbers were wrong. At their real widths those 71 carriers need 7,306,000 MHz on a section that has 4,800,000 MHz, 52 percent oversubscribed. The collision check reports 91 overlapping pairs against that plan. The shipped dataset now holds 40 carriers and fits. ## The width a carrier occupies[​](#the-width-a-carrier-occupies "Direct link to The width a carrier occupies") ``` occupied_width_mhz = round(baud_mbaud x (1 + 0.1)) + 9,200 ``` Integer arithmetic, in megahertz, in `src/infrahub_demo_otn/units.py`, which is the only file in the repository allowed to hold a scale factor. Every consumer calls `occupied_width_mhz` and none of them recomputes it. | Mode | Symbol rate | Occupied width | | ---------------------------------------------- | ----------- | -------------- | | `DP-QPSK 32GBd 100G`, `DP-16QAM 32GBd 200G` | `32 GBd` | 44,400 MHz | | `400ZR` | `59.84 GBd` | 75,024 MHz | | `OpenZR+ 200G`, `OpenZR+ 300G`, `OpenZR+ 400G` | `60.14 GBd` | 75,354 MHz | | `DP-16QAM 64GBd 400G`, `DP-64QAM 64GBd 600G` | `64 GBd` | 79,600 MHz | | `800ZR` | `118 GBd` | 139,000 MHz | | `DP-QPSK 128GBd 400G` | `128 GBd` | 150,000 MHz | **Roll-off 0.1** is the root-raised-cosine shaping factor coherent DWDM transponders use, and it is the figure OpenROADM and vendor literature quote for modern flexible-rate line cards. ### What one carrier takes out of the grid[​](#what-one-carrier-takes-out-of-the-grid "Direct link to What one carrier takes out of the grid") A carrier is named by one channel number and occupies several channels' worth of spectrum. `DP-QPSK 128GBd 400G` anchored on channel 50 runs from 193,725,000 to 193,875,000 MHz, which is exactly the slots of channels 49, 50 and 51. Ask where a `DP-16QAM 64GBd 400G` carrier may anchor beside it, and the answer is not "anywhere except 50". Channels 48 and 52 are two grid positions clear of the anchor and still overlap it. One carrier removes five anchors, and a check comparing channel numbers would see none of that. `checks/channel_collision.py` compares intervals for this reason. ### The guard band is fitted to one anchor, and it misses the second[​](#the-guard-band-is-fitted-to-one-anchor-and-it-misses-the-second "Direct link to The guard band is fitted to one anchor, and it misses the second") **9.2 GHz has no clean citation and this page is not going to pretend otherwise.** It was chosen because it lands `128 GBd` on exactly 150.0 GHz, which is a published flexgrid media-channel width. That is one anchor. Checked against a second published width, the model comes off worse. A real deployment allocates 87.5 GHz to a `64 GBd` carrier. This model gives it 79.6 GHz, **7.9 GHz under**. So the model is slightly optimistic at `64 GBd`, and since 22 of the 40 shipped carriers ride a `64 GBd` mode, the optimism is not a corner case. A deployment planner reading a section as 86.1 percent full here would find it fuller in the field. ### What moves if the guard band moves[​](#what-moves-if-the-guard-band-moves "Direct link to What moves if the guard band moves") | Guard band | `128 GBd` width | Consequence | | ---------- | --------------- | ------------------------------------------------------- | | 9.2 GHz | 150.0 GHz | Lands exactly on the published media-channel width | | 10 GHz | 150.8 GHz | Over it, so a 150 GHz media channel no longer holds one | **`128 GBd` sits precisely on a boundary.** At 9.2 GHz it is 150.000 GHz, not 149.9 and not 150.1, so any change to the guard in either direction changes what fits on a section. The guard band is the most sensitive constant in this feature and the least defensible, which is why it is written out here rather than left in a docstring. ## The band, edge to edge[​](#the-band-edge-to-edge "Direct link to The band, edge to edge") | Figure | Value | What it measures | | ---------------- | ------------- | --------------------------------------------------------------- | | Centre to centre | 4,750,000 MHz | Channel 1's centre to channel 96's centre, 191.35 to 196.10 THz | | Edge to edge | 4,800,000 MHz | 191.325 to 196.125 THz, which is 96 slots of 50 GHz | Width semantics need the second figure. Using 4,750,000 where 4,800,000 belongs silently loses one channel of capacity, and the loss looks like a rounding error rather than a bug. `units.py` names both and says which is which. The `OtnOpticalPort.center_frequency_mhz` bounds of 191,350,000 to 196,100,000 are centre-based and stay correct for a port's centre frequency. No schema change was needed. ## Anchors are quantised, and that is the whole finding[​](#anchors-are-quantised-and-that-is-the-whole-finding "Direct link to Anchors are quantised, and that is the whole finding") A carrier's centre may only sit on one of the 96 grid positions. A carrier is therefore usable only when its **whole width** fits inside a **single** free block with a grid position at that block's centre. Free megahertz divided by carrier width counts fragments no anchor can reach into. On the shipped dataset, `oms-fra-mil` measures: | Figure | Value | | -------------------------------------- | ---------------------------------------- | | Occupied | 4,134,400 MHz of 4,800,000, 86.1 percent | | Free | 665,600 MHz | | Free blocks | 26 | | Anchors that can take another 400G | **1**, channel 95 | | Free spectrum divided by carrier width | 8 | **Eight against one.** That gap is the single most important number this model produces. Counting free spectrum overstates capacity whenever anchors are quantised onto a grid, and the overstatement grows as the plan fragments. The left branch is the answer a spreadsheet gives. The right branch is the answer the model gives. Two steps decide it, and the spreadsheet skips both. A free block has to be wide enough for the whole carrier, and it has to have a grid position far enough from its own edges to centre that carrier on. ### What does not fit, and why[​](#what-does-not-fit-and-why "Direct link to What does not fit, and why") The 665,600 MHz free on that corridor breaks down like this: | Block width | How many | Fits | | ----------- | -------- | ------------------------- | | 152,800 MHz | 1 | Every mode in the catalog | | 38,000 MHz | 1 | Nothing | | 35,200 MHz | 1 | Nothing | | 20,400 MHz | 21 | Nothing | | 5,600 MHz | 2 | Nothing | **Twenty-five of the twenty-six blocks fit no mode at all.** The narrowest mode in the catalog is `DP-QPSK 32GBd 100G` at 44,400 MHz, and the widest block below the top one is 38,000. Naming that is the point of the table: a free-megahertz total hides fragmentation, and fragmentation is a capacity finding in its own right. The answer to it is a rewrite of the anchor plan, which is different work from buying more spectrum. The report says which of the two problems it hit. A block too narrow for any mode is a spectrum problem. A block wide enough with no grid position inside it is a fragmentation problem. Collapsing them into one "no" sends an operator looking in the wrong place. ### The anchor range narrows as the mode widens[​](#the-anchor-range-narrows-as-the-mode-widens "Direct link to The anchor range narrows as the mode widens") Channels 1 and 96 cannot anchor a 400G carrier. Centring 79,600 MHz on channel 1 puts the lower edge 14,800 MHz below the bottom of the band, and 150,000 MHz is worse. | Mode | Width | Usable anchors on an empty section | | --------------------- | ----------- | ---------------------------------- | | `DP-QPSK 32GBd 100G` | 44,400 MHz | 96, channels 1 to 96 | | `DP-16QAM 64GBd 400G` | 79,600 MHz | 94, channels 2 to 95 | | `DP-QPSK 128GBd 400G` | 150,000 MHz | 94, channels 2 to 95 | Both 400G modes fit flush on channel 2 and channel 95: at `128 GBd`, channel 2's lower edge is 191,325,000 MHz, exactly the bottom of the band. This shows up on the emptiest route in the demo. Provision Berlin to Amsterdam over Hamburg, where no wavelength exists at all, and the carrier lands on **channel 2, not channel 1**. There is nothing wrong with the route and nothing in the way. The grid is not the capacity. ## Reading it off a branch[​](#reading-it-off-a-branch "Direct link to Reading it off a branch") ``` uv run invoke demo-capacity --branch probe ``` The capacity report gives, per section, the occupied and free megahertz, the free blocks with their edges, the anchors that can take another 400G, and a sentence saying which of the three answers applies. It also includes two standing notes. A route's free spectrum is the band minus the union of its sections' occupancy, and is never wider than the narrowest section. Free spectrum overstates capacity because anchors are quantised. `checks/channel_collision.py` is the gate. Two carriers no longer have to share an anchor to collide: a `128 GBd` carrier reaches three grid positions either side of its own centre, so the check compares intervals. It also refuses a carrier whose width crosses a band edge, which is the rule that makes channels 1 and 96 unusable for a 400G mode rather than merely unattractive. ## Where the numbers live[​](#where-the-numbers-live "Direct link to Where the numbers live") | File | What it holds | | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | `src/infrahub_demo_otn/units.py` | The roll-off, the guard band, the two band edges, `occupied_width_mhz`, `carrier_interval_mhz` and `free_blocks` | | `src/infrahub_demo_otn/plant.py` | The occupancy map, built from a carrier's anchor and its mode's symbol rate | | `src/infrahub_demo_otn/routing.py` | `fitting_channels`, which is what turns free blocks into usable anchors | | `transforms/capacity_view.py` | The report the capacity scenario prints | | `checks/channel_collision.py` | Overlap and band-edge refusal | | `tests/unit/test_units.py` | One assertion per mode, so a change to either constant cannot pass silently | No file under `src/` imports `infrahub_sdk`, so every figure on this page is reproducible with no server running. --- # What this demo shows Two readers land on this page. One is evaluating Infrahub and has no interest in optics. The other works in optics every day and already owns a planner and a controller. The first half below is written for the first reader and the second half for the second. The last two sections are for both. If you want the result before the reading, [the quick start](/demo-otn/quickstart.md) takes you from a fresh clone to a merge Infrahub refuses. ## For the network automation engineer[​](#for-the-network-automation-engineer "Direct link to For the network automation engineer") Each row below names a capability first, then the scenario in this demo that exercises it, then a link to the page that walks it. The optical detail is the example rather than the subject. A wavelength here plays the part a prefix, a VLAN or a circuit would play in your own domain, and the mechanism is the same one you would use. ### The capability map[​](#the-capability-map "Direct link to The capability map") | Infrahub capability | What this demo does with it | Scenario, and where it is walked | | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Branches | Every service is provisioned on a branch cut from the default one. The plant sits on the default branch and no service is ever written there. | Scenario one, [create the branch and the request](/demo-otn/provisioning-scenarios.md#create-the-branch-and-the-request) | | Proposed changes | Opening a proposed change from the branch runs the whole pipeline: fourteen checks, one generator and three artifact definitions, each reported on the change. The branch has to be created with `--sync-with-git` for that, or the change runs the two built-in validators and stops. | Scenario one, [review and merge](/demo-otn/provisioning-scenarios.md#review-and-merge) | | Checks that gate a merge | `.infrahub.yml` registers fourteen check definitions. `provisionable` reads `status` and fails the change for a service the generator left at `rejected` that nobody signed for. The reason code and the detail are what make the message readable. | Madrid to Warsaw at DP-16QAM, [the regenerator pair](/demo-otn/loadable-scenarios.md#two-loadable-scenarios-for-the-regenerator) and [the quick start](/demo-otn/quickstart.md#the-refusal-that-blocks-the-merge) | | Generators | One generator definition turns a service request that names no route into a carrier, an optical path, 25 ordered hops and two containers. | Scenario one, [run the generator](/demo-otn/provisioning-scenarios.md#run-the-generator) | | Artifacts | Three artifact definitions: a JSON service trace on every service, and two SVG maps rendered once per PoP, each from its own stored query. | Scenario five and [the two maps every PoP carries](/demo-otn/reporting-scenarios.md#the-two-maps-every-pop-carries) | | Python transforms | Ten transforms, each bound to its own stored query, render every report in the demo from live data. | Scenario eight, [the AI services against their budgets](/demo-otn/reporting-scenarios.md#scenario-eight-the-ai-services-against-their-budgets) | | Stored GraphQL queries | 25 queries, one per report or check, so a query two consumers share is never a query neither can change. | [Adding a report](/demo-otn/developer-guide.md#adding-a-report) | | Object files in Git | `objects/` holds the plant and loads on every init. `demo/` holds scenario input and is loaded by hand onto a branch. | [`objects/` against `demo/`](/demo-otn/developer-guide.md#objects-against-demo) | | Schema as code | Eight schema files loaded as one payload, with flat generics composed on the concrete nodes. | [The files](/demo-otn/schema-reference.md#the-files) | | Computed attributes | 32 of them across the eight files, every one Jinja2, most turning a scaled integer into a value a person can read. | [Display attributes](/demo-otn/schema-reference.md#display-attributes) | | Schema constraints | Enums, `min_value` and `max_value`, `optional: false` with a `default_value`, and uniqueness constraints in the `["relationship", "attribute__value"]` form. | The enum and the `default_value` at [the six reason codes](/demo-otn/schema-reference.md#the-six-reason-codes-and-the-signature-beside-them), the bounds at [the rules that constrain a change](/demo-otn/developer-guide.md#the-rules-that-constrain-a-change), the uniqueness constraint at [a regenerated circuit](/demo-otn/client-mapping.md#a-regenerated-circuit-has-containers-on-both-sides) | | `on_delete: cascade` | Used on the two relationships that own their children: a device's ports, and an optical path's hops. No page walks it. | Not walked | | Custom menus | `menus/otn.yml` builds the sidebar as five top-level entries and fifteen leaves, and every kind sets `include_in_menu: false` so nothing appears twice. | [Menu and navigation](/demo-otn/schema-reference.md#menu-and-navigation) | | Groups as targets | A `CoreGeneratorGroup` fires the generator. A `CoreStandardGroup` scoped to the PoPs is what the two map artifacts render against. | Scenario one, [create the branch and the request](/demo-otn/provisioning-scenarios.md#create-the-branch-and-the-request) | | The repository as a Git integration | The schema, the object files, the checks, the generator and the transforms all come from this repository. `invoke init` registers it, and that is what turns the pipeline on. | [Registering the repository](/demo-otn/developer-guide.md#registering-the-repository) | ### How a check decides a merge[​](#how-a-check-decides-a-merge "Direct link to How a check decides a merge") Three facts carry most of the weight, and they are worth stating on their own because they are the part that translates directly to your domain. **Any check that logs an error blocks the merge.** `log_error` blocks and `log_info` annotates. There is no `log_warning` in between, so a check has no way to soften a finding into advice. **A check can be global rather than targeted.** `provisionable` is registered with no `targets`, so it judges every service on the branch instead of the ones your change touched. Adding a span, retiring a mode or filling a corridor can make a service nobody edited impossible to build, and a targeted check bound to the edited objects would report green over it. **A refusal can be signed for.** `refusal_accepted` is a `Boolean` on the service with `default_value: false`. Set it and the branch merges with the refusal on the record, which is why scenario two merges: the 400G on the saturated Frankfurt to Milan corridor is refused for slots, and `demo/90_fra_mil_saturated.yml` signs for that refusal. The scenario that actually holds a merge shut is `demo/06_mad_waw_16qam.yml`, where nothing signs for anything. [Concepts](/demo-otn/concepts.md#a-refusal-is-an-answer-and-it-does-not-merge-unless-somebody-signed-for-it) has the decision tree. ## For the optical engineer[​](#for-the-optical-engineer "Direct link to For the optical engineer") An optical team does three jobs with a corridor. Design it, turn it up, and then answer questions about it for as long as it carries traffic: which services a duct cut takes down, whether two circuits are diverse, what is committed on a corridor nobody has built yet. The first two jobs have tools. A planning tool designs the corridor and needs no live equipment to do it. A domain controller turns the service up, and it holds planned objects too. Both are scoped, the planning tool to one project and one vendor, the controller to one domain. The third job crosses every vendor, every layer and every project, which is why it usually ends up in a spreadsheet. This demo holds that third record: versioned, held in one graph, and reviewed before it becomes true. It runs no propagation simulation, writes to no device, and reconciles against no external system. [What this demo does not exercise](#what-this-demo-does-not-exercise) is the full list. ### Where this sits next to your planner and your NMS[​](#where-this-sits-next-to-your-planner-and-your-nms "Direct link to Where this sits next to your planner and your NMS") | Question | Offline planner | Domain controller | This demo | | --------------------------------------------------------------- | --------------------------------- | ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Design a corridor: amplifier siting, budget, bill of materials | Yes. This is the tool for it. | No | No. It holds the result of that work, not the work | | Receive power and error rate on a channel right now | No | Yes, with history | The last reading only, carried as a fact on a monitor | | Turn the service up on the hardware | No | Yes | No. Nothing here writes to a device | | What is committed but not yet built, across vendors | Inside one project file | As planned objects, inside one domain | On a branch, across the whole plant | | Why this route and not the other one | With whoever ran the tool | No | The path, its ordered hops, the running budget, and the proposed change where a person signed | | What drops if one duct is cut | Within its own project | Within its own domain | [Scenario four](/demo-otn/reporting-scenarios.md#scenario-four-cut-the-frankfurt-to-amsterdam-fiber) | | Where a coloured pluggable in a router lands on the line system | It is not that tool's transponder | Split across two controllers | One chain of objects, the one below. Three OpenZR+ wavelengths run router to router here. A 400ZR reaches none of these sections, and the model says so and says why | | Which IP link rides which wavelength rides which duct | No | No | One query, and [scenario seven](/demo-otn/reporting-scenarios.md#scenario-seven-which-services-are-not-diverse) reads it | | What the plan looked like last quarter | An older file, if anyone kept it | Rarely | The branch and the commit history | The first three rows are jobs the planning tool and the controller already do, and this demo does not attempt any of them. The remaining six fall between the two tools. ### What varies by vendor, and where the model puts it[​](#what-varies-by-vendor-and-where-the-model-puts-it "Direct link to What varies by vendor, and where the model puts it") | What varies | Where the model puts it | What a second vendor costs | | ---------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | The mode catalog: reach, required OSNR, baud, forward error correction | `OtnOpticalMode` objects in `objects/03_optical_modes.yml` | More objects. No schema change and no code change | | How much spectrum a mode occupies | A width on the mode, fitted against `OtnFrequencyGrid` | Nothing. `channel_collision` reads the width and never asks who built the card | | Insertion loss and noise figure | `vendor` and `model` on `OtnOpticalElement`, with the loss fields beside them | Objects per model | | An alien wavelength on a line system another operator built | A transponder whose carrier crosses sections built from other equipment | Nothing. No relationship in the hop chain is scoped to a vendor | | Framing and correction latency | `fec_latency_ns` on the mode, `framing_latency_ns` on the ODU switch | Values per model, added up by [scenario eight](/demo-otn/reporting-scenarios.md#scenario-eight-the-ai-services-against-their-budgets) | | Which optic supplies a mode | `OtnTransceiverType`, with the modes each part number supports | More catalog entries. `transceiver_mode_support` refuses a mode the fitted part cannot run, whoever made it | | Naming and slot conventions | The `human_friendly_id` on each kind | A different string | Every racked element in this plant loads as `vendor: Generic`. The field exists and the dataset leaves it unnamed, so the table above says where a vendor difference goes rather than which vendor. A second vendor is more objects in `objects/`, reviewed on a branch like any other change. ### One model, every layer it touches[​](#one-model-every-layer-it-touches "Direct link to One model, every layer it touches") | Layer | What the demo holds | Kinds | | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | | Duct and shared risk | Which routes share a duct, which is what makes two services not diverse | `OtnConduit` | | Fiber | Attenuation, dispersion, group index, Raman gain where it is pumped, and loss added on purpose | `OtnFiberSpan`, `OtnFiberType`, `OtnRamanPump`, `OtnFixedAttenuator`, `OtnVariableAttenuator` | | Spectrum | The 50 GHz grid, the coarse grid, what each carrier occupies on both, and which fiber a channel lands on | `OtnFrequencyGrid`, `OtnCwdmChannel`, `OtnMuxClientPort`, `OtnMuxLinePort` | | Optical carrier | One provisioned wavelength, the mode it runs, and every section it crosses | `OtnOpticalCarrier`, `OtnOpticalMode` | | Digital wrapper | The container hierarchy, tributary slots, and where a wavelength terminates and the next one starts | `OtnContainer`, `OtnOduSwitch` | | Client handover | Ethernet, SDH, PDH, Fibre Channel and InfiniBand, each with the container it maps into | `OtnClientSignal` | | Router edge | The router, its grey port as the endpoint a service names, and the coloured line port where a pluggable puts a wavelength straight on the line system | `OtnRouter`, `OtnRouterPort`, `OtnLinePort`, `OtnTransceiver`, `OtnTransceiverType` | | Above that | Nothing. No VLAN, no prefix, no routing protocol | [Not exercised](#what-this-demo-does-not-exercise) | The last row counts as much as the others. This model stops at the router port. A layer boundary is a relationship like any other, so reaching one layer further up is the same work as the layers already modelled here. Other repositories have done that work for the layers above. [infrahub-demo-dc](https://github.com/opsmill/infrahub-demo-dc) models a data center down to cables, VLANs and IP space. [infrahub-demo-sp](https://github.com/opsmill/infrahub-demo-sp) models a service provider core with MPLS, BGP and L3VPN services. [infrahub-solution-ai-dc](https://github.com/opsmill/infrahub-solution-ai-dc) models an AI data center from the physical location up through the routing overlay to the workload. This repository is the layer below all three, and none of them shares code with it. Each defines its layer in schema files, which is what makes a layer a modelling decision rather than a product feature. ### From a router port to a duct[​](#from-a-router-port-to-a-duct "Direct link to From a router port to a duct") Eight steps, each one a relationship that already exists in the schema. | Step | Kind | Reached through | What it fixes | | ---- | ---------------------------- | ------------------------------ | --------------------------------- | | 1 | `OtnRouterPort` | the service endpoint's `ports` | the interface the IP side sees | | 2 | `OtnClientPort` | `connected_to` | the grey handover | | 3 | `OtnTransponder` | `device` | the card doing the adaptation | | 4 | `OtnLinePort` | `ports` | the coloured line side | | 5 | `OtnOpticalCarrier` | `carrier` | the wavelength and its mode | | 6 | `OtnOpticalMultiplexSection` | `sections` | ROADM to ROADM | | 7 | `OtnFiberSpan` | `spans` | the physical section and its loss | | 8 | `OtnConduit` | `conduit` | the shared risk | Two IP links are diverse only when step eight differs for both. `diversity.py` checks that on every proposed change, and [scenario seven](/demo-otn/reporting-scenarios.md#scenario-seven-which-services-are-not-diverse) walks it. The IP team asks whether two links share a duct. The optical team asks which services a duct cut takes down. Both questions read this chain, so both teams need it in the same place. ## What this demo does not exercise[​](#what-this-demo-does-not-exercise "Direct link to What this demo does not exercise") Absence read from silence is a bad inference, so the list below is explicit. Each row was checked against the repository rather than recalled. | Capability | Status here | | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Physical-layer propagation modelling | Not done. The budget is a running total against a mode's required OSNR, walked at [the link budget](/demo-otn/link-budget.md). Nothing here solves for a nonlinear penalty or fits a spectrum plan the way a planning tool does. | | Live performance monitoring | Not held. A monitor carries its last reading and the timestamp it was taken at, and `invoke demo-drift` compares that against what the model says it should be. There is no time series and nothing polls anything. | | Resource pools, `CoreNumberPool`, prefix and address pools | Not used. No pool of any kind is declared. A channel is chosen by fitting a mode's occupied width into free spectrum, which is a bin-packing decision rather than an allocation from a range. | | Profiles, `generate_profile` | Not used. Shared default values are written into the object files. | | Object templates, `generate_template` | Not used. The generator writes structure instead, because what it writes depends on which route closed. | | IPAM, prefixes and addresses | Not used. There is no `IpamPrefix` and no address anywhere in the schema. The vendored location generic drops `LocationHosting` deliberately, and that is the relationship that would have brought IPAM in. | | Transform-backed computed attributes | Not used. All 32 computed attributes are Jinja2 over the node's own fields. Nothing here aggregates over a cardinality-many relationship inside the schema. | | Jinja2 transforms and device configuration artifacts | Not used. All ten transforms are Python, and no artifact in this demo is a router configuration. | | Hierarchical locations | Declared and then switched off. The vendored generic carries `hierarchical: true`, but `OtnSite` sets `parent: ""` and `children: ""`, so the fifteen sites are one flat set with no region or country above them. | | Webhooks and event-driven integration | Not used. Every run here is a command you type or a proposed change you open. | | Accounts, roles and permissions | Not used. The repository ships no account, no account group and no permission object, and every command runs as the admin token in `.env`. | | Synchronisation with an external system | Not used. Nothing in this repository reads from or writes to anything outside Infrahub, a management system or a planning tool included. The monitor readings in `objects/` are loaded from files, not polled from equipment. | Two capabilities look absent from that list and are not. `on_delete: cascade` is used, and so are uniqueness constraints of the `["relationship", "attribute__value"]` form. Both are in the capability map above. ## Where to go from here[​](#where-to-go-from-here "Direct link to Where to go from here") * [Quick start](/demo-otn/quickstart.md) is the shortest path to a blocked merge. * [Provisioning scenarios](/demo-otn/provisioning-scenarios.md) walk the two scenarios that write: one service that provisions, and one corridor where a 400G is refused and a 100G is not. * [Reporting scenarios](/demo-otn/reporting-scenarios.md) walk the six that read, including what a fiber cut takes down and which services are not diverse. * [The link budget](/demo-otn/link-budget.md) is where the optical arithmetic is set out, corridor by corridor, including the two that do not close. * [Developer guide](/demo-otn/developer-guide.md) is where to look if you want to add a report, a check or an element of your own. --- ## Infrahub - PoC of a service catalog[​](#infrahub---poc-of-a-service-catalog "Direct link to Infrahub - PoC of a service catalog") This repository demonstrates a proof of concept for a service catalog using Infrahub and Streamlit. For more information about the business case and the development of this PoC, read the [blog post](https://opsmill.com/blog/how-to-turn-your-source-of-truth-into-a-service-factory/) on the OpsMill website. [Blog posthttps://opsmill.com/blog/how-to-turn-your-source-of-truth-into-a-service-factory/](https://opsmill.com/blog/how-to-turn-your-source-of-truth-into-a-service-factory/) ## The problem[​](#the-problem "Direct link to The problem") Organizations strive to deliver services efficiently, as this is where value is created. The stakes are high: the structure of a service determines everything downstream—from invoicing and lifecycle management to resource allocation and capacity planning. A poorly designed service layer can lead to inefficiencies and challenges at every stage. The service catalog forms the foundation of this structure, serving as a blueprint for how services are defined, managed, and delivered. It enables infrastructure self-service across the organization to streamline operations and enhance service delivery. ## Use case[​](#use-case "Direct link to Use case") ![Otternet](/assets/images/otternet-491510e07a59395b882e6f8abe580b5f.png) This demo follows a fictional ISP, Otter-net, which provides standard internet connectivity. ![Backbone](/assets/images/network-backbone-230ca1a3cfa0412b7a5a7825ea721e8c.png) The company operates multiple points of presence across Europe and the United States. Currently, it offers a single service: dedicated internet access. This service provides customers with a physical port and a set of public IP addresses for hosting services. Additional services are planned for the future! ![Service](/assets/images/network-customer-service-355b7b6ffae51cc19bb9cc1d0468d873.png) The operational team at Otter-net is divided into two groups: * Network Architects: Experts with extensive networking experience, responsible for operating and maintaining the backbone network. * Service Delivery Team: Customer-facing professionals responsible for provisioning and connecting services to the backbone. The goal The goal of the company is to automate the service delivery process, allowing the service delivery team to request new services without needing to involve network architects for every request. This will enable faster and more efficient service delivery. ## The solution[​](#the-solution "Direct link to The solution") Implementing a service catalog is a complex operation that many organizations struggle with. It demands a deep understanding of the product lifecycle, the interplay of various components, and coordination among numerous stakeholders. Beyond that, it requires a robust technical implementation to automate all the associated rules and processes properly. In many ways, Infrahub is the perfect tool to support your service catalog implementation: * **Flexible Schema**: Infrahub's schema can be tailored to fit the specific needs of your service catalog, allowing you to define services, components, and their relationships in a way that makes sense for your organization. * **Version Control**: Infrahub's native version control capabilities enable you to track changes, ensuring that you have full visibility into the evolution of your network. * **Generators**: Infrahub's generators can be used to codify the rules and processes associated with your service implementation, enabling fast and consistent implementation across the board. * **Resource Managers**: Infrahub's resource managers can be used to automate the allocation and management of resources, ensuring robust and efficient resource management. * **Branching**: Infrahub's branching capabilities allow you to isolate changes in a separate branch, paving the way for testing and validation before deploying changes to production. To complete the workflow from service request to configuration deployment, this demo also integrates [Semaphore](https://semaphoreui.com/), an open-source Ansible automation UI. Once a proposed change is reviewed and merged in Infrahub, Semaphore runs Ansible playbooks that fetch the generated device configurations from Infrahub and deploy them to network devices — closing the loop from self-service request to production deployment. If you want more information about the business case and how this PoC was built, you can read the corresponding [blog post](https://opsmill.com/blog/how-to-turn-your-source-of-truth-into-a-service-factory/) on the OpsMill website. [Blog posthttps://opsmill.com/blog/how-to-turn-your-source-of-truth-into-a-service-factory/](https://opsmill.com/blog/how-to-turn-your-source-of-truth-into-a-service-factory/) --- # Developer Walkthrough In this walkthrough, we detail the key building blocks of the demo, explain how it works, and describe how the components interact. This guide is intended for developers who want to understand the implementation details of the service catalog PoC. [Need help or have questions? Join our Discord community for support!https://discord.gg/opsmill](https://discord.gg/opsmill) ## The schema[​](#the-schema "Direct link to The schema") To structure and store data, we define a schema in Infrahub. You can find the schema files in the `schemas` folder. ### Schema architecture overview[​](#schema-architecture-overview "Direct link to Schema architecture overview") The following diagram illustrates the complete schema architecture, showing how different objects relate to each other in the Service Catalog demo: #### Key concepts in the diagram[​](#key-concepts-in-the-diagram "Direct link to Key concepts in the diagram") 1. **Generic vs Concrete Objects**: * **Generic** (dashed boxes): Abstract base classes that define common attributes * **Concrete** (solid boxes): Actual objects that can be instantiated 2. **Object Categories**: * **Service Layer** (blue): Service definitions and instances * **Location Layer** (pink): Geographic hierarchy and hosting locations * **DCIM Layer** (orange): Physical devices and interfaces * **IPAM Layer** (green): Network resources (IPs, VLANs, prefixes) 3. **Relationship Types**: * **Inheritance** (dashed arrows): Shows class hierarchy * **Associations** (solid arrows): Shows data relationships with cardinality * **Resource Allocation** (dotted arrows): Shows dynamic provisioning flow ### Consuming the schema library[​](#consuming-the-schema-library "Direct link to Consuming the schema library") Much of the schema is based on the [Infrahub schema library](https://github.com/opsmill/schema-library), which provides reusable schema components for quickly scaffolding a schema. * `base`: Contains generic definitions for `IPAM` (IP address, prefix, etc.), `DCIM` (network device, interface, etc.), `location`, and `organization` (provider, manufacturer). Importing this folder is mandatory as it provides the basic definitions required for extensions. * `location_minimal`: Defines a hierarchical tree for country, metro, and site. * `vlan`: Includes nodes for VLANs and L2 domains. * `circuit` (optional): Defines circuits with endpoints for site-to-site connections. This schema is available in `schemas/circuit/` but is not loaded by default. [Learn about schema libraryhttps://docs.infrahub.app/schema-library](https://docs.infrahub.app/schema-library) ### Custom service schema[​](#custom-service-schema "Direct link to Custom service schema") The service layer is unique to each organization, so we define a custom schema to represent services and their components. /schemas/service/service.yml ``` --- # yaml-language-server: $schema=https://schema.infrahub.app/infrahub/schema/latest.json version: "1.0" generics: # To enable future expansions, we define a generic service object. # This object holds all common attributes shared across the services. # Additionally, we can leverage this generic structure to simplify relationships. - name: Generic namespace: Service description: Generic service... label: Service icon: mdi:package-variant include_in_menu: true human_friendly_id: - service_identifier__value order_by: - service_identifier__value display_labels: - service_identifier__value attributes: - name: service_identifier kind: Text unique: true order_weight: 1000 optional: false branch: agnostic - name: account_reference kind: Text order_weight: 1010 optional: false branch: agnostic nodes: # The DedicatedInternet schema node inherits from the generic service object and includes a few additional attributes. # These attributes are relatively high-level (e.g., an ip_package with T-shirt size values) and are primarily intended as inputs for users. - name: DedicatedInternet namespace: Service description: This service provides customers with a dedicated physical port, ensuring complete internet connectivity. label: Dedicated Internet icon: mdi:ethernet menu_placement: ServiceGeneric inherit_from: - ServiceGeneric include_in_menu: true # By default, Infrahub creates data within branches (parallel realities), but it also supports branch-agnostic objects. # A branch-agnostic object is propagated to all branches, regardless of where it was created. # Here, branch-agnostic behavior is applied in the schema to the service object and key attributes, such as service_identifier. # This ensures consistent tracking of a service across all ongoing implementations and branches. branch: agnostic attributes: - name: status kind: Dropdown optional: false default_value: draft order_weight: 1050 # Putting this one as branch aware otherwise generator put it as active in the branch and so on main as well # even tho the service is really active only when the branch is merged... branch: aware choices: - name: draft label: Draft color: "#D3D3D3" - name: in-delivery label: In Delivery color: "#A8E6A2" - name: active label: Active color: "#66CC66" - name: in-decomissioning label: In Decomissioning color: "#FFAB59" - name: decomissioned label: Decomissioned color: "#FF6B6B" - name: bandwidth kind: Dropdown optional: false order_weight: 1100 branch: aware choices: - name: "100" label: Hundred Megabits description: Provides a 100 Mbps bandwidth. - name: "1000" label: One Gigabit description: Provides a 1 Gbps bandwidth. - name: "10000" label: Ten Gigabits description: Provides a 10 Gbps bandwidth. - name: ip_package kind: Dropdown optional: false order_weight: 1120 branch: aware choices: - name: small label: Small description: Provide customer with 6 IPs. color: "#6a5acd" - name: medium label: Medium description: Provide customer with 14 IPs. color: "#9090de" - name: large label: Large description: Provide customer with 30 IPs. color: "#ffa07a" # We implement various relationships to capture all the building blocks of the service (such as prefixes, interfaces, etc.) relationships: # From a site’s perspective, I only need a list of services and do not want multiple relationships for each type of service. # However, for a specific type of service, I want to enforce rules within the relationships. # For example, a distributed service could link to multiple sites, whereas a DedicatedInternet service is tied to a single site. # By configuring directions in relationships to point toward the generic service from a site’s perspective and initiating the relationship in the node pointing toward the site, we achieve the desired behavior. # Using the same identifier in the relationship allows Infrahub to recognize it as a single, unified relationship. - name: location peer: LocationSite order_weight: 1150 cardinality: one direction: inbound identifier: service_site optional: false branch: agnostic - name: dedicated_interfaces peer: DcimInterface kind: Attribute order_weight: 1200 cardinality: many direction: inbound identifier: service_interface - name: vlan peer: IpamVLAN kind: Attribute order_weight: 1300 cardinality: one direction: inbound identifier: service_vlan - name: gateway_ip_address peer: IpamIPAddress order_weight: 1350 cardinality: one direction: inbound identifier: service_ip_address - name: prefix peer: IpamPrefix kind: Attribute order_weight: 1400 cardinality: one direction: inbound identifier: service_prefix extensions: nodes: - kind: LocationSite relationships: - name: services peer: ServiceGeneric cardinality: many direction: outbound identifier: service_site branch: agnostic - kind: DcimInterface relationships: - name: service peer: ServiceGeneric cardinality: one direction: outbound identifier: service_interface - kind: IpamVLAN relationships: - name: service peer: ServiceGeneric cardinality: one direction: outbound identifier: service_vlan - kind: IpamIPAddress relationships: - name: service peer: ServiceGeneric cardinality: one direction: outbound identifier: service_ip_address - kind: IpamPrefix relationships: - name: service peer: ServiceGeneric cardinality: one direction: outbound identifier: service_prefix - kind: CoreProposedChange relationships: - name: tags peer: BuiltinTag cardinality: many ``` success We now have the data model and data to support our use case. It captures everything from services to the backbone, with some abstraction for flexibility. This setup is a strong foundation for automation. [Learn about Infrahub flexible schemahttps://docs.infrahub.app/guides/create-schema](https://docs.infrahub.app/guides/create-schema) ## The generator[​](#the-generator "Direct link to The generator") The generator is a powerful feature of Infrahub that allows you to codify the rules and processes associated with your service implementation. It enables fast and consistent implementation across the board. important We want to build the generator with the concept of **idempotency** in mind, meaning it should be repeatable: it assigns resources the first time it runs, and if run again, it changes nothing if the desired state is already achieved. This approach ensures the code is robust and predictable. Infrahub provides a set of features to help: * Resource manager: It allows users to create pools and allocate resources from them, such as prefixes, IP addresses, or even numbers. We will use this feature to allocate our prefixes/vlan in a branch-agnostic and idempotent way. Learn more about [resource managers](https://docs.infrahub.app/topics/resource-manager). * `allow_upsert=True`: This parameter is provided when saving the node, allowing it to be created if it doesn't exist or updated if it does. This is useful for ensuring that the generator can run multiple times without creating duplicates or errors. /generators/implement\_dedicated\_internet.py ``` from __future__ import annotations import logging import random from infrahub_sdk.generator import InfrahubGenerator from infrahub_sdk.node import InfrahubNode from infrahub_sdk.protocols import CoreIPPrefixPool, CoreNumberPool from service_catalog.protocols_async import ( DcimDevice, DcimInterfaceL3, IpamIPAddress, IpamPrefix, IpamVLAN, ServiceDedicatedInternet, ) ACTIVE_STATUS = "active" SERVICE_VLAN_POOL: str = "Customer vlan pool" SERVICE_PREFIX_POOL: str = "Customer prefixes pool" IP_PACKAGE_TO_PREFIX_SIZE: dict[str, int] = {"small": 29, "medium": 28, "large": 27} class DedicatedInternetGenerator(InfrahubGenerator): customer_service: ServiceDedicatedInternet | None = None allocated_vlan: IpamVLAN | None = None allocated_prefix: IpamPrefix | None = None gateway_ip: IpamIPAddress | None = None log = logging.getLogger("infrahub.tasks") async def generate(self, data: dict) -> None: service_dict: dict = data["ServiceDedicatedInternet"]["edges"][0]["node"] # Translate the dict to proper object self.customer_service: ServiceDedicatedInternet = await InfrahubNode.from_graphql( client=self.client, data=service_dict, branch=self.branch, ) # Move the service as active # TODO: Not happy with having this one here... self.customer_service.status.value = "active" await self.customer_service.save(allow_upsert=True) # Allocate the VLAN to the service await self.allocate_vlan() # Translate teeshirt size to int self.prefix_length: int = IP_PACKAGE_TO_PREFIX_SIZE[self.customer_service.ip_package.value] # Allocate the prefix to the service await self.allocate_prefix() # Allocate port await self.allocate_port() # Create L3 interface for gateway await self.allocate_gateway() async def allocate_vlan(self) -> None: """Create a VLAN with ID coming from the pool provided and assign this VLAN to the service.""" self.log.info("Allocating VLAN to this service...") # Get resource pool resource_pool = await self.client.get( kind=CoreNumberPool, name__value=SERVICE_VLAN_POOL, ) # Craft and save the vlan self.allocated_vlan = await self.client.create( kind=IpamVLAN, name=f"vlan__{self.customer_service.service_identifier.value}", vlan_id=resource_pool, # Here we get the vlan ID from the pool description=f"VLAN allocated to service {self.customer_service.service_identifier.value}", status=ACTIVE_STATUS, role="customer", l2domain=["default"], service=self.customer_service, ) # And save it to Infrahub await self.allocated_vlan.save(allow_upsert=True) self.log.info(f"VLAN `{self.allocated_vlan.name.value}` assigned!") async def allocate_prefix(self) -> None: """Allocate a prefix coming from a resource pool to the service.""" self.log.info("Allocating prefix from pool...") # Get resource pool resource_pool = await self.client.get( kind=CoreIPPrefixPool, name__value=SERVICE_PREFIX_POOL, ) # Craft the data dict for prefix prefix_data: dict = { "status": "active", "description": f"Prefix allocated to service {self.customer_service.service_identifier.value}", "service": [self.customer_service.id], "role": "customer", "vlan": [self.allocated_vlan.id], } # Create resource from the pool self.allocated_prefix = await self.client.allocate_next_ip_prefix( resource_pool, kind=IpamPrefix, data=prefix_data, prefix_length=self.prefix_length, identifier=self.customer_service.service_identifier.value, ) self.log.info(f"Prefix `{self.allocated_prefix}` assigned!") await self.allocated_prefix.save(allow_upsert=True) async def allocate_port(self) -> None: """Allocate a port to the service.""" allocated_port = None self.log.info("Allocating port to this service...") # Fetch interfaces records await self.customer_service.dedicated_interfaces.fetch() self.log.info( f"There are {len(self.customer_service.dedicated_interfaces.peers)} interfaces attached to this service.", ) # If we have any interface attached to the service if len(self.customer_service.dedicated_interfaces.peers) > 0: # Loop over interfaces attached to the service for interface in self.customer_service.dedicated_interfaces.peers: # Get device related to the interface await interface.peer.device.fetch() # If the device is "core" if interface.peer.device.peer.role.value == "core": self.log.info(f"Port `{interface.peer.display_label}` already allocated to the service.") # Big assomption but we assume port is already allocated self.index = interface.peer.device.peer.index.value allocated_port = interface break # If we don't have yet a port, we need to find one if allocated_port is None: self.log.info("Haven't found any port allocated to this service.") # Here, we pick randomly. In a real-life scenario, we might want to give this more thought self.index = random.randint(1, 2) # Find the switch on the site switch = await self.client.get( kind=DcimDevice, location__ids=[self.customer_service.location.id], role__value="core", index__value=self.index, ) self.log.info(f"Looking for port on {switch}...") # Fetch switch interface data await switch.interfaces.fetch() # Find first interface on that switch that is free selected_interface = next( ( interface for interface in switch.interfaces.peers if interface.peer.role.value == "customer" and interface.peer.status.value == "free" and interface.peer.service.id is None ), None, # Default value if no match is found ) # If we don't have any interface available if selected_interface is None: msg: str = f"There is no physical port to allocate to customer on {switch}" self.log.exception(msg) raise Exception(msg) self.log.info(f"Found port {selected_interface.peer.display_label} to allocate to the service.") allocated_port = selected_interface allocated_port = allocated_port.peer # Enforce all params of this interface allocated_port.enabled.value = True allocated_port.status.value = "active" allocated_port.l2_mode.value = "Access" allocated_port.role.value = "customer" allocated_port.description.value = f"Port allocated to service {self.customer_service.service_identifier.value}" allocated_port.speed.value = int(self.customer_service.bandwidth.value) allocated_port.service = self.customer_service allocated_port.untagged_vlan = self.allocated_vlan # Finally save await allocated_port.save(allow_upsert=True) async def allocate_gateway(self) -> None: """Allocate a gateway to the service.""" self.log.info("Allocating gateway to this service...") # Find the corresponding router router = await self.client.get( kind=DcimDevice, location__ids=[self.customer_service.location.id], role__value="edge", index__value=self.index, ) # Work around issue if isinstance(self.allocated_vlan.vlan_id.value, int): vlan_id: int = self.allocated_vlan.vlan_id.value else: vlan_id: int = self.allocated_vlan.vlan_id.value["value"] # Create interface gateway_interface = await self.client.create( kind=DcimInterfaceL3, name=f"vlan_{vlan_id!s}", speed=1000, device=router, status="active", role="customer", description=f"Gateway interface for service {self.customer_service.service_identifier.value}", enabled=True, service=self.customer_service, untagged_vlan=self.allocated_vlan, ) await gateway_interface.save(allow_upsert=True) # Compute the gateway ip address: str = f"{next(self.allocated_prefix.prefix.value.hosts())!s}/{self.prefix_length!s}" # Create IP object self.gateway_ip = await self.client.create( kind=IpamIPAddress, address=address, service=self.customer_service, interface=gateway_interface, ) await self.gateway_ip.save(allow_upsert=True) self.log.info(f"Gateway `{self.gateway_ip.address.value}` assigned!") # Add gateway to prefix self.allocated_prefix.gateway = self.gateway_ip # Save prefix await self.allocated_prefix.save(allow_upsert=True) ``` ### Generator architecture patterns[​](#generator-architecture-patterns "Direct link to Generator architecture patterns") 1. **Resource Tracking**: Every allocation linked to the service 2. **Relationship Integrity**: Bi-directional references maintained 3. **Error Handling**: Fail fast with clear messages 4. **Logging**: Detailed progress tracking for troubleshooting ### Service provisioning flow[​](#service-provisioning-flow "Direct link to Service provisioning flow") Here's a focused view of how resources flow during service provisioning: ### Summary: From request to reality[​](#summary-from-request-to-reality "Direct link to Summary: From request to reality") The generator transforms a service request: ``` Service: DI-12345 Location: Paris Bandwidth: 1 Gbps IP Package: Medium ``` Into fully provisioned infrastructure: * VLAN 142 allocated and configured * IP prefix 10.0.42.0/28 assigned * Switch port ge-0/0/5 configured * Router interface vlan\_142 with IP 10.0.42.1/28 All in seconds, consistently, and idempotently! [Learn about Infrahub generatorshttps://docs.infrahub.app/topics/generator](https://docs.infrahub.app/topics/generator) ## The service portal architecture[​](#the-service-portal-architecture "Direct link to The service portal architecture") ### Streamlit for the user interface[​](#streamlit-for-the-user-interface "Direct link to Streamlit for the user interface") The demo uses [Streamlit](https://streamlit.io/) for the user interface because it: * Builds web UIs with pure Python (no JavaScript required) * Provides ready-made components for forms and data display * Integrates seamlessly with the Infrahub Python SDK * Enables rapid prototyping of service catalogs ### Portal design patterns[​](#portal-design-patterns "Direct link to Portal design patterns") #### Infrahub SDK integration[​](#infrahub-sdk-integration "Direct link to Infrahub SDK integration") The portal uses several patterns to efficiently interact with Infrahub: Client Caching Pattern ``` @st.cache_resource def get_client(branch: str = "main") -> InfrahubClientSync: """Create and cache Infrahub client.""" address = get_instance_address() return InfrahubClientSync( address=address, config=Config(default_branch=branch) ) ``` Why Cache the Client? The `@st.cache_resource` decorator: * Creates client once and reuses it * Reduces connection overhead * Maintains consistent state * Improves page load performance [Learn about Streamlithttps://streamlit.io/](https://streamlit.io/) ## The deployment layer[​](#the-deployment-layer "Direct link to The deployment layer") ### Semaphore for Ansible automation[​](#semaphore-for-ansible-automation "Direct link to Semaphore for Ansible automation") [Semaphore](https://semaphoreui.com/) (SemaphoreUI) is an open-source web UI for running Ansible playbooks. In this demo, it serves as the deployment step: after a service is provisioned in Infrahub and the proposed change is merged, Semaphore runs playbooks that fetch the generated device configurations and deploy them to network equipment. ### Dynamic inventory[​](#dynamic-inventory "Direct link to Dynamic inventory") The inventory file (`ansible/inventory.yml`) uses the `opsmill.infrahub.inventory` plugin to build the device inventory dynamically from Infrahub at runtime. Instead of maintaining a static list of hosts, the plugin queries Infrahub for all `DcimDevice` nodes and composes host variables from their attributes: ansible/inventory.yml ``` plugin: opsmill.infrahub.inventory api_endpoint: "http://infrahub-server:8000" nodes: DcimDevice: include: - name - location.name - role - device_type.name - primary_address.address - status compose: hostname: name ansible_host: primary_address.address | ansible.utils.ipaddr('address') platform: device_type.platform.ansible_network_os location: location.name ``` This means any device added or modified in Infrahub is automatically reflected in the Ansible inventory — no manual synchronization required. ### The deployment playbook[​](#the-deployment-playbook "Direct link to The deployment playbook") The `ansible/deploy.yml` playbook performs two tasks per device: 1. **Fetch configuration artifact** — Uses `opsmill.infrahub.artifact_fetch` to retrieve the `startup-config` artifact from Infrahub, identified by the device's node ID 2. **Save configuration** — Writes the fetched configuration to a local file (`/tmp/_startup_config.txt`) ansible/deploy.yml ``` - name: Deploy configuration to network devices hosts: all connection: local gather_facts: false tasks: - name: "Fetch configuration artifact from Infrahub" opsmill.infrahub.artifact_fetch: artifact_name: "startup-config" target_id: "{{ id }}" register: artifact - name: Save configs to localhost ansible.builtin.copy: content: "{{ artifact.text }}" dest: "/tmp/{{ inventory_hostname }}_startup_config.txt" ``` ### Semaphore container setup[​](#semaphore-container-setup "Direct link to Semaphore container setup") The Semaphore container (`semaphore/Dockerfile`) extends the official image with the dependencies needed to interact with Infrahub: * **infrahub-sdk**: Python SDK installed via pip, required by the Ansible collection * **`opsmill.infrahub` collection**: Ansible collection installed via `ansible-galaxy`, providing the `inventory` plugin and `artifact_fetch` module At runtime, the container mounts the `ansible/` directory as its playbook source, so any changes to playbooks or inventory are reflected immediately without rebuilding. ## Architecture takeaways[​](#architecture-takeaways "Direct link to Architecture takeaways") ### Key design decisions[​](#key-design-decisions "Direct link to Key design decisions") 1. **Schema as Foundation**: Well-designed schema enables everything else 2. **Automation through Generators**: Complex provisioning made straightforward 3. **Resource Management**: Centralized pools prevent conflicts 4. **Branch-Based Workflows**: Safe testing and rollback capabilities 5. **User-Friendly Abstractions**: Hide complexity behind clear choices 6. **Deployment Automation**: Semaphore bridges the gap between source of truth and network devices, using dynamic inventory and artifact-based configuration delivery ### Extending the demo[​](#extending-the-demo "Direct link to Extending the demo") Consider these enhancements for production use: * Additional service types (MPLS, Cloud Connect, etc.) * Approval workflows with human checkpoints * Integration with ITSM platforms * Automated testing of provisioned services * Rollback and decommissioning automation * Event-driven deployment: trigger Semaphore runs automatically when proposed changes are merged in Infrahub ## Next steps[​](#next-steps "Direct link to Next steps") Now that you understand the architecture: 1. **Run the Demo**: Follow the [installation guide](/demo-service-catalog/getting-started/installation.md) to set up locally 2. **Try the Portal**: Walk through the [user experience](/demo-service-catalog/getting-started/user-walkthrough.md) 3. **Explore Further**: Modify the schema or create new service types 4. **Build Your Own**: Apply these patterns to your infrastructure needs The combination of Infrahub's flexible schema, powerful generators, and Python-based portals enables you to build sophisticated service automation tailored to your organization's needs. --- # How to Install the Service Catalog Demo This guide walks you through installing the Service Catalog demo, which demonstrates how to build a self-service portal on top of Infrahub's source of truth system. By the end of this guide, you'll have a working environment where users can request network services through a web interface. [Need help or have questions? Join our Discord community for support!https://discord.gg/opsmill](https://discord.gg/opsmill) ## What you'll achieve[​](#what-youll-achieve "Direct link to What you'll achieve") After completing this installation: * **Infrahub instance** running with network infrastructure data * **Service Catalog portal** accessible for requesting services * **Demo data** loaded including locations, devices, and network resources * **Git integration** configured for infrastructure as code workflows * **Semaphore automation** ready to deploy configurations to network devices ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before you get started, make sure you have these tools installed: ### Required software[​](#required-software "Direct link to Required software") * 🐳 **Docker**: Container runtime for running Infrahub and its dependencies * [Docker Desktop](https://docs.docker.com/desktop/) for local development * Or Docker Engine on a Linux VM for server deployments * Minimum 4GB RAM allocated to Docker * 🐍 **uv**: Fast Python package and project manager * Install via the [official installation guide](https://docs.astral.sh/uv/getting-started/installation/) * Used to manage Python packages and virtual environments * Version 0.4.0 or higher recommended ### System requirements[​](#system-requirements "Direct link to System requirements") * **Operating System**: macOS, Linux, or Windows with WSL2 * **Memory**: At least 8GB RAM (16GB recommended) * **Storage**: 10GB free disk space * **Network**: Internet connection for downloading dependencies ## Installation steps[​](#installation-steps "Direct link to Installation steps") ### Step 1: Clone the repository[​](#step-1-clone-the-repository "Direct link to Step 1: Clone the repository") First, download the demo code from GitHub. This repository contains all the necessary configuration files, schemas, and demo data. ``` git clone https://github.com/opsmill/infrahub-demo-service-catalog.git cd infrahub-demo-service-catalog ``` What's in the Repository? The repository includes: * **Infrahub schemas** defining the data models for network infrastructure * **Demo data** with sample locations, devices, and network resources * **Service Catalog application** built with Streamlit * **Docker configuration** for running all components ### Step 2: Configure environment variables[​](#step-2-configure-environment-variables "Direct link to Step 2: Configure environment variables") Set up the connection details for Infrahub. These variables tell the Service Catalog application how to communicate with the Infrahub API. ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_API_TOKEN="06438eb2-8019-4776-878c-0941b1f1d1ec" ``` #### Understanding the variables[​](#understanding-the-variables "Direct link to Understanding the variables") * **`INFRAHUB_ADDRESS`**: The URL where Infrahub's API is accessible * Default uses `localhost:8000` for local development * Change this if running Infrahub on a different host or port * **`INFRAHUB_API_TOKEN`**: Authentication token for API access * The demo uses a pre-configured token for convenience * In production, use secure token generation and storage ### Step 3: Install Python dependencies[​](#step-3-install-python-dependencies "Direct link to Step 3: Install Python dependencies") Use uv to create a virtual environment and install all required Python packages. ``` uv sync ``` This command: * Creates an isolated Python virtual environment * Installs the Infrahub SDK for API interactions * Installs Streamlit for the web portal interface * Sets up development tools (pytest, ruff, mypy) First time using uv? uv automatically manages virtual environments. All subsequent commands should be run with `uv run` prefix to ensure they use the correct environment. ### Step 4: Start the application stack[​](#step-4-start-the-application-stack "Direct link to Step 4: Start the application stack") Launch Infrahub and the Service Catalog using Docker Compose. This single command starts multiple services. ``` uv run invoke start ``` Why Use Invoke? The `invoke start` command is a wrapper that: * Ensures the correct Docker Compose configuration is used * Mounts the `service_catalog` module into the Infrahub container * Starts Infrahub, the Streamlit application, and Semaphore * Sets up networking between containers #### What gets started[​](#what-gets-started "Direct link to What gets started") This command launches: * **Infrahub Core**: The main application server (port 8000) * **PostgreSQL**: Database for storing infrastructure data * **Redis**: Cache and message broker * **RabbitMQ**: Task queue for asynchronous operations * **Service Catalog**: Streamlit web portal (port 8501) * **Semaphore**: Ansible automation UI (port 3000) ### Step 5: Initialize the demo environment[​](#step-5-initialize-the-demo-environment "Direct link to Step 5: Initialize the demo environment") Once Infrahub is up and running, initialize the demo by loading the repository, permissions, and seeding Semaphore. ``` uv run invoke init ``` This single command performs three things: 1. **Seeds Semaphore** with the project, SSH key, repository, inventory, environment, and task template (idempotent — safe to run multiple times) 2. **Loads the repository** configuration from `bootstrap/repository.yaml` into Infrahub — a read-write `CoreRepository` that references the local checkout mounted at `/upstream` in the container 3. **Loads permissions** from `bootstrap/permissions.yml`, creating an operator account (`john` / `infrahub`) with permissions to manage repositories, merge branches, and review proposed changes #### What the repository provides[​](#what-the-repository-provides "Direct link to What the repository provides") Once Infrahub imports the repository, it picks up: * **Generator Scripts**: Python code that automates service provisioning * **Configuration Templates**: Jinja2 templates for device configurations * **Schema**: Data models for DCIM, IPAM, services, and locations * **Objects and data**: Demo infrastructure loaded from YAML fixtures * **Menu**: Custom navigation structure for the Infrahub UI Understanding the Data Model The repository imports: * **Locations**: Sites in Paris, Brussels, New York, and Dallas * **Devices**: Core switches and edge routers at each site * **Providers**: Transit and peering providers (Cogent, Zayo, Colt, Lumen, Equinix) * **Network Resources**: VLAN pools, IP prefix pools, interface profiles * **Templates**: Device configurations and service definitions The schema uses a hierarchical structure: * Locations contain devices * Devices have interfaces and connections * Services allocate resources from pools * Everything is version-controlled and auditable ## Verify your installation[​](#verify-your-installation "Direct link to Verify your installation") Confirm everything is working by accessing both interfaces: ### Infrahub web UI[​](#infrahub-web-ui "Direct link to Infrahub web UI") * **URL**: * **Admin account**: `admin` / `infrahub` * **Operator account**: `john` / `infrahub` (limited permissions) * **What to Check**: * Navigate to the Schema page to see loaded models * Check the Objects page to verify demo data is present * Confirm the repository appears under Repositories * Log in as `john` to verify operator-level access works ### Service catalog portal[​](#service-catalog-portal "Direct link to Service catalog portal") * **URL**: * **What to Check**: * Home page displays available services * Service request forms are accessible * No connection errors appear ### Semaphore automation UI[​](#semaphore-automation-ui "Direct link to Semaphore automation UI") * **URL**: * **Username**: `admin` * **Password**: `semaphore` * **What to Check**: * Login page loads successfully * The "Service Catalog" project is visible (created by `invoke init`) * The "Deploy" task template and "Infrahub" inventory are configured ## Troubleshooting common issues[​](#troubleshooting-common-issues "Direct link to Troubleshooting common issues") ### Services won't start[​](#services-wont-start "Direct link to Services won't start") * Ensure Docker is running and has sufficient resources * Check for port conflicts (3000, 8000, 8501, 5432, 6379) * Run `uv run invoke destroy` then `uv run invoke start` to reset ### Schema load fails[​](#schema-load-fails "Direct link to Schema load fails") * Wait for Infrahub to fully start (check logs with `docker compose logs infrahub`) * Retry the schema load command after 30 seconds ### Can't access web interfaces[​](#cant-access-web-interfaces "Direct link to Can't access web interfaces") * Verify environment variables are set correctly * Check firewall rules if running on a remote server * Ensure all containers are running: `docker compose ps` ## Next steps[​](#next-steps "Direct link to Next steps") Now that your environment is running: 1. **For Users**: Follow the [user walkthrough](/demo-service-catalog/getting-started/user-walkthrough.md) to learn how to request services 2. **For Developers**: Check the [developer walkthrough](/demo-service-catalog/getting-started/developer-walkthrough.md) to understand the architecture 3. **Explore Further**: Browse the Infrahub UI to understand the data model and relationships --- # How to Use the Service Catalog Demo This guide demonstrates the complete service delivery workflow through two user perspectives: requesting services through the catalog and managing them through Infrahub. You'll see how automated provisioning transforms service requests into fully configured network services. [Need help or have questions? Join our Discord community for support!https://discord.gg/opsmill](https://discord.gg/opsmill) ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting, ensure you have: * Completed the [installation guide](/demo-service-catalog/getting-started/installation.md) * Both services running: * Service Catalog at `http://localhost:8501` * Infrahub at `http://localhost:8000` ## Understanding the workflow[​](#understanding-the-workflow "Direct link to Understanding the workflow") The demo simulates a typical enterprise service delivery process: 1. **Service Request**: Customer/engineer requests a service through self-service portal 2. **Automatic Processing**: Infrahub generates proposed changes with allocated resources 3. **Review & Approval**: Network architect reviews and approves the implementation 4. **Resource Allocation**: System automatically provisions VLANs, IPs, and configurations 5. **Service Delivery**: Customer receives access details for their new service 6. **Automated Deployment**: Semaphore runs Ansible playbooks to fetch generated configurations from Infrahub and deploy them to network devices [Blog posthttps://opsmill.com/blog/how-to-turn-your-source-of-truth-into-a-service-factory/](https://opsmill.com/blog/how-to-turn-your-source-of-truth-into-a-service-factory/) ## Step 1: Request a new service (service engineer role)[​](#step-1-request-a-new-service-service-engineer-role "Direct link to Step 1: Request a new service (service engineer role)") ### Understanding the service catalog interface[​](#understanding-the-service-catalog-interface "Direct link to Understanding the service catalog interface") The Service Catalog provides a simplified, user-friendly interface for requesting network services without requiring deep technical knowledge. ### Create a dedicated internet service request[​](#create-a-dedicated-internet-service-request "Direct link to Create a dedicated internet service request") 1. **Access the Service Catalog** * Open your browser to `http://localhost:8501` * You'll see the main dashboard with available service types 2. **Navigate to Service Creation** * Click on the "🔌 Dedicated Internet" card from the home page * Or use the sidebar navigation to go to "Dedicated Internet" 3. **Fill Out the Service Request Form** The form presents user-friendly options that hide technical complexity: * **Service Identifier**: Choose a unique name (for example: "DI-DEMO-001") * **Account Reference**: Customer or project identifier * **Location**: Select a site within a location (for example: "Paris 1", "Brussels 1", "New York 1", "Dallas 1") * **Bandwidth**: Choose from human-readable options: * "Ten Gigabits" (10 Gbps) * "One Gigabit" (1 Gbps) * "Hundred Megabits" (100 Mbps) * **IP Package**: Select based on customer needs: * "Small" (6 usable IPs) * "Medium" (14 usable IPs) * "Large" (30 usable IPs) 4. **Submit the Request** * Click "Submit" to create the service request * The system creates a new branch in Infrahub for this change ![Streamlit Form](/assets/images/form-dfaa078ad523dfcaa985c5bc4388cb42.png) What Happens Behind the Scenes When you submit the request: 1. A new git branch is created in Infrahub 2. The service object is created with "draft" status 3. A generator is triggered to allocate resources 4. A "Proposed Change" is created for review ### Track your service requests[​](#track-your-service-requests "Direct link to Track your service requests") 1. **View All Requests** * Navigate to "📦 Service Requests" in the sidebar * You'll see a table of all service requests with their current status 2. **Monitor Request Progress** The table shows key information: * **Service ID**: Your unique identifier * **Status**: Current state (Draft, In Delivery, Active) * **Location**: Where the service will be deployed * **Bandwidth & IP Package**: Technical specifications * **Account**: Customer reference 3. **Understanding Status Progression** * **Draft**: Initial request submitted, awaiting processing * **In Delivery**: Resources allocated, implementation in progress * **Active**: Service fully provisioned and operational In this example, the Service Request is in Draft state. ![Service Requests](/assets/images/service_requests-7add0941927656196a3950d16a0fc29b.png) Why Two Interfaces? The Service Catalog provides a simplified view for business users, while Infrahub offers detailed technical views for network engineers. This separation allows each role to work with appropriate tools. ## Step 2: Review infrastructure state (network architect role)[​](#step-2-review-infrastructure-state-network-architect-role "Direct link to Step 2: Review infrastructure state (network architect role)") ### Understanding Infrahub's role[​](#understanding-infrahubs-role "Direct link to Understanding Infrahub's role") Infrahub serves as the "source of truth" for your network infrastructure, maintaining detailed models of: * Physical locations and equipment * Network configurations and relationships * Service allocations and dependencies * Change history and version control ### Access the Infrahub interface[​](#access-the-infrahub-interface "Direct link to Access the Infrahub interface") 1. **Log into Infrahub** * Navigate to `http://localhost:8000` * Username: `admin` * Password: `infrahub` 2. **Explore the Infrastructure Model** The left sidebar organizes infrastructure data by category: **Locations** * Countries, metros, and sites * Geographic hierarchy for service placement **DCIM (Data Center Infrastructure Management)** * Devices (switches, routers, servers) * Interfaces and connections * Hardware specifications **IPAM (IP Address Management)** * IP prefixes and addresses * VLANs and L2 domains * Resource pools for allocation **Services** * Service definitions and instances * Resource assignments * Customer relationships 3. **Examine Site Details** * Click `Location` > `Site` to see all sites * Select a site (for example: "Paris 1") * Review the site's components: * Associated devices and their roles * Current VLAN assignments * Available resource pools * Existing services ![Site View](/assets/images/site-da02a5777ea882adf9d47fc0a172674c.png) Infrastructure as Code Benefits By modeling infrastructure in Infrahub: * All changes are version-controlled * Relationships between components are maintained * Resource conflicts are automatically prevented * Changes can be tested before implementation ### Review and approve service implementation[​](#review-and-approve-service-implementation "Direct link to Review and approve service implementation") 1. **Access Proposed Changes** * Navigate to `Proposed Changes` in the left sidebar * You'll see the service request you created earlier * Each request creates a separate branch for safe testing 2. **Examine the Automated Provisioning** Click on your service request to see detailed tabs: **Overview Tab** * Summary of the proposed change * Author and timestamp information * Current status and available actions **Checks Tab** * Automated validation results * Generator execution status (should show success) * Any constraint violations or conflicts **Data Tab** * Original service request data * Your form inputs (location, bandwidth, etc.) * Service metadata and relationships 3. **Review Resource Allocation** * Click "Refresh Diff" to see what the generator created * Examine the provisioned resources: **VLAN Assignment** ``` New VLAN: vlan__DI-DEMO-001 ID: 142 (allocated from pool) Role: customer Status: active ``` **IP Prefix Allocation** ``` New Prefix: 10.0.42.0/28 Gateway: 10.0.42.1 Usable IPs: 10.0.42.2 - 10.0.42.14 ``` **Interface Configuration** ``` Switch Port: ge-0/0/5 on core-switch-1 Speed: 1000 Mbps VLAN: 142 (untagged) Status: active ``` **Router Interface** ``` Interface: vlan_142 on edge-router-1 IP Address: 10.0.42.1/28 Status: active ``` 4. **Approve the Implementation** * Return to the "Overview" tab * Review the summary of changes * Click "Merge" to approve and implement * The branch merges into main, making changes live ![Proposed Change](/assets/images/proposed_change-c128332802247f048fd4d9b2e4810332.png) Approval Workflow Benefits This review process provides: * **Safety**: Changes tested in isolation before going live * **Visibility**: Full resource allocation visible before approval * **Audit Trail**: Complete history of who approved what and when * **Rollback**: Ability to revert changes if issues arise ## Step 3: Verify service delivery[​](#step-3-verify-service-delivery "Direct link to Step 3: Verify service delivery") ### Confirm active service[​](#confirm-active-service "Direct link to Confirm active service") 1. **Check Service Status** * Return to the Service Catalog (`http://localhost:8501`) * Navigate to "Service Requests" * Your service should now show "Active" status 2. **Review Allocated Resources** * In Infrahub, navigate to the service under `Service` > `Dedicated Internet` * Click on your service to see all allocated components * Verify relationships to VLANs, interfaces, and IP addresses 3. **Understand the Customer Experience** At this point, a real customer would receive: * **Connection Details**: Switch port and patch panel location * **IP Configuration**: Gateway IP, subnet mask, and usable IP range * **VLAN Information**: VLAN ID for their traffic * **Support Contacts**: Who to contact for service issues ## Step 4: Deploy the configurations[​](#step-4-deploy-the-configurations "Direct link to Step 4: Deploy the configurations") After the proposed change is merged, you can use Semaphore to deploy the generated configurations to network devices. ### Access the Semaphore UI[​](#access-the-semaphore-ui "Direct link to Access the Semaphore UI") 1. **Open Semaphore** * Navigate to `http://localhost:3000` * Username: `admin` * Password: `semaphore` 2. **Locate the Deployment Task** * Find the deployment project in the Semaphore dashboard * The project is configured with the `deploy.yml` playbook and the dynamic Infrahub inventory ### Run the deployment playbook[​](#run-the-deployment-playbook "Direct link to Run the deployment playbook") 1. **Start a Task Run** * Click "Run" to launch the deployment playbook * Semaphore uses the dynamic inventory plugin to query Infrahub for all devices and their attributes 2. **Monitor Real-Time Output** * Watch the live output as Ansible connects to each device * The playbook fetches the `startup-config` artifact from Infrahub for each device * Configurations are saved locally (and in a real environment, pushed to devices) 3. **Verify Deployment Success** * Check that all tasks complete with "ok" or "changed" status * Review the output for any errors or skipped hosts * Confirm that configuration files were generated for each device What the Playbook Does The `deploy.yml` playbook performs two tasks per device: 1. **Fetch configuration artifact** — Uses the `opsmill.infrahub.artifact_fetch` module to retrieve the `startup-config` artifact from Infrahub 2. **Save configuration** — Writes the configuration to a local file for deployment Workflow Complete! You've successfully: 1. ✅ Requested a service through the self-service portal 2. ✅ Reviewed the automated resource allocation 3. ✅ Approved the implementation as a network architect 4. ✅ Delivered a fully configured network service 5. ✅ Deployed configurations to network devices via Semaphore All in minutes, with zero manual configuration errors! ## Key takeaways[​](#key-takeaways "Direct link to Key takeaways") ### Business benefits[​](#business-benefits "Direct link to Business benefits") * **Speed**: Service delivery in minutes instead of days/weeks * **Consistency**: Standardized configurations eliminate human error * **Self-Service**: Reduces bottlenecks and improves customer satisfaction * **Visibility**: Complete audit trail and resource tracking ### Technical benefits[​](#technical-benefits "Direct link to Technical benefits") * **Resource Management**: Automatic allocation prevents conflicts * **Version Control**: All changes tracked and reversible * **Validation**: Automated checks prevent invalid configurations * **Integration**: API-driven architecture enables broader automation ### Next steps[​](#next-steps "Direct link to Next steps") Now that you understand the user experience: * Explore the [developer walkthrough](/demo-service-catalog/getting-started/developer-walkthrough.md) to understand the technical implementation * Try creating additional service types or modifying existing ones * Consider how similar patterns could apply to your organization's services --- # Welcome to Emma Emma is an experimental AI-powered assistant designed to help you interact with Infrahub, OpsMill's next-generation infrastructure management platform. Emma helps you manage infrastructure schemas and data through an intuitive web interface. ![Emma Homepage](/assets/images/home_page-688ffc9365ffba7db2bec7b3ee5c6c4f.png) Experimental Software Emma is experimental by design and may not work as expected. She tests out new ideas before her big brother Otto gets involved and implements them in Infrahub. ## What Emma can do[​](#what-emma-can-do "Direct link to What Emma can do") Emma provides powerful tools for infrastructure management: ### 🏗️ **Schema management**[​](#️-schema-management "Direct link to ️-schema-management") * **[AI Schema Builder](/emma/features/schema-builder.md)** - Create schemas using natural language descriptions * **[Schema Library](https://docs.infrahub.app/schema-library)** - Browse and use community-validated schemas * **[Schema Loader](/emma/features/schema-management.md)** - Load schemas into your Infrahub instance * **[Schema Visualizer](/emma/features/schema-management.md)** - Visual representation of schema relationships ### 📊 **Data operations**[​](#-data-operations "Direct link to -data-operations") * **[Data Import](/emma/features/data-import-export.md)** - Import CSV data into Infrahub with intelligent mapping * **[Data Export](/emma/features/data-import-export.md)** - Export Infrahub data to CSV for analysis and backup ### 🔬 **Experimental features**[​](#-experimental-features "Direct link to -experimental-features") * **Query Builder** - Interactive GraphQL query interface (feature flag required) * **Template Builder** - Create and manage infrastructure templates (feature flag required) ## Quick start[​](#quick-start "Direct link to Quick start") Ready to get started? Choose your preferred installation method: ### 🚀 **Fastest setup**[​](#-fastest-setup "Direct link to -fastest-setup") Use the one-command quickstart to get Emma and Infrahub running together: ``` curl https://infrahub.opsmill.io/latest-emma | docker compose -f - up -d ``` Then open [Emma](http://localhost:8501) and [Infrahub](http://localhost:8000) in your browser. ### 📚 **Full installation guide**[​](#-full-installation-guide "Direct link to -full-installation-guide") For detailed installation instructions, including local development setup and production deployments, see the **[Installation Guide](/emma/getting-started/installation.md)**. ## Documentation structure[​](#documentation-structure "Direct link to Documentation structure") ### 🚀 Getting started[​](#-getting-started "Direct link to 🚀 Getting started") Perfect for new users: * **[Installation](/emma/getting-started/installation.md)** - Set up Emma in various environments * **[Configuration](/emma/getting-started/configuration.md)** - Configure Emma and connect to Infrahub * **[First Steps](/emma/getting-started/first-steps.md)** - Your first experience with Emma ### ⚙️ Features[​](#️-features "Direct link to ⚙️ Features") Deep dives into Emma's capabilities: * **[AI Schema Builder](/emma/features/schema-builder.md)** - AI-powered schema creation * **[Data Import & Export](/emma/features/data-import-export.md)** - Comprehensive data management * **[Schema Management](/emma/features/schema-management.md)** - Complete schema lifecycle * **[Schema Library](https://docs.infrahub.app/schema-library)** - Community schema collection ### 📖 Guides[​](#-guides "Direct link to 📖 Guides") Step-by-step tutorials: * **[Building Your First Schema](/emma/guides/building-your-first-schema.md)** - Complete walkthrough ### 📚 Reference[​](#-reference "Direct link to 📚 Reference") Complete reference materials: * **[Configuration Reference](/emma/reference/configuration.md)** - All configuration options * **[Feature Flags](/emma/reference/feature-flags.md)** - Experimental feature management * **[Troubleshooting](/emma/reference/troubleshooting.md)** - Solutions to common issues ## Common use cases[​](#common-use-cases "Direct link to Common use cases") ### Infrastructure modeling[​](#infrastructure-modeling "Direct link to Infrastructure modeling") Start with Emma's Schema Library to model your infrastructure: 1. Browse [pre-built schemas](https://docs.infrahub.app/schema-library) for common components 2. Use the [AI Schema Builder](/emma/features/schema-builder.md) for custom requirements 3. [Load schemas](/emma/features/schema-management.md) into your Infrahub instance 4. [Visualize relationships](/emma/features/schema-management.md) to understand your model ### Data migration[​](#data-migration "Direct link to Data migration") Migrate from existing systems using Emma's data tools: 1. Export data from your current systems to CSV 2. Use Emma's [Data Importer](/emma/features/data-import-export.md) to map and load data 3. Validate and clean data during the import process 4. Export data for backup and integration with other tools ### Schema development[​](#schema-development "Direct link to Schema development") Develop and iterate on infrastructure schemas: 1. [Plan your schema](/emma/guides/building-your-first-schema.md) requirements 2. Use AI assistance to generate initial schema definitions 3. Test and refine schemas with sample data 4. Share schemas with your team through the library ## Getting help[​](#getting-help "Direct link to Getting help") ### 📖 Documentation[​](#-documentation "Direct link to 📖 Documentation") This documentation covers everything you need to know about Emma. Use the navigation menu to explore topics in detail. ### 🐛 Issues and support[​](#-issues-and-support "Direct link to 🐛 Issues and support") * **Bug Reports**: [GitHub Issues](https://github.com/opsmill/emma/issues) * **General Questions**: [OpsMill Discord Community](https://discord.gg/opsmill) ### 🔗 Related resources[​](#-related-resources "Direct link to 🔗 Related resources") * **[Infrahub Documentation](https://docs.infrahub.app)** - Complete Infrahub documentation * **[Schema Library Repository](https://docs.infrahub.app/schema-library)** - Community schema collection * **[OpsMill Website](https://opsmill.com)** - Learn more about OpsMill ## Next steps[​](#next-steps "Direct link to Next steps") 1. **New to Emma?** Start with the [Installation Guide](/emma/getting-started/installation.md) 2. **Ready to build?** Try [Building Your First Schema](/emma/guides/building-your-first-schema.md) 3. **Need schemas?** Explore the [Schema Library](https://docs.infrahub.app/schema-library) 4. **Have data to import?** Check out [Data Import & Export](/emma/features/data-import-export.md) Emma is designed to make infrastructure management more accessible and efficient. Whether you're modeling a basic network or a complex multi-cloud environment, Emma provides the tools you need to succeed. Happy infrastructure modeling! --- # Data Import and Export Emma provides powerful tools for importing data into Infrahub from CSV files and exporting Infrahub data back to CSV format. This enables efficient migration from existing systems and data sharing between tools. ![Data Exporter](/assets/images/data_exporter-5758ce92cd8b320f30c4da3baca07860.png) ## Data import[​](#data-import "Direct link to Data import") The Data Importer allows you to upload CSV files and map their columns to Infrahub schema attributes, enabling you to bulk-load infrastructure data. ### Supported file formats[​](#supported-file-formats "Direct link to Supported file formats") * **CSV files** with headers * **UTF-8 encoding** (recommended) * **Common delimiters**: comma, semicolon, tab * **File size**: Up to 100MB per upload ### Import process[​](#import-process "Direct link to Import process") #### Step 1: Prepare your data[​](#step-1-prepare-your-data "Direct link to Step 1: Prepare your data") Ensure your CSV file: * **Has column headers** that roughly match your schema attributes * **Contains clean data** with consistent formatting * **Includes required fields** as defined by your schema * **Uses consistent values** for relationships and enums **Example CSV structure:** ``` name,model,serial_number,location,ip_address,status switch-01,Cisco 3850,FCW2140L0EF,datacenter-1,192.168.1.10,active switch-02,Cisco 3850,FCW2140L0EG,datacenter-1,192.168.1.11,active ``` #### Step 2: Upload and map[​](#step-2-upload-and-map "Direct link to Step 2: Upload and map") 1. **Select your CSV file** using the file upload interface 2. **Choose the target schema** from your Infrahub instance 3. **Map CSV columns** to schema attributes using the dropdown selectors 4. **Configure relationship handling** for foreign key references 5. **Set data validation options** like duplicate handling #### Step 3: Preview and validate[​](#step-3-preview-and-validate "Direct link to Step 3: Preview and validate") Emma provides a preview showing: * **Mapped data** - How your CSV data will appear in Infrahub * **Validation results** - Any data quality issues or constraint violations * **Relationship resolution** - How foreign keys will be resolved * **Import statistics** - Number of records to be created/updated #### Step 4: execute import[​](#step-4-execute-import "Direct link to Step 4: execute import") After reviewing the preview: 1. **Confirm the import** if everything looks correct 2. **Monitor progress** through the real-time status updates 3. **Review results** including success/failure counts and any error details ### Advanced import features[​](#advanced-import-features "Direct link to Advanced import features") * Relationship Handling * Data Validation * Performance Options Emma intelligently handles relationships between objects: **By Name/ID Lookup:** * Reference existing objects by name or ID * Automatic resolution of relationship targets * Error reporting for unresolved references **Nested Object Creation:** * Create related objects inline during import * Support for hierarchical data structures * Maintains referential integrity **Example:** ``` device_name,interface_name,ip_address,vlan_name switch-01,GigabitEthernet1/0/1,192.168.1.10,production switch-01,GigabitEthernet1/0/2,192.168.1.11,management ``` Emma validates data during import: **Schema Validation:** * Required field checking * Data type validation * Format validation (IP addresses, emails, etc.) * Enum value verification **Business Rules:** * Uniqueness constraints * Custom validation rules * Cross-field dependencies **Error Handling:** * Detailed error reporting * Skip invalid records option * Partial import support For large datasets: **Batch Processing:** * Configurable batch sizes * Memory-efficient processing * Progress tracking **Incremental Updates:** * Update existing records * Insert new records only * Upsert functionality **Parallel Processing:** * Multiple worker threads * Optimized for large files * Automatic resource management ## Data export[​](#data-export "Direct link to Data export") The Data Exporter extracts data from Infrahub and formats it as CSV files for use in other tools or for backup purposes. ### Export options[​](#export-options "Direct link to Export options") #### Schema-based export[​](#schema-based-export "Direct link to Schema-based export") Export all objects of a specific schema type: 1. **Select schema** from your Infrahub instance 2. **Choose attributes** to include in the export 3. **Configure relationship handling** (include related object details) 4. **Set filtering criteria** to limit exported records #### Query-based export[​](#query-based-export "Direct link to Query-based export") Export data using custom filters: * **Field filters** - Filter by specific attribute values * **Date ranges** - Export data from specific time periods * **Relationship filters** - Filter based on related object properties * **Complex queries** - Combine multiple filter criteria #### Bulk export[​](#bulk-export "Direct link to Bulk export") Export multiple schema types in a single operation: * **Related schemas** - Export parent and child objects together * **Dependency order** - Automatic ordering for reimport compatibility * **Consistent snapshots** - Ensure data consistency across exports ### Export formats[​](#export-formats "Direct link to Export formats") Emma supports multiple CSV format options: **Standard CSV:** * Comma-separated values * Header row with attribute names * UTF-8 encoding **Customizable Format:** * Custom delimiters * Quote character options * Line ending preferences * Character encoding selection **Relationship Handling:** * Flatten relationships into columns * Include relationship IDs * Export related object details ### Advanced export features[​](#advanced-export-features "Direct link to Advanced export features") #### Incremental export[​](#incremental-export "Direct link to Incremental export") Export only changes since last export: * **Timestamp-based** - Export records modified after a specific date * **Change tracking** - Export based on Infrahub's change tracking * **Delta exports** - Include only modified fields #### Scheduled exports[​](#scheduled-exports "Direct link to Scheduled exports") Configure automated exports: * **Recurring schedules** - Daily, weekly, monthly exports * **Event-triggered** - Export on data changes * **Multiple formats** - Generate different exports for different consumers ## Data quality and validation[​](#data-quality-and-validation "Direct link to Data quality and validation") ### Import validation[​](#import-validation "Direct link to Import validation") Emma validates imported data against: * **Schema definitions** - Ensure data matches expected structure * **Data types** - Validate numeric, date, and other typed fields * **Constraints** - Check uniqueness, required fields, and custom rules * **Relationships** - Verify referenced objects exist ### Export verification[​](#export-verification "Direct link to Export verification") Exported data includes: * **Metadata** - Export timestamp, schema versions, record counts * **Integrity checks** - Checksums for data verification * **Audit trail** - Information about data source and transformation ## Best practices[​](#best-practices "Direct link to Best practices") ### Import best practices[​](#import-best-practices "Direct link to Import best practices") 1. **Validate data quality** before import using external tools 2. **Start with small batches** to test mappings and validations 3. **Use consistent naming** for relationship references 4. **Clean up data** to remove duplicates and inconsistencies 5. **Backup Infrahub** before large imports ### Export best practices[​](#export-best-practices "Direct link to Export best practices") 1. **Document export purposes** to choose appropriate options 2. **Test import compatibility** if exporting for reimport 3. **Use incremental exports** for regular synchronization 4. **Include metadata** for audit and tracking purposes 5. **Validate exported data** before using in downstream systems ### Performance optimization[​](#performance-optimization "Direct link to Performance optimization") 1. **Use appropriate batch sizes** (typically 100-1000 records) 2. **Limit concurrent operations** to avoid overwhelming Infrahub 3. **Monitor system resources** during large operations 4. **Use filtering** to export only necessary data 5. **Schedule large operations** during low-usage periods ## Integration scenarios[​](#integration-scenarios "Direct link to Integration scenarios") ### Common use cases[​](#common-use-cases "Direct link to Common use cases") **Migration from Legacy Systems:** * Export data from existing tools to CSV * Transform and clean data as needed * Import into Infrahub schemas **Data Synchronization:** * Regular exports to external systems * Incremental updates based on changes * Bidirectional synchronization workflows **Backup and Recovery:** * Full data exports for backup purposes * Schema-consistent exports for disaster recovery * Point-in-time data snapshots **Reporting and Analytics:** * Export data for business intelligence tools * Create regular data extracts for reporting * Integration with data lakes and warehouses ### Integration with other Emma features[​](#integration-with-other-emma-features "Direct link to Integration with other Emma features") * **Schema Library** - Use library schemas as import targets * **Schema Builder** - Create schemas for imported data * **Schema Visualizer** - Understand data relationships before import/export ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Common import issues[​](#common-import-issues "Direct link to Common import issues") **File Upload Problems:** * Check file size limits (100MB maximum) * Verify file format and encoding * Ensure proper CSV structure with headers **Mapping Errors:** * Verify schema attribute names * Check data type compatibility * Review required field mappings **Validation Failures:** * Check data quality and formatting * Verify relationship references exist * Review constraint violations **Performance Issues:** * Reduce batch sizes * Check network connectivity * Monitor Infrahub resource usage ### Common export issues[​](#common-export-issues "Direct link to Common export issues") **Missing Data:** * Verify user permissions for accessed schemas * Check filtering criteria * Review relationship configurations **Format Problems:** * Verify encoding settings * Check delimiter configuration * Review quote character handling **Performance Issues:** * Use filtering to reduce data volume * Increase timeout settings * Monitor export progress For detailed troubleshooting, see the [Troubleshooting Guide](/emma/reference/troubleshooting.md). --- # AI Schema Builder Emma's Schema Builder is an AI-powered tool that helps you create Infrahub schemas using natural language descriptions. Instead of manually writing YAML schema definitions, you can describe what you need and let Emma's AI assistant generate the appropriate schema structure. ![Schema Builder](/assets/images/schema_builder-525dc0ecdbbdd761a54677abe2f9f557.png) ## How it works[​](#how-it-works "Direct link to How it works") The Schema Builder leverages OpenAI's language models to understand infrastructure concepts and translate natural language requirements into properly structured Infrahub schemas. It understands: * Common infrastructure components (devices, interfaces, IP addresses, etc.) * Relationships between infrastructure objects * Industry-standard naming conventions and best practices * Infrahub schema syntax and requirements ## Getting started[​](#getting-started "Direct link to Getting started") ### Prerequisites[​](#prerequisites "Direct link to Prerequisites") To use the AI Schema Builder, you need: * Basic understanding of your infrastructure requirements * Knowledge of how your schema fits into your overall infrastructure model ### Basic workflow[​](#basic-workflow "Direct link to Basic workflow") 1. **Describe your schema** in natural language 2. **Review the AI-generated schema** for accuracy and completeness 3. **Refine your description** if needed to get better results 4. **Generate the final schema** and optionally load it into Infrahub ## Creating your first schema[​](#creating-your-first-schema "Direct link to Creating your first schema") ### Step 1: Describe your requirements[​](#step-1-describe-your-requirements "Direct link to Step 1: Describe your requirements") In the Schema Builder interface, describe what you want to model. Be specific about: * **Object type**: What kind of infrastructure component * **Attributes**: What properties it should have * **Relationships**: How it connects to other objects * **Constraints**: Any validation rules or requirements **Example descriptions:** ``` Create a schema for network switches that includes: - Basic device information (name, model, serial number) - Management IP address - Physical location reference - Multiple network interfaces - VLAN configuration capabilities ``` ``` Model a cloud instance with: - Instance type and size - Operating system details - Network interfaces with public/private IPs - Associated security groups - Tags for metadata ``` ### Step 2: Review generated schema[​](#step-2-review-generated-schema "Direct link to Step 2: Review generated schema") The AI will generate a complete Infrahub schema in YAML format. Review: * **Schema name and namespace** - Ensure they follow your naming conventions * **Attributes** - Check data types and validation rules * **Relationships** - Verify connections to other schemas make sense * **Inheritance** - Confirm the schema inherits from appropriate base schemas ### Step 3: Refine and iterate[​](#step-3-refine-and-iterate "Direct link to Step 3: Refine and iterate") If the generated schema isn't quite right: * **Add more details** to your description * **Specify constraints** more clearly * **Mention missing attributes** or relationships * **Reference existing schemas** that should be related The AI learns from your feedback and will generate improved schemas. ## Advanced features[​](#advanced-features "Direct link to Advanced features") ### Schema inheritance[​](#schema-inheritance "Direct link to Schema inheritance") Emma understands Infrahub's inheritance model. You can specify base schemas: ``` Create a server schema that inherits from the base Device schema and adds CPU, memory, and storage specifications ``` ### Complex relationships[​](#complex-relationships "Direct link to Complex relationships") Define sophisticated relationships between schemas: ``` Create a cluster schema that: - Has multiple nodes (many-to-one relationship with servers) - Belongs to a single environment - Can span multiple availability zones - Has a load balancer that references multiple backend servers ``` ### Validation rules[​](#validation-rules "Direct link to Validation rules") Specify validation requirements: ``` Create an IP address schema with: - IPv4 and IPv6 address validation - Subnet mask validation - Ensure addresses are within allowed ranges - Prevent duplicate address assignment ``` ## Best practices[​](#best-practices "Direct link to Best practices") ### Writing effective descriptions[​](#writing-effective-descriptions "Direct link to Writing effective descriptions") 1. **Be specific**: Vague descriptions lead to generic schemas 2. **Use industry terms**: The AI understands networking, cloud, and infrastructure terminology 3. **Mention relationships**: Always describe how your schema connects to others 4. **Consider the data lifecycle**: Think about how data will be created, updated, and deleted ### Schema design principles[​](#schema-design-principles "Direct link to Schema design principles") 1. **Start with basics**: Begin with basic attributes and add complexity iteratively 2. **Follow conventions**: Use consistent naming and follow Infrahub best practices 3. **Consider inheritance**: Leverage base schemas to avoid duplication 4. **Plan relationships**: Think about how schemas connect before creating them ### Integration with existing schemas[​](#integration-with-existing-schemas "Direct link to Integration with existing schemas") * **Reference the Schema Library**: Mention existing schemas in your descriptions * **Check dependencies**: Ensure required base schemas are loaded first * **Validate relationships**: Verify that referenced schemas exist in your Infrahub instance * **Test thoroughly**: Always test generated schemas before using in production ## Limitations and considerations[​](#limitations-and-considerations "Direct link to Limitations and considerations") ### AI limitations[​](#ai-limitations "Direct link to AI limitations") * **Context awareness**: The AI doesn't know your specific environment setup * **Validation**: Always review generated schemas for accuracy * **Best practices**: The AI follows general practices but may not match your specific standards * **Complex logic**: Very complex business rules may need manual refinement ### Schema complexity[​](#schema-complexity "Direct link to Schema complexity") * **Start with core attributes**: Complex schemas with many relationships can be challenging * **Iterate**: Build complex schemas in stages * **Test**: Validate each iteration before adding more complexity ### Performance considerations[​](#performance-considerations "Direct link to Performance considerations") * **API calls**: Each generation requires an OpenAI API call * **Response time**: Complex descriptions may take longer to process * **Rate limits**: Be aware of OpenAI API rate limitations ## Integration with other Emma features[​](#integration-with-other-emma-features "Direct link to Integration with other Emma features") The Schema Builder integrates seamlessly with other Emma features: * **Schema Loader**: Generated schemas can be loaded directly into Infrahub * **Schema Visualizer**: View the structure and relationships of generated schemas * **Data Importer**: Use generated schemas as targets for data import * **Schema Library**: Save generated schemas to your local library ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Common issues[​](#common-issues "Direct link to Common issues") **AI generates incorrect schemas:** * Provide more specific descriptions * Include examples of desired attributes * Reference existing schemas for context **Missing relationships:** * Explicitly mention all required relationships * Verify referenced schemas exist * Check relationship cardinality requirements **Validation errors:** * Review Infrahub schema requirements * Check attribute data types * Verify namespace and naming conventions **Performance issues:** * Simplify complex descriptions * Break large schemas into smaller components * Check network connectivity to OpenAI API For more troubleshooting help, see the [Troubleshooting Guide](/emma/reference/troubleshooting.md). ## Next steps[​](#next-steps "Direct link to Next steps") * Try the [Building Your First Schema](/emma/guides/building-your-first-schema.md) guide * Explore the [Schema Library](https://docs.infrahub.app/schema-library) for examples and templates * Learn about [Schema Management](/emma/features/schema-management.md) best practices --- # Schema Management Emma provides comprehensive tools for managing Infrahub schemas throughout their lifecycle. From loading and visualizing schemas to organizing them in libraries, Emma makes schema management accessible and efficient. ## Overview[​](#overview "Direct link to Overview") Schema management in Emma encompasses: * **Schema Loading** - Installing schemas into your Infrahub instance * **Schema Visualization** - Understanding schema relationships and structure * **Schema Library** - Organizing and browsing available schemas * **Version Control** - Managing schema changes over time ## Schema loader[​](#schema-loader "Direct link to Schema loader") The Schema Loader allows you to install schemas into your Infrahub instance from various sources. ![Schema Loader](/assets/images/schema_loader-48332680a3b72f079c92cf7f1df1b076.png) ### Loading sources[​](#loading-sources "Direct link to Loading sources") * Schema Library * YAML Editor * File Upload Load schemas from Emma's built-in library: 1. **Browse available schemas** in categories (Base, Extensions, Experimental) 2. **Select schemas** to load (multiple selection supported) 3. **Review dependencies** - Emma shows required base schemas 4. **Load in dependency order** - Emma automatically orders schema loading **Benefits:** * Pre-tested, community-validated schemas * Proper dependency management * Documentation and examples included * Regular updates and improvements Load schemas from YAML definitions: 1. **Paste or type YAML** directly into the editor 2. **Validate syntax** - Emma checks YAML formatting 3. **Preview schema structure** before loading 4. **Load individual or multiple schemas** from a single YAML file **Use cases:** * Loading custom schemas * Testing schema modifications * Importing schemas from other sources * Quick schema prototyping Upload schema files from your local system: 1. **Select YAML files** (single or multiple) 2. **Review file contents** before loading 3. **Validate and load** with dependency checking 4. **Track loading progress** for multiple files **Supported formats:** * Individual .yaml or .yml files * Multiple files in zip archives * Schema collections from exports ### Dependency management[​](#dependency-management "Direct link to Dependency management") Emma automatically handles schema dependencies: **Dependency Detection:** * Analyzes `inherit_from` relationships * Identifies referenced schemas in relationships * Detects circular dependencies **Load Ordering:** * Loads base schemas before dependent schemas * Provides clear dependency visualization * Prevents load failures due to missing dependencies **Conflict Resolution:** * Warns about schema name conflicts * Handles version differences * Provides override options for updates ### Validation and error handling[​](#validation-and-error-handling "Direct link to Validation and error handling") The Schema Loader validates schemas before installation: **Syntax Validation:** * YAML formatting and structure * Required field presence * Data type correctness **Semantic Validation:** * Schema naming conventions * Relationship target validity * Attribute definition consistency **Infrahub Validation:** * Compatibility with Infrahub version * Permission checks * Namespace conflicts ## Schema visualizer[​](#schema-visualizer "Direct link to Schema visualizer") The Schema Visualizer provides graphical representations of your schema structure and relationships. ![Schema Visualizer](/assets/images/schema_visualizer-7b17e46228491d6e7a2c4f3e29e05ae1.png) ### Visualization types[​](#visualization-types "Direct link to Visualization types") **Graph View:** * Interactive network diagram of schemas * Shows inheritance and relationship connections * Allows zooming and panning for large schema sets * Color-coded by schema type or namespace **Tree View:** * Hierarchical display of schema inheritance * Expandable/collapsible nodes * Shows parent-child relationships clearly * Useful for understanding schema hierarchies **List View:** * Tabular display of all schemas * Sortable and filterable columns * Shows key schema metadata * Quick access to schema details ### Interactive features[​](#interactive-features "Direct link to Interactive features") **Schema Exploration:** * Click schemas to view detailed information * Hover for quick attribute and relationship previews * Expand/collapse related schema groups * Navigate between connected schemas **Filtering and Search:** * Filter by schema type, namespace, or category * Search by schema name or description * Hide/show specific relationship types * Focus on schema subsets **Export Options:** * Export visualizations as images (PNG, SVG) * Generate schema documentation * Create relationship reports * Export schema lists as CSV ### Analysis tools[​](#analysis-tools "Direct link to Analysis tools") **Impact Analysis:** * Show what schemas depend on a selected schema * Identify unused schemas * Find circular dependencies * Analyze schema complexity metrics **Relationship Analysis:** * Count incoming and outgoing relationships * Identify highly connected schemas * Find isolated schema groups * Analyze relationship patterns ## Schema library integration[​](#schema-library-integration "Direct link to Schema library integration") Emma's Schema Library provides curated, ready-to-use schemas for common infrastructure components. ### Library organization[​](#library-organization "Direct link to Library organization") **Base Schemas:** * Fundamental infrastructure components * Required by most other schemas * Examples: Device, Location, Organization, Person **Extensions:** * Specialized schemas for specific domains * Build on base schemas * Examples: Routing protocols, Security policies, Cloud resources **Experimental:** * Cutting-edge schemas for new technologies * Community contributions * May change significantly ### Using library schemas[​](#using-library-schemas "Direct link to Using library schemas") **Browse and Discover:** * Search by category, technology, or use case * View schema descriptions and examples * Check compatibility with your environment * Review community ratings and feedback **Customize and Extend:** * Copy library schemas as starting points * Modify attributes and relationships * Add organization-specific extensions * Maintain local variations **Contribute Back:** * Share useful schemas with the community * Submit improvements to existing schemas * Participate in schema standardization efforts ## Best practices[​](#best-practices "Direct link to Best practices") ### Schema design[​](#schema-design "Direct link to Schema design") **Start with Standards:** * Use library schemas as foundations * Follow established naming conventions * Design for reusability and extension * Document schema purposes and constraints **Plan Dependencies:** * Load base schemas first * Minimize circular dependencies * Use inheritance appropriately * Design clear relationship hierarchies **Version Management:** * Use Infrahub branches for schema changes * Test schema modifications thoroughly * Plan migration strategies for breaking changes * Maintain schema documentation ### Performance considerations[​](#performance-considerations "Direct link to Performance considerations") **Large Schema Sets:** * Load schemas in batches * Use dependency ordering * Monitor Infrahub performance during loads * Consider schema consolidation **Complex Relationships:** * Avoid overly complex relationship networks * Use visualization tools to understand complexity * Simplify where possible * Document complex relationships clearly ### Team collaboration[​](#team-collaboration "Direct link to Team collaboration") **Schema Governance:** * Establish schema review processes * Define naming and design standards * Use version control for schema files * Document schema decisions and rationale **Knowledge Sharing:** * Use schema visualization for training * Maintain schema documentation * Share library schemas across teams * Regular schema review meetings ## Advanced features[​](#advanced-features "Direct link to Advanced features") ### Schema diff and comparison[​](#schema-diff-and-comparison "Direct link to Schema diff and comparison") Compare schemas across branches or versions: * Side-by-side schema comparisons * Highlight attribute and relationship changes * Impact analysis for modifications * Migration planning assistance ### Bulk operations[​](#bulk-operations "Direct link to Bulk operations") Manage multiple schemas efficiently: * Bulk schema loading and updates * Batch validation and testing * Mass schema exports * Automated dependency resolution ### Integration APIs[​](#integration-apis "Direct link to Integration APIs") Programmatic schema management: * REST APIs for schema operations * GraphQL queries for schema metadata * Webhook notifications for schema changes * CI/CD integration capabilities ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Common issues[​](#common-issues "Direct link to Common issues") **Schema Load Failures:** * Check dependency requirements * Verify YAML syntax * Ensure proper permissions * Review Infrahub compatibility **Visualization Problems:** * Clear browser cache * Check for large schema sets * Verify network connectivity * Update browser if needed **Performance Issues:** * Reduce loaded schema count * Use filtering in visualizations * Load schemas in smaller batches * Monitor system resources For detailed troubleshooting, see the [Troubleshooting Guide](/emma/reference/troubleshooting.md). ## Next steps[​](#next-steps "Direct link to Next steps") * Explore the [Schema Library](https://docs.infrahub.app/schema-library) for available schemas * Try [Building Your First Schema](/emma/guides/building-your-first-schema.md) * Learn about [Data Import & Export](/emma/features/data-import-export.md) for populating your schemas * Check out the [Schema Builder](/emma/features/schema-builder.md) for creating custom schemas --- # Configuration Emma connects to Infrahub using standard environment variables and configuration options. This guide covers all the ways to configure Emma for your environment. ## Environment variables[​](#environment-variables "Direct link to Environment variables") Emma uses the same environment variables as other Infrahub tools for consistency: ### Required configuration[​](#required-configuration "Direct link to Required configuration") | Variable | Description | Example | | -------------------- | ----------------------------- | -------------------------------------- | | `INFRAHUB_ADDRESS` | URL of your Infrahub instance | `http://localhost:8000` | | `INFRAHUB_API_TOKEN` | API token for authentication | `06438eb2-8019-4776-878c-0941b1f1d1ec` | ### Optional configuration[​](#optional-configuration "Direct link to Optional configuration") | Variable | Description | Default | Example | | -------------------- | --------------------------------------------- | ------- | -------------------------------- | | `EMMA_FEATURE_FLAGS` | Comma-separated list of experimental features | `""` | `query_builder,template_builder` | | `INFRAHUB_TIMEOUT` | API request timeout in seconds | `30` | `60` | ## Setting environment variables[​](#setting-environment-variables "Direct link to Setting environment variables") * Local Development * Docker * Production For local development, create a `.env` file in the Emma root directory: ``` # .env file INFRAHUB_ADDRESS=http://localhost:8000 INFRAHUB_API_TOKEN=your-api-token-here EMMA_FEATURE_FLAGS=query_builder ``` Or export them in your shell: ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_API_TOKEN="06438eb2-8019-4776-878c-0941b1f1d1ec" export EMMA_FEATURE_FLAGS="query_builder,template_builder" ``` When using Docker, set environment variables in your `docker-compose.yml`: ``` version: '3.8' services: emma: build: . ports: - "8501:8501" environment: - INFRAHUB_ADDRESS=http://infrahub:8000 - INFRAHUB_API_TOKEN=your-api-token-here - EMMA_FEATURE_FLAGS=query_builder ``` Or use an environment file: ``` services: emma: build: . env_file: - .env ``` For production deployments, use secure methods to inject environment variables: **Kubernetes:** ``` apiVersion: apps/v1 kind: Deployment metadata: name: emma spec: template: spec: containers: - name: emma env: - name: INFRAHUB_ADDRESS value: "https://infrahub.example.com" - name: INFRAHUB_API_TOKEN valueFrom: secretKeyRef: name: emma-secrets key: api-token ``` **Docker Swarm:** ``` version: '3.8' services: emma: environment: - INFRAHUB_ADDRESS=https://infrahub.example.com - INFRAHUB_API_TOKEN_FILE=/run/secrets/api_token secrets: - api_token ``` ## Getting an API token[​](#getting-an-api-token "Direct link to Getting an API token") To get an API token from Infrahub: 1. **Log into your Infrahub instance** at your `INFRAHUB_ADDRESS` 2. **Navigate to Settings** → **API Tokens** 3. **Create a new token** with appropriate permissions 4. **Copy the token** and use it as your `INFRAHUB_API_TOKEN` warning API tokens provide full access to your Infrahub instance. Store them securely and never commit them to version control. ## In-app configuration[​](#in-app-configuration "Direct link to In-app configuration") Emma also provides configuration options within the web interface: ### Infrahub connection settings[​](#infrahub-connection-settings "Direct link to Infrahub connection settings") * **Address**: Override the environment variable for the current session * **Branch**: Select which Infrahub branch to work with * **Timeout**: Adjust request timeout for slow connections ### AI assistant settings[​](#ai-assistant-settings "Direct link to AI assistant settings") * **OpenAI API Key**: Configure AI features for schema generation * **Model Selection**: Choose between different AI models * **Temperature**: Adjust AI creativity level Access these settings through the sidebar in any Emma page. ## Feature flags[​](#feature-flags "Direct link to Feature flags") Emma uses feature flags to enable experimental functionality: | Flag | Description | Status | | ------------------ | ------------------------------- | ------ | | `query_builder` | GraphQL query builder interface | Alpha | | `template_builder` | Template creation tools | Alpha | Enable feature flags by setting `EMMA_FEATURE_FLAGS`: ``` export EMMA_FEATURE_FLAGS="query_builder,template_builder" ``` ## Next steps[​](#next-steps "Direct link to Next steps") With Emma configured, you're ready to take your [First Steps](/emma/getting-started/first-steps.md) with the application. ## Configuration validation[​](#configuration-validation "Direct link to Configuration validation") Emma validates your configuration on startup. Common configuration issues: * **Invalid Infrahub address**: Check the URL format and network connectivity * **Authentication errors**: Verify your API token is valid and has necessary permissions * **Connection timeouts**: Increase timeout values for slow networks For detailed troubleshooting, see the [Troubleshooting Guide](/emma/reference/troubleshooting.md). --- # First Steps Welcome to Emma! This guide will walk you through your first experience using Emma to interact with Infrahub. ## Opening Emma[​](#opening-emma "Direct link to Opening Emma") 1. **Navigate to Emma** at `http://localhost:8501` (or your configured port) 2. **Verify connection** - Emma should show a connection status to your Infrahub instance 3. **Select a branch** - Choose which Infrahub branch to work with (usually `main`) ![Emma Homepage](/assets/images/home_page-688ffc9365ffba7db2bec7b3ee5c6c4f.png) ## Understanding Emma's interface[​](#understanding-emmas-interface "Direct link to Understanding Emma's interface") Emma is organized into several key areas accessible from the sidebar: ### Core features[​](#core-features "Direct link to Core features") * **🏗️ Schema Builder** - AI-powered schema creation and editing * **📁 Schema Library** - Browse and use pre-built schema templates * **🔄 Schema Loader** - Load schemas into your Infrahub instance * **👁️ Schema Visualizer** - Visual representation of your schemas * **📥 Data Importer** - Import CSV data into Infrahub * **📤 Data Exporter** - Export Infrahub data to CSV ### Experimental features[​](#experimental-features "Direct link to Experimental features") * **🔍 Query Builder** - Build GraphQL queries (feature flag required) * **📋 Template Builder** - Create templates (feature flag required) ## Your first task: explore the schema library[​](#your-first-task-explore-the-schema-library "Direct link to Your first task: explore the schema library") Let's start by exploring Emma's schema library: 1. **Click "Schema Library"** in the sidebar 2. **Browse available schemas** - You'll see categories like Base, Extensions, and Experimental 3. **Select a schema** to view its details, such as the "Device" schema from Base/DCIM 4. **Review the schema structure** - Notice how it defines attributes, relationships, and inheritance This gives you an understanding of how Infrahub schemas are structured. ## Load your first schema[​](#load-your-first-schema "Direct link to Load your first schema") Next, let's load a schema into Infrahub: 1. **Navigate to "Schema Loader"** 2. **Select schemas** from the library (start with Base schemas like Organization and Location) 3. **Preview the schema** to understand what will be loaded 4. **Click "Load Schema"** to add it to your Infrahub instance tip Start with foundational schemas like Organization and Location before adding more complex schemas that depend on them. ## Visualize your schema[​](#visualize-your-schema "Direct link to Visualize your schema") After loading schemas: 1. **Go to "Schema Visualizer"** 2. **View the graphical representation** of your schema relationships 3. **Explore connections** between different schema objects 4. **Use filters** to focus on specific schema types or relationships ## Import sample data[​](#import-sample-data "Direct link to Import sample data") To see Emma in action with real data: 1. **Prepare a CSV file** with data matching your schema structure 2. **Navigate to "Data Importer"** 3. **Upload your CSV** and map columns to schema attributes 4. **Preview the import** to verify mappings are correct 5. **Execute the import** to add data to Infrahub ## Key concepts to understand[​](#key-concepts-to-understand "Direct link to Key concepts to understand") ### Schemas vs data[​](#schemas-vs-data "Direct link to Schemas vs data") * **Schemas** define the structure and relationships of your infrastructure models * **Data** is the actual instances that follow those schemas (specific devices, locations, etc.) ### Branches[​](#branches "Direct link to Branches") * Emma works with Infrahub branches, similar to Git branches * Changes can be made in development branches before merging to main * Always be aware of which branch you're working in ### AI integration[​](#ai-integration "Direct link to AI integration") * Emma's Schema Builder uses AI to help create schemas from natural language * The AI understands infrastructure concepts and can suggest appropriate attributes and relationships * Review AI-generated schemas carefully before using them in production ## Next steps[​](#next-steps "Direct link to Next steps") Now that you've completed your first steps: 1. **Explore Features** - Learn about each feature in detail in the [Features](/emma/features/schema-builder.md) section 2. **Follow Guides** - Try the step-by-step [guides](/emma/guides/building-your-first-schema.md) for common tasks 3. **Build Your Own** - Start creating schemas and importing data for your infrastructure ## Getting help[​](#getting-help "Direct link to Getting help") If you encounter issues: * Check the [Troubleshooting Guide](/emma/reference/troubleshooting.md) * Review feature-specific documentation in the Features section * Remember that Emma is experimental - some features may not work as expected Emma is designed to make working with Infrahub more accessible and efficient. Take your time to explore each feature and don't hesitate to experiment! --- # Installation Emma is an experimental AI-powered assistant for Infrahub that helps you manage infrastructure schemas and data through an intuitive web interface. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before installing Emma, ensure you have: * **Infrahub instance**: Emma requires a running Infrahub instance to connect to * **Docker** (for containerized deployment) or **Python 3.10+** and **uv** (for local development) ## Installation methods[​](#installation-methods "Direct link to Installation methods") * Quick Start (Recommended) * Docker Compose * Local Development The easiest way to get started is using the Infrahub quickstart with Emma included: ``` # Download and start Infrahub + Emma with Docker Compose curl https://infrahub.opsmill.io/latest-emma | docker compose -f - up -d ``` Or if you want to add Emma on to a specific release of Infrahub: ``` # Download and start Infrahub + Emma with Docker Compose curl https://infrahub.opsmill.io/1.7.6-emma | docker compose -f - up -d ``` This command will: * Download the latest Infrahub and Emma configuration * Start both services with proper networking * Make Emma available at `http://localhost:8501` * Make Infrahub available at `http://localhost:8000` tip Update the version number in the URL to the [latest Infrahub release](https://github.com/opsmill/infrahub/tags) for the most recent features. To run Emma alongside your existing Infrahub setup: 1. **Clone the Emma repository:** ``` git clone https://github.com/opsmill/emma.git cd emma ``` 2. **Start Emma with Docker Compose:** ``` docker compose up --build -d ``` 3. **Connect to Infrahub network** (if running Infrahub in containers): ``` # Find your Infrahub network name docker network ls # Connect Emma to the Infrahub network docker network connect emma-emma-1 ``` Emma will be available at `http://localhost:8501` For development or custom setups: 1. **Clone the repository:** ``` git clone https://github.com/opsmill/emma.git cd emma ``` 2. **Install dependencies with uv:** ``` uv sync ``` 3. **Run Emma:** ``` uv run streamlit run main.py ``` Emma will start on `http://localhost:8501` ## Next steps[​](#next-steps "Direct link to Next steps") After installation, proceed to [Configuration](/emma/getting-started/configuration.md) to set up your connection to Infrahub. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") If you encounter issues during installation: * **Port conflicts**: Ensure ports 8501 (Emma) and 8000 (Infrahub) are available * **Docker issues**: Check Docker is running and you have sufficient resources allocated * **Network connectivity**: Verify Emma can reach your Infrahub instance For more detailed troubleshooting, see the [Troubleshooting Guide](/emma/reference/troubleshooting.md). --- # Building Your First Schema This guide walks you through creating your first Infrahub schema using Emma's AI Schema Builder. We'll create a practical schema for modeling network devices. ## What we'll build[​](#what-well-build "Direct link to What we'll build") By the end of this guide, you'll have created a "NetworkDevice" schema that includes: * Basic device information (name, model, serial number) * Network configuration (management IP, location) * Operational status and metadata * Relationships to location and vendor schemas ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting, ensure you have: * Emma running and connected to your Infrahub instance * Basic understanding of what schemas are (see [First Steps](/emma/getting-started/first-steps.md)) * An OpenAI API key configured for AI features ## Step 1: Plan your schema[​](#step-1-plan-your-schema "Direct link to Step 1: Plan your schema") Before jumping into the Schema Builder, think about what you want to model: **Core Attributes:** * `name` - Unique device identifier * `model` - Device model/part number * `serial_number` - Hardware serial number * `management_ip` - IP address for device management * `status` - Operational status (active, maintenance, decommissioned) **Relationships:** * `location` - Reference to a Location object * `vendor` - Reference to a Vendor/Organization object **Metadata:** * `description` - Human-readable description * `tags` - Flexible tagging system ## Step 2: Open the schema builder[​](#step-2-open-the-schema-builder "Direct link to Step 2: Open the schema builder") 1. **Navigate to Schema Builder** in Emma's sidebar 2. **Verify AI configuration** - Ensure your OpenAI API key is set 3. **Check Infrahub connection** - Confirm you're connected to the right instance and branch ## Step 3: Describe your schema[​](#step-3-describe-your-schema "Direct link to Step 3: Describe your schema") In the Schema Builder interface, enter this description: ``` Create a NetworkDevice schema for modeling network infrastructure devices with the following requirements: Core Information: - name: unique identifier for the device - model: device model or part number - serial_number: hardware serial number - description: human-readable description Network Configuration: - management_ip: IPv4 address for device management - management_interface: name of the management interface Location and Organization: - location: relationship to a Location object - vendor: relationship to an Organization object representing the device vendor Operational Status: - status: enumeration with values: active, maintenance, decommissioned, planned - commissioned_date: when the device was put into service - last_seen: timestamp of last successful connection The schema should inherit from the base Device schema if available, and include proper validation for IP addresses and required fields. ``` ## Step 4: Review the generated schema[​](#step-4-review-the-generated-schema "Direct link to Step 4: Review the generated schema") The AI will generate a complete YAML schema. Review it carefully: ### Check the schema structure[​](#check-the-schema-structure "Direct link to Check the schema structure") ``` # Example generated schema (yours may vary) --- version: '1.0' schemas: - name: NetworkDevice namespace: Infrastructure description: Network infrastructure device inherit_from: - CoreDevice attributes: - name: name kind: Text unique: true optional: false - name: model kind: Text optional: true # ... more attributes relationships: - name: location peer: LocationGeneric optional: true cardinality: one # ... more relationships ``` ### Verify key elements[​](#verify-key-elements "Direct link to Verify key elements") ✅ **Naming**: Schema name follows your conventions ✅ **Namespace**: Appropriate namespace for organization ✅ **Inheritance**: Inherits from appropriate base schema ✅ **Attributes**: All required attributes are present with correct types ✅ **Relationships**: References to Location and Organization schemas ✅ **Validation**: IP address validation and required field constraints ## Step 5: Refine if needed[​](#step-5-refine-if-needed "Direct link to Step 5: Refine if needed") If the generated schema needs adjustments, refine your description: **Add missing details:** ``` Also include: - hostname: DNS hostname for the device - firmware_version: current firmware version - power_consumption: power usage in watts ``` **Adjust relationships:** ``` The location relationship should be required (not optional) and the vendor relationship should reference Organization objects with a role filter for vendors only. ``` **Modify validation:** ``` The management_ip should validate as a proper IPv4 address, and the serial_number should be required and unique. ``` ## Step 6: Load the schema[​](#step-6-load-the-schema "Direct link to Step 6: Load the schema") Once you're satisfied with the generated schema: 1. **Copy the YAML** from the Schema Builder output 2. **Navigate to Schema Loader** in Emma's sidebar 3. **Paste the schema** into the YAML editor 4. **Preview the schema** to verify it looks correct 5. **Load into Infrahub** by clicking "Load Schema" ## Step 7: Verify the schema[​](#step-7-verify-the-schema "Direct link to Step 7: Verify the schema") After loading, verify your schema worked correctly: ### Using the schema visualizer[​](#using-the-schema-visualizer "Direct link to Using the schema visualizer") 1. **Go to Schema Visualizer** 2. **Find your NetworkDevice schema** in the schema list 3. **Examine the relationships** to Location and Organization 4. **Verify the attributes** appear correctly ### Using Infrahub directly[​](#using-infrahub-directly "Direct link to Using Infrahub directly") 1. **Open your Infrahub instance** in a web browser 2. **Navigate to Schema** section 3. **Find NetworkDevice** in the schema list 4. **Review the generated schema** definition ## Step 8: Test with sample data[​](#step-8-test-with-sample-data "Direct link to Step 8: Test with sample data") Now test your schema by adding some sample data: ### Using the data importer[​](#using-the-data-importer "Direct link to Using the data importer") Create a CSV file: ``` name,model,serial_number,management_ip,status,description switch-01,Cisco 3850,FCW2140L0EF,192.168.1.10,active,Core distribution switch router-01,Cisco ISR4431,FJC2140L0AG,192.168.1.1,active,Main internet gateway switch-02,Cisco 3850,FCW2140L0EH,192.168.1.11,maintenance,Access layer switch ``` Then: 1. **Go to Data Importer** in Emma 2. **Upload your CSV file** 3. **Map columns** to your NetworkDevice schema attributes 4. **Preview the import** to verify mappings 5. **Execute the import** to create your first devices ### Verify the data[​](#verify-the-data "Direct link to Verify the data") 1. **Check Infrahub** to see your imported devices 2. **Use the Data Exporter** to export and verify the data 3. **Navigate between relationships** to see how location and vendor references work ## Common issues and solutions[​](#common-issues-and-solutions "Direct link to Common issues and solutions") ### Schema generation issues[​](#schema-generation-issues "Direct link to Schema generation issues") **Problem**: AI generates incorrect attribute types **Solution**: Be more specific about data types in your description, for example: "IPv4 address" instead of "IP address" **Problem**: Missing relationships **Solution**: Explicitly mention all required relationships and their cardinality **Problem**: Schema doesn't inherit correctly **Solution**: Check if base schemas exist in your Infrahub instance first ### Schema loading issues[​](#schema-loading-issues "Direct link to Schema loading issues") **Problem**: Validation errors when loading **Solution**: Check that referenced schemas (Location, Organization) exist in Infrahub **Problem**: Namespace conflicts **Solution**: Use unique namespaces or check existing schema names ### Data import issues[​](#data-import-issues "Direct link to Data import issues") **Problem**: Relationship resolution fails **Solution**: Ensure referenced objects (locations, vendors) exist before importing devices **Problem**: Validation errors on IP addresses **Solution**: Verify IP address format in your CSV data ## Next steps[​](#next-steps "Direct link to Next steps") Congratulations! You've created your first schema with Emma. Here's what to explore next: ### Extend your schema[​](#extend-your-schema "Direct link to Extend your schema") * **Add more attributes** like rack position, power requirements, or warranty information * **Create related schemas** for interfaces, VLANs, or IP addresses * **Model hierarchical relationships** like chassis and line cards ### Build a complete model[​](#build-a-complete-model "Direct link to Build a complete model") * **Create supporting schemas** for racks, cabinets, and rooms * **Add network connectivity** with cable and interface schemas * **Include service modeling** for applications and services ### Advanced features[​](#advanced-features "Direct link to Advanced features") * **Try the Schema Library** to find pre-built schemas you can extend * **Use the Schema Visualizer** to understand complex relationships * **Explore experimental features** like the Query Builder ### Best practices[​](#best-practices "Direct link to Best practices") * **Start with basics** and add complexity gradually * **Plan relationships** before creating dependent schemas * **Test with sample data** before importing production data * **Document your schemas** for team collaboration ## Resources[​](#resources "Direct link to Resources") * [Schema Builder Reference](/emma/features/schema-builder.md) - Detailed Schema Builder documentation * [Schema Library](https://docs.infrahub.app/schema-library) - Pre-built schemas and examples * [Data Import & Export](/emma/features/data-import-export.md) - Advanced data import techniques * [Infrahub Schema Documentation](https://docs.infrahub.app/reference/schema/) - Official schema reference You're now ready to build more complex schemas and models with Emma! --- # Configuration Reference This reference covers all configuration options available in Emma, including environment variables, feature flags, and runtime settings. ## Environment variables[​](#environment-variables "Direct link to Environment variables") ### Required variables[​](#required-variables "Direct link to Required variables") | Variable | Description | Example | Notes | | -------------------- | ------------------------ | -------------------------------------- | ---------------------------- | | `INFRAHUB_ADDRESS` | URL of Infrahub instance | `http://localhost:8000` | Must be accessible from Emma | | `INFRAHUB_API_TOKEN` | API authentication token | `06438eb2-8019-4776-878c-0941b1f1d1ec` | Generate in Infrahub admin | ### Optional variables[​](#optional-variables "Direct link to Optional variables") | Variable | Description | Default | Example | | -------------------------- | ------------------------------------- | --------- | -------------------------------- | | `INFRAHUB_TIMEOUT` | API request timeout (seconds) | `30` | `60` | | `INFRAHUB_RETRIES` | Number of API retry attempts | `3` | `5` | | `EMMA_FEATURE_FLAGS` | Comma-separated experimental features | `""` | `query_builder,template_builder` | | `STREAMLIT_SERVER_PORT` | Port for Emma web interface | `8501` | `8080` | | `STREAMLIT_SERVER_ADDRESS` | Interface to bind to | `0.0.0.0` | `127.0.0.1` | ### AI configuration[​](#ai-configuration "Direct link to AI configuration") | Variable | Description | Default | Example | | -------------------- | ------------------------------ | --------------- | -------- | | `OPENAI_API_KEY` | OpenAI API key for AI features | `""` | `sk-...` | | `OPENAI_MODEL` | AI model to use | `gpt-3.5-turbo` | `gpt-4` | | `OPENAI_TEMPERATURE` | AI creativity level (0.0-2.0) | `0.7` | `0.3` | | `OPENAI_MAX_TOKENS` | Maximum tokens per AI response | `2000` | `4000` | ### Logging configuration[​](#logging-configuration "Direct link to Logging configuration") | Variable | Description | Default | Example | | ------------------------ | -------------------------- | ------- | ------- | | `STREAMLIT_LOGGER_LEVEL` | Streamlit logging level | `INFO` | `DEBUG` | | `PYTHONPATH` | Python path for modules | `""` | `/app` | | `LOG_LEVEL` | Emma application log level | `INFO` | `DEBUG` | ## Feature flags[​](#feature-flags "Direct link to Feature flags") Feature flags enable experimental functionality in Emma. Set via `EMMA_FEATURE_FLAGS` environment variable. ### Available flags[​](#available-flags "Direct link to Available flags") | Flag | Description | Status | Dependencies | | ------------------ | -------------------------------- | ------------ | -------------- | | `query_builder` | GraphQL query builder interface | Alpha | None | | `template_builder` | Template creation and management | Alpha | None | | `advanced_import` | Enhanced data import features | Beta | None | | `bulk_operations` | Bulk schema and data operations | Alpha | None | | `ai_suggestions` | AI-powered suggestions in UI | Experimental | OpenAI API key | ### Setting feature flags[​](#setting-feature-flags "Direct link to Setting feature flags") **Single flag:** ``` export EMMA_FEATURE_FLAGS="query_builder" ``` **Multiple flags:** ``` export EMMA_FEATURE_FLAGS="query_builder,template_builder,ai_suggestions" ``` **Docker Compose:** ``` environment: - EMMA_FEATURE_FLAGS=query_builder,template_builder ``` **Kubernetes:** ``` env: - name: EMMA_FEATURE_FLAGS value: "query_builder,template_builder" ``` ## Runtime configuration[​](#runtime-configuration "Direct link to Runtime configuration") ### In-app settings[​](#in-app-settings "Direct link to In-app settings") Emma provides runtime configuration through the web interface: #### Connection settings[​](#connection-settings "Direct link to Connection settings") * **Infrahub Address**: Override environment variable for current session * **API Token**: Override environment variable for current session * **Branch**: Select Infrahub branch to work with * **Timeout**: Adjust request timeout for slow connections #### AI settings[​](#ai-settings "Direct link to AI settings") * **OpenAI API Key**: Configure AI features * **Model Selection**: Choose AI model (gpt-3.5-turbo, gpt-4) * **Temperature**: Adjust AI creativity (0.0 = deterministic, 2.0 = very creative) * **Max Tokens**: Limit AI response length #### Import/export settings[​](#importexport-settings "Direct link to Import/export settings") * **Batch Size**: Records to process per batch (default: 100) * **Validation Level**: Strict, moderate, or lenient validation * **Error Handling**: Stop on error or continue with warnings * **Relationship Resolution**: Automatic or manual relationship handling #### Display settings[​](#display-settings "Direct link to Display settings") * **Theme**: Light or dark mode * **Page Layout**: Wide or narrow layout * **Auto-refresh**: Automatic data refresh interval * **Pagination**: Number of items per page ### Configuration persistence[​](#configuration-persistence "Direct link to Configuration persistence") Settings configured in the Emma interface are stored in: **Local Development:** * Browser session storage (temporary) * Browser local storage (persistent across sessions) **Docker Deployment:** * Volume-mounted configuration file * Environment variables take precedence **Kubernetes:** * ConfigMaps for application settings * Secrets for sensitive data (API keys, tokens) ## Deployment configurations[​](#deployment-configurations "Direct link to Deployment configurations") ### Docker Compose[​](#docker-compose "Direct link to Docker Compose") Complete docker-compose.yml configuration: ``` version: '3.8' services: emma: build: . ports: - "8501:8501" environment: # Required - INFRAHUB_ADDRESS=http://infrahub:8000 - INFRAHUB_API_TOKEN=${INFRAHUB_API_TOKEN} # Optional - INFRAHUB_TIMEOUT=60 - EMMA_FEATURE_FLAGS=query_builder,template_builder - OPENAI_API_KEY=${OPENAI_API_KEY} # Streamlit Configuration - STREAMLIT_SERVER_PORT=8501 - STREAMLIT_SERVER_ADDRESS=0.0.0.0 - STREAMLIT_LOGGER_LEVEL=INFO volumes: - emma-config:/app/config networks: - infrahub-network depends_on: - infrahub restart: unless-stopped volumes: emma-config: networks: infrahub-network: external: true ``` ### Kubernetes[​](#kubernetes "Direct link to Kubernetes") Emma deployment configuration: ``` apiVersion: apps/v1 kind: Deployment metadata: name: emma labels: app: emma spec: replicas: 1 selector: matchLabels: app: emma template: metadata: labels: app: emma spec: containers: - name: emma image: emma:latest ports: - containerPort: 8501 env: - name: INFRAHUB_ADDRESS value: "http://infrahub:8000" - name: INFRAHUB_API_TOKEN valueFrom: secretKeyRef: name: emma-secrets key: api-token - name: OPENAI_API_KEY valueFrom: secretKeyRef: name: emma-secrets key: openai-key - name: EMMA_FEATURE_FLAGS valueFrom: configMapKeyRef: name: emma-config key: feature-flags resources: requests: memory: "256Mi" cpu: "100m" limits: memory: "512Mi" cpu: "500m" volumeMounts: - name: config mountPath: /app/config volumes: - name: config configMap: name: emma-config --- apiVersion: v1 kind: Service metadata: name: emma spec: selector: app: emma ports: - port: 8501 targetPort: 8501 type: ClusterIP --- apiVersion: v1 kind: ConfigMap metadata: name: emma-config data: feature-flags: "query_builder,template_builder" --- apiVersion: v1 kind: Secret metadata: name: emma-secrets type: Opaque data: api-token: openai-key: ``` ## Security configuration[​](#security-configuration "Direct link to Security configuration") ### API token security[​](#api-token-security "Direct link to API token security") **Best Practices:** * Use dedicated service accounts for Emma * Rotate tokens regularly * Limit token permissions to required operations * Never log or expose tokens in plaintext **Token Permissions:** Emma requires the following Infrahub permissions: * Schema read/write access * Data read/write access * Branch access (if using feature branches) * API access ### Network security[​](#network-security "Direct link to Network security") **Firewall Rules:** * Emma port (default 8501) should be restricted to authorized users * Infrahub connection should use HTTPS in production * Consider VPN or private networks for sensitive deployments **TLS/SSL:** For production deployments, configure TLS: ``` # Using reverse proxy (nginx, traefik, etc.) # Emma itself runs HTTP, proxy handles HTTPS ``` ### Data security[​](#data-security "Direct link to Data security") **Sensitive Data Handling:** * API tokens stored securely (secrets management) * Temporary files cleaned up automatically * No sensitive data logged by default * CSV uploads processed in memory when possible ## Performance configuration[​](#performance-configuration "Direct link to Performance configuration") ### Resource limits[​](#resource-limits "Direct link to Resource limits") **Memory:** * Minimum: 256MB * Recommended: 512MB * Large datasets: 1GB+ **CPU:** * Minimum: 0.1 cores * Recommended: 0.5 cores * Heavy AI usage: 1+ cores **Storage:** * Application: \~100MB * Temporary files: Variable based on data volume * Configuration: <1MB ### Optimization settings[​](#optimization-settings "Direct link to Optimization settings") **For Large Datasets:** ``` # Increase timeouts export INFRAHUB_TIMEOUT=120 # Optimize batch sizes # Set in Emma UI: Import/Export Settings → Batch Size: 50 ``` **For Slow Networks:** ``` # Increase retries and timeout export INFRAHUB_TIMEOUT=60 export INFRAHUB_RETRIES=5 ``` **For High-Volume Usage:** ``` # Enable performance features export EMMA_FEATURE_FLAGS="bulk_operations,advanced_import" ``` ## Configuration validation[​](#configuration-validation "Direct link to Configuration validation") Emma validates configuration on startup and provides helpful error messages: **Common Validation Errors:** ``` ERROR: INFRAHUB_ADDRESS is required ERROR: Cannot connect to Infrahub at http://localhost:8000 WARNING: OPENAI_API_KEY not set - AI features disabled WARNING: Unknown feature flag 'invalid_flag' ignored ``` **Configuration Check:** ``` # Test configuration without starting full application uv run python -c "from emma.utils import validate_config; validate_config()" ``` ## Migration and upgrades[​](#migration-and-upgrades "Direct link to Migration and upgrades") ### Configuration migration[​](#configuration-migration "Direct link to Configuration migration") When upgrading Emma, configuration may need updates: **Version 1.x to 2.x:** * New required environment variables * Changed feature flag names * Updated AI model configurations **Check Release Notes:** Always review release notes for configuration changes and migration steps. ### Backup configuration[​](#backup-configuration "Direct link to Backup configuration") **Docker:** ``` # Backup configuration volume docker run --rm -v emma-config:/data -v $(pwd):/backup busybox tar czf /backup/emma-config.tar.gz -C /data . ``` **Kubernetes:** ``` # Export ConfigMaps and Secrets kubectl get configmap emma-config -o yaml > emma-config-backup.yaml kubectl get secret emma-secrets -o yaml > emma-secrets-backup.yaml ``` This configuration reference provides comprehensive guidance for setting up Emma in various environments. For troubleshooting configuration issues, see the [Troubleshooting Guide](/emma/reference/troubleshooting.md). --- # Feature Flags Emma uses feature flags to enable experimental and beta functionality. This allows you to try new features while maintaining stability in production environments. ## Overview[​](#overview "Direct link to Overview") Feature flags in Emma control access to: * **Experimental features** - Early implementations that may change significantly * **Beta features** - Stable implementations undergoing final testing * **Advanced features** - Complex functionality for power users ## Configuration[​](#configuration "Direct link to Configuration") Set feature flags using the `EMMA_FEATURE_FLAGS` environment variable: ``` # Single feature export EMMA_FEATURE_FLAGS="query_builder" # Multiple features (comma-separated) export EMMA_FEATURE_FLAGS="query_builder,template_builder" ``` ## Available feature flags[​](#available-feature-flags "Direct link to Available feature flags") ### `query_builder` (Alpha)[​](#query_builder-alpha "Direct link to query_builder-alpha") **Description:** Interactive GraphQL query builder for Infrahub **Status:** Alpha - Basic functionality implemented, interface may change **What it enables:** * Visual query builder interface * GraphQL query generation and execution * Result formatting and export * Query history and favorites **Use cases:** * Custom data extraction from Infrahub * Ad-hoc reporting and analysis * Learning GraphQL query syntax * Debugging data relationships **Requirements:** * None (works with basic Infrahub connection) **Limitations:** * Limited to read-only queries * Basic error handling * No query optimization suggestions * Interface is subject to change **Example usage:** 1. Enable the feature flag 2. Navigate to "Query Builder" in Emma's sidebar 3. Use the visual interface to build queries 4. Execute queries and view results ### `template_builder` (Alpha)[​](#template_builder-alpha "Direct link to template_builder-alpha") **Description:** Template creation and management system **Status:** Alpha - Core functionality working, needs refinement **What it enables:** * Template creation interface * Template library management * Variable substitution * Template sharing and export **Use cases:** * Creating reusable configuration templates * Standardizing infrastructure deployments * Generating documentation templates * Configuration as code workflows **Requirements:** * Write access to Infrahub * Understanding of template syntax **Limitations:** * Basic template syntax support * Limited validation * No template versioning * Experimental UI **Example usage:** 1. Enable the feature flag 2. Access "Template Builder" from the sidebar 3. Create templates with variables 4. Test template rendering with sample data ### `advanced_import` (Beta)[​](#advanced_import-beta "Direct link to advanced_import-beta") **Description:** Enhanced data import capabilities **Status:** Beta - Feature complete, undergoing testing **What it enables:** * Advanced CSV parsing options * Custom data transformation rules * Relationship auto-resolution * Import validation previews * Batch processing optimizations **Use cases:** * Complex data migrations * Custom data format handling * Large dataset imports * Data quality validation **Requirements:** * Standard Emma data import permissions * Understanding of data transformation concepts **Limitations:** * Increased memory usage * Longer processing times for complex transformations **Example usage:** 1. Enable the feature flag 2. Use Data Importer with enhanced options 3. Configure advanced parsing and transformation rules 4. Preview and validate before importing ### `bulk_operations` (Alpha)[​](#bulk_operations-alpha "Direct link to bulk_operations-alpha") **Description:** Bulk operations for schemas and data **Status:** Alpha - Basic implementation, needs performance optimization **What it enables:** * Bulk schema loading * Mass data operations * Batch delete functionality * Progress tracking for large operations **Use cases:** * Initial system setup with many schemas * Large-scale data cleanup * Mass updates across multiple objects * System migrations **Requirements:** * Administrator-level permissions * Understanding of bulk operation impacts **Limitations:** * Performance limitations with very large datasets * Limited rollback capabilities * Potential impact on Infrahub performance **Example usage:** 1. Enable the feature flag 2. Access bulk operation interfaces 3. Select multiple items for batch operations 4. Monitor progress and handle errors ### `ai_suggestions` (experimental)[​](#ai_suggestions-experimental "Direct link to ai_suggestions-experimental") **Description:** AI-powered suggestions throughout Emma's interface **Status:** Experimental - Research implementation, may be removed **What it enables:** * Schema attribute suggestions * Data validation suggestions * Relationship recommendations * Import mapping assistance **Use cases:** * First-time users learning schema design * Data quality improvement * Relationship discovery * Import optimization **Requirements:** * OpenAI API key configured * Sufficient OpenAI API credits * Network access to OpenAI services **Limitations:** * Requires external API calls * May slow down interface * Suggestions quality varies * Additional API costs **Example usage:** 1. Configure OpenAI API key 2. Enable the feature flag 3. Look for AI suggestion indicators in the UI 4. Review and apply suggestions as appropriate ## Feature flag management[​](#feature-flag-management "Direct link to Feature flag management") ### Checking active flags[​](#checking-active-flags "Direct link to Checking active flags") Emma displays active feature flags in several places: **Settings Panel:** * Lists all enabled flags * Shows feature status (Alpha, Beta, Experimental) * Provides links to feature documentation **Feature Indicators:** * Icons next to experimental features * Tooltips explaining feature status * Warnings about stability ### Dynamic configuration[​](#dynamic-configuration "Direct link to Dynamic configuration") Some feature flags can be toggled at runtime: **Session-level flags** (temporary): * Set via URL parameters: `?features=query_builder,template_builder` * Override environment variables for testing * Reset when browser session ends **User preferences** (persistent): * Stored in browser local storage * Survive browser restarts * Can be cleared via settings panel ### Validation and error handling[​](#validation-and-error-handling "Direct link to Validation and error handling") Emma validates feature flags on startup: **Valid flag formats:** ``` # Correct export EMMA_FEATURE_FLAGS="query_builder,template_builder" # Also correct (spaces are stripped) export EMMA_FEATURE_FLAGS="query_builder, template_builder" ``` **Invalid configurations:** ``` # Unknown flags are ignored with warnings export EMMA_FEATURE_FLAGS="invalid_flag,query_builder" # Warning: Unknown feature flag 'invalid_flag' ignored # Empty values are acceptable export EMMA_FEATURE_FLAGS="" ``` ## Best practices[​](#best-practices "Direct link to Best practices") ### Development environments[​](#development-environments "Direct link to Development environments") **Enable experimental features** for testing: ``` export EMMA_FEATURE_FLAGS="query_builder,template_builder,ai_suggestions" ``` **Benefits:** * Access to latest functionality * Opportunity to provide feedback * Early adoption of useful features **Considerations:** * Features may break or change * Performance may be sub-optimal * Some features require additional setup ### Production environments[​](#production-environments "Direct link to Production environments") **Use only stable features** in production: ``` # Only enable well-tested features export EMMA_FEATURE_FLAGS="advanced_import" ``` **Recommended approach:** 1. Test features in development first 2. Enable beta features in staging 3. Only use stable features in production 4. Monitor for issues after enabling new features ### Team deployments[​](#team-deployments "Direct link to Team deployments") **Coordinate feature usage** across teams: * Document which features are enabled in each environment * Communicate feature changes to team members * Provide training for new experimental features * Establish rollback procedures ## Feature lifecycle[​](#feature-lifecycle "Direct link to Feature lifecycle") Emma features follow a development lifecycle: ### Experimental[​](#experimental "Direct link to Experimental") * **Purpose:** Research and initial implementation * **Stability:** May be removed or significantly changed * **Support:** Limited documentation and support * **Recommendation:** Development environments only ### Alpha[​](#alpha "Direct link to Alpha") * **Purpose:** Core functionality complete, needs refinement * **Stability:** Basic functionality works, interfaces may change * **Support:** Basic documentation, community support * **Recommendation:** Development and staging environments ### Beta[​](#beta "Direct link to Beta") * **Purpose:** Feature complete, undergoing final testing * **Stability:** Stable functionality, minor interface changes possible * **Support:** Full documentation, supported by team * **Recommendation:** Safe for production with monitoring ### Stable[​](#stable "Direct link to Stable") * **Purpose:** Production-ready feature * **Stability:** Fully stable, changes follow deprecation policy * **Support:** Full support and documentation * **Recommendation:** Safe for all environments ### Deprecated[​](#deprecated "Direct link to Deprecated") * **Purpose:** Feature being phased out * **Stability:** Still works but may be removed in future versions * **Support:** Limited support, migration guidance provided * **Recommendation:** Plan migration to replacement features ## Migration and compatibility[​](#migration-and-compatibility "Direct link to Migration and compatibility") ### Version compatibility[​](#version-compatibility "Direct link to Version compatibility") Feature flags may change between Emma versions: **Major versions**, for example: 1.x to 2.x: * Feature flags may be renamed or removed * New features may be added * Check migration guide for changes **Minor versions**, for example: 1.1 to 1.2: * New features may be added * Existing features may change status * Backward compatibility maintained **Patch versions**, for example: 1.1.1 to 1.1.2: * Feature flags unchanged * Bug fixes only ### Feature graduation[​](#feature-graduation "Direct link to Feature graduation") When experimental features become stable: 1. **Feature flag removed** - Feature becomes always available 2. **Configuration maintained** - No changes to your setup required 3. **Documentation updated** - Feature moves to main documentation 4. **Migration period** - Old flag maintained temporarily for compatibility ### Deprecation process[​](#deprecation-process "Direct link to Deprecation process") When features are deprecated: 1. **Deprecation notice** - Warning in documentation and UI 2. **Migration guide** - Instructions for moving to replacement 3. **Grace period** - Feature continues to work for several versions 4. **Removal** - Feature flag and functionality removed ## Troubleshooting feature flags[​](#troubleshooting-feature-flags "Direct link to Troubleshooting feature flags") ### Common issues[​](#common-issues "Direct link to Common issues") **Feature not appearing:** * Check feature flag spelling * Verify environment variable is set correctly * Restart Emma after changing environment variables * Check Emma logs for feature flag warnings **Feature causing issues:** * Disable the feature flag temporarily * Check feature-specific documentation * Report issues with feature flag and Emma version information **Performance problems:** * Some experimental features may impact performance * Disable CPU-intensive features if needed * Monitor system resources when testing new features ### Getting help[​](#getting-help "Direct link to Getting help") When reporting feature flag issues: 1. **Include Emma version** and deployment method 2. **List active feature flags** from your configuration 3. **Describe the specific feature** causing issues 4. **Provide steps to reproduce** the problem 5. **Include relevant logs** with feature flag context ### Feedback and contributions[​](#feedback-and-contributions "Direct link to Feedback and contributions") Emma's experimental features benefit from user feedback: * **Test new features** and report issues * **Suggest improvements** for experimental interfaces * **Share use cases** for features you find valuable * **Contribute documentation** for features you use Feature flags allow Emma to evolve rapidly while maintaining stability. Use them to access cutting-edge functionality and help shape Emma's future development. --- # Troubleshooting This guide covers common issues you might encounter while using Emma and how to resolve them. ## Connection issues[​](#connection-issues "Direct link to Connection issues") ### Cannot connect to Infrahub[​](#cannot-connect-to-infrahub "Direct link to Cannot connect to Infrahub") **Symptoms:** * Emma shows "Connection Failed" status * API calls timeout or fail * Cannot load schemas or data **Solutions:** 1. **Check Infrahub Address** ``` # Verify the address is correct and accessible curl http://localhost:8000/api/schema/ ``` 2. **Verify API Token** * Ensure the token is valid and not expired * Check token permissions in Infrahub * Try generating a new token 3. **Network Connectivity** * Check if Infrahub is running: `docker ps` or service status * Verify port accessibility: `telnet localhost 8000` * Check firewall rules and network configuration 4. **Docker Network Issues** (when using containers) ``` # Check if containers can communicate docker network ls docker network inspect # Connect Emma to Infrahub network docker network connect emma-emma-1 ``` ### Authentication errors[​](#authentication-errors "Direct link to Authentication errors") **Error:** "Unauthorized" or "403 Forbidden" **Solutions:** * Verify `INFRAHUB_API_TOKEN` is set correctly * Check token permissions in Infrahub admin interface * Ensure token hasn't expired * Try accessing Infrahub directly with the token: ``` curl -H "X-INFRAHUB-KEY: your-token" http://localhost:8000/api/schema/ ``` ## Schema issues[​](#schema-issues "Direct link to Schema issues") ### Schema loading failures[​](#schema-loading-failures "Direct link to Schema loading failures") **Error:** "Failed to load schema" or validation errors **Common Causes and Solutions:** 1. **Missing Dependencies** * Load base schemas before dependent schemas * Check inheritance chain requirements * Verify referenced schemas exist 2. **Invalid YAML Syntax** * Use a YAML validator to check syntax * Check indentation (use spaces, not tabs) * Verify quotes and special characters 3. **Schema Conflicts** * Check for naming conflicts with existing schemas * Verify namespace uniqueness * Review attribute naming conventions 4. **Permission Issues** * Ensure API token has schema modification permissions * Check branch permissions if using feature branches ### AI schema builder issues[​](#ai-schema-builder-issues "Direct link to AI schema builder issues") **Problem:** AI generates incorrect or incomplete schemas **Solutions:** 1. **Improve Descriptions** * Be more specific about requirements * Include examples of desired attributes * Mention relationships explicitly * Reference existing schemas 2. **Check OpenAI Configuration** * Verify API key is valid and has credits * Check rate limits and quotas * Try a simpler description first 3. **Validate Generated Schemas** * Always review AI output carefully * Test schemas with sample data * Compare with similar schemas in the library ## Data import/export issues[​](#data-importexport-issues "Direct link to Data import/export issues") ### Import failures[​](#import-failures "Direct link to Import failures") **Common Problems:** 1. **File Format Issues** ``` Error: "Unable to parse CSV file" ``` **Solutions:** * Ensure file is UTF-8 encoded * Check for proper CSV format with headers * Remove or escape special characters * Verify file size is under 100MB 2. **Mapping Errors** ``` Error: "Column 'xyz' cannot be mapped to attribute 'abc'" ``` **Solutions:** * Check attribute names match schema definition * Verify data types are compatible * Review required vs optional fields * Check relationship target schemas exist 3. **Validation Failures** ``` Error: "Validation failed for record 123" ``` **Solutions:** * Check data format (dates, IPs, emails) * Verify enum values are valid * Ensure required fields are populated * Check uniqueness constraints 4. **Relationship Resolution** ``` Error: "Cannot resolve relationship to 'Location'" ``` **Solutions:** * Ensure referenced objects exist in Infrahub * Check relationship field naming * Verify object identifiers are correct * Create referenced objects first ### Export failures[​](#export-failures "Direct link to Export failures") **Common Problems:** 1. **Permission Errors** * Verify read permissions for target schemas * Check branch access permissions * Ensure API token has export rights 2. **Performance Issues** * Reduce export size with filters * Export in smaller batches * Check Infrahub resource usage * Increase timeout settings 3. **Format Issues** * Verify encoding settings * Check delimiter configuration * Review special character handling ## Performance issues[​](#performance-issues "Direct link to Performance issues") ### Slow response times[​](#slow-response-times "Direct link to Slow response times") **Symptoms:** * Long loading times for schemas or data * Timeouts during operations * Unresponsive interface **Solutions:** 1. **Check System Resources** ``` # Monitor system resources htop # or top docker stats # for containerized deployments ``` 2. **Optimize Operations** * Reduce batch sizes for imports * Use filters to limit data exports * Close unused browser tabs * Clear browser cache 3. **Network Optimization** * Check network latency to Infrahub * Use local deployments for development * Increase timeout settings 4. **Infrahub Performance** * Check Infrahub logs for errors * Monitor database performance * Verify adequate system resources ### Memory issues[​](#memory-issues "Direct link to Memory issues") **Error:** "Out of memory" or browser crashes **Solutions:** * Reduce file sizes for imports * Process data in smaller batches * Close other applications * Increase system memory if possible * Use command-line tools for very large datasets ## Browser and interface issues[​](#browser-and-interface-issues "Direct link to Browser and interface issues") ### Display problems[​](#display-problems "Direct link to Display problems") **Issues:** * Missing UI elements * Formatting problems * Broken layouts **Solutions:** 1. **Clear Browser Cache** * Hard refresh (Ctrl+F5 or Cmd+Shift+R) * Clear browser cache and cookies * Try incognito/private mode 2. **Check Browser Compatibility** * Use supported browsers (Chrome, Firefox, Safari, Edge) * Update to latest browser version * Disable problematic browser extensions 3. **Check Console Errors** * Open browser developer tools (F12) * Look for JavaScript errors in console * Check network tab for failed requests ### Streamlit-specific issues[​](#streamlit-specific-issues "Direct link to Streamlit-specific issues") **Error:** "Please run this app with streamlit run" **Solution:** ``` # Ensure you're using the correct command uv run streamlit run main.py # Or for direct Python execution python -m streamlit run main.py ``` **Error:** Session state errors **Solutions:** * Refresh the page * Clear browser session storage * Restart Emma application ## Configuration issues[​](#configuration-issues "Direct link to Configuration issues") ### Environment variables[​](#environment-variables "Direct link to Environment variables") **Problem:** Configuration not being applied **Check List:** 1. **Verify Variables Are Set** ``` echo $INFRAHUB_ADDRESS echo $INFRAHUB_API_TOKEN env | grep INFRAHUB ``` 2. **Docker Environment Issues** ``` # In docker-compose.yml environment: - INFRAHUB_ADDRESS=http://infrahub:8000 - INFRAHUB_API_TOKEN=${INFRAHUB_API_TOKEN} ``` 3. **Environment File Issues** ``` # Check .env file exists and format cat .env # Ensure no spaces around = signs # CORRECT: INFRAHUB_ADDRESS=http://localhost:8000 # WRONG: INFRAHUB_ADDRESS = http://localhost:8000 ``` ### Feature flags[​](#feature-flags "Direct link to Feature flags") **Problem:** Experimental features not working **Solutions:** ``` # Check feature flags are set echo $EMMA_FEATURE_FLAGS # Set feature flags correctly export EMMA_FEATURE_FLAGS="query_builder,template_builder" # For Docker docker run -e EMMA_FEATURE_FLAGS="query_builder" emma ``` ## Logging and debugging[​](#logging-and-debugging "Direct link to Logging and debugging") ### Enable debug logging[​](#enable-debug-logging "Direct link to Enable debug logging") **For Local Development:** ``` # Set Streamlit logging level export STREAMLIT_LOGGER_LEVEL=debug # Set Python logging level export PYTHONPATH=. uv run python -c "import logging; logging.basicConfig(level=logging.DEBUG)" ``` **For Docker:** ``` # In docker-compose.yml environment: - STREAMLIT_LOGGER_LEVEL=debug - PYTHONPATH=/app ``` ### Check application logs[​](#check-application-logs "Direct link to Check application logs") **Local Development:** ``` # Run with verbose output uv run streamlit run main.py --logger.level=debug # Check Emma logs tail -f ~/.streamlit/logs/ ``` **Docker Deployment:** ``` # Check container logs docker logs emma-emma-1 # Follow logs in real-time docker logs -f emma-emma-1 ``` ### Common log patterns[​](#common-log-patterns "Direct link to Common log patterns") **Connection Issues:** ``` ERROR: Failed to connect to Infrahub at http://localhost:8000 ERROR: Authentication failed - invalid token ``` **Schema Issues:** ``` ERROR: Schema validation failed WARNING: Referenced schema 'Location' not found ``` **Import Issues:** ``` ERROR: Failed to parse CSV at line 123 WARNING: Relationship resolution failed for 'device-01' ``` ## Getting help[​](#getting-help "Direct link to Getting help") ### Before asking for help[​](#before-asking-for-help "Direct link to Before asking for help") 1. **Check this troubleshooting guide** 2. **Review Emma documentation** 3. **Check Infrahub documentation** at [docs.infrahub.app](https://docs.infrahub.app) 4. **Search existing issues** on GitHub ### Gathering information[​](#gathering-information "Direct link to Gathering information") When reporting issues, include: 1. **Emma version** and deployment method 2. **Infrahub version** and configuration 3. **Error messages** (full text, not screenshots) 4. **Steps to reproduce** the issue 5. **Environment details** (OS, browser, Docker version) 6. **Relevant logs** with debug logging enabled ### Where to get help[​](#where-to-get-help "Direct link to Where to get help") * **GitHub Issues**: [GitHub](https://github.com/opsmill/emma/issues) * **Infrahub Community**: [GitHub Discussions](https://github.com/opsmill/infrahub/discussions) * **Documentation**: [docs.infrahub.app](https://docs.infrahub.app) ### Emergency procedures[​](#emergency-procedures "Direct link to Emergency procedures") **If Emma is completely broken:** 1. **Check Infrahub directly** to ensure it's working 2. **Restart Emma** application or containers 3. **Clear browser cache** and try again 4. **Check recent changes** and revert if necessary 5. **Use Infrahub directly** as a workaround **If data import corrupted data:** 1. **Stop the import** immediately 2. **Use Infrahub branches** to isolate changes 3. **Review import logs** for affected records 4. **Use Infrahub's version control** to revert if needed 5. **Export and verify** data integrity Remember: Emma is experimental software. Always backup your Infrahub data before major operations and test changes in non-production environments first. --- # Infrahub Exporter Infrahub Exporter is a service that exports metrics and service discovery information from Infrahub to monitoring systems like Prometheus and OpenTelemetry. ## Overview[​](#overview "Direct link to Overview") Infrahub Exporter acts as a bridge between your Infrahub instance and monitoring tools, providing: 1. **Metrics Export**: Collects and exposes metrics from Infrahub nodes for monitoring 2. **Service Discovery**: Provides dynamic service discovery for Prometheus based on Infrahub data 3. **OpenTelemetry Integration**: Supports sending metrics to OpenTelemetry collectors ## Features[​](#features "Direct link to Features") * **Prometheus Integration**: Exposes metrics in Prometheus format * **OpenTelemetry Support**: Sends metrics to OTLP-compatible collectors * **Dynamic Service Discovery**: Generates Prometheus service discovery files based on GraphQL queries * **Flexible Configuration**: Configurable via YAML with environment variable overrides * **Caching**: Efficient data retrieval with caching to reduce load on Infrahub * **Resilience**: Automatic retries and error handling for API calls ## Guides[​](#guides "Direct link to Guides") * [Install and run Infrahub Exporter](/exporter/guides/installation.md) * [Configure Infrahub Exporter](/exporter/guides/configuration.md) --- # Configure Infrahub Exporter ## Configuration[​](#configuration "Direct link to Configuration") Create a `config.yml` file with the following structure: ``` # Infrahub connection settings infrahub: address: "http://localhost:8000" # Infrahub server address token: "your-api-token" # Infrahub SDK API key branch: main # Which branch to fetch # Poll interval in seconds (default is XXX) poll_interval_seconds: 30 # HTTP server configuration listen_address: "0.0.0.0" listen_port: 8001 # Log level (default level is INFO) log_level: "INFO" ``` ## Exporter configuration[​](#exporter-configuration "Direct link to Exporter configuration") Infrahub Exporter separates what metrics to collect from how to send them. You define metrics (resources and fields) once, and then enable one or more exporters to deliver those metrics: ### Defining metrics[​](#defining-metrics "Direct link to Defining metrics") ``` metrics: # Define which resource types and fields to include kind: - kind: InfraDevice include: - name - description - platform - role - status # You can apply filters to this `kind` filters: - site__name__value: "dc1" - role__value: "edge" ``` These metrics will be available to all configured exporters. You can choose one or more exporters: * Pull-based (Prometheus): Exposes a `/metrics` HTTP endpoint for Prometheus or other scrapers. * Push-based (OTLP): Sends metrics to an OpenTelemetry collector. ### Prometheus integration[​](#prometheus-integration "Direct link to Prometheus integration") Configure the Prometheus exporter in your `config.yml` : ``` exporters: prometheus: enabled: true # Pull-based endpoint at /metrics ``` And in your Prometheus : ``` scrape_configs: - job_name: 'infrahub' scrape_interval: 30s static_configs: - targets: ['infrahub-exporter:8001'] ``` ### Opentelemetry integration[​](#opentelemetry-integration "Direct link to Opentelemetry integration") Configure the OTLP exporter in your `config.yml` : ``` exporters: otlp: enabled: true endpoint: "http://otel-collector:4317" timeout_seconds: 10 ``` ## Service discovery configuration[​](#service-discovery-configuration "Direct link to Service discovery configuration") The exporter can provide HTTP-based service-discovery for Prometheus. Define your queries and how to map fields into target labels: ``` service_discovery: enabled: true queries: - name: "devices" file_path: "./queries/devices.gql" endpoint_path: "devices" refresh_interval_seconds: 60 target_field: "primary_address.node.address.ip" label_mappings: "device_name": "name.value" "location": "site.node.name.value" "role": "role.value" "platform": "platform.node.name.value" ``` This section ensures Prometheus can automatically discover and scrape dynamic targets managed by Infrahub. To integrate it with your monitoring stack, add this section in your Prometheus configuration: ``` scrape_configs: - job_name: 'infrahub-sd' http_sd_configs: - url: http://infrahub-exporter:8001/sd/devices refresh_interval: 60s ``` --- # Install and run Infrahub Exporter This guide walks you through installing and running the Infrahub Exporter on your system. ## Installation[​](#installation "Direct link to Installation") Choose one of the following methods to install Infrahub Exporter. ### Using poetry[​](#using-poetry "Direct link to Using poetry") 1. **Clone the repository** ``` git clone https://github.com/opsmill/infrahub-exporter.git cd infrahub-exporter ``` 1. **Install dependencies** ``` poetry install ``` 1. **Configure your exporter** See the [Configuration guide](/exporter/guides/configuration.md) 1. **Run the exporter** ``` poetry run python -m infrahub_exporter --config path/to/config.yml ``` ### Using docker (recommended)[​](#using-docker-recommended "Direct link to Using docker (recommended)") 1. **Clone the repository** ``` git clone https://github.com/opsmill/infrahub-exporter.git cd infrahub-exporter ``` 1. **Build the Docker image** ``` docker build -t infrahub-exporter:latest -f development/Dockerfile . ``` 1. **Configure your exporter** See the [Configuration guide](/exporter/guides/configuration.md) 1. **Run the container** ``` docker run \ -v "path/to/config.yml:/app/config.yml" \ -p 8001:8001 \ infrahub-exporter:latest ``` 1. \*\*Docker compose example ``` infrahub-exporter: image: infrahub-exporter:latest restart: unless-stopped ports: - "8001:8001" volumes: - path/to/config.yml:/app/config.yml:ro - path/to/queries:/app/queries:ro environment: - INFRAHUB_SIDECAR_LOG_LEVEL=INFO depends_on: - infrahub-server # adjust to your Infrahub service name ``` --- # Infrahub demo-sp example Welcome to the **Infrahub demo-sp** example — a service-provider bundle that shows how Infrahub manages a multi-vendor MPLS backbone and provisions customer-facing services end to end. It demonstrates Infrahub's core capabilities through a realistic SP workload: * Schema-driven data modeling for backbone and customer services * Generator-driven provisioning (VRF, RT, PE-CE interfaces, IP allocation, BGP) * Per-vendor configuration artifacts from a shared schema * Version-controlled changes with proposed-change diff review * A Streamlit Service Catalog for self-service L3VPN and SD-WAN provisioning Whether you're a network engineer evaluating automation, a developer building on Infrahub, or an architect comparing infrastructure-data platforms, this demo gives you a hands-on view of how an SP team would model and operate a real backbone. ## Documentation guide[​](#documentation-guide "Direct link to Documentation guide") This documentation follows the [Diataxis framework](https://diataxis.fr/) so you can find the right kind of page for what you need to do. ### Getting started[​](#getting-started "Direct link to Getting started") | Page | Purpose | | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | **[Quick start](/infrahub-demo-sp/quickstart.md)** | Step-by-step instructions to clone the repository, configure `.env`, start Infrahub, and load the bootstrap data. Start here on your first run. | ### Tutorials[​](#tutorials "Direct link to Tutorials") | Page | Purpose | | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **[L3VPN service walkthrough](/infrahub-demo-sp/services/l3vpn.md)** | End-to-end tour of the L3VPN service: how the catalog form drives `ServiceL3Vpn` creation, what the generator materialises (VRF, RTs, PE-CE links, eBGP), what each per-vendor template emits, and how the validation checks fit together. | | **[SD-WAN service walkthrough](/infrahub-demo-sp/services/sdwan.md)** | Companion tour of the SD-WAN service: vendor choice (Cisco Viptela vs Versa Networks), hub-spoke vs full-mesh topology, generator-created edge devices, and per-vendor edge configurations. | | **[Deploy with containerlab](/infrahub-demo-sp/lab/containerlab.md)** | Take the rendered MPLS backbone artifacts and bring them up in a virtual lab with containerlab. Cover Arista cEOS, Nokia SR Linux substitution, and how to reach the lab CLIs. | ### Topics[​](#topics "Direct link to Topics") | Page | Purpose | | ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **[Architecture](/infrahub-demo-sp/architecture.md)** | The "why" behind the demo — backbone design choices, service modelling, generator vs catalog responsibilities, and the lifecycle of a customer service. | | **[Schema reference](/infrahub-demo-sp/schema-reference.md)** | Field-level documentation of every node (`ServiceL3Vpn`, `ServiceSdwan`, `TopologyMplsBackbone`, …), the resource pools, and the user-vs-generator split for each attribute. | ### Operations[​](#operations "Direct link to Operations") | Page | Purpose | | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | **[Troubleshooting](/infrahub-demo-sp/troubleshooting.md)** | Symptoms and fixes for common issues — stuck repository sync, generator races, broken artifacts, and how to inspect live state when something looks off. | ## Quick start[​](#quick-start "Direct link to Quick start") If you're ready to dive in: 1. Follow the **[quick start](/infrahub-demo-sp/quickstart.md)** to clone the repository, configure `.env`, and run `uv run invoke init`. 2. Open the **Infrahub UI** at `http://localhost:8000` and explore the seeded data — Devices, MPLS Backbones, Service Catalog → L3 VPNs and SD-WAN. 3. Provision a new service from the **[Streamlit Service Catalog](/infrahub-demo-sp/services/l3vpn.md)** at `http://localhost:8501`. 4. (Optional) Bring up the MPLS backbone in **[containerlab](/infrahub-demo-sp/lab/containerlab.md)**. 5. Read the **[architecture](/infrahub-demo-sp/architecture.md)** and **[schema reference](/infrahub-demo-sp/schema-reference.md)** to understand the moving parts before you extend the demo. ## What you'll learn[​](#what-youll-learn "Direct link to What you'll learn") Through this demo, you'll gain practical experience with: * **Modelling SP services in Infrahub** — `ServiceL3Vpn` / `ServiceSdwan` with uniqueness constraints, parent/child relationships, and lifecycle status. * **Resource pools** — `vpn_id_pool`, `sdwan_id_pool`, `customer_asn_pool`, `pe_loopback_pool`, `backbone_p2p_pool`, `pe_ce_pool` for deterministic allocation. * **Generators** — Materialising VRFs, route targets, PE-CE /30 allocations, eBGP sessions, and SD-WAN edge devices from a single user-facing object. * **Per-vendor transforms** — Rendering the same intent into Arista EOS, Cisco IOS-XR, Juniper Junos, Nokia SR OS, Cisco Viptela, and Versa VOS configurations. * **Validation checks** — Catching duplicate RDs, overlapping customer prefixes, exhausted interface pools, and missing iBGP sessions before merge. * **Branch-based workflows** — Catalog form → feature branch → generator → artifact regeneration → proposed change → review → merge. * **Containerlab** — Booting the rendered backbone in a virtual lab for hands-on testing. ## Architecture at a glance[​](#architecture-at-a-glance "Direct link to Architecture at a glance") Out of the box the **financial** dataset gives you the Sterling Financial MPLS backbone — eight Arista cEOS PEs (`pe-01`…`pe-08`) in a partial mesh, with four pre-provisioned Arista cEOS CE routers hanging off the two hub PEs — plus three seeded customer services — two L3VPNs and one SD-WAN — wired up end to end: **Backbone** — Eight PE routers, all Arista cEOS (`pe-01`…`pe-08`), in eight EMEA sites. The topology is a partial mesh of 15 p2p links: `pe-01` (London) and `pe-08` (Zurich) are the degree-2 hub PEs carrying every customer attachment, and `pe-02`…`pe-07` form the meshed core between them at degree 4–5. Over the top runs a full iBGP/VPNv4 mesh, ISIS for underlay reachability, and LDP for label distribution. (The **isp** dataset instead ships a four-PE, one-per-vendor backbone — see below.) **Customer edge** — Four pre-provisioned Arista cEOS CE routers, two per customer, on the hub PEs: `ce-trading-lon`/`ce-ib-lon` on `pe-01` and `ce-trading-zrh`/`ce-ib-zrh` on `pe-08`. Each has its own rendered configuration artifact, so the PE-CE eBGP session comes up for real in the lab. **L3VPN services** — `trading-floor-vpn` (tenant `markets-trading`) and `ib-advisory-vpn` (tenant `investment-banking`), each with a London and a Zurich site, eBGP PE-CE sessions, and one /24 customer subnet per site in the `10.200.0.0/16` supernet. Each customer peers from its own AS, allocated from `customer_asn_pool`. **SD-WAN service (`treasury-branch-sdwan`)** — Tenant `treasury-ops`, hub-spoke on Cisco Viptela cEdge-1000 edges at London (hub), Frankfurt and Amsterdam (spokes); LAN subnets in the `10.250.0.0/16` supernet. An alternate **isp** dataset (Lumina Networks pan-European ISP) ships its own backbone — four PEs, one per vendor (`pe-lon-arista`, `pe-fra-cisco`, `pe-ams-juniper`, `pe-par-nokia`), full-mesh iBGP over six p2p links — plus eight customer tenants and a different default L3VPN and SD-WAN. Select it via the `INFRAHUB_DATASET` env var. ## Key features demonstrated[​](#key-features-demonstrated "Direct link to Key features demonstrated") ### Schema-driven service modelling[​](#schema-driven-service-modelling "Direct link to Schema-driven service modelling") `ServiceL3Vpn`, `ServiceL3VpnSite`, `ServiceSdwan`, `ServiceSdwanSite` define not just object shape but uniqueness constraints, parent/child relationships, dropdown enums for vendor and topology, and lifecycle status. See the **[schema reference](/infrahub-demo-sp/schema-reference.md)** for the full field tables. ### Generator-driven provisioning[​](#generator-driven-provisioning "Direct link to Generator-driven provisioning") When a catalog form creates a `ServiceL3Vpn` or `ServiceSdwan` and the row joins the `l3vpns` or `sdwans` group, Infrahub auto-fires the matching generator. The L3VPN generator allocates a VRF, route targets, a customer AS from `customer_asn_pool`, a /30 from `pe_ce_pool`, sets up the PE interface, and writes the eBGP session on both ends of each site — the CE end too, when the site names a pre-provisioned CE router. The SD-WAN generator materialises one CPE per site, allocates the LAN address from the customer subnet, and adds the edge to the vendor-specific group so the artifact pipeline targets it. ### Per-vendor configuration artifacts[​](#per-vendor-configuration-artifacts "Direct link to Per-vendor configuration artifacts") A single schema drives seven Jinja2 templates: Arista EOS (PE), Arista EOS (CE), Cisco IOS-XR, Juniper Junos, Nokia SR OS / SR Linux, Cisco Viptela cEdge (IOS-XE SD-WAN), and Versa VOS (FlexVNF). Each PE, CE, or edge ends up with a `text/plain` configuration artifact keyed by `name__value`, ready to feed into a CI/CD push. The default **financial** backbone is all-Arista, so it exercises the two Arista templates; the **isp** dataset, with one PE per vendor, exercises the full per-vendor set. ### Branch-based workflows[​](#branch-based-workflows "Direct link to Branch-based workflows") Every catalog submission opens a feature branch (`l3vpn/` or `sdwan/`), creates objects on the branch, polls the generator to completion, regenerates artifacts so the diff is meaningful, and opens a `CoreProposedChange` against `main`. Validation checks run on the PC before merge. ### Containerlab integration[​](#containerlab-integration "Direct link to Containerlab integration") The `clab-mpls-topology` artifact emits a containerlab YAML wiring the lab-deployable PEs and their backbone links, derived from the shared /31 addressing, plus every pre-provisioned CE and its PE-CE access link. For the default **financial** backbone that's eight Arista cEOS PEs, 15 backbone links, and four cEOS CEs that bring up real eBGP sessions against their PEs; for the **isp** dataset it's the Arista and Nokia PEs (Nokia SR OS substituted to SR Linux for the lab image). See the **[containerlab guide](/infrahub-demo-sp/lab/containerlab.md)**. ## Community and support[​](#community-and-support "Direct link to Community and support") * **Source code**: [GitHub repository](https://github.com/opsmill/infrahub-demo-sp) * **Infrahub documentation**: [docs.infrahub.app](https://docs.infrahub.app) * **Discord community**: [Discord](https://discord.gg/opsmill) * **OpsMill website**: [opsmill.com](https://opsmill.com) ## Next steps[​](#next-steps "Direct link to Next steps") Ready to get started? Head to the **[quick start](/infrahub-demo-sp/quickstart.md)** to set up your environment. --- # Architecture This page summarises the data flow and the role of each directory. ## Data flow[​](#data-flow "Direct link to Data flow") ``` Schema definition → Bootstrap data → Generators → Transforms → Artifacts ↓ Checks ``` ## Directory map[​](#directory-map "Direct link to Directory map") | Path | Purpose | | --------------------- | --------------------------------------------------------------- | | `schemas/base/` | Core node definitions copied from schema-library | | `schemas/extensions/` | VRF, routing, BGP, topology extensions from schema-library | | `schemas/sp/` | SP-demo-specific schemas (MPLS, L3VPN service, PE role) | | `objects/` | Pre-loaded bootstrap data (PEs, backbone, pools, tenants) | | `generators/` | `L3VpnGenerator` materialises VRF + interfaces + IPs | | `transforms/` | One Python+Jinja transform per vendor + the clab transform | | `templates/` | Jinja2 templates (one per vendor + clab + macros) | | `checks/` | Five checks gating the proposed-change pipeline (incl. Batfish) | | `service_catalog/` | Streamlit sidecar (Dashboard + Create L3VPN) | | `queries/` | GraphQL queries used by generators, transforms, checks | | `menus/` | Sidebar menu configuration | | `lab/` | Runtime-only; not committed | ## Backbone[​](#backbone "Direct link to Backbone") The MPLS backbone is **static demo data** and is dataset-specific — each dataset ships its own backbone. The default **financial** dataset has 8 PEs, all Arista cEOS (`pe-01`…`pe-08`), in a partial mesh of 15 p2p links over a full iBGP mesh. `pe-01` (London) and `pe-08` (Zurich) are the two-degree hub PEs that carry every customer attachment; `pe-02`…`pe-07` form the meshed core between them at degree 4–5. The **isp** dataset has 4 PEs, one per vendor, in a full mesh (6 p2p links). Both run ISIS L2 for the underlay and LDP for label distribution. To add or move PEs, edit `objects/datasets//60_backbone.yml` (and `70_topology.yml` for the p2p links) by hand and rerun `invoke bootstrap`. ### Customer edge[​](#customer-edge "Direct link to Customer edge") The **financial** dataset also ships four pre-provisioned CE routers (`objects/datasets/financial/65_ce.yml`), two per customer, attached to the hub PEs: | CE | Customer (tenant) | PE | L3VPN | | ---------------- | -------------------- | ----------------- | ------------------- | | `ce-trading-lon` | `markets-trading` | `pe-01` Ethernet3 | `trading-floor-vpn` | | `ce-trading-zrh` | `markets-trading` | `pe-08` Ethernet3 | `trading-floor-vpn` | | `ce-ib-lon` | `investment-banking` | `pe-01` Ethernet4 | `ib-advisory-vpn` | | `ce-ib-zrh` | `investment-banking` | `pe-08` Ethernet4 | `ib-advisory-vpn` | They are Arista cEOS boxes with their own rendered configuration artifact (`ce-arista-eos`), so the PE-CE eBGP session actually establishes in the containerlab topology rather than being modelled only. The PE ports they land on are seeded in `60_backbone.yml` with `status: active` and a description matching the L3VPN generator's idempotency key (`L3VPN `). That is what pins the wiring: the generator binds the site to that exact port instead of allocating the next free one, and the port is already out of the free pool so no later service can claim it. ### Customer autonomous system numbers[​](#customer-autonomous-system-numbers "Direct link to Customer autonomous system numbers") PE-CE eBGP peer ASNs come from the `customer_asn_pool` number pool (65100–65199, `objects/50_pools.yml`). The L3VPN generator allocates **one AS per VPN** — a customer is one routing domain across its sites — and links it as `customer_asn` on the VPN. A site may still set `bgp_peer_asn` to override it, which is how the **isp** dataset peers with pre-agreed customer AS numbers; when every eBGP site overrides, no pool ASN is consumed. ## L3VPN service flow[​](#l3vpn-service-flow "Direct link to L3VPN service flow") When an operator creates an L3VPN through the Streamlit catalog: 1. The catalog allocates a `vpn_id` from the `vpn_id_pool` number pool. 2. It opens a feature branch and creates `ServiceL3Vpn` + one or more `ServiceL3VpnSite` objects. 3. It adds the `ServiceL3Vpn` to the `l3vpns` group. A group-membership trigger (`objects/events/00_triggers.yml`) fires `L3VpnGenerator` **on the branch**, which materialises the VRF, route targets, PE-CE interfaces, IP addresses, and an eBGP session if the routing protocol is set to `ebgp`, all inside the customer's own IP namespace. The generator runs event-driven rather than in the proposed-change pipeline (`execute_in_proposed_change: false`) so its data lands before artifacts render — otherwise the pipeline races and the configuration diff comes out empty. A second rule in the same file re-runs the generator when a site is added to or removed from a service that is already in the group; see [services/l3vpn](/infrahub-demo-sp/services/l3vpn.md) for what that rule cannot cover. 4. Once the generator finishes, per-PE configuration artifacts are rendered by the transform layer against the now-complete branch data. 5. The catalog opens a `CoreProposedChange` targeting `main`. 6. The proposed-change checks run in the pipeline — any failure blocks the merge. 7. The operator reviews the diff in the Infrahub UI and merges. See [services/l3vpn](/infrahub-demo-sp/services/l3vpn.md) for the full service reference. ## Transform layer[​](#transform-layer "Direct link to Transform layer") Each vendor has a dedicated Python transform module (in `transforms/`) backed by a Jinja2 template (in `templates/`). The transform fetches the full PE state via GraphQL and renders a single, complete device configuration fragment. The mapping is: | Vendor | Transform | Template | | ------------- | -------------------------------- | ------------------------------- | | Arista EOS | `transforms/pe_arista_eos.py` | `templates/pe_arista_eos.j2` | | Cisco IOS-XR | `transforms/pe_cisco_iosxr.py` | `templates/pe_cisco_iosxr.j2` | | Juniper Junos | `transforms/pe_juniper_junos.py` | `templates/pe_juniper_junos.j2` | | Nokia SR OS | `transforms/pe_nokia_sros.py` | `templates/pe_nokia_sros.j2` | | Containerlab | `transforms/clab_topology.py` | `templates/clab_topology.j2` | All four per-vendor PE transforms are still in play: the **isp** dataset has one PE per vendor and exercises all of them. The default **financial** backbone is all-Arista, so it only renders through `pe_arista_eos`. ## Checks[​](#checks "Direct link to Checks") Five checks gate the proposed-change pipeline: | Check | What it enforces | | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `l3vpn_overlap` | No duplicate VPN IDs across active L3VPNs | | `l3vpn_site_subnet` | Customer subnet is reachable / not already allocated in the same VRF | | `pe_interface_alloc` | The nominated PE interface is free (status = `free`) | | `backbone_session_count` | Every PE has its full-mesh complement of N−1 iBGP sessions (N = PE count: 7 each for the 8-PE financial backbone, 3 each for the 4-PE isp backbone) | | `batfish_backbone` | Batfish-driven static validation of rendered backbone configs (parse status, undefined references, BGP session compatibility, IS-IS adjacency mesh). See [Batfish validation](/infrahub-demo-sp/validation/batfish.md). | ## Per-vendor interface name macros[​](#per-vendor-interface-name-macros "Direct link to Per-vendor interface name macros") The schema uses an abstract interface name (`Ethernet1`, `Ethernet2`, …); each vendor template translates that to the platform-native form via a macro in `templates/_macros.j2`: | Vendor | Macro | `Ethernet1` becomes | | ------------------------- | ---------------- | ------------------------ | | Arista EOS | (no translation) | `Ethernet1` | | Cisco IOS-XR | `iosxr_iface` | `GigabitEthernet0/0/0/0` | | Juniper Junos | `junos_iface` | `ge-0/0/0` | | SR Linux (lab substitute) | `srl_iface` | `ethernet-1/1` | Without these macros the rendered IOS-XR / Junos configs would carry Arista-style `Ethernet` interface names — invalid on the target platform and reported as parse errors by `BatfishBackboneCheck`. ## Schema layering[​](#schema-layering "Direct link to Schema layering") Schemas are loaded in three passes so each layer can reference the previous: ``` invoke bootstrap └── infrahubctl schema load schemas/base/ └── infrahubctl schema load schemas/extensions/ └── infrahubctl schema load schemas/sp/ ``` The SP layer adds `ServiceL3Vpn`, `ServiceL3VpnSite`, `TopologyMplsBackbone`, `MplsIsisProcess`, `MplsLdpProcess`, and `MplsBgpProcess`. See [schema-reference](/infrahub-demo-sp/schema-reference.md) for field-level details. --- # Containerlab This page covers the optional containerlab integration: what runs, which PEs are substituted, how to deploy, and known limitations. *** ## Overview[​](#overview "Direct link to Overview") The containerlab artifact (`clab-mpls-topology`) is generated from Infrahub data by the `clab_topology` transform and renders a valid containerlab YAML topology. It wires up the lab-deployable PE nodes plus the L3VPN CE nodes for end-to-end testing. The backbone links are **data-driven** — derived from the shared /31 addressing rather than fixed in the template — so the lab tracks whatever backbone the active dataset defines. Which PEs are deployable depends on the dataset: * **financial** (default) — all 8 PEs are Arista cEOS, so all 8 boot in the lab and all 15 backbone links render, plus the 4 cEOS CE routers and their PE-CE access links. * **isp** — only the Arista and Nokia PEs have a lab-runnable image, so 2 PE nodes boot with the single backbone link between them. The topology is written to `lab/mpls-backbone.clab.yml` by `invoke lab.deploy` before containerlab brings it up. *** ## Node map[​](#node-map "Direct link to Node map") A device is lab-deployable only when its platform carries a `containerlab_os` image. Today that means two vendors: | Vendor | Platform | Containerlab image | Notes | | ------------- | --------------- | ------------------------------------------------ | ---------------------------------------------------- | | Arista EOS | `arista_eos` | `ceos-image` (see [Architecture](#architecture)) | Full configuration push via `invoke lab.push-arista` | | Nokia SR OS | `nokia_sros` | SR Linux (`ghcr.io/nokia/srlinux`) | SR Linux used in place of SR OS (see below) | | Cisco IOS-XR | `cisco_iosxr` | — | No lab image — omitted from the topology | | Juniper Junos | `juniper_junos` | — | No lab image — omitted from the topology | Which PEs that yields depends on the active dataset: * **financial** (default) — all 8 PEs (`pe-01`…`pe-08`) are Arista cEOS, so all 8 boot in the lab and all 15 backbone links render. * **isp** — only `pe-lon-arista` (cEOS) and `pe-par-nokia` (SR Linux) are deployable; `pe-fra-cisco` and `pe-ams-juniper` are omitted. MPLS data plane The lab brings up IS-IS, the full iBGP mesh and PE-CE eBGP, and every customer prefix lands in the right VRF. It does **not** forward traffic between customer sites: cEOS never starts its LDP agent, so there is no MPLS label path between PEs. See [Troubleshooting](/infrahub-demo-sp/troubleshooting.md#ldp-does-not-start-so-inter-site-vpn-traffic-fails). CE nodes are real routers, not stand-ins. On **financial** the four pre-provisioned CEs are Arista cEOS and boot from their own `ce-arista-eos` configuration artifact, so the PE-CE eBGP session establishes in the lab. A site whose CE is unmanaged (no `ce_device`), or whose L3VPN generator has not run yet, contributes no CE node and no PE-CE link. Behind each CE the topology also synthesises one **customer host** — a `nicolaka/netshoot` container named `cust-` (for example `cust-ib-zrh` behind `ce-ib-zrh`). It is lab scaffolding, not Infrahub data: it exists so the LAN port on the CE has carrier. Without it the port stays down, the dot1q sub-interface sits `lowerlayerdown`, and the customer prefix is never advertised because a down interface contributes no connected route for the BGP `network` statement to match. Each host tags its own frames with the same VLAN the CE expects and takes `.10` of the customer subnet, with the CE sub-interface `.1` as its default gateway — so the encapsulation is exercised, and the host is a usable ping source across the VPN. *** ## SR OS → SR Linux substitution[​](#sr-os--sr-linux-substitution "Direct link to SR OS → SR Linux substitution") This applies only when the active backbone includes a Nokia PE — i.e. the **isp** dataset. The default **financial** backbone is all-Arista, so no substitution happens there. Nokia's containerlab-friendly image is **SR Linux**, not SR OS. SR OS is available only with a paid Nokia licence and is not suitable for a public demo. The demo uses SR Linux as a stand-in for the Nokia PE. The Infrahub data model remains `nokia_sros` for schema consistency; the containerlab artifact swaps in the SR Linux image automatically. The SR Linux node boots with the generated Nokia SR OS configuration loaded as a startup configuration, but **SR Linux does not parse SR OS CLI syntax**. Configuration push to the Nokia PE is therefore not available in v1 (see [Known gaps](#known-gaps)). *** ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") ### Host[​](#host "Direct link to Host") containerlab drives the host kernel directly (network namespaces, veth pairs, bridges), so the lab needs a **Linux host**. On macOS or Windows, run it inside a Linux VM — OrbStack, Lima, UTM or a cloud instance — not against Docker Desktop from the host side. The **financial** dataset is the demanding one: 12 Arista cEOS routers and 4 customer hosts, alongside the Infrahub stack itself. | Resource | financial (default) | isp | | -------------------------------- | -------------------------------- | ----------------------- | | Lab containers | 16 (12 cEOS + 4 hosts) | 2 (1 cEOS + 1 SR Linux) | | RAM for the lab | \~24 GB (cEOS wants \~2 GB/node) | \~4 GB | | RAM including the Infrahub stack | **32 GB or more** | \~16 GB | | vCPU | 8+ recommended | 4 | A host with less RAM than that does not fail cleanly — cEOS nodes thrash or get OOM-killed and the deploy stalls in postdeploy. If you have less, use the **isp** dataset, or run the Infrahub side only: everything except `invoke lab.deploy` works on a modest machine. ### Architecture[​](#architecture "Direct link to Architecture") **The lab needs an x86\_64 host. Apple Silicon cannot run it.** Every cEOS-lab build Arista publishes has a 32-bit x86 userland — `SWI_ARCH=i686` in the image's `/etc/swi-version`, amd64 tags included. Rosetta 2 does not translate 32-bit x86 at all, so the only thing left to execute it on Apple Silicon is qemu-i386 user-mode emulation. Under emulation EOS gets as far as `agentsToStart=[...]`, loops on `ProcMgr.service`, and systemd eventually gives up: the container exits 255 about 17 minutes in, with nothing naming the cause. There is no arm64 cEOS-lab tag to substitute — a tag whose registry metadata claims `arm64` is a mislabelled x86 image, not an ARM build. `invoke lab.deploy` therefore refuses 64-bit ARM hosts outright, in under a second, rather than letting twelve routers die one at a time: ``` ✗ cEOS-lab cannot run on this host (arm64). ``` Set `LAB_ALLOW_UNSUPPORTED_ARCH=true` to attempt it anyway. | Host `uname -m` | cEOS image | | ------------------- | --------------------------------------------------- | | `x86_64` | `registry.opsmill.io/external/ceos-image:4.36.0.1F` | | `aarch64` / `arm64` | unsupported — `lab.deploy` refuses | The rendered topology reads `${CEOS_IMAGE:=}`, so a different build can be pinned with `CEOS_IMAGE` without re-rendering the artifact. On an Apple Silicon machine everything except `invoke lab.deploy` works unchanged — schemas, generators, transforms, checks, artifact rendering and the service catalog. Only the containerlab side needs an x86\_64 host. SR Linux is the exception: `ghcr.io/nokia/srlinux:26.7.1-554` is a genuine multi-arch image and boots natively on ARM. That only covers the single `srl` node in the **isp** dataset; the **financial** dataset is all cEOS. Do not pin a cEOS-lab image older than **4.32.0F**. Earlier builds require a cgroups v1 host and never finish booting on a cgroups v2 one — the default on Ubuntu 21.04+, OrbStack and most current distributions — and the failure is silent. See [Troubleshooting](/infrahub-demo-sp/troubleshooting.md#containerlab-deploy-hangs-at-running-postdeploy-actions). ### Software[​](#software "Direct link to Software") * containerlab >= 0.50 * Docker, with network access to pull: * `registry.opsmill.io/external/ceos-image` (public mirror — no Arista account or manual import needed) * `ghcr.io/nokia/srlinux` (isp dataset only) * `nicolaka/netshoot` (the `cust-*` customer hosts on the CE LAN side) * A running Infrahub instance with bootstrap data loaded (`invoke init`) *** ## Deploy[​](#deploy "Direct link to Deploy") ``` # Fetch the clab artifact from Infrahub and write lab/mpls-backbone.clab.yml, # then call containerlab deploy. uv run invoke lab.deploy ``` Expected output (default **financial** dataset — 8 Arista cEOS PEs, the four pre-provisioned Arista cEOS CE routers, and one customer host behind each CE): ``` [INFO] Fetching clab-mpls-topology artifact from Infrahub... [INFO] Writing lab/mpls-backbone.clab.yml [INFO] Running: containerlab deploy -t lab/mpls-backbone.clab.yml ... +----+---------------------------------+-----------+---------------------------------+ | # | Name | Kind | Image | +----+---------------------------------+-----------+---------------------------------+ | 1 | pe-01 | ceos | ceos-image:4.36.0.1F | | 2 | pe-02 | ceos | ceos-image:4.36.0.1F | | 3 | pe-03 | ceos | ceos-image:4.36.0.1F | | 4 | pe-04 | ceos | ceos-image:4.36.0.1F | | 5 | pe-05 | ceos | ceos-image:4.36.0.1F | | 6 | pe-06 | ceos | ceos-image:4.36.0.1F | | 7 | pe-07 | ceos | ceos-image:4.36.0.1F | | 8 | pe-08 | ceos | ceos-image:4.36.0.1F | | 9 | ce-ib-lon | ceos | ceos-image:4.36.0.1F | | 10 | ce-ib-zrh | ceos | ceos-image:4.36.0.1F | | 11 | ce-trading-lon | ceos | ceos-image:4.36.0.1F | | 12 | ce-trading-zrh | ceos | ceos-image:4.36.0.1F | | 13 | cust-ib-lon | linux | nicolaka/netshoot:latest | | 14 | cust-ib-zrh | linux | nicolaka/netshoot:latest | | 15 | cust-trading-lon | linux | nicolaka/netshoot:latest | | 16 | cust-trading-zrh | linux | nicolaka/netshoot:latest | +----+---------------------------------+-----------+---------------------------------+ ``` On the **isp** dataset the same command brings up just `pe-lon-arista` (Arista cEOS) and `pe-par-nokia` (Nokia SR Linux), with the single backbone link between them. That dataset defines no CE routers, and a site without a `ce_device` contributes no CE node and no PE-CE link, so the isp lab exercises the backbone only — use the **financial** dataset for the PE-CE story. *** ## Push Arista configuration[​](#push-arista-configuration "Direct link to Push Arista configuration") After the lab is up, push the Arista EOS configuration artifact from Infrahub: ``` uv run invoke lab.push-arista ``` This pushes the per-device configs that `invoke lab.deploy` already wrote to `lab/devices/.cfg`, applying each over the containerlab management network via eAPI. It does not re-fetch them, so if you changed a template since the last deploy, re-run `invoke lab.deploy` (or `scripts/fetch_lab_configs.py`) first — otherwise the previous render is what reaches the device. On the default **financial** backbone every PE is Arista (`pe-01`…`pe-08`) plus the four CEs; on the **isp** dataset it is the single `pe-lon-arista` node. The command shows a diff of what changed. Verify with (financial default — substitute `pe-lon-arista` on the isp dataset): ``` containerlab exec -t lab/mpls-backbone.clab.yml --label clab-node-name=pe-01 \ --cmd "Cli -p 15 -c 'show ip bgp summary vrf all'" ``` `containerlab exec` runs its `--cmd` in the container, not in the EOS CLI, so the `Cli -p 15 -c '…'` wrapper is what gets an EOS command executed; without it the node answers `executable file not found in $PATH`. `show ip bgp summary vrf all` covers both the iBGP mesh in the default VRF and each PE-CE session in its customer VRF. Expect seven `Estab` neighbours in `VRF default` on every PE, plus one per L3VPN site on `pe-01` and `pe-08`. Note that the VPNv4 address family has no `show` command on the pinned cEOS 4.29 image — `show bgp vpn-ipv4 summary` returns `% Not supported`. That is a CLI gap, not a signalling failure: the sessions carry the address family, and `show ip route vrf ` on a hub PE lists the customer prefix learned from its CE. *** ## Destroy[​](#destroy "Direct link to Destroy") ``` uv run invoke lab.destroy ``` This calls `containerlab destroy` on the generated topology. The topology file itself is left in place — `invoke lab.status` and `invoke lab.destroy` both read it, so removing it would strand the lab. `invoke lab.deploy` overwrites it from Infrahub on the next run. *** ## Lab status[​](#lab-status "Direct link to Lab status") ``` uv run invoke lab.status ``` Shows container health without modifying anything. *** ## Login credentials[​](#login-credentials "Direct link to Login credentials") containerlab registers each node in its embedded DNS as `clab--`. For the bundled MPLS backbone the lab name is `mpls-backbone-1`, so on the default **financial** dataset the Arista PEs are at `clab-mpls-backbone-1-pe-01` … `clab-mpls-backbone-1-pe-08`. On the **isp** dataset the Arista PE is `clab-mpls-backbone-1-pe-lon-arista` and the Nokia PE is `clab-mpls-backbone-1-pe-par-nokia`. | Node | Image | User | Password | Notes | | ----------------------------------------------------------------------------------------------------- | ----------------- | ------------ | ------------ | ----------------------------------------- | | Arista PE (`pe-01`…`pe-08` on financial, `pe-lon-arista` on isp) | Arista cEOS | `admin` | `admin` | containerlab's cEOS default, demo-only | | Arista CE (`ce-trading-lon`, `ce-trading-zrh`, `ce-ib-lon`, `ce-ib-zrh`) — financial only | Arista cEOS | `admin` | `admin` | Same containerlab default, demo-only | | `pe-par-nokia` (CLI) — isp only | Nokia SR Linux | `admin` | `NokiaSrl1!` | SR Linux 22.11+ vendor default | | `pe-par-nokia` (shell) — isp only | Nokia SR Linux | `linuxadmin` | `NokiaSrl1!` | Shell access, same default | | Customer host (`cust-trading-lon`, `cust-trading-zrh`, `cust-ib-lon`, `cust-ib-zrh`) — financial only | nicolaka/netshoot | `root` | (none) | Plain Linux container, `docker exec` only | To reach the CLIs: ``` # Arista — via SSH or docker exec (financial default; use pe-lon-arista on isp) ssh admin@clab-mpls-backbone-1-pe-01 docker exec -it clab-mpls-backbone-1-pe-01 Cli # Nokia SR Linux (isp dataset only) ssh admin@clab-mpls-backbone-1-pe-par-nokia docker exec -it clab-mpls-backbone-1-pe-par-nokia sr_cli # CE router (financial default) — same cEOS CLI as the PEs ssh admin@clab-mpls-backbone-1-ce-trading-lon docker exec -it clab-mpls-backbone-1-ce-trading-lon Cli # Customer host behind a CE (financial default) — plain shell, no SSH docker exec -it clab-mpls-backbone-1-cust-trading-lon bash ``` The cEOS password (`admin`) is set by the startup-configuration we render — specifically the `username admin privilege 15 role network-admin secret 0 admin` line in `templates/pe_arista_eos.j2`. It's **demo-only**; if you ever publish from this template to real Arista hardware, swap the secret or gate the line behind a flag. The Nokia password is the SR Linux 22.11+ default. Older SR Linux releases used `admin` as the password — adjust if you've pinned a different image tag in `templates/clab_topology.j2`. *** ## Known gaps[​](#known-gaps "Direct link to Known gaps") | Gap | Details | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Nokia SR Linux configuration push | SR Linux uses a different CLI/API from SR OS, and the public SR Linux 23.10 image lacks several constructs SR OS supports (LDP, `l3vpn-ipv4-unicast` afi-safi, `bgp-vpn` under `network-instance`). The dedicated `pe-nokia-srlinux` artifact intentionally renders a **lab-minimum** configuration — hostname + interfaces + ISIS underlay only. BGP and per-VPN ip-vrf signalling are skipped on the SRL lab node. The production-quality configuration lives in the `pe-nokia-sros` artifact (real SR OS syntax) and is meant for a real Nokia box. | | Cisco / Juniper PEs | IOS-XR and Junos container images require vendor licences; these PEs are absent from the v1 lab topology. | | Inter-site VPN forwarding | PE-CE eBGP establishes and every customer prefix reaches the right VRF, but cEOS never starts its LDP agent, so there is no MPLS label path between PEs and traffic does not cross the backbone. See [Troubleshooting](/infrahub-demo-sp/troubleshooting.md#ldp-does-not-start-so-inter-site-vpn-traffic-fails). | | Unmanaged CEs | Only sites with a `ce_device` get a lab CE. On the **isp** dataset no site names one, so that lab has no CE nodes, no PE-CE links, and no customer hosts. | *** ## Image pull failures[​](#image-pull-failures "Direct link to Image pull failures") If containerlab fails with `Error response from daemon: pull access denied`: 1. **SR Linux** (`ghcr.io/nokia/srlinux`) — requires a GitHub login: `docker login ghcr.io -u -p ` 2. **cEOS** — the OpsMill mirror (`registry.opsmill.io/external/ceos-image`) is public and needs no login, so a pull failure here is usually a tag that does not exist rather than an access problem. Note that the mirror's registry metadata is not a reliable guide to architecture: at least one tag declares `arm64` while carrying an x86 image, and pulling it on Apple Silicon succeeds and then fails to boot. Check what you asked for: ``` uname -m echo "${CEOS_IMAGE:-}" ``` To use your own Arista-supplied build instead, import it and point `CEOS_IMAGE` at it: ``` docker import cEOS-lab-4.36.0F.tar.xz ceos:local CEOS_IMAGE=ceos:local uv run invoke lab.deploy ``` 3. **netshoot** (`nicolaka/netshoot`) — public image used by the `cust-*` customer hosts; if it fails, check Docker Hub rate limits or mirror it via a local registry. The CE routers use the same cEOS image as the PEs, so one image covers both. --- # Quick start This page gets you from clone to a running demo in about 10 minutes. For an overview of what `invoke init` creates, see the [landing page](/infrahub-demo-sp.md#architecture-at-a-glance). ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Docker / Docker Compose * Python 3.10+ * [`uv`](https://docs.astral.sh/uv/) * (Optional) containerlab >= 0.50 — only for `invoke lab.deploy`, which has much heavier requirements than the rest of the demo: a **Linux** host and **32 GB+ RAM** for the default dataset's 12 cEOS routers. See [Containerlab prerequisites](/infrahub-demo-sp/lab/containerlab.md#prerequisites). Everything else on this page runs fine on a laptop. * (Optional) [`pnpm`](https://pnpm.io/) — only needed for `uv run invoke docs`, which builds the Docusaurus site locally ## 1. Clone and install[​](#1-clone-and-install "Direct link to 1. Clone and install") ``` git clone https://github.com/opsmill/infrahub-demo-sp.git cd infrahub-demo-sp cp .env.example .env set -a; source .env; set +a uv sync ``` Open `.env` and review the values — see [Configuration](#configuration) below for what each variable does. The defaults are fine for a local demo; you typically only edit `INFRAHUB_SERVICE_CATALOG` (to enable the Streamlit sidecar) and `INFRAHUB_DATASET` (to switch tenant theme). `set -a` is required, and re-running it is required in every new shell: `.env` holds quoted assignments, so a bare `source .env` leaves them as shell variables that `invoke` and `infrahubctl` never see, and the next step fails at "Loading schemas" with `Authentication is required`. ## 2. Start Infrahub and bootstrap data[​](#2-start-infrahub-and-bootstrap-data "Direct link to 2. Start Infrahub and bootstrap data") ``` uv run invoke init ``` This destroys any prior state, starts the containers, loads the schemas, the menu, all bootstrap objects (resource pools plus the selected dataset — its backbone, tenants, and services), and generates schema protocol bindings. The backbone ships inside the dataset: the default **financial** dataset loads 8 Arista cEOS PEs (`pe-01`…`pe-08`) in a partial mesh plus 4 pre-provisioned Arista cEOS CE routers on the two hub PEs, while **isp** loads 4 multi-vendor PEs in a full mesh. Wait \~30 s after the containers come up before bootstrap runs. ## 3. Open the Infrahub UI[​](#3-open-the-infrahub-ui "Direct link to 3. Open the Infrahub UI") Visit `http://localhost:8000` — log in with `admin` / `infrahub`. The sidebar menu shows **Service Catalog → L3 VPNs**, **Topology → MPLS Backbones**, and **MPLS** (ISIS / LDP / MP-BGP processes). ## 4. Start the Streamlit Service Catalog[​](#4-start-the-streamlit-service-catalog "Direct link to 4. Start the Streamlit Service Catalog") Set `INFRAHUB_SERVICE_CATALOG="true"` in `.env`, then run: ``` uv run invoke start ``` Every `invoke start` / `invoke init` will now also build and start the Streamlit sidecar. Visit `http://localhost:8501`. Create your first L3VPN. The Service Catalog also exposes a second wizard — **Create SD-WAN service** — for provisioning Cisco Viptela or Versa Networks SD-WAN overlays. The flow mirrors Create L3VPN: pick vendor and topology, list the sites, submit. See [services/sdwan](/infrahub-demo-sp/services/sdwan.md) for the lifecycle, vendor differences, and known gaps. ## 5. (Optional) Run Batfish validation[​](#5-optional-run-batfish-validation "Direct link to 5. (Optional) Run Batfish validation") Under the sidebar **Validation → Batfish Check** the catalog runs `BatfishBackboneCheck` against the rendered MPLS backbone configs and shows findings bucketed by severity. Each query (parse status, undefined references, BGP session compatibility, IS-IS adjacency mesh) progresses with its own status indicator so a slow Batfish init isn't a blank screen. The Batfish sidecar is enabled by default in `docker-compose.override.yml` and started by `invoke start`. You can also run the same check from the command line: ``` uv run invoke batfish ``` See [validation/batfish](/infrahub-demo-sp/validation/batfish.md) for what each finding means and how severity is tiered. ## 6. (Optional) Bring up the containerlab[​](#6-optional-bring-up-the-containerlab "Direct link to 6. (Optional) Bring up the containerlab") See [`lab/containerlab`](/infrahub-demo-sp/lab/containerlab.md). ## Configuration[​](#configuration "Direct link to Configuration") All configurable behaviour is driven by environment variables in `.env` (loaded with `set -a; source .env; set +a` before any `uv run invoke` command). `uv run invoke info` prints the resolved values. | Variable | Default | Effect | | -------------------------- | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `INFRAHUB_ADDRESS` | `http://localhost:8000` | Where the SDK / Streamlit catalog reach the server | | `INFRAHUB_API_TOKEN` | demo token | Authentication. Rotate before any non-local use | | `INFRAHUB_UI_URL` | `http://localhost:8000` | Used by the Streamlit catalog for "open in Infrahub" links | | `INFRAHUB_GIT_LOCAL` | `false` | `true` registers a `CoreRepository` pointed at the bind-mounted `/upstream` (no GitHub clone needed). `false` registers a `CoreReadOnlyRepository` against the public GitHub repository | | `INFRAHUB_SERVICE_CATALOG` | `false` | `true` builds + starts the Streamlit sidecar on every `invoke start` / `invoke init` | | `INFRAHUB_DATASET` | `financial` | Selects a dataset overlay — one subdirectory of `objects/datasets/`. Each dataset ships its own backbone plus its tenants and services. `financial` (internal bank): 8 Arista cEOS PEs (`pe-01`…`pe-08`) in a partial mesh, 8 division tenants, `trading-floor-vpn`. `isp` (Lumina Networks pan-European ISP): 4 multi-vendor PEs in a full mesh, 8 customer tenants, `kestrel-bank-mpls` | | `INFRAHUB_ENTERPRISE` | `false` | `true` streams the Enterprise edition compose file from `https://infrahub.opsmill.io/enterprise/` instead of the Community one. Requires an Enterprise license / pull credentials | | `INFRAHUB_VERSION` | `stable` | Compose file version tag pulled from `infrahub.opsmill.io` (use `latest`, `stable`, or a specific release like `1.4.0`) | | `INFRAHUB_PORT` | `8000` | Override the Infrahub server port | | `PREFECT_PORT` | `4200` | Override the Prefect task-manager port | | `STREAMLIT_PORT` | `8501` | Override the Streamlit catalog port | To switch dataset: ``` # in .env INFRAHUB_DATASET="isp" ``` …then `uv run invoke init` to wipe and reload from the new dataset. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") See [`troubleshooting`](/infrahub-demo-sp/troubleshooting.md). --- # Schema reference This page covers the SP-demo-specific schema nodes defined in `schemas/sp/`. Base and extension schemas (devices, interfaces, IPAM, routing) come from the Infrahub schema-library and are not repeated here. For the overall schema loading order and layering, see [architecture](/infrahub-demo-sp/architecture.md). *** ## ServiceL3Vpn[​](#servicel3vpn "Direct link to ServiceL3Vpn") **Namespace:** `Service` · **Kind:** `ServiceL3Vpn` **File:** `schemas/sp/service_l3vpn.yml` Represents a single L3VPN service offered to a tenant. One `ServiceL3Vpn` has one or more `ServiceL3VpnSite` attachments (component relationship). ### Attributes[​](#attributes "Direct link to Attributes") | Field | Kind | Required | Description | Who sets it | | ---------------- | -------- | -------- | ------------------------------------------------------------------------------------- | ------------------------------------------------- | | `name` | Text | Required | Unique human name for the VPN (for example, `acme-prod`) | User | | `description` | Text | Optional | Free-text description | User | | `vpn_id` | Number | Required | Unique integer allocated from `vpn_id_pool`. Used to derive RD/RT as `:` | User (catalog allocates from pool) | | `address_family` | Dropdown | Required | `ipv4` or `ipv4_ipv6`. Defaults to `ipv4` | User | | `status` | Dropdown | Required | `draft` → `active` → `decommissioned`. Defaults to `draft` | User sets `draft`; generator promotes to `active` | ### Relationships[​](#relationships "Direct link to Relationships") | Field | Peer | Cardinality | Required | Description | Who sets it | | -------------- | ------------------------- | ----------- | -------- | ------------------------------------------------------------ | -------------------------------------------- | | `tenant` | `OrganizationGeneric` | one | Required | Owning organisation | User | | `vrf` | `IpamVRF` | one | Optional | VRF materialised by the generator | Generator | | `customer_asn` | `RoutingAutonomousSystem` | one | Optional | The customer AS used on every PE-CE eBGP session of this VPN | Generator allocates from `customer_asn_pool` | | `sites` | `ServiceL3VpnSite` | many | Optional | PE attachment points (component) | User / catalog | *** ## ServiceL3VpnSite[​](#servicel3vpnsite "Direct link to ServiceL3VpnSite") **Namespace:** `Service` · **Kind:** `ServiceL3VpnSite` **File:** `schemas/sp/service_l3vpn.yml` Represents a single PE attachment of an L3VPN to a customer subnet. One site = one PE + one customer-facing interface + a /30 PE-CE link. ### Attributes[​](#attributes-1 "Direct link to Attributes") | Field | Kind | Required | Description | Who sets it | | ------------------ | -------- | -------- | ------------------------------------------------------------------------------------- | ------------------------------ | | `name` | Text | Required | Unique within a given `l3vpn` (uniqueness constraint) | User | | `routing_protocol` | Dropdown | Required | `ebgp`, `static`, or `connected` | User | | `bgp_peer_asn` | Number | Optional | Per-site override for the PE-CE peer AS. Leave unset to use `customer_asn` on the VPN | User | | `static_routes` | JSON | Optional | List of static routes to inject into the VRF | User | | `status` | Dropdown | Required | `provisioning` → `active` → `decommissioned`. Defaults to `provisioning` | Generator promotes to `active` | ### Relationships[​](#relationships-1 "Direct link to Relationships") | Field | Peer | Cardinality | Required | Description | Who sets it | | ----------------- | ------------------- | ----------- | -------- | ----------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | | `l3vpn` | `ServiceL3Vpn` | one | Required | Parent VPN (parent relationship) | User / catalog | | `pe_device` | `DcimDevice` | one | Required | PE router for this attachment | User | | `pe_interface` | `InterfacePhysical` | one | Optional | PE-facing physical interface | Generator: reuses a port already described `L3VPN `, else allocates a free one | | `ce_device` | `DcimDevice` | one | Optional | Pre-provisioned CE router this site terminates on. Unset = unmanaged CE | User | | `ce_interface` | `InterfacePhysical` | one | Optional | CE port facing the PE | User | | `customer_subnet` | `IpamPrefix` | one | Required | Customer prefix to be placed into the VRF | User | | `pe_address` | `IpamIPAddress` | one | Optional | PE IP on the PE-CE /30 link | Generator allocates from `pe_ce_pool` | | `ce_address` | `IpamIPAddress` | one | Optional | CE IP on the PE-CE /30 link | Generator allocates from `pe_ce_pool` | *** ## ServiceSdwan[​](#servicesdwan "Direct link to ServiceSdwan") **Namespace:** `Service` · **Kind:** `ServiceSdwan` **File:** `schemas/sp/service_sdwan.yml` Represents a single SD-WAN service offered to a tenant. One `ServiceSdwan` has one or more `ServiceSdwanSite` attachments (component relationship). ### Attributes[​](#attributes-2 "Direct link to Attributes") | Field | Kind | Required | Description | Who sets it | | ------------- | -------- | -------- | --------------------------------------------------------------------------------- | ------------------------------------------------- | | `name` | Text | Required | Unique human name for the SD-WAN service (for example, `treasury-branch-sdwan`) | User | | `description` | Text | Optional | Free-text description | User | | `service_id` | Number | Required | Unique integer allocated from `sdwan_id_pool`. Used for SD-WAN site-id derivation | User (catalog allocates from pool) | | `vendor` | Dropdown | Required | `viptela` (Cisco SD-WAN) or `versa` (Versa Networks). Defaults to `viptela` | User | | `topology` | Dropdown | Required | `full-mesh` or `hub-spoke`. Defaults to `full-mesh` | User | | `status` | Dropdown | Required | `draft` → `active` → `decommissioned`. Defaults to `draft` | User sets `draft`; generator promotes to `active` | ### Relationships[​](#relationships-2 "Direct link to Relationships") | Field | Peer | Cardinality | Required | Description | Who sets it | | -------- | --------------------- | ----------- | -------- | ----------------------------- | -------------- | | `tenant` | `OrganizationGeneric` | one | Required | Owning organisation | User | | `sites` | `ServiceSdwanSite` | many | Optional | SD-WAN edge sites (component) | User / catalog | *** ## ServiceSdwanSite[​](#servicesdwansite "Direct link to ServiceSdwanSite") **Namespace:** `Service` · **Kind:** `ServiceSdwanSite` **File:** `schemas/sp/service_sdwan.yml` Represents a single SD-WAN edge site: one location, one edge device, one LAN subnet, one LAN IP. The edge `DcimDevice` is created automatically by the generator (it isn't pre-seeded in any dataset). ### Attributes[​](#attributes-3 "Direct link to Attributes") | Field | Kind | Required | Description | Who sets it | | -------- | -------- | -------- | ------------------------------------------------------------------------ | ------------------------------ | | `name` | Text | Required | Unique within a given `sdwan` (uniqueness constraint) | User | | `role` | Dropdown | Required | `hub`, `spoke`, or `branch`. Defaults to `spoke` | User | | `status` | Dropdown | Required | `provisioning` → `active` → `decommissioned`. Defaults to `provisioning` | Generator promotes to `active` | ### Relationships[​](#relationships-3 "Direct link to Relationships") | Field | Peer | Cardinality | Required | Description | Who sets it | | ------------- | --------------- | ----------- | -------- | ------------------------------------------- | ------------------------------------- | | `sdwan` | `ServiceSdwan` | one | Required | Parent SD-WAN service (parent relationship) | User / catalog | | `location` | `LocationSite` | one | Required | Customer site / PoP this edge attaches to | User | | `lan_subnet` | `IpamPrefix` | one | Required | Customer LAN subnet behind this edge | User | | `lan_address` | `IpamIPAddress` | one | Optional | Edge's LAN IP (network address + 1) | Generator allocates from `lan_subnet` | | `sdwan_edge` | `DcimDevice` | one | Optional | Edge router materialised at this site | Generator creates per-site | *** ## TopologyMplsBackbone[​](#topologymplsbackbone "Direct link to TopologyMplsBackbone") **Namespace:** `Topology` · **Kind:** `TopologyMplsBackbone` **File:** `schemas/sp/topology_mpls.yml` Inherits from `TopologyGeneric`. Represents the MPLS backbone as a whole — one row per backbone domain. The demo has exactly one row: `mpls-backbone-1`. ### Attributes[​](#attributes-4 "Direct link to Attributes") | Field | Kind | Required | Description | Who sets it | | ------------ | -------- | -------- | -------------------------------------------------------- | ---------------- | | `name` | Text | Required | Unique backbone name | User (bootstrap) | | `isis_area` | Text | Required | ISIS area ID (default `49.0001`) | User (bootstrap) | | `isis_level` | Dropdown | Required | `level-1`, `level-2`, or `level-1-2` (default `level-2`) | User (bootstrap) | ### Relationships[​](#relationships-4 "Direct link to Relationships") | Field | Peer | Cardinality | Required | Description | Who sets it | | ----- | ------------------------- | ----------- | -------- | -------------------------------------- | ---------------- | | `asn` | `RoutingAutonomousSystem` | one | Required | Provider ASN used for RD/RT derivation | User (bootstrap) | | `pes` | `DcimDevice` | many | Optional | All PE routers in this backbone | User (bootstrap) | *** ## MplsIsisProcess[​](#mplsisisprocess "Direct link to MplsIsisProcess") **Namespace:** `Mpls` · **Kind:** `MplsIsisProcess` **File:** `schemas/sp/mpls.yml` Inherits from `RoutingProtocol`. One row per PE representing the IS-IS routing process. The `device` relationship is inherited from `RoutingProtocol`. ### Attributes[​](#attributes-5 "Direct link to Attributes") | Field | Kind | Required | Description | Who sets it | | --------- | -------- | -------- | ------------------------------------------------------------- | ---------------- | | `area_id` | Text | Required | IS-IS area (default `49.0001`) | User (bootstrap) | | `level` | Dropdown | Required | `level-1`, `level-2`, or `level-1-2` (default `level-2`) | User (bootstrap) | | `net_id` | Text | Required | ISO NET identifier (for example, `49.0001.0100.0000.0001.00`) | User (bootstrap) | ### Relationships[​](#relationships-5 "Direct link to Relationships") | Field | Peer | Cardinality | Required | Description | | ------------ | ------------------- | ----------- | -------- | --------------------------------- | | `interfaces` | `InterfacePhysical` | many | Optional | Backbone interfaces running IS-IS | ### Inherited from RoutingProtocol[​](#inherited-from-routingprotocol "Direct link to Inherited from RoutingProtocol") | Field | Description | | ------------- | ------------------------------------- | | `device` | The PE device this process belongs to | | `description` | Free-text description | | `status` | `active` / `inactive` | *** ## MplsLdpProcess[​](#mplsldpprocess "Direct link to MplsLdpProcess") **Namespace:** `Mpls` · **Kind:** `MplsLdpProcess` **File:** `schemas/sp/mpls.yml` Inherits from `RoutingProtocol`. One row per PE representing the LDP label distribution process. ### Attributes[​](#attributes-6 "Direct link to Attributes") | Field | Kind | Required | Description | Who sets it | | ----------- | ---- | -------- | ------------------------------------------ | ---------------- | | `router_id` | Text | Required | LDP router-ID (typically the Loopback0 IP) | User (bootstrap) | ### Relationships[​](#relationships-6 "Direct link to Relationships") | Field | Peer | Cardinality | Required | Description | | ------------------- | ------------------- | ----------- | -------- | ------------------------------- | | `transport_address` | `IpamIPAddress` | one | Optional | LDP transport address override | | `interfaces` | `InterfacePhysical` | many | Optional | Backbone interfaces running LDP | *** ## MplsBgpProcess[​](#mplsbgpprocess "Direct link to MplsBgpProcess") **Namespace:** `Mpls` · **Kind:** `MplsBgpProcess` **File:** `schemas/sp/mpls.yml` Inherits from `RoutingProtocol`. One row per PE representing the MP-BGP overlay process carrying VPNv4 / VPNv6 address families. ### Attributes[​](#attributes-7 "Direct link to Attributes") | Field | Kind | Required | Description | Who sets it | | ------------------ | ---- | -------- | ------------------------------------------------------------------- | ---------------- | | `router_id` | Text | Required | BGP router-ID (typically the Loopback0 IP) | User (bootstrap) | | `address_families` | List | Required | MP-BGP address families enabled (for example, `["vpnv4", "vpnv6"]`) | User (bootstrap) | ### Relationships[​](#relationships-7 "Direct link to Relationships") | Field | Peer | Cardinality | Required | Description | | ---------- | ------------------- | ----------- | -------- | ------------------------------ | | `sessions` | `RoutingBGPSession` | many | Optional | iBGP sessions for this process | *** ## LocationRegion / LocationSite[​](#locationregion--locationsite "Direct link to LocationRegion / LocationSite") **Namespace:** `Location` · **File:** `schemas/sp/location_concrete.yml` Concrete location kinds for this demo. `LocationRegion` inherits from `LocationGeneric` (the schema-library generic), and `LocationSite` adds hosting capability via `LocationHosting`. Both render under `LocationGeneric` in the sidebar menu. The bootstrap data uses `LocationRegion` for `emea` and `LocationSite` for the four PoPs (`lon`, `fra`, `ams`, `par`). | Kind | Inherits | Purpose | | ---------------- | ------------------------------------ | --------------------------------------- | | `LocationRegion` | `LocationGeneric` | Geographic region (for example, `emea`) | | `LocationSite` | `LocationGeneric`, `LocationHosting` | PoP / site that hosts PEs | *** ## DcimDevice role extension[​](#dcimdevice-role-extension "Direct link to DcimDevice role extension") **File:** `schemas/sp/dcim_role_pe.yml` This file is a schema *extension* (not a new node) that adds SP-specific choices to the existing `DcimDevice.role` dropdown: | Role | Label | Use | | ------ | -------------------------- | ----------------------------------------- | | `pe` | Provider Edge | PE router terminating customer VPNs | | `p` | Provider Core | P router (transit only, no customer VRFs) | | `rr` | Route Reflector | BGP route reflector | | `core` | Core Router | Central part of the network | | `edge` | Edge Router | Network boundary with external networks | | `cpe` | Customer Premise Equipment | Devices at the customer's premises | The demo bootstrap sets `role = pe` on every backbone PE — 8 on the default **financial** dataset (`pe-01`…`pe-08`), 4 on the **isp** dataset. *** ## Resource pools[​](#resource-pools "Direct link to Resource pools") The following pools are bootstrapped in `objects/50_pools.yml` and consumed by the generator and the Streamlit catalog: | Pool name | Kind | Purpose | Allocation unit | | ------------------- | ------------------ | -------------------------------------------------- | ---------------------------------------------------------------------------------- | | `vpn_id_pool` | `CoreNumberPool` | Globally unique VPN IDs | Integer (range 200–9999; values 100–199 are reserved for bootstrap-seeded L3VPNs) | | `sdwan_id_pool` | `CoreNumberPool` | Globally unique SD-WAN service IDs | Integer (range 200–9999; values 100–199 are reserved for bootstrap-seeded SD-WANs) | | `pe_loopback_pool` | `CoreIPPrefixPool` | Loopback /32 addresses | /32 from `10.0.0.0/24` | | `backbone_p2p_pool` | `CoreIPPrefixPool` | Backbone p2p /31 links | /31 from `10.1.0.0/16` | | `pe_ce_pool` | `CoreIPPrefixPool` | PE-CE /30 links per L3VPN site | /30 from `10.100.0.0/16` | | `customer_asn_pool` | `CoreNumberPool` | Customer AS numbers for PE-CE eBGP — one per L3VPN | Integer (range 65100–65199; the backbone AS 65000 sits outside it) | *** ## User-provided vs generator-filled summary[​](#user-provided-vs-generator-filled-summary "Direct link to User-provided vs generator-filled summary") The following table consolidates which fields an operator must supply and which the `L3VpnGenerator` fills in automatically: | Object | Field | Source | | ------------------------------------- | ------------------------------------------------------------------- | ----------------------------- | | `ServiceL3Vpn` | `name`, `vpn_id`, `address_family`, `tenant` | User / catalog | | `ServiceL3Vpn` | `vrf`, `customer_asn`, `status = active` | Generator | | `ServiceL3VpnSite` | `name`, `pe_device`, `routing_protocol`, `customer_subnet`, `l3vpn` | User / catalog | | `ServiceL3VpnSite` | `bgp_peer_asn`, `ce_device`, `ce_interface` | User / catalog (all optional) | | `ServiceL3VpnSite` | `pe_interface`, `pe_address`, `ce_address`, `status = active` | Generator | | `IpamVRF` | all | Generator | | `IpamIPAddress` (PE-CE) | all | Generator | | `RoutingAutonomousSystem` (customer) | all — ASN from `customer_asn_pool` | Generator | | `RoutingBGPSession` (PE-CE and CE-PE) | all (only when `routing_protocol = ebgp`) | Generator | --- # L3VPN service This page documents the L3VPN service end-to-end: how an operator creates one through the Streamlit catalog, what Infrahub does automatically, and how each vendor's configuration differs. For field-level schema details see [schema-reference](/infrahub-demo-sp/schema-reference.md). *** ## User flow[​](#user-flow "Direct link to User flow") ### Step-by-step[​](#step-by-step "Direct link to Step-by-step") 1. **Catalog form** — The operator fills in the VPN name, selects a tenant, picks a PE and a customer subnet for each site, and chooses a routing protocol (eBGP / static / connected). 2. **Branch + objects** — The catalog allocates a `vpn_id` from `vpn_id_pool`, opens a feature branch, and writes `ServiceL3Vpn` + `ServiceL3VpnSite` objects to Infrahub. All objects start with `status = draft / provisioning`. 3. **Generator run (event-driven)** — Once the sites exist, the catalog adds the `ServiceL3Vpn` to the `l3vpns` group. That membership change fires the `trigger-l3vpn-generator` group trigger (a `CoreGroupTriggerRule` + `CoreGeneratorAction`, see `objects/events/00_triggers.yml`), which runs `L3VpnGenerator` **on the branch**. The generator is deliberately **not** run inside the proposed-change pipeline (`execute_in_proposed_change: false` in `.infrahub.yml`): in the pipeline it races artifact rendering, which render before the VRF/IPs exist and produce an empty configuration diff. Running it on the branch-change event guarantees the data lands before artifacts render. `L3VpnGenerator` does all heavy lifting: * Creates an `IpamVRF` with `vrf_rd = :`, bound to the namespace the customer prefixes were created in, so two customers may use the same private prefix. * Creates matching import and export route targets. * Selects the lowest-numbered free interface on the PE. * Allocates a /30 from `pe_ce_pool` and creates the PE and CE IP addresses. * Places the customer subnet prefix into the VRF. * Hangs a dot1q sub-interface off the private port on the CE, carrying a VLAN from the `vlan_pool` named by the VPN, and the customer LAN gateway address. * Creates a `RoutingBGPSession` (eBGP, `session_type = EXTERNAL`) when `routing_protocol = ebgp`. * Promotes `ServiceL3Vpn.status` to `active` and `ServiceL3VpnSite.status` to `active`. **Editing a service that is already in the group** does not change membership, so the group rule cannot fire. A second rule, `trigger-l3vpn-generator-sites-changed` (a `CoreNodeTriggerRule` on `ServiceL3Vpn`), covers adding a site to — or removing one from — an existing service; removal matters because the re-run is what lets the generator reap the PE port, /30 and session the deleted site left behind. Editing an existing site's own fields — `customer_subnet`, `bgp_peer_asn`, `ce_device` — fires nothing: an attribute change on the child produces no event for the parent, and a rule scoped to the site cannot dispatch a generator whose target group holds VPNs. To force a run on the branch, remove the VPN from the `l3vpns` group and add it back; on `main`, run `uv run python scripts/run_generator.py generate_l3vpn`. 4. **Proposed Change** — Once the generator has finished (service `active`) and the per-PE artifacts are rendered, the catalog opens a `CoreProposedChange` from the feature branch into `main` for review. 5. **Checks** — Six checks must pass before merge is allowed (see [Checks](#checks) below). 6. **Transforms** — Configuration artifacts are rendered for each PE that has at least one L3VPN site. 7. **Merge** — The operator reviews the diff in the Infrahub UI and merges. On the target platform the artifact is pushed via the relevant `invoke` task (for example, `invoke lab.push-arista`). If the merge is refused because a `Schema Integrity` validator failed while every check listed below passed, see [Troubleshooting](/infrahub-demo-sp/troubleshooting.md#proposed-change-schema-integrity-failure-blocks-the-merge) — it is an intermittent false positive, and recreating the service on a fresh branch clears it. *** ## Checks[​](#checks "Direct link to Checks") | Check | File | What it enforces | | ------------------------ | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `l3vpn_overlap` | `checks/l3vpn_overlap.py` | No two `ServiceL3Vpn` objects share an RD (derived from `vpn_id`) | | `l3vpn_site_subnet` | `checks/l3vpn_site_subnet.py` | The customer subnet is not already claimed by another site in the same VRF | | `l3vpn_peer_asn_range` | `checks/l3vpn_peer_asn_range.py` | No site's `bgp_peer_asn` override falls inside `customer_asn_pool`'s range, where the pool could later issue the same number to another customer | | `pe_interface_alloc` | `checks/pe_interface_alloc.py` | No PE interface is claimed by more than one L3VPN site | | `backbone_session_count` | `checks/backbone_session_count.py` | Every PE in the backbone has its full-mesh complement of N−1 iBGP sessions (N = PE count — 7 each on the 8-PE financial backbone, 3 each on the 4-PE isp backbone) | | `batfish_backbone` | `checks/batfish_backbone.py` | Static validation of the rendered backbone configs via Batfish — see [Batfish validation](/infrahub-demo-sp/validation/batfish.md) for the full query battery | A failing check blocks merge of the proposed change. The operator can see the check result in the Infrahub UI under the proposed change's **Checks** tab. *** ## Customer address space[​](#customer-address-space "Direct link to Customer address space") Each L3VPN's customer prefixes live in an `IpamNamespace` of their own, named `vrf-`. `IpamPrefix` and `IpamIPAddress` are unique on `[value, ip_namespace]`, so without one, two customers using the same private prefix would resolve to a single row and their services would compete for its VRF — and both shipped datasets do hand `10.200.10.0/24` to a different customer. The namespace is created by whoever creates the prefix: the datasets declare it, and the Service Catalog creates it before the prefix. The generator only reads it, off the site's own `customer_subnet`, and puts the LAN gateway in the same namespace. It deliberately never writes the namespace node — generators run with `delete_unused_nodes=True`, so a node the generator saves joins its tracking group, and any later run that does not save it makes the reaper try to delete it. For a namespace still holding customer prefixes that delete fails and takes the whole generator run down with an unreadable error. A prefix created by hand, without a namespace, lands in `default` and still works — it just gets no isolation. Provider-owned space stays in `default` throughout: the PE-CE /30s come from `pe_ce_pool`, and pool allocations take the namespace of the pool. *** ## Generator: what it creates[​](#generator-what-it-creates "Direct link to Generator: what it creates") | Object | Created by | Notes | | ------------------------------------ | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `IpamVRF` | `_ensure_vrf` | Name = VPN name; `vrf_rd` = `:`; bound to the namespace read off the customer prefix | | `IpamRouteTarget` | `find_or_create_route_target` | Same value used for import and export RT | | `RoutingAutonomousSystem` (customer) | `_ensure_customer_as` | One per VPN; ASN allocated from `customer_asn_pool`. Skipped when every eBGP site sets `bgp_peer_asn` | | `InterfacePhysical` (updated) | `_ensure_pe_interface` | Reuses a PE port already described `L3VPN ` (pre-wired CE links); otherwise the lowest free-status interface. Role set to `cust`, status to `active` | | `IpamPrefix` (/30) | `allocate_prefix_from_pool` | Allocated from `pe_ce_pool`; placed in VRF | | `IpamIPAddress` × 2 | `_ensure_ip_address` | `.1` = PE, `.2` = CE within the /30. The CE address is attached to `ce_interface` when the site names one | | `RoutingBGPSession` (PE side) | `_ensure_ebgp_session` | Only when `routing_protocol = ebgp`; `session_type = EXTERNAL`, bound to the VRF | | `RoutingBGPSession` (CE side) | `_bind_ce_side` | Only when the site names a `ce_device`; the mirror of the PE session, with no VRF (a CE is not VPN-aware) | | `InterfaceVirtual` (dot1q) | `_ensure_private_vlan` | Only when the site names a `ce_private_interface` and the VPN a `vlan_pool`. One sub-interface per site, so a CE port shared by two services carries one VLAN each | | `IpamIPAddress` (LAN gateway) | `_ensure_private_vlan` | First usable address of the `customer_subnet` for that site, on the dot1q sub-interface, in the per-VPN namespace | The generator is idempotent: re-running it on an L3VPN that already has a VRF and allocated addresses will skip re-creation and only update missing fields. *** ## Per-vendor configuration[​](#per-vendor-configuration "Direct link to Per-vendor configuration") The transform layer renders a full configuration fragment for each PE when that PE has at least one active L3VPN site. Configuration sections differ by vendor. On the default **financial** backbone every PE is Arista, so `trading-floor-vpn` and `ib-advisory-vpn` render the Arista EOS form at all their sites; the **isp** dataset, with one PE per vendor, exercises all four forms below: ### Arista EOS[​](#arista-eos "Direct link to Arista EOS") ``` vrf instance rd : ! router bgp vrf rd : route-target import route-target export neighbor remote-as network ``` ### Cisco IOS-XR[​](#cisco-ios-xr "Direct link to Cisco IOS-XR") ``` vrf address-family ipv4 unicast import route-target export route-target ! router bgp vrf rd : address-family ipv4 unicast neighbor remote-as ``` ### Juniper Junos[​](#juniper-junos "Direct link to Juniper Junos") ``` routing-instances { { instance-type vrf; interface ; vrf-target target:; protocols { bgp { group PE-CE { neighbor { peer-as ; } } } } } } ``` ### Nokia SR OS[​](#nokia-sr-os "Direct link to Nokia SR OS") ``` service { vprn "" customer "1" create route-distinguisher vrf-target target: interface "" create address / sap :0 create exit exit bgp-vpn-backup no shutdown exit } ``` --- # SD-WAN service The SD-WAN service models a customer's SD-WAN overlay across one or more sites. Each site has a dedicated edge device (a per-site `DcimDevice` created automatically by the generator). The default vendor is Cisco Viptela (cEdge / IOS-XE SD-WAN); Versa Networks VOS is available as an alternate. For field-level schema details see [schema-reference](/infrahub-demo-sp/schema-reference.md). *** ## User flow[​](#user-flow "Direct link to User flow") ### Step-by-step[​](#step-by-step "Direct link to Step-by-step") 1. **Catalog form** — Operator picks vendor (Viptela / Versa), topology (hub-spoke / full-mesh), and lists one site per location with a LAN subnet. 2. **Branch + objects** — Catalog allocates a `service_id` from `sdwan_id_pool`, opens a feature branch, writes `ServiceSdwan` and `ServiceSdwanSite` rows, and adds the service to the `sdwans` group. 3. **Generator** — `SdwanGenerator` fires automatically on group membership. For each site it creates one edge `DcimDevice` (`--edge`) with the vendor's platform / device type / manufacturer, adds the device to the vendor-specific edge group (`sdwan_viptela_edges` or `sdwan_versa_edges`), allocates a LAN address from the site's subnet, and flips both site and service status to `active`. 4. **Artifacts** — Infrahub renders one configuration per edge via the matching transform (`sdwan_viptela` or `sdwan_versa`). 5. **Proposed Change** — Reviewable diff in the UI; merging promotes the service to `main`. *** ## Schema shape[​](#schema-shape "Direct link to Schema shape") The shape parallels `ServiceL3Vpn`: * `ServiceSdwan` — name, `service_id`, `vendor`, `topology`, tenant, sites. * `ServiceSdwanSite` — name, `role` (hub / spoke / branch), `location`, `lan_subnet`, `lan_address`, `sdwan_edge`. *** ## Vendor differences[​](#vendor-differences "Direct link to Vendor differences") | Aspect | Viptela (cEdge) | Versa (FlexVNF) | | -------------------- | ------------------------------------------------ | ----------------------------------------- | | Platform | `cisco_viptela` | `versa_flexvnf` | | Device type | `cEdge-1000` | `FlexVNF-200` | | Edge group | `sdwan_viptela_edges` | `sdwan_versa_edges` | | Artifact definition | `sdwan-viptela-config` | `sdwan-versa-config` | | Configuration flavor | IOS-XE SD-WAN CLI (`system` / `sdwan` / `vpn N`) | Versa VOS CLI (`set orgs org-services …`) | *** ## Checks[​](#checks "Direct link to Checks") * `sdwan_id_overlap` — no two services share `service_id`. Safety net behind the pool. * `sdwan_site_subnet` — no two sites within the same service have overlapping LAN subnets. *** ## Known gaps[​](#known-gaps "Direct link to Known gaps") * **No SD-WAN controllers** modelled — vManage / vSmart / vBond and Versa Director / Analytics are out of scope for v1. * **No transport circuits** — edges have only a LAN-side address; no MPLS-vs-Internet-vs-LTE distinction. * **No overlay tunnels or BGP** in rendered configs — templates emit the intent (system identity, `vpn 1` LAN block, organization name) but peer-site lists are comments only. * **No containerlab support** — the `clab-mpls-topology` artifact stays MPLS-only. SD-WAN edges would require a `data.ServiceSdwanSite.edges` loop in the clab template and a new `linux` CE per SD-WAN site. --- # Troubleshooting Common failure modes and their fixes. If your problem is not here, check the [Infrahub documentation](https://docs.infrahub.app) or open an issue. *** ## Bootstrap timeout[​](#bootstrap-timeout "Direct link to Bootstrap timeout") **Symptom:** ``` [ERROR] ConnectionRefusedError: [Errno 111] Connection refused ``` or the bootstrap script exits immediately with "Infrahub not reachable". **Cause:** The bootstrap script runs too soon after `docker compose up`. Infrahub takes 20–40 seconds to initialise its database and API server. **Fix:** ``` # Wait for Infrahub to be ready, then bootstrap manually: uv run invoke start sleep 40 uv run invoke bootstrap ``` Alternatively, run `invoke init` which includes a built-in wait, but if your machine is slow, increase the sleep in `tasks.py`: ``` # tasks.py — find the sleep call in init_demo and increase it time.sleep(40) # change to 60 or more on slow machines ``` *** ## Generator fails: "No free physical interface"[​](#generator-fails-no-free-physical-interface "Direct link to Generator fails: \"No free physical interface\"") **Symptom:** ``` RuntimeError: No free physical interface on pe-01 ``` **Cause:** The `L3VpnGenerator` first looks for an `InterfacePhysical` on the target PE already described `L3VPN ` — that is how a pre-wired PE-CE port is bound. When there is no such port it falls back to the lowest-numbered interface with `status = free`, and this error means there is none left: every interface on that PE is already `active`. On the **financial** dataset each PE ships with two free customer-facing spares; the ports wired to the pre-provisioned CEs (`Ethernet3`/`Ethernet4` on `pe-01` and `pe-08`) are deliberately `active` so they can never be handed to another service. **Fix:** Create a free interface on the PE via the Infrahub UI or GraphQL: ``` curl -s -X POST http://localhost:8000/graphql \ -H "X-INFRAHUB-KEY: 06438eb2-8019-4776-878c-0941b1f1d1ec" \ -H "Content-Type: application/json" \ -d '{ "query": "mutation { InterfacePhysicalCreate(data: { name: {value: \"Ethernet10\"}, status: {value: \"free\"}, mtu: {value: 9000}, device: {hfid: [\"pe-01\"]} }) { ok object { id } } }" }' ``` Repeat for whichever PE is out of free interfaces. *** ## Configuration push fails: "not supported on this hardware platform"[​](#configuration-push-fails-not-supported-on-this-hardware-platform "Direct link to Configuration push fails: \"not supported on this hardware platform\"") **Symptom:** ``` eAPI error: CLI command 11 of 57 'ip routing' failed: invalid command last result: {'errors': ["Unavailable command (not supported on this hardware platform) (at token 1: 'routing')"]} ``` **Cause:** The node answered eAPI before its agents finished registering their CLI commands, so the parser genuinely did not know the `routing` keyword yet. It is a timing problem, not a configuration problem — the same file pushes cleanly to the nodes that had longer to boot, which is why the failure tends to appear part-way through a run. **Fix:** None needed. `scripts/push_arista.py` polls a read-only command served by the same agent that owns `ip routing`, and only pushes once that answers. If it times out instead, the node is genuinely stuck — check `docker logs clab-mpls-backbone-1-`. Booting twelve cEOS nodes is memory-hungry; give Docker more RAM if nodes are being killed. *** ## Pushed configuration does not match Infrahub[​](#pushed-configuration-does-not-match-infrahub "Direct link to Pushed configuration does not match Infrahub") **Symptom:** The pushed command counts or interface names do not match what Infrahub shows, or a device rejects interfaces the topology does not have. **Cause:** `lab/devices/*.cfg` are fetched once by `invoke lab.deploy` and then reused by `invoke lab.push-arista`. After the dataset or a service changes, the files on disk are stale. **Fix:** Re-run `invoke lab.deploy`. It re-fetches the topology artifact and every per-device configuration, then redeploys the lab so the running nodes and the configuration files agree. *** ## LDP does not start, so inter-site VPN traffic fails[​](#ldp-does-not-start-so-inter-site-vpn-traffic-fails "Direct link to LDP does not start, so inter-site VPN traffic fails") **Symptom:** Every prefix is exchanged correctly — IS-IS is up, the iBGP mesh is up, PE-CE eBGP is up, and each PE holds the customer prefixes in the right VRF — but a host on one customer site cannot reach the same customer's other site. On any PE: ``` show mpls ldp neighbor ! Agent 'LdpAgent' is not running ! LDP is not enabled ``` **Cause:** Not a configuration defect. The rendered PE configuration contains the full `mpls ldp` block, including `no shutdown` and the per-interface statements, and it pushes without error — but cEOS never starts the LDP agent. It appears to be a limitation of the cEOS-lab image rather than something the configuration can influence. **Consequence:** An L3VPN needs an MPLS label path between PEs to carry VPN traffic. Without LDP there is no transport, so VPNv4 routes cannot be used for forwarding even though BGP has exchanged them. The control plane is complete and demonstrable; the data plane between sites is not. **What still works:** everything up to the label path — IS-IS adjacencies, the full iBGP mesh, PE-CE eBGP, VRF route import/export, and customer LAN reachability *within* a site (host to its own CE gateway over the tagged VLAN). **Fix:** Unverified. This was diagnosed against cEOS 4.29.0.2F (which reports itself as 4.28.9M), whose LDP agent never starts. The lab now runs 4.36.0.1F, so the gap may already be closed — if you can confirm either way on a real deploy (which needs an x86\_64 host), update this page. To test another build, point `CEOS_IMAGE` at it (`CEOS_IMAGE= uv run invoke lab.deploy`). Segment routing instead of LDP would be the other option, and a larger change. *** ## Containerlab deploy hangs at "Running postdeploy actions"[​](#containerlab-deploy-hangs-at-running-postdeploy-actions "Direct link to Containerlab deploy hangs at \"Running postdeploy actions\"") **Symptom:** `invoke lab.deploy` creates every node and every link within a second or two, then stops making progress. The last lines repeat once per cEOS node and nothing follows them, for tens of minutes: ``` INFO Created link: pe-07:eth4 ▪-▪ pe-08:eth2 INFO Running postdeploy actions for Arista cEOS 'pe-07' node INFO Running postdeploy actions for Arista cEOS 'pe-08' node ``` There is no error, no traceback and no exit — `containerlab inspect` shows the containers up, and `docker logs` on a node is empty or stops early. **Cause:** the nodes booted as containers but EOS never came up inside them. containerlab's cEOS postdeploy step opens the EOS CLI (`docker exec … Cli`) and waits for a prompt before pushing the management and interface configuration. That wait is an unbounded retry loop — `for { …; time.Sleep(2s); continue }` in containerlab's `utils/networkcli.go`, with no deadline and no attempt cap — so an EOS that never finishes booting shows up as an indefinite stall rather than a failure. `invoke lab.deploy` bounds it at 30 minutes and prints the per-node container state on timeout, which is what separates a node that died from one still booting. Adjust with `LAB_DEPLOY_TIMEOUT_SECONDS`. Three things cause it: 1. **An ARM host.** The most common one, and the one that looks least like itself. cEOS-lab is 32-bit x86 in every published build and Rosetta 2 does not translate 32-bit x86, so on Apple Silicon EOS reaches `agentsToStart=[...]`, loops on `ProcMgr.service`, and exits 255 roughly 17 minutes later — while the `cust-*` netshoot hosts stay up, so `docker ps` still looks like a deployed lab. Confirm with: ``` uname -m # aarch64 / arm64 = unsupported docker ps -a --filter name=clab- --format '{{.Names}} {{.Status}}' ``` `Exited (255)` on the `ceos` nodes with the `cust-*` hosts up is the signature. `invoke lab.deploy` now refuses ARM hosts before deploying, so this only appears on an older checkout or with `LAB_ALLOW_UNSUPPORTED_ARCH=true`. There is no image that fixes it — see [Architecture](/infrahub-demo-sp/lab/containerlab.md#architecture). 2. **cEOS-lab older than 4.32.0F on a cgroups v2 host.** Earlier builds require cgroups v1 and silently fail to boot otherwise. cgroups v2 is the default on Ubuntu 21.04+, OrbStack, and most current distributions. Check the host: ``` stat -fc %T /sys/fs/cgroup/ # cgroup2fs = v2, tmpfs = v1 ``` From 4.32.0F onward cEOS-lab detects the host's cgroup version itself. The images `invoke lab.deploy` selects are all newer than that, so this only bites a pinned older build — check `CEOS_IMAGE`. 3. **Not enough memory.** cEOS wants \~2 GB per node, and the default **financial** dataset runs 12 of them. On an undersized host the nodes thrash or get OOM-killed mid-boot, which looks identical from the outside: ``` free -h dmesg -T | grep -iE 'oom|killed process' docker stats --no-stream ``` See [Containerlab prerequisites](/infrahub-demo-sp/lab/containerlab.md#host) for sizing. Use the **isp** dataset (2 lab nodes) if the host cannot take the financial one. **Confirming which:** ask a node directly. ``` docker exec -it clab-mpls-backbone-1-pe-01 systemctl is-system-running ``` `starting` long after deploy means EOS is stuck mid-boot (any of the three); `running` means EOS is up and the problem is elsewhere. `systemctl list-jobs` on the node shows which unit is blocking. **Fix:** deploy on a host that meets the [prerequisites](/infrahub-demo-sp/lab/containerlab.md#prerequisites), with a cEOS image of 4.32.0F or newer matching the host architecture. Tear the partial lab down first — `uv run invoke lab.destroy` — so the retry starts clean. *** ## Port conflict: Infrahub already running[​](#port-conflict-infrahub-already-running "Direct link to Port conflict: Infrahub already running") **Symptom:** ``` Error response from daemon: driver failed programming external connectivity ... Bind for 0.0.0.0:8000 failed: port is already allocated ``` **Cause:** Another process (or a leftover container) is using port 8000, 4200 (Prefect), or 8501 (Streamlit). **Fix:** ``` # Find what is using the port: lsof -i :8000 # Stop leftover containers from this project: docker compose -p sp-demo down # If another unrelated project is running on the same port, # override the port in .env: echo "INFRAHUB_PORT=8001" >> .env # Then update INFRAHUB_ADDRESS accordingly: echo 'INFRAHUB_ADDRESS="http://localhost:8001"' >> .env uv run invoke start ``` *** ## Containerlab: image pull failure[​](#containerlab-image-pull-failure "Direct link to Containerlab: image pull failure") ### SR Linux (`ghcr.io/nokia/srlinux`)[​](#sr-linux-ghcrionokiasrlinux "Direct link to sr-linux-ghcrionokiasrlinux") ``` Error response from daemon: Head "https://ghcr.io/...": unauthorized ``` SR Linux is on GitHub Container Registry and requires a GitHub token: ``` docker login ghcr.io -u -p ``` Create a PAT at [GitHub settings](https://github.com/settings/tokens) with `read:packages` scope. ### Arista cEOS[​](#arista-ceos "Direct link to Arista cEOS") The OpsMill mirror (`registry.opsmill.io/external/ceos-image`) is public — no Arista account and no manual import. Do not read the registry's architecture metadata as fact. `docker import` stamps the *importing* host's architecture into the image configuration, so a tag imported on an ARM machine claims `arm64` while carrying the same 32-bit x86 EOS as every other tag. Docker then treats it as native, arranges no emulation, warns about nothing, and the node fails to boot. cEOS-lab has no ARM build; `invoke lab.deploy` refuses ARM hosts for that reason. ``` uname -m # aarch64/arm64 cannot run cEOS-lab echo "${CEOS_IMAGE:-}" # what you asked for, if anything ``` To run your own Arista-supplied build, import it and point `CEOS_IMAGE` at it: ``` docker import cEOS-lab-4.36.0F.tar.xz ceos:local CEOS_IMAGE=ceos:local uv run invoke lab.deploy ``` *** ## Streamlit catalog: "Infrahub not reachable"[​](#streamlit-catalog-infrahub-not-reachable "Direct link to Streamlit catalog: \"Infrahub not reachable\"") **Symptom:** The Streamlit app loads but shows "Cannot connect to Infrahub". **Cause:** `INFRAHUB_ADDRESS` is not set or points to the wrong host/port. **Fix:** ``` # Verify the env var is set: grep INFRAHUB_ADDRESS .env # If running Streamlit outside Docker Compose, ensure the address # points to the correct host: echo 'INFRAHUB_ADDRESS="http://localhost:8000"' >> .env uv run streamlit run service_catalog/app.py ``` If you are running the Streamlit app inside Docker Compose, use the internal service name: ``` INFRAHUB_ADDRESS=http://infrahub-server:8000 ``` *** ## Schema load order error[​](#schema-load-order-error "Direct link to Schema load order error") **Symptom:** ``` SchemaNotFound: Node 'RoutingProtocol' not found ``` **Cause:** The SP schemas (`schemas/sp/`) reference base and extension nodes that were not loaded first. **Fix:** Always load schemas in order: ``` infrahubctl schema load schemas/base/ infrahubctl schema load schemas/extensions/ infrahubctl schema load schemas/sp/ ``` `invoke bootstrap` does this automatically. If you loaded schemas manually in the wrong order, run `invoke destroy && invoke init` to start clean. *** ## Proposed Change: check fails after generator runs[​](#proposed-change-check-fails-after-generator-runs "Direct link to Proposed Change: check fails after generator runs") **Symptom:** The `pe_interface_alloc` check fails even though the generator ran and set the interface. **Cause:** The check reads the current branch state. If the generator ran on a previous branch and the change was already merged, the interface status may already be `cust` on `main`, making it appear unavailable for a new site. **Fix:** Use a different (free) interface for the new site, or set an existing `cust`-status interface back to `free` if it was decommissioned. *** ## Proposed Change: schema integrity failure blocks the merge[​](#proposed-change-schema-integrity-failure-blocks-the-merge "Direct link to Proposed Change: schema integrity failure blocks the merge") **Symptom:** Every check the demo owns passes, every artifact validator passes, and the diff is correct — but the proposed change carries a failed `Schema Integrity` validator and merging is refused: ``` Unable to merge proposed change containing failing checks ``` Its checks report conflicts on schema paths nothing in the change touched, most often: ``` schema/IpamIPAddress/uniqueness_constraints schema/RoutingBGPSession/local_ip/cardinality schema/RoutingBGPSession/remote_ip/min_count ``` **Cause:** Not a data or schema problem in this repository. Compare the schema on the two branches and they are identical: ``` curl -s -H "X-INFRAHUB-KEY: $INFRAHUB_API_TOKEN" \ "$INFRAHUB_ADDRESS/api/schema?branch=main" > /tmp/main.json curl -s -H "X-INFRAHUB-KEY: $INFRAHUB_API_TOKEN" \ "$INFRAHUB_ADDRESS/api/schema?branch=" > /tmp/branch.json diff <(jq -S '.nodes[] | select(.name=="BGPSession")' /tmp/main.json) \ <(jq -S '.nodes[] | select(.name=="BGPSession")' /tmp/branch.json) ``` The conflicts also carry `"branch": "placeholder"` and `"id": "unknown"` rather than real references. Seen on Infrahub 1.10.6. **Behaviour:** intermittent, and sticky to the branch once it happens. Closing the proposed change and opening a new one against the *same* branch reproduces it exactly; a service created on a fresh branch is unaffected. Most branches never see it — an empty branch does not raise the validator at all. **Fix:** Recreate the service on a new branch. The catalog does that for you: submit the wizard again and delete the stuck branch afterwards. There is no way to clear the failed validator on the branch that has it, and no configuration in this repository influences whether it appears. *** ## yamllint: line too long[​](#yamllint-line-too-long "Direct link to yamllint: line too long") **Symptom:** ``` [error] line too long (105 > 100 characters) ``` **Cause:** Bootstrap YAML files sometimes have long inline object specs. **Fix:** Break the long line or raise the limit in `.yamllint.yml`: ``` rules: line-length: max: 120 level: warning # demote from error to warning if needed ``` *** ## uv sync fails[​](#uv-sync-fails "Direct link to uv sync fails") **Symptom:** `uv sync` fails with a resolver error or `requires-python` mismatch. **Fix:** ``` # Verify your Python version: python --version # must be 3.10, 3.11, or 3.12 # If using pyenv: pyenv install 3.12 pyenv local 3.12 uv sync ``` --- # Batfish validation `BatfishBackboneCheck` runs the [Batfish](https://www.batfish.org/) query battery against the rendered MPLS backbone configs and reports findings as Infrahub log entries. It runs automatically on every proposed change touching the backbone, and can also be invoked ad-hoc from the CLI or the Streamlit service catalog. ## What it checks[​](#what-it-checks "Direct link to What it checks") For each MPLS backbone, the check: 1. Fetches the rendered configuration artifact for every PE whose platform is in `SUPPORTED_PLATFORMS` (`arista_eos`, `cisco_iosxr`, `juniper_junos`). Nokia SR OS and SR Linux are not parsed by Batfish today — they're reported as INFO so the gap is explicit, not invisible. On the default **financial** backbone every PE is `arista_eos`, so all 8 are parsed; the **isp** dataset also contributes Cisco and Juniper PEs (and one Nokia PE that surfaces as INFO). 2. Loads the configs into a temporary Batfish snapshot. 3. Runs five Batfish questions and maps each answer row to a `Finding`: | Query | What it flags | Severity | | ------------------------- | ---------------------------------------------------------------------------------------------------- | -------- | | `fileParseStatus` | Files Batfish couldn't model at all (`PARSE_FAIL`, `ORPHANED`, `WILL_NOT_COMMIT`) | ERROR | | `fileParseStatus` | Files partially parsed (`PARTIALLY_UNRECOGNIZED` — Batfish grammar gaps) | WARNING | | `parseWarning` | Per-line "Batfish doesn't model this construct" notes | WARNING | | `undefinedReferences` | Configuration references a structure (route-map, ACL, peer-group) that isn't defined anywhere | ERROR | | `bgpSessionCompatibility` | Configured BGP session doesn't pair cleanly with the other end (mismatched AS, missing remote, etc.) | WARNING | | `isisEdges` | Expected IS-IS adjacency between two PEs isn't present | WARNING | Only ERROR findings block the merge. WARNING and INFO surface in the proposed-change log so reviewers see them without preventing the merge. ## Run from the CLI[​](#run-from-the-cli "Direct link to Run from the CLI") ``` uv run invoke batfish # or against a specific backbone: uv run invoke batfish --backbone mpls-backbone-1 ``` Requires: * Infrahub running (`uv run invoke start`) * The `batfish` sidecar running (started automatically by `invoke start`) * At least one rendered `pe-*` artifact (`invoke bootstrap` if you've never bootstrapped, or `invoke init` for a clean rebuild) The CLI exit code is 0 when the check passes — i.e. zero ERROR findings. WARNING and INFO findings still print but don't fail the run. ## Run from the Service Catalog[​](#run-from-the-service-catalog "Direct link to Run from the Service Catalog") The Streamlit catalog (port 8501) has a **Validation → Batfish Check** page. It runs the same query battery as the CLI, but with stage-by-stage progress: 1. *Fetching backbone topology …* 2. *Fetching N rendered configuration(s) …* (one tick per PE) 3. *Waiting for Batfish coordinator at batfish:9996 …* 4. *Initializing snapshot …* 5. *Running query: Parse status / Parse warnings / Undefined references / BGP / IS-IS edges* Findings then render in three tabs (🚫 Errors / ⚠️ Warnings / ℹ️ Info) with per-query expanders inside each tab. The Branch + Backbone selectors are populated from Infrahub so typos can't silently no-op the check. ## Run as a proposed-change check[​](#run-as-a-proposed-change-check "Direct link to Run as a proposed-change check") The check is registered in `.infrahub.yml` against the `topologies_mpls` target. Every proposed change touching an MPLS backbone runs it automatically; the result appears in the proposed change's **Checks** tab in the Infrahub UI. ## The Batfish sidecar[​](#the-batfish-sidecar "Direct link to The Batfish sidecar") `docker-compose.override.yml` adds a `batfish/allinone:latest` container. It exposes the coordinator on port 9996 (`/v2/networks`) and is reachable from any other container on the `sp-demo` network as `batfish:9996`. No host port mapping is set up by default — the sidecar talks only inside the compose network. Set `BATFISH_DISABLED=1` to skip the check entirely (for example, for CI environments that don't want to start the sidecar): ``` BATFISH_DISABLED=1 uv run pytest ``` ## Known limitations[​](#known-limitations "Direct link to Known limitations") * **No SR Linux / Nokia SR OS support.** Batfish doesn't parse either grammar, so those PEs are filtered out and surfaced as INFO findings. * **Partial parses are common.** Batfish's per-vendor grammars lag real vendor syntax — features like `mpls ip`, `vrf-import` policies, and `route-target import vpn-ipv4` show up as parseWarnings even though the configs are valid on the target platform. That's why parseWarning is mapped to WARNING (informational) rather than ERROR. * **IS-IS adjacencies are inferred from configuration, not observed.** Batfish compares advertised IS-IS interface tuples to predict who *should* peer; it doesn't see live protocol state. A `isisEdges` finding may mean "configuration drift between PEs" or "this PE isn't fully meshed with the others yet." --- # Demo guide This guide walks through the AI/DC solution from start to finish. You will load a data center design into Infrahub, trigger Generators that build the infrastructure from that design, and inspect the results. No code changes are required. Prerequisites Python 3.11+, Docker, and [uv](https://docs.astral.sh/uv/) are required. Refer to [Installation & Setup](/infrahub-solution-ai-dc/solution-ai-dc/installation-setup.md) for detailed environment setup instructions. ## What you are about to see[​](#what-you-are-about-to-see "Direct link to What you are about to see") ### The problem[​](#the-problem "Direct link to The problem") AI data center fabrics involve hundreds of standardized switches, thousands of interfaces, and hierarchical IP addressing schemes that must be consistent across every pod and rack. Traditional scripting can automate the initial build, but it discards design intent after execution — the script knows what to create, but that knowledge lives in code, not in the data layer. Day-two changes (adding a rack, expanding a pod) require re-running or patching scripts with no guarantee that existing infrastructure remains untouched. ### Design-driven automation[​](#design-driven-automation "Direct link to Design-driven automation") Infrahub inverts this approach. Operators define *what* the infrastructure should look like — topology, device counts, IP ranges — as structured design objects stored in Infrahub. Generators read those design objects and produce the implementation: devices, IP allocations, cabling plans, and configuration data. Because the design stays in Infrahub as the source of truth, day-two changes are surgical — a Generator rebuilds only its target scope, leaving everything else intact. For more on how Generators work, see the [Infrahub documentation on Generators](https://docs.infrahub.app/topics/generator). ### The fabric topology[​](#the-fabric-topology "Direct link to The fabric topology") Fabric-A is a 5-stage Clos network with the following hierarchy: * **Fabric-A**: 6 super spine switches * **Pod-A1** (role: `fabric`): represents the super spine tier — the FabricGenerator places super spine devices here * **Pod-A2** and **Pod-A3**: each with 4 spine switches * **4 racks under Pod-A2**: Rack-A2-1 through Rack-A2-4, each containing 1–2 leaf switches * **4 racks under Pod-A3**: Rack-A3-1 through Rack-A3-4, each containing 1–2 leaf switches ![Fabric-A topology diagram](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAxMDYwIDUyMCIgZm9udC1mYW1pbHk9IkludGVyLCAtYXBwbGUtc3lzdGVtLCBCbGlua01hY1N5c3RlbUZvbnQsICdTZWdvZSBVSScsIFJvYm90bywgc2Fucy1zZXJpZiI+CiAgPGRlZnM+CiAgICA8ZmlsdGVyIGlkPSJzaGFkb3ciIHg9Ii0yJSIgeT0iLTIlIiB3aWR0aD0iMTA0JSIgaGVpZ2h0PSIxMDQlIj4KICAgICAgPGZlRHJvcFNoYWRvdyBkeD0iMCIgZHk9IjEiIHN0ZERldmlhdGlvbj0iMiIgZmxvb2Qtb3BhY2l0eT0iMC4xIi8+CiAgICA8L2ZpbHRlcj4KICA8L2RlZnM+CgogIDwhLS0gRmFicmljLUEgY29udGFpbmVyIC0tPgogIDxyZWN0IHg9IjEwIiB5PSIxMCIgd2lkdGg9IjEwNDAiIGhlaWdodD0iNTAwIiByeD0iMTIiIGZpbGw9IiNmNWY1ZjUiIHN0cm9rZT0iI2QwZDBkMCIgc3Ryb2tlLXdpZHRoPSIxLjUiLz4KICA8dGV4dCB4PSIzMCIgeT0iMzgiIGZvbnQtc2l6ZT0iMTYiIGZvbnQtd2VpZ2h0PSI3MDAiIGZpbGw9IiMzMzMiPkZhYnJpYy1BPC90ZXh0PgoKICA8IS0tIFN1cGVyIFNwaW5lcyAtLT4KICA8cmVjdCB4PSIzNTAiIHk9IjU1IiB3aWR0aD0iMzYwIiBoZWlnaHQ9IjU1IiByeD0iOCIgZmlsbD0iIzRhOTBkOSIgZmlsdGVyPSJ1cmwoI3NoYWRvdykiLz4KICA8dGV4dCB4PSI1MzAiIHk9Ijc4IiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmb250LXNpemU9IjE1IiBmb250LXdlaWdodD0iNzAwIiBmaWxsPSIjMzMzIj42IFN1cGVyIFNwaW5lczwvdGV4dD4KICA8dGV4dCB4PSI1MzAiIHk9Ijk3IiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmb250LXNpemU9IjEyIiBmaWxsPSIjMzMzIj5zcy1mYWJyaWMtYS0xIOKApiBzcy1mYWJyaWMtYS02PC90ZXh0PgoKICA8IS0tIEZ1bGwgbWVzaCBsaW5lcyAtLT4KICA8bGluZSB4MT0iNDMwIiB5MT0iMTEwIiB4Mj0iMjcwIiB5Mj0iMTc1IiBzdHJva2U9IiMzMzMiIHN0cm9rZS13aWR0aD0iMS41IiBzdHJva2UtZGFzaGFycmF5PSI1LDMiLz4KICA8dGV4dCB4PSIzMzAiIHk9IjE0MiIgZm9udC1zaXplPSIxMSIgZmlsbD0iIzMzMyIgdHJhbnNmb3JtPSJyb3RhdGUoLTIwIDMzMCAxNDIpIj5mdWxsIG1lc2g8L3RleHQ+CiAgPGxpbmUgeDE9IjYzMCIgeTE9IjExMCIgeDI9Ijc5MCIgeTI9IjE3NSIgc3Ryb2tlPSIjMzMzIiBzdHJva2Utd2lkdGg9IjEuNSIgc3Ryb2tlLWRhc2hhcnJheT0iNSwzIi8+CiAgPHRleHQgeD0iNzE1IiB5PSIxNDIiIGZvbnQtc2l6ZT0iMTEiIGZpbGw9IiMzMzMiIHRyYW5zZm9ybT0icm90YXRlKDIwIDcxNSAxNDIpIj5mdWxsIG1lc2g8L3RleHQ+CgogIDwhLS0gUG9kLUEyIGNvbnRhaW5lcjogeD0yMCwgd2lkdGg9NTA1IChoYWxmIG9mIDEwNDAtMjAtMTAgbWludXMgNXB4IGdhcCBlYWNoIHNpZGUpIC0tPgogIDxyZWN0IHg9IjIwIiB5PSIxNjUiIHdpZHRoPSI1MDUiIGhlaWdodD0iMjMwIiByeD0iMTAiIGZpbGw9IiNmZmYiIHN0cm9rZT0iI2RkZCIgc3Ryb2tlLXdpZHRoPSIxLjUiLz4KICA8dGV4dCB4PSI0MCIgeT0iMTkwIiBmb250LXNpemU9IjE0IiBmb250LXdlaWdodD0iNzAwIiBmaWxsPSIjMzMzIj5Qb2QtQTI8L3RleHQ+CiAgPHRleHQgeD0iOTgiIHk9IjE5MCIgZm9udC1zaXplPSIxMyIgZmlsbD0iIzY2NiI+wrcgcm9sZTogY3B1PC90ZXh0PgoKICA8IS0tIFBvZC1BMiBTcGluZXMgLSBjZW50ZXJlZCBhdCB4PTI3MiAtLT4KICA8cmVjdCB4PSIxNDIiIHk9IjIwNSIgd2lkdGg9IjI2MCIgaGVpZ2h0PSI1MCIgcng9IjgiIGZpbGw9IiNmNWE2MjMiIGZpbHRlcj0idXJsKCNzaGFkb3cpIi8+CiAgPHRleHQgeD0iMjcyIiB5PSIyMjYiIHRleHQtYW5jaG9yPSJtaWRkbGUiIGZvbnQtc2l6ZT0iMTQiIGZvbnQtd2VpZ2h0PSI3MDAiIGZpbGw9IiMzMzMiPjQgU3BpbmVzPC90ZXh0PgogIDx0ZXh0IHg9IjI3MiIgeT0iMjQ0IiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmb250LXNpemU9IjExIiBmaWxsPSIjMzMzIj5zcGluZS1wb2QtYTItMSDigKYgNDwvdGV4dD4KCiAgPCEtLSBQb2QtQTIgUmFja3MgLSA0IGluIGEgcm93LCAxMTBweCBlYWNoLCAxMHB4IGdhcHMsIGNlbnRlcmVkIGluIDUwNXB4IHBvZAogICAgICAgdG90YWwgcmFjayB3aWR0aCA9IDQqMTEwICsgMyoxMCA9IDQ3MCwgbGVmdCBtYXJnaW4gPSAoNTA1LTQ3MCkvMiA9IDE3LjUsIHN0YXJ0IGF0IDIwKzE3LjU9MzcgLS0+CiAgPHJlY3QgeD0iMzciIHk9IjI5MCIgd2lkdGg9IjExMCIgaGVpZ2h0PSI1MCIgcng9IjYiIGZpbGw9IiM3ZWQzMjEiIGZpbHRlcj0idXJsKCNzaGFkb3cpIi8+CiAgPHRleHQgeD0iOTIiIHk9IjMxMCIgdGV4dC1hbmNob3I9Im1pZGRsZSIgZm9udC1zaXplPSIxMiIgZm9udC13ZWlnaHQ9IjYwMCIgZmlsbD0iIzMzMyI+UmFjay1BMi0xPC90ZXh0PgogIDx0ZXh0IHg9IjkyIiB5PSIzMjYiIHRleHQtYW5jaG9yPSJtaWRkbGUiIGZvbnQtc2l6ZT0iMTAiIGZpbGw9IiMzMzMiPmNvbXB1dGUgwrcgMiBsZWFmczwvdGV4dD4KCiAgPHJlY3QgeD0iMTU3IiB5PSIyOTAiIHdpZHRoPSIxMTAiIGhlaWdodD0iNTAiIHJ4PSI2IiBmaWxsPSIjN2VkMzIxIiBmaWx0ZXI9InVybCgjc2hhZG93KSIvPgogIDx0ZXh0IHg9IjIxMiIgeT0iMzEwIiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmb250LXNpemU9IjEyIiBmb250LXdlaWdodD0iNjAwIiBmaWxsPSIjMzMzIj5SYWNrLUEyLTI8L3RleHQ+CiAgPHRleHQgeD0iMjEyIiB5PSIzMjYiIHRleHQtYW5jaG9yPSJtaWRkbGUiIGZvbnQtc2l6ZT0iMTAiIGZpbGw9IiMzMzMiPmNvbXB1dGUgwrcgMSBsZWFmPC90ZXh0PgoKICA8cmVjdCB4PSIyNzciIHk9IjI5MCIgd2lkdGg9IjExMCIgaGVpZ2h0PSI1MCIgcng9IjYiIGZpbGw9IiM3ZWQzMjEiIGZpbHRlcj0idXJsKCNzaGFkb3cpIi8+CiAgPHRleHQgeD0iMzMyIiB5PSIzMTAiIHRleHQtYW5jaG9yPSJtaWRkbGUiIGZvbnQtc2l6ZT0iMTIiIGZvbnQtd2VpZ2h0PSI2MDAiIGZpbGw9IiMzMzMiPlJhY2stQTItMzwvdGV4dD4KICA8dGV4dCB4PSIzMzIiIHk9IjMyNiIgdGV4dC1hbmNob3I9Im1pZGRsZSIgZm9udC1zaXplPSIxMCIgZmlsbD0iIzMzMyI+Y29tcHV0ZSDCtyAyIGxlYWZzPC90ZXh0PgoKICA8cmVjdCB4PSIzOTciIHk9IjI5MCIgd2lkdGg9IjExMCIgaGVpZ2h0PSI1MCIgcng9IjYiIGZpbGw9IiM3ZWQzMjEiIGZpbHRlcj0idXJsKCNzaGFkb3cpIi8+CiAgPHRleHQgeD0iNDUyIiB5PSIzMTAiIHRleHQtYW5jaG9yPSJtaWRkbGUiIGZvbnQtc2l6ZT0iMTIiIGZvbnQtd2VpZ2h0PSI2MDAiIGZpbGw9IiMzMzMiPlJhY2stQTItNDwvdGV4dD4KICA8dGV4dCB4PSI0NTIiIHk9IjMyNiIgdGV4dC1hbmNob3I9Im1pZGRsZSIgZm9udC1zaXplPSIxMCIgZmlsbD0iIzMzMyI+c3RvcmFnZSDCtyAxIGxlYWY8L3RleHQ+CgogIDwhLS0gUG9kLUEyIHNwaW5lLXRvLXJhY2sgbGluZXMgLS0+CiAgPGxpbmUgeDE9IjIwMiIgeTE9IjI1NSIgeDI9IjkyIiB5Mj0iMjkwIiBzdHJva2U9IiMzMzMiIHN0cm9rZS13aWR0aD0iMS4yIi8+CiAgPGxpbmUgeDE9IjI0MiIgeTE9IjI1NSIgeDI9IjIxMiIgeTI9IjI5MCIgc3Ryb2tlPSIjMzMzIiBzdHJva2Utd2lkdGg9IjEuMiIvPgogIDxsaW5lIHgxPSIzMDIiIHkxPSIyNTUiIHgyPSIzMzIiIHkyPSIyOTAiIHN0cm9rZT0iIzMzMyIgc3Ryb2tlLXdpZHRoPSIxLjIiLz4KICA8bGluZSB4MT0iMzQyIiB5MT0iMjU1IiB4Mj0iNDUyIiB5Mj0iMjkwIiBzdHJva2U9IiMzMzMiIHN0cm9rZS13aWR0aD0iMS4yIi8+CgogIDwhLS0gUG9kLUEzIGNvbnRhaW5lcjogeD01MzUsIHdpZHRoPTUwNSAtLT4KICA8cmVjdCB4PSI1MzUiIHk9IjE2NSIgd2lkdGg9IjUwNSIgaGVpZ2h0PSIyMzAiIHJ4PSIxMCIgZmlsbD0iI2ZmZiIgc3Ryb2tlPSIjZGRkIiBzdHJva2Utd2lkdGg9IjEuNSIvPgogIDx0ZXh0IHg9IjU1NSIgeT0iMTkwIiBmb250LXNpemU9IjE0IiBmb250LXdlaWdodD0iNzAwIiBmaWxsPSIjMzMzIj5Qb2QtQTM8L3RleHQ+CiAgPHRleHQgeD0iNjEzIiB5PSIxOTAiIGZvbnQtc2l6ZT0iMTMiIGZpbGw9IiM2NjYiPsK3IHJvbGU6IGNwdTwvdGV4dD4KCiAgPCEtLSBQb2QtQTMgU3BpbmVzIC0gY2VudGVyZWQgYXQgeD03ODcgLS0+CiAgPHJlY3QgeD0iNjU3IiB5PSIyMDUiIHdpZHRoPSIyNjAiIGhlaWdodD0iNTAiIHJ4PSI4IiBmaWxsPSIjZjVhNjIzIiBmaWx0ZXI9InVybCgjc2hhZG93KSIvPgogIDx0ZXh0IHg9Ijc4NyIgeT0iMjI2IiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmb250LXNpemU9IjE0IiBmb250LXdlaWdodD0iNzAwIiBmaWxsPSIjMzMzIj40IFNwaW5lczwvdGV4dD4KICA8dGV4dCB4PSI3ODciIHk9IjI0NCIgdGV4dC1hbmNob3I9Im1pZGRsZSIgZm9udC1zaXplPSIxMSIgZmlsbD0iIzMzMyI+c3BpbmUtcG9kLWEzLTEg4oCmIDQ8L3RleHQ+CgogIDwhLS0gUG9kLUEzIFJhY2tzIC0gNCBpbiBhIHJvdywgY2VudGVyZWQgaW4gNTA1cHggcG9kCiAgICAgICBzdGFydCBhdCA1MzUrMTcuNT01NTIgLS0+CiAgPHJlY3QgeD0iNTUyIiB5PSIyOTAiIHdpZHRoPSIxMTAiIGhlaWdodD0iNTAiIHJ4PSI2IiBmaWxsPSIjN2VkMzIxIiBmaWx0ZXI9InVybCgjc2hhZG93KSIvPgogIDx0ZXh0IHg9IjYwNyIgeT0iMzEwIiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmb250LXNpemU9IjEyIiBmb250LXdlaWdodD0iNjAwIiBmaWxsPSIjMzMzIj5SYWNrLUEzLTE8L3RleHQ+CiAgPHRleHQgeD0iNjA3IiB5PSIzMjYiIHRleHQtYW5jaG9yPSJtaWRkbGUiIGZvbnQtc2l6ZT0iMTAiIGZpbGw9IiMzMzMiPmNvbXB1dGUgwrcgMiBsZWFmczwvdGV4dD4KCiAgPHJlY3QgeD0iNjcyIiB5PSIyOTAiIHdpZHRoPSIxMTAiIGhlaWdodD0iNTAiIHJ4PSI2IiBmaWxsPSIjN2VkMzIxIiBmaWx0ZXI9InVybCgjc2hhZG93KSIvPgogIDx0ZXh0IHg9IjcyNyIgeT0iMzEwIiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmb250LXNpemU9IjEyIiBmb250LXdlaWdodD0iNjAwIiBmaWxsPSIjMzMzIj5SYWNrLUEzLTI8L3RleHQ+CiAgPHRleHQgeD0iNzI3IiB5PSIzMjYiIHRleHQtYW5jaG9yPSJtaWRkbGUiIGZvbnQtc2l6ZT0iMTAiIGZpbGw9IiMzMzMiPmNvbXB1dGUgwrcgMSBsZWFmPC90ZXh0PgoKICA8cmVjdCB4PSI3OTIiIHk9IjI5MCIgd2lkdGg9IjExMCIgaGVpZ2h0PSI1MCIgcng9IjYiIGZpbGw9IiM3ZWQzMjEiIGZpbHRlcj0idXJsKCNzaGFkb3cpIi8+CiAgPHRleHQgeD0iODQ3IiB5PSIzMTAiIHRleHQtYW5jaG9yPSJtaWRkbGUiIGZvbnQtc2l6ZT0iMTIiIGZvbnQtd2VpZ2h0PSI2MDAiIGZpbGw9IiMzMzMiPlJhY2stQTMtMzwvdGV4dD4KICA8dGV4dCB4PSI4NDciIHk9IjMyNiIgdGV4dC1hbmNob3I9Im1pZGRsZSIgZm9udC1zaXplPSIxMCIgZmlsbD0iIzMzMyI+c3RvcmFnZSDCtyAxIGxlYWY8L3RleHQ+CgogIDxyZWN0IHg9IjkxMiIgeT0iMjkwIiB3aWR0aD0iMTEwIiBoZWlnaHQ9IjUwIiByeD0iNiIgZmlsbD0iIzdlZDMyMSIgZmlsdGVyPSJ1cmwoI3NoYWRvdykiLz4KICA8dGV4dCB4PSI5NjciIHk9IjMxMCIgdGV4dC1hbmNob3I9Im1pZGRsZSIgZm9udC1zaXplPSIxMiIgZm9udC13ZWlnaHQ9IjYwMCIgZmlsbD0iIzMzMyI+UmFjay1BMy00PC90ZXh0PgogIDx0ZXh0IHg9Ijk2NyIgeT0iMzI2IiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmb250LXNpemU9IjEwIiBmaWxsPSIjMzMzIj5jb21wdXRlIMK3IDEgbGVhZjwvdGV4dD4KCiAgPCEtLSBQb2QtQTMgc3BpbmUtdG8tcmFjayBsaW5lcyAtLT4KICA8bGluZSB4MT0iNzE3IiB5MT0iMjU1IiB4Mj0iNjA3IiB5Mj0iMjkwIiBzdHJva2U9IiMzMzMiIHN0cm9rZS13aWR0aD0iMS4yIi8+CiAgPGxpbmUgeDE9Ijc1NyIgeTE9IjI1NSIgeDI9IjcyNyIgeTI9IjI5MCIgc3Ryb2tlPSIjMzMzIiBzdHJva2Utd2lkdGg9IjEuMiIvPgogIDxsaW5lIHgxPSI4MTciIHkxPSIyNTUiIHgyPSI4NDciIHkyPSIyOTAiIHN0cm9rZT0iIzMzMyIgc3Ryb2tlLXdpZHRoPSIxLjIiLz4KICA8bGluZSB4MT0iODU3IiB5MT0iMjU1IiB4Mj0iOTY3IiB5Mj0iMjkwIiBzdHJva2U9IiMzMzMiIHN0cm9rZS13aWR0aD0iMS4yIi8+CgogIDwhLS0gTGVnZW5kIC0tPgogIDxyZWN0IHg9IjUwIiB5PSI0MjAiIHdpZHRoPSIxNiIgaGVpZ2h0PSIxNiIgcng9IjMiIGZpbGw9IiM0YTkwZDkiLz4KICA8dGV4dCB4PSI3NCIgeT0iNDMzIiBmb250LXNpemU9IjEyIiBmaWxsPSIjMzMzIj5TdXBlciBTcGluZTwvdGV4dD4KICA8cmVjdCB4PSIxNzAiIHk9IjQyMCIgd2lkdGg9IjE2IiBoZWlnaHQ9IjE2IiByeD0iMyIgZmlsbD0iI2Y1YTYyMyIvPgogIDx0ZXh0IHg9IjE5NCIgeT0iNDMzIiBmb250LXNpemU9IjEyIiBmaWxsPSIjMzMzIj5TcGluZTwvdGV4dD4KICA8cmVjdCB4PSIyNjAiIHk9IjQyMCIgd2lkdGg9IjE2IiBoZWlnaHQ9IjE2IiByeD0iMyIgZmlsbD0iIzdlZDMyMSIvPgogIDx0ZXh0IHg9IjI4NCIgeT0iNDMzIiBmb250LXNpemU9IjEyIiBmaWxsPSIjMzMzIj5MZWFmIChwZXIgcmFjayk8L3RleHQ+CgogIDx0ZXh0IHg9IjUwIiB5PSI0NjAiIGZvbnQtc2l6ZT0iMTIiIGZpbGw9IiMzMzMiPkVhY2ggc3BpbmUgY29ubmVjdHMgdG8gYWxsIHN1cGVyIHNwaW5lcyDCtyBFYWNoIGxlYWYgY29ubmVjdHMgdG8gYWxsIHNwaW5lcyBpbiBpdHMgcG9kPC90ZXh0PgogIDx0ZXh0IHg9IjUwIiB5PSI0NzgiIGZvbnQtc2l6ZT0iMTIiIGZpbGw9IiMzMzMiPlRvdGFsOiA2IHN1cGVyIHNwaW5lcyArIDggc3BpbmVzICsgMTEgbGVhZnMgPSAyNSBkZXZpY2VzPC90ZXh0Pgo8L3N2Zz4K) ### Minimal inputs, complete infrastructure[​](#minimal-inputs-complete-infrastructure "Direct link to Minimal inputs, complete infrastructure") The contrast between what you define and what Generators produce illustrates the power of design-driven automation. **Design inputs** (loaded by `inv load`): * 4 single-vendor fabrics (Fabric-A Cisco, Fabric-B Arista, Fabric-C Dell, Fabric-D Juniper), 12 pods, 32 racks * Per-vendor device templates (defining interface layouts and roles) * 1 IP supernet (`10.0.0.0/8`) with a `FabricSupernetPool`, plus overlay resource pools (ASN, VNI, VLAN) * A seed overlay tenant (`Blue` on Fabric-A) with a VRF and segments **Generated output** (for Fabric-A alone): * 25 devices: 6 super spines + 8 spines + 11 leafs * Hierarchical IP pools carved from the supernet through fabric, pod, and rack levels * Interface-level cabling with point-to-point /31 addressing between tiers * Computed interface descriptions on every connected interface ## Setting up the demo[​](#setting-up-the-demo "Direct link to Setting up the demo") ### Start the environment[​](#start-the-environment "Direct link to Start the environment") ``` uv sync --all-packages uv run inv start ``` This installs Python dependencies and launches the Infrahub stack via Docker Compose — including the server, task workers, Neo4j, Redis, RabbitMQ, and PostgreSQL. ### Load the design data[​](#load-the-design-data "Direct link to Load the design data") `infrahubctl` needs credentials to authenticate against the Infrahub API — export the default admin credentials before loading: ``` export INFRAHUB_USERNAME="admin" export INFRAHUB_PASSWORD="infrahub" uv run inv load ``` This runs four steps in sequence: loads schema definitions (`infrahubctl schema load`), loads UI menus (`infrahubctl menu load`), loads all design objects from `objects/`, and registers the Git repository from `repository.yml`. After this command completes, Infrahub contains the full design — fabrics, pods, racks, device templates, and IP pools — but no actual devices. What `inv load` does not create No devices exist yet. The fabrics, pods, and racks define design intent — they describe *what* the infrastructure should look like. Browse the NetworkDevice list in the Infrahub UI to confirm it is empty. ### Wait for repository sync[​](#wait-for-repository-sync "Direct link to Wait for repository sync") ``` uv run infrahubctl repository list ``` Verify the repository status shows as synced. When Infrahub imports the repository, it reads `.infrahub.yml` to register Generator definitions, Transforms, queries, and artifact definitions. Generators cannot run until this import completes. ### Load trigger rules[​](#load-trigger-rules "Direct link to Load trigger rules") ``` uv run infrahubctl object load triggers.yml ``` This creates `CoreGeneratorAction` and `CoreNodeTriggerRule` objects that enable automatic execution of the modular Generators — including the tenant/overlay trigger (`run-tenant-generator`). `triggers.yml` is a committed top-level file that is not part of `inv load`; it must be loaded after the repository has synced, because the rules reference Generator definitions that need to exist first. Branch scope Triggers fire only on non-main branches. All trigger rules are configured with `branch_scope: other_branches` by design. Running a Generator on `main` will not trigger downstream Generators. ## Running the Generators[​](#running-the-generators "Direct link to Running the Generators") With trigger rules loaded, the modular Generators run automatically. The FabricGenerator writes checksums to Pods — triggers fire the PodGenerator for each Pod — the PodGenerator writes checksums to Racks — triggers fire the RackGenerator for each Rack. One action at the fabric level builds the entire data center. ### How it works[​](#how-it-works "Direct link to How it works") 1. **FabricGenerator** completes and writes a checksum to each child Pod 2. The `CoreNodeTriggerRule` for `NetworkPod` detects the checksum update and fires the `run-pod-generator` action (`CoreGeneratorAction`) 3. **PodGenerator** runs for each Pod (skipping fabric-role pods), creates spines and links, then writes a checksum to each child Rack 4. The `CoreNodeTriggerRule` for `LocationRack` detects the checksum update and fires the `run-rack-generator` action 5. **RackGenerator** runs for each Rack, creating leafs and links The pattern is: **checksum write → trigger rule → Generator action → next tier**, repeated at each level. A fourth Generator — the **OverlayGenerator** (`generate-tenant`) — sits alongside this physical build. It is triggered by changes to a tenant's design (a `NetworkTenant` checksum update) rather than by the fabric cascade, and it layers EVPN/VXLAN services onto the leaf switches the fabric build produced. See [Overlay services (EVPN/VXLAN)](#overlay-services-evpnvxlan) below. ### Try it[​](#try-it "Direct link to Try it") Create a branch and run the FabricGenerator for Fabric-A: All Generator work happens on branches, not on `main`. This keeps the main branch clean and lets you review changes before merging. In the Infrahub UI, navigate to the Generator Definitions page (accessible from the Actions menu). Find `generate-fabric`, click **Run**, and select **Fabric-A** as the target. Watch the modular Generators in action: PodGenerators fire automatically for Pod-A2 and Pod-A3 (Pod-A1 is skipped — its `fabric` role marks it as the super spine tier), followed by RackGenerators for all 8 racks. After all Generators complete Verify in the UI: 25 devices in the NetworkDevice list (6 super spines, 8 spines, 11 leafs), hierarchical IP pools carved from the supernet, NetworkLink objects with point-to-point /31 addressing, and computed interface descriptions on every connected interface. Monitoring progress You can track progress through the Infrahub UI task execution list. Each Generator execution appears as a separate task, so you can follow the execution as it propagates through the tiers. ## Reviewing changes and generating artifacts[​](#reviewing-changes-and-generating-artifacts "Direct link to Reviewing changes and generating artifacts") With the Generators finished, create a proposed change to review what was built and trigger artifact generation. 1. In the Infrahub menu, navigate to **Proposed Change** and click **New proposed change**. 2. Select the branch that we previously created, give it a title (for example, *Build Fabric-A*) and click *Open*. The proposed change shows a diff of every object the Generators created — devices, IP allocations, links, and pools. This is the same review workflow used for any change in Infrahub. Creating the proposed change also triggers Infrahub's CI pipeline, which runs the Transforms and artifact definitions registered from the repository. Once the pipeline completes, the following artifacts are available: * **Startup configuration** — a text artifact per device, rendered from that device's vendor template. Fabric-A is Cisco, so its devices render from the Cisco template; Fabric-B (Arista), Fabric-C (Dell) and Fabric-D (Juniper) render from theirs. See [Multivendor configuration](/infrahub-solution-ai-dc/solution-ai-dc/multivendor-config.md). * **Cabling plan** — a CSV artifact per fabric, produced by a Python Transformation Browse the generated artifacts from the proposed change view or from individual device and fabric objects. Computed interface descriptions Computed interface descriptions are not generated by CI — they are computed attributes that update automatically whenever an interface's relationships change. They are already visible on interfaces after the Generators run. On the proposed change overview tab, click the **Merge** button to merge the changes into the main branch. We have now generated a full fabric from our design intend, and the startup configuration artifacts for each device can now be deployed, using a configuration deployment tool such as Ansible or Nornir. ## Day-two operations: adding a rack[​](#day-two-operations-adding-a-rack "Direct link to Day-two operations: adding a rack") Design-driven automation extends beyond the initial build. Adding a rack demonstrates how Generators handle incremental changes without rebuilding existing infrastructure. 1. Create a new branch 2. In the new branch, create a new **LocationRack** in the Infrahub UI: * **Name**: `Rack-A2-5` * **Index**: `5` * **Rack type**: `compute` * **Pod**: `Pod-A2` * **Parent**: `Hall-A1` * **Amount of leafs**: `2` * **Leaf switch template**: `Object template Device` >> `leaf-switch-compute` * **Member Of Groups**: `racks` 3. Run `generate-rack` manually for **Rack-A2-5** 4. Observe the results: 2 new leaf devices (`leaf-pod-a2-5-1` and `leaf-pod-a2-5-2`), new NetworkLink objects connecting them to the existing spines — and all existing devices, links, and IP addresses in other racks remain unchanged. Surgical updates The RackGenerator creates objects only for its target rack. Existing devices, links, and IPs in other racks are not modified. Triggers fire only on updates to existing objects, not on new object creation — which is why the RackGenerator is run manually here. 4. Open a new proposed change for the newly created branch The proposed change shows a diff of every object the Generator created — devices, IP allocations, links. Creating the proposed change also triggers Infrahub's CI pipeline, which runs the Transforms and artifact definitions registered from the repository. Once the pipeline completes, the following artifacts are available: * **Startup configuration** — new startup configurations for the 2 rack leaf switches, updated startup configurations for the pod spine switches * **Cabling plan** — an updated cabling plan for the whole fabric Browse the generated artifacts from the proposed change view or from individual device and fabric objects. On the proposed change overview tab, click the **Merge** button to merge the changes into the main branch. ## Overlay services (EVPN/VXLAN)[​](#overlay-services-evpnvxlan "Direct link to Overlay services (EVPN/VXLAN)") On top of the physical fabric, the solution layers multi-tenant EVPN/VXLAN overlay services. `inv load` already seeded a tenant — **Blue** on Fabric-A — with a VRF (`blue-prod`) and several segments (routed, rack-scoped, and L2-only examples). Nothing about the underlay or the fabric build changes; the overlay rides on top of it. 1. Create a new branch. 2. Run `generate-tenant` manually for the **Blue** tenant (Generator Definitions → `generate-tenant` → **Run** → target **Blue**). The OverlayGenerator allocates every overlay identifier (an L2VNI and VLAN per segment, an L3VNI and transit VLAN per VRF, route targets derived from the fabric ASN), allocates a subnet and `.1` anycast gateway for each routed segment, and materializes which leaf switches carry which segments. 3. Open a proposed change for the branch, let CI render the artifacts, then inspect a **leaf** device's startup configuration. Confirm it now contains the iBGP L2VPN-EVPN neighbors, the `nve1` VXLAN interface, per-segment VLAN/VNI mappings, the VRF context, and an anycast-gateway SVI for each routed segment. 4. Inspect a **spine** or **super-spine** artifact and confirm it carries the EVPN control plane and route-reflector role but **no** NVE, VLAN, VRF, or SVI state — only leaves are VTEPs. Scoped, queryable overlay Every allocated identifier and every iBGP session is stored as queryable data in Infrahub (browse **Network → Overlay** for Tenants, VRFs, Segments, and BGP Sessions). Adding, changing, or removing a tenant/VRF/segment regenerates configuration for only the affected leaves. To see a scoped day-two multi-tenant change, load the second tenant and run its Generator: ``` # data/ is not part of `inv load` — load the Red tenant manually uv run infrahubctl object load data/tenant-red.yml ``` Then run `generate-tenant` for **Red**: only the leaves carrying Red's segments change; Blue's configuration and every unrelated device are untouched. For the full model, allocation rules, and rendered-configuration contract, see [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md). ## What to take away[​](#what-to-take-away "Direct link to What to take away") ### Key concepts demonstrated[​](#key-concepts-demonstrated "Direct link to Key concepts demonstrated") * **Design-driven automation** — you defined topology, device counts, and IP ranges; Generators produced 25 devices with full connectivity and addressing * **Modular Generators** — a single fabric-level action built the entire data center through checksum-triggered execution across layers * **Hierarchical IP delegation** — the supernet was carved automatically through fabric, pod, and rack levels via [Resource Manager](https://docs.infrahub.app/topics/resource-manager) * **Proposed change review** — a diff of every generated object, with CI-triggered artifact generation (startup configs, cabling plans) * **Day-two change support** — adding a rack produced only new objects with no rebuild of existing infrastructure * **Multi-tenant overlay** — declaring a tenant's intent produced a full EVPN/VXLAN control and data plane, with all identifiers auto-allocated and leaf-only state * **Multivendor rendering** — the same design produced Cisco, Arista, Dell, and Juniper Junos configuration, each device routed to its vendor's template * **Branch-based workflow** — all changes happened on a branch, reviewable and mergeable through Infrahub's standard workflow ### Next steps[​](#next-steps "Direct link to Next steps") * [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md) — the multi-tenant overlay model, allocation, and rendered configuration * [Multivendor configuration](/infrahub-solution-ai-dc/solution-ai-dc/multivendor-config.md) — how per-vendor configuration is resolved and rendered * [Design-driven automation](/infrahub-solution-ai-dc/solution-ai-dc/design-driven-automation.md) — deeper explanation of the architecture and concepts * [Modular Generator architecture](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md) — how the modular Generators work internally * [Generator patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md) — implementation patterns for building your own Generators * [Infrahub documentation: Generators](https://docs.infrahub.app/topics/generator) * [Infrahub documentation: Resource Manager](https://docs.infrahub.app/topics/resource-manager) Beyond AI data centers The modular Generator pattern demonstrated here — hierarchical Generators connected by checksum triggers — applies to any standardized, layered infrastructure: edge deployments, service provider networks, enterprise campuses. The Fabric → Pod → Rack hierarchy is illustrative, not prescriptive. --- # Design-driven automation Operators describe what infrastructure should look like; Generators produce it. This page explains how that separation works in the AI/DC solution, what the design objects look like, and why it matters for ongoing operations. See the [Demo Guide](/infrahub-solution-ai-dc/solution-ai-dc/demo-guide.md) for hands-on execution and [Modular Generator architecture](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md) for how the modular Generators connect. ## What design-driven automation means[​](#what-design-driven-automation-means "Direct link to What design-driven automation means") ### Intent over procedure[​](#intent-over-procedure "Direct link to Intent over procedure") Traditional automation encodes *how*: create device X, assign IP Y, cable port Z. The design exists only inside the script — it is consumed during execution and discarded. To understand what was intended, you read code. Design-driven automation encodes *what*: "this fabric has 6 super spines." The *how* is the Generator's job. The design persists as structured data in Infrahub — queryable, versionable, and auditable — independent of whether Generators have run. An operator can inspect the intended state of a rack at any time without reading Generator code. ### Schema as design language[​](#schema-as-design-language "Direct link to Schema as design language") The solution's schema makes design intent machine-readable. Each schema node type represents a level in the data center hierarchy: * `NetworkFabric` — the top level, defining super spine counts and templates * `NetworkPod` — the middle level, defining spine counts, roles, and templates * `LocationRack` — the bottom level, defining rack type, leaf counts, and templates Relationships encode topology: a Fabric contains Pods, Pods reference Racks. Device templates (`CoreObjectTemplate`) capture interface layouts per device role — Generators use these to stamp out devices with the correct ports and interface profiles. Overlay services add a parallel intent hierarchy on top of the physical one — `NetworkTenant` → `NetworkVrf` → `NetworkSegment` — where operators declare *what* tenant services should exist and the OverlayGenerator produces the implementation. See [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md). Schema reference The physical design hierarchy is defined in `schemas/logical_design.yml` (Fabric and Pod) and `schemas/physical_location.yml` (Rack); the overlay intent model is in `schemas/overlay.yml`. See [Installation & Setup: Schemas](/infrahub-solution-ai-dc/solution-ai-dc/installation-setup.md#schemas) for the full schema file list. ### Generators read design, produce infrastructure[​](#generators-read-design-produce-infrastructure "Direct link to Generators read design, produce infrastructure") A Generator reads design objects via a GraphQL query and creates implementation objects — devices, IP allocations, links. Each Generator owns one hierarchy layer: * **FabricGenerator** reads `NetworkFabric` → produces super spine devices, fabric IP pools * **PodGenerator** reads `NetworkPod` → produces spine devices, spine-to-super-spine links * **RackGenerator** reads `LocationRack` → produces leaf devices, leaf-to-spine links * **OverlayGenerator** reads `NetworkTenant` → allocates overlay identifiers and places tenant segments on leaf switches The same principle extends beyond the physical fabric: the overlay describes tenant *intent* (a tenant, its VRFs, its segments) and the OverlayGenerator produces the EVPN/VXLAN implementation. See [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md). Generators are idempotent: running one again produces the same result, creating only what is missing. If the design changes, a Generator rebuilds only its scope — other layers remain untouched. Generators also connect through triggers: the output of one tier triggers the next, building a full data center from a single action. See [Modular Generator architecture](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md) for how the modular Generators work and [Generator patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md) for implementation details. ## The design in this solution[​](#the-design-in-this-solution "Direct link to The design in this solution") ### The Fabric-Pod-Rack hierarchy[​](#the-fabric-pod-rack-hierarchy "Direct link to The Fabric-Pod-Rack hierarchy") The AI/DC solution organizes design intent into three levels, each owning a layer of the data center topology: * **Fabric** — super spine count, super spine switch template * **Pod** — spine count, role, spine switch template * **Rack** — rack type, leaf count, leaf switch template Each level carries enough information for its Generator to produce a complete set of devices, IP allocations, and links — without knowledge of the layers above or below. ### What operators define[​](#what-operators-define "Direct link to What operators define") **Design hierarchy objects:** | Design object | Key attributes | Example from demo data | | --------------- | ------------------------------------------------------- | ----------------------------------------------------------------------------------- | | `NetworkFabric` | `amount_of_super_spines`, `super_spine_switch_template` | Fabric-A: 6 super spines, `cisco-9364d-gx2-super-spine-switch` template | | `NetworkPod` | `amount_of_spines`, `role`, `spine_switch_template` | Pod-A2: `cisco-9364d-gx2-spine-switch` template | | `LocationRack` | `rack_type`, `amount_of_leafs`, `leaf_switch_template` | Rack-A2-1: type `compute`, 2 leafs, `cisco-93400ld-h1-leaf-switch-compute` template | **Supporting design objects:** | Design object | Purpose | Example | | ---------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------ | | Device templates | Interface layouts and port-role assignments per device role | `spine-switch`: Loopback0 + Ethernet ranges with leaf and super-spine profiles | | IP supernet | Top-level address space for all allocations | `10.0.0.0/8` with role `supernet` | | Prefix pool | Allocates fabric subnets from the supernet | `FabricSupernetPool` → one /16 per fabric | Data files The design objects above are loaded from the `objects/` directory. Device templates are defined in `objects/06_device_template.yml`, IP resources in `objects/04_ipam.yml`, overlay resource pools in `objects/07_pools.yml`, the physical hierarchy in `objects/10_fabric.yml` and `objects/11_rack.yml`, and the seed overlay tenant in `objects/12_overlay.yml`. See [Installation & Setup: Demo data](/infrahub-solution-ai-dc/solution-ai-dc/installation-setup.md#demo-data-object-files) for the full file list. ### What Generators produce[​](#what-generators-produce "Direct link to What Generators produce") | Generator | Reads | Produces | | ---------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | FabricGenerator | `NetworkFabric` | Fabric supernet prefix, super spine loopback pool, super spine devices with loopback IPs, fabric overlay ASN | | PodGenerator | `NetworkPod` | Pod supernet prefix, pod loopback pool, spine devices with loopback IPs, spine-to-super-spine links with /31 IPs, iBGP EVPN sessions | | RackGenerator | `LocationRack` | Leaf devices with loopback IPs, leaf-to-spine links with /31 IPs, VTEP loopbacks | | OverlayGenerator | `NetworkTenant` | Allocated overlay identifiers (VNI/VLAN/ASN/route target), tenant subnets and anycast gateways, segment-to-leaf placement | The demo defines 48 hierarchy objects (4 fabrics, 12 pods, 32 racks), per-vendor device templates, 1 IP supernet with a prefix pool, and overlay resource pools. From these inputs, the Generators produce devices across all four fabrics, hierarchical IP pool trees, hundreds of links with point-to-point addressing, per-vendor configuration, and computed attributes on every interface. Hands-on walkthrough The [Demo Guide](/infrahub-solution-ai-dc/solution-ai-dc/demo-guide.md) walks through each Generator's output step by step. This section focuses on the mapping between design and implementation. ## Why this matters for day-two operations[​](#why-this-matters-for-day-two-operations "Direct link to Why this matters for day-two operations") Design-driven automation is not only about the initial build — the stronger differentiator is what happens afterward. 1. **Design persists after Generators run.** Fabric, Pod, and Rack objects remain in Infrahub as queryable data. An operator can inspect "what was this rack designed to look like" at any time, independent of the devices that were generated from it. 2. **Changes are scoped.** Adding a rack means creating one `LocationRack` object and running the RackGenerator for that rack. Existing racks, devices, links, and IP allocations remain untouched. 3. **Design is versionable.** Changes happen on branches with diffs, reviews, and merges — the same workflow used for configuration changes. There is a full audit trail from design intent through to generated infrastructure. Try it The [Demo Guide: Day-two operations](/infrahub-solution-ai-dc/solution-ai-dc/demo-guide.md#day-two-operations-adding-a-rack) walks through adding a rack and running the RackGenerator to see scoped generation in action. ## Learn more[​](#learn-more "Direct link to Learn more") For deeper coverage of the concepts and patterns introduced on this page: * [Infrahub documentation: Generators](https://docs.infrahub.app/topics/generator) * [Infrahub documentation: Resource Manager](https://docs.infrahub.app/topics/resource-manager) * [Modular Generator architecture](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md) * [Generator patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md) * [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md) * [Multivendor configuration](/infrahub-solution-ai-dc/solution-ai-dc/multivendor-config.md) --- # EVPN/VXLAN overlay The overlay adds **multi-tenant Layer-2 and Layer-3 network services** on top of the fabric the AI/DC solution already builds. Operators declare tenant intent — a tenant, its routing instances, and its segments — and the solution allocates every overlay identifier, decides which leaf switches carry which segments, and renders the complete per-device EVPN/VXLAN configuration. The existing OSPF underlay and the Fabric → Pod → Rack build are untouched: the overlay rides on top of them with zero disruption. This page covers the operator-facing model, how generation works, and the key behaviors. For the implementation patterns behind the Generators, see [Generator patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md); for how Generators chain together, see [Modular Generator architecture](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md). ## What it does[​](#what-it-does "Direct link to What it does") The fabric provides loopback-to-loopback reachability between switches over an OSPF-routed IP **underlay**. The overlay is the iBGP **L2VPN-EVPN** control plane and **VXLAN** data plane carried on top of that underlay. It lets multiple tenants share one physical fabric while staying isolated from one another: * **Bridged Layer-2 services** stretched across the fabric (a VLAN extended fabric-wide over VXLAN). * **Routed Layer-3 services** with a **distributed anycast gateway** — the same default gateway present on every leaf that carries the segment, so a host's gateway is always local. * **Inter-subnet routing within a tenant**, while keeping different tenants' routes fully separated. Each fabric is an **independent overlay domain** with its own routing number (ASN) and identifier space. Cross-fabric interconnect (DCI) is out of scope for this release. ## The operator-facing model[​](#the-operator-facing-model "Direct link to The operator-facing model") Operators work entirely in intent. The model is a three-level hierarchy, all scoped to a single fabric: | Object | What it represents | | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Tenant** | An owner of overlay services (a customer or workload owner), scoped to exactly one fabric. Owns one or more VRFs. | | **VRF** | A tenant's isolated Layer-3 routing instance (IP-VRF). Owns one or more segments and defines the boundary for inter-subnet routing and tenant isolation. | | **Segment** | A tenant Layer-2 service — one VLAN bridged fabric-wide. Belongs to one VRF. Optionally carries a subnet and an anycast gateway. | Two operator choices shape how a segment behaves: * **Routed (IRB) vs L2-only.** A segment **with** a subnet and gateway is routed: it gets a distributed anycast gateway and participates in inter-subnet routing within its tenant (symmetric IRB). A segment **without** a gateway is **L2-only** — bridged across the fabric with no gateway and no inter-subnet routing, for cases where routing happens outside the fabric. * **Rack placement (optional).** By default a segment is available on **every leaf in the fabric**. An operator may restrict it to **specific racks**, in which case only those racks' leaf switches receive it. Operators never assign VNIs, VLANs, route targets, or the fabric ASN by hand — those are allocated automatically (see below). The tenant/VRF/segment model is the stable contract: a future eBGP control-plane mode can be introduced without changing what operators declare. Domain language The terms used here — tenant, VRF, segment, anycast gateway, VTEP, leaf/spine/super-spine, route reflector — carry precise meanings in this solution. See [`CONTEXT.md`](https://github.com/opsmill/infrahub-solution-ai-dc/blob/main/CONTEXT.md) for the shared language. ## How it works[​](#how-it-works "Direct link to How it works") A new **OverlayGenerator** turns tenant intent into implementation objects, and the per-device configuration transform renders those objects into device configuration. Generator naming The overlay Generator is registered as `generate-tenant` in `.infrahub.yml` (targeting the `tenants` group) — that is the name you select in the Infrahub UI. Its Python class is `OverlayGenerator`, the name used throughout this page. ### Allocation and materialization (OverlayGenerator)[​](#allocation-and-materialization-overlaygenerator "Direct link to Allocation and materialization (OverlayGenerator)") Triggered by changes to a tenant's design, the OverlayGenerator: 1. **Allocates overlay identifiers** from Infrahub Resource Manager pools — an **L2VNI** and **VLAN** per segment, an **L3VNI** and transit VLAN per VRF, and the per-fabric **ASN** — with no collisions inside a fabric. **Route targets** are derived from the fabric ASN and the VNI. 2. **Allocates the addressing** for routed segments: a tenant subnet from the overlay address space, and the segment's **anycast gateway** as the first usable host in that subnet. 3. **Materializes placement** — it writes the relationship that records *which leaf switches carry which segments*. Empty placement means every leaf in the tenant's fabric; otherwise only the leaves in the listed racks. All of these allocations are retained as **queryable data** in Infrahub — the system of record — inspectable independently of the generated configuration. The supporting fabric/pod/rack Generators are extended to allocate the per-fabric ASN, stamp each device's ASN and route-reflector role, give every leaf a dedicated **VTEP loopback** address, and materialize the iBGP **BGP sessions** along the cabling — so the whole control plane is queryable data, not just rendered text. ### Rendering the configuration (per-device transform)[​](#rendering-the-configuration-per-device-transform "Direct link to Rendering the configuration (per-device transform)") The per-device `startup_configuration` transform is expanded to render the overlay. The control plane is **iBGP L2VPN-EVPN** with **hierarchical route reflection**: leaf switches are clients of their spines, spines reflect to leaves and are themselves clients of the super-spines, which reflect to the spines. **VTEPs are leaf-only** — only leaves encapsulate VXLAN, sourced from their VTEP loopback. Routed segments use **symmetric IRB** (L2VNI bridging plus L3VNI routing with the distributed anycast gateway). The sections rendered depend on the device's role: | Section | Leaf | Spine | Super-spine | | ------------------------------------------------------ | --------------------- | ------------ | ------------- | | OSPF underlay (now also advertising the VTEP loopback) | yes | yes | yes | | `router bgp ` + iBGP EVPN neighbors | yes | yes | yes | | `route-reflector-client` toward lower-tier neighbors | — | yes (→ leaf) | yes (→ spine) | | `interface nve1` (VXLAN tunnel endpoint) | yes | — | — | | L2VNI bridging (`vlan` / `vn-segment`, `member vni`) | yes | — | — | | L3VNI routing (`vrf context`, transit SVI) | yes | — | — | | Anycast gateway SVI | yes (routed segments) | — | — | Configuration is rendered **per vendor**: each device is routed to its vendor's template (Cisco NX-OS, Arista EOS, Dell OS10, or Juniper Junos), so the same overlay model produces the correct syntax for whichever fabric a device belongs to. In the demo data each fabric is single-vendor — Fabric-A Cisco, Fabric-B Arista, Fabric-C Dell, Fabric-D Juniper. See [Multivendor configuration](/infrahub-solution-ai-dc/solution-ai-dc/multivendor-config.md) for how vendor is resolved and rendered. The leaf overlay block below is the **Cisco NX-OS** example (the Arista, Dell and Juniper templates render the equivalent constructs in their own syntax). The shape (see the [configuration artifact contract](https://github.com/opsmill/infrahub-solution-ai-dc/blob/main/specs/001-evpn-overlay/contracts/config-artifact.md) for the full definition): ``` router bgp router-id neighbor remote-as ! one per cabled spine update-source loopback0 address-family l2vpn evpn send-community extended interface nve1 source-interface loopback1 ! the leaf VTEP loopback host-reachability protocol bgp member vni ! per carried segment member vni associate-vrf ! per carried VRF vrf context ! per carried VRF vni address-family ipv4 unicast route-target both evpn interface Vlan ! only if the segment has a gateway vrf member ip address fabric forwarding mode anycast-gateway ``` A spine or super-spine renders the EVPN control plane and acts as a route reflector toward its lower tier, but renders **no** NVE, VLAN, VRF, or SVI. ## Key behaviors[​](#key-behaviors "Direct link to Key behaviors") * **Zero-disruption coexistence.** Introducing the overlay causes no change to the previously generated underlay configuration or to the existing fabric/pod/rack build. A fabric with no tenants still establishes the EVPN control-plane baseline and keeps operating its underlay. * **Scoped day-two changes.** Adding, modifying, or removing a tenant, VRF, or segment regenerates configuration for **only the affected leaves** — every unrelated device's configuration is byte-identical. Removing a segment releases its allocated identifiers. Day-two overlay changes follow the same scoped, branch-based, reviewable workflow as adding a rack. * **Leaves carry tenant state; spines do not.** Only leaves are VTEPs and only leaves render per-tenant segment, VRF, and gateway state. Spines and super-spines participate in the control plane purely as route reflectors and never carry tenant state. * **No collisions.** Every overlay identifier is allocated from a managed pool, so identifiers are unique within a fabric by construction. ## Try it[​](#try-it "Direct link to Try it") The [overlay quickstart](https://github.com/opsmill/infrahub-solution-ai-dc/blob/main/specs/001-evpn-overlay/quickstart.md) walks through end-to-end validation: lint and test, load the schema and overlay seed data, run the full build, then inspect a leaf's rendered configuration and confirm the spine/super-spine carry no tenant state. It also covers verifying scoped day-two changes and the rack-placement and L2-only variations. ## Learn more[​](#learn-more "Direct link to Learn more") * [Multivendor configuration](/infrahub-solution-ai-dc/solution-ai-dc/multivendor-config.md) — how the overlay (and underlay) configuration is rendered per vendor (Cisco/Arista/Dell/Juniper). * [`CONTEXT.md`](https://github.com/opsmill/infrahub-solution-ai-dc/blob/main/CONTEXT.md) — the domain language for the overlay (tenant, VRF, segment, VTEP, anycast gateway, route reflector). * Architecture Decision Records [`dev/adr/0001`–`0005`](https://github.com/opsmill/infrahub-solution-ai-dc/tree/main/dev/adr) — the rationale behind the design split, the OverlayGenerator's ownership, hierarchical route reflection, and the stored BGP-session and route-reflector model. * The feature [specification](https://github.com/opsmill/infrahub-solution-ai-dc/blob/main/specs/001-evpn-overlay/spec.md) and [implementation plan](https://github.com/opsmill/infrahub-solution-ai-dc/blob/main/specs/001-evpn-overlay/plan.md) for the full requirements and design. --- # Generator patterns This page covers the implementation patterns used in the AI/DC solution's Generators. It is aimed at developers who want to understand, adapt, or extend the code. Each section focuses on a specific pattern with annotated code from the solution. See [Modular Generator architecture](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md) for the architectural concepts (signaling mechanism, validation gates, interface allocation design) and [Design-driven automation](/infrahub-solution-ai-dc/solution-ai-dc/design-driven-automation.md) for what the Generators read and produce. ## Generator definition wiring[​](#generator-definition-wiring "Direct link to Generator definition wiring") Each Generator is registered in `.infrahub.yml` with a definition that connects a Python class to a GraphQL query and a target group: ``` generator_definitions: - name: generate-fabric file_path: "./generators/generate_fabric.py" query: generate_fabric targets: fabrics parameters: fabric_name: name__value class_name: FabricGenerator convert_query_response: false execute_in_proposed_change: false execute_after_merge: false ``` Key fields: | Field | Purpose | | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `query` | References a named GraphQL query defined in the `queries` section of `.infrahub.yml` | | `targets` | The group whose members are valid targets for this Generator (e.g., `fabrics` is a `CoreStandardGroup`) | | `parameters` | Maps GraphQL query variables to target object attributes. `fabric_name: name__value` passes the target's `name` as the `$fabric_name` query variable | | `class_name` | The Python class inside `file_path` that Infrahub instantiates | | `convert_query_response` | Set to `false` to receive the raw GraphQL response as a dict. The Generators parse it with Pydantic models for type safety | The overlay Generator follows the same wiring pattern, targeting the `tenants` group. Note the naming split: the definition, file, and query use `generate-tenant`/`generate_tenant`, while the Python class is `OverlayGenerator`. ``` generator_definitions: - name: generate-tenant file_path: "./generators/generate_tenant.py" query: generate_tenant targets: tenants parameters: name: name__value class_name: OverlayGenerator convert_query_response: false execute_in_proposed_change: false execute_after_merge: false ``` ## GraphQL queries[​](#graphql-queries "Direct link to GraphQL queries") Each Generator has a paired `.gql` file that fetches the design object and its context. The query receives parameters mapped from the target object via the definition in `.infrahub.yml`. ### Fetching design context in a single query[​](#fetching-design-context-in-a-single-query "Direct link to Fetching design context in a single query") The PodGenerator needs data from both the Pod itself and its parent Fabric (super spine count, sorting methods). The query uses an inline fragment (`... on NetworkFabric`) to fetch parent-specific fields in a single request: ``` query PodGeneratorQuery($pod_name: String!) { NetworkPod(name__value: $pod_name) { edges { node { id amount_of_spines { value } name { value } index { value } role { value } spine_switch_template { node { __typename, id } } parent { node { __typename id name { value } ... on NetworkFabric { amount_of_super_spines { value } fabric_interface_sorting_method { value } spine_interface_sorting_method { value } } } } } } } } ``` The RackGenerator follows the same pattern — it fetches the Rack's attributes and traverses the `pod` relationship to get the pod's IP pools, spine count, and sorting methods. ### Parsing query responses with Pydantic[​](#parsing-query-responses-with-pydantic "Direct link to Parsing query responses with Pydantic") Each query has a corresponding Pydantic model file (e.g., `fabric_generator_query.py`, `pod_generator_query.py`) that mirrors the GraphQL response structure. The Generator parses the raw dict into typed models as the first step of `generate()`: ``` from .pod_generator_query import PodGeneratorQuery async def generate(self, data: dict) -> None: data: PodGeneratorQuery = PodGeneratorQuery(**data) self.pod_id = data.network_pod.edges[0].node.id self.pod_index = data.network_pod.edges[0].node.index.value self.pod_name = data.network_pod.edges[0].node.name.value.lower() self.fabric_amount_of_super_spines = ( data.network_pod.edges[0].node.parent.node.amount_of_super_spines.value ) ``` This gives type-safe access to nested fields and catches schema mismatches at parse time rather than deep in the generation logic. ## Device creation[​](#device-creation "Direct link to Device creation") Devices are created from object templates using the Infrahub SDK. The pattern is the same across all three Generators — only the role, naming convention, and template differ. ### Creating a device from a template[​](#creating-a-device-from-a-template "Direct link to Creating a device from a template") ``` device = await self.client.create( NetworkDevice, hostname=f"spine-{self.pod_name}-{idx}", object_template={"id": self.pod_spine_switch_template}, pod={"id": self.pod_id}, loopback_ip=self.loopback_pool, role="spine", member_of_groups=["devices"], ) await device.save(allow_upsert=True) ``` Key aspects: * **`object_template`** — references a `CoreObjectTemplate` by ID. Infrahub stamps out the device with the template's predefined interfaces (Loopback0, Ethernet ranges with role profiles) * **`loopback_ip`** — passing a pool object triggers automatic IP allocation from that pool * **`allow_upsert=True`** — if a device with this hostname already exists, the save updates it rather than creating a duplicate. This is what makes re-runs safe * **`member_of_groups`** — adds the device to the `devices` group, making it a target for artifact definitions ### Assigning the loopback IP to an interface[​](#assigning-the-loopback-ip-to-an-interface "Direct link to Assigning the loopback IP to an interface") After device creation, the allocated loopback IP must be assigned to the device's Loopback0 interface: ``` device = await self.client.get( NetworkDevice, id=device.id, include=["ip_address"], exclude=["rack", "pod", "role", "hostname", "object_template", "member_of_groups"], ) loopback_interface = await self.client.get( NetworkInterface, device__ids=[device.id], role__value="loopback" ) loopback_interface.status.value = "active" loopback_interface.ip_address = device.loopback_ip.id await loopback_interface.save(allow_upsert=True) ``` The device is re-fetched with `include=["ip_address"]` to retrieve the pool-allocated IP address ID, which is then assigned to the loopback interface. ## IP pool allocation[​](#ip-pool-allocation "Direct link to IP pool allocation") The Generators build a hierarchical IP pool tree using Infrahub's [Resource Manager](https://docs.infrahub.app/topics/resource-manager). Each tier carves a subnet from the pool created by the tier above. ### Hierarchical pool delegation[​](#hierarchical-pool-delegation "Direct link to Hierarchical pool delegation") The FabricGenerator creates the top-level pools: ``` # Allocate a /16 from the global FabricSupernetPool fabric_supernet_pool = await self.client.get( kind=CoreIPPrefixPool, name__value="FabricSupernetPool" ) fabric_supernet = await self.client.allocate_next_ip_prefix( resource_pool=fabric_supernet_pool, identifier=self.fabric_id, data={"role": "fabric_supernet"}, ) # Create a fabric-scoped prefix pool backed by the allocated /16 fabric_prefix_pool = await self.client.create( kind=CoreIPPrefixPool, name=f"{self.fabric_name}-prefix-pool", default_prefix_type="IpamIPPrefix", default_prefix_length=24, ip_namespace={"hfid": ["default"]}, resources=[fabric_supernet], ) await fabric_prefix_pool.save(allow_upsert=True) ``` The PodGenerator then allocates from the fabric prefix pool: ``` # Allocate a /19 pod supernet from the fabric prefix pool pod_supernet = await self.client.allocate_next_ip_prefix( resource_pool=fabric_prefix_pool, identifier=self.pod_id, member_type="prefix", prefix_length=19, data={"role": "pod_supernet"}, ) ``` ### Deterministic allocation with identifiers[​](#deterministic-allocation-with-identifiers "Direct link to Deterministic allocation with identifiers") Every `allocate_next_ip_prefix` call uses the object's ID as the `identifier` parameter. This makes allocation idempotent — calling the method again with the same identifier returns the previously allocated prefix rather than allocating a new one. For point-to-point link addressing, the identifier is the concatenation of both interface IDs: ``` prefix = await client.allocate_next_ip_prefix( resource_pool=pool, identifier=src_interface.id + dst_interface.id, member_type="address", prefix_length=31, data={"role": prefix_role}, ) ``` The same interface pair always produces the same /31 prefix. ## Cabling[​](#cabling "Direct link to Cabling") Cabling connects devices across tiers — spines to super spines, leafs to spines. The pattern has three phases: sort interfaces, build a cabling plan, then create links and assign IPs. See [Modular Generator architecture: Deterministic interface allocation](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md#deterministic-interface-allocation) for the design rationale. ### Interface sorting[​](#interface-sorting "Direct link to Interface sorting") Interfaces are grouped by device and sorted using `netutils.interface.sort_interface_list`. The sorting function is selected at runtime based on the schema's sorting method attribute: ``` from infrahub_solution_ai_dc import sorting # Resolve the sorting function from the schema dropdown value fabric_interface_sorting_method = ( data.network_pod.edges[0].node.parent.node.fabric_interface_sorting_method.value ) self.fabric_interface_sorting_function = getattr( sorting, fabric_interface_sorting_method ) ``` The sorting functions produce a `dict[NetworkDevice, list[NetworkInterface]]` — a stable, ordered mapping from device to interfaces that the cabling plan consumes. ### Building and executing a cabling plan[​](#building-and-executing-a-cabling-plan "Direct link to Building and executing a cabling plan") The PodGenerator's cabling sequence illustrates the full pattern: ``` async def connect_spine_to_super_spine(self) -> None: # 1. Fetch interfaces by role spine_interfaces = await self.client.filters( kind=NetworkInterface, device__ids=[spine.id for spine in self.spine_switches], role__value="super_spine", ) super_spine_interfaces = await self.client.filters( kind=NetworkInterface, device__ids=[ss.id for ss in self.super_spine_switches], role__value="spine", ) # 2. Sort interfaces per device spine_interface_map = self.spine_interface_sorting_function(spine_interfaces) super_spine_interface_map = self.fabric_interface_sorting_function( super_spine_interfaces ) # 3. Build the cabling plan using the pod index cabling_plan = build_pod_cabling_plan( pod_index=self.pod_index, src_interface_map=spine_interface_map, dst_interface_map=super_spine_interface_map, ) # 4. Create NetworkLink objects and mark interfaces active await connect_interface_maps( client=self.client, logger=self.logger, cabling_plan=cabling_plan ) # 5. Allocate /31 prefixes and assign IPs to both endpoints await assign_ip_addresses_to_p2p_connections( client=self.client, logger=self.logger, connections=cabling_plan, prefix_len=31, prefix_role="pod_super_spine_spine", pool=self.pod_prefix_pool, ) ``` The RackGenerator follows the same five-step pattern with `build_rack_cabling_plan` and the rack index. ### Link creation[​](#link-creation "Direct link to Link creation") Each interface pair in the cabling plan becomes a `NetworkLink` object: ``` network_link = await client.create( kind="NetworkLink", name=f"{src_interface.device.display_label}-{src_interface.name.value}" f"__{dst_interface.device.display_label}-{dst_interface.name.value}", medium="copper", endpoints=[src_interface, dst_interface], ) await network_link.save(allow_upsert=True) ``` Both interfaces are then marked as `active`. The `allow_upsert=True` on the link save ensures that re-running the Generator does not create duplicate links. ## Checksum propagation[​](#checksum-propagation "Direct link to Checksum propagation") After the FabricGenerator and PodGenerator complete their work, they write a checksum to each child object to trigger the next tier. The RackGenerator is the final tier of the physical cascade and does not propagate a checksum to a child. The OverlayGenerator uses the same checksum mechanism differently: it stamps a content checksum on the tenant it just materialized (guarded so it only changes when the tenant's segment set changes), which is what the tenant trigger re-fires on — without creating a self-retrigger loop. ### Calculating the checksum[​](#calculating-the-checksum "Direct link to Calculating the checksum") The checksum is a SHA-256 hash of all object IDs accessed during the Generator run: ``` class GeneratorMixin: def calculate_checksum(self) -> str: related_ids = ( self.client.group_context.related_group_ids + self.client.group_context.related_node_ids ) sorted_ids = sorted(related_ids) joined = ",".join(sorted_ids) return hashlib.sha256(joined.encode("utf-8")).hexdigest() ``` The IDs are sorted before hashing, so the order in which objects were accessed does not matter — only the set of objects determines the checksum. ### Writing the checksum to children[​](#writing-the-checksum-to-children "Direct link to Writing the checksum to children") The FabricGenerator writes the checksum to all child Pods; the PodGenerator writes it to all child Racks: ``` async def update_checksum(self) -> None: racks = await self.client.filters(kind=LocationRack, pod__ids=[self.pod_id]) checksum = self.calculate_checksum() for rack in racks: if rack.checksum.value != checksum: rack.checksum.value = checksum await rack.save(allow_upsert=True) ``` The `if rack.checksum.value != checksum` guard prevents unnecessary saves — if the checksum has not changed (because the Generator produced identical results), no trigger fires and downstream execution stops. ## Shared library[​](#shared-library "Direct link to Shared library") The Generators share common logic through the `infrahub_solution_ai_dc` package (in `src/infrahub_solution_ai_dc/`). This package is installed into the custom Infrahub Docker image so that task workers can import it at runtime. | Module | Purpose | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `generator.py` | `GeneratorMixin` — checksum calculation | | `cabling.py` | `build_pod_cabling_plan`, `build_rack_cabling_plan`, `connect_interface_maps` — cabling plan algorithms and link creation | | `sorting.py` | `create_sorted_device_interface_map`, `create_reverse_sorted_device_interface_map` — interface sorting strategies | | `addressing.py` | `assign_ip_addresses_to_p2p_connections`, `assign_ip_address_to_interface` — IP allocation and assignment | | `overlay.py` | `route_target`, `rr_client`, `upsert_evpn_session`, `resolve_segment_devices` — overlay route-target/route-reflector helpers, iBGP session upserts, and segment-to-leaf placement | | `vendors.py` | `vendor_group_for_manufacturer`, `vendor_group_for_template` — resolves a device's vendor group (`{manufacturer}_devices`, from the manufacturer name in lower case) for per-vendor artifact targeting | | `protocols.py` | Auto-generated SDK protocol classes (`infrahubctl protocols`) — typed interfaces for `NetworkDevice`, `NetworkInterface`, `LocationRack`, etc. | ## Learn more[​](#learn-more "Direct link to Learn more") * [Modular Generator architecture](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md) — signaling mechanism, validation gates, interface allocation design * [Design-driven automation](/infrahub-solution-ai-dc/solution-ai-dc/design-driven-automation.md) — what the Generators read and produce * [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md) — the OverlayGenerator's allocation and placement patterns * [Multivendor configuration](/infrahub-solution-ai-dc/solution-ai-dc/multivendor-config.md) — per-vendor resolution and artifact targeting * [Installation & Setup](/infrahub-solution-ai-dc/solution-ai-dc/installation-setup.md) — repository structure and environment setup * [Infrahub documentation: Generators](https://docs.infrahub.app/topics/generator) * [Infrahub documentation: Resource Manager](https://docs.infrahub.app/topics/resource-manager) --- # Installation & Setup Everything needed to run the AI/DC solution locally is included in the repository. The environment runs in Docker; you interact with it through the `infrahubctl` CLI and the Infrahub web UI. ## Requirements[​](#requirements "Direct link to Requirements") * **Python 3.11+** (3.12 recommended) * **Docker** and **Docker Compose** (v2) * **[uv](https://docs.astral.sh/uv/)** — Python package manager * **Git** — for cloning the repository infrahubctl is included The `infrahubctl` CLI is installed automatically as part of the `infrahub-sdk` dependency. No separate installation is needed. ## Getting started[​](#getting-started "Direct link to Getting started") ### Cloning the repository[​](#cloning-the-repository "Direct link to Cloning the repository") First we have to clone the repository. ``` git clone https://github.com/opsmill/infrahub-solution-ai-dc.git ``` ### Install dependencies[​](#install-dependencies "Direct link to Install dependencies") ``` uv sync --all-packages ``` This installs all Python dependencies including `infrahub-sdk` (which provides `infrahubctl`), `invoke` (task runner), and the solution's own `infrahub-solution-ai-dc` package. ### Configure environment variables[​](#configure-environment-variables "Direct link to Configure environment variables") ``` export INFRAHUB_USERNAME="admin" export INFRAHUB_PASSWORD="infrahub" ``` `INFRAHUB_USERNAME` and `INFRAHUB_PASSWORD` are used by `infrahubctl` to authenticate with the Infrahub API. `VERSION` sets the Infrahub image tag. If not using direnv, source the file manually or export these variables. One optional variable belongs to the MCP server rather than to `infrahubctl`, so keep it out of the block above: `INFRAHUB_API_TOKEN` sets the token an MCP client presents to the [Infrahub MCP server](#infrahub-mcp-server). Setting it takes precedence over `INFRAHUB_USERNAME` / `INFRAHUB_PASSWORD` for `infrahubctl` too, so export it only once you have a token to use: ``` export INFRAHUB_API_TOKEN="" ``` Alternatively, you can use [direnv](https://direnv.net) and define the environment variables in a `.envrc` file at the root of the project. ### Choose an Infrahub edition[​](#choose-an-infrahub-edition "Direct link to Choose an Infrahub edition") The solution runs on either Infrahub Community or Infrahub Enterprise. `INFRAHUB_EDITION` picks between them and defaults to `community`, so no configuration is needed to stay on Community. ``` export INFRAHUB_EDITION="enterprise" ``` It is the only variable to set. From it, `tasks.py` derives everything that differs between the editions: | | `community` | `enterprise` | | ---------------- | -------------------------------------- | ------------------------------------------------- | | Compose file | `https://infrahub.opsmill.io` | `https://infrahub.opsmill.io/enterprise` | | Base image | `registry.opsmill.io/opsmill/infrahub` | `registry.opsmill.io/opsmill/infrahub-enterprise` | | Image built here | `opsmill/infrahub-solution-ai-dc` | `opsmill/infrahub-enterprise-solution-ai-dc` | The two editions build to different image names, so switching back and forth never runs one edition's stack on the other's image. Enterprise adds one service, `task-manager-background-svc`, which runs the stock Enterprise image rather than the custom build. Enterprise images live in a private registry. Authenticate before building: ``` docker login registry.opsmill.io ``` Every task that drives Compose also takes `--edition` to override the variable for a single command, for example `inv build --edition=enterprise`. note `docker-compose.yml` is the compose file for one edition, and it is committed to the repository. After changing `INFRAHUB_EDITION`, fetch the matching one: ``` uv run inv download-compose-file --override ``` The tasks refuse to run against a compose file from the other edition rather than starting a mixed stack, and the error names this command. ### Build container image[​](#build-container-image "Direct link to Build container image") ``` uv run inv build ``` This will build a custom Infrahub container image, this is needed because this solution is using a custom Python package `src/infrahub_solution_ai_dc` that we want to use in our Generators. This makes importing classes and functions from this package easier and more reliable. ### Start Infrahub[​](#start-infrahub "Direct link to Start Infrahub") ``` uv run inv start ``` This downloads the base `docker-compose.yml` for the selected [edition](#choose-an-infrahub-edition) (if not already present), builds a custom Docker image, then runs `docker compose up -d`. The custom image extends the standard Infrahub image with the `infrahub-solution-ai-dc` Python package — a shared library (in `src/infrahub_solution_ai_dc/`) that the Generators and Transforms use for common logic such as IP addressing, cabling, and interface sorting. Without it, Generators running inside the task workers would not have access to that shared code. The override file (`docker-compose.override.yml`) ensures the custom image is used in place of the standard one. **Services started:** | Service | Description | Port | | ----------------------------- | --------------------------------------------- | -------------- | | `infrahub-server` | Infrahub API and web UI | `8000` | | `task-manager` | Prefect API server for workflow orchestration | `4200` | | `task-worker` | Infrahub task workers (2 replicas) | — | | `database` | Neo4j graph database | `7474`, `7687` | | `cache` | Redis | — | | `message-queue` | RabbitMQ | — | | `task-manager-db` | PostgreSQL for Prefect | — | | `infrahub-mcp` | Infrahub MCP server for AI agents | `8001` | | `task-manager-background-svc` | Prefect background services (Enterprise only) | — | Verify the environment Open `http://localhost:8000` in a browser and log in with username `admin`, password `infrahub`. ### Infrahub MCP server[​](#infrahub-mcp-server "Direct link to Infrahub MCP server") `inv start` also brings up `infrahub-mcp`, a sidecar container running the Infrahub MCP server. It exposes Infrahub as a set of MCP tools so an AI agent can read the graph and make changes to it, reaching `infrahub-server` over the internal Docker network. Once the stack is up, check it responds: ``` curl http://localhost:8001/health ``` **Connecting an agent.** The repository ships a `.mcp.json` at its root, so Claude Code opened in the repository detects the server with no configuration. Detection is not approval: a server that a project defines stays at *pending approval* until you accept it in an interactive session, and a fresh clone cannot approve its own servers. Approve `infrahub` at that prompt on first run and the `mcp__infrahub__*` tools become available. You do this once per clone. **Authentication.** The container is configured with no Infrahub credential. Every client authenticates for itself by sending its own token in an `Authorization: Bearer` header, and a request that carries no such header is rejected. Like Infrahub's own port `8000`, the MCP port is published on all interfaces. **Version.** The image is pinned in `docker-compose.override.yml` to `registry.opsmill.io/opsmill/infrahub-mcp:v1.1.7`. Bump it to another published version with `INFRAHUB_MCP_VERSION`: ``` export INFRAHUB_MCP_VERSION="v1.1.8" uv run inv start ``` **Other settings.** `INFRAHUB_MCP_LOG_LEVEL` (default `info`) sets how much the container logs, readable with `docker compose logs infrahub-mcp`. `INFRAHUB_MCP_READ_ONLY=true` disables the write tools, so agents can query the graph but not change it. **Other MCP clients.** Any client that speaks the `streamable-http` transport can use the same endpoint: ``` { "mcpServers": { "infrahub": { "type": "streamable-http", "url": "http://localhost:8001/mcp", "headers": { "Authorization": "Bearer " } } } } ``` **Agent changes.** Writes made through the MCP server do not land on `main`. Each session gets its own Infrahub branch named `mcp/session--`, and the changes on it reach `main` through a proposed change that you review — nothing is merged automatically. These are Infrahub branches carrying data changes only; no git branch is created. ### Load data[​](#load-data "Direct link to Load data") ``` uv run inv load ``` This runs four steps in sequence: 1. `infrahubctl schema load schemas` — loads the 7 schema files defining the data model 2. `infrahubctl menu load menus/` — loads UI menu configuration 3. `infrahubctl object load objects/` — loads all design objects (groups, manufacturers, device types, IPAM, overlay resource pools, profiles, device templates, fabrics, pods, racks, and the seed `Blue` overlay tenant) 4. `infrahubctl object load repository.yml` — registers the Git repository with Infrahub Repository sync After `inv load`, Infrahub imports the repository and reads `.infrahub.yml` to register Generators, Transforms, queries, and artifact definitions. Verify with `infrahubctl repository list`. Generators cannot run until this completes. ### Load trigger rules[​](#load-trigger-rules "Direct link to Load trigger rules") ``` uv run infrahubctl object load triggers.yml ``` This creates the `CoreGeneratorAction` and `CoreNodeTriggerRule` objects that drive automatic modular Generator execution — including the tenant/overlay trigger (`run-tenant-generator`). `triggers.yml` is a committed top-level file that is **not** part of `inv load`: it must be loaded after the repository has synced, because the trigger rules reference Generator definitions that need to exist first. Load triggers after repository sync If loaded before Generator definitions are imported, the trigger rules will fail to resolve their action references. ### Load supplementary data (optional)[​](#load-supplementary-data-optional "Direct link to Load supplementary data (optional)") ``` # NOTE: the data/ directory is NOT loaded by `inv load`. uv run infrahubctl object load data/permissions.yml # sample operator account `john` uv run infrahubctl object load data/tenant-red.yml # second, day-two overlay tenant `Red` ``` The `data/` directory holds material that is intentionally kept out of the default load: a sample operator persona (account `john`, role and group definitions) and a second overlay tenant (`Red` on Fabric-A) used to demonstrate scoped, non-clobbering day-two multi-tenant generation. Load these manually when you want them. ## Repository structure[​](#repository-structure "Direct link to Repository structure") The repository is self-contained — everything below is included. You do not need to build any of these from scratch. ### Schemas[​](#schemas "Direct link to Schemas") | File | Contents | | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `schemas/logical_design.yml` | `NetworkFabric` and `NetworkPod` — the design hierarchy with Generator signaling attributes (checksum, amount\_of\_spines) and the fabric overlay ASN | | `schemas/physical_location.yml` | `LocationHall` and `LocationRack` — physical locations with Generator target support | | `schemas/device.yml` | `NetworkDevice`, `NetworkInterface`, and `NetworkLink` — devices with computed attributes, ASN, route-reflector role, and VTEP loopback | | `schemas/ipam.yml` | `IpamIPPrefix` with role-based allocation and `IpamIPAddress` | | `schemas/overlay.yml` | `NetworkTenant`, `NetworkVrf`, and `NetworkSegment` — the overlay intent model | | `schemas/routing.yml` | `NetworkBGPSession` — the stored iBGP EVPN control-plane sessions | | `schemas/generator.yml` | `GeneratorTarget` generic enabling trigger-based modular Generator execution via checksum attribute | ### Generators[​](#generators "Direct link to Generators") | File | Responsibility | | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | `generators/generate_fabric.py` | FabricGenerator — IP pool and overlay ASN allocation, super spine creation, writes checksums to child Pods | | `generators/generate_pod.py` | PodGenerator — validates fabric complete, spine creation, spine-to-super-spine cabling, iBGP EVPN sessions, writes checksums to child Racks | | `generators/generate_rack.py` | RackGenerator — validates pod complete, leaf creation, leaf-to-spine cabling, VTEP loopbacks | | `generators/generate_tenant.py` | OverlayGenerator (registered as `generate-tenant`, targets the `tenants` group) — allocates overlay identifiers and materializes tenant/VRF/segment state | Each Generator has a paired `.gql` query file in the same directory. The `.infrahub.yml` file in the repository root wires Generator definitions to their queries and target groups. Note the naming split for the overlay Generator: the definition, file, and query are named `generate-tenant`/`generate_tenant`, while the Python class is `OverlayGenerator`. ### Transforms and artifacts[​](#transforms-and-artifacts "Direct link to Transforms and artifacts") | File | Purpose | | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `transforms/startup_config.gql` + `templates/startup_config_{cisco,arista,dell,juniper}.j2` | Four per-vendor Jinja2 Transforms, each producing a startup configuration artifact for the devices in its vendor group (`{manufacturer}_devices`). See [Multivendor configuration](/infrahub-solution-ai-dc/solution-ai-dc/multivendor-config.md). | | `transforms/cabling_plan.py` / `.gql` | Python Transform producing a CSV cabling plan artifact per fabric | | `transforms/computed_interface_description.py` / `.gql` | Python Transform applied as a computed attribute on each interface | ### Demo data (object files)[​](#demo-data-object-files "Direct link to Demo data (object files)") Loaded in numbered order by `infrahubctl object load objects/`: | File | Contents | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `objects/01_groups.yml` | `CoreStandardGroup` objects: `halls`, `racks`, `fabrics`, `pods`, `devices`, `tenants`, and the per-vendor device groups `cisco_devices` / `arista_devices` / `dell_devices` / `juniper_devices` | | `objects/02_manufacturer.yml` | Equipment manufacturers (Cisco, Arista, Dell, Juniper) | | `objects/03_device_type.yml` | Per-vendor device types for leaf, spine, and super spine switches | | `objects/04_ipam.yml` | IP supernet (`10.0.0.0/8`) and `FabricSupernetPool` | | `objects/05_profiles.yml` | Interface role profiles (MTU, role assignments) | | `objects/06_device_template.yml` | Per-vendor device templates defining interface layouts per device role | | `objects/07_pools.yml` | Overlay resource pools: ASN, L2VNI, L3VNI, and VLAN number pools | | `objects/10_fabric.yml` | Fabric and Pod objects: Fabric-A (Cisco), Fabric-B (Arista), Fabric-C (Dell), Fabric-D (Juniper) — each single-vendor | | `objects/11_rack.yml` | Rack objects across all four fabrics with leaf counts and per-vendor template assignments | | `objects/12_overlay.yml` | Seed overlay tenant `Blue` on Fabric-A, with a VRF and segments (routed, rack-scoped, and L2-only examples) | Trigger rules live in the top-level `triggers.yml` (not under `objects/`) and are loaded separately after repository sync — see [Load trigger rules](#load-trigger-rules). Supplementary data (`data/permissions.yml`, `data/tenant-red.yml`) is loaded manually — see [Load supplementary data](#load-supplementary-data-optional). ### Infrastructure[​](#infrastructure "Direct link to Infrastructure") | File | Purpose | | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Dockerfile` | Builds a custom Infrahub image with the `infrahub-solution-ai-dc` Python package installed; the base image and runtime user come from the selected edition | | `docker-compose.override.yml` | Replaces standard Infrahub image with the custom build; mounts `src/` for live code changes; defines the `infrahub-mcp` sidecar | | `.mcp.json` | MCP client configuration pointing at the `infrahub-mcp` sidecar | | `repository.yml` | `CoreRepository` object pointing to `/upstream` (the mounted repository) | | `tasks.py` | Invoke task definitions for environment management | | `.infrahub.yml` | Central configuration wiring Generators, Transforms, queries, and artifacts | ## Invoke tasks[​](#invoke-tasks "Direct link to Invoke tasks") | Task | Command | Description | | ---------------- | -------------------------------------- | ------------------------------------------------------------------ | | Start | `inv start` | Download compose file (if needed) and start all services | | Stop | `inv stop` | Stop containers, remove networks (data preserved) | | Destroy | `inv destroy` | Stop containers, remove networks **and volumes** (full reset) | | Restart | `inv restart` | Restart all services (or `inv restart --component=` for one) | | Load | `inv load` | Load schemas, menus, objects, and register repository | | Load schema | `inv load-schema` | Load schema files only | | Load menu | `inv load-menu` | Load menu definitions only | | Build | `inv build` | Build the custom Docker image locally | | Download compose | `inv download-compose-file --override` | Fetch the compose file for the selected edition | | Test | `inv test` | Run pytest test suite | | Format | `inv format` | Run Ruff formatter and auto-fix linting issues | ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Services fail to start[​](#services-fail-to-start "Direct link to Services fail to start") * Check Docker is running: `docker info` * Check port conflicts: Infrahub uses ports 8000, 4200, 7474, 7687, and 8001 for the MCP server * View logs: `docker compose logs infrahub-server` or `docker compose logs task-worker` * Full reset: `uv run inv destroy && uv run inv start` ### Compose file does not match the edition[​](#compose-file-does-not-match-the-edition "Direct link to Compose file does not match the edition") * The tasks stop with `docker-compose.yml is the compose file but the edition is ` — the committed compose file belongs to the other edition * Fetch the matching one: `uv run inv download-compose-file --override` * Check which one is on disk: `grep -c infrahub-enterprise docker-compose.yml` returns `0` for Community ### Enterprise image fails to pull[​](#enterprise-image-fails-to-pull "Direct link to Enterprise image fails to pull") * Authenticate against the private registry: `docker login registry.opsmill.io` * Confirm the tag exists for the pinned version: `docker manifest inspect registry.opsmill.io/opsmill/infrahub-enterprise:$VERSION` * `VERSION` and `INFRAHUB_BASE_VERSION` have to name a version published for Enterprise, which does not always match the Community release list ### Repository not syncing[​](#repository-not-syncing "Direct link to Repository not syncing") * Verify with `infrahubctl repository list` * The repository mounts from `/upstream` inside the container — check `docker compose logs task-worker` for import errors * Schema mismatches between what `.infrahub.yml` references and what is loaded can block sync ### MCP tool calls fail authentication[​](#mcp-tool-calls-fail-authentication "Direct link to MCP tool calls fail authentication") * Check what your shell is presenting: `echo $INFRAHUB_API_TOKEN` * A token exported for a different Infrahub instance fails closed here rather than falling back — unset it or export one for this stack * Confirm the server is up: `curl http://localhost:8001/health` ### MCP port already in use[​](#mcp-port-already-in-use "Direct link to MCP port already in use") * Move both the published port and the client address with one variable: `export INFRAHUB_MCP_PORT=8011`, then `uv run inv start` * Export it in the shell your agent runs in as well, and restart the agent — it reads the port when it loads `.mcp.json` * Confirm what the container publishes: `docker compose ps infrahub-mcp` ### MCP image fails to pull[​](#mcp-image-fails-to-pull "Direct link to MCP image fails to pull") * Confirm the pinned tag exists: `docker manifest inspect registry.opsmill.io/opsmill/infrahub-mcp:v1.1.7` * Select a different published version: `export INFRAHUB_MCP_VERSION=` * Point at a different registry or repository: `export INFRAHUB_MCP_DOCKER_IMAGE=` --- # Modular Generator architecture The AI/DC solution uses four modular Generators. Three build the physical fabric — FabricGenerator, PodGenerator, RackGenerator — connecting automatically through checksum-based triggers. A fourth, the OverlayGenerator (`generate-tenant`), layers EVPN/VXLAN services on top; it uses the same checksum-trigger mechanism but is signaled by tenant design changes rather than by the fabric cascade. This page explains the signaling mechanism, how each tier validates that the previous tier completed, and how interface allocation works. See [Design-driven automation](/infrahub-solution-ai-dc/solution-ai-dc/design-driven-automation.md) for what the Generators read and produce, or [Generator patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md) for implementation-level code patterns. ![Modular Generator execution flow](/assets/images/modular-generator-flow-9c11d62bb17e252a49c7ce53da2ca303.svg) ## How the modular Generators connect[​](#how-the-modular-generators-connect "Direct link to How the modular Generators connect") ### Checksum-driven signaling[​](#checksum-driven-signaling "Direct link to Checksum-driven signaling") Each Generator writes a checksum to its child objects after it finishes. That checksum change fires a trigger rule, which runs the next Generator tier. The process repeats until the lowest tier completes. The `GeneratorTarget` generic (defined in `schemas/generator.yml`) adds a `checksum` attribute to both `NetworkPod` and `LocationRack`. This attribute is the only thing that flows from parent to child — it carries no configuration, only a signal that "something changed upstream." The checksum itself is a SHA-256 hash of all object IDs fetched during the Generator run. It is deterministic: the same set of objects produces the same hash. If a Generator re-runs and nothing has changed, the checksum stays the same and no downstream triggers fire. ### Trigger rules[​](#trigger-rules "Direct link to Trigger rules") Trigger rules connect checksum changes to Generator actions. Each rule watches for an attribute update on a specific node type and fires the corresponding Generator. | Trigger | Watches | Attribute | Fires | | ------------------------ | --------------- | ----------------------------------------------- | ----------------- | | Pod Generator triggers | `NetworkPod` | `checksum`, `index`, `amount_of_spines`, `role` | `generate-pod` | | Rack Generator triggers | `LocationRack` | `checksum`, `index`, `rack_type` | `generate-rack` | | Tenant Generator trigger | `NetworkTenant` | `checksum` | `generate-tenant` | The checksum trigger handles the modular execution flow. The additional attribute triggers allow a Generator to re-run when an operator changes a design parameter (for example, updating `amount_of_spines` on a Pod). The tenant trigger re-runs the OverlayGenerator whenever a tenant's overlay design changes — the same scoped, checksum-driven pattern, applied to overlay intent. Branch scope Trigger rules are scoped to branches other than `main`. Generators run on branches where changes can be reviewed before merging — they do not fire on `main` directly. ### Full execution sequence[​](#full-execution-sequence "Direct link to Full execution sequence") Running the FabricGenerator for a single fabric triggers the following sequence: 1. **FabricGenerator** creates super spine devices and IP pools, then writes a checksum to each child Pod 2. Checksum change on each Pod fires the **PodGenerator** 3. **PodGenerator** validates the parent Fabric is complete, creates spine devices and spine-to-super-spine links, then writes a checksum to each child Rack 4. Checksum change on each Rack fires the **RackGenerator** 5. **RackGenerator** validates the parent Pod is complete, creates leaf devices and leaf-to-spine links One action at the fabric level produces the entire data center — devices, links, and IP addressing across all tiers. ### Overlay generation[​](#overlay-generation "Direct link to Overlay generation") The overlay is not part of the fabric cascade — it is triggered independently by a tenant design change. When a `NetworkTenant`'s checksum updates, the trigger fires the **OverlayGenerator** (`generate-tenant`), which allocates the tenant's overlay identifiers and materializes which leaf switches carry which segments. It relies on the fabric build having already run: the three physical Generators were extended to support the overlay by allocating the per-fabric overlay **ASN**, stamping each device's ASN and **route-reflector role**, giving every leaf a dedicated **VTEP loopback**, and materializing the iBGP **EVPN sessions** along the cabling. The OverlayGenerator then only has to layer tenant services onto that control-plane baseline. See [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md) for the full flow. ## Validation gates[​](#validation-gates "Direct link to Validation gates") Each Generator validates that the parent tier completed before it runs. If validation fails, the Generator raises an error and aborts, preventing partial or inconsistent infrastructure from being generated. ### PodGenerator validation[​](#podgenerator-validation "Direct link to PodGenerator validation") Before creating spines, the PodGenerator checks: 1. **Role exclusion** — Pods with role `fabric` represent the super spine tier and are skipped 2. **Parent completeness** — the number of super spine devices in the fabric must match the fabric's `amount_of_super_spines`. If the FabricGenerator created only 4 of 6 expected super spines (for example, due to a failure), the PodGenerator refuses to run 3. **Template defined** — the Pod must have a `spine_switch_template` assigned ### RackGenerator validation[​](#rackgenerator-validation "Direct link to RackGenerator validation") Before creating leafs, the RackGenerator checks: 1. **Parent completeness** — the number of spine devices in the pod must match the pod's `amount_of_spines`. If the PodGenerator has not finished, the RackGenerator refuses to run 2. **Template defined** — the Rack must have a `leaf_switch_template` assigned These gates make the modular setup self-healing in practice: if a Generator fails partway through, the checksum still changes for child objects whose parent data was modified. The child Generators fire, detect that the parent is incomplete, and abort. When the failed Generator is re-run and completes, child checksums update again and execution resumes. ## Deterministic interface allocation[​](#deterministic-interface-allocation "Direct link to Deterministic interface allocation") Cabling — connecting spines to super spines, and leafs to spines — requires allocating specific interfaces on each device. The allocation must be **stable**: the same rack must always land on the same spine interfaces, regardless of what happens to other racks. ![Deterministic interface allocation](/assets/images/interface-allocation-ae79414d6cf69634a7b547edd6dbd6d2.svg) ### Why stability matters[​](#why-stability-matters "Direct link to Why stability matters") Consider a pod with 4 spines and 4 racks, fully cabled. Rack-3 is decommissioned — its leaf switches and links are removed. The spine interfaces that were connected to Rack-3 are now free, sitting in the middle of each spine's used range. If the Generator allocated interfaces by filling the next available slot, re-running it for the remaining racks would reassign them to a denser packing — shifting Rack-4's leafs onto the interfaces that Rack-3 previously occupied. In the data model, the cabling would look correct. In the physical world, those cables are already plugged in. The generated design would no longer match the actual infrastructure, and operators would need to re-cable interfaces to restore alignment. The AI/DC solution avoids this by tying interface selection to the rack's `index` attribute, not to the set of racks that currently exist. Rack-4 always connects to the same spine interfaces whether Rack-3 is present or not. Decommissioning a rack leaves a gap in the interface range — which is exactly what the physical fabric looks like. ### How it works[​](#how-it-works "Direct link to How it works") The allocation follows three steps: 1. **Collect** — fetch all interfaces of a given role (for example, all super-spine-facing interfaces on spine devices) 2. **Sort** — sort each device's interfaces by name using `netutils.interface.sort_interface_list`, producing a stable ordering (Ethernet1, Ethernet2, ... Ethernet10, Ethernet11 — not lexicographic) 3. **Index** — use the device's position (pod index or rack index) to calculate which slot in the sorted list each cable connects to Because the index is derived from the design object (`NetworkPod.index`, `LocationRack.index`), the same design always produces the same cabling. Re-running a Generator selects the same interfaces every time. ### Pod cabling: spine-to-super-spine[​](#pod-cabling-spine-to-super-spine "Direct link to Pod cabling: spine-to-super-spine") Each spine connects to every super spine. The pod's `index` attribute determines which slot on each super spine the connections use: * Pod with index 2 → spines connect starting at super spine interface slot 0 * Pod with index 3 → spines connect starting at a higher offset Within a pod, each spine occupies one slot per super spine. The offset formula ensures that pods do not compete for the same interfaces — each pod owns a contiguous range of slots on every super spine device. ### Rack cabling: leaf-to-spine[​](#rack-cabling-leaf-to-spine "Direct link to Rack cabling: leaf-to-spine") Each leaf connects to every spine in the pod. The rack's `index` attribute determines which slot on each spine the connections use: * Rack with index 1 → leafs connect to spine interface slots 0–1 (for a 2-leaf rack) * Rack with index 2 → leafs connect to spine interface slots 2–3 Within a rack, leaf 1 uses the first slot in the range and leaf 2 uses the second. Racks with a single leaf use only one slot. ### Configurable sorting direction[​](#configurable-sorting-direction "Direct link to Configurable sorting direction") The sorting direction is not hardcoded. Operators can choose between ascending (`create_sorted_device_interface_map`) and reverse (`create_reverse_sorted_device_interface_map`) per fabric level: | Schema attribute | Controls | | ----------------------------------------------- | ------------------------------------------- | | `NetworkFabric.fabric_interface_sorting_method` | Super spine interface ordering | | `NetworkFabric.spine_interface_sorting_method` | Spine interface ordering (for pod cabling) | | `NetworkPod.leaf_interface_sorting_method` | Leaf interface ordering (for rack cabling) | | `NetworkPod.spine_interface_sorting_method` | Spine interface ordering (for rack cabling) | This allows different cabling patterns on different fabrics — for example, one fabric might allocate interfaces top-down while another allocates bottom-up. ## Idempotency[​](#idempotency "Direct link to Idempotency") Every layer of the system is designed for safe re-runs: | Operation | Mechanism | | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | Device creation | `allow_upsert=True` — creating a device with an existing name updates rather than duplicates | | Link creation | `allow_upsert=True` — creating a link with existing endpoints is a no-op | | Interface allocation | Index-based — same design produces the same cabling plan every time | | IP allocation | Deterministic identifiers — the resource allocator key is derived from the interface pair IDs, so the same pair always receives the same prefix | | Checksum propagation | Deterministic hash — if nothing changed, the checksum stays the same and no downstream triggers fire | If a Generator fails mid-run, the already-created objects remain. Re-running the Generator picks up where it left off: existing objects are upserted (no duplicates), and new objects are created. The validation gates ensure that downstream Generators do not run until the failed tier completes. ## Learn more[​](#learn-more "Direct link to Learn more") * [Design-driven automation](/infrahub-solution-ai-dc/solution-ai-dc/design-driven-automation.md) — what the Generators read and produce * [Generator patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md) — implementation-level code patterns * [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md) — how the OverlayGenerator layers tenant services on the fabric * [Infrahub documentation: Generators](https://docs.infrahub.app/topics/generator) * [Infrahub documentation: Resource Manager](https://docs.infrahub.app/topics/resource-manager) --- # Multivendor configuration The AI/DC solution is truly multivendor: the same design model produces **Cisco, Arista, Dell, and Juniper Junos** device configuration. A single set of Generators builds every fabric, and each device's startup configuration is rendered from a template owned by that device's vendor — not one template applied to every make. This page explains how a device's vendor is resolved, how per-vendor rendering is wired, and how the demo data is organized. For the fabric build itself, see [Design-driven automation](/infrahub-solution-ai-dc/solution-ai-dc/design-driven-automation.md) and [Modular Generator architecture](/infrahub-solution-ai-dc/solution-ai-dc/modular-generator-architecture.md). ## What it does[​](#what-it-does "Direct link to What it does") * **One design, per-vendor output.** Operators define fabrics, pods, and racks the same way regardless of vendor. When the Generators run, each device is placed into its manufacturer's device group and rendered from that vendor's configuration template. * **One Generator per layer, not per vendor.** The Generators are not forked per vendor. `generate-fabric`, `generate-pod`, and `generate-rack` each resolve a device's vendor from its template and route it to the right group — so adding a vendor does not mean adding Generators. * **Single-vendor fabrics in the demo.** The shipped dataset provides four clean single-vendor fabrics so you can exercise the solution on any of the four platforms. ## Vendor resolution[​](#vendor-resolution "Direct link to Vendor resolution") Every device is built from an object template that carries a `device_type`, and every device type carries a `manufacturer`. Vendor resolution walks that chain: ``` TemplateNetworkDevice.device_type → NetworkDeviceType.manufacturer → OrganizationManufacturer.name ``` The manufacturer name is mapped to a vendor device group by `src/infrahub_solution_ai_dc/vendors.py`: * `SUPPORTED_VENDORS = ("cisco", "arista", "dell", "juniper")`. * `vendor_group_for_manufacturer(name, ...)` returns `"{vendor}_devices"` for a supported manufacturer. * `vendor_group_for_template(client, template_id)` resolves the whole chain for a device template. Because every device built from a template shares that template's manufacturer, each Generator resolves the vendor group **once per run** and stamps it onto every device it creates via `member_of_groups=["devices", "_devices"]`. Each device therefore belongs to the parent `devices` group **and** exactly one vendor group. Fail-loud resolution Resolution is deliberately fail-loud: a device whose manufacturer is missing or unsupported raises a `ValueError` that names the offending device. There is no silent skip and no default configuration — a data gap surfaces immediately rather than producing a device with no (or wrong) configuration. ## Device groups[​](#device-groups "Direct link to Device groups") `objects/01_groups.yml` defines four vendor groups as children of the existing `devices` group: | Group | Parent | Members | | ----------------- | --------- | ------------------------------------------ | | `cisco_devices` | `devices` | Every device whose manufacturer is Cisco | | `arista_devices` | `devices` | Every device whose manufacturer is Arista | | `dell_devices` | `devices` | Every device whose manufacturer is Dell | | `juniper_devices` | `devices` | Every device whose manufacturer is Juniper | Group membership is what drives per-vendor artifact targeting — the artifact definitions target these groups directly. ## Per-vendor templates and wiring[​](#per-vendor-templates-and-wiring "Direct link to Per-vendor templates and wiring") The single `startup_config.j2` template was replaced by per-vendor templates, one per supported vendor. They all share one GraphQL query (`transforms/startup_config.gql`); `.infrahub.yml` wires a transform and an artifact definition per vendor: | Vendor | Template | Jinja2 transform | Artifact definition (targets) | | ------- | ------------------------------------------------ | ------------------------------- | --------------------------------------------------- | | Cisco | `transforms/templates/startup_config_cisco.j2` | `cisco_device_startup_config` | `cisco_startup_configuration` (`cisco_devices`) | | Arista | `transforms/templates/startup_config_arista.j2` | `arista_device_startup_config` | `arista_startup_configuration` (`arista_devices`) | | Dell | `transforms/templates/startup_config_dell.j2` | `dell_device_startup_config` | `dell_startup_configuration` (`dell_devices`) | | Juniper | `transforms/templates/startup_config_juniper.j2` | `juniper_device_startup_config` | `juniper_startup_configuration` (`juniper_devices`) | Because no artifact definition targets the parent `devices` group, each device renders **exactly one** startup configuration artifact — the one for its vendor. Every artifact uses the same `artifact_name` (`Startup configuration`), so the artifact reads consistently across vendors regardless of which template produced it. ## Demo data[​](#demo-data "Direct link to Demo data") The dataset is organized as four single-vendor fabrics: | Fabric | Vendor | Example switch templates | | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | | Fabric-A | Cisco | `cisco-9364d-gx2-super-spine-switch`, `cisco-9364d-gx2-spine-switch`, `cisco-93400ld-h1-leaf-switch-compute` | | Fabric-B | Arista | `arista-7060dx5-64s-super-spine-switch`, `arista-7060dx5-64s-spine-switch` | | Fabric-C | Dell | `dell-z9864f-on-super-spine-switch`, `dell-z9864f-on-spine-switch` | | Fabric-D | Juniper | `juniper-qfx5230-64cd-super-spine-switch`, `juniper-qfx5230-64cd-spine-switch`, `juniper-qfx5120-48y-8c-leaf-switch-compute` | Supporting data: * `objects/02_manufacturer.yml` — Cisco, Arista, Dell, and Juniper manufacturers. * `objects/12_overlay.yml` — overlay tenants. `Blue` is scoped to Fabric-A and `Green` to Fabric-D, so both the Cisco and Juniper fabrics render tenant overlay configuration. * `objects/03_device_type.yml` — device types, each declaring a manufacturer. No vendor-less device types remain. * `objects/06_device_template.yml` — per-vendor device templates for each device role (including per-vendor compute and storage leaf templates). ## How to try it[​](#how-to-try-it "Direct link to How to try it") Because switching a fabric's vendor renames its interfaces, the multivendor dataset is meant to be applied to a **fresh stack** rather than in place: ``` uv sync --all-packages uv run inv destroy && uv run inv start uv run inv load ``` Then build a fabric (via the trigger pipeline, or by running `generate-fabric` for a target fabric — see the [Demo Guide](/infrahub-solution-ai-dc/solution-ai-dc/demo-guide.md)). To verify: * Each device is a member of `devices` and exactly one `{vendor}_devices` group. * Fabric-A is 100% Cisco, Fabric-B 100% Arista, Fabric-C 100% Dell, Fabric-D 100% Juniper. * Each device yields exactly one `Startup configuration` artifact, rendered from its vendor's template. * Editing one vendor's template changes only that vendor's device configs; the others are byte-identical. The Cisco, Arista and Dell templates started near-identical (NX-OS-style) and are diverged toward correct per-vendor syntax over time. The Juniper template is structurally different: Junos configuration is hierarchical (curly-brace) rather than flat, and it models both loopbacks as units of a single `lo0`. Editing one template in isolation is safe precisely because each vendor's devices render only from their own template. ## Learn more[​](#learn-more "Direct link to Learn more") * [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md) — the overlay configuration is also rendered per vendor. * [Generator patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md) — how the Generators resolve and stamp the vendor group. * The feature [specification](https://github.com/opsmill/infrahub-solution-ai-dc/blob/main/specs/002-multivendor-config/spec.md) and [quickstart](https://github.com/opsmill/infrahub-solution-ai-dc/blob/main/specs/002-multivendor-config/quickstart.md). --- # AI/DC solution The AI/DC solution is a reference implementation demonstrating how to use Infrahub to automate the full lifecycle of a large-scale AI data center. It takes a small number of design inputs — fabric topology, pod configuration, rack assignments — and generates a complete data center: devices, IP allocations, cabling plans, and configuration data. Four modular Generators connect automatically via event-driven triggers: three build the physical fabric (Fabric → Pod → Rack), and a fourth layers multi-tenant EVPN/VXLAN overlay services on top. Triggering one Generator at the fabric level causes the entire data center to build itself, and the same design renders per-vendor (Cisco, Arista, Dell, Juniper) device configuration. The solution is fully functional today and is positioned as a demo and reference implementation. All of the content provided in this solution can be found on the [`infrahub-solution-ai-dc` GitHub repository](https://github.com/opsmill/infrahub-solution-ai-dc). ## The problem this solves[​](#the-problem-this-solves "Direct link to The problem this solves") * Large-scale data center operators must deploy standardized environments rapidly, repeatedly, and without deviation — while extending those environments over time without disrupting what is already running. * The capital cost of AI hardware means time-to-production directly affects ROI. Inconsistency between environments introduces risk at every layer. * Traditional scripted automation discards design intent after the run. Day-two operations require rebuilding from scratch. There is no parallelism, and no stored relationship between the configuration produced and the design that produced it. * Infrahub's design-driven approach solves this: operators define what infrastructure should look like, and Generators produce it — consistently, in parallel, and with surgical day-two change support. * The pattern is not AI-specific. It applies to any standardized, layered infrastructure domain. ## What the AI/DC solution is[​](#what-the-aidc-solution-is "Direct link to What the AI/DC solution is") The AI/DC solution builds a 5-stage Clos data center fabric from minimal design inputs. The hierarchy follows a three-level structure: * **Fabric** — the top level, containing super spine switches * **Pod** — the middle level, containing spine switches connected to the super spines above * **Rack** — the bottom level, containing leaf switches connected to the spines above Three Generators each own one layer of this hierarchy and connect automatically via checksum-triggered events: 1. **FabricGenerator** allocates IP pools and creates super spine switches, then signals child Pods to run. 2. **PodGenerator** creates spine switches, connects them to super spines, and signals child Racks to run. 3. **RackGenerator** creates leaf switches and connects them to spines. Trigger once at the fabric level — the entire data center builds itself. On top of this physical fabric, a fourth Generator — the **OverlayGenerator** (registered as `generate-tenant`) — turns tenant intent into multi-tenant EVPN/VXLAN services. See [EVPN/VXLAN overlay](/infrahub-solution-ai-dc/solution-ai-dc/evpn-vxlan-overlay.md). The demo data includes three complete, single-vendor fabrics: * **Fabric-A** — Cisco equipment * **Fabric-B** — Arista equipment * **Fabric-C** — Dell equipment * **Fabric-D** — Juniper equipment Each fabric is built entirely from one vendor's device templates, and each device's configuration is rendered from that vendor's template. See [Multivendor configuration](/infrahub-solution-ai-dc/solution-ai-dc/multivendor-config.md). The solution generates IP pool allocations, interface-level cabling plans, OSPF underlay configuration, EVPN/VXLAN overlay configuration, and computed interface descriptions. ## Who this is for[​](#who-this-is-for "Direct link to Who this is for") Choose your path **Running a demo or evaluating Infrahub?** Start with the [Demo Guide](/infrahub-solution-ai-dc/solution-ai-dc/demo-guide.md) — it walks through the solution end to end with no code modifications required. **Building your own Generators or adapting the patterns?** Start with the [Reference Guide: Generator Patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md) — it covers the implementation patterns in detail. ### Persona 1 — Evaluator / Learner[​](#persona-1--evaluator--learner "Direct link to Persona 1 — Evaluator / Learner") You want to see design-driven automation in action. You do not need to modify code. Your path: run the demo, understand what happened, and connect it to your own use case. → [Demo Guide](/infrahub-solution-ai-dc/solution-ai-dc/demo-guide.md) ### Persona 2 — Advanced Implementer[​](#persona-2--advanced-implementer "Direct link to Persona 2 — Advanced Implementer") You are already working with Infrahub. You want a production-quality reference for modular Generator patterns, checksum-triggered cascades, and IP space delegation. You will read and adapt the code. → [Reference Guide: Generator Patterns](/infrahub-solution-ai-dc/solution-ai-dc/generator-patterns.md) **Broader audience:** The patterns demonstrated here apply to any standardized, layered infrastructure at scale — edge deployments, service provider networks, enterprise campuses. The Fabric → Pod → Rack hierarchy is illustrative, not prescriptive. ## What's included[​](#whats-included "Direct link to What's included") ### Schema[​](#schema "Direct link to Schema") Seven schema files define the data model: | File | Contents | | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------- | | `logical_design.yml` | NetworkFabric and NetworkPod — the design hierarchy with Generator signaling attributes and the fabric overlay ASN | | `physical_location.yml` | LocationHall and LocationRack — physical locations with Generator target support | | `device.yml` | NetworkDevice, NetworkInterface, and NetworkLink — devices with computed attributes, ASN, route-reflector role, and VTEP loopback | | `ipam.yml` | IpamIPPrefix with role-based allocation and IpamIPAddress | | `overlay.yml` | NetworkTenant, NetworkVrf, and NetworkSegment — the overlay intent model | | `routing.yml` | NetworkBGPSession — the stored iBGP EVPN control plane | | `generator.yml` | GeneratorTarget generic with checksum attribute enabling trigger-based cascades | ### Generators[​](#generators "Direct link to Generators") Four Generators — three own one layer of the physical hierarchy, and one materializes the overlay: | Generator | Responsibility | | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------- | | FabricGenerator | IP pool and overlay ASN allocation, super spine switch creation | | PodGenerator | Spine switch creation, spine-to-super-spine cabling, iBGP EVPN sessions | | RackGenerator | Leaf switch creation, leaf-to-spine cabling, VTEP loopbacks | | OverlayGenerator (`generate-tenant`) | Allocates overlay identifiers (VNI/VLAN/ASN/route target), places segments on leaf switches, materializes tenant/VRF/segment state | ### Transforms and artifacts[​](#transforms-and-artifacts "Direct link to Transforms and artifacts") * **Startup configuration** — four per-vendor Jinja2 templates (Cisco, Arista, Dell, Juniper), each producing a text/plain artifact for the devices in its vendor group * **Cabling plan** — Python Transformation producing a CSV artifact per fabric * **Computed interface descriptions** — Python Transformation applied as a computed attribute on each interface ### Demo data[​](#demo-data "Direct link to Demo data") * Four complete single-vendor fabrics (Cisco, Arista, Dell, Juniper) with pods, racks, device templates, and IP pools * A seed overlay tenant (`Blue` on Fabric-A) with VRFs and segments * Manufacturer and device type definitions * Overlay resource pools (ASN, L2VNI, L3VNI, VLAN) * Event trigger rules (`CoreNodeTriggerRule` and `CoreGeneratorAction`) for automatic modular Generator execution ### Tooling[​](#tooling "Direct link to Tooling") * Docker Compose environment for local development * Invoke tasks for setup, loading, and testing * Python package management via `uv` → [Installation & Setup](/infrahub-solution-ai-dc/solution-ai-dc/installation-setup.md) → [GitHub repository](https://github.com/opsmill/infrahub-solution-ai-dc) --- # infrahubctl `infrahubctl` is a command line utility designed to help with the day to day management of an Infrahub installation. It's meant to run on any laptop or server and it communicates with a remote Infrahub server over the network. `infrahubctl` can help you to: * Manage the branches in Infrahub: List, Create, Merge, Rebase, Delete. * Manage the schema and load new schema files into Infrahub. * Browse and download schemas and collections from the Infrahub Marketplace. * Execute any Python script that requires access to the Python SDK. * Render a Jinja Template locally for troubleshooting. * Execute a GraphQL query store in a Git repository for troubleshooting. * Validate that input files conform with the format expected by Infrahub. ## Installation[​](#installation "Direct link to Installation") The `infrahubctl` command line utility is installed as a part of the [Infrahub SDK for Python](/python-sdk/guides/installation.md#ctl). ## Configuration[​](#configuration "Direct link to Configuration") `infrahubctl` requires a minimum configuration in order to connect to the right Infrahub server with the correct credentials. These settings can be provided either in a configuration file, `infrahubctl.toml`, or via environment variables. ### Environment variables[​](#environment-variables "Direct link to Environment variables") | Name | Example value | | ------------------------- | -------------------------------------- | | `INFRAHUB_ADDRESS` | | | `INFRAHUB_API_TOKEN` | `06438eb2-8019-4776-878c-0941b1f1d1ec` | | `INFRAHUB_DEFAULT_BRANCH` | main | > You can also provide the location of a configuration file via the environment variable `INFRAHUBCTL_CONFIG`. ### `infrahubctl.toml` file[​](#infrahubctltoml-file "Direct link to infrahubctltoml-file") infrahubctl.toml ``` server_address="http://localhost:8000" api_token="06438eb2-8019-4776-878c-0941b1f1d1ec" ``` --- # `infrahubctl branch` Manage the branches in a remote Infrahub instance. List, create, merge, rebase .. **Usage**: ``` $ infrahubctl branch [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `list`: List all existing branches. * `create`: Create a new branch. * `delete`: Delete a branch. * `rebase`: Rebase a Branch with main. * `merge`: Merge a Branch with main. * `validate`: Validate if a branch has some conflict and... * `report`: Generate branch cleanup status report. ## `infrahubctl branch list`[​](#infrahubctl-branch-list "Direct link to infrahubctl-branch-list") List all existing branches. **Usage**: ``` $ infrahubctl branch list [OPTIONS] ``` **Options**: * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl branch create`[​](#infrahubctl-branch-create "Direct link to infrahubctl-branch-create") Create a new branch. **Usage**: ``` $ infrahubctl branch create [OPTIONS] BRANCH_NAME ``` **Arguments**: * `BRANCH_NAME`: Name of the branch to create \[required] **Options**: * `--description TEXT`: Description of the branch * `--sync-with-git / --no-sync-with-git`: Extend the branch to Git and have Infrahub create the branch in connected repositories. \[default: no-sync-with-git] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl branch delete`[​](#infrahubctl-branch-delete "Direct link to infrahubctl-branch-delete") Delete a branch. **Usage**: ``` $ infrahubctl branch delete [OPTIONS] BRANCH_NAME ``` **Arguments**: * `BRANCH_NAME`: \[required] **Options**: * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl branch rebase`[​](#infrahubctl-branch-rebase "Direct link to infrahubctl-branch-rebase") Rebase a Branch with main. **Usage**: ``` $ infrahubctl branch rebase [OPTIONS] BRANCH_NAME ``` **Arguments**: * `BRANCH_NAME`: \[required] **Options**: * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl branch merge`[​](#infrahubctl-branch-merge "Direct link to infrahubctl-branch-merge") Merge a Branch with main. **Usage**: ``` $ infrahubctl branch merge [OPTIONS] BRANCH_NAME ``` **Arguments**: * `BRANCH_NAME`: \[required] **Options**: * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl branch validate`[​](#infrahubctl-branch-validate "Direct link to infrahubctl-branch-validate") Validate if a branch has some conflict and is passing all the tests (NOT IMPLEMENTED YET). **Usage**: ``` $ infrahubctl branch validate [OPTIONS] BRANCH_NAME ``` **Arguments**: * `BRANCH_NAME`: \[required] **Options**: * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl branch report`[​](#infrahubctl-branch-report "Direct link to infrahubctl-branch-report") Generate branch cleanup status report. **Usage**: ``` $ infrahubctl branch report [OPTIONS] BRANCH_NAME ``` **Arguments**: * `BRANCH_NAME`: Branch name to generate report for \[required] **Options**: * `--update-diff`: Update diff before generating report * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. --- # `infrahubctl check` Execute user-defined checks. **Usage**: ``` $ infrahubctl check [OPTIONS] [CHECK_NAME] [VARIABLES]... ``` **Arguments**: * `[CHECK_NAME]`: Name of the Python check * `[VARIABLES]...`: Variables to pass along with the query. Format key=value key=value. **Options**: * `--branch TEXT` * `--path TEXT`: Root directory \[default: .] * `--debug / --no-debug`: \[default: no-debug] * `--format-json / --no-format-json`: \[default: no-format-json] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--list`: Show available Python checks * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # `infrahubctl dump` Export nodes and their relationships out of the database. **Usage**: ``` $ infrahubctl dump [OPTIONS] ``` **Options**: * `--namespace TEXT`: Namespace(s) to export * `--directory PATH`: Directory path to store export \[default: (dynamic)] * `--quiet / --no-quiet`: No console output \[default: no-quiet] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--branch TEXT`: Branch from which to export * `--concurrent INTEGER`: Maximum number of requests to execute at the same time. \[env var: INFRAHUB\_MAX\_CONCURRENT\_EXECUTION; default: 4] * `--timeout INTEGER`: Timeout in sec \[env var: INFRAHUB\_TIMEOUT; default: 60] * `--exclude TEXT`: Prevent node kind(s) from being exported, CoreAccount is excluded by default \[default: CoreAccount] * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # `infrahubctl generator` Run a generator script. **Usage**: ``` $ infrahubctl generator [OPTIONS] [GENERATOR_NAME] [VARIABLES]... ``` **Arguments**: * `[GENERATOR_NAME]`: Name of the Generator * `[VARIABLES]...`: Variables to pass along with the query. Format key=value key=value. **Options**: * `--branch TEXT` * `--path TEXT`: Root directory \[default: .] * `--debug / --no-debug`: \[default: no-debug] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--list`: Show available Generators * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # `infrahubctl graphql` Various GraphQL related commands. **Usage**: ``` $ infrahubctl graphql [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `query-report`: Report how Infrahub will interpret a... * `export-schema`: Export the GraphQL schema to a file. * `generate-return-types`: Create Pydantic Models for GraphQL query... ## `infrahubctl graphql query-report`[​](#infrahubctl-graphql-query-report "Direct link to infrahubctl-graphql-query-report") Report how Infrahub will interpret a GraphQL query. Reports whether the query targets unique nodes, meaning every query it contains returns a single object. Artifact and generator definitions use this to decide how much to regenerate when data changes: when it is true, only the artifacts or generator instances of the objects that changed are regenerated, and when it is false, all of them are. A query returns a single object when it filters on ids or hfid, or on every part of one uniqueness constraint of the model, and the values it filters on are always provided. **Usage**: ``` $ infrahubctl graphql query-report [OPTIONS] NAME ``` **Arguments**: * `NAME`: Name of the GraphQL query to analyze, as declared under queries in .infrahub.yml. \[required] **Options**: * `--online`: Fetch the query from the Infrahub server (CoreGraphQLQuery by name) instead of reading it from the local .infrahub.yml file. * `--branch TEXT`: Branch on which to run the report. Uniqueness constraints come from the schema, so the result can differ between branches. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl graphql export-schema`[​](#infrahubctl-graphql-export-schema "Direct link to infrahubctl-graphql-export-schema") Export the GraphQL schema to a file. **Usage**: ``` $ infrahubctl graphql export-schema [OPTIONS] ``` **Options**: * `--destination PATH`: Path to the GraphQL schema file. \[default: schema.graphql] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl graphql generate-return-types`[​](#infrahubctl-graphql-generate-return-types "Direct link to infrahubctl-graphql-generate-return-types") Create Pydantic Models for GraphQL query return types. **Usage**: ``` $ infrahubctl graphql generate-return-types [OPTIONS] [QUERY] ``` **Arguments**: * `[QUERY]`: Location of the GraphQL query file(s). Defaults to current directory if not specified. **Options**: * `--schema PATH`: Path to the GraphQL schema file. \[default: schema.graphql] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. --- # `infrahubctl info` Display the status of the Python SDK. **Usage**: ``` $ infrahubctl info [OPTIONS] ``` **Options**: * `--detail / --no-detail`: Display detailed information. \[default: no-detail] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # `infrahubctl load` Import nodes and their relationships into the database. **Usage**: ``` $ infrahubctl load [OPTIONS] ``` **Options**: * `--directory PATH`: Directory path of exported data \[default: (dynamic)] * `--continue-on-error / --no-continue-on-error`: Allow exceptions during loading and display them when complete \[default: no-continue-on-error] * `--quiet / --no-quiet`: No console output \[default: no-quiet] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--branch TEXT`: Branch from which to export * `--concurrent INTEGER`: Maximum number of requests to execute at the same time. \[env var: INFRAHUB\_MAX\_CONCURRENT\_EXECUTION] * `--timeout INTEGER`: Timeout in sec \[env var: INFRAHUB\_TIMEOUT; default: 60] * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # `infrahubctl marketplace` Browse and download schemas from the Infrahub Marketplace. **Usage**: ``` $ infrahubctl marketplace [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `show`: Show full details of a schema or... * `list`: List schemas (default) or collections... * `search`: Search the Infrahub Marketplace for... * `get`: Fetch a schema or collection from the... ## `infrahubctl marketplace show`[​](#infrahubctl-marketplace-show "Direct link to infrahubctl-marketplace-show") Show full details of a schema or collection from the Infrahub Marketplace. **Usage**: ``` $ infrahubctl marketplace show [OPTIONS] IDENTIFIER ``` **Arguments**: * `IDENTIFIER`: Schema or collection identifier in namespace/name format \[required] **Options**: * `-c, --collection`: Force collection lookup. Default: auto-detect whether the identifier is a schema or collection. * `--json`: Output raw JSON to stdout instead of a table. * `--marketplace-url TEXT`: Base URL of the Infrahub Marketplace. Overrides configuration and environment. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl marketplace list`[​](#infrahubctl-marketplace-list "Direct link to infrahubctl-marketplace-list") List schemas (default) or collections available on the Infrahub Marketplace. **Usage**: ``` $ infrahubctl marketplace list [OPTIONS] ``` **Options**: * `--collections`: List collections instead of schemas. * `-l, --limit INTEGER`: Maximum number of results to display. * `--json`: Output raw JSON to stdout instead of a table. * `--marketplace-url TEXT`: Base URL of the Infrahub Marketplace. Overrides configuration and environment. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl marketplace search`[​](#infrahubctl-marketplace-search "Direct link to infrahubctl-marketplace-search") Search the Infrahub Marketplace for schemas (default) or collections. **Usage**: ``` $ infrahubctl marketplace search [OPTIONS] TERM ``` **Arguments**: * `TERM`: Search term matched against name, display name, and description. \[required] **Options**: * `--collections`: Search collections instead of schemas. * `-l, --limit INTEGER`: Maximum number of results to display. * `--json`: Output raw JSON to stdout instead of a table. * `--marketplace-url TEXT`: Base URL of the Infrahub Marketplace. Overrides configuration and environment. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl marketplace get`[​](#infrahubctl-marketplace-get "Direct link to infrahubctl-marketplace-get") Fetch a schema or collection from the Infrahub Marketplace. By default, auto-detects whether `namespace/name` is a schema or a collection. Pass --collection to force the collection path when an identifier exists as both. **Usage**: ``` $ infrahubctl marketplace get [OPTIONS] IDENTIFIER ``` **Arguments**: * `IDENTIFIER`: Schema or collection identifier in namespace/name format \[required] **Options**: * `-v, --version TEXT`: Specific schema version, for example 1.2.0. Default: latest published. * `-c, --collection`: Force collection download. Default: auto-detect whether the identifier is a schema or collection. * `--dependencies`: Also download the schemas this schema or collection depends on. * `-y, --yes`: Overwrite schemas that already exist in the output directory without prompting. * `-s, --stdout`: Print content to stdout instead of writing to disk. Status messages go to stderr. * `-o, --output-dir PATH`: Directory to save downloaded files. \[default: schemas] * `--marketplace-url TEXT`: Base URL of the Infrahub Marketplace. Overrides configuration and environment. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. --- # `infrahubctl menu` Manage the menu in a remote Infrahub instance. **Usage**: ``` $ infrahubctl menu [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `load`: Load one or multiple menu files into... * `validate`: Validate one or multiple menu files. ## `infrahubctl menu load`[​](#infrahubctl-menu-load "Direct link to infrahubctl-menu-load") Load one or multiple menu files into Infrahub. **Usage**: ``` $ infrahubctl menu load [OPTIONS] MENUS... ``` **Arguments**: * `MENUS...`: \[required] **Options**: * `--debug / --no-debug`: \[default: no-debug] * `--branch TEXT`: Branch on which to load the menu. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl menu validate`[​](#infrahubctl-menu-validate "Direct link to infrahubctl-menu-validate") Validate one or multiple menu files. **Usage**: ``` $ infrahubctl menu validate [OPTIONS] PATHS... ``` **Arguments**: * `PATHS...`: \[required] **Options**: * `--debug / --no-debug`: \[default: no-debug] * `--branch TEXT`: Branch on which to validate the objects. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. --- # `infrahubctl object` Manage objects in a remote Infrahub instance. **Usage**: ``` $ infrahubctl object [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `get`: Query and display Infrahub objects. * `create`: Create a new object in Infrahub. * `update`: Update an existing object in Infrahub. * `delete`: Delete an Infrahub object. * `load`: Load one or multiple objects files into... * `validate`: Validate one or multiple objects files. ## `infrahubctl object get`[​](#infrahubctl-object-get "Direct link to infrahubctl-object-get") Query and display Infrahub objects. When IDENTIFIER is omitted the command lists all objects of the given KIND. When IDENTIFIER is provided it displays a single object in detail view. Empty columns are hidden by default (use --all-columns). Examples: infrahubctl object get InfraDevice infrahubctl object get InfraDevice spine01 infrahubctl object get InfraDevice --filter name\_\_value=spine01 infrahubctl object get InfraDevice --output json infrahubctl object get InfraDevice --output yaml > backup.yml Exit codes: 0 = results found, 1 = error (including not found in detail mode), 80 = list query succeeded but returned zero objects. **Usage**: ``` $ infrahubctl object get [OPTIONS] KIND [IDENTIFIER] ``` **Arguments**: * `KIND`: Infrahub schema kind to query \[required] * `[IDENTIFIER]`: UUID, name, or HFID (use / for multi-part, for example: Cisco/NX-OS) **Options**: * `--filter TEXT`: Filter in attr\_\_value=x format * `-o, --output [table|json|csv|yaml]`: Output format * `-b, --branch TEXT`: Target branch * `--limit INTEGER`: Maximum results * `--offset INTEGER`: Skip first N results * `--all-columns`: Show all columns including empty ones * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl object create`[​](#infrahubctl-object-create "Direct link to infrahubctl-object-create") Create a new object in Infrahub. Provide field values with repeatable --set flags or supply a JSON/YAML object file via --file. The two modes are mutually exclusive. Examples: infrahubctl object create InfraDevice --set name=spine01 --set status=active infrahubctl object create InfraDevice --set name=spine01 --set location=DC1 infrahubctl object create InfraDevice --file devices.yml **Usage**: ``` $ infrahubctl object create [OPTIONS] KIND ``` **Arguments**: * `KIND`: Infrahub schema kind to create \[required] **Options**: * `--set TEXT`: Field value in key=value format * `-f, --file PATH`: JSON or YAML file with object data * `-b, --branch TEXT`: Target branch * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl object update`[​](#infrahubctl-object-update "Direct link to infrahubctl-object-update") Update an existing object in Infrahub. Fetches the object by KIND and IDENTIFIER, applies the requested changes, and saves back to the server. Use --set or --file. Examples: infrahubctl object update InfraDevice spine01 --set status=active infrahubctl object update InfraDevice spine01 --set location=DC1 infrahubctl object update InfraDevice spine01 --file updates.yml **Usage**: ``` $ infrahubctl object update [OPTIONS] KIND IDENTIFIER ``` **Arguments**: * `KIND`: Infrahub schema kind \[required] * `IDENTIFIER`: UUID, name, or HFID (use / for multi-part, for example: Cisco/NX-OS) \[required] **Options**: * `--set TEXT`: Field value in key=value format * `-f, --file PATH`: JSON or YAML file with update data * `-b, --branch TEXT`: Target branch * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl object delete`[​](#infrahubctl-object-delete "Direct link to infrahubctl-object-delete") Delete an Infrahub object. Fetches the object by KIND and IDENTIFIER, then deletes it. Unless --yes is provided, a confirmation prompt is shown first. Examples: infrahubctl object delete InfraDevice spine01 infrahubctl object delete InfraDevice spine01 --yes **Usage**: ``` $ infrahubctl object delete [OPTIONS] KIND IDENTIFIER ``` **Arguments**: * `KIND`: Infrahub schema kind \[required] * `IDENTIFIER`: UUID, name, or HFID (use / for multi-part, for example: Cisco/NX-OS) \[required] **Options**: * `-y, --yes`: Skip confirmation prompt * `-b, --branch TEXT`: Target branch * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl object load`[​](#infrahubctl-object-load "Direct link to infrahubctl-object-load") Load one or multiple objects files into Infrahub. **Usage**: ``` $ infrahubctl object load [OPTIONS] PATHS... ``` **Arguments**: * `PATHS...`: \[required] **Options**: * `--debug / --no-debug`: \[default: no-debug] * `--branch TEXT`: Branch on which to load the objects. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl object validate`[​](#infrahubctl-object-validate "Direct link to infrahubctl-object-validate") Validate one or multiple objects files. **Usage**: ``` $ infrahubctl object validate [OPTIONS] PATHS... ``` **Arguments**: * `PATHS...`: \[required] **Options**: * `--debug / --no-debug`: \[default: no-debug] * `--branch TEXT`: Branch on which to validate the objects. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. --- # `infrahubctl protocols` Export Python protocols corresponding to a schema. **Usage**: ``` $ infrahubctl protocols [OPTIONS] ``` **Options**: * `--schemas PATH`: List of schemas or directory to load. * `--branch TEXT`: Branch of schema to export Python protocols for. * `--sync / --no-sync`: Generate for sync or async. \[default: no-sync] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--out TEXT`: Path to a file to save the result. \[default: schema\_protocols.py] * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # `infrahubctl render` Render a local Jinja2 Transform for debugging purpose. **Usage**: ``` $ infrahubctl render [OPTIONS] [TRANSFORM_NAME] [VARIABLES]... ``` **Arguments**: * `[TRANSFORM_NAME]`: Name of the Python transformation * `[VARIABLES]...`: Variables to pass along with the query. Format key=value key=value. **Options**: * `--branch TEXT`: Branch on which to render the transform. * `--debug / --no-debug`: \[default: no-debug] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--list`: Show available transforms * `--out TEXT`: Path to a file to save the result. * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # `infrahubctl repository` Manage the repositories in a remote Infrahub instance. List, create, delete .. **Usage**: ``` $ infrahubctl repository [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `add`: Add a new repository. * `list` * `init`: Initialize a new Infrahub repository. ## `infrahubctl repository add`[​](#infrahubctl-repository-add "Direct link to infrahubctl-repository-add") Add a new repository. **Usage**: ``` $ infrahubctl repository add [OPTIONS] NAME LOCATION ``` **Arguments**: * `NAME`: \[required] * `LOCATION`: \[required] **Options**: * `--description TEXT` * `--username TEXT` * `--password TEXT` * `--ref TEXT` * `--read-only / --no-read-only`: \[default: no-read-only] * `--debug / --no-debug`: \[default: no-debug] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl repository list`[​](#infrahubctl-repository-list "Direct link to infrahubctl-repository-list") **Usage**: ``` $ infrahubctl repository list [OPTIONS] ``` **Options**: * `--branch TEXT`: Branch on which to list repositories. * `--debug / --no-debug`: \[default: no-debug] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl repository init`[​](#infrahubctl-repository-init "Direct link to infrahubctl-repository-init") Initialize a new Infrahub repository. **Usage**: ``` $ infrahubctl repository init [OPTIONS] ``` **Options**: * `--help`: Show this message and exit. --- # `infrahubctl run` Execute a script. **Usage**: ``` $ infrahubctl run [OPTIONS] SCRIPT [VARIABLES]... ``` **Arguments**: * `SCRIPT`: \[required] * `[VARIABLES]...`: Variables to pass along with the query. Format key=value key=value. **Options**: * `--method TEXT`: \[default: run] * `--debug / --no-debug`: \[default: no-debug] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--branch TEXT`: Branch on which to run the script. * `--concurrent INTEGER`: Maximum number of requests to execute at the same time. \[env var: INFRAHUB\_MAX\_CONCURRENT\_EXECUTION] * `--timeout INTEGER`: Timeout in sec \[env var: INFRAHUB\_TIMEOUT; default: 60] * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # `infrahubctl schema` Manage the schema in a remote Infrahub instance. **Usage**: ``` $ infrahubctl schema [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `load`: Load one or multiple schema files into Infrahub. * `check`: Check if schema files are valid and their impact on Infrahub. * `export`: Export the schema from Infrahub as YAML... * `list`: List all available schema kinds. * `show`: Show details for a specific schema kind. * `format`: Format Infrahub schema files with a... ## `infrahubctl schema load`[​](#infrahubctl-schema-load "Direct link to infrahubctl-schema-load") Load one or multiple schema files into Infrahub. **Usage**: ``` $ infrahubctl schema load [OPTIONS] SCHEMAS... ``` **Arguments**: * `SCHEMAS...`: \[required] **Options**: * `--debug / --no-debug`: \[default: no-debug] * `--branch TEXT`: Branch on which to load the schema. * `--wait INTEGER`: Time in seconds to wait until the schema has converged across all workers \[default: 0] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl schema check`[​](#infrahubctl-schema-check "Direct link to infrahubctl-schema-check") Check if schema files are valid and what would be the impact of loading them with Infrahub. **Usage**: ``` $ infrahubctl schema check [OPTIONS] SCHEMAS... ``` **Arguments**: * `SCHEMAS...`: \[required] **Options**: * `--debug / --no-debug`: \[default: no-debug] * `--branch TEXT`: Branch on which to check the schema. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl schema export`[​](#infrahubctl-schema-export "Direct link to infrahubctl-schema-export") Export the schema from Infrahub as YAML files, one per namespace. **Usage**: ``` $ infrahubctl schema export [OPTIONS] ``` **Options**: * `--directory PATH`: Directory path to store schema files \[default: (dynamic)] * `--branch TEXT`: Branch from which to export the schema * `--namespaces TEXT`: Namespace(s) to export (default: all user-defined) * `--debug / --no-debug`: \[default: no-debug] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl schema list`[​](#infrahubctl-schema-list "Direct link to infrahubctl-schema-list") List all available schema kinds. Displays a table of all node schema entries. Use --filter to narrow results by a case-insensitive match on the kind name. Examples: infrahubctl schema list infrahubctl schema list --filter Device **Usage**: ``` $ infrahubctl schema list [OPTIONS] ``` **Options**: * `--filter TEXT`: Filter kinds by name * `-b, --branch TEXT`: Target branch * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl schema show`[​](#infrahubctl-schema-show "Direct link to infrahubctl-schema-show") Show details for a specific schema kind. Displays metadata, attributes, and relationships for the requested schema kind in a human-readable format. Examples: infrahubctl schema show InfraDevice **Usage**: ``` $ infrahubctl schema show [OPTIONS] KIND ``` **Arguments**: * `KIND`: Schema kind to display \[required] **Options**: * `-b, --branch TEXT`: Target branch * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl schema format`[​](#infrahubctl-schema-format "Direct link to infrahubctl-schema-format") Format Infrahub schema files with a canonical key ordering. Reorders the keys within each node, generic, attribute, relationship and dropdown choice into a consistent, opinionated order so schema files read the same way and produce small diffs. Only your own nodes are formatted; nodes in Infrahub-reserved namespaces are left untouched. Comments, quoting, and inline (flow) sequences are preserved. By default the change is purely key ordering. The opt-in flags additionally change content: --strip-defaults drops redundant default values, --sort-by-order-weight reorders attributes/relationships, and --backfill-order-weight fills in a missing order\_weight. Examples: infrahubctl schema format schemas/ infrahubctl schema format schemas/dcim.yml --diff infrahubctl schema format schemas/ --check infrahubctl schema format schemas/ --strip-defaults --sort-by-order-weight **Usage**: ``` $ infrahubctl schema format [OPTIONS] SCHEMAS... ``` **Arguments**: * `SCHEMAS...`: \[required] **Options**: * `--check`: Do not write files; exit 1 if any file would be reformatted. * `--diff`: Print a diff of the changes instead of writing files. * `--strip-defaults`: Remove attribute/relationship/node keys whose value equals the schema default. * `--sort-by-order-weight`: Sort attributes and relationships by order\_weight (items without one keep their order and go last). * `--backfill-order-weight`: Give attributes/relationships that lack an order\_weight the value 1000. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. --- # `infrahubctl task` Manage Infrahub tasks. **Usage**: ``` $ infrahubctl task [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `list`: List Infrahub tasks. ## `infrahubctl task list`[​](#infrahubctl-task-list "Direct link to infrahubctl-task-list") List Infrahub tasks. **Usage**: ``` $ infrahubctl task list [OPTIONS] ``` **Options**: * `-s, --state TEXT`: Filter by task state. Can be provided multiple times. * `--limit INTEGER`: Maximum number of tasks to retrieve. * `--offset INTEGER`: Offset for pagination. * `--include-related-nodes / --no-include-related-nodes`: Include related nodes in the output. \[default: no-include-related-nodes] * `--include-logs / --no-include-logs`: Include task logs in the output. \[default: no-include-logs] * `--json`: Output the result as JSON. * `--debug / --no-debug`: \[default: no-debug] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. --- # `infrahubctl telemetry` **Usage**: ``` $ infrahubctl telemetry [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `list`: List telemetry snapshots with summary... * `export`: Export telemetry snapshots to a JSON file. ## `infrahubctl telemetry list`[​](#infrahubctl-telemetry-list "Direct link to infrahubctl-telemetry-list") List telemetry snapshots with summary information. **Usage**: ``` $ infrahubctl telemetry list [OPTIONS] ``` **Options**: * `--start-date [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%dT%H:%M:%S%z]`: Start date filter (ISO 8601) * `--end-date [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%dT%H:%M:%S%z]`: End date filter (ISO 8601) * `--limit INTEGER`: Maximum number of results \[default: 50] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl telemetry export`[​](#infrahubctl-telemetry-export "Direct link to infrahubctl-telemetry-export") Export telemetry snapshots to a JSON file. Pages through the API automatically so that all matching snapshots are exported, not just the first page. **Usage**: ``` $ infrahubctl telemetry export [OPTIONS] ``` **Options**: * `--output TEXT`: Output file path \[default: telemetry-export.json] * `--start-date [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%dT%H:%M:%S%z]`: Start date filter (ISO 8601) * `--end-date [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%dT%H:%M:%S%z]`: End date filter (ISO 8601) * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. --- # `infrahubctl transform` Render a local transform (TransformPython) for debugging purpose. **Usage**: ``` $ infrahubctl transform [OPTIONS] [TRANSFORM_NAME] [VARIABLES]... ``` **Arguments**: * `[TRANSFORM_NAME]`: Name of the Python transformation * `[VARIABLES]...`: Variables to pass along with the query. Format key=value key=value. **Options**: * `--branch TEXT`: Branch on which to run the transformation * `--debug / --no-debug`: \[default: no-debug] * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--list`: Show available transforms * `--out TEXT`: Path to a file to save the result. * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # `infrahubctl validate` Helper to validate the format of various files. **Usage**: ``` $ infrahubctl validate [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `schema`: Validate the format of a schema file... * `graphql-query`: Validate the format of a GraphQL Query... ## `infrahubctl validate schema`[​](#infrahubctl-validate-schema "Direct link to infrahubctl-validate-schema") Validate the format of a schema file either in JSON or YAML. **Usage**: ``` $ infrahubctl validate schema [OPTIONS] SCHEMA ``` **Arguments**: * `SCHEMA`: \[required] **Options**: * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--help`: Show this message and exit. ## `infrahubctl validate graphql-query`[​](#infrahubctl-validate-graphql-query "Direct link to infrahubctl-validate-graphql-query") Validate the format of a GraphQL Query stored locally by executing it on a remote GraphQL endpoint. **Usage**: ``` $ infrahubctl validate graphql-query [OPTIONS] QUERY [VARIABLES]... ``` **Arguments**: * `QUERY`: \[required] * `[VARIABLES]...`: Variables to pass along with the query. Format key=value key=value. **Options**: * `--debug / --no-debug`: Display more troubleshooting information. \[default: no-debug] * `--branch TEXT`: Branch on which to validate the GraphQL Query. * `--config-file TEXT`: \[env var: INFRAHUBCTL\_CONFIG; default: infrahubctl.toml] * `--out TEXT`: Path to a file to save the result. * `--help`: Show this message and exit. --- # `infrahubctl version` Display the version of Python and the version of the Python SDK in use. **Usage**: ``` $ infrahubctl version [OPTIONS] ``` **Options**: * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. --- # Infrahub Integrations Infrahub integrates with a wide range of infrastructure management and automation tools to help you synchronize data, automate workflows, and manage your infrastructure. This page provides an overview of all available integrations and their capabilities. ## Network automation and validation platforms[​](#network-automation-and-validation-platforms "Direct link to Network automation and validation platforms") ### Ansible[​](#ansible "Direct link to Ansible") The OpsMill Infrahub Ansible Collection provides modules and plugins to seamlessly interact with Infrahub through Ansible. This integration enables you to define and enforce the desired state of your infrastructure using Ansible playbooks. **Key capabilities:** * Dynamic inventory from Infrahub GraphQL data * Node creation, updates, and deletion * Branch management and manipulation * GraphQL queries and lookup operations * Artifact retrieval and management [Ansible Documentation/ansible](/ansible.md) ### Kriten[​](#kriten "Direct link to Kriten") Kriten is an API-first automation engine that transforms containerized scripts into secure REST API endpoints. The integration with Infrahub creates a powerful automation pipeline that separates data modeling from task execution, enabling secure cross-domain automation. **Key capabilities:** * Webhook-triggered automation from Infrahub events * Automatic API generation from containerized scripts * Enterprise-grade security with RBAC and audit trails * Support for synchronous and asynchronous job execution * Multi-language script support for diverse automation needs [Infrahub-Kriten Integrationhttps://opsmill.com/blog/infrahub-kriten-integration/](https://opsmill.com/blog/infrahub-kriten-integration/) ### Netpicker[​](#netpicker "Direct link to Netpicker") Netpicker provides network testing and compliance validation capabilities that integrate with Infrahub as a source of truth. This integration enables automated compliance checking by comparing live device configurations against Infrahub data. **Key capabilities:** * Device interface compliance validation * Configuration drift detection between Infrahub and live devices * Automated assertion-based testing for network infrastructure * Integration with pytest frameworks for network testing workflows [NetPicker Infrahub Integrationhttps://github.com/netpicker/pytests-for-networking/blob/main/Integrations/Infrahub/infrahub.py](https://github.com/netpicker/pytests-for-networking/blob/main/Integrations/Infrahub/infrahub.py) ### Nornir[​](#nornir "Direct link to Nornir") A Nornir plugin that allows Infrahub to serve as an inventory source for Nornir-based network automation workflows. This integration simplifies network automation by providing structured data directly from your Infrahub instance. **Key capabilities:** * Inventory management with GraphQL-based host and group data * Artifact management for configuration templates and files * Host-specific artifact regeneration * Bulk artifact generation across multiple devices [Nornir Documentation/nornir](/nornir.md) ## Data synchronization with Infrahub Sync[​](#data-synchronization-with-infrahub-sync "Direct link to Data synchronization with Infrahub Sync") Infrahub Sync is a versatile Python package that synchronizes data between source systems and Infrahub. Built on `diffsync`, it provides flexible and efficient data synchronization across different network management platforms. [Infrahub Sync Overview/sync](/sync.md) ### Network management systems[​](#network-management-systems "Direct link to Network management systems") #### LibreNMS[​](#librenms "Direct link to LibreNMS") Integrates with the open-source LibreNMS network monitoring system to import device and monitoring data into Infrahub for centralized infrastructure management. [LibreNMS Adapter/sync/adapters/librenms](/sync/adapters/librenms.md) #### Observium[​](#observium "Direct link to Observium") Connects with Observium network monitoring platform to synchronize device discovery and performance data with Infrahub for comprehensive network visibility. [Observium Adapter/sync/adapters/observium](/sync/adapters/observium.md) ### Infrastructure documentation systems[​](#infrastructure-documentation-systems "Direct link to Infrastructure documentation systems") #### Nautobot[​](#nautobot "Direct link to Nautobot") Synchronizes data from Nautobot, an open-source network source of truth platform. Supports both Nautobot v1 and v2 data models with flexible schema mapping. **Key data synchronized:** * Device and platform information * Location and site hierarchies * Manufacturer and hardware data * Tags and custom attributes [Nautobot Adapter/sync/adapters/nautobot](/sync/adapters/nautobot.md) #### Netbox[​](#netbox "Direct link to Netbox") Imports infrastructure data from NetBox, the popular open-source IPAM and DCIM tool, into Infrahub while maintaining data relationships and hierarchies. **Key data synchronized:** * Device inventory and rack layouts * IP address management (IPAM) data * Location hierarchies (regions, sites, racks) * Circuit and connection documentation [Netbox Adapter/sync/adapters/netbox](/sync/adapters/netbox.md) #### Peering manager[​](#peering-manager "Direct link to Peering manager") Bi-directional synchronization with Peering Manager for comprehensive BGP session and interconnection management. **Key data synchronized:** * BGP communities and routing policies * Peering session configurations * Internet exchange point (IXP) data * AS (Autonomous System) information [Peering Manager Adapter/sync/adapters/peering-manager](/sync/adapters/peering-manager.md) ### Network data collection[​](#network-data-collection "Direct link to Network data collection") #### IP Fabric[​](#ip-fabric "Direct link to IP Fabric") Synchronizes network discovery data from IP Fabric's network discovery and validation platform into Infrahub. IP Fabric provides automatic network discovery and advanced analytics through SSH/Telnet connectivity. **Key data synchronized:** * Device inventory with hardware details * Network topology and relationships * Platform and software version data [IP Fabric Adapter/sync/adapters/ipfabric](/sync/adapters/ipfabric.md) #### Slurp'it[​](#slurpit "Direct link to Slurp'it") Integrates with Slurp'it network data collection platform to import structured network configuration and operational data into Infrahub. **Key data synchronized:** * Device vendor and model information * Network configuration templates * Routing table and interface data * Custom planning and operational data [Slurp'It Adapter/sync/adapters/slurpit](/sync/adapters/slurpit.md) --- # Infrahub MCP Server Infrahub MCP Server connects AI assistants and IDE agents to [Infrahub](https://github.com/opsmill/infrahub) using the open [Model Context Protocol](https://modelcontextprotocol.io) standard — so agents can query, create, update, and propose changes to your infrastructure data through a consistent, audited interface. It works with any MCP-compatible client (Claude Desktop, VS Code, Cursor, CLI agents, and more) with no custom glue code required. All writes are branch-isolated and require human approval before merging — agents never modify your default branch directly. ## What you can do with it[​](#what-you-can-do-with-it "Direct link to What you can do with it") * **Query your infrastructure data from natural language** — ask an AI assistant to find devices, interfaces, IP addresses, or any node kind defined in your Infrahub schema, with attribute filtering and partial-match search. * **Explore your schema without leaving the conversation** — the server exposes your full schema catalog, per-kind attribute and filter maps, and the GraphQL SDL as MCP resources that agents read automatically. * **Make changes on isolated branches** — all writes happen on an auto-created session branch (`mcp/session-YYYYMMDD-`), so the default branch is never modified directly. * **Submit changes for human review** — when edits are ready, call `propose_changes` to open a Proposed Change (the Infrahub equivalent of a pull request) for approval before merging. * **Run arbitrary GraphQL queries** — for advanced use cases, execute any GraphQL query or mutation directly against the Infrahub API. ## Who this is for[​](#who-this-is-for "Direct link to Who this is for") **An infrastructure or network team already using Infrahub** that wants their AI coding assistants to understand and interact with their source of truth. You get schema-aware querying, safe branch-isolated writes, and a human-in-the-loop review path — all from your IDE or chat assistant. **A developer building AI-powered infrastructure automation** who needs a standardized interface between agents and Infrahub. The MCP server handles authentication, schema discovery, and branch management so you can focus on the agent logic. ## What's included[​](#whats-included "Direct link to What's included") * **Tools (read)** — `get_nodes` retrieves objects by kind with filters, `search_nodes` finds nodes by partial name match, `query_graphql` executes arbitrary GraphQL queries. * **Tools (write)** — `node_upsert` creates or updates nodes, `node_delete` removes nodes, `mutate_graphql` executes GraphQL mutations, `propose_changes` opens a Proposed Change for human review — all on an auto-created session branch. * **Resources** — `infrahub://schema` lists all available kinds, `infrahub://schema/{kind}` returns full attribute/filter details, `infrahub://graphql-schema` exposes the GraphQL SDL, `infrahub://branches` lists all branches. * **Prompts** — a built-in `infrahub_agent` system prompt that teaches agents the branch-per-session workflow and available tools. * **Middleware** — 17-layer production middleware stack with rate limiting, retry with exponential backoff, response caching, scope-based auth, OpenTelemetry tracing, Prometheus metrics, and read-only mode — all opt-in via environment variables. * **Infrastructure** — Dockerfile for container deployment, Docker Compose sidecar configuration, Streamable HTTP transport support, `/health` and `/metrics` endpoints. See the [Methods reference](/mcp/references/methods.md) for full details on every tool, resource, and prompt. ## Start here[​](#start-here "Direct link to Start here") * [Install the server](/mcp/getting-started/installation.md) — from clone to running in a few minutes. * [Set up authentication](/mcp/getting-started/authentication.md) — shared API token, per-client token pass-through, or OIDC. * [Your first agent run](/mcp/getting-started/first-agent-run.md) — connect Claude Desktop and ask an infrastructure question. * [Make a change through an agent](/mcp/getting-started/make-a-change.md) — walk the session-branch and `propose_changes` flow end-to-end. ## Go deeper[​](#go-deeper "Direct link to Go deeper") * **Deployment** — [Docker Compose](/mcp/guides/docker-compose.md) sidecar next to Infrahub. * **Use cases** — [Troubleshooting queries](/mcp/use-cases/troubleshooting-queries.md), [natural language to GraphQL](/mcp/use-cases/natural-language-graphql.md), [cross-system correlation](/mcp/use-cases/cross-system-correlation.md), [compliance analysis](/mcp/use-cases/compliance-analysis.md), [brownfield onboarding](/mcp/use-cases/brownfield-onboarding.md), and [safe changes via branch isolation](/mcp/use-cases/safe-changes-branch-isolation.md). * **Integrations** — [Claude Desktop](/mcp/integrations/claude-desktop.md), [Claude Code](/mcp/integrations/claude-code.md), [Cursor](/mcp/integrations/cursor.md), [VS Code](/mcp/integrations/vscode.md), [OpenAI Agents SDK](/mcp/integrations/openai-agents-sdk.md), [Claude Agent SDK](/mcp/integrations/claude-agent-sdk.md). * **Reference** — [Authentication architecture](/mcp/references/authentication.md), [configuration](/mcp/references/configuration.md), [methods](/mcp/references/methods.md). ## About Infrahub[​](#about-infrahub "Direct link to About Infrahub") [Infrahub](https://github.com/opsmill/infrahub) is an open source infrastructure data management and automation platform (AGPLv3), developed by [OpsMill](https://opsmill.com). It gives infrastructure and network teams a unified, schema-driven source of truth for all infrastructure data — devices, topology, IP space, configuration — with built-in version control, a generator framework for automation, and native integrations with Git, Ansible, Terraform, and CI/CD pipelines. --- # Set up authentication This guide walks through configuring each authentication mode. For a deeper explanation of how the auth architecture works — sequence diagrams, the two-layer model, audit logs, and branch-placeholder resolution — see [Authentication architecture](/mcp/references/authentication.md). ## Choosing a mode[​](#choosing-a-mode "Direct link to Choosing a mode") | Mode | When to use | Transport | Identity per request | | ------------------- | -------------------------------------------------------------------------------------------------------------- | ------------- | ----------------------------------- | | `none` (default) | Local development, single-user workstations, VPN-only deployments | stdio or HTTP | Shared `INFRAHUB_API_TOKEN` | | `token-passthrough` | Multi-tenant HTTP deployments where each user should act as themselves in Infrahub | HTTP only | Client sends own Infrahub API token | | `oidc` | Enterprise SSO (Google, Okta, Entra, Keycloak) with centralized identity and optional scope-based write gating | HTTP only | OIDC token validated per request | Username/password authentication is not supported — use OIDC when you need external identity. ## No authentication (default)[​](#no-authentication-default "Direct link to No authentication (default)") No configuration needed beyond Infrahub connection variables: ``` export INFRAHUB_ADDRESS=https://infrahub.example.com export INFRAHUB_API_TOKEN=your-token infrahub-mcp --transport streamable-http ``` All clients share the same Infrahub credentials. Use this for local development or behind a VPN. ## Token passthrough[​](#token-passthrough "Direct link to Token passthrough") Each client sends their own Infrahub API token. No shared server-side credentials required. ``` export INFRAHUB_ADDRESS=https://infrahub.example.com export INFRAHUB_MCP_AUTH_MODE=token-passthrough # No INFRAHUB_API_TOKEN needed — credentials come from the client infrahub-mcp --transport streamable-http ``` Clients include their token in the request header: ``` Authorization: Bearer ``` To use a different header (for example, `X-Infrahub-Token`), set: ``` export INFRAHUB_MCP_TOKEN_PASSTHROUGH_HEADER=X-Infrahub-Token ``` note Token passthrough requires Streamable HTTP transport. The server rejects `--transport stdio` at startup since stdio has no HTTP headers. For per-client `.mcp.json` configuration examples, see [Streamable HTTP with authentication](/mcp/guides/installation.md#streamable-http-with-authentication). ## Basic passthrough[​](#basic-passthrough "Direct link to Basic passthrough") Same fail-closed passthrough model as token passthrough, but clients send Infrahub username and password via the standard HTTP `Basic` scheme: ``` export INFRAHUB_ADDRESS=https://infrahub.example.com export INFRAHUB_MCP_AUTH_MODE=basic-passthrough # No INFRAHUB_USERNAME/INFRAHUB_PASSWORD needed — credentials come from the client infrahub-mcp --transport streamable-http ``` Clients include their credentials in the request header: ``` Authorization: Basic ``` The server decodes the header per request, constructs a fresh `InfrahubClient` with those credentials, and resets the per-request context after the call so credentials are never shared between requests. note Basic passthrough requires Streamable HTTP transport, same as token passthrough. To encode your credentials for the `Authorization` header: ``` echo -n "username:password" | base64 # dXNlcm5hbWU6cGFzc3dvcmQ= ``` For per-client `.mcp.json` configuration examples, see [Streamable HTTP with authentication](/mcp/guides/installation.md#streamable-http-with-authentication). ## OIDC[​](#oidc "Direct link to OIDC") ### Step 1: Register an OAuth client[​](#step-1-register-an-oauth-client "Direct link to Step 1: Register an OAuth client") In your Identity Provider, create an OAuth/OIDC client application: * **Application type**: Web application * **Redirect URI**: `/oauth/callback` (FastMCP handles this automatically) * **Scopes**: At minimum, `openid`, `email`, and `profile`. Add a custom scope (for example, `write` or `infrahub:write`) if you want to gate write access. Record the **client ID**, **client secret** (if not using PKCE), and the **OIDC discovery URL**. ### Step 2: Configure the MCP server[​](#step-2-configure-the-mcp-server "Direct link to Step 2: Configure the MCP server") ``` export INFRAHUB_MCP_AUTH_MODE=oidc export INFRAHUB_MCP_OIDC_CONFIG_URL=https://accounts.google.com/.well-known/openid-configuration export INFRAHUB_MCP_OIDC_CLIENT_ID=my-mcp-client export INFRAHUB_MCP_OIDC_BASE_URL=https://mcp.example.com # Optional — omit for PKCE flow (recommended for public clients) export INFRAHUB_MCP_OIDC_CLIENT_SECRET=my-secret ``` The server validates that all required OIDC fields are set at startup and fails fast with a clear error if anything is missing. ### Step 3: Configure write scopes (optional)[​](#step-3-configure-write-scopes-optional "Direct link to Step 3: Configure write scopes (optional)") By default, any authenticated user can call both read and write tools. To restrict write access to users with a specific scope in their token: ``` # Only users whose token contains "infrahub:write" can call write tools export INFRAHUB_MCP_AUTH_SCOPES_WRITE=infrahub:write ``` When configured, tools tagged `write` (`node_upsert`, `node_delete`, `propose_changes`, `mutate_graphql`) are: 1. **Hidden from discovery** — users without the required scope never see write tools in `tools/list`. 2. **Blocked at call time** — even if a client hardcodes a tool name, the call is rejected. Users without the write scope effectively get a read-only experience without needing `INFRAHUB_MCP_READ_ONLY=true`. ### Step 4: Verify[​](#step-4-verify "Direct link to Step 4: Verify") Start the server and confirm the OIDC configuration in the logs: ``` INFO oidc_auth enabled=true config_url=https://accounts.google.com/... client_id=my-mcp-client INFO auth_middleware enabled=true auth_mode=oidc write_scopes=['infrahub:write'] ``` ## Provider examples[​](#provider-examples "Direct link to Provider examples") ### Google[​](#google "Direct link to Google") ``` INFRAHUB_MCP_AUTH_MODE=oidc INFRAHUB_MCP_OIDC_CONFIG_URL=https://accounts.google.com/.well-known/openid-configuration INFRAHUB_MCP_OIDC_CLIENT_ID=123456789.apps.googleusercontent.com INFRAHUB_MCP_OIDC_CLIENT_SECRET=GOCSPX-... INFRAHUB_MCP_OIDC_BASE_URL=https://mcp.example.com ``` ### Okta[​](#okta "Direct link to Okta") ``` INFRAHUB_MCP_AUTH_MODE=oidc INFRAHUB_MCP_OIDC_CONFIG_URL=https://your-org.okta.com/.well-known/openid-configuration INFRAHUB_MCP_OIDC_CLIENT_ID=0oa... INFRAHUB_MCP_OIDC_BASE_URL=https://mcp.example.com INFRAHUB_MCP_AUTH_SCOPES_WRITE=infrahub:write ``` ### Microsoft Entra[​](#microsoft-entra "Direct link to Microsoft Entra") ``` INFRAHUB_MCP_AUTH_MODE=oidc INFRAHUB_MCP_OIDC_CONFIG_URL=https://login.microsoftonline.com/{tenant-id}/v2.0/.well-known/openid-configuration INFRAHUB_MCP_OIDC_CLIENT_ID=your-app-id INFRAHUB_MCP_OIDC_BASE_URL=https://mcp.example.com INFRAHUB_MCP_OIDC_AUDIENCE=api://your-app-id ``` ### Keycloak[​](#keycloak "Direct link to Keycloak") ``` INFRAHUB_MCP_AUTH_MODE=oidc INFRAHUB_MCP_OIDC_CONFIG_URL=https://keycloak.example.com/realms/infrahub/.well-known/openid-configuration INFRAHUB_MCP_OIDC_CLIENT_ID=infrahub-mcp INFRAHUB_MCP_OIDC_BASE_URL=https://mcp.example.com INFRAHUB_MCP_AUTH_SCOPES_WRITE=write ``` ## Configuration reference[​](#configuration-reference "Direct link to Configuration reference") For the complete list of environment variables, see the [Configuration reference](/mcp/references/configuration.md#authentication-and-authorization). For an architectural deep-dive on each mode, see [Authentication architecture](/mcp/references/authentication.md). | Variable | Required | Default | Description | | --------------------------------------- | ---------- | --------------- | ------------------------------------------------------------------------------ | | `INFRAHUB_MCP_AUTH_MODE` | No | `none` | Authentication mode (`none`, `oidc`, `token-passthrough`, `basic-passthrough`) | | `INFRAHUB_MCP_OIDC_CONFIG_URL` | Yes (oidc) | | OIDC discovery endpoint URL | | `INFRAHUB_MCP_OIDC_CLIENT_ID` | Yes (oidc) | | OAuth client ID registered with the IdP | | `INFRAHUB_MCP_OIDC_CLIENT_SECRET` | No | | OAuth client secret (omit for PKCE flow) | | `INFRAHUB_MCP_OIDC_BASE_URL` | Yes (oidc) | | Public URL where the MCP server is accessible | | `INFRAHUB_MCP_OIDC_AUDIENCE` | No | | Token audience claim | | `INFRAHUB_MCP_OIDC_USER_CLAIM` | No | `email` | JWT claim used for user identity | | `INFRAHUB_MCP_AUTH_SCOPES_WRITE` | No | `write` | Scopes required for write operations (comma-separated) | | `INFRAHUB_MCP_TOKEN_PASSTHROUGH_HEADER` | No | `Authorization` | HTTP header carrying the Infrahub API token (token-passthrough mode) | --- # Your first agent run This walkthrough connects an LLM client to your running Infrahub MCP server and asks it a read-only question. By the end you'll have confirmed the server is reachable, the agent can discover your schema, and the built-in `infrahub_agent` system prompt is guiding the conversation correctly. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * A running Infrahub MCP server — see [Install the Infrahub MCP server](/mcp/getting-started/installation.md). * An MCP-capable client installed. This guide uses **Claude Desktop** on stdio; the same flow works with [Claude Code](/mcp/integrations/claude-code.md), [Cursor](/mcp/integrations/cursor.md), [VS Code](/mcp/integrations/vscode.md), or any client that speaks MCP. * `INFRAHUB_ADDRESS` and `INFRAHUB_API_TOKEN` set in your environment. ## 1. Configure Claude Desktop[​](#1-configure-claude-desktop "Direct link to 1. Configure Claude Desktop") Open **Settings → Developer** in Claude Desktop, then click `Edit config` and add the server to `claude_desktop_config.json`: ``` { "mcpServers": { "infrahub_mcp": { "transport": "stdio", "command": "uv", "args": [ "run", "fastmcp", "run", "src/infrahub_mcp/server.py:mcp" ], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000", "INFRAHUB_API_TOKEN": "your-api-token" } } } } ``` Save the file and restart Claude Desktop. The `infrahub_mcp` server should appear in the developer panel with a green status dot. If you're connecting to a remote server over HTTP instead, replace the `command`/`args`/`env` block with a `url` field — see the [Docker Compose guide](/mcp/guides/docker-compose.md#connect-your-mcp-client) for the HTTP transport configuration. ## 2. Load the agent prompt[​](#2-load-the-agent-prompt "Direct link to 2. Load the agent prompt") In a new Claude Desktop conversation, attach the built-in `infrahub_agent` prompt from the MCP server. The prompt teaches the model: * How to discover your schema via the `infrahub://schema` resource. * Which tools are read-only (`get_nodes`, `search_nodes`, `query_graphql`, `get_schema`) versus write (`node_upsert`, `node_delete`, `propose_changes`, `mutate_graphql`). * The session-branch workflow for any mutation. The prompt dynamically adjusts to read-only mode (`INFRAHUB_MCP_READ_ONLY=true`) and hides the write path when writes are disabled. ## 3. Ask a read-only question[​](#3-ask-a-read-only-question "Direct link to 3. Ask a read-only question") Try a prompt like: > What device kinds are defined in the schema? List the five most common ones. Behind the scenes the agent should: 1. **Read `infrahub://schema`** to list all non-internal kinds. 2. **Call `get_nodes(kind=...)`** for each candidate to sample the data. 3. **Return a natural-language summary** grounded in what's actually in your Infrahub. If your client shows tool-call traces, you should see those two MCP calls in the transcript. If the client hides them, watch the server logs — every call is logged with a request ID and latency. ## 4. Verify observability (optional)[​](#4-verify-observability-optional "Direct link to 4. Verify observability (optional)") If you're running the server with Streamable HTTP transport and enabled Prometheus metrics (`INFRAHUB_MCP_PROMETHEUS_ENABLED=true`), scrape `/metrics` after a few calls: ``` curl -s http://localhost:8001/metrics | grep infrahub_mcp_requests_total ``` You should see counters for `tools/call` and `resources/read` incrementing. See the [Configuration reference](/mcp/references/configuration.md#observability) for the full metrics list. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") | Symptom | Likely cause | Fix | | -------------------------------------------------------------- | ---------------------------------------------------------- | --------------------------------------------------------------------------- | | Server won't start — `INFRAHUB_MCP_AUTH_MODE` validation error | Missing OIDC configuration or token-passthrough over stdio | Use `auth_mode=none` for stdio, or switch to `--transport streamable-http`. | | Client shows "server connected" but no tools | Stale `tools/list` cache | Restart the client; MCP capabilities are cached per session. | | `get_nodes` returns empty | Wrong kind casing, missing `__value` suffix in filters | Read `infrahub://schema/{kind}` first — it lists the exact filter keys. | | 401/403 from Infrahub | API token expired or lacks permissions | Regenerate the token in Infrahub → **Admin → API tokens**. | ## Next steps[​](#next-steps "Direct link to Next steps") * [Make a change through an agent](/mcp/getting-started/make-a-change.md) — walk the write path end-to-end. * [Use cases](/mcp/use-cases/troubleshooting-queries.md) — recipe-style scenarios for day-to-day work. * [Integrations](/mcp/integrations/claude-desktop.md) — per-client configuration details. --- # Install the Infrahub MCP server This guide gets you from a fresh checkout to a running Infrahub MCP server in a few minutes. Once it's running, continue with [Authentication](/mcp/getting-started/authentication.md) to pick a credential model, then [Your first agent run](/mcp/getting-started/first-agent-run.md) to connect an LLM. For a containerized deployment (Docker Compose, sidecar alongside Infrahub), see the [Docker Compose guide](/mcp/guides/docker-compose.md). ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Python 3.13 or newer * [uv](https://docs.astral.sh/uv/) for dependency management * An Infrahub instance you can reach over HTTP * An Infrahub API token (create one from **Admin → API tokens** in the Infrahub UI) ## Install[​](#install "Direct link to Install") The recommended way is via `uvx` (no clone needed): ``` uvx infrahub-mcp --help ``` Or install into an existing environment: ``` pip install infrahub-mcp ``` To develop or contribute, clone the repository: ``` git clone https://github.com/opsmill/infrahub-mcp.git cd infrahub-mcp uv sync ``` ## Configure the connection[​](#configure-the-connection "Direct link to Configure the connection") The server reads its configuration from environment variables. At minimum, set the Infrahub URL and an API token: ``` export INFRAHUB_ADDRESS=http://localhost:8000 export INFRAHUB_API_TOKEN=06438eb2-8019-4776-878c-0941b1f1d1ec ``` For the complete list of variables (rate limiting, caching, observability, auth), see the [Configuration reference](/mcp/references/configuration.md). ## Run the server[​](#run-the-server "Direct link to Run the server") ### Stdio transport (local clients)[​](#stdio-transport-local-clients "Direct link to Stdio transport (local clients)") Stdio is the default transport for MCP clients that spawn the server as a subprocess (Claude Desktop, Claude Code with `.mcp.json`): ``` uvx infrahub-mcp ``` ### Streamable HTTP transport (remote clients, Docker)[​](#streamable-http-transport-remote-clients-docker "Direct link to Streamable HTTP transport (remote clients, Docker)") HTTP is required for token pass-through, OIDC, or any client connecting over the network: ``` uvx infrahub-mcp --transport streamable-http --host 0.0.0.0 --port 8001 ``` The server exposes three endpoints on the configured port: | Path | Description | | ---------- | ----------------------------------------------------------- | | `/mcp` | MCP Streamable HTTP transport | | `/health` | Health probe — returns `{"status": "healthy"}` or HTTP 503 | | `/metrics` | Request counts, errors, latency (JSON or Prometheus format) | ## Verify it is running[​](#verify-it-is-running "Direct link to Verify it is running") With the HTTP transport: ``` curl -s http://localhost:8001/health # {"status":"healthy"} ``` Or inspect the server logs — you should see a line confirming the Infrahub connection and the middleware stack that is active. ## Next steps[​](#next-steps "Direct link to Next steps") * [Set up authentication](/mcp/getting-started/authentication.md) — choose between a shared API token, per-client token pass-through, or OIDC. * [Your first agent run](/mcp/getting-started/first-agent-run.md) — connect Claude Desktop and ask Infrahub a read-only question end-to-end. * [Make a change through an agent](/mcp/getting-started/make-a-change.md) — walk the session-branch + `propose_changes` workflow. --- # Make a change through an agent This walkthrough takes you through the full write path: ask an agent to update something in Infrahub, watch it auto-create an isolated session branch, and open a Proposed Change for human review. No data on your default branch is modified until a human approves the merge. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * The [First agent run](/mcp/getting-started/first-agent-run.md) walkthrough is complete (server reachable, client connected, read-only calls working). * `INFRAHUB_MCP_READ_ONLY` is not set (or is `false`) — writes are enabled. * Your Infrahub API token has permission to create branches and proposed changes. ## How it works[​](#how-it-works "Direct link to How it works") Every write starts by auto-creating a **session branch** on the first mutation. The default naming pattern is: ``` mcp/session-YYYYMMDD- ``` All subsequent writes in the same MCP session go to the same branch. When the agent calls `propose_changes`, Infrahub opens a `CoreProposedChange` (a pull-request equivalent) from the session branch back to the default branch. A human reviews, approves, and merges — only then does the change land. You can customize the pattern via `INFRAHUB_MCP_BRANCH_PATTERN`. Supported placeholders: | Placeholder | Resolves to | | ----------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `{date}` | Current date in `YYYYMMDD` format | | `{hex}` | Short random hex suffix for uniqueness | | `{user}` | Authenticated user identity (OIDC mode only — see [Authentication architecture](/mcp/references/authentication.md#branch-placeholder)) | If the generated branch already exists (rare, with `{hex}`), the server retries up to `INFRAHUB_MCP_MAX_BRANCH_RETRIES` times (default 5). ## 1. Ask the agent to make a change[​](#1-ask-the-agent-to-make-a-change "Direct link to 1. Ask the agent to make a change") In your MCP client, prompt something like: > Update the description of the device named `leaf-01` to `"Rack 3A, upgraded 2026-04"`. The agent should: 1. **Discover the schema** — read `infrahub://schema` and then `infrahub://schema/{kind}` for the relevant device kind to confirm the attribute name. 2. **Look up the node** — call `search_nodes(query="leaf-01", kind="...")` or `get_nodes` with a `name__value` filter to resolve the UUID or human-friendly ID. 3. **Create the session branch** — happens implicitly on the first write. You'll see it in the `infrahub://branches` resource. 4. **Call `node_upsert`** with `id` or `hfid` to update the `description` attribute. 5. **Confirm the write** — report the branch name and node ID. ## 2. Open a Proposed Change for review[​](#2-open-a-proposed-change-for-review "Direct link to 2. Open a Proposed Change for review") Follow up with: > Open a proposed change titled "Update leaf-01 description" so the team can review. The agent calls `propose_changes(title=..., description=..., destination_branch=...)`. The response includes the `CoreProposedChange` ID and the source/destination branches. The session branch stays active, so you can keep making related changes and they'll all show up in the same proposed change. ## 3. Review in Infrahub[​](#3-review-in-infrahub "Direct link to 3. Review in Infrahub") Open the Infrahub UI and navigate to **Proposed Changes**. The change you just opened will show: * The diff between the session branch and the default branch. * The agent that created the branch (in audit logs, if OIDC is enabled). * Any check results, if you have [Infrahub checks](https://docs.infrahub.app/topics/proposed-change) configured. Approve and merge when ready. Until a human approves, the default branch is untouched. ## Guardrails at a glance[​](#guardrails-at-a-glance "Direct link to Guardrails at a glance") * **Branch isolation** — every write lives on an ephemeral session branch; the default branch is never modified directly. * **Read-only kill switch** — setting `INFRAHUB_MCP_READ_ONLY=true` hides all write tools from discovery and rejects calls that hardcode them. * **Scope gating (OIDC only)** — set `INFRAHUB_MCP_AUTH_SCOPES_WRITE=infrahub:write` to require a specific OAuth scope for write tools. * **Audit logs** — every tool call is logged with a request ID, tool name, branch, and (under OIDC) the authenticated user. * **Human approval** — `propose_changes` is the only way changes reach the default branch. Agents cannot merge. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") | Symptom | Likely cause | Fix | | --------------------------------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | `node_upsert` fails with "kind not found" | Schema name mismatch (case, namespace) | Ask the agent to read `infrahub://schema` first and copy the exact kind name. | | Agent tries to update a relationship and fails | `node_upsert` accepts scalar attributes only | Use `mutate_graphql` with a GraphQL mutation for relationship changes. | | `propose_changes` returns an error about no changes | Session branch is empty | Make at least one successful `node_upsert` / `node_delete` / `mutate_graphql` call before proposing. | | Write tools are hidden from the agent | Server is in read-only mode, or user lacks the write scope | Check `INFRAHUB_MCP_READ_ONLY` and `INFRAHUB_MCP_AUTH_SCOPES_WRITE` on the server. | ## Next steps[​](#next-steps "Direct link to Next steps") * [Safe changes via branch isolation](/mcp/use-cases/safe-changes-branch-isolation.md) — deeper look at the branch-per-session model. * [Brownfield network onboarding](/mcp/use-cases/brownfield-onboarding.md) — use the write path to bulk-import existing state. * [Methods reference](/mcp/references/methods.md) — every tool, resource, and prompt the server exposes. --- # Running with Docker This guide covers deploying the Infrahub MCP server as a container, including running it alongside Infrahub in the same Docker Compose stack. ## Quick start with Docker Compose[​](#quick-start-with-docker-compose "Direct link to Quick start with Docker Compose") ``` docker compose up -d ``` This pulls the published image from the Opsmill registry and starts the MCP server. Configure it with a `.env` file or environment variables (see [Configuration reference](/mcp/references/configuration.md)). ## Build the image locally[​](#build-the-image-locally "Direct link to Build the image locally") ``` docker build -f development/Dockerfile -t infrahub-mcp . ``` ## Run standalone[​](#run-standalone "Direct link to Run standalone") ``` docker run -p 8001:8001 \ -e INFRAHUB_ADDRESS=http://your-infrahub-host:8000 \ -e INFRAHUB_API_TOKEN=your-token \ registry.opsmill.io/opsmill/infrahub-mcp:latest ``` The server listens on `http://localhost:8001/mcp` using the [MCP Streamable HTTP](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#streamable-http) transport. ## Deploy alongside Infrahub[​](#deploy-alongside-infrahub "Direct link to Deploy alongside Infrahub") The MCP server connects to Infrahub over HTTP, making it a natural sidecar in a Docker Compose stack. Add the `infrahub-mcp` service to your existing `docker-compose.yml`: ``` services: # ... your existing Infrahub services ... infrahub-mcp: image: registry.opsmill.io/opsmill/infrahub-mcp:latest ports: - "8001:8001" environment: INFRAHUB_ADDRESS: "http://infrahub-server:8000" INFRAHUB_API_TOKEN: "${INFRAHUB_API_TOKEN}" MCP_HOST: "0.0.0.0" MCP_PORT: "8001" depends_on: - infrahub-server restart: unless-stopped ``` Replace `infrahub-server` with the service name of your Infrahub instance. ## Connect your MCP client[​](#connect-your-mcp-client "Direct link to Connect your MCP client") Once the container is running, point your MCP client at the HTTP endpoint instead of running a local subprocess: * Cursor * VS Code * Claude Desktop ``` { "mcpServers": { "infrahub_mcp": { "type": "streamable-http", "url": "http://localhost:8001/mcp" } } } ``` ``` { "servers": { "infrahub_mcp": { "type": "http", "url": "http://localhost:8001/mcp" } } } ``` ``` { "mcpServers": { "infrahub_mcp": { "transport": "streamable-http", "url": "http://localhost:8001/mcp" } } } ``` ## Configuration reference[​](#configuration-reference "Direct link to Configuration reference") | Variable | Description | Default | | --------------------------------- | ---------------------------------------- | ------------ | | `INFRAHUB_ADDRESS` | URL of your Infrahub instance | **required** | | `INFRAHUB_API_TOKEN` | API token *(or use username + password)* | — | | `INFRAHUB_USERNAME` | Username for basic-auth login | — | | `INFRAHUB_PASSWORD` | Password for basic-auth login | — | | `INFRAHUB_TIMEOUT` | HTTP request timeout in seconds | `30` | | `MCP_HOST` | Bind address for the HTTP server | `0.0.0.0` | | `MCP_PORT` | Port for the HTTP server | `8001` | | `INFRAHUB_MCP_READ_ONLY` | Disable write operations | `false` | | `INFRAHUB_MCP_RATE_LIMIT_RPS` | Rate limit (requests/sec, `0` = off) | `0` | | `INFRAHUB_MCP_CACHE_ENABLED` | Enable response caching | `false` | | `INFRAHUB_MCP_PROMETHEUS_ENABLED` | Prometheus format for `/metrics` | `false` | For the full list including retry, auth, OpenTelemetry, and more, see the [Configuration reference](/mcp/references/configuration.md). ## Endpoints[​](#endpoints "Direct link to Endpoints") | Path | Method | Description | | ---------- | ------ | -------------------------------------------------------- | | `/mcp` | POST | MCP Streamable HTTP transport | | `/health` | GET | Health check for container orchestration probes | | `/metrics` | GET | Request counts, errors, and latency (JSON or Prometheus) | --- # Docker Compose deployment This guide covers running the Infrahub MCP server as a container — standalone for development, or as a sidecar alongside Infrahub in the same Docker Compose stack for production. Kubernetes/Helm deployment is planned. Until then, you can run the MCP server as a plain `Deployment` next to an Infrahub installation managed by [`opsmill/infrahub-helm`](https://github.com/opsmill/infrahub-helm). ## Build the image[​](#build-the-image "Direct link to Build the image") From the repository root: ``` docker build -t infrahub-mcp . ``` The `Dockerfile` is a two-stage build: Python 3.13 + `uv` installs dependencies into `/opt/venv`, and the runtime stage copies them into a minimal `python:3.13-slim` image running as a non-root `app` user. The container exposes port 8001 and runs `infrahub-mcp --transport streamable-http --host 0.0.0.0 --port 8001` by default. ## Run standalone[​](#run-standalone "Direct link to Run standalone") ``` docker run --rm -p 8001:8001 \ -e INFRAHUB_ADDRESS=http://your-infrahub-host:8000 \ -e INFRAHUB_API_TOKEN=your-token \ infrahub-mcp ``` The server listens on `http://localhost:8001/mcp` using the [MCP Streamable HTTP transport](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#streamable-http). Verify it's up: ``` curl -s http://localhost:8001/health # {"status":"healthy"} ``` ## Deploy as a sidecar to Infrahub[​](#deploy-as-a-sidecar-to-infrahub "Direct link to Deploy as a sidecar to Infrahub") The repository ships a `docker-compose.yml` that uses a YAML anchor (`x-infrahub-mcp-config`) to centralize the env-var list. This is the recommended starting point for real deployments — drop it next to (or merge it into) your existing Infrahub Compose file. ``` --- x-infrahub-mcp-config: &infrahub_mcp_config INFRAHUB_ADDRESS: ${INFRAHUB_ADDRESS:-http://infrahub-server:8000} INFRAHUB_API_TOKEN: ${INFRAHUB_INITIAL_AGENT_TOKEN:-44af444d-3b26-410d-9546-b758657e026c} INFRAHUB_MCP_AUTH_MODE: ${INFRAHUB_MCP_AUTH_MODE:-none} INFRAHUB_MCP_TOKEN_PASSTHROUGH_HEADER: ${INFRAHUB_MCP_TOKEN_PASSTHROUGH_HEADER:-Authorization} INFRAHUB_MCP_READ_ONLY: ${INFRAHUB_MCP_READ_ONLY:-false} INFRAHUB_MCP_BRANCH_PATTERN: ${INFRAHUB_MCP_BRANCH_PATTERN:-mcp/session-{date}-{hex}} INFRAHUB_MCP_MAX_BRANCH_RETRIES: ${INFRAHUB_MCP_MAX_BRANCH_RETRIES:-5} INFRAHUB_MCP_LOG_LEVEL: ${INFRAHUB_MCP_LOG_LEVEL:-info} INFRAHUB_MCP_RATE_LIMIT_RPS: ${INFRAHUB_MCP_RATE_LIMIT_RPS:-0} INFRAHUB_MCP_RATE_LIMIT_BURST: ${INFRAHUB_MCP_RATE_LIMIT_BURST:-0} INFRAHUB_MCP_RETRY_MAX_ATTEMPTS: ${INFRAHUB_MCP_RETRY_MAX_ATTEMPTS:-0} INFRAHUB_MCP_RETRY_BASE_DELAY: ${INFRAHUB_MCP_RETRY_BASE_DELAY:-1.0} INFRAHUB_MCP_CACHE_ENABLED: ${INFRAHUB_MCP_CACHE_ENABLED:-false} INFRAHUB_MCP_CACHE_LIST_TTL: ${INFRAHUB_MCP_CACHE_LIST_TTL:-300} INFRAHUB_MCP_CACHE_READ_TTL: ${INFRAHUB_MCP_CACHE_READ_TTL:-3600} INFRAHUB_MCP_OTEL_ENABLED: ${INFRAHUB_MCP_OTEL_ENABLED:-false} INFRAHUB_MCP_PROMETHEUS_ENABLED: ${INFRAHUB_MCP_PROMETHEUS_ENABLED:-false} INFRAHUB_MCP_DEREFERENCE_SCHEMAS: ${INFRAHUB_MCP_DEREFERENCE_SCHEMAS:-false} INFRAHUB_MCP_PING_INTERVAL_MS: ${INFRAHUB_MCP_PING_INTERVAL_MS:-0} INFRAHUB_MCP_AUTH_SCOPES_WRITE: ${INFRAHUB_MCP_AUTH_SCOPES_WRITE:-} services: infrahub-mcp: build: context: . dockerfile: Dockerfile ports: - "8001:8001" environment: <<: [*infrahub_mcp_config] depends_on: - infrahub-server restart: unless-stopped ``` Replace `infrahub-server` with the service name of your Infrahub instance. Every variable falls back to a sensible default; override the ones you care about in a `.env` file or your deployment pipeline. ## Choose a credential model[​](#choose-a-credential-model "Direct link to Choose a credential model") The Compose file defaults to shared API-token auth (`INFRAHUB_MCP_AUTH_MODE=none`, `INFRAHUB_API_TOKEN` set from `INFRAHUB_INITIAL_AGENT_TOKEN`). Switch to token pass-through or OIDC for multi-user deployments — see [Set up authentication](/mcp/getting-started/authentication.md). ## Connect your MCP client[​](#connect-your-mcp-client "Direct link to Connect your MCP client") With the container running on `http://localhost:8001/mcp`, point your client at the HTTP endpoint instead of spawning a subprocess: * Cursor * VS Code * Claude Desktop ``` { "mcpServers": { "infrahub_mcp": { "type": "streamable-http", "url": "http://localhost:8001/mcp" } } } ``` ``` { "servers": { "infrahub_mcp": { "type": "http", "url": "http://localhost:8001/mcp" } } } ``` ``` { "mcpServers": { "infrahub_mcp": { "transport": "streamable-http", "url": "http://localhost:8001/mcp" } } } ``` For per-client setup details (prompts, tool approval, troubleshooting), see the [Integrations section](/mcp/integrations/claude-desktop.md). ## Frequently-overridden variables[​](#frequently-overridden-variables "Direct link to Frequently-overridden variables") | Variable | Description | Default | | --------------------------------- | ----------------------------------------------- | ------------ | | `INFRAHUB_ADDRESS` | URL of your Infrahub instance | **required** | | `INFRAHUB_API_TOKEN` | API token for the MCP server's Infrahub session | — | | `INFRAHUB_TIMEOUT` | HTTP request timeout in seconds | `30` | | `MCP_HOST` | Bind address for the HTTP server | `0.0.0.0` | | `MCP_PORT` | Port for the HTTP server | `8001` | | `INFRAHUB_MCP_READ_ONLY` | Disable write operations | `false` | | `INFRAHUB_MCP_RATE_LIMIT_RPS` | Rate limit (requests/sec, `0` = off) | `0` | | `INFRAHUB_MCP_CACHE_ENABLED` | Enable response caching | `false` | | `INFRAHUB_MCP_PROMETHEUS_ENABLED` | Prometheus format for `/metrics` | `false` | For the full list including retry, auth, OpenTelemetry, and more, see the [Configuration reference](/mcp/references/configuration.md). ## Endpoints[​](#endpoints "Direct link to Endpoints") | Path | Method | Description | | ---------- | ------ | -------------------------------------------------------- | | `/mcp` | POST | MCP Streamable HTTP transport | | `/health` | GET | Health check for container orchestration probes | | `/metrics` | GET | Request counts, errors, and latency (JSON or Prometheus) | ## Operational tips[​](#operational-tips "Direct link to Operational tips") * **Health probes** — `/health` returns HTTP 503 when the server can't reach Infrahub. Wire it into your orchestrator's readiness probe with a generous timeout (Infrahub can be slow to start). * **Metrics** — set `INFRAHUB_MCP_PROMETHEUS_ENABLED=true` and scrape `/metrics` into Prometheus or a Grafana Agent. The JSON format (default) is useful for ad-hoc curls; the Prometheus format is what production monitoring wants. * **Log level** — `INFRAHUB_MCP_LOG_LEVEL=debug` logs full tracebacks and per-call timing; use it when diagnosing an issue, not in steady state. * **Token pass-through requires HTTP** — the Compose deployment is HTTP-only, which is what token pass-through and OIDC require. Don't mix stdio transport into a container deployment. ## Related reading[​](#related-reading "Direct link to Related reading") * [Installation](/mcp/getting-started/installation.md) — running the server without Docker. * [Set up authentication](/mcp/getting-started/authentication.md) — choosing between shared token, pass-through, and OIDC. * [Configuration reference](/mcp/references/configuration.md) — every environment variable. --- # Installing Infrahub MCP This guide provides step-by-step instructions for installing and configuring different MCP clients to connect to the Infrahub MCP server. For container-based deployments (Docker Compose, sidecar alongside Infrahub), see the [Docker deployment guide](/mcp/guides/docker.md). ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Python 3.13+ * Access to an Infrahub instance * An API token or credentials for your Infrahub instance ## Install the Infrahub MCP server[​](#install-the-infrahub-mcp-server "Direct link to Install the Infrahub MCP server") * uvx (recommended) * pip / uv pip * Docker * From source (development) No installation needed — `uvx` runs the package directly from PyPI: ``` uvx infrahub-mcp --help ``` note The `infrahub-mcp` CLI entry point requires version 1.1.0 or later. For earlier versions, use `uvx --from infrahub-mcp fastmcp run infrahub_mcp.server:mcp` instead. ``` pip install infrahub-mcp # or uv pip install infrahub-mcp ``` ``` docker pull registry.opsmill.io/opsmill/infrahub-mcp:latest ``` See the [Docker deployment guide](/mcp/guides/docker.md) for Docker Compose and sidecar setups. ``` git clone https://github.com/opsmill/infrahub-mcp.git cd infrahub-mcp uv sync ``` ## Run the server[​](#run-the-server "Direct link to Run the server") ``` # With uvx (no install) uvx infrahub-mcp # Or if installed via pip infrahub-mcp # With explicit transport infrahub-mcp --transport streamable-http --host 0.0.0.0 --port 8001 ``` ### Configuration[​](#configuration "Direct link to Configuration") Set the following environment variables as needed: | Variable | Description | Default | | ------------------------ | ---------------------------------------- | ------------ | | `INFRAHUB_ADDRESS` | URL of your Infrahub instance | **required** | | `INFRAHUB_API_TOKEN` | API token *(or use username + password)* | — | | `MCP_HOST` | Host for the web server | `0.0.0.0` | | `MCP_PORT` | Port for the web server | `8001` | | `INFRAHUB_MCP_READ_ONLY` | Disable write operations | `false` | | `INFRAHUB_MCP_LOG_LEVEL` | Set to `debug` for verbose logging | `info` | For the full list of configuration options (rate limiting, caching, auth, observability), see the [Configuration reference](/mcp/references/configuration.md). ## Add to your MCP client[​](#add-to-your-mcp-client "Direct link to Add to your MCP client") * Cursor * VS Code * Claude Desktop * Claude Code Go to **Settings > Cursor Settings > Tools & Integrations**, click **Add Custom MCP**, and paste the following into `mcp.json`: ``` { "mcpServers": { "infrahub_mcp": { "command": "uvx", "args": ["infrahub-mcp"], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000", "INFRAHUB_API_TOKEN": "your-api-token" } } } } ``` For tool-approval tuning, workspace rules, and troubleshooting, see the [Cursor integration guide](/mcp/integrations/cursor.md). Open the MCP Servers configuration file (Cmd+Shift+P, type "MCP: Open User Configuration") and add: ``` { "servers": { "infrahub_mcp": { "type": "stdio", "command": "uvx", "args": ["infrahub-mcp"], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000", "INFRAHUB_API_TOKEN": "your-api-token" } } } } ``` For Copilot compatibility (`$ref` inlining) and troubleshooting, see the [VS Code integration guide](/mcp/integrations/vscode.md). Go to **Settings > Developer**, click **Edit config** to open `claude_desktop_config.json`, and add to the `mcpServers` section: ``` { "mcpServers": { "infrahub_mcp": { "transport": "stdio", "command": "uvx", "args": ["infrahub-mcp"], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000", "INFRAHUB_API_TOKEN": "your-api-token" } } } } ``` For remote server setup, prompts, and troubleshooting, see the [Claude Desktop integration guide](/mcp/integrations/claude-desktop.md). Add an `.mcp.json` file at the root of your project (Claude Code auto-detects it on startup): ``` { "mcpServers": { "infrahub": { "command": "uvx", "args": ["infrahub-mcp"], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000", "INFRAHUB_API_TOKEN": "your-api-token" } } } } ``` For CLAUDE.md conventions, bundled prompts, and troubleshooting, see the [Claude Code integration guide](/mcp/integrations/claude-code.md). ## Streamable HTTP with authentication[​](#streamable-http-with-authentication "Direct link to Streamable HTTP with authentication") When the MCP server runs over Streamable HTTP with `token-passthrough` or `basic-passthrough` auth, clients connect via URL and pass credentials in the `Authorization` header instead of using stdio with env vars. See [Set up authentication](/mcp/getting-started/authentication.md) for server-side configuration. warning These configuration files contain credentials in plain text. Do not commit them to version control — add `.mcp.json` to `.gitignore`. * Claude Code * Cursor * Claude Desktop * VS Code In your `.mcp.json`: ``` { "mcpServers": { "infrahub": { "type": "http", "url": "http://localhost:8001/mcp", "headers": { "Authorization": "Bearer " } } } } ``` For basic passthrough, replace the `Authorization` value with `Basic `. In your `.cursor/mcp.json`: ``` { "mcpServers": { "infrahub_mcp": { "url": "http://localhost:8001/mcp", "headers": { "Authorization": "Bearer " } } } } ``` For basic passthrough, replace the `Authorization` value with `Basic `. In your `claude_desktop_config.json`: ``` { "mcpServers": { "infrahub_mcp": { "transport": "streamable-http", "url": "http://localhost:8001/mcp", "headers": { "Authorization": "Bearer " } } } } ``` For basic passthrough, replace the `Authorization` value with `Basic `. In your `.vscode/mcp.json`: ``` { "servers": { "infrahub_mcp": { "type": "http", "url": "http://localhost:8001/mcp", "headers": { "Authorization": "Bearer " } } } } ``` For basic passthrough, replace the `Authorization` value with `Basic `. ## Related resources[​](#related-resources "Direct link to Related resources") --- # Claude Agent SDK The [Claude Agent SDK](https://docs.claude.com/en/api/agent-sdk/overview) is Anthropic's framework for building custom agents on top of the Claude API. It has native MCP support, so connecting to the Infrahub MCP server is a short wiring exercise. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Python 3.13+ and an Anthropic API key. * The Infrahub MCP server running on HTTP (see [Docker Compose](/mcp/guides/docker-compose.md)). * `pip install claude-agent-sdk`. ## Minimal read-only agent[​](#minimal-read-only-agent "Direct link to Minimal read-only agent") ``` import anyio from claude_agent_sdk import ( ClaudeAgentOptions, ClaudeSDKClient, create_sdk_mcp_server, ) async def main() -> None: options = ClaudeAgentOptions( mcp_servers={ "infrahub": { "type": "http", "url": "http://localhost:8001/mcp", "headers": { # Token pass-through: each user forwards their own Infrahub token "Authorization": "Bearer your-infrahub-api-token", }, }, }, system_prompt=( "You are an infrastructure analyst with access to the Infrahub MCP server. " "Always read infrahub://schema before guessing kind names. Prefer read-only " "tools. For any mutation, call propose_changes at the end." ), ) async with ClaudeSDKClient(options=options) as client: await client.query( "List every schema kind whose name starts with Dcim, then use get_nodes " "(limit 3) to sample three of them." ) async for message in client.receive_response(): print(message) if __name__ == "__main__": anyio.run(main) ``` The SDK discovers `tools/list`, `resources/list`, and `prompts/list` automatically and exposes them to Claude as structured tools. ## Using the bundled `infrahub_agent` prompt[​](#using-the-bundled-infrahub_agent-prompt "Direct link to using-the-bundled-infrahub_agent-prompt") Instead of writing a system prompt, fetch the server's own `infrahub_agent` prompt: ``` async with ClaudeSDKClient(options=options) as client: prompt = await client.get_prompt("mcp__infrahub__infrahub_agent") await client.query( f"{prompt}\n\nList the five most populated device roles.", ) ``` The prompt adapts to read-only mode at registration time — if `INFRAHUB_MCP_READ_ONLY=true` on the server, the prompt omits the write workflow entirely. ## Multi-tenant: per-user tokens[​](#multi-tenant-per-user-tokens "Direct link to Multi-tenant: per-user tokens") For a service that serves many users, rebuild `ClaudeAgentOptions` per request with the user's token: ``` async def handle_user_request(user_token: str, question: str) -> str: options = ClaudeAgentOptions( mcp_servers={ "infrahub": { "type": "http", "url": "http://mcp.internal:8001/mcp", "headers": {"Authorization": f"Bearer {user_token}"}, }, }, system_prompt="Answer using Infrahub MCP tools.", ) async with ClaudeSDKClient(options=options) as client: await client.query(question) result_parts: list[str] = [] async for message in client.receive_response(): if hasattr(message, "content"): result_parts.append(str(message.content)) return "\n".join(result_parts) ``` Set `INFRAHUB_MCP_AUTH_MODE=token-passthrough` on the server (see [Authentication setup](/mcp/getting-started/authentication.md#token-passthrough)). ## Production considerations[​](#production-considerations "Direct link to Production considerations") * **Write guardrails** — set `INFRAHUB_MCP_AUTH_SCOPES_WRITE=infrahub:write` under OIDC so users need an explicit scope to mutate Infrahub. * **Audit** — the SDK captures tool-call traces; the server logs every call with request IDs. Correlate both for forensics. * **Timeouts** — tune `INFRAHUB_TIMEOUT` for slow Infrahub installations so the agent gets a clear error instead of hanging. * **Caching** — enable `INFRAHUB_MCP_CACHE_ENABLED=true` to cache schema lookups; this is a big win for agents that iterate. ## Related reading[​](#related-reading "Direct link to Related reading") * [Safe changes via branch isolation](/mcp/use-cases/safe-changes-branch-isolation.md) * [Authentication setup](/mcp/getting-started/authentication.md) * [Methods reference](/mcp/references/methods.md) --- # Claude Code [Claude Code](https://claude.com/claude-code) is Anthropic's CLI/IDE agent. It auto-detects `.mcp.json` files at project roots and loads the servers on startup — a perfect fit for the Infrahub MCP server when you're working inside a specific Infrahub consumer repository (IaC, Ansible, Python clients, etc.). ## Quick start[​](#quick-start "Direct link to Quick start") See the [Installation guide](/mcp/guides/installation.md#add-to-your-mcp-client) for the minimal Claude Code configuration snippet. This page covers Claude Code-specific workflows. ## Project-scoped setup[​](#project-scoped-setup "Direct link to Project-scoped setup") Create `.mcp.json` at the root of the repository where you want the Infrahub server available: ``` { "mcpServers": { "infrahub": { "command": "uvx", "args": ["infrahub-mcp"], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000", "INFRAHUB_API_TOKEN": "your-api-token" } } } } ``` Claude Code loads the server on the next session. Run `claude` in the project directory and you should see `infrahub` listed under **/mcp**. ## Keeping the token out of `.mcp.json`[​](#keeping-the-token-out-of-mcpjson "Direct link to keeping-the-token-out-of-mcpjson") `.mcp.json` is usually committed to your repository, so it should not contain secrets. Instead, keep only the non-secret `INFRAHUB_ADDRESS` in its `env:` block and put the token in a `.env` file (kept out of version control) in the same directory. `.mcp.json` (committed): ``` { "mcpServers": { "infrahub": { "command": "uvx", "args": ["infrahub-mcp"], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000" } } } } ``` `.env` (add it to `.gitignore`; place it in the directory Claude Code launches from): ``` INFRAHUB_API_TOKEN=your-token-here # or, instead of a token: # INFRAHUB_USERNAME=admin # INFRAHUB_PASSWORD=infrahub ``` At server startup the server loads exactly four keys from `.env`: `INFRAHUB_ADDRESS`, `INFRAHUB_API_TOKEN`, `INFRAHUB_USERNAME`, and `INFRAHUB_PASSWORD`. Every other key is ignored, including other Infrahub SDK settings such as `INFRAHUB_PROXY` and `INFRAHUB_TLS_INSECURE`, and the `INFRAHUB_MCP_*` server settings (set those in the real environment or the `.mcp.json` `env:` block). Values already set in the real environment take precedence, so `.env` never overrides an explicitly set variable. Set `INFRAHUB_MCP_ENV_FILE=/path/to/file` to load a different file (`~` is expanded), or `INFRAHUB_MCP_ENV_FILE=` (empty) to disable loading. Use either a token or a username and password, not both. If `.env` and the `.mcp.json` `env:` block each supply one of the two, the server stops at startup with a message naming the conflict. In the HTTP `token-passthrough` and `basic-passthrough` modes the Infrahub credential comes from the request `Authorization` header, so a token in `.env` is ignored; `.env` still supplies `INFRAHUB_ADDRESS`. ## User-scoped setup[​](#user-scoped-setup "Direct link to User-scoped setup") For always-on access across every project, add the same block to `~/.claude/settings.json` instead. The server is spawned once per Claude Code session. ## Using the bundled prompt[​](#using-the-bundled-prompt "Direct link to Using the bundled prompt") Invoke the `infrahub_agent` system prompt directly: ``` /infrahub_agent ``` The prompt teaches Claude the schema-discovery pattern, the session-branch write workflow, and when to use each tool. It dynamically reflects read-only mode — if the server is running with `INFRAHUB_MCP_READ_ONLY=true`, the prompt tells the model that write paths are unavailable. ## CLAUDE.md conventions[​](#claudemd-conventions "Direct link to CLAUDE.md conventions") If you're adding the MCP server to a shared repository, document the expected workflow in `CLAUDE.md` so collaborators' sessions behave consistently. Example snippet: ``` ## Infrahub MCP This repo has the Infrahub MCP server wired up via `.mcp.json`. Claude should: 1. Read `infrahub://schema` before guessing kind names. 2. Use `search_nodes` for human-entered identifiers. 3. Always call `propose_changes` after writes — never merge yourself. 4. Prefer read-only tools unless the user explicitly asks for a mutation. ``` ## Remote server[​](#remote-server "Direct link to Remote server") For a shared HTTP deployment, replace the stdio block with: ``` { "mcpServers": { "infrahub": { "transport": "streamable-http", "url": "http://mcp.internal:8001/mcp" } } } ``` Combine with [token pass-through](/mcp/getting-started/authentication.md#token-passthrough) or [OIDC](/mcp/getting-started/authentication.md#oidc) for multi-user deployments. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") | Symptom | Fix | | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | | `.mcp.json` not picked up | Claude Code requires the file at the project root. Confirm with `ls .mcp.json` in the directory where you launched Claude. | | "Unknown tool" errors | Cache stale; restart Claude Code to re-fetch `tools/list`. | | Permissions dialog on every call | Pre-approve write tools in `.claude/settings.json` or use the `/allow` command. | ## Related reading[​](#related-reading "Direct link to Related reading") * [Your first agent run](/mcp/getting-started/first-agent-run.md) * [Make a change through an agent](/mcp/getting-started/make-a-change.md) * [Methods reference](/mcp/references/methods.md) --- # Claude Desktop Claude Desktop is the default starting point for most users — it speaks MCP over stdio and can spawn the Infrahub MCP server locally without any infrastructure. ## Quick start[​](#quick-start "Direct link to Quick start") See the [Installation guide](/mcp/guides/installation.md#add-to-your-mcp-client) for the minimal Claude Desktop configuration snippet. This page covers Claude Desktop-specific details. ## Configure[​](#configure "Direct link to Configure") Open **Settings → Developer**, then click `Edit config`. Claude Desktop opens `claude_desktop_config.json`. Add an entry under `mcpServers`: ``` { "mcpServers": { "infrahub_mcp": { "transport": "stdio", "command": "uvx", "args": ["infrahub-mcp"], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000", "INFRAHUB_API_TOKEN": "your-api-token" } } } } ``` Save and restart Claude Desktop. The `infrahub_mcp` server appears under **Developer → MCP servers** with a green status indicator once the handshake succeeds. ## Connect to a remote server[​](#connect-to-a-remote-server "Direct link to Connect to a remote server") If the MCP server runs in Docker or on a remote host, use the Streamable HTTP transport: ``` { "mcpServers": { "infrahub_mcp": { "transport": "streamable-http", "url": "http://mcp.example.com:8001/mcp" } } } ``` For multi-user deployments, combine this with [token pass-through](/mcp/getting-started/authentication.md#token-passthrough) — each user's Claude Desktop sends its own Infrahub token in the `Authorization` header. ## Try it[​](#try-it "Direct link to Try it") In a new conversation, attach the `infrahub_agent` prompt (available under the **Prompts** picker). Try: > List every schema kind whose name starts with `Dcim`. Or with writes enabled: > Create a new `DcimInterface` named `Ethernet99` on `spine-03` and open a proposed change for review. See [Your first agent run](/mcp/getting-started/first-agent-run.md) for the full walkthrough. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") | Symptom | Fix | | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | | Server shows red status | Check logs at **Developer → Open Logs Folder** — the server prints startup errors there. | | `command not found: uvx` | Claude Desktop doesn't inherit your shell's PATH. Use an absolute path to `uvx` in `command`. | | Connection keeps dropping | Enable [ping middleware](/mcp/references/configuration.md#client-compatibility) — set `INFRAHUB_MCP_PING_INTERVAL_MS=30000`. | | No tools appear in the UI | The server is starting but failing to register. Check `INFRAHUB_MCP_LOG_LEVEL=debug` for the full traceback. | ## Related reading[​](#related-reading "Direct link to Related reading") * [Your first agent run](/mcp/getting-started/first-agent-run.md) * [Authentication setup](/mcp/getting-started/authentication.md) --- # Cursor [Cursor](https://cursor.com) supports MCP servers via its **Tools & Integrations** panel. This page shows the Infrahub MCP server configuration and notes specific to Cursor's tool-approval UX. ## Quick start[​](#quick-start "Direct link to Quick start") See the [Installation guide](/mcp/guides/installation.md#add-to-your-mcp-client) for the minimal Cursor configuration snippet. This page covers Cursor-specific UX tuning beyond the basics. ## Configure[​](#configure "Direct link to Configure") 1. Open **Settings → Cursor Settings → Tools & Integrations**. 2. Under **MCP tools**, click **Add Custom MCP**. 3. Paste the configuration below into `mcp.json`: ``` { "mcpServers": { "infrahub_mcp": { "command": "uvx", "args": ["infrahub-mcp"], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000", "INFRAHUB_API_TOKEN": "your-api-token" } } } } ``` 4. Save the file. Restart Cursor if prompted. The server appears under **MCP tools** once Cursor starts it. Click the entry to expand tool/resource details. ## Remote (HTTP) transport[​](#remote-http-transport "Direct link to Remote (HTTP) transport") For a shared MCP server deployment: ``` { "mcpServers": { "infrahub_mcp": { "type": "streamable-http", "url": "http://localhost:8001/mcp" } } } ``` ## Cursor's tool-approval model[​](#cursors-tool-approval-model "Direct link to Cursor's tool-approval model") Cursor prompts the user before every tool call by default, which is useful during onboarding but noisy for chatty read-only flows. Two ways to tune this: * **Auto-approve read tools** — in **Settings → MCP tools → infrahub\_mcp**, whitelist `get_schema`, `get_nodes`, `search_nodes`, `query_graphql`. * **Keep manual approval for writes** — leave `node_upsert`, `node_delete`, `propose_changes`, `mutate_graphql` on interactive approval so a human explicitly confirms each mutation. This matches Cursor's default for destructive actions. For a server-side hard lock, set `INFRAHUB_MCP_READ_ONLY=true` and Cursor will only ever see read tools. ## Workspace rules[​](#workspace-rules "Direct link to Workspace rules") Add a Cursor rule (`.cursor/rules/infrahub.mdc`) to bake the agent workflow into Cursor's system prompt: ``` When the Infrahub MCP server is connected: - Read infrahub://schema before guessing kind names - Use get_nodes or search_nodes before query_graphql - Always follow node_upsert / node_delete with propose_changes - Ask before calling write tools even if auto-approve is on ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") | Symptom | Fix | | ----------------------------------------- | -------------------------------------------------------------------- | | Cursor shows `Failed to start MCP server` | Check Cursor's MCP log (**Output → MCP**) for the exact stderr. | | Tool descriptions look truncated | Some Cursor versions truncate MCP responses — upgrade to the latest. | | `$ref` errors in tool schemas | Enable `INFRAHUB_MCP_DEREFERENCE_SCHEMAS=true` to inline references. | ## Related reading[​](#related-reading "Direct link to Related reading") * [Your first agent run](/mcp/getting-started/first-agent-run.md) * [Configuration reference](/mcp/references/configuration.md) --- # OpenAI Agents SDK The [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) has first-class support for MCP servers. This page shows how to build a custom infrastructure-automation agent that talks to the Infrahub MCP server over Streamable HTTP. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Python 3.13+ and an OpenAI API key. * The Infrahub MCP server running on HTTP (see [Docker Compose](/mcp/guides/docker-compose.md) or the [installation guide](/mcp/getting-started/installation.md) for the `--transport streamable-http` path). * `pip install openai-agents httpx`. ## Minimal read-only agent[​](#minimal-read-only-agent "Direct link to Minimal read-only agent") ``` import asyncio from agents import Agent, Runner from agents.mcp import MCPServerStreamableHttp async def main() -> None: async with MCPServerStreamableHttp( name="infrahub", params={ "url": "http://localhost:8001/mcp", # Token pass-through: forward the user's Infrahub API token "headers": { "Authorization": "Bearer your-infrahub-api-token", }, }, ) as infrahub_server: agent = Agent( name="Infra Analyst", instructions=( "You are a network analyst with read-only access to Infrahub " "via MCP. Always read infrahub://schema before guessing kind " "names. Prefer get_nodes and search_nodes over query_graphql " "for simple lookups." ), mcp_servers=[infrahub_server], ) result = await Runner.run( agent, "List every schema kind whose name starts with Dcim, and pick three " "to sample with get_nodes (limit 3).", ) print(result.final_output) if __name__ == "__main__": asyncio.run(main()) ``` The SDK handles `tools/list`, `resources/list`, and translates MCP tool calls into the OpenAI tool-call format automatically. You get structured tool-call traces in the result object for auditing. ## Adding the built-in system prompt[​](#adding-the-built-in-system-prompt "Direct link to Adding the built-in system prompt") The server ships an `infrahub_agent` prompt that encodes the expected workflow. Fetch it and use it as your `instructions` override: ``` prompt_response = await infrahub_server.get_prompt("infrahub_agent", {}) agent = Agent( name="Infra Agent", instructions=prompt_response.messages[0].content.text, mcp_servers=[infrahub_server], ) ``` The prompt dynamically reflects the server's read-only mode. If the server runs with `INFRAHUB_MCP_READ_ONLY=true`, the prompt tells the agent that write tools are unavailable. ## Multi-tenant: per-user tokens[​](#multi-tenant-per-user-tokens "Direct link to Multi-tenant: per-user tokens") For a web app where each authenticated user has their own Infrahub API token, forward the token per request instead of hardcoding it: ``` async def handle_request(user_token: str, question: str) -> str: async with MCPServerStreamableHttp( name="infrahub", params={ "url": "http://mcp.internal:8001/mcp", "headers": {"Authorization": f"Bearer {user_token}"}, }, ) as server: agent = Agent( name="Infra Agent", instructions="Use Infrahub tools to answer the user's question.", mcp_servers=[server], ) result = await Runner.run(agent, question) return result.final_output ``` The MCP server must be configured with `INFRAHUB_MCP_AUTH_MODE=token-passthrough`. See [Authentication setup](/mcp/getting-started/authentication.md#token-passthrough). ## Production considerations[​](#production-considerations "Direct link to Production considerations") * **Rate limit** — set `INFRAHUB_MCP_RATE_LIMIT_RPS` on the server to cap per-client load. * **Retry** — enable server-side retries via `INFRAHUB_MCP_RETRY_MAX_ATTEMPTS=3` for transient network errors. * **Observability** — scrape `/metrics` (JSON or Prometheus) to track tool-call rates and error budgets. * **Audit** — with OIDC enabled, every call is logged with the authenticated user. See the [Authentication architecture](/mcp/references/authentication.md#audit-logs). ## Related reading[​](#related-reading "Direct link to Related reading") * [Docker Compose deployment](/mcp/guides/docker-compose.md) — to run the MCP server for your SDK client. * [Authentication setup](/mcp/getting-started/authentication.md) * [Methods reference](/mcp/references/methods.md) --- # VS Code VS Code supports MCP through its extension API — the Infrahub MCP server shows up as a tool provider alongside Copilot. This page covers the `code --add-mcp` setup and the VS Code Copilot compatibility flag. ## Quick start[​](#quick-start "Direct link to Quick start") See the [Installation guide](/mcp/guides/installation.md#add-to-your-mcp-client) for the minimal VS Code configuration snippet. This page covers VS Code-specific details and the Copilot compatibility flag. ## Register via CLI[​](#register-via-cli "Direct link to Register via CLI") The fastest way to register the server is with VS Code's `--add-mcp` flag: ``` code --add-mcp '{ "type": "stdio", "name": "infrahub-mcp", "description": "MCP server to interact with Infrahub", "command": "uvx", "args": ["infrahub-mcp"], "env": { "INFRAHUB_ADDRESS": "http://localhost:8000", "INFRAHUB_API_TOKEN": "placeholder-token" } }' ``` Then open the MCP user configuration (**Cmd/Ctrl+Shift+P → MCP: Open User Configuration**) and replace `placeholder-token` with your real API token. ## Remote (HTTP) transport[​](#remote-http-transport "Direct link to Remote (HTTP) transport") ``` { "servers": { "infrahub_mcp": { "type": "http", "url": "http://localhost:8001/mcp" } } } ``` ## VS Code Copilot compatibility[​](#vs-code-copilot-compatibility "Direct link to VS Code Copilot compatibility") VS Code Copilot Chat does not follow JSON Schema `$ref` references in tool definitions. If you see errors about unresolved schemas, enable `$ref` inlining on the server: ``` export INFRAHUB_MCP_DEREFERENCE_SCHEMAS=true ``` This makes every tool definition self-contained. There's no downside for clients that do handle `$ref` — the schemas just become more verbose. ## Using the tools[​](#using-the-tools "Direct link to Using the tools") Once registered, MCP tools show up alongside Copilot's native tools. In Copilot Chat, reference them like any other tool: > Use the Infrahub server to list all `DcimDevice` kinds and summarize their roles. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") | Symptom | Fix | | ----------------------- | ------------------------------------------------------------------------------- | | MCP command not found | Update VS Code — MCP support lands in recent versions. | | `$ref` schema errors | Set `INFRAHUB_MCP_DEREFERENCE_SCHEMAS=true`. | | Server shows as stopped | Right-click the extension, choose **Start Server**, and check the Output panel. | ## Related reading[​](#related-reading "Direct link to Related reading") * [Configuration reference — Client compatibility](/mcp/references/configuration.md#client-compatibility) * [Your first agent run](/mcp/getting-started/first-agent-run.md) --- # Authentication architecture For the step-by-step configuration recipes, see [Set up authentication](/mcp/getting-started/authentication.md). This page explains *how* each mode works end-to-end. The MCP server sits between AI clients (Claude, Cursor, VS Code Copilot) and the Infrahub API. Authentication happens at two separate layers: 1. **MCP layer** — controls *who* can reach the MCP server and *what* they can do (read-only vs read-write). Configured via `INFRAHUB_MCP_AUTH_MODE`. 2. **Infrahub layer** — controls *what* the MCP server can do in Infrahub. Configured via `INFRAHUB_API_TOKEN` or `INFRAHUB_USERNAME`/`INFRAHUB_PASSWORD` (for `none` and `oidc` modes), or via the client's own credentials (for `token-passthrough` and `basic-passthrough` modes). In `none` and `oidc` modes, the Infrahub credentials are shared across all MCP sessions — they define the ceiling of what any client can do, and the MCP auth layer narrows that ceiling per-user. In `token-passthrough` and `basic-passthrough` modes, each client provides their own Infrahub credentials, so there are no shared credentials. ## Auth modes[​](#auth-modes "Direct link to Auth modes") ### `none` — shared credentials, no identity[​](#none--shared-credentials-no-identity "Direct link to none--shared-credentials-no-identity") No identity gate at the MCP level. Any client that can reach the server gets access using the shared Infrahub credentials. This is the right choice when: * You are running the server locally via **stdio** transport (single-user, no network exposure). * The server is behind a VPN or network-level access control and you trust all clients equally. * You want the simplest possible setup. No additional configuration is required beyond the standard Infrahub connection variables. ### `oidc` — identity + scopes via external IdP[​](#oidc--identity--scopes-via-external-idp "Direct link to oidc--identity--scopes-via-external-idp") External Identity Provider authentication via OpenID Connect. When enabled, the MCP server delegates authentication to a provider like Google, Okta, or Microsoft Entra using FastMCP's `OIDCProxy`. This is the right choice when: * Multiple users share a single MCP server instance over HTTP. * You need **audit trails** that identify who performed each action. * You want **role-based access control** — some users get read-only, others get read-write. * Compliance requires identity verification before infrastructure changes. OIDC mode requires the **Streamable HTTP transport** — it is not supported over stdio. ### `token-passthrough` — client's own Infrahub token[​](#token-passthrough--clients-own-infrahub-token "Direct link to token-passthrough--clients-own-infrahub-token") Each MCP client provides their own Infrahub API token in the HTTP request header. The server extracts it and creates a per-request `InfrahubClient` using that token — no shared server-side credentials needed. This is the right choice when: * Each user already has their own Infrahub API token. * You want **end-to-end credential isolation** — the server never stores or shares tokens between users. * You need a simpler setup than OIDC but still want multi-user support. * You want Infrahub's native RBAC to control what each user can do. Token passthrough is **fail-closed**: every request must carry a valid token. If the header is missing or empty, the request is rejected — there is no silent fallback to server-side credentials. Token passthrough requires the **Streamable HTTP transport** — it is not supported over stdio (no HTTP headers available). The server rejects this combination at startup. ### `basic-passthrough` — client's own Infrahub username + password[​](#basic-passthrough--clients-own-infrahub-username--password "Direct link to basic-passthrough--clients-own-infrahub-username--password") Same fail-closed per-request model as `token-passthrough`, but clients authenticate with their Infrahub username and password via the standard HTTP `Basic` scheme. Each request constructs a fresh `InfrahubClient` — credentials are never cached or shared between requests. Use this mode when clients hold Infrahub user accounts rather than API tokens. Like `token-passthrough`, this mode requires the **Streamable HTTP transport**. ## User identity and branch naming[​](#user-identity-and-branch-naming "Direct link to User identity and branch naming") When OIDC is enabled, the authenticated user identity flows into two features: ### Audit logs[​](#audit-logs "Direct link to Audit logs") Every tool call and resource read includes the user identity in structured log output: ``` INFO tool_call tool=get_nodes user=alice@example.com INFO resource_read uri=infrahub://schema user=alice@example.com ``` The claim used for identity is configurable via `INFRAHUB_MCP_OIDC_USER_CLAIM` (default: `email`). Common alternatives include `sub` (subject ID) or `preferred_username`. ### Branch placeholder[​](#branch-placeholder "Direct link to Branch placeholder") The `{user}` placeholder in `INFRAHUB_MCP_BRANCH_PATTERN` resolves to the authenticated user's identity, sanitized for git ref compatibility: ``` INFRAHUB_MCP_BRANCH_PATTERN=mcp/{user}/{date}-{hex} # Produces: mcp/alice-example.com/20260409-a1b2c3d4 ``` The sanitization follows `git check-ref-format` rules: characters in the set `[a-zA-Z0-9._/-]` (letters, digits, dot, underscore, slash, and dash) are preserved — all others are replaced with hyphens. For example, `alice@example.com` becomes `alice-example.com` because the `@` is replaced while the dot is preserved. Additionally, `..` sequences are collapsed to a single dot, `//` to a single slash, `/.` components are stripped (no component may start with a dot), and a trailing `.lock` suffix is removed. Leading/trailing dots, slashes, and hyphens are trimmed. If no user is available, `anonymous` is used. ## Read-only mode[​](#read-only-mode "Direct link to Read-only mode") Independent of auth mode, the server supports a hard read-only mode: ``` INFRAHUB_MCP_READ_ONLY=true ``` This provides two layers of protection: 1. **Registration time** — write tools are not mounted on the FastMCP server, so they do not appear in `tools/list`. 2. **Middleware time** — `ReadOnlyMiddleware` filters any tool tagged `write` from discovery and rejects calls, catching hardcoded tool names that bypass discovery. The `infrahub_agent` system prompt dynamically reflects the access mode, telling the AI agent that write operations are unavailable. Read-only mode and OIDC scope-based authorization can be combined. Use read-only mode for server-wide enforcement (for example, a monitoring-only deployment) and scopes for per-user access control. ## How auth modes interact with transports[​](#how-auth-modes-interact-with-transports "Direct link to How auth modes interact with transports") | Transport | `none` | `oidc` | `token-passthrough` | `basic-passthrough` | | ------------------- | --------------------------------------------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------- | | **stdio** | Full access via shared credentials. | Not supported — no HTTP headers. | Not supported — no HTTP headers. | Not supported — no HTTP headers. | | **Streamable HTTP** | Full access via shared credentials. No identity tracking. | Full OIDC flow. Identity in audit logs. Scope-based write gating. | Per-request token. Each client uses their own Infrahub credentials. | Per-request username/password. Each client uses their own Infrahub credentials. | --- # Configuration reference The Infrahub MCP server is configured entirely through environment variables. All middleware features are **opt-in** — the server works out of the box with only the required connection variables set. ## Connection[​](#connection "Direct link to Connection") | Variable | Description | Default | | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------ | | `INFRAHUB_ADDRESS` | URL of your Infrahub instance | **required** | | `INFRAHUB_API_TOKEN` | API token used by the MCP server to call Infrahub (not needed in `token-passthrough` auth mode) | — | | `INFRAHUB_MCP_ENV_FILE` | Path to a `.env` file (`~` is expanded) whose Infrahub connection keys are loaded at startup: `INFRAHUB_ADDRESS`, `INFRAHUB_API_TOKEN`, `INFRAHUB_USERNAME`, `INFRAHUB_PASSWORD`, and nothing else (real env vars win). A missing default file is ignored; set to empty to disable loading | `./.env` | | `MCP_HOST` | Bind address for the HTTP server | `0.0.0.0` | | `MCP_PORT` | Port for the HTTP server | `8001` | Username/password authentication against Infrahub is not supported. Use [OIDC](/mcp/getting-started/authentication.md#oidc) when you need per-user identity. ## Server behavior[​](#server-behavior "Direct link to Server behavior") | Variable | Description | Default | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | | `INFRAHUB_MCP_READ_ONLY` | Disable all write operations (`true`, `1`, or `yes`) | `false` | | `INFRAHUB_MCP_BRANCH_PATTERN` | Branch naming pattern. Supports `{date}`, `{hex}`, `{user}` placeholders. Fixed names (no placeholders) must not already exist. | `mcp/session-{date}-{hex}` | | `INFRAHUB_MCP_MAX_BRANCH_RETRIES` | Max collision retries when generating branch names (1–20) | `5` | | `INFRAHUB_MCP_LOG_LEVEL` | Set to `debug` for verbose logging with tracebacks | `info` | ## Rate limiting[​](#rate-limiting "Direct link to Rate limiting") Token bucket rate limiter that protects the server from abuse. Disabled by default. | Variable | Description | Default | | ------------------------------- | -------------------------------------------------- | ------- | | `INFRAHUB_MCP_RATE_LIMIT_RPS` | Max sustained requests per second (`0` = disabled) | `0` | | `INFRAHUB_MCP_RATE_LIMIT_BURST` | Token bucket burst capacity (`0` = auto: 2x RPS) | `0` | ``` # Allow 10 requests/sec with bursts up to 20 INFRAHUB_MCP_RATE_LIMIT_RPS=10 INFRAHUB_MCP_RATE_LIMIT_BURST=20 ``` ## Retry[​](#retry "Direct link to Retry") Automatic retry with exponential backoff for transient failures (`ConnectionError`, `TimeoutError`, `OSError`). Disabled by default. | Variable | Description | Default | | --------------------------------- | ---------------------------------------- | ------- | | `INFRAHUB_MCP_RETRY_MAX_ATTEMPTS` | Max retry attempts (`0` = disabled) | `0` | | `INFRAHUB_MCP_RETRY_BASE_DELAY` | Initial delay between retries in seconds | `1.0` | ## Response caching[​](#response-caching "Direct link to Response caching") TTL-based caching for schema discovery and list operations. Reduces load on Infrahub for repeated schema lookups. | Variable | Description | Default | | ----------------------------- | -------------------------------------------------------------- | ------- | | `INFRAHUB_MCP_CACHE_ENABLED` | Enable response caching (`true`/`false`) | `false` | | `INFRAHUB_MCP_CACHE_LIST_TTL` | TTL for list operations (tools, resources, prompts) in seconds | `300` | | `INFRAHUB_MCP_CACHE_READ_TTL` | TTL for read operations (resources, tool calls) in seconds | `3600` | Only the `get_schema` tool is cached for tool calls — node queries and writes are never cached. ## Authentication and authorization[​](#authentication-and-authorization "Direct link to Authentication and authorization") The MCP server supports four authentication modes controlled by `INFRAHUB_MCP_AUTH_MODE`: ### `none` (default)[​](#none-default "Direct link to none-default") No identity gate. Any client that reaches the MCP server gets access using the shared `INFRAHUB_API_TOKEN`. Works with both stdio and HTTP transports. ### `oidc`[​](#oidc "Direct link to oidc") External Identity Provider authentication via OpenID Connect. When enabled, the MCP server uses FastMCP's OIDCProxy to authenticate users against providers like Google, Okta, or Microsoft Entra. This provides: * **Identity** — who is calling (for audit logs and `{user}` branch placeholder) * **Role gating** — two roles: read-only (read tools only) and read-write (all tools) Infrahub API calls still use the shared env var credentials. The OIDC token is for MCP-level access control only. Streamable HTTP transport is required (OIDC is not supported over stdio). ### `token-passthrough`[​](#token-passthrough "Direct link to token-passthrough") Each client sends their own Infrahub API token in the HTTP request header. The server creates a per-request `InfrahubClient` using that token — no shared server-side credentials required. Every request must carry a valid token (fail-closed). Streamable HTTP transport is required. ### `basic-passthrough`[​](#basic-passthrough "Direct link to basic-passthrough") Same fail-closed per-request model as `token-passthrough`, but the client sends an `Authorization: Basic ` header. The server decodes the header and constructs a fresh `InfrahubClient` with those credentials for each request. Streamable HTTP transport is required. | Variable | Required | Default | Description | | --------------------------------------- | ---------- | --------------- | ------------------------------------------------------------------------------ | | `INFRAHUB_MCP_AUTH_MODE` | No | `none` | Authentication mode (`none`, `oidc`, `token-passthrough`, `basic-passthrough`) | | `INFRAHUB_MCP_OIDC_CONFIG_URL` | Yes (oidc) | | OIDC discovery endpoint URL | | `INFRAHUB_MCP_OIDC_CLIENT_ID` | Yes (oidc) | | OAuth client ID registered with the IdP | | `INFRAHUB_MCP_OIDC_CLIENT_SECRET` | No | | OAuth client secret (omit for PKCE flow) | | `INFRAHUB_MCP_OIDC_BASE_URL` | Yes (oidc) | | Public URL where the MCP server is accessible | | `INFRAHUB_MCP_OIDC_AUDIENCE` | No | | Token audience claim | | `INFRAHUB_MCP_OIDC_USER_CLAIM` | No | `email` | JWT claim used for user identity | | `INFRAHUB_MCP_AUTH_SCOPES_WRITE` | No | `write` | Scopes required for write operations (comma-separated) | | `INFRAHUB_MCP_TOKEN_PASSTHROUGH_HEADER` | No | `Authorization` | HTTP header carrying the Infrahub API token (token-passthrough mode) | ### Scope-based write authorization[​](#scope-based-write-authorization "Direct link to Scope-based write authorization") When `INFRAHUB_MCP_AUTH_SCOPES_WRITE` is configured (in either auth mode over HTTP transport), tools tagged `write` are filtered from discovery and blocked at call time unless the access token contains all required scopes. STDIO connections skip auth checks. ``` # Require "infrahub:write" scope for node_upsert, node_delete, etc. INFRAHUB_MCP_AUTH_SCOPES_WRITE=infrahub:write ``` ## Observability[​](#observability "Direct link to Observability") ### OpenTelemetry tracing[​](#opentelemetry-tracing "Direct link to OpenTelemetry tracing") Wraps every MCP request in an OpenTelemetry span (`mcp.`). Requires the `opentelemetry-api` package — degrades gracefully to a no-op if not installed. | Variable | Description | Default | | --------------------------- | ---------------------------- | ------- | | `INFRAHUB_MCP_OTEL_ENABLED` | Enable OpenTelemetry tracing | `false` | ### Prometheus metrics[​](#prometheus-metrics "Direct link to Prometheus metrics") The `/metrics` endpoint returns JSON by default. Enable Prometheus exposition format for scraping by Prometheus, Grafana Agent, or Datadog. | Variable | Description | Default | | --------------------------------- | --------------------------------------------- | ------- | | `INFRAHUB_MCP_PROMETHEUS_ENABLED` | Return Prometheus text format from `/metrics` | `false` | Metrics exposed: * `infrahub_mcp_requests_total{method}` — total requests by MCP method * `infrahub_mcp_errors_total{method}` — total errors by MCP method * `infrahub_mcp_latency_ms_total{method}` — cumulative latency in milliseconds The JSON format also includes `error_stats` (per-exception-type counts from error handling) and `cache` statistics when caching is enabled. ## Client compatibility[​](#client-compatibility "Direct link to Client compatibility") | Variable | Description | Default | | ---------------------------------- | ------------------------------------------------------------------------------------------------------ | ------- | | `INFRAHUB_MCP_DEREFERENCE_SCHEMAS` | Inline `$ref` in JSON schemas for clients that do not handle references (for example, VS Code Copilot) | `false` | | `INFRAHUB_MCP_PING_INTERVAL_MS` | Send periodic pings to keep HTTP connections alive, in milliseconds (`0` = disabled, max `300000`) | `0` | ## Middleware stack[​](#middleware-stack "Direct link to Middleware stack") The server applies middleware in registration order. The full stack when all features are enabled: | Layer | Middleware | Always on | | ----- | ---------------------------------------------------------------------------------------- | --------- | | 1 | **RequestIdMiddleware** — unique correlation ID per request | yes | | 2 | **MetricsMiddleware** — request/error/latency counters for `/metrics` | yes | | 3 | **OTelTracingMiddleware** — OpenTelemetry spans | no | | 4 | **ErrorHandlingMiddleware** — exception-to-MCP-error mapping with error tracking | yes | | 5 | **InfrahubConnectionMiddleware** — friendly errors for unreachable/unresponsive Infrahub | yes | | 6 | **RetryMiddleware** — exponential backoff for transient failures | no | | 7 | **RateLimitingMiddleware** — token bucket rate limiting | no | | 8 | **StructuredLoggingMiddleware** — JSON logs with token estimates | yes | | 9 | **DetailedTimingMiddleware** — per-operation timing breakdown | yes | | 10 | **AuthMiddleware** — scope-based write tool authorization (HTTP only) | no | | 11 | **TokenPassthroughMiddleware** — fail-closed gate for token/basic passthrough modes | no | | 12 | **ReadOnlyMiddleware** — tag-based write tool blocking | no | | 13 | **AuditMiddleware** — structured audit trail for tool/resource access | yes | | 14 | **ResponseCachingMiddleware** — TTL-based response caching | no | | 15 | **DereferenceRefsMiddleware** — inline `$ref` in JSON schemas | no | | 16 | **PingMiddleware** — periodic keepalive pings | no | | 17 | **ResponseLimitingMiddleware** — truncates responses over 500 KB | yes | ## Endpoints[​](#endpoints "Direct link to Endpoints") | Path | Method | Description | | ---------------- | ------ | -------------------------------------------------------------------- | | `/mcp` | POST | MCP Streamable HTTP transport | | `/health` | GET | Health check — returns `{"status": "healthy"}` or 503 | | `/metrics` | GET | Request counts, error counts, latency (JSON or Prometheus format) | | `/.well-known/*` | GET | OAuth discovery — returns RFC 6749 JSON 404 when OIDC is not enabled | --- # Infrahub MCP methods ## Response formats[​](#response-formats "Direct link to Response formats") Structured array responses (schema details, node attributes) are encoded in **TOON** (Token-Oriented Object Notation) to reduce token usage by 33–45% compared to JSON. TOON declares field names once in a header and lists rows as CSV: ``` items[N]{field1,field2,field3}: value1,value2,value3 value1,value2,value3 ``` Scalar fields use standard `key: value` notation. Flat string maps (the schema catalog, branch lists) are returned as compact JSON where TOON offers no benefit. ## Schema[​](#schema "Direct link to Schema") ### get\_schema[​](#get_schema "Direct link to get_schema") #### Capabilities[​](#capabilities "Direct link to Capabilities") * read-only * idempotent * no destroy Discover available schema kinds or retrieve the full detail for a single kind. * Without `kind`: returns the catalog of all available kinds as compact JSON. * With `kind`: returns the kind's attributes, relationships, and the full set of filter keys accepted by `get_nodes` (TOON-encoded). Prefer reading the `infrahub://schema` resource if your client supports MCP resources — this tool provides the same data for clients that don't. #### Parameters[​](#parameters "Direct link to Parameters") * **kind** (string): kind to get detail for; omit to list all kinds * **branch** (string): branch to read from; default is the server's default branch * **expand** (boolean): inline one level of each relationship's peer schema; defaults to the server's `INFRAHUB_MCP_SCHEMA_EXPAND_PEERS` setting ## Nodes[​](#nodes "Direct link to Nodes") ### get\_nodes[​](#get_nodes "Direct link to get_nodes") #### Capabilities[​](#capabilities-1 "Direct link to Capabilities") * read-only * idempotent * no destroy List nodes of a specific kind — the default read path for typed queries. Supports filtering and pagination. Use `get_schema` to discover kinds and their filter keys. #### Parameters[​](#parameters-1 "Direct link to Parameters") * **kind** (string, required): kind of the objects to retrieve * **branch** (string): branch name (default: server's default branch) * **filters** (object): attribute/relationship filter map (e.g. `{"name__value": "atl1"}` or `{"site__name__value": "atl1"}`) * **partial\_match** (boolean, default: false): enable substring matches for string filters * **include\_attributes** (boolean, default: false): return full attribute values in TOON tabular format instead of just display labels * **limit** (integer, default: 50, min: -1): maximum nodes to return; `-1` returns all results * **offset** (integer, default: 0, min: 0): number of results to skip for pagination The response dict contains `nodes`, `count`, `total_count`, `has_more`, `offset`, and `limit`. `has_more` is `null` when the total count query failed and pagination cannot be confirmed. ### search\_nodes[​](#search_nodes "Direct link to search_nodes") #### Capabilities[​](#capabilities-2 "Direct link to Capabilities") * read-only * idempotent * no destroy Find nodes by partial name. Performs substring matching against the `name` attribute only (via `name__value` with `partial_match=true`). For other attributes, or combining filters, use `get_nodes` instead. #### Parameters[​](#parameters-2 "Direct link to Parameters") * **query** (string, required): partial name to search for (minimum 1 character) * **kind** (string, required): kind to search within * **branch** (string): branch name * **limit** (integer, default: 10, range: 1–100): maximum results to return ## Session[​](#session "Direct link to Session") ### get\_session\_info[​](#get_session_info "Direct link to get_session_info") #### Capabilities[​](#capabilities-3 "Direct link to Capabilities") * read-only * idempotent * no destroy Return the current MCP session state. Reports the active session branch (if any) and the Infrahub instance address. A session branch is lazily auto-created on the first write tool call and is named `mcp/session-YYYYMMDD-`. Before the first write, `session_branch` is `null` and all read tools target the default branch. ## GraphQL[​](#graphql "Direct link to GraphQL") ### query\_graphql[​](#query_graphql "Direct link to query_graphql") #### Capabilities[​](#capabilities-4 "Direct link to Capabilities") * read-only * idempotent * no destroy Execute a read-only GraphQL query against Infrahub. Mutations are rejected at the AST level — use `mutate_graphql` instead. For straightforward attribute reads, prefer `get_nodes` or `search_nodes`. #### Parameters[​](#parameters-3 "Direct link to Parameters") * **query** (string, required): GraphQL query document (mutations rejected) * **branch** (string): branch to execute against; default is the server's default branch ## Graph traversal[​](#graph-traversal "Direct link to Graph traversal") Graph traversal walks the live data graph to find how objects connect. It requires Infrahub 1.10 or later. Node arguments — `source` (both tools) and `destination` (`find_paths` only) — accept either a node UUID or a kind-qualified human-friendly ID (for example `InfraDevice__atl1-edge1`, the form `get_nodes` returns); an unresolvable human-friendly ID returns an error pointing at `get_nodes`. ### find\_paths[​](#find_paths "Direct link to find_paths") #### Capabilities[​](#capabilities-5 "Direct link to Capabilities") * read-only * idempotent * no destroy Find the shortest path(s) between two nodes. A `count` of 0 means no path within `max_depth`. Prefer this over a hand-built deep GraphQL query when you need to know how two objects are connected. #### Parameters[​](#parameters-4 "Direct link to Parameters") * **source** (string, required): start node — UUID or kind-qualified human-friendly ID * **destination** (string, required): end node — UUID or kind-qualified human-friendly ID * **branch** (string): branch to read from; default is the server's default branch * **max\_depth** (integer): maximum relationship hops to explore * **kind\_filter** (array of strings): only traverse through nodes of these kinds * **relationship\_filter** (array of strings): only follow these schema relationship identifiers ### find\_reachable[​](#find_reachable "Direct link to find_reachable") #### Capabilities[​](#capabilities-6 "Direct link to Capabilities") * read-only * idempotent * no destroy Find nodes of the given kinds reachable from a source node — impact and dependency analysis. #### Parameters[​](#parameters-5 "Direct link to Parameters") * **source** (string, required): source node — UUID or kind-qualified human-friendly ID * **target\_kinds** (array of strings, required): node kinds to search for * **branch** (string): branch to read from; default is the server's default branch * **max\_depth** (integer): maximum traversal depth * **max\_results** (integer, default: 20): maximum distinct reachable nodes returned * **shortest\_paths\_only** (boolean, default: true): return only the shortest path to each target ## Write operations[​](#write-operations "Direct link to Write operations") Write tools target the **active session branch**, auto-created on the first write of a session (`mcp/session-YYYYMMDD-`). Use `propose_changes` to open a review once your changes are ready. Write tools are hidden when the server runs in read-only mode. The session branch is tracked per session. If it has been merged or deleted, the next write automatically recovers onto a fresh branch — no server restart is needed. Use `reset_session_branch` to reset or switch branches deliberately. ### node\_upsert[​](#node_upsert "Direct link to node_upsert") #### Capabilities[​](#capabilities-7 "Direct link to Capabilities") * write * non-idempotent * non-destructive Create or update a node on the active session branch. * **Create**: omit both `id` and `hfid`. * **Update**: supply either `id` or `hfid` to identify the target node. Only scalar attribute fields are accepted in `data`. To set relationship fields, use `mutate_graphql` with an appropriate GraphQL mutation. #### Parameters[​](#parameters-6 "Direct link to Parameters") * **kind** (string, required): kind of the node to create or update * **data** (object, required): flat `{attribute: value}` map; scalar attributes only * **id** (string): UUID of an existing node to update (update mode) * **hfid** (array of string): human-friendly ID segments of an existing node (update mode) ### node\_delete[​](#node_delete "Direct link to node_delete") #### Capabilities[​](#capabilities-8 "Direct link to Capabilities") * write * non-idempotent * destructive Delete a node on the active session branch. The deletion is applied to the session branch only and is not visible on the default branch until a proposed change is merged. #### Parameters[​](#parameters-7 "Direct link to Parameters") * **kind** (string, required): kind of the node to delete * **id** (string): UUID of the node to delete * **hfid** (array of string): human-friendly ID segments of the node to delete Provide either `id` or `hfid` — both cannot be omitted. ### propose\_changes[​](#propose_changes "Direct link to propose_changes") #### Capabilities[​](#capabilities-9 "Direct link to Capabilities") * write * non-idempotent * non-destructive Open a proposed change (pull request) from the active session branch to the default branch. Creates a `CoreProposedChange` in Infrahub so a human can review, approve, and merge the changes made during this session. The session branch remains active after calling this — you can continue making changes. #### Parameters[​](#parameters-8 "Direct link to Parameters") * **title** (string, required): title for the proposed change * **description** (string): optional description explaining the motivation * **destination\_branch** (string): branch to merge into; defaults to the instance's default branch ### mutate\_graphql[​](#mutate_graphql "Direct link to mutate_graphql") #### Capabilities[​](#capabilities-10 "Direct link to Capabilities") * write * non-idempotent * potentially destructive Execute a GraphQL mutation against Infrahub — use only for complex writes that the typed tools can't express (relationship edits, bulk operations, mutations not covered by `node_upsert` / `node_delete`). The mutation always runs on the active session branch; there is no branch override. To target a different branch deliberately, switch the session with `reset_session_branch` first. Branch-management mutations (`BranchMerge`, `BranchRebase`, `BranchDelete`, …) and schema mutations are rejected — they would bypass session isolation and the review gate. #### Parameters[​](#parameters-9 "Direct link to Parameters") * **query** (string, required): GraphQL mutation to execute on the session branch ### reset\_session\_branch[​](#reset_session_branch "Direct link to reset_session_branch") #### Capabilities[​](#capabilities-11 "Direct link to Capabilities") * write * non-idempotent * non-destructive Reset or switch the active session branch for the current session. Call with no arguments to clear the cached branch so the next write creates a fresh one (for example, after merging your work). Pass a branch name to switch this session to it — the branch is created when it does not exist and the name matches the configured branch pattern. The instance default branch and merged or read-only branches are rejected. Affects only the calling session. A merged or deleted session branch is recovered automatically on the next write; this tool is the explicit override on top of that. #### Parameters[​](#parameters-10 "Direct link to Parameters") * **branch** (string): target branch; omit to reset to a fresh auto-created branch --- ## [📄️Release 1.1.7](/mcp/release-notes/release-1_1_7.md) [Trace how two infrastructure objects are connected and find what a change could reach, with two read-only graph-traversal tools backed by Infrahub 1.10.](/mcp/release-notes/release-1_1_7.md) --- # Release 1.0.0 | Release Number | 1.0.0 | | -------------- | --------------------------------------------------------------------- | | Release Type | Feature | | Release Date | March 26th, 2026 | | Tag | [v1.0.0](https://github.com/opsmill/infrahub-mcp/releases/tag/v1.0.0) | ## Release summary[​](#release-summary "Direct link to Release summary") Your AI agents can now gather Infrahub context directly through the MCP server instead of switching between GraphQL queries, the UI, and documentation. This allows AI tools to better understand relationships between objects and safely propose changes for review. ### Query Infrahub data from MCP-compatible clients[​](#query-infrahub-data-from-mcp-compatible-clients "Direct link to Query Infrahub data from MCP-compatible clients") Infrahub data is now available to AI agents through the MCP server, making it possible to retrieve objects, search relationships, and execute GraphQL queries from MCP-compatible clients. **What changed** * Retrieve infrastructure objects by kind using `get_nodes`, with support for attribute and relationship filters, partial matching, attribute selection, and limit/offset paging to narrow large datasets. * Search for infrastructure objects by partial name using `search_nodes`, making it easier to locate objects when only part of an identifier is known. * Run GraphQL read queries using `query_graphql` for cases that require more complex data retrieval. Mutations are rejected; write mutations go through `mutate_graphql` on the session branch. * Retrieve schema using `get_schema` when working with MCP clients that do not support resources. * Check which branch a write will target using `get_session_info`, which reports the active session branch (or `none` before the first write). ### Help AI understand Infrahub relationships and schema[​](#help-ai-understand-infrahub-relationships-and-schema "Direct link to Help AI understand Infrahub relationships and schema") Schema and branch information are now exposed as MCP resources, allowing AI agents to discover available kinds, attributes, relationships, and branches directly from Infrahub. **What changed** * Explore available kinds using `infrahub://schema`, allowing AI agents to discover the Infrahub data model dynamically. * Explore attributes, relationships, and filters for a specific kind using `infrahub://schema/{kind}`, making it possible to build queries from the live schema. * Access the GraphQL SDL through `infrahub://graphql-schema` when working directly with GraphQL-aware clients. * Access branch names, descriptions, and default branch information through `infrahub://branches` to understand the current branch context. ### Review and merge changes created with AI using existing Infrahub workflows[​](#review-and-merge-changes-created-with-ai-using-existing-infrahub-workflows "Direct link to Review and merge changes created with AI using existing Infrahub workflows") Changes created with AI are kept separate from the default branch and can be explicitly submitted as a Proposed Change, allowing them to follow the same review and merge process used elsewhere in Infrahub. **What changed** * Create a session branch automatically on the first write so changes remain isolated from the default branch. * Generate branch names using `INFRAHUB_MCP_BRANCH_PATTERN` to keep branch creation consistent across sessions. * Retry branch creation automatically when naming collisions occur. * Open a Proposed Change explicitly with `propose_changes`, which records the source and destination branches so reviewers can see exactly where changes will be merged. ## Upgrade notes[​](#upgrade-notes "Direct link to Upgrade notes") ### Reading schema and branches[​](#reading-schema-and-branches "Direct link to Reading schema and branches") Schema and branch information are available as MCP resources (`infrahub://schema`, `infrahub://branches`). Use `get_schema` when working with MCP clients that do not support resources. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * Branch support for GraphQL queries ([#32](https://github.com/opsmill/infrahub-mcp/pull/32)). ### Changed[​](#changed "Direct link to Changed") * Exposed schema and branches as MCP resources and adopted a branch-per-session write model ([#35](https://github.com/opsmill/infrahub-mcp/pull/35)). * Migrated integration tests to the Anthropic SDK. --- # Release 1.0.1 | Release Number | 1.0.1 | | -------------- | --------------------------------------------------------------------- | | Release Type | Improvement | | Release Date | April 7th, 2026 | | Tag | [v1.0.1](https://github.com/opsmill/infrahub-mcp/releases/tag/v1.0.1) | ## Release summary[​](#release-summary "Direct link to Release summary") Use AI agents to query Infrahub data without being interrupted by invalid queries. Error messages now provide enough context for AI agents to identify valid alternatives and retry automatically. ### Query Infrahub data without interrupting a conversation to correct invalid queries[​](#query-infrahub-data-without-interrupting-a-conversation-to-correct-invalid-queries "Direct link to Query Infrahub data without interrupting a conversation to correct invalid queries") Schema information is now included in common error paths, allowing AI agents to correct invalid queries and continue working without manual intervention. **What changed** * Return valid kinds when an unknown kind is referenced, making it easier to correct invalid requests. * Return valid filters for the requested kind so AI agents can build a corrected query automatically. * Include `get_schema` in remediation guidance when additional schema information is needed. * Apply the same behavior to read, write, and GraphQL operations. ## Minor changes[​](#minor-changes "Direct link to Minor changes") ### Security[​](#security "Direct link to Security") * Updated dependencies, including `ujson`, `requests`, `aiohttp`, and `fastmcp`. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * Schema discovery hints in MCP error messages ([#54](https://github.com/opsmill/infrahub-mcp/pull/54)). --- # Release 1.1.1 | Release Number | 1.1.1 | | -------------- | --------------------------------------------------------------------- | | Release Type | Feature | | Release Date | May 4th, 2026 | | Tag | [v1.1.1](https://github.com/opsmill/infrahub-mcp/releases/tag/v1.1.1) | ## Release summary[​](#release-summary "Direct link to Release summary") Run the MCP server as a shared service instead of a local process while preserving individual identities, controlling write access, and monitoring server health through dedicated operational endpoints. This makes it practical to deploy a single MCP server for an organization instead of asking each person to install and run their own. ### Access Infrahub using individual identities[​](#access-infrahub-using-individual-identities "Direct link to Access Infrahub using individual identities") Authentication can now be passed through to Infrahub so permissions and audit history remain associated with the person making the request instead of a shared service account. **What changed** * Configure authentication using `none`, `oidc`, `token-passthrough`, or `basic-passthrough`. * Pass user tokens through `INFRAHUB_MCP_TOKEN_PASSTHROUGH_HEADER` when using token-based authentication. * Pass Infrahub credentials through HTTP basic authentication. * Apply passthrough authentication to HTTP transports. * Reject unauthenticated requests in passthrough and OIDC modes. ### Share Infrahub data without enabling writes[​](#share-infrahub-data-without-enabling-writes "Direct link to Share Infrahub data without enabling writes") Infrahub data can now be exposed in read-only mode, making it possible to investigate, report on, and correlate data across systems without changing state. **What changed** * Remove write-tagged tools by setting `INFRAHUB_MCP_READ_ONLY=true`. * Block GraphQL mutations when read-only mode is enabled. ### Monitor the MCP server like any other service[​](#monitor-the-mcp-server-like-any-other-service "Direct link to Monitor the MCP server like any other service") Health endpoints, metrics, and middleware are now available so the MCP server can be operated and observed like any other service. **What changed** * Collect request IDs, Prometheus metrics, OpenTelemetry traces, structured logs, retries, response caching, and rate limiting through middleware. * Access `/health` for readiness checks. * Access `/metrics` for Prometheus or JSON-based monitoring. * Record audit logs for write operations. ## Upgrade notes[​](#upgrade-notes "Direct link to Upgrade notes") ### Configure authentication before exposing the server over HTTP[​](#configure-authentication-before-exposing-the-server-over-http "Direct link to Configure authentication before exposing the server over HTTP") If multiple people use a shared deployment, configure `INFRAHUB_MCP_AUTH_MODE` along with token passthrough or OIDC settings before exposing the server. `stdio` deployments continue to use environment variable credentials. ## Minor changes[​](#minor-changes "Direct link to Minor changes") ### Documentation[​](#documentation "Direct link to Documentation") * Upgraded the documentation build to Docusaurus 3.10.0. ### Release pipeline[​](#release-pipeline "Direct link to Release pipeline") * Added automated, categorized release drafts. ### Security[​](#security "Direct link to Security") * Updated dependencies, including `cryptography`, `authlib`, and `python-multipart`. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * Production middleware stack: authentication, rate limiting, observability, read-only mode, and `/health` and `/metrics` endpoints ([#62](https://github.com/opsmill/infrahub-mcp/pull/62)). ### Changed[​](#changed "Direct link to Changed") * Documentation upgraded to Docusaurus 3.10.0 ([#65](https://github.com/opsmill/infrahub-mcp/pull/65)). ### Fixed[​](#fixed "Direct link to Fixed") * Removed a broken release-drafter workflow ([#69](https://github.com/opsmill/infrahub-mcp/pull/69)). ## Notes[​](#notes "Direct link to Notes") * `v1.1.0` was prepared internally but never published; this work ships in `v1.1.1`. --- # Release 1.1.2 | Release Number | 1.1.2 | | -------------- | --------------------------------------------------------------------- | | Release Type | Maintenance | | Release Date | May 4th, 2026 | | Tag | [v1.1.2](https://github.com/opsmill/infrahub-mcp/releases/tag/v1.1.2) | ## Release summary[​](#release-summary "Direct link to Release summary") Dependency and internal tooling updates only. There are no changes to MCP tools, workflows, or runtime behavior. ## Minor changes[​](#minor-changes "Direct link to Minor changes") ### Developer experience[​](#developer-experience "Direct link to Developer experience") * Updated Speckit and related extensions. * Hardened the release and packaging pipeline. ### Security[​](#security "Direct link to Security") * Updated `urllib3` and `python-multipart`. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Changed[​](#changed "Direct link to Changed") * Updated internal developer tooling (Speckit and extensions) and hardened the release and packaging pipeline ([#85](https://github.com/opsmill/infrahub-mcp/pull/85)). ### Security[​](#security-1 "Direct link to Security") * Grouped dependency updates ([#91](https://github.com/opsmill/infrahub-mcp/pull/91), [#86](https://github.com/opsmill/infrahub-mcp/pull/86)). --- # Release 1.1.3 | Release Number | 1.1.3 | | -------------- | --------------------------------------------------------------------- | | Release Type | Improvement | | Release Date | May 12th, 2026 | | Tag | [v1.1.3](https://github.com/opsmill/infrahub-mcp/releases/tag/v1.1.3) | ## Release summary[​](#release-summary "Direct link to Release summary") Run non-HTTP transports without additional startup configuration. This also includes pipeline and developer tooling maintenance. ### Run non-HTTP transports correctly[​](#run-non-http-transports-correctly "Direct link to Run non-HTTP transports correctly") Host and port settings are now applied only to the `streamable-http` transport, allowing other transports to start correctly without additional configuration. **What changed** * Pass host and port to the runner only when using `streamable-http`. ## Minor changes[​](#minor-changes "Direct link to Minor changes") ### Release pipeline[​](#release-pipeline "Direct link to Release pipeline") * Corrected the release auto-labeling pipeline. ### Developer experience[​](#developer-experience "Direct link to Developer experience") * Refreshed Speckit and related extensions. No runtime behavior changed. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Fixed[​](#fixed "Direct link to Fixed") * Startup now works correctly for non-HTTP transports ([#90](https://github.com/opsmill/infrahub-mcp/pull/90)). * Corrected the release auto-labeling pipeline ([#94](https://github.com/opsmill/infrahub-mcp/pull/94)). ### Changed[​](#changed "Direct link to Changed") * Refreshed internal developer tooling (Speckit and extensions). --- # Release 1.1.4 | Release Number | 1.1.4 | | -------------- | --------------------------------------------------------------------- | | Release Type | Maintenance | | Release Date | May 12th, 2026 | | Tag | [v1.1.4](https://github.com/opsmill/infrahub-mcp/releases/tag/v1.1.4) | ## Release summary[​](#release-summary "Direct link to Release summary") Alignment with current Infrahub SDK releases, plus routine dependency maintenance. There are no changes to MCP tools, workflows, or runtime behavior. ## Minor changes[​](#minor-changes "Direct link to Minor changes") ### SDK compatibility[​](#sdk-compatibility "Direct link to SDK compatibility") * Raised the Infrahub SDK minimum version to `>=1.20.0`. * Migrated a deprecated SDK keyword argument. ### Security[​](#security "Direct link to Security") * Updated dependencies and GitHub Actions. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Changed[​](#changed "Direct link to Changed") * Raised the Infrahub SDK minimum version to `>=1.20.0` and migrated a deprecated SDK keyword argument ([#96](https://github.com/opsmill/infrahub-mcp/pull/96)). ### Security[​](#security-1 "Direct link to Security") * Grouped dependency and GitHub Actions updates ([#98](https://github.com/opsmill/infrahub-mcp/pull/98), [#97](https://github.com/opsmill/infrahub-mcp/pull/97)). --- # Release 1.1.5 | Release Number | 1.1.5 | | -------------- | --------------------------------------------------------------------- | | Release Type | Improvement | | Release Date | May 29th, 2026 | | Tag | [v1.1.5](https://github.com/opsmill/infrahub-mcp/releases/tag/v1.1.5) | ## Release summary[​](#release-summary "Direct link to Release summary") Know immediately when a session branch can no longer be used. Branch state is validated before reuse, making long-running sessions more predictable and laying the groundwork for the automatic recovery in [1.1.6](/mcp/release-notes/release-1_1_6.md). ### Get a clear signal when a session branch can no longer be used[​](#get-a-clear-signal-when-a-session-branch-can-no-longer-be-used "Direct link to Get a clear signal when a session branch can no longer be used") Branch state is now validated before reuse, so a session whose branch has been merged or deleted fails with a clear reason instead of an opaque error. This is the groundwork for the automatic recovery in [1.1.6](/mcp/release-notes/release-1_1_6.md). **What changed** * Detect when a cached branch has been deleted. * Detect when a cached branch has already been merged or is being deleted. * Report why a branch can no longer be used before attempting additional writes. ## Minor changes[​](#minor-changes "Direct link to Minor changes") ### Documentation[​](#documentation "Direct link to Documentation") * Updated the README. ### Security[​](#security "Direct link to Security") * Updated dependencies, including `idna` and `authlib`. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Fixed[​](#fixed "Direct link to Fixed") * Validate the cached session branch before reuse ([#110](https://github.com/opsmill/infrahub-mcp/pull/110)). ### Security[​](#security-1 "Direct link to Security") * Updated dependencies, including `idna` ([#105](https://github.com/opsmill/infrahub-mcp/pull/105)) and `authlib` ([#103](https://github.com/opsmill/infrahub-mcp/pull/103)). --- # Release 1.1.6 | Release Number | 1.1.6 | | -------------- | --------------------------------------------------------------------- | | Release Type | Feature | | Release Date | June 8th, 2026 | | Tag | [v1.1.6](https://github.com/opsmill/infrahub-mcp/releases/tag/v1.1.6) | ## Release summary[​](#release-summary "Direct link to Release summary") A session works on a single branch and opens one Proposed Change from it. When that branch is merged or is no longer available, the session recovers automatically so work can continue on a fresh branch, and you can reset or switch branches explicitly to start the next change set — all while still following existing Infrahub review workflows. ### Continue a session after its branch is merged or deleted[​](#continue-a-session-after-its-branch-is-merged-or-deleted "Direct link to Continue a session after its branch is merged or deleted") When a Proposed Change has been merged or its branch is no longer available, a replacement branch is created automatically so work can continue in the same session. **What changed** * Detect when a branch has been deleted, merged, or is in the process of being deleted. * Create a replacement branch automatically when the current branch can no longer accept changes. * Surface a retryable error when a branch changes state mid-write — the failed write is not replayed automatically (a partial mutation could apply), so the next attempt runs on a fresh branch. ### Start fresh or switch to a different branch[​](#start-fresh-or-switch-to-a-different-branch "Direct link to Start fresh or switch to a different branch") A session branch can now be reset or changed explicitly, making it easier to start over or continue work on a branch that was prepared ahead of time. **What changed** * Reset the current branch using `reset_session_branch()` so the next write starts with a new branch. * Switch to a branch by name using `reset_session_branch()` — created automatically if it does not exist and matches the configured branch pattern. * Validate branch names against the configured branch pattern. * Reject the default branch and merged or deleting branches. ### Review and merge changes created with AI using existing Infrahub workflows[​](#review-and-merge-changes-created-with-ai-using-existing-infrahub-workflows "Direct link to Review and merge changes created with AI using existing Infrahub workflows") Changes created with AI are kept separate from the default branch and can be explicitly submitted as a Proposed Change, allowing them to follow the same review and merge process used elsewhere in Infrahub. **What changed** * Always write to the active session branch. * Block branch and schema mutations, including those nested in GraphQL fragments. * Continue using `propose_changes` to submit changes for review. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * `reset_session_branch` tool to reset or switch the session branch ([#114](https://github.com/opsmill/infrahub-mcp/pull/114)). ### Changed[​](#changed "Direct link to Changed") * `search_nodes` now uses `any__value` with `partial_match`, making substring searches consistent across concrete and abstract kinds ([#93](https://github.com/opsmill/infrahub-mcp/pull/93)). * Writes always target the active session branch, and privileged branch and schema mutations are blocked ([#114](https://github.com/opsmill/infrahub-mcp/pull/114)). ### Fixed[​](#fixed "Direct link to Fixed") * Recover stale (merged or deleted) session branches automatically ([#114](https://github.com/opsmill/infrahub-mcp/pull/114)). --- # Release 1.1.7 | Release Number | 1.1.7 | | -------------- | --------------------------------------------------------------------- | | Release Type | Feature | | Release Date | July 3rd, 2026 | | Tag | [v1.1.7](https://github.com/opsmill/infrahub-mcp/releases/tag/v1.1.7) | ## Release summary[​](#release-summary "Direct link to Release summary") Trace how two objects in your infrastructure are connected, or find everything of a given kind that a change could reach — without inspecting the schema and hand-building a deep, nested GraphQL query. Two read-only tools, `find_paths` and `find_reachable`, expose the server-side graph traversal added in Infrahub 1.10. Reading a schema also becomes more controllable, with peer-schema expansion you can turn on or off. ### Trace how objects connect and what a change can reach[​](#trace-how-objects-connect-and-what-a-change-can-reach "Direct link to Trace how objects connect and what a change can reach") Instead of studying the schema and hand-building a deep, nested GraphQL query, an agent can now ask two direct questions of the graph. Both tools build on the server-side path traversal added in Infrahub 1.10. **What changed** * `find_paths` returns the shortest path(s) between a source and a destination node, as ordered hops — answering "how are these two connected?". A `count` of `0` means no path exists within the search depth. * `find_reachable` returns the nodes of the kinds you name that are reachable from a source node — answering "what could a change here affect?". * Both accept either a UUID or the kind-qualified HFID that `get_nodes` and `search_nodes` already return (for example, `InfraDevice__atl1-edge1`), and both can be bounded by depth, node kind, or relationship. * Both are read-only and return compact TOON output. Against an Infrahub server older than 1.10, they fail with a clear version message instead of an opaque error. ### Expand related schemas only when you need them[​](#expand-related-schemas-only-when-you-need-them "Direct link to Expand related schemas only when you need them") When reading a schema, related peer schemas are expanded one level deep so an agent can see how kinds connect. That expansion is now optional, so you can keep schema responses lean when the extra detail is not needed. **What changed** * Peer-schema expansion is controlled by the new `INFRAHUB_MCP_SCHEMA_EXPAND_PEERS` setting, enabled by default so existing behavior is preserved. * `get_schema` accepts an `expand` argument to override that default for a single call. ## Minor changes[​](#minor-changes "Direct link to Minor changes") ### Testing[​](#testing "Direct link to Testing") * Added an opt-in integration test suite (`tests/integration/`, run with `pytest -m integration`) that exercises the server end to end against an ephemeral Infrahub provisioned by infrahub-testcontainers. The default `uv run pytest` remains Docker-free. ### Security[​](#security "Direct link to Security") * Resolved Python and npm dependency security alerts. ## Upgrade notes[​](#upgrade-notes "Direct link to Upgrade notes") * The graph-traversal tools require Infrahub 1.10 or later and `infrahub-sdk >= 1.22`, which is now the minimum supported version. On Infrahub servers older than 1.10, `find_paths` and `find_reachable` return a clear version error; every other tool is unaffected. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * Graph-traversal tools `find_paths` and `find_reachable`, exposing the server-side path traversal added in Infrahub 1.10 ([#74](https://github.com/opsmill/infrahub-mcp/pull/74)). * `INFRAHUB_MCP_SCHEMA_EXPAND_PEERS` setting and a per-call `expand` argument on `get_schema` to control peer-schema expansion ([#74](https://github.com/opsmill/infrahub-mcp/pull/74)). * Opt-in integration test suite backed by infrahub-testcontainers ([#118](https://github.com/opsmill/infrahub-mcp/pull/118)). ### Changed[​](#changed "Direct link to Changed") * Raised the Infrahub SDK minimum version to `>=1.22.0` ([#74](https://github.com/opsmill/infrahub-mcp/pull/74)). * Grouped dependency and GitHub Actions updates ([#136](https://github.com/opsmill/infrahub-mcp/pull/136), [#131](https://github.com/opsmill/infrahub-mcp/pull/131)). ### Security[​](#security-1 "Direct link to Security") * Resolved Python dependency security alerts ([#133](https://github.com/opsmill/infrahub-mcp/pull/133)). * Resolved npm dependency security alerts in the documentation site ([#134](https://github.com/opsmill/infrahub-mcp/pull/134)). --- # Brownfield network onboarding Onboarding an existing ("brownfield") network into Infrahub is a long tail of small writes: one device per rack, one interface per device, one cable per interface. Agents shine here — they can read source-of-truth data from CSV exports, SNMP, LLDP neighbours, or NetBox, and turn it into idempotent Infrahub upserts on a session branch you can review all at once. ## Scenario[​](#scenario "Direct link to Scenario") A team is migrating from a spreadsheet-based inventory to Infrahub. They have a CSV with \~500 devices. They want an agent to import them safely, with a single Proposed Change at the end that a senior engineer reviews. ## The plan[​](#the-plan "Direct link to The plan") 1. Confirm the Infrahub schema covers every field in the CSV. 2. Do a dry-run import on a session branch (no merge). 3. Review the diff, iterate on any mapping issues. 4. Open a Proposed Change and hand it off for human review. 5. Merge once approved. ## 1. Align the CSV to the schema[​](#1-align-the-csv-to-the-schema "Direct link to 1. Align the CSV to the schema") Before any writes, the agent reads `infrahub://schema` and then `infrahub://schema/{kind}` for each kind it plans to populate. It maps CSV columns to Infrahub attributes and flags anything that doesn't align: | CSV column | Kind | Attribute | Notes | | ----------- | ------------ | ----------------- | ------------------------------------- | | `hostname` | `DcimDevice` | `name` | Also used as HFID segment | | `role` | `DcimDevice` | `role` | Dropdown — check allowed values | | `site_code` | `DcimDevice` | `site` (relation) | Resolve by `LocationSite.name__value` | If the schema is missing a field, that's the first write to make — via the Infrahub UI or a schema-extension proposed change — before the bulk import. ## 2. Use human-friendly IDs for idempotency[​](#2-use-human-friendly-ids-for-idempotency "Direct link to 2. Use human-friendly IDs for idempotency") Brownfield imports run multiple times as mappings get fixed. Use **human-friendly IDs (HFIDs)** rather than UUIDs so re-running the import updates existing nodes instead of creating duplicates: ``` node_upsert( kind="DcimDevice", hfid=["spine-01"], data={ "name": "spine-01", "role": "spine", "serial": "JPE12345", }, ) ``` If a `DcimDevice` with `name=spine-01` already exists on the session branch, `node_upsert` updates it. If not, it's created. Either way, running the same command twice has the same effect. `node_upsert` accepts **scalar attributes only**. For relationships (assigning a device to a site, wiring interfaces to circuits), use `mutate_graphql` with a targeted mutation. ## 3. Watch the session branch accumulate[​](#3-watch-the-session-branch-accumulate "Direct link to 3. Watch the session branch accumulate") All writes in the session go to the auto-created `mcp/session-YYYYMMDD-` branch. The default branch stays clean. You can pause mid-import, read the branches resource, and confirm what's been written: ``` resource: infrahub://branches ``` If the agent catches a mapping issue at row 200, it can course-correct — re-run the earlier writes with corrected data, and the HFID-keyed upserts deduplicate transparently. ## 4. Batch size and rate limiting[​](#4-batch-size-and-rate-limiting "Direct link to 4. Batch size and rate limiting") For a 500-device import, you'll want two operational controls: * `INFRAHUB_MCP_RATE_LIMIT_RPS=10` — cap agent throughput so it doesn't saturate Infrahub. * `INFRAHUB_MCP_RETRY_MAX_ATTEMPTS=3` and `INFRAHUB_MCP_RETRY_BASE_DELAY=1.0` — retry transient failures with exponential backoff. See the [Configuration reference](/mcp/references/configuration.md#rate-limiting) for the full list. ## 5. Propose the change[​](#5-propose-the-change "Direct link to 5. Propose the change") Once the session branch has the full import: ``` propose_changes( title="Brownfield import: 512 devices from legacy inventory", description="Automated import of devices.csv exported 2026-04-17. See attached diff.", ) ``` A senior engineer reviews the diff in Infrahub, runs any repository-defined checks, and merges. The default branch only changes once a human approves. ## 6. Iterate[​](#6-iterate "Direct link to 6. Iterate") The session branch remains active after `propose_changes`. If review turns up issues, the agent can: * Keep writing to the same branch — new changes show up in the same proposed change. * Or close the proposed change, start fresh on a new session, and redo the import. ## Patterns worth keeping[​](#patterns-worth-keeping "Direct link to Patterns worth keeping") * **HFID-keyed upserts** are the cornerstone of brownfield work: they let you re-run imports without duplicates. * **Scalar vs. relationship writes**: `node_upsert` for scalars, `mutate_graphql` for relationships. * **One proposed change per logical batch**: one for devices, another for interfaces, another for cabling — reviewers can reason about each independently. * **Dry-run with `INFRAHUB_MCP_READ_ONLY=true` first**: have the agent generate the *plan* as text, not writes, so you can sanity-check the mapping before enabling writes. ## Related reading[​](#related-reading "Direct link to Related reading") * [Safe changes via branch isolation](/mcp/use-cases/safe-changes-branch-isolation.md) — the branch-per-session model in depth. * [Make a change through an agent](/mcp/getting-started/make-a-change.md) — the same workflow for a single change. * [Methods reference — node\_upsert](/mcp/references/methods.md) — full `node_upsert` contract. --- # Compliance analysis Agents are a natural fit for ad-hoc compliance questions: "do all edge routers have dual uplinks to different spines?", "are all production VLANs documented?", "which devices are missing an owner?". This recipe shows how to run those checks as read-only agent workflows against Infrahub. ## Scenario[​](#scenario "Direct link to Scenario") A network architect asks: > Every device tagged `role=edge` must have at least two uplink interfaces, and those uplinks must connect to different spine devices. Find any violations. ## Recommended setup[​](#recommended-setup "Direct link to Recommended setup") Compliance checks are read-only by definition. Run the MCP server with writes disabled so no accidental mutation slips through: ``` export INFRAHUB_MCP_READ_ONLY=true ``` Under read-only mode, the write tools (`node_upsert`, `node_delete`, `propose_changes`, `mutate_graphql`) are hidden from discovery and rejected if hard-coded — the agent literally cannot write. If you want compliance checks to run as a specific identity (for audit logs), also enable [OIDC](/mcp/references/authentication.md#oidc--identity--scopes-via-external-idp) and give the compliance user read-only scopes. ## The flow[​](#the-flow "Direct link to The flow") ### 1. Enumerate the population[​](#1-enumerate-the-population "Direct link to 1. Enumerate the population") ``` query EdgeDevices { DcimDevice(role__value: "edge") { count edges { node { name { value } id } } } } ``` `query_graphql` returns the full list. For very large populations, iterate in batches via `get_nodes(kind=..., limit=N)`. ### 2. For each device, walk its uplinks[​](#2-for-each-device-walk-its-uplinks "Direct link to 2. For each device, walk its uplinks") The agent could make one GraphQL call that returns everything (uplinks + remote device per uplink) and then analyze client-side: ``` query EdgeUplinks { DcimDevice(role__value: "edge") { edges { node { name { value } interfaces(role__value: "uplink") { edges { node { name { value } connected_endpoint { node { device { node { name { value } role { value } } } } } } } } } } } } ``` ### 3. Score each device against the rule[​](#3-score-each-device-against-the-rule "Direct link to 3. Score each device against the rule") Per device: * Count interfaces with `role=uplink` → must be ≥ 2. * Map each uplink to its remote device → the set must have ≥ 2 distinct spines. A violation is any device that fails either condition. The agent returns a structured report: | Device | Uplinks | Unique remote spines | Status | | ------- | ------- | -------------------- | ------------------------------------ | | edge-01 | 2 | 2 | pass | | edge-02 | 1 | 1 | **fail: single uplink** | | edge-03 | 2 | 1 | **fail: both uplinks to same spine** | ### 4. Persist the report (optional)[​](#4-persist-the-report-optional "Direct link to 4. Persist the report (optional)") The agent can write the report to a file, post it to Slack, or attach it to a GitHub issue — whatever tools it already has. The Infrahub MCP server stays strictly read-only; remediation happens through a separate human-driven workflow. ## Tips for compliance agents[​](#tips-for-compliance-agents "Direct link to Tips for compliance agents") * **Cache read-only resources**: enable `INFRAHUB_MCP_CACHE_ENABLED=true` to cut schema-lookup chatter. * **Use GraphQL fragments**: when the same check runs repeatedly, define the query once and parameterize it. * **Run on a frozen branch**: pass `branch="compliance-snapshot-2026-04-17"` to make the check reproducible even as the default branch drifts. * **Rate-limit in production**: set `INFRAHUB_MCP_RATE_LIMIT_RPS` to cap the load an agent can put on Infrahub. * **Keep a spot-check human loop**: have the agent post its findings for a human reviewer before filing tickets or opening PRs. ## Related reading[​](#related-reading "Direct link to Related reading") * [Natural-language to GraphQL](/mcp/use-cases/natural-language-graphql.md) — for writing the underlying queries. * [Safe changes via branch isolation](/mcp/use-cases/safe-changes-branch-isolation.md) — if your compliance flow needs to *fix* violations. * [Configuration reference — Observability](/mcp/references/configuration.md#observability) — scrape metrics into Grafana for compliance dashboards. --- # Cross-system correlation Infrastructure data doesn't live in one place. A ticket in Jira mentions a device, a Git commit touches a configuration file, an Ansible run fails against a hostname. Connecting those dots — "which device is this about, what depends on it, what changed recently?" — is a natural fit for an agent that can read Infrahub through MCP and your other systems through their own tools. ## Scenario[​](#scenario "Direct link to Scenario") A support engineer is triaging a Jira ticket: > "SR-4812 — `spine-03` flapping interface `Ethernet1/5`, downstream services degraded." The agent's job: enrich the ticket with Infrahub context — what is `spine-03`, what connects to `Ethernet1/5`, which services depend on that link, and when did it last change. ## The flow[​](#the-flow "Direct link to The flow") ### 1. Start with the signal[​](#1-start-with-the-signal "Direct link to 1. Start with the signal") The agent already has the ticket text from the Jira MCP server (or its own Jira tool). It extracts the device name `spine-03` and the interface `Ethernet1/5`. ### 2. Resolve the device in Infrahub[​](#2-resolve-the-device-in-infrahub "Direct link to 2. Resolve the device in Infrahub") ``` search_nodes(query="spine-03", kind="DcimDevice", limit=5) ``` Once the agent has the canonical name and UUID, it fetches the full record: ``` get_nodes( kind="DcimDevice", filters={"name__value": "spine-03"}, include_attributes=True, ) ``` ### 3. Walk the relationships[​](#3-walk-the-relationships "Direct link to 3. Walk the relationships") The interesting context lives in related nodes — interfaces, connected circuits, upstream/downstream peers, the rack, the site, any services that reference this device. Use `get_nodes` with `prefetch_relationships` to pull related objects in a single call, or drop to `query_graphql` for multi-hop traversals: ``` get_nodes( kind="DcimDevice", filters={"name__value": "spine-03"}, prefetch_relationships=True, include_attributes=True, ) ``` For the specific interface: ``` query { DcimInterface( device__name__value: "spine-03", name__value: "Ethernet1/5" ) { edges { node { description { value } connected_endpoint { node { name { value } device { node { name { value } } } } } } } } } ``` ### 4. Correlate with change history[​](#4-correlate-with-change-history "Direct link to 4. Correlate with change history") Ask your Git MCP server (or a git log tool) for recent commits touching `spine-03`'s configuration. Infrahub's own branch history surfaces recently-merged proposed changes via the branches resource: ``` resource: infrahub://branches ``` ### 5. Return a triage summary[​](#5-return-a-triage-summary "Direct link to 5. Return a triage summary") The agent synthesizes: > `spine-03` is an Arista 7280 in Site AMS1, Rack 4B. Interface `Ethernet1/5` is a 100G uplink to `leaf-12 Ethernet49`. Two services depend on this link: `edge-fabric-eu` and `transit-carrier-a`. No Infrahub changes in the last 7 days; the last Git configuration push was 3 days ago by `@alice`. ## Patterns[​](#patterns "Direct link to Patterns") * **Name → ID**: always resolve human-friendly names to UUIDs before fetching related data. Use `search_nodes` for fuzzy names, `get_nodes` with exact filters for clean ones. * **Relationship names are schema-defined**: use `infrahub://schema/{kind}` to discover the exact relationship field names (`interfaces`, `connected_endpoint`, `site`, etc.) rather than guessing. * **GraphQL for multi-hop**: if you need to cross 3+ relationships in one call, write a GraphQL query via `query_graphql` — it's cheaper than chaining multiple `get_nodes` calls. * **Branch awareness**: if the ticket is about a pending change, pass `branch=...` to your queries. The same device can have different state on different branches. ## Related reading[​](#related-reading "Direct link to Related reading") * [Natural-language to GraphQL](/mcp/use-cases/natural-language-graphql.md) — for multi-hop joins. * [Compliance analysis](/mcp/use-cases/compliance-analysis.md) — for correlating across *many* devices. * [Troubleshooting data queries](/mcp/use-cases/troubleshooting-queries.md) — when the correlation returns nothing. --- # Natural language to GraphQL `get_nodes` and `search_nodes` handle most everyday queries, but some questions need the full expressive power of GraphQL: aggregates, deep joins, multi-kind filters, and computed fields. This recipe shows how an agent goes from a free-form English question to a working `query_graphql` call, using the server's own schema as the grounding. ## Scenario[​](#scenario "Direct link to Scenario") A capacity planner asks: > Show me every /24 prefix in the `EU` region that has less than 50% of its addresses allocated, sorted by available space descending. No canned tool covers that shape. The agent needs to generate a GraphQL query against Infrahub's schema. ## The flow[​](#the-flow "Direct link to The flow") ### 1. Ground the model in the real schema[​](#1-ground-the-model-in-the-real-schema "Direct link to 1. Ground the model in the real schema") Reading `infrahub://graphql-schema` returns the full GraphQL SDL for *this* Infrahub instance — including any schema extensions. This is the single source of truth the agent should consult before writing a query. For clients that don't support the `text/plain` resource type, the agent can fall back to calling `get_schema()` for the kind catalog, then `get_schema(kind="IpamIPPrefix")` (or whichever kind applies) for the full field list. ### 2. Draft the query[​](#2-draft-the-query "Direct link to 2. Draft the query") With the SDL in context, the agent drafts: ``` query PrefixCapacity($region: String!) { IpamIPPrefix( prefix_length__value: 24, region__name__value: $region ) { count edges { node { prefix { value } utilization { value } region { node { name { value } } } } } } } ``` The agent inspects the returned `utilization` values, filters client-side to those below 50%, and sorts by available space. ### 3. Execute via `query_graphql`[​](#3-execute-via-query_graphql "Direct link to 3-execute-via-query_graphql") ``` query_graphql( query="", branch=None, ) ``` Mutations are rejected by `query_graphql` — this is a read-only path. If the agent tries to mutate, it gets a clear error telling it to use `mutate_graphql` or `node_upsert` instead. ### 4. Narrate the result[​](#4-narrate-the-result "Direct link to 4. Narrate the result") The agent summarizes in natural language: "8 prefixes in EU have less than 50% utilization. The three with the most headroom are ..." ## When to use which tool[​](#when-to-use-which-tool "Direct link to When to use which tool") | Question shape | Use | | ------------------------------------------ | ---------------------------------------------- | | "Show me all X where attribute = Y" | `get_nodes` with filters | | "Find X whose name contains Y" | `search_nodes` | | "What relationships does X have?" | `get_nodes` with `prefetch_relationships=True` | | "Aggregate, join, or compute across kinds" | `query_graphql` | | "Build a custom change pipeline" | `mutate_graphql` (write) | ## The `answer_infra_question` prompt[​](#the-answer_infra_question-prompt "Direct link to the-answer_infra_question-prompt") The server ships a prompt called `answer_infra_question` that encodes this exact pipeline — discover kinds, read schema detail, pick the right tool, traverse relationships, summarize with provenance. Invoke it directly from your client when you want a canned read-only investigation flow: ``` /answer_infra_question question="How many /24s in EU have <50% utilization?" ``` ## Related reading[​](#related-reading "Direct link to Related reading") * [Troubleshooting data queries](/mcp/use-cases/troubleshooting-queries.md) — when the query structure is wrong. * [Cross-system correlation](/mcp/use-cases/cross-system-correlation.md) — combining Infrahub data with external signals. * [Methods reference](/mcp/references/methods.md#query_graphql) — the full `query_graphql` contract. --- # Safe changes via branch isolation Branch isolation is the single most important safety property of the Infrahub MCP server: **an agent can never write directly to your default branch**. Every mutation lives on an ephemeral session branch that only becomes part of the default branch once a human reviews and merges a Proposed Change. This page explains the model end-to-end and shows how to tune it. ## The lifecycle of a session branch[​](#the-lifecycle-of-a-session-branch "Direct link to The lifecycle of a session branch") Key properties: * **The branch is auto-created on the first write** — the agent never names or creates it explicitly. * **All subsequent writes in the same MCP session go to the same branch**, so a multi-step change is atomic from the reviewer's perspective. * **`propose_changes` does not close the branch** — the session stays active, so the agent can keep iterating and the new writes land in the same proposed change. * **Merging is human-only** — no MCP tool merges branches; that happens in the Infrahub UI or API, under whatever approval policy your organization enforces. ## Customizing the branch name[​](#customizing-the-branch-name "Direct link to Customizing the branch name") The default pattern is `mcp/session-{date}-{hex}`. Override it via `INFRAHUB_MCP_BRANCH_PATTERN`: ``` # Fixed branch name (must not already exist) INFRAHUB_MCP_BRANCH_PATTERN=mcp/automation # Per-user branch (requires OIDC auth mode) INFRAHUB_MCP_BRANCH_PATTERN=mcp/{user}/{date}-{hex} ``` Supported placeholders: | Placeholder | Resolves to | | ----------- | -------------------------------------------------- | | `{date}` | `YYYYMMDD` | | `{hex}` | Short random hex | | `{user}` | OIDC-authenticated user ID, sanitized for git refs | If the generated name collides, the server retries up to `INFRAHUB_MCP_MAX_BRANCH_RETRIES` times (default 5, max 20). A fixed name with no placeholders always resolves to the same branch, so startup fails if that branch already exists. See [Authentication architecture — Branch placeholder](/mcp/references/authentication.md#branch-placeholder) for how `{user}` is sanitized. ## Making a change dry-run-able[​](#making-a-change-dry-run-able "Direct link to Making a change dry-run-able") There's no explicit dry-run flag — the session branch *is* the dry run. Review it before calling `propose_changes`: 1. Agent makes writes via `node_upsert` / `node_delete` / `mutate_graphql`. 2. Open the Infrahub UI and navigate to the session branch (listed under `infrahub://branches`). 3. Review the diff. If something is wrong, just discard the branch — the default branch never saw it. 4. Re-run the agent on a fresh session. For a hard read-only posture (compliance analysis, monitoring agents), set `INFRAHUB_MCP_READ_ONLY=true` — writes are hidden from discovery and rejected if hardcoded. ## Combining with scope-based authorization[​](#combining-with-scope-based-authorization "Direct link to Combining with scope-based authorization") Under OIDC auth, you can gate write tools behind an OAuth scope: ``` INFRAHUB_MCP_AUTH_MODE=oidc INFRAHUB_MCP_AUTH_SCOPES_WRITE=infrahub:write ``` Users whose token doesn't include `infrahub:write` cannot see or call write tools. Branch isolation stops agents from damaging the default branch by accident; scope gating stops *specific users* from writing at all. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") | Symptom | Likely cause | Fix | | -------------------------------------------- | ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | "Branch already exists" error at startup | Fixed `INFRAHUB_MCP_BRANCH_PATTERN` and the branch is left over from a previous run | Merge or delete the branch in Infrahub, or use a pattern with `{hex}`. | | `propose_changes` fails with "no changes" | No successful writes on the session branch | Confirm at least one `node_upsert`/`node_delete`/`mutate_graphql` succeeded. | | Agent can't find the session branch | Branch creation failed silently | Check server logs for the branch-creation error; often a permission issue on the Infrahub API token. | | `{user}` placeholder resolves to `anonymous` | Auth mode is `none` | Enable OIDC to get a real user identity, or use a different placeholder. | ## Related reading[​](#related-reading "Direct link to Related reading") * [Make a change through an agent](/mcp/getting-started/make-a-change.md) — the first-time walkthrough. * [Brownfield network onboarding](/mcp/use-cases/brownfield-onboarding.md) — branch isolation applied to bulk imports. * [Authentication architecture](/mcp/references/authentication.md) — how identity flows into branch naming and audit logs. --- # Troubleshooting data queries When an agent answers "I couldn't find that device" — it's almost always a data-query problem, not a data problem. This recipe walks through the diagnostic flow for "why is my query returning nothing?" ## Scenario[​](#scenario "Direct link to Scenario") A network engineer asks their assistant: > What interfaces are on `leaf-01`? The agent replies "no device named `leaf-01` found," but the engineer can see it in the Infrahub UI. Something is wrong with the query, not the data. ## The diagnostic flow[​](#the-diagnostic-flow "Direct link to The diagnostic flow") ### 1. Confirm the kind exists[​](#1-confirm-the-kind-exists "Direct link to 1. Confirm the kind exists") Ask the agent to read `infrahub://schema` (or call `get_schema()` with no arguments). This returns the catalog of all non-internal schema kinds. Expected output includes entries like `DcimDevice`, `InfraDevice`, or whatever your schema defines. If `leaf-01` belongs to a kind the agent didn't try, you've already found the bug. ### 2. Inspect the kind's filters[​](#2-inspect-the-kinds-filters "Direct link to 2. Inspect the kind's filters") Read `infrahub://schema/{kind}` (or call `get_schema(kind="DcimDevice")`) to see: * Every attribute (with its type). * Every relationship. * The exact filter keys accepted by `get_nodes`. The filter keys follow a strict naming convention: | Filter type | Example | | ----------------------- | ------------------------------------ | | Attribute equality | `name__value` | | Attribute partial match | `name__value` + `partial_match=True` | | Relationship attribute | `site__name__value` | | Relationship ID | `site__id` | A common mistake: filtering on `name` instead of `name__value`. The latter is correct. ### 3. Try partial match for human-entered names[​](#3-try-partial-match-for-human-entered-names "Direct link to 3. Try partial match for human-entered names") `leaf-01` might actually be stored as `leaf-01.example.com` or `LEAF-01`. Use `search_nodes`, which is a convenience wrapper around `get_nodes` with `partial_match=True`: ``` search_nodes(query="leaf-01", kind="DcimDevice", limit=10) ``` This returns up to 10 display labels containing the substring. If it returns one match, you have your canonical name. ### 4. Traverse relationships[​](#4-traverse-relationships "Direct link to 4. Traverse relationships") If the engineer actually wants *interfaces on leaf-01*, the agent can use `get_nodes` with `prefetch_relationships` to pull the device and its interfaces in one call: ``` get_nodes( kind="DcimDevice", filters={"name__value": "leaf-01"}, prefetch_relationships=True, include_attributes=True, ) ``` Or in one GraphQL call via `query_graphql`: ``` query { DcimDevice(name__value: "leaf-01") { edges { node { interfaces { edges { node { name { value } } } } } } } } ``` ## Common filter pitfalls[​](#common-filter-pitfalls "Direct link to Common filter pitfalls") | Pitfall | Fix | | --------------------------------------------- | -------------------------------------------------------------------------- | | `filters={"name": "leaf-01"}` returns nothing | Use `name__value`, not `name` | | Case mismatch (`LEAF-01` vs `leaf-01`) | Add `partial_match=True` or use `search_nodes` | | Filtering a relationship as a string | Use the nested `relation__attribute__value` syntax | | No results because of the default limit | Pass `limit=-1` to `get_nodes` for all results | | Wrong branch | Pass `branch=` explicitly or confirm the default via `infrahub://branches` | ## Related reading[​](#related-reading "Direct link to Related reading") * [Methods reference](/mcp/references/methods.md) — the full tool and resource list. * [Natural-language to GraphQL](/mcp/use-cases/natural-language-graphql.md) — when filters aren't expressive enough. --- # Nornir Integration with Infrahub This integration bridges [Nornir's](https://github.com/nornir-automation/nornir) powerful automation framework with [Infrahub's](https://github.com/opsmill/infrahub) next-generation infrastructure data management platform. ## Why use this integration?[​](#why-use-this-integration "Direct link to Why use this integration?") Traditional network automation tools struggle with: * **Static inventory management**: Manually maintained host files that become outdated * **Configuration drift**: No systematic way to track and validate infrastructure state * **Environment isolation**: Difficulty managing different configurations across dev/staging/production * **Audit trails**: Limited visibility into who changed what and when The Nornir-Infrahub integration solves these challenges by: * **Dynamic inventory**: Automatically sync device inventory from Infrahub's graph database * **Version control**: Leverage Infrahub's Git-like branching for configuration management * **Artifact management**: Generate, validate, and deploy configurations with full traceability * **Flexible grouping**: Create dynamic device groups based on any Infrahub attribute or relationship ## Common use cases[​](#common-use-cases "Direct link to Common use cases") * **Multi-environment deployments**: Use Infrahub branches to manage configurations across development, staging, and production environments * **Compliance automation**: Generate compliance reports and configuration audits as artifacts * **Configuration validation**: Compare intended vs. actual device configurations * **Disaster recovery**: Maintain authoritative source of network configurations with full history ## Quick start[​](#quick-start "Direct link to Quick start") ### Installation[​](#installation "Direct link to Installation") ``` pip install nornir-infrahub ``` ### Next steps[​](#next-steps "Direct link to Next steps") * [Getting Started](/nornir/getting-started.md) - Set up your first Nornir-Infrahub integration * [Understanding the Integration](/nornir/topics/understanding-nornir-infrahub-integration.md) - Learn core concepts and architecture * [Your First Automation Workflow](/nornir/guides/your-first-automation-workflow.md) - Build a complete automation workflow ## Documentation[​](#documentation "Direct link to Documentation") ### Topics[​](#topics "Direct link to Topics") * [Understanding Nornir-Infrahub Integration](/nornir/topics/understanding-nornir-infrahub-integration.md) * [Infrahub Inventory Concepts](/nornir/topics/infrahub-inventory-concepts.md) * [Artifact Lifecycle Management](/nornir/topics/artifact-lifecycle-management.md) ### Guides[​](#guides "Direct link to Guides") * [Your First Automation Workflow](/nornir/guides/your-first-automation-workflow.md) * [Configuring Schema Mappings](/nornir/guides/configuring-schema-mappings.md) ### References[​](#references "Direct link to References") #### Inventory plugin[​](#inventory-plugin "Direct link to Inventory plugin") * [InfrahubInventory](/nornir/references/plugins/infrahub_inventory.md#infrahubinventory) #### Artifact management tasks[​](#artifact-management-tasks "Direct link to Artifact management tasks") * [regenerate\_host\_artifact](/nornir/references/plugins/artifact_tasks.md#regenerate_host_artifact) * [generate\_artifacts](/nornir/references/plugins/artifact_tasks.md#generate_artifacts) * [get\_artifact](/nornir/references/plugins/artifact_tasks.md#get_artifact) #### CoreFileObject management tasks[​](#corefileobject-management-tasks "Direct link to CoreFileObject management tasks") * [upload\_file\_object](/nornir/references/plugins/file_object_tasks.md#upload_file_object) * [download\_file\_object](/nornir/references/plugins/file_object_tasks.md#download_file_object) ## Additional resources[​](#additional-resources "Direct link to Additional resources") * [Blog: Simplifying Network Automation Workflows with Infrahub, Nornir, and Jinja2](https://www.opsmill.com/simplifying-network-automation-workflows-with-infrahub-nornir-and-jinja2/) * [Infrahub Documentation](https://docs.infrahub.app/) * [Nornir Documentation](https://nornir.readthedocs.io/) --- # Getting Started This guide helps you set up your first integration between Nornir and Infrahub. By the end, you'll have a working automation setup that demonstrates the power of dynamic inventory management. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting, ensure you have: * Python 3.10 through 3.14 installed * An Infrahub instance running (local or remote) * You can use the [Infrahub sandbox instance](https://sandbox.infrahub.app) for testing * Basic familiarity with Nornir concepts * At least one device defined in your Infrahub schema ## Installation[​](#installation "Direct link to Installation") Install the Nornir-Infrahub plugin: * pip * uv ``` pip install nornir-infrahub nornir_netmiko ``` ``` uv add nornir-infrahub nornir_netmiko ``` This automatically installs all required dependencies including: * `nornir` - The automation framework * `infrahub-sdk` - Python SDK for Infrahub API * `nornir-utils` - Helpful utilities for Nornir ## Setting up your first integration[​](#setting-up-your-first-integration "Direct link to Setting up your first integration") ### Step 1: Obtain your Infrahub API token[​](#step-1-obtain-your-infrahub-api-token "Direct link to Step 1: Obtain your Infrahub API token") note You can skip this step if you will be using the Infrahub Sandbox as the `config.yaml` file below already has the Infrahub Sandbox token! First, you'll need an API token to authenticate with Infrahub: 1. Log into your Infrahub web interface 2. Navigate to your user profile settings 3. Generate or copy your API token 4. Store it securely (we'll use it in the next step) ### Step 2: Understand your Infrahub schema[​](#step-2-understand-your-infrahub-schema "Direct link to Step 2: Understand your Infrahub schema") Before configuring Nornir, you need to understand your Infrahub schema structure. The sandbox instance at [sandbox.infrahub.app](https://sandbox.infrahub.app) uses a schema similar to this example: #### Example schema structure[​](#example-schema-structure "Direct link to Example schema structure") For the examples in this documentation to work, you'll need a schema with at least these node types: ``` # Device node definition nodes: - name: Device namespace: Infra attributes: - name: name kind: Text unique: true - name: description kind: Text optional: true - name: type kind: Text relationships: - name: site peer: BuiltinLocation cardinality: one - name: role peer: BuiltinRole cardinality: one - name: primary_address peer: InfraIPAddress cardinality: one optional: true - name: platform peer: InfraPlatform cardinality: one optional: true # Supporting nodes - name: IPAddress namespace: Infra attributes: - name: address kind: IPHost - name: Platform namespace: Infra attributes: - name: name kind: Text - name: nornir_platform kind: Text ``` #### Key schema requirements[​](#key-schema-requirements "Direct link to Key schema requirements") For Nornir integration, ensure your schema has: 1. **Device node** (`InfraDevice` or similar) representing network devices 2. **IP Address relationship** for device connectivity (`primary_address`) 3. **Platform relationship** for device type identification 4. **Location/Site relationship** for grouping devices by location 5. **Role relationship** for grouping devices by function #### Checking your schema[​](#checking-your-schema "Direct link to Checking your schema") To verify your schema structure: 1. Open your Infrahub web interface 2. Navigate to the Schema section 3. Look for device node types and their relationships 4. Note the exact attribute names and relationship paths for mapping ### Step 3: Create your Nornir configuration[​](#step-3-create-your-nornir-configuration "Direct link to Step 3: Create your Nornir configuration") Create a file named `config.yaml` with your Nornir configuration: ``` --- inventory: plugin: InfrahubInventory options: address: "https://sandbox.infrahub.app" # Infrahub instance URL token: "1808d43b-c370-48e8-d0ef-c51781c02ddf" # Replace with your actual token branch: "main" # Infrahub branch to use # Define which Infrahub node type represents network devices # Using the schema example above host_node: kind: "InfraDevice" # Matches the Device node with Infra namespace include: - "platform" # Map Infrahub attributes to Nornir host properties schema_mappings: - name: "hostname" mapping: "name" # Use device name as hostname - name: "platform" mapping: "platform.nornir_platform" # Platform attribute for Nornir compatibility # Create Nornir groups based on Infrahub attributes # Note: Disabled due to some devices missing site/role in sandbox # group_mappings: # - "site.name" # - "role.name" runner: plugin: threaded options: num_workers: 20 ``` ### Step 4: Test your connection[​](#step-4-test-your-connection "Direct link to Step 4: Test your connection") Create a Python script called `test_connection.py` to verify your setup: ``` from nornir import InitNornir from nornir_utils.plugins.functions import print_result def main(): # Initialize Nornir with your configuration nr = InitNornir(config_file="config.yaml") # Display discovered inventory print("Discovered hosts:") for host in nr.inventory.hosts.values(): print(f" - {host.name} ({host.hostname})") print("\nDiscovered groups:") for group in nr.inventory.groups.values(): print(f" - {group.name}") return 0 if __name__ == "__main__": raise SystemExit(main()) ``` Run the script: ``` python test_connection.py ``` You should see a list of hosts and groups discovered from your Infrahub instance. ### Step 5: Run your first task[​](#step-5-run-your-first-task "Direct link to Step 5: Run your first task") Now let's run a task against your inventory: ``` from nornir import InitNornir from nornir.core.task import Result, Task from nornir_utils.plugins.functions import print_result def hello_world(task: Task) -> Result: """A task that returns a greeting.""" return Result( host=task.host, result=f"Hello from {task.host.name}!" ) def main(): # Initialize Nornir nr = InitNornir(config_file="config.yaml") # Run the hello_world task on all hosts results = nr.run(task=hello_world) # Display results print_result(results) return 0 if __name__ == "__main__": raise SystemExit(main()) ``` ## Verifying your setup[​](#verifying-your-setup "Direct link to Verifying your setup") To ensure everything is working correctly: 1. **Check Host Discovery**: Verify that all expected hosts appear in your inventory 2. **Validate Groups**: Confirm that dynamic groups are created based on your mappings 3. **Test Connectivity**: Run a task to ensure Nornir can process your inventory ## Common issues and solutions[​](#common-issues-and-solutions "Direct link to Common issues and solutions") ### Authentication errors[​](#authentication-errors "Direct link to Authentication errors") If you see authentication errors: * Verify your API token is correct * Ensure your Infrahub instance is accessible at the specified address * Check that your token has appropriate permissions ### Missing hosts[​](#missing-hosts "Direct link to Missing hosts") If hosts aren't appearing: * Verify the `host_node.kind` matches your Infrahub schema * Check that devices exist in the specified branch * Ensure devices have the required attributes for your schema mappings ### Mapping errors[​](#mapping-errors "Direct link to Mapping errors") If you see errors about missing attributes: * Verify your schema mappings match your Infrahub data model * Use the Infrahub GraphQL explorer to check available attributes * Ensure related objects (like `primary_address`) are properly defined ## Next steps[​](#next-steps "Direct link to Next steps") Now that you have a working integration: * Explore [Your First Automation Workflow](/nornir/guides/your-first-automation-workflow.md) to build a complete automation * Learn about [Schema Mappings](/nornir/guides/configuring-schema-mappings.md) for advanced inventory configuration * Understand [Artifact Management](/nornir/topics/artifact-lifecycle-management.md) for configuration generation ## Getting help[​](#getting-help "Direct link to Getting help") * Check the [Infrahub Documentation](https://docs.infrahub.app/) for schema and API details * Review the [Nornir Documentation](https://nornir.readthedocs.io/) for automation concepts * Join the Infrahub community for support and best practices --- # How to Configure Schema Mappings This guide shows you how to configure schema mappings to control how Infrahub data translates into Nornir inventory properties. Schema mappings are the bridge between your Infrahub data model and Nornir's expected host attributes. ## What you'll accomplish[​](#what-youll-accomplish "Direct link to What you'll accomplish") By following this guide, you'll be able to: * Map Infrahub node attributes to Nornir host properties * Handle nested relationships and complex data structures * Create custom mappings for specialized use cases * Debug and troubleshoot mapping issues ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Working Nornir-Infrahub installation * Infrahub instance with network devices configured * You can use the [Infrahub sandbox instance](https://sandbox.infrahub.app) to follow along * Understanding of your Infrahub schema (see example schema in [Getting Started](/nornir/getting-started.md#example-schema-structure)) * Access to Infrahub GraphQL explorer (for testing queries) * Sandbox GraphQL explorer: [sandbox.infrahub.app/graphql](https://sandbox.infrahub.app/graphql) ## Understanding schema mappings[​](#understanding-schema-mappings "Direct link to Understanding schema mappings") Schema mappings define how to extract values from Infrahub nodes and assign them to Nornir host properties. Each mapping consists of: * `name`: The Nornir property to set (i.e., "hostname", "platform") * `mapping`: The path to the value in the Infrahub node (i.e., "primary\_address.address") ## Step 1: Identify required Nornir properties[​](#step-1-identify-required-nornir-properties "Direct link to Step 1: Identify required Nornir properties") First, determine which Nornir properties your automation tasks need: | Property | Purpose | Common Mapping Source | | ---------- | ----------------------- | -------------------------- | | `hostname` | IP/FQDN for connections | Device primary address | | `platform` | Device OS type | Platform or OS attribute | | `username` | Authentication user | Credential or static value | | `password` | Authentication password | Credential or static value | | `port` | Connection port | Interface or static value | | `data` | Custom data | Any Infrahub attributes | ## Step 2: Explore your Infrahub schema[​](#step-2-explore-your-infrahub-schema "Direct link to Step 2: Explore your Infrahub schema") Use the Infrahub GraphQL explorer to understand available attributes. For the [Infrahub sandbox](https://sandbox.infrahub.app/graphql), you can query: ``` query { InfraDevice { edges { node { name { value } primary_address { node { address { value } } } platform { node { name { value } nornir_platform { value } } } site { node { name { value } } } role { value } } } } } ``` note The query structure depends on your schema. In the sandbox, `role` is a Dropdown attribute (use `value`), while `site` and `platform` are relationships (use `node`). Adapt your queries to match your schema structure. This helps you identify the exact paths to your data based on your schema structure. ## Step 3: Configure basic mappings[​](#step-3-configure-basic-mappings "Direct link to Step 3: Configure basic mappings") ### Basic attribute mapping[​](#basic-attribute-mapping "Direct link to Basic attribute mapping") For direct attributes on the node: ``` schema_mappings: - name: "hostname" mapping: "name" # Maps device name to hostname ``` ### Nested relationship mapping[​](#nested-relationship-mapping "Direct link to Nested relationship mapping") For attributes through relationships (based on the example schema): ``` schema_mappings: - name: "hostname" mapping: "primary_address.address" # Traverses to InfraIPAddress node - name: "platform" mapping: "platform.nornir_platform" # Traverses to InfraPlatform node ``` important Every relationship traversed in `schema_mappings` or `group_mappings` must be listed in `host_node.include`. Without this, the related node's attributes are not fetched and mappings will fail. ``` host_node: kind: "InfraDevice" include: - "primary_address" # required for primary_address.address - "platform" # required for platform.nornir_platform ``` ### Customizing the Nornir host name[​](#customizing-the-nornir-host-name "Direct link to Customizing the Nornir host name") By default, the plugin uses the node's `name` attribute as the Nornir host name. If your schema does not define a `name` attribute, or you want a different attribute to drive the host name, add a schema mapping whose `name` is `"name"`: ``` schema_mappings: - name: "name" mapping: "hostname" # Use the node's "hostname" attribute as the Nornir host name ``` The `mapping` follows the same rules as any other mapping, so you can also resolve the name through a single relationship (for example, `primary_address.address`). When no `name` mapping is provided and the node has no `name` attribute, the host is skipped. ### Multiple level nesting[​](#multiple-level-nesting "Direct link to Multiple level nesting") The plugin supports traversing one relationship level before reaching an attribute. For example, `site.name` follows the `site` relationship then reads its `name` attribute. Deeper chains (e.g., `site.location.name`) are not supported. ## Step 4: Handle complex mappings[​](#step-4-handle-complex-mappings "Direct link to Step 4: Handle complex mappings") ### IP address handling[​](#ip-address-handling "Direct link to IP address handling") Infrahub stores IP addresses as interface objects. The integration automatically extracts the IP: ``` schema_mappings: - name: "hostname" mapping: "primary_address.address" # Automatically converts IPHost to string ``` ### Infrahub Node Object[​](#infrahub-node-object "Direct link to Infrahub Node Object") Along with schema mappings that allow mapping Infrahub inventory data to Nornir host attributes, we also include the Infrahub object that can be used in tasks, etc. ``` def example_task(task): # Access the Infrahub node data infrahub_node = task.host.data["InfrahubNode"] # Access node attributes directly if hasattr(infrahub_node, 'serial_number'): serial = infrahub_node.serial_number.value print(f"Device {task.host.name} serial: {serial}") ``` ## Step 5: Configure group mappings[​](#step-5-configure-group-mappings "Direct link to Step 5: Configure group mappings") Group mappings create dynamic Nornir groups from Infrahub node attributes or relationships: ``` host_node: kind: "InfraDevice" include: - "site" # required for site.name group mapping - "platform" # required for platform.name group mapping group_mappings: - "site.name" # Creates groups like "site__chicago" - "platform.name" # Creates groups like "platform__eos" ``` Groups are automatically prefixed with the first part of the mapping path to avoid conflicts. Additionally, the plugin will auto add groups based on each node's groups it is assigned to within Infrahub. ## Step 6: Test your mappings[​](#step-6-test-your-mappings "Direct link to Step 6: Test your mappings") Create a test script to verify mappings: ``` from nornir import InitNornir def test_mappings(): nr = InitNornir(config_file="config.yaml") # Check hosts print("Sample hosts:") for host in list(nr.inventory.hosts.values())[:3]: # First 3 hosts print(f"\n {host.name}:") print(f" Hostname: {host.hostname}") print(f" Platform: {host.platform}") print(f" Groups: {[g.name for g in host.groups]}") # List all groups print("\nAll groups:") for group in nr.inventory.groups: print(f" - {group}") if __name__ == "__main__": test_mappings() ``` ## Validation[​](#validation "Direct link to Validation") ### Check mapping success[​](#check-mapping-success "Direct link to Check mapping success") After configuration, verify: 1. All hosts have required properties set 2. Groups are created as expected 3. No mapping errors in logs ### Validation Example[​](#validation-example "Direct link to Validation Example") ``` # Check all hosts have hostname for host in nr.inventory.hosts.values(): assert host.hostname is not None, f"{host.name} missing hostname" # Verify platform mappings platforms = {h.platform for h in nr.inventory.hosts.values()} print(f"Discovered platforms: {platforms}") # Check group membership for group in nr.inventory.groups.values(): print(f"{group.name}: {len(group.hosts)} hosts") ``` ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Using mapping results in tasks[​](#using-mapping-results-in-tasks "Direct link to Using mapping results in tasks") Access mapped data in your tasks: ``` def device_info(task): # Access standard properties print(f"Connecting to {task.host.hostname}") print(f"Platform: {task.host.platform}") # Access Infrahub node node = task.host.data["InfrahubNode"] print(f"Infrahub ID: {node.id}") ``` ### Troubleshooting mapping failures[​](#troubleshooting-mapping-failures "Direct link to Troubleshooting mapping failures") If mappings fail silently: 1. Enable debug logging: ``` import logging logging.basicConfig(level=logging.DEBUG) ``` 2. Check the Infrahub GraphQL query result for the specific node: ``` # Add to your script print(vars(nr.inventory.hosts["device1"].data["InfrahubNode"])) ``` 3. Verify relationship cardinality: * Single relationships: `mapping: "platform.name"` * Many relationships not supported directly ## Related resources[​](#related-resources "Direct link to Related resources") * [Understanding Nornir-Infrahub Integration](/nornir/topics/understanding-nornir-infrahub-integration.md) * [Infrahub Inventory Concepts](/nornir/topics/infrahub-inventory-concepts.md) * [Your First Automation Workflow](/nornir/guides/your-first-automation-workflow.md) --- # How to Create Your First Automation Workflow This guide walks you through building a complete network automation workflow using Infrahub as your inventory source and artifact management system. You'll retrieve device configurations managed by Infrahub. ## What you'll accomplish[​](#what-youll-accomplish "Direct link to What you'll accomplish") By the end of this guide, you'll have: * Connected Nornir to your Infrahub instance * Retrieved device configurations from Infrahub artifacts * Used dynamic device groups for targeted operations * Understood how to integrate Nornir with Infrahub's artifact system ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Infrahub instance with network devices and artifact definitions configured * You can use the [Infrahub sandbox instance](https://sandbox.infrahub.app) to try this workflow * Nornir-Infrahub plugin installed (`pip install nornir-infrahub`) * Basic Python and Nornir knowledge ## Step 1: Set up your project structure[​](#step-1-set-up-your-project-structure "Direct link to Step 1: Set up your project structure") Create a project directory with the following structure: ``` mkdir network-automation cd network-automation # Create directory structure mkdir -p configs touch config.yaml touch workflow.py ``` ## Step 2: Configure Nornir with Infrahub[​](#step-2-configure-nornir-with-infrahub "Direct link to Step 2: Configure Nornir with Infrahub") Create your `config.yaml` file: config.yaml ``` --- inventory: plugin: InfrahubInventory options: address: "https://sandbox.infrahub.app" # Infrahub instance URL token: "1808d43b-c370-48e8-d0ef-c51781c02ddf" # Replace with your actual token branch: "main" # Infrahub branch to use # Define which Infrahub node type represents network devices # Using the schema example above host_node: kind: "InfraDevice" # Matches the Device node with Infra namespace include: - "platform" # Map Infrahub attributes to Nornir host properties schema_mappings: - name: "hostname" mapping: "name" # Use device name as hostname - name: "platform" mapping: "platform.nornir_platform" # Platform attribute for Nornir compatibility # Create Nornir groups based on Infrahub attributes # Note: Disabled due to some devices missing site/role in sandbox # group_mappings: # - "site.name" # - "role.name" runner: plugin: threaded options: num_workers: 20 ``` ## Step 3: Understand Infrahub artifact system[​](#step-3-understand-infrahub-artifact-system "Direct link to Step 3: Understand Infrahub artifact system") Before proceeding, ensure your Infrahub instance has artifact definitions configured. In this workflow, we'll use Infrahub's artifact system to retrieve generated configurations rather than creating templates locally. Infrahub artifacts provide: * **Centralized Templates**: Jinja2 templates stored in Git repositories * **Data-Driven Generation**: Configurations generated from your Infrahub data model * **Version Control**: All templates and generated artifacts are versioned * **Consistency**: Ensure all network configurations follow organizational standards The Nornir-Infrahub plugin provides tasks to interact with these artifacts: * `get_artifact()`: Retrieve generated configuration content * `generate_artifacts()`: Trigger artifact generation for targets * `regenerate_host_artifact()`: Regenerate configuration for a specific device ## Step 4: Build the workflow script[​](#step-4-build-the-workflow-script "Direct link to Step 4: Build the workflow script") Create `workflow.py` with your automation workflow: workflow\.py ``` #!/usr/bin/env python3 """Network automation workflow using Nornir and Infrahub artifacts. This module implements a complete configuration management workflow that: 1. Connects to Infrahub to load device inventory 2. Regenerates device configurations using Infrahub's artifact system 3. Retrieves the generated configurations 4. Validates the generated configurations The workflow is designed to work with the Infrahub sandbox environment and targets edge devices that have the "Startup Config for Edge devices" artifact definition configured. Requirements: - nornir: Core automation framework - nornir-infrahub: Infrahub inventory and task plugins - nornir-utils: Result printing utilities Example: Run the workflow from the command line:: $ python workflow.py Or import and customize:: from workflow import main main() """ import logging from pathlib import Path from nornir import InitNornir from nornir.core.task import Task, Result from nornir_utils.plugins.functions import print_result from nornir_infrahub.plugins.tasks import ( get_artifact, regenerate_host_artifact, ) # Configure logging logging.basicConfig( level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s" ) logger = logging.getLogger(__name__) def retrieve_configuration(task: Task) -> Result: """Retrieve device configuration from Infrahub artifacts. Fetches the generated configuration artifact from Infrahub and saves it locally for validation. The configuration is saved to the `configs/` directory with the filename `{hostname}.cfg`. Args: task: Nornir task object containing host information and Infrahub client. Returns: Result object with: - On success: Path to saved configuration file - On failure: Error message describing the issue Raises: No exceptions are raised; all errors are captured in the Result object. """ try: # Get the configuration artifact from Infrahub # Using the artifact name from the sandbox result = get_artifact( task, artifact="Startup Config for Edge devices", ) # Check if we got actual content if not result.result or result.result.strip() == "": return Result( host=task.host, failed=True, result="Artifact exists but has no content", ) # Save configuration locally output_dir = Path("configs") output_dir.mkdir(exist_ok=True) config_file = output_dir / f"{task.host.name}.cfg" config_file.write_text(result.result) return Result( host=task.host, result=f"Configuration retrieved from Infrahub and saved to {config_file}", ) except Exception as e: error_msg = str(e) if "NodeNotFoundError" in error_msg or "Unable to find" in error_msg: return Result( host=task.host, failed=True, result="No artifact found for this device (may not be configured in Infrahub)", ) return Result( host=task.host, failed=True, result=f"Failed to retrieve configuration: {error_msg}", ) def regenerate_configuration(task: Task) -> Result: """Trigger regeneration of device configuration in Infrahub. Requests Infrahub to regenerate the configuration artifact for the host. This ensures the latest data from Infrahub is used to generate a fresh configuration before retrieval. Args: task: Nornir task object containing host information and Infrahub client. Returns: Result object with: - On success: Confirmation message - On failure: Error message describing the issue """ try: # Regenerate the artifact in Infrahub regenerate_host_artifact( task, artifact="Startup Config for Edge devices", ) return Result( host=task.host, result="Configuration regenerated successfully in Infrahub" ) except Exception as e: error_msg = str(e) if "NodeNotFoundError" in error_msg or "Unable to find" in error_msg: return Result( host=task.host, failed=True, result="No artifact definition found for this device", ) return Result( host=task.host, failed=True, result=f"Failed to regenerate configuration: {error_msg}", ) def validate_configuration(task: Task) -> Result: """Validate the retrieved configuration. Performs basic validation checks on the configuration file to ensure it meets basic requirements. Currently validates that the configuration is not empty. Args: task: Nornir task object containing host information. Returns: Result object with: - On success: Validation passed message with file size - On failure: List of failed validation checks """ config_file = Path("configs") / f"{task.host.name}.cfg" if not config_file.exists(): return Result( host=task.host, failed=True, result="Configuration file not found" ) config_content = config_file.read_text() # Basic validation checks - relaxed for sandbox artifacts validations = { "not_empty": len(config_content.strip()) > 0, } failed_checks = [check for check, passed in validations.items() if not passed] if failed_checks: return Result( host=task.host, failed=True, result=f"Validation failed: {', '.join(failed_checks)}", ) return Result( host=task.host, result=f"Configuration validation passed ({len(config_content)} bytes)", ) def main() -> int: """Execute the complete network automation workflow. Orchestrates the following steps: 1. Load device inventory from Infrahub 2. Filter to edge devices (which have startup config artifacts) 3. Regenerate configurations in Infrahub 4. Retrieve configurations to local `configs/` directory 5. Validate all retrieved configurations The workflow will abort if any validation failures occur. Returns: Exit code: 0 for success, 1 for validation failures. """ # Initialize Nornir # Step 1: Load devices/init Nornir dynamic inventory from Infrahub nr = InitNornir(config_file="config.yaml") logger.info(f"Loaded {len(nr.inventory.hosts)} hosts from Infrahub") # Filter to only edge devices (which have the Startup Config artifact) edge_devices = nr.filter(filter_func=lambda h: "edge" in h.name) logger.info( f"Filtered to {len(edge_devices.inventory.hosts)} edge devices with artifacts" ) # Step 2: Regenerate configurations in Infrahub logger.info("Step 2: Regenerating configurations in Infrahub...") regen_results = edge_devices.run(task=regenerate_configuration) print_result(regen_results) # Step 3: Retrieve updated configurations from Infrahub logger.info("Step 3: Retrieving configurations from Infrahub...") retrieve_results = edge_devices.run(task=retrieve_configuration) print_result(retrieve_results) # Step 4: Validate configurations logger.info("Step 4: Validating configurations...") validation_results = edge_devices.run(task=validate_configuration) print_result(validation_results) # Check for validation failures failed_hosts = [ host for host, result in validation_results.items() if result.failed ] if failed_hosts: logger.error(f"Validation failed for hosts: {failed_hosts}") logger.info("Aborting due to validation failures") return 1 return 0 if __name__ == "__main__": raise SystemExit(main()) ``` ## Step 5: Execute the workflow[​](#step-5-execute-the-workflow "Direct link to Step 5: Execute the workflow") Run your automation workflow: ``` python workflow.py ``` The workflow will: 1. Connect to Infrahub and load device inventory 2. Regenerate configurations in Infrahub using artifact system 3. Retrieve updated configurations from Infrahub 4. Validate the retrieved configurations ## Validation[​](#validation "Direct link to Validation") ### Check retrieved configurations[​](#check-retrieved-configurations "Direct link to Check retrieved configurations") Review the configuration files retrieved from Infrahub in the `configs/` directory: ``` ls -la configs/ cat configs/router1.cfg ``` ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Filtering devices[​](#filtering-devices "Direct link to Filtering devices") Target specific devices or groups: ``` from nornir.core.filter import F # Filter by platform junos_devices = nr.filter(platform="juniper_junos") results = junos_devices.run(task=retrieve_configuration) # Filter by group core_devices = nr.filter(F(groups__contains="core_router")) core_results = core_devices.run(task=retrieve_configuration) ``` Further information on [Nornir filters](https://nornir.readthedocs.io/en/latest/tutorial/inventory.html#Filtering-the-inventory). ### Parallel execution control[​](#parallel-execution-control "Direct link to Parallel execution control") Adjust parallelism for different tasks: ``` # Fewer workers for controlled execution nr.config.runner.options["num_workers"] = 2 results = nr.run(task=retrieve_configuration) # More workers for parallel operations nr.config.runner.options["num_workers"] = 50 results = nr.run(task=retrieve_configuration) ``` ### Integration with CI/CD[​](#integration-with-cicd "Direct link to Integration with CI/CD") Create a CI-friendly version: ``` # ci_workflow.py import sys import os from nornir.core.filter import F def ci_workflow(): # Get parameters from environment target_branch = os.getenv("INFRAHUB_BRANCH", "main") target_site = os.getenv("TARGET_SITE", None) # Initialize with CI parameters nr = InitNornir( config_file="config.yaml", inventory={ "options": { "branch": target_branch } } ) # Filter by site group if specified (site.name group_mapping creates groups like "site__chicago") if target_site: nr = nr.filter(F(groups__contains=f"site__{target_site}")) # Run workflow without prompts results = nr.run(task=retrieve_configuration) # Exit with proper code failed_count = len([r for r in results.values() if r.failed]) sys.exit(failed_count) ``` ## Related resources[​](#related-resources "Direct link to Related resources") * [Understanding Nornir-Infrahub Integration](/nornir/topics/understanding-nornir-infrahub-integration.md) * [Configuring Schema Mappings](/nornir/guides/configuring-schema-mappings.md) * [Artifact Lifecycle Management](/nornir/topics/artifact-lifecycle-management.md) * [Nornir Task Documentation](https://nornir.readthedocs.io/) --- # Artifact management plugin ## `regenerate_host_artifact`[​](#regenerate_host_artifact "Direct link to regenerate_host_artifact") Regenerates a host artifact for a given task. This function regenerates the named artifact for the host's InfrahubNode through the Infrahub SDK. ### Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ---------- | ------ | -------- | ------- | ----------------------------------------------- | | `task` | `Task` | Yes | | The task instance containing host-related data. | | `artifact` | `str` | Yes | | The name of the artifact to regenerate. | ### Returns[​](#returns "Direct link to Returns") `Result` — An object representing the outcome of the operation, indicating success or failure. ### Raises[​](#raises "Direct link to Raises") | Exception | Description | | ----------- | ------------------------------------------------------------------------------------ | | `Exception` | Propagates any error raised by the Infrahub SDK if the artifact cannot be generated. | ### Examples[​](#examples "Direct link to Examples") #### Regenerate artifact for a given device[​](#regenerate-artifact-for-a-given-device "Direct link to Regenerate artifact for a given device") ``` from nornir import InitNornir from nornir.core.plugins.inventory import InventoryPluginRegister from nornir_infrahub.plugins.inventory.infrahub import InfrahubInventory from nornir_infrahub.plugins.tasks import regenerate_host_artifact from nornir_utils.plugins.functions import print_result def main(): InventoryPluginRegister.register("InfrahubInventory", InfrahubInventory) nr = InitNornir(inventory=...) eos_devices = nr.filter(platform="eos") # regenerate an artifact for a host print_result(eos_devices.run(task=regenerate_host_artifact, artifact="startup-config")) return 0 if __name__ == "__main__": raise SystemExit(main()) ``` ## `generate_artifacts`[​](#generate_artifacts "Direct link to generate_artifacts") Generates an artifact for a given task. This function retrieves the artifact definition from the InfrahubNode and triggers its generation through the Infrahub SDK. ### Parameters[​](#parameters-1 "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ---------- | ------ | -------- | ------- | -------------------------------------------------------------------- | | `task` | `Task` | Yes | | The task instance containing host-related data. | | `artifact` | `str` | Yes | | The name of the artifact to generate. | | `timeout` | `int` | No | 10 | Retained for compatibility; ignored by the SDK path. Defaults to 10. | ### Returns[​](#returns-1 "Direct link to Returns") `Result` — An object representing the outcome of the operation, indicating success or failure. ### Raises[​](#raises-1 "Direct link to Raises") | Exception | Description | | ----------- | ------------------------------------------------------------------------------------ | | `Exception` | Propagates any error raised by the Infrahub SDK if the artifact cannot be generated. | ### Examples[​](#examples-1 "Direct link to Examples") #### Example generating artifacts[​](#example-generating-artifacts "Direct link to Example generating artifacts") ``` from nornir import InitNornir from nornir.core.plugins.inventory import InventoryPluginRegister from nornir_infrahub.plugins.inventory.infrahub import InfrahubInventory from nornir_infrahub.plugins.tasks import generate_artifacts def main(): InventoryPluginRegister.register("InfrahubInventory", InfrahubInventory) nr = InitNornir(inventory=...) # generate_artifacts, generates the artifact for all the targets in the Artifact definition # we only need to run this task once, per artifact definition run_once = nr.filter(name="jfk1-edge1") result = run_once.run(task=generate_artifacts, artifact="startup-config", timeout=20) ocfg_result = run_once.run(task=generate_artifacts, artifact="openconfig-interfaces", timeout=20) return 0 if __name__ == "__main__": raise SystemExit(main()) ``` ## `get_artifact`[​](#get_artifact "Direct link to get_artifact") Retrieves the specified artifact from the Infrahub storage. This function fetches an artifact node associated with the given artifact name or id and sends a request to retrieve its stored content. The response is returned as JSON or text, depending on the artifact's content type. ### Parameters[​](#parameters-2 "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ------------- | ------ | -------- | ------- | ----------------------------------------------- | | `task` | `Task` | Yes | | The task instance containing host-related data. | | `artifact` | `str` | No | | The name of the artifact to retrieve. | | `artifact_id` | `str` | No | | The id of the artifact to retrieve. | ### Returns[​](#returns-2 "Direct link to Returns") `Result` — An object containing the retrieved artifact data, its content type, and the success status of the operation. ### Raises[​](#raises-2 "Direct link to Raises") | Exception | Description | | -------------- | ------------------------------------------------------------------------------------ | | `RuntimeError` | If neither or both of `artifact` and `artifact_id` are provided. | | `Exception` | Propagates any error raised by the Infrahub SDK if the artifact cannot be retrieved. | ### Examples[​](#examples-2 "Direct link to Examples") #### Example getting artifacts from Infrahub[​](#example-getting-artifacts-from-infrahub "Direct link to Example getting artifacts from Infrahub") ``` from nornir import InitNornir from nornir.core.plugins.inventory import InventoryPluginRegister from nornir_infrahub.plugins.inventory.infrahub import InfrahubInventory from nornir_infrahub.plugins.tasks import get_artifact from nornir_utils.plugins.functions import print_result def main(): InventoryPluginRegister.register("InfrahubInventory", InfrahubInventory) nr = InitNornir(inventory=...) eos_devices = nr.filter(platform="eos") # retrieves the artifact for all the hosts in the inventory result = eos_devices.run(task=get_artifact, artifact="startup-config") print_result(result) return 0 if __name__ == "__main__": raise SystemExit(main()) ``` --- # CoreFileObject management tasks ## `upload_file_object`[​](#upload_file_object "Direct link to upload_file_object") Uploads a file to an Infrahub CoreFileObject object. Creates the object if it does not exist, or updates it if the file content has changed (SHA-1 checksum comparison). When the local content is identical to the one stored on the server the upload is skipped (idempotent). Provide either `file_path` (to upload a file from disk) or `content` with `file_name` (to upload raw bytes). ### Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ----------- | ------------- | -------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `task` | `Task` | Yes | | The Nornir task instance containing host-related data. | | `kind` | `str` | Yes | | The schema kind that inherits from CoreFileObject. | | `file_path` | `str \| Path` | No | | Filesystem path to the file to upload. | | `content` | `bytes` | No | | Raw file content to upload. Requires `file_name`. | | `file_name` | `str` | No | | File name to associate with `content`. Ignored when `file_path` is used. | | `data` | `dict` | No | | Additional object fields. On create, this is forwarded to `client.create` (attributes and relationships). On update, only attribute keys are applied; passing a relationship or unknown key fails the task. | | `object_id` | `str` | No | | UUID of an existing object to update. | | `hfid` | `list[str]` | No | | HFID components identifying an existing object. | | `branch` | `str` | No | the client's default branch | Target Infrahub branch. Defaults to the client's default branch. | | `**kwargs` | `Any` | No | | Extra keyword arguments forwarded to `client.create` (e.g. `allow_upsert`, `timeout`). Applies only on the create path; the update path does not call `client.create`, so these are not used when updating an existing object. | ### Returns[​](#returns "Direct link to Returns") `Result` — A Nornir Result with `changed` indicating whether the object was created or updated, `object_id` with the UUID, and a descriptive `result` message. ### Raises[​](#raises "Direct link to Raises") | Exception | Description | | ------------ | ----------------------------------------------------------------------------- | | `ValueError` | If *kind* does not inherit from CoreFileObject or arguments are inconsistent. | ### Examples[​](#examples "Direct link to Examples") #### Upload a contract PDF to a CoreFileObject object[​](#upload-a-contract-pdf-to-a-corefileobject-object "Direct link to Upload a contract PDF to a CoreFileObject object") ``` from nornir_infrahub.plugins.tasks import upload_file_object result = nr.run( task=upload_file_object, kind="NetworkCircuitContract", file_path="/path/to/contract.pdf", data={"contract_start": "2026-01-01"}, ) ``` ## `download_file_object`[​](#download_file_object "Direct link to download_file_object") Downloads file content from an Infrahub CoreFileObject object. Returns the file content as base64-encoded binary data, with a UTF-8 text representation for text MIME types. Optionally saves the file to a local path. When `save_to` points to an existing file whose SHA-1 matches the server-side checksum, the download is skipped and `changed` is `False` (idempotent, similar to `nornir_utils.plugins.tasks.files.write_file`). Note: The downloaded content is held in memory and embedded as base64 in the Nornir `Result`. Nornir aggregates results across hosts, so downloading large files across many hosts can be heavy. For large payloads, prefer `save_to` and ignore the `binary`/`text` fields, or call the infrahub-sdk download API directly. ### Parameters[​](#parameters-1 "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ----------- | ------------- | -------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `task` | `Task` | Yes | | The Nornir task instance containing host-related data. | | `kind` | `str` | Yes | | The schema kind that inherits from CoreFileObject. | | `object_id` | `str` | No | | UUID of the object to download from. | | `hfid` | `list[str]` | No | | HFID components identifying the object. | | `save_to` | `str \| Path` | No | | Local path where the downloaded file should be written. A directory receives the original file name; an explicit path is used as-is. | | `branch` | `str` | No | the client's default branch | Target Infrahub branch. Defaults to the client's default branch. | ### Returns[​](#returns-1 "Direct link to Returns") `Result` — A Nornir Result containing `binary` (base64 string), `text` (UTF-8 string or None), `file_name`, `file_type`, `file_size`, `checksum`, `object_id`, `save_to`, and `changed` (True only when a local file was created or overwritten). ### Raises[​](#raises-1 "Direct link to Raises") | Exception | Description | | ------------ | ----------------------------------------------- | | `ValueError` | If *kind* does not inherit from CoreFileObject. | ### Examples[​](#examples-1 "Direct link to Examples") #### Download a file from a CoreFileObject object[​](#download-a-file-from-a-corefileobject-object "Direct link to Download a file from a CoreFileObject object") ``` from nornir_infrahub.plugins.tasks import download_file_object result = nr.run( task=download_file_object, kind="NetworkCircuitContract", hfid=["contract-2026"], ) ``` --- # Inventory plugin ## `InfrahubInventory`[​](#infrahubinventory "Direct link to infrahubinventory") Nornir inventory plugin for integrating with `Opsmill - Infrahub` (). This plugin fetches inventory data from Infrahub, maps it to Nornir Hosts, and supports the creation of Nornir groups based on attributes or relations from Infrahub Nodes. ### Parameters[​](#parameters "Direct link to Parameters") | Parameter | Type | Required | Default | Description | | ----------------- | ------ | -------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `address` | `str` | No | "" | The Infrahub URL to connect to. Defaults to "". | | `branch` | `str` | No | "main" | The Infrahub branch to use. Defaults to "main". | | `host_node` | `dict` | Yes | | A dictionary defining the Infrahub Node kind that will be mapped to Nornir Hosts. Example: `{"kind": "InfraDevice"}` | | `schema_mappings` | `list` | Yes | | A list of mappings that define how Nornir Host properties correspond to attributes or relations from Infrahub Nodes. A mapping with `name: "name"` customizes the Nornir host name (default: the node's `name` attribute). Example: `[{"name": "hostname", "mapping": "primary_address.address"}, {"name": "name", "mapping": "hostname"}]`. | | `group_mappings` | `list` | Yes | | A list of Infrahub Node attributes or relations used to create Nornir groups. Example: `["site.name"]`. | | `defaults_file` | `str` | No | "defaults.yaml" | Path to the defaults YAML file. Defaults to "defaults.yaml". | | `group_file` | `str` | No | "group.yaml" | Path to the group YAML file. Defaults to "group.yaml". | ### Examples[​](#examples "Direct link to Examples") #### Basic usage of `InfrahubInventory` with Nornir[​](#basic-usage-of-infrahubinventory-with-nornir "Direct link to basic-usage-of-infrahubinventory-with-nornir") ``` from nornir import InitNornir from nornir.core.plugins.inventory import InventoryPluginRegister from nornir_infrahub.plugins.inventory.infrahub import InfrahubInventory def main(): # Register the custom InfrahubInventory plugin InventoryPluginRegister.register("InfrahubInventory", InfrahubInventory) # Initialize Nornir with InfrahubInventory as the inventory plugin nr = InitNornir( inventory={ "plugin": "InfrahubInventory", "options": { "address": "http://localhost:8000", # Infrahub API URL "token": "06438eb2-8019-4776-878c-0941b1f1d1ec", # Infrahub API token "host_node": {"kind": "InfraDevice"}, # Infrahub Node kind to map to Nornir Hosts "schema_mappings": [ {"name": "hostname", "mapping": "primary_address.address"}, {"name": "platform", "mapping": "platform.nornir_platform"}, ], # Mapping Nornir Host properties to Infrahub Node attributes "group_mappings": ["site.name"], # Create Nornir groups from Infrahub Node attributes "group_file": "dummy.yml", # Path to the group file }, } ) # Print Nornir inventory host and group names print(nr.inventory.hosts.keys()) print(nr.inventory.groups.keys()) return 0 if __name__ == "__main__": raise SystemExit(main()) ``` --- # Understanding Artifact Lifecycle Management This topic explores how Infrahub's artifact system works with Nornir to enable sophisticated configuration management workflows. Understanding artifact lifecycle management is crucial for implementing robust, auditable network automation. ## Introduction[​](#introduction "Direct link to Introduction") Artifacts in Infrahub are generated outputs based on your infrastructure data and templates. They represent the "rendered truth" of your infrastructure at any point in time. This document explains: * What artifacts are and why they matter * How artifacts flow through generation, storage, and deployment * The relationship between artifacts, templates, and infrastructure data * How version control applies to generated configurations ## What are artifacts?[​](#what-are-artifacts "Direct link to What are artifacts?") ### Definition[​](#definition "Direct link to Definition") An artifact is any generated content that represents infrastructure state: * Device configurations * Compliance reports * Documentation * API payloads * Validation results ### Key characteristics[​](#key-characteristics "Direct link to Key characteristics") Artifacts are: * **Generated**: Created from templates + data, not manually written * **Versioned**: Each generation creates a new version * **Immutable**: Once generated, artifacts don't change * **Traceable**: Linked to the exact data and templates used ### References[​](#references "Direct link to References") For detailed information about artifacts in Infrahub: * [Artifact Topic](https://docs.infrahub.app/topics/artifact) - How artifacts work in Infrahub * [Transformation Topic](https://docs.infrahub.app/topics/transformation) - How transformations generate artifacts * [Repository Configuration Reference](https://docs.infrahub.app/reference/dotinfrahub) - `.infrahub.yml` artifact definition syntax ## Artifact management with Nornir[​](#artifact-management-with-nornir "Direct link to Artifact management with Nornir") ### Task plugin functions[​](#task-plugin-functions "Direct link to Task plugin functions") The Nornir-Infrahub plugin provides three key functions: #### [generate\_artifacts()](/nornir/references/plugins/artifact_tasks.md#generate_artifacts)[​](#generate_artifacts "Direct link to generate_artifacts") Triggers generation for all targets in a definition: ``` # Generate configs for all edge devices nr.run(task=generate_artifacts, artifact="Startup Config for Edge devices") ``` This is typically used: * After template updates * On scheduled intervals * Before deployment windows #### [regenerate\_host\_artifact()](/nornir/references/plugins/artifact_tasks.md#regenerate_host_artifact)[​](#regenerate_host_artifact "Direct link to regenerate_host_artifact") Regenerates artifact for specific hosts: ``` # Regenerate after device-specific change changed_devices = nr.filter(name="atl1-edge1") changed_devices.run(task=regenerate_host_artifact, artifact="Startup Config for Edge devices") ``` Use cases: * After individual device updates * Testing template changes * Troubleshooting specific devices #### [get\_artifact()](/nornir/references/plugins/artifact_tasks.md#get_artifact)[​](#get_artifact "Direct link to get_artifact") Retrieves stored artifact content: ``` # Get current config for deployment result = nr.run(task=get_artifact, artifact="Startup Config for Edge devices") for host, task_result in result.items(): config = task_result.result # Deploy config to device ``` Use cases: * Deploying configurations to devices * Comparing current vs desired state * Feeding into validation pipelines ## Version control integration[​](#version-control-integration "Direct link to Version control integration") ### Branch-aware artifacts[​](#branch-aware-artifacts "Direct link to Branch-aware artifacts") Artifacts are generated per branch: ``` main branch: atl1-edge1 startup-config v3 (current) atl1-edge1 startup-config v2 atl1-edge1 startup-config v1 feature branch: atl1-edge1 startup-config v2 (testing changes) atl1-edge1 startup-config v1 ``` This enables: * Safe testing in feature branches * Comparison between branches * Controlled promotion to production ## Practical workflows[​](#practical-workflows "Direct link to Practical workflows") ### Configuration deployment[​](#configuration-deployment "Direct link to Configuration deployment") 1. **Generate**: Create new configs from current data 2. **Validate**: Check configs before deployment 3. **Deploy**: Push to devices 4. **Verify**: Confirm successful application ### Compliance auditing[​](#compliance-auditing "Direct link to Compliance auditing") 1. **Define**: Create compliance report template 2. **Generate**: Run against all devices 3. **Review**: Identify non-compliant devices 4. **Remediate**: Fix issues and regenerate ### Disaster recovery[​](#disaster-recovery "Direct link to Disaster recovery") 1. **Regular Generation**: Keep artifacts current 2. **Backup Storage**: Replicate artifact store 3. **Rapid Recovery**: Deploy last-known-good configs ## Advanced concepts[​](#advanced-concepts "Direct link to Advanced concepts") ### Conditional generation[​](#conditional-generation "Direct link to Conditional generation") Generate only when needed: ``` # Check if regeneration needed if device.last_modified > artifact.generated_at: nr.run(task=regenerate_host_artifact, artifact="Startup Config for Edge devices") ``` ## Further reading[​](#further-reading "Direct link to Further reading") * [Your First Automation Workflow](/nornir/guides/your-first-automation-workflow.md) - Implement artifact workflows with Nornir * [Artifact Management Plugin Reference](/nornir/references/plugins/artifact_tasks.md) - Full API documentation for artifact tasks --- # Infrahub Inventory Concepts This topic provides a deep understanding of how Infrahub's graph database model maps to Nornir's inventory concepts. Understanding these mappings is essential for designing effective automation workflows. ## Introduction[​](#introduction "Direct link to Introduction") The InfrahubInventory plugin bridges two different paradigms: * **Infrahub**: Graph database with nodes, relationships, and attributes * **Nornir**: Host-centric inventory with groups and connection parameters This document explores: * How graph concepts translate to inventory structures * The role of schema mappings in this translation * Dynamic group generation from relationships * Performance considerations and best practices ## Core mapping concepts[​](#core-mapping-concepts "Direct link to Core mapping concepts") ### Nodes to hosts[​](#nodes-to-hosts "Direct link to Nodes to hosts") In Infrahub, devices are nodes in the graph. The inventory plugin transforms these into Nornir hosts: ``` Infrahub Node Nornir Host ┌─────────────────────┐ ┌─────────────────────┐ │ InfraDevice │ ──▶ │ Host │ │ - id: abc123 │ │ - name: router1 │ │ - name: router1 │ │ - hostname: 10.1.1.1│ │ - primary_address │ │ - platform: ios │ │ └─▶ 10.1.1.1 │ │ - groups: [...] │ └─────────────────────┘ └─────────────────────┘ ``` ### Attributes to properties[​](#attributes-to-properties "Direct link to Attributes to properties") Node attributes become host properties through schema mappings: | Infrahub Attribute | Mapping Path | Nornir Property | | ------------------ | ------------------------- | --------------- | | Direct attribute | `name` | `host.name` | | Nested relation | `primary_address.address` | `host.hostname` | ## The inventory loading process[​](#the-inventory-loading-process "Direct link to The inventory loading process") ### Step 1: query construction[​](#step-1-query-construction "Direct link to Step 1: query construction") The plugin builds a GraphQL query based on your configuration: ``` query { InfraDevice(status__value: "active") { edges { node { id name { value } primary_address { node { address { value } } } site { node { name { value } } } member_of_groups { edges { node { name { value } } } } } } } } ``` ### Step 2: data transformation[​](#step-2-data-transformation "Direct link to Step 2: data transformation") Retrieved data undergoes transformation: 1. **Flatten nested structures**: Navigate relationships to extract values 2. **Apply type conversions**: Convert IP objects to strings 3. **Handle missing data**: Use defaults or skip mappings 4. **Preserve metadata**: Store full node data for tasks ### Step 3: group generation[​](#step-3-group-generation "Direct link to Step 3: group generation") Groups are created from: 1. **Infrahub group memberships**: Devices assigned to `CoreStandardGroup` in Infrahub 2. **Dynamic groups**: Generated from `group_mappings` configuration 3. **Hierarchical groups**: Groups can have parent groups config.yaml ``` group_mappings: - "site.name" - "role.name" ``` ``` # For a device where site.name = "chicago" and role.name = "spine" # Results in Nornir groups: host.groups = [ "core_router", # Infrahub group membership "site__chicago", # dynamic groups from group_mappings "role__spine" # dynamic groups from group_mappings ] ``` ## Advanced inventory concepts[​](#advanced-inventory-concepts "Direct link to Advanced inventory concepts") ### Filtering at source[​](#filtering-at-source "Direct link to Filtering at source") Reduce data transfer by filtering in Infrahub: ``` host_node: kind: "InfraDevice" filters: status__value: "active" site__name__values: ["chicago", "dallas"] role__value: "spine" ``` This translates to GraphQL filters, ensuring only relevant nodes are fetched. Use `__value` for single values and `__values` for lists. ### Include optimization[​](#include-optimization "Direct link to Include optimization") Minimize query complexity with selective includes: ``` host_node: kind: "InfraDevice" include: - "name" - "primary_address" - "platform" # Only fetch what you need ``` ### Relationship cardinality[​](#relationship-cardinality "Direct link to Relationship cardinality") Understanding cardinality is crucial: * **One-to-One**: Device → Primary Address (direct mapping) * **One-to-Many**: Device → Interfaces (requires special handling) * **Many-to-Many**: Devices ↔ Services (complex relationships) Currently, the plugin handles one-to-one and many-to-one relationships directly. ## Performance considerations[​](#performance-considerations "Direct link to Performance considerations") ### Query optimization[​](#query-optimization "Direct link to Query optimization") Large inventories require optimization: 1. **Pagination**: Automatic handling of large result sets 2. **Selective fetching**: Only request needed attributes 3. **Relationship depth**: Limit traversal depth ### Inventory loading[​](#inventory-loading "Direct link to Inventory loading") Nornir loads the inventory once during initialization: ``` # Inventory is fetched from Infrahub during initialization nr = InitNornir(config_file="config.yaml") # Subsequent operations use the loaded inventory nr.filter(platform="ios") # No new query nr.run(task=my_task) # Uses loaded inventory ``` ### Parallel processing[​](#parallel-processing "Direct link to Parallel processing") Inventory loading is optimized for parallel task execution: * Host data is independent * Groups are pre-computed * No shared state between hosts ## Dynamic inventory patterns[​](#dynamic-inventory-patterns "Direct link to Dynamic inventory patterns") ### Environment-based inventory[​](#environment-based-inventory "Direct link to Environment-based inventory") Use branches for environments: ``` def get_inventory(environment="production"): branch_map = { "production": "main", "staging": "staging", "development": "dev" } return InitNornir( inventory={ "plugin": "InfrahubInventory", "options": { "host_node": {"kind": "InfraDevice"}, "branch": branch_map.get(environment, "main") } } ) ``` ### Role-based filtering[​](#role-based-filtering "Direct link to Role-based filtering") Create specialized inventories: ``` # Core network devices only core_nr = InitNornir( inventory={ "plugin": "InfrahubInventory", "options": { "host_node": { "kind": "InfraDevice", "filters": { "role__values": ["core", "edge"] } } } } ) ``` ### Multi-platform support[​](#multi-platform-support "Direct link to Multi-platform support") Handle different device platforms: config.yaml ``` schema_mappings: - name: "hostname" mapping: "primary_address.address" - name: "platform" mapping: "platform.nornir_platform" # Groups by platform automatically group_mappings: - "platform.name" ``` ## Troubleshooting inventory issues[​](#troubleshooting-inventory-issues "Direct link to Troubleshooting inventory issues") ### Debug mode[​](#debug-mode "Direct link to Debug mode") Enable detailed logging: ``` import logging logging.basicConfig(level=logging.DEBUG) nr = InitNornir(config_file="config.yaml") # Watch for GraphQL queries and responses ``` ### Validation queries[​](#validation-queries "Direct link to Validation queries") Test your mappings with GraphQL using the Infrahub GraphQL explorer or SDK: ``` import asyncio from infrahub_sdk import InfrahubClient async def test_query(): client = InfrahubClient(address="http://localhost:8000") query = """ query { InfraDevice { edges { node { name { value } primary_address { node { address { value } } } } } } } """ result = await client.execute_graphql(query=query) print(result) asyncio.run(test_query()) ``` ### Common issues[​](#common-issues "Direct link to Common issues") 1. **Missing Hosts**: Check filters and node kind 2. **Empty Groups**: Verify relationship paths exist 3. **Mapping Errors**: Ensure attributes are populated 4. **Performance**: Reduce included attributes ## Integration with Nornir ecosystem[​](#integration-with-nornir-ecosystem "Direct link to Integration with Nornir ecosystem") ### Transform functions[​](#transform-functions "Direct link to Transform functions") Nornir provides a `transform_function` mechanism to enrich host data during inventory loading. Register a transform function and reference it by name: ``` from nornir import InitNornir from nornir.core.plugins.inventory import TransformFunctionRegister def enrich_host(host): """Transform function called for each host during inventory loading.""" # Add computed properties based on naming conventions host.data["is_core"] = "core" in host.name host.data["is_edge"] = "edge" in host.name # Derive datacenter from hostname prefix if "-" in host.name: host.data["datacenter"] = host.name.split("-")[0] # Register the transform function TransformFunctionRegister.register("enrich_host", enrich_host) # Use transform_function in InitNornir nr = InitNornir( inventory={ "plugin": "InfrahubInventory", "options": { "address": "http://localhost:8000", "host_node": {"kind": "InfraDevice"} }, "transform_function": "enrich_host" } ) # Filter using enriched data core_devices = nr.filter(filter_func=lambda h: h.data.get("is_core", False)) print(f"Found {len(core_devices.inventory.hosts)} core devices") ``` ### Using Infrahub node data in tasks[​](#using-infrahub-node-data-in-tasks "Direct link to Using Infrahub node data in tasks") Access the full Infrahub node object within tasks for attributes not mapped to Nornir properties: ``` from nornir.core.task import Task, Result def device_audit(task: Task) -> Result: """Audit device using both Nornir properties and Infrahub data.""" # Access standard Nornir host properties hostname = task.host.hostname platform = task.host.platform # Access the full Infrahub node for additional attributes infrahub_node = task.host.data.get("InfrahubNode") audit_info = { "name": task.host.name, "hostname": hostname, "platform": platform, } # Extract additional Infrahub attributes if available if infrahub_node: if hasattr(infrahub_node, "role") and infrahub_node.role: audit_info["role"] = infrahub_node.role.value return Result(host=task.host, result=audit_info) # Run the audit task results = nr.run(task=device_audit) ``` ### Custom inventory plugins[​](#custom-inventory-plugins "Direct link to Custom inventory plugins") Create a custom inventory class to apply transformations during loading: ``` from nornir_infrahub.plugins.inventory.infrahub import InfrahubInventory from nornir.core.plugins.inventory import InventoryPluginRegister class CustomInfrahubInventory(InfrahubInventory): """Custom inventory with platform name normalization.""" def load(self): inventory = super().load() # Normalize platform names for Nornir/NAPALM compatibility platform_map = { "juniper_junos": "junos", "arista_eos": "eos", "cisco_ios": "ios", } for host in inventory.hosts.values(): if host.platform in platform_map: host.platform = platform_map[host.platform] return inventory # Register and use the custom plugin InventoryPluginRegister.register("CustomInfrahubInventory", CustomInfrahubInventory) nr = InitNornir( inventory={ "plugin": "CustomInfrahubInventory", "options": { "address": "http://localhost:8000", "host_node": { "kind": "InfraDevice", "include": ["platform"] }, "schema_mappings": [ {"name": "platform", "mapping": "platform.nornir_platform"} ] } } ) ``` ## Further reading[​](#further-reading "Direct link to Further reading") * [Understanding Nornir-Infrahub Integration](/nornir/topics/understanding-nornir-infrahub-integration.md) - High-level concepts * [Configuring Schema Mappings](/nornir/guides/configuring-schema-mappings.md) - Practical mapping guide * [Infrahub Schema Documentation](https://docs.infrahub.app/topics/schema) - Schema design guide --- # Understanding the Nornir-Infrahub Integration This topic explains the core concepts behind the Nornir-Infrahub integration, how it works, and why it represents a paradigm shift in network automation. ## Introduction[​](#introduction "Direct link to Introduction") The Nornir-Infrahub integration fundamentally changes how network automation handles inventory and configuration management. Instead of maintaining static files or basic databases, this integration leverages Infrahub's graph database and version control capabilities to create a dynamic, versioned, and relationship-aware automation platform. This document answers key questions: * How does Infrahub's data model translate to Nornir's inventory system? * What makes this approach different from traditional inventory management? * How do version control concepts apply to network automation? * What new capabilities does this integration enable? ## Core concepts[​](#core-concepts "Direct link to Core concepts") ### The graph database advantage[​](#the-graph-database-advantage "Direct link to The graph database advantage") Traditional network inventory systems store data in flat files or relational databases. This creates challenges: * **Rigid relationships**: Difficult to model complex network topologies * **Limited context**: Device data exists in isolation * **No history**: Changes overwrite previous state Infrahub's graph database approach solves these issues: ``` ┌─────────────┐ belongs_to ┌──────────┐ │ Device │──────────────────▶│ Site │ └─────────────┘ └──────────┘ │ │ │ has_interface │ located_in ▼ ▼ ┌─────────────┐ ┌──────────┐ │ Interface │ │ Location │ └─────────────┘ └──────────┘ ``` This graph structure naturally represents network relationships, making it straightforward to: * Navigate from devices to their interfaces, sites, and roles * Query based on any relationship or attribute * Maintain referential integrity automatically ### Dynamic inventory generation[​](#dynamic-inventory-generation "Direct link to Dynamic inventory generation") The integration transforms Infrahub's graph data into Nornir's inventory structure in real-time: 1. **Query Infrahub**: Fetch nodes based on configured criteria 2. **Apply mappings**: Transform Infrahub attributes to Nornir properties 3. **Build relationships**: Create groups from Infrahub relationships 4. **Inject metadata**: Include full Infrahub node data for tasks This process happens transparently when you initialize Nornir, ensuring your inventory always reflects the current state in Infrahub. ### Version control for infrastructure[​](#version-control-for-infrastructure "Direct link to Version control for infrastructure") Infrahub brings Git-like version control to infrastructure data: #### Branches[​](#branches "Direct link to Branches") Just as code has development branches, your infrastructure can too: ``` # Work with production data nr_prod = InitNornir( inventory={ "plugin": "InfrahubInventory", "options": { "host_node": {"kind": "InfraDevice"}, "branch": "main" } } ) # Test changes in development nr_dev = InitNornir( inventory={ "plugin": "InfrahubInventory", "options": { "host_node": {"kind": "InfraDevice"}, "branch": "development" } } ) ``` #### Change tracking[​](#change-tracking "Direct link to Change tracking") Every modification in Infrahub is tracked: * Who made the change * When it was made * What was changed * Why (via commit messages) This audit trail is invaluable for compliance and troubleshooting. ## Architecture and design[​](#architecture-and-design "Direct link to Architecture and design") ### Component interaction[​](#component-interaction "Direct link to Component interaction") The integration consists of three main components: 1. **InfrahubInventory Plugin**: Bridges Infrahub and Nornir 2. **Schema Mappings**: Define how Infrahub data maps to Nornir 3. **Task Plugins**: Interact with Infrahub artifacts ``` ┌─────────────┐ HTTP/GraphQL ┌──────────────┐ │ Nornir │◀──────────────────▶│ Infrahub │ │ │ │ │ │ ┌─────────┐ │ │ ┌──────────┐ │ │ │Inventory│ │ │ │ Graph │ │ │ │ Plugin │ │ │ │Database │ │ │ └─────────┘ │ │ └──────────┘ │ │ │ │ │ │ ┌─────────┐ │ │ ┌──────────┐ │ │ │ Task │ │ │ │Artifacts │ │ │ │Plugins │ │ │ │ Storage │ │ │ └─────────┘ │ │ └──────────┘ │ └─────────────┘ └──────────────┘ ``` ### Data flow[​](#data-flow "Direct link to Data flow") 1. **Initialization**: Nornir requests inventory from the plugin 2. **Query**: Plugin queries Infrahub via GraphQL 3. **Transform**: Results are mapped to Nornir's data model 4. **Enrichment**: Additional metadata is preserved 5. **Execution**: Tasks run with access to full Infrahub context ## Further reading[​](#further-reading "Direct link to Further reading") * [Infrahub Inventory Concepts](/nornir/topics/infrahub-inventory-concepts.md) - Deep dive into inventory mechanics * [Artifact Lifecycle Management](/nornir/topics/artifact-lifecycle-management.md) - Understanding configuration artifacts * [Your First Automation Workflow](/nornir/guides/your-first-automation-workflow.md) - Practical implementation guide --- # Executing queries in a batch The Python SDK client allows you to group the execution of multiple queries in a batch. Using a batch to execute code concurrently helps reducing the amount of boilerplate code you would have to write. Additionally a batch allows you to control how many queries can be executed concurrently, preventing you from overwhelming the Infrahub API server. A batch is only available with the async version of the SDK client. ## Using batches[​](#using-batches "Direct link to Using batches") Using a batch is a 3 step process: 1. first, we instantiate a batch object 2. we can then add a tasks to the batch, a task can be any Python `Callable` that returns an `Awaitable`, like a coroutine 3. finally we execute the batch and process the results ``` import asyncio from infrahub_sdk import InfrahubClient async def main(): client = InfrahubClient() batch = await client.create_batch() for tag in ["red", "green", "blue", "yellow", "orange"]: batch.add(task=client.get, kind="BuiltinTag", name__value=tag) async for _, result in batch.execute(): print(result.name.value) if __name__ == "__main__": asyncio.run(main()) ``` Additional arguments passed in the `batch.add` method call, are being passed as arguments to the task when it gets executed. ## Controlling the amount of concurrent queries[​](#controlling-the-amount-of-concurrent-queries "Direct link to Controlling the amount of concurrent queries") By default a batch will execute 5 tasks concurrently, but this can be controlled with the `max_concurrent_execution` configuration variable for the SDK client. ``` import asyncio from infrahub_sdk import Config, InfrahubClient async def main(): client = InfrahubClient(Config(max_concurrent_execution=10)) batch = await client.create_batch() for tag in ["red", "green", "blue", "yellow", "orange"]: batch.add(task=client.get, kind="BuiltinTag", name__value=tag) async for _, result in batch.execute(): print(result.name.value) if __name__ == "__main__": asyncio.run(main()) ``` ## Handling exceptions in tasks[​](#handling-exceptions-in-tasks "Direct link to Handling exceptions in tasks") When a batch executes a task that raises an exception, it will re-raise the exception and this will need to be handled in the calling function. Other tasks that were executing or tasks that have not started yet will not be executed or complete. ``` import asyncio from infrahub_sdk import InfrahubClient async def will_raise(swallowed: bool): raise Exception() async def main(): client = InfrahubClient() batch = await client.create_batch() batch.add(task=client.get, kind="BuiltinTag", name__value="red") batch.add(task=will_raise, swallowed=True) batch.add(task=client.get, kind="BuiltinTag", name__value="green") async for _, result in batch.execute(): print(result.name.value) if __name__ == "__main__": asyncio.run(main()) ``` Alternatively we can choose to return the exception raised in a task. This has the advantage that we can still handle the exception in the calling function and that the other batch tasks will be executed. ``` import asyncio from infrahub_sdk import InfrahubClient async def will_raise(swallowed: bool): raise Exception() async def main(): client = InfrahubClient() batch = await client.create_batch(return_exceptions=True) batch.add(task=client.get, kind="BuiltinTag", name__value="red") batch.add(task=will_raise, swallowed=True) batch.add(task=client.get, kind="BuiltinTag", name__value="green" ) async for _, result in batch.execute(): if isinstance(result, Exception): print("this task has failed") print(result.name.value) if __name__ == "__main__": asyncio.run(main()) ``` ## Adding extra contexts for tasks[​](#adding-extra-contexts-for-tasks "Direct link to Adding extra contexts for tasks") A task can be provided with extra context, through the `node` argument. The goal is to be able to provide more context when a task has completed. This can be especially useful when a task does not return a useful value. Imagine a scenario where we want to create 100 tag objects. The `save` method that we have to call to save the object in the database does not return a useful result. Therefor we will pass the tag object as the node argument to the task, so we can inform the user when the task completes. ``` import asyncio from infrahub_sdk import InfrahubClient async def main(): client = InfrahubClient() batch = await client.create_batch() for idx in range(100): tag = await client.create(kind="BuiltinTag", name=f"tag-{idx}") batch.add(task=tag.save, node=tag, allow_upsert=True) async for node, result in batch.execute(): print(f"{node.name.value} was created in Infrahub succesfully") if __name__ == "__main__": asyncio.run(main()) ``` --- # Branch management The Python SDK provides multiple methods to manage the branches in an Infrahub instance. ## Get a single branch[​](#get-a-single-branch "Direct link to Get a single branch") * Async * Sync ``` from infrahub_sdk import InfrahubClient client = InfrahubClient() branch = await client.branch.get(branch_name="main") ``` ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync() branch = client.branch.get(branch_name="main") ``` ## Get all the branches[​](#get-all-the-branches "Direct link to Get all the branches") * Async * Sync ``` from asyncio import run as aiorun from rich import print as rprint from infrahub_sdk import InfrahubClient async def main(): client = InfrahubClient(address="http://localhost:8000") branches = await client.branch.all() rprint(branches) if __name__ == "__main__": aiorun(main()) ``` ``` from rich import print as rprint from infrahub_sdk import InfrahubClientSync def main(): client = InfrahubClientSync(address="http://localhost:8000") branches = client.branch.all() rprint(branches) if __name__ == "__main__": main() ``` ## Create a branch[​](#create-a-branch "Direct link to Create a branch") * Async * Sync ``` from asyncio import run as aiorun from infrahub_sdk import InfrahubClient async def main(): client = InfrahubClient(address="http://localhost:8000") await client.branch.create(branch_name="new-branch", description="description", sync_with_git=False) print("New branch created") if __name__ == "__main__": aiorun(main()) ``` ``` from infrahub_sdk import InfrahubClientSync def main(): client = InfrahubClientSync(address="http://localhost:8000") client.branch.create(branch_name="new-branch2", description="description", sync_with_git=False) print("New branch created") if __name__ == "__main__": main() ``` ## Rebase a branch[​](#rebase-a-branch "Direct link to Rebase a branch") * Async * Sync ``` from asyncio import run as aiorun from infrahub_sdk import InfrahubClient async def main(): client = InfrahubClient(address="http://localhost:8000") await client.branch.rebase(branch_name="new-branch") if __name__ == "__main__": aiorun(main()) ``` ``` from infrahub_sdk import InfrahubClientSync def main(): client = InfrahubClientSync(address="http://localhost:8000") client.branch.rebase(branch_name="new-branch") if __name__ == "__main__": main() ``` ## Merge a branch[​](#merge-a-branch "Direct link to Merge a branch") * Async * Sync ``` from asyncio import run as aiorun from infrahub_sdk import InfrahubClient async def main(): client = InfrahubClient(address="http://localhost:8000") await client.branch.merge(branch_name="new-branch") if __name__ == "__main__": aiorun(main()) ``` ``` from infrahub_sdk import InfrahubClientSync def main(): client = InfrahubClientSync(address="http://localhost:8000") client.branch.merge(branch_name="new-branch") if __name__ == "__main__": main() ``` ## Delete a branch[​](#delete-a-branch "Direct link to Delete a branch") * Async * Sync ``` from infrahub_sdk import InfrahubClient client = InfrahubClient() await client.branch.delete(branch_name="new-branch") ``` ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync() client.branch.delete(branch_name="new-branch") ``` ## Getting the diff for a branch[​](#getting-the-diff-for-a-branch "Direct link to Getting the diff for a branch") Use `get_diff_tree` to retrieve the full diff of a branch compared to its base branch, including summary counts and the list of changed nodes. It returns `None` if no diff exists for the branch. * Async * Sync ``` from infrahub_sdk import InfrahubClient client = InfrahubClient() diff = await client.get_diff_tree(branch="new-branch") ``` ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync() diff = client.get_diff_tree(branch="new-branch") ``` If you only need the list of changed nodes, `get_diff_summary` returns them without the diff metadata. * Async * Sync ``` from infrahub_sdk import InfrahubClient client = InfrahubClient() node_diffs = await client.get_diff_summary(branch="new-branch") ``` ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync() node_diffs = client.get_diff_summary(branch="new-branch") ``` --- # How to create and configure an Infrahub client This guide shows you how to create and configure an Infrahub client using the Python SDK. You'll learn how to set up authentication, configure proxy settings, and customize client behavior for your infrastructure automation workflows. At the end of this guide, you'll have a fully configured client ready to interact with your Infrahub instance. TL;DR; If you prefer to jump right in, check out the ["Hello World" example](#hello-world-example) at the end of this guide for a quick reference implementation. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Python 3.9 or higher * Infrahub SDK installed (`pip install infrahub-sdk` or `uv add infrahub-sdk`) * Access to an Infrahub instance (local or remote) * Valid credentials (API token or username/password) if authentication is required ## Step 1: Create a client instance[​](#step-1-create-a-client-instance "Direct link to Step 1: Create a client instance") Asynchronous vs. Synchronous Clients The SDK offers both asynchronous and synchronous client implementations. Choose the one that best fits your application architecture: * **Asynchronous client** (`InfrahubClient`): Ideal for modern async applications using Python's `async`/`await` syntax * **Synchronous client** (`InfrahubClientSync`): Better for traditional synchronous workflows or scripts - Async - Sync Create an `InfrahubClient` for async/await workflows: ``` from infrahub_sdk import InfrahubClient # Connect to local Infrahub instance client = InfrahubClient() ``` Create an `InfrahubClientSync` for traditional synchronous code: ``` from infrahub_sdk import InfrahubClientSync # Connect to local Infrahub instance client = InfrahubClientSync() ``` ## Step 2: Configure authentication and address[​](#step-2-configure-authentication-and-address "Direct link to Step 2: Configure authentication and address") Next, configure the address of your Infrahub instance and set up authentication credentials. Infrahub supports two authentication methods: API tokens and username/password. Configuration Methods You can configure the client in two ways: 1. **Environment variables**: Ideal for sensitive information 2. **`Config` object**: Better for explicit configuration in code You can also combine both methods, where the `Config` object takes precedence over environment variables. You can find the full list of configuration options in the [SDK configuration reference](/python-sdk/reference/config.md). ### Using API tokens[​](#using-api-tokens "Direct link to Using API tokens") API tokens provide secure, long-lived authentication and are the recommended method for most use cases, especially for automation scripts and non-interactive applications. * Async * Sync - Code - Environment We can set the address and API token directly in the `Config` object: ``` from infrahub_sdk import Config, InfrahubClient client = InfrahubClient(config=Config(address="http://localhost:8000", api_token="token")) ``` We need to export the address and API token as an environment variable: ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_API_TOKEN="token" ``` ``` from infrahub_sdk import InfrahubClient client = InfrahubClient() # token and address are read from the INFRAHUB_API_TOKEN and INFRAHUB_ADDRESS environment variables ``` * Code * Environment We can set the address and API token directly in the `Config` object: ``` from infrahub_sdk import Config, InfrahubClientSync client = InfrahubClientSync(config=Config(address="http://localhost:8000", api_token="token")) ``` We need to export the address and API token as an environment variable: ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_API_TOKEN="token" ``` ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync() # token and address are read from the INFRAHUB_API_TOKEN and INFRAHUB_ADDRESS environment variables ``` ### Using username/password[​](#using-usernamepassword "Direct link to Using username/password") For interactive authentication, use username and password. The SDK automatically handles JWT token generation and refresh. * Async * Sync - Code - Environment We can set the address, username and password directly in the `Config` object: ``` from infrahub_sdk import Config, InfrahubClient client = InfrahubClient(config=Config(address="http://localhost:8000", username="admin", password="infrahub")) ``` We need to export the address, username and password as environment variables: ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_USERNAME="admin" export INFRAHUB_PASSWORD="infrahub" ``` ``` from infrahub_sdk import InfrahubClient client = InfrahubClient() # username and password are read from the INFRAHUB_USERNAME and INFRAHUB_PASSWORD environment variables ``` * Code * Environment We can set the address, username and password directly in the `Config` object: ``` from infrahub_sdk import Config, InfrahubClientSync client = InfrahubClientSync(config=Config(address="http://localhost:8000", username="admin", password="infrahub")) ``` We need to export the address, username and password as environment variables: ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_USERNAME="admin" export INFRAHUB_PASSWORD="infrahub" ``` ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync() # username and password are read from the INFRAHUB_USERNAME and INFRAHUB_PASSWORD environment variables ``` success Your client is now configured and ready to use! ## Step 3: Set the default branch (optional)[​](#step-3-set-the-default-branch-optional "Direct link to Step 3: Set the default branch (optional)") Configure your client to work with a specific Infrahub branch instead of the default `main` branch. This is particularly useful when working on feature branches or experimental changes, as it eliminates the need to specify the branch name in every method call. * Code * Environment ``` from infrahub_sdk import InfrahubClient, Config config = Config(address="http://localhost:8000", api_token="token", default_branch="other-branch") client_other_branch = InfrahubClient(config=config) tag_other_branch = await client_other_branch.get(kind="BuiltinTag", name__value="RED") tag_main_branch = await client_other_branch.get(kind="BuiltinTag", name__value="RED", branch="main") ``` ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_API_TOKEN="token" export INFRAHUB_DEFAULT_BRANCH="other-branch" ``` ``` from infrahub_sdk import InfrahubClient client_other_branch = InfrahubClient() tag_other_branch = await client_other_branch.get(kind="BuiltinTag", name__value="RED") tag_main_branch = await client_other_branch.get(kind="BuiltinTag", name__value="RED", branch="main") ``` Branch Configuration Your client is now configured to use the specified default branch instead of `main`. Remember that you can always override this setting by explicitly passing the `branch` argument to any method call. ## Hello world example[​](#hello-world-example "Direct link to Hello world example") Let's create a "Hello World" example to verify your client configuration works correctly. This example will connect to your Infrahub instance and query the available accounts. 1. Create a new file called `hello_world.py`: ``` touch hello_world.py ``` File Naming Avoid naming your script `test.py` as this name could conflict with Python's testing modules. 2. Add the following code to `hello_world.py`: * Async * Sync ``` import asyncio from infrahub_sdk import Config, InfrahubClient async def hello_world(): client = InfrahubClient(config=Config(address="http://localhost:8000")) # Try to query accounts to validate connection try: accounts = await client.all(kind="CoreAccount") print(f"Successfully connected to Infrahub! Found {len(accounts)} account(s)") except Exception as e: print(f"Something went wrong: {e}") asyncio.run(hello_world()) ``` ``` from infrahub_sdk import Config, InfrahubClientSync # Test connection and authentication def hello_world(): client = InfrahubClientSync(config=Config(address="http://localhost:8000")) # Try to query accounts to validate connection try: accounts = client.all(kind="CoreAccount") print(f"Successfully connected to Infrahub! Found {len(accounts)} account(s)") except Exception as e: print(f"Something went wrong: {e}") hello_world() ``` 3. Export the necessary environment variables: ``` export INFRAHUB_API_TOKEN="06438eb2-8019-4776-878c-0941b1f1d1ec" # This is the default token ``` 4. Run the script: ``` python hello_world.py ``` If everything is configured correctly, you should see: success `Successfully connected to Infrahub! Found 1 account(s)` ## Advanced use cases[​](#advanced-use-cases "Direct link to Advanced use cases") ### Enable debug logging[​](#enable-debug-logging "Direct link to Enable debug logging") When developing or troubleshooting, you can enable GraphQL query logging to see exactly what requests are being sent to Infrahub: * Code * Environment ``` from infrahub_sdk import Config, InfrahubClient config = Config(echo_graphql_queries=True) client = InfrahubClient(config=config) ``` ``` export INFRAHUB_ECHO_GRAPHQL_QUERIES=true ``` ``` from infrahub_sdk import InfrahubClient client = InfrahubClient() # debug flag state is read from the INFRAHUB_ECHO_GRAPHQL_QUERIES environment variables ``` ### Configure proxy settings[​](#configure-proxy-settings "Direct link to Configure proxy settings") When operating in environments with restricted network access or behind corporate firewalls, you can configure the client to route requests through HTTP/HTTPS proxies. #### Single proxy for all requests[​](#single-proxy-for-all-requests "Direct link to Single proxy for all requests") To route all traffic through the same proxy server, use the `INFRAHUB_PROXY` environment variable or the `proxy` configuration parameter: * Async * Sync ``` from infrahub_sdk import Config, InfrahubClient config = Config(proxy="http://proxy.example.com:8080") client = InfrahubClient(config=config) # Or using environment variable # export INFRAHUB_PROXY=http://proxy.example.com:8080 client = InfrahubClient() ``` ``` from infrahub_sdk import Config, InfrahubClientSync config = Config(proxy="http://proxy.example.com:8080") client = InfrahubClientSync(config=config) # Or using environment variable # export INFRAHUB_PROXY=http://proxy.example.com:8080 client = InfrahubClientSync() ``` #### Separate proxies for HTTP and HTTPS[​](#separate-proxies-for-http-and-https "Direct link to Separate proxies for HTTP and HTTPS") In some network environments, you might need to route HTTP and HTTPS traffic through different proxy servers. Use the `INFRAHUB_PROXY_MOUNTS_HTTP` and `INFRAHUB_PROXY_MOUNTS_HTTPS` environment variables for this purpose: ``` export INFRAHUB_PROXY_MOUNTS_HTTP=http://http-proxy.example.com:8080 export INFRAHUB_PROXY_MOUNTS_HTTPS=http://https-proxy.example.com:8080 ``` * Async * Sync ``` from infrahub_sdk import InfrahubClient # Proxy configuration is read from environment variables client = InfrahubClient() ``` ``` from infrahub_sdk import InfrahubClientSync # Proxy configuration is read from environment variables client = InfrahubClientSync() ``` Using both `proxy` and `proxy_mounts` The `proxy` and `proxy_mounts` configurations are mutually exclusive and cannot be used together. Specifying both will cause a `ValueError` to be raised when the client is initialized. ### Retry transient failures[​](#retry-transient-failures "Direct link to Retry transient failures") Long-running scripts and generators can fail halfway through when Infrahub is temporarily unavailable: a database restart, an overloaded backend or a short network outage. With `retry_on_failure` enabled, the client retries a request when it fails with a transient error and raises immediately for every other error. A failure is transient when it is one of: * A connection error, or a connection dropped before the response arrived, such as when a load balancer restarts. * A timeout, while connecting, sending the request or reading the response. * An HTTP response with a status listed in `retry_status_codes` (`500`, `502`, `503` and `504` by default). * A GraphQL response in which every error is flagged by the server with one of those statuses. Retries use exponential backoff with jitter, starting at `retry_delay` seconds and capped at `retry_max_delay`. They stop after `max_retry_duration` seconds (5 minutes by default). Set `max_retry_duration` to `0` to keep retrying until the request succeeds, which lets a generator survive an outage instead of aborting. * Code * Environment ``` from infrahub_sdk import Config, InfrahubClient config = Config(retry_on_failure=True, max_retry_duration=0) client = InfrahubClient(config=config) ``` ``` export INFRAHUB_RETRY_ON_FAILURE=true export INFRAHUB_MAX_RETRY_DURATION=0 ``` ``` from infrahub_sdk import InfrahubClient client = InfrahubClient() # retry settings are read from the environment variables ``` Every retry is logged at `WARNING` level with the attempt number and the time spent so far. After 5 minutes of retrying the log level switches to `ERROR`, so an operation that keeps retrying stays visible. Retrying mutations A mutation that timed out may have been applied by the server before the retry. Saving with `allow_upsert=True` is safe to retry. A plain create that had already succeeded fails on retry with a non-transient error, which is raised. Unclassified server errors Infrahub reports some transient database failures as HTTP `500` without further classification, so `500` is part of the default `retry_status_codes`. The same status also covers genuine bugs, which are therefore retried until `max_retry_duration` expires. Remove `500` from `retry_status_codes` to fail fast on them instead. ## Next steps[​](#next-steps "Direct link to Next steps") Now that you have a fully configured Infrahub client, you're ready to start working with your infrastructure data. Here's what you can explore next: * **[Query objects and data](/python-sdk/guides/query_data.md)**: Learn how to retrieve objects and data from your Infrahub instance * **[Create, update and delete objects](/python-sdk/guides/create_update_delete.md)**: Manage your infrastructure data programmatically * **[Work with branches](/python-sdk/guides/branches.md)**: Use Infrahub's branch-based workflow to manage infrastructure changes * **[Batch operations](/python-sdk/guides/batch.md)**: Optimize performance with batch operations for multiple objects ## Related resources[​](#related-resources "Direct link to Related resources") * **[SDK Configuration Reference](/python-sdk/reference/config.md)**: Complete list of all configuration options * **[Installation Guide](/python-sdk/guides/installation.md)**: Detailed instructions for installing the SDK * **[Python Typing Guide](/python-sdk/guides/python-typing.md)**: Improve your development experience with type hints * **[Tracking Guide](/python-sdk/guides/tracking.md)**: Learn how to monitor and trace SDK operations --- # Create, update and deleting nodes We will be using the following schema in this guide: ``` --- version: "1.0" nodes: - name: Device namespace: Test attributes: - name: name kind: Text unique: true optional: false relationships: - name: tags cardinality: many kind: Attribute peer: BuiltinTag optional: true - name: site cardinality: one kind: Attribute peer: TestSite optional: false - name: interfaces cardinality: many kind: Component peer: TestInterface optional: false - name: Site namespace: Test attributes: - name: name kind: Text unique: true optional: false relationships: - name: device cardinality: many kind: Attribute peer: TestDevice optional: true - name: Interface namespace: Test attributes: - name: name kind: Text unique: true optional: false relationships: - name: device cardinality: one kind: Parent peer: TestDevice optional: true ``` ## Creating a node[​](#creating-a-node "Direct link to Creating a node") A node can be created using the `create` method. The `create` method will first construct a `InfrahubNode` object in memory. This `InfrahubNode` object will then need to be saved into Infrahub using the `save` method. The attributes and relationships of the `InfrahubNode` you want to create can be passed as arguments to the `create` method, or you can pass them using a dictionary. * Async * Sync ``` device = await client.create(kind="TestDevice", name="atl1-edge1") await device.save() ``` ``` device = client.create(kind="TestDevice", name="atl1-edge1") device.save() ``` * Async * Sync ``` data = {"name": "atl1-edge1"} device = await client.create(kind="TestDevice", data=data) await device.save() ``` ``` data = {"name": "atl1-edge1"} device = client.create(kind="TestDevice", data=data) device.save() ``` ### Creating a node with a relation of cardinality one[​](#creating-a-node-with-a-relation-of-cardinality-one "Direct link to Creating a node with a relation of cardinality one") * Async * Sync ``` site = await client.get(kind="TestSite", name__value="atl1") device = await client.create(kind="TestDevice", name="atl1-edge1", site=site) await device.save() ``` ``` site = client.get(kind="TestSite", name__value="atl1") device = client.create(kind="TestDevice", name="atl1-edge1", site=site) device.save() ``` ### Creating a node with a relation of cardinality many[​](#creating-a-node-with-a-relation-of-cardinality-many "Direct link to Creating a node with a relation of cardinality many") * Async * Sync ``` interfaces = await client.filters(kind="TestInterface", name__values=["Ethernet1", "Ethernet2"]) device = await client.create(kind="TestDevice", name="atl1-edge1", interfaces=interfaces) await device.save() ``` ``` interfaces = client.filters(kind="TestInterface", name__values=["Ethernet1", "Ethernet2"]) device = client.create(kind="TestDevice", name="atl1-edge1", interfaces=interfaces) device.save() ``` ## Updating a node[​](#updating-a-node "Direct link to Updating a node") To update a node from Infrahub using the SDK, we first have to retrieve the node using one of the query methods, then update the attributes and/or relations of the node and `save` the node. * Async * Sync ``` interface = await client.get(kind="TestInterface", name__value="Ethernet1") interface.name.value = "Ethernet3" await interface.save() ``` ``` interface = client.get(kind="TestInterface", name__value="Ethernet1") interface.name.value = "Ethernet3" interface.save() ``` ### Updating a relation of cardinality one on a node[​](#updating-a-relation-of-cardinality-one-on-a-node "Direct link to Updating a relation of cardinality one on a node") * Async * Sync ``` site = await client.get(kind="TestSite", name__value="atl2") device = await client.get(kind="TestDevice", name__value="atl1-edge1") device.site = site await device.save() ``` ``` site = client.get(kind="TestSite", name__value="atl2") device = client.get(kind="TestDevice", name__value="atl1-edge1") device.site = site device.save() ``` ### Updating a relation of cardinality many on a node[​](#updating-a-relation-of-cardinality-many-on-a-node "Direct link to Updating a relation of cardinality many on a node") Adding a single relation: * Async * Sync ``` interface = await client.get(kind="TestInterface", name__value="Ethernet1") device = await client.get(kind="TestDevice", name__value="atl1-edge1") device.interfaces.add(interface) await device.save() ``` ``` interface = client.get(kind="TestInterface", name__value="Ethernet1") device = client.get(kind="TestDevice", name__value="atl1-edge1") device.interfaces.add(interface) device.save() ``` Adding multiple relations: * Async * Sync ``` interfaces = await client.filters(kind="TestInterface", name__values=["Ethernet1", "Ethernet2"]) device = await client.get(kind="TestDevice", name__value="atl1-edge1") device.interfaces.extend(interfaces) await device.save() ``` ``` interfaces = client.filters(kind="TestInterface", name__values=["Ethernet1", "Ethernet2"]) device = client.get(kind="TestDevice", name__value="atl1-edge1") device.interfaces.extend(interfaces) device.save() ``` ## Deleting a node[​](#deleting-a-node "Direct link to Deleting a node") To delete a node from Infrahub using the SDK, we first have to retrieve the node using one of the query methods. The `delete` method can then be used on the retrieved `InfrahubNode` object. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1") await device.delete() ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1") device.delete() ``` ## Upserting a node[​](#upserting-a-node "Direct link to Upserting a node") The word upserting is formed out of the combination of updating and inserting. It refers to the action of creating a node if it does not yet exist, or to update the existing node if it already exists. We first have to create the node in memory, then use the `save` method on the node with the `allow_upsert` argument set to `True`. * Async * Sync ``` site = await client.get(kind="TestSite", name="atl1") device = await client.create(kind="TestDevice", name="atl1-edge1") device.site = site await device.save(allow_upsert=True) ``` ``` site = client.get(kind="TestSite", name="atl1") device = client.create(kind="TestDevice", name="atl1-edge1") device.site = site device.save(allow_upsert=True) ``` --- # Traversing the graph Graph traversal lets you discover how nodes are connected without knowing the relationship path in advance. The SDK exposes two client methods: * `traverse_paths` — find the shortest path(s) between a **source** and a **destination** node. * `reachable_nodes` — find every node of given **kinds** that is reachable from a **source** node, with the path used to reach each one. note Graph traversal requires **Infrahub 1.10 or later**. Calling either method against an older server raises `VersionNotSupportedError`. ## Finding the path between two nodes[​](#finding-the-path-between-two-nodes "Direct link to Finding the path between two nodes") Pass the source and destination as node UUID strings **or** `InfrahubNode` instances (the SDK reads the node's id for you). Paths are returned shortest-first. Each path is a list of `hops`, and each hop exposes the `node` visited and the `relationship` traversed to reach it (the first hop's `relationship` is `None`). ``` # both of these work — and you can mix them device = await client.get(kind="InterfacePhysical", id=src_id) await client.traverse_paths(device, dst_id) # node + id await client.traverse_paths(src_id, dst_id) # id + id ``` * Async * Sync ``` result = await client.traverse_paths( source="1891a122-8875-bae7-3866-10658751d7cc", destination="1891a12b-27e5-fe3e-386c-1065983045b0", max_depth=8, ) print(f"{result.count} path(s) found") for path in result.paths: for hop in path.hops: arrow = f" --[{hop.relationship.from_rel}]--> " if hop.relationship else "" print(f"{arrow}{hop.node.display_label}", end="") print() ``` ``` result = client.traverse_paths( source="1891a122-8875-bae7-3866-10658751d7cc", destination="1891a12b-27e5-fe3e-386c-1065983045b0", max_depth=8, ) print(f"{result.count} path(s) found") ``` ### Constraining the traversal[​](#constraining-the-traversal "Direct link to Constraining the traversal") All limits and filters are optional; when omitted, the server applies its own defaults. By default the server excludes IP namespaces — they act as hubs and are rarely useful for path tracing — along with internal namespaces such as `Builtin`, `Internal`, and `Core`. The excluded kinds are reported back in `result.excluded_kinds`, and any of them can be brought back into the traversal with `included_kinds`. The kind filters (`kind_filter`, `excluded_kinds`, `included_kinds`, and `target_kinds` on `reachable_nodes`) accept either kind-name strings or generated protocol classes — mix them freely: ``` from infrahub_sdk.protocols import DcimCable, InterfacePhysical result = await client.traverse_paths( source=src, destination=dst, kind_filter=[DcimCable, InterfacePhysical, "DcimFrontPatchPanelInterface"], ) ``` note `relationship_filter` matches the schema **relationship identifier** (the canonical name shared by both sides of a relationship, for example `dcimconnector__dcimendpoint`) — **not** the per-side `from_rel` / `to_rel` names shown in the result. Find identifiers via the schema: `(await client.schema.get(kind="InterfacePhysical")).relationships[i].identifier`. `shortest_paths_only` controls how many paths are considered through each intermediate node. When `True` (the server default), only the shortest path through each intermediate object is returned; when `False`, every loopless path up to `max_paths` is returned (exhaustive mode). * Async * Sync ``` result = await client.traverse_paths( source=src, destination=dst, max_depth=10, max_paths=5, kind_filter=["DcimCable", "InterfacePhysical"], # only traverse through these kinds relationship_filter=["dcimconnector__dcimendpoint"], # schema relationship identifier included_kinds=["IpamIPPrefix"], # re-include a default-excluded kind shortest_paths_only=False, # return all loopless paths, not just the shortest ) ``` ``` result = client.traverse_paths( source=src, destination=dst, max_depth=10, max_paths=5, kind_filter=["DcimCable", "InterfacePhysical"], relationship_filter=["dcimconnector__dcimendpoint"], included_kinds=["IpamIPPrefix"], shortest_paths_only=False, ) ``` ## Checking that a path exists[​](#checking-that-a-path-exists "Direct link to Checking that a path exists") A common use in an Infrahub check is "is A still connected to B?". `path_exists` answers that in one call — it requests a single path (the cheapest way to know) and returns a boolean. It takes the same source/destination and filter arguments as `traverse_paths`. * Async * Sync ``` connected = await client.path_exists( device_a, device_b, max_depth=8, kind_filter=["DcimCable", "InterfacePhysical"], ) if not connected: self.log_error(message=f"No path between {device_a.display_label} and {device_b.display_label}") ``` ``` connected = client.path_exists(device_a, device_b, max_depth=8) if not connected: self.log_error(message="Expected path is missing") ``` ## Discovering reachable nodes[​](#discovering-reachable-nodes "Direct link to Discovering reachable nodes") Use `reachable_nodes` for impact or dependency analysis: "what nodes of these kinds can I reach from here?" Each entry includes the reachable `node`, the `depth` at which it was found, and the full `path` from the source. * Async * Sync ``` result = await client.reachable_nodes( source=device_id, target_kinds=["DcimCable", "InfraCircuit"], max_depth=5, max_results=100, shortest_paths_only=True, ) for dep in result.dependencies: print(f"{dep.node.kind:20} {dep.node.display_label} (depth {dep.depth})") ``` ``` result = client.reachable_nodes( source=device_id, target_kinds=["DcimCable", "InfraCircuit"], max_depth=5, max_results=100, shortest_paths_only=True, ) ``` ## Working with the results[​](#working-with-the-results "Direct link to Working with the results") Results are typed objects, not raw dictionaries. A returned `PathNode` is a lightweight identity (`id`, `kind`, `label`, `display_label`, `hfid`) — it does not carry the node's attributes or relationships. To get the full node, call `.fetch()` on any `PathNode`. It resolves the node through the same client (and branch) that produced the traversal and adds it to the client store, so fetching the same id again is served from the store. * Async * Sync ``` for hop in result.paths[0].hops: print(hop.node.display_label) # identity, no request full = await hop.node.fetch() # resolve the full node print(full.name.value) ``` ``` for hop in result.paths[0].hops: print(hop.node.display_label) # identity, no request full = hop.node.fetch() # resolve the full node print(full.name.value) ``` note `.fetch()` issues one request per node. For large results where you need many full nodes, prefer fetching only the ones you actually need, or batch them with `client.get` inside an `InfrahubBatch`. ### Detecting truncated results[​](#detecting-truncated-results "Direct link to Detecting truncated results") The server caps the number of results. Note that `result.count` is the number of *paths* returned, not the number of distinct nodes — when several paths lead to the same node, one node is reported multiple times. To check whether `max_results` (the cap on distinct target nodes) was reached, count the distinct nodes by their UUID: ``` result = await client.reachable_nodes(source=src, target_kinds=["InfraDevice"], max_results=50) reached = {dep.node.id for dep in result.dependencies} if len(reached) >= 50: print("Result may be truncated — increase max_results to see more.") ``` For `traverse_paths`, the `PathTraversalResult` reports truncation directly via `result.truncated_at_depth`. It is `None` when the search completed within budget; otherwise it is the depth at which the server ran out of budget. In that case the returned paths are complete only for depths *less than* that value, and deeper paths may exist. ``` result = await client.traverse_paths(source=src, destination=dst) if result.truncated_at_depth is not None: print(f"Search ran out of budget at depth {result.truncated_at_depth}; deeper paths may exist.") ``` ## Common check patterns[​](#common-check-patterns "Direct link to Common check patterns") These are the recurring questions graph traversal answers in an Infrahub check. The examples are async; the sync client mirrors them without `await`. **Connectivity required** — A must reach B: ``` if not await client.path_exists(a, b): self.log_error(message="A is no longer connected to B") ``` **Isolation / segmentation** — A must *not* reach B (the same primitive, negated): ``` if await client.path_exists(a, b): self.log_error(message="A and B must remain isolated") ``` **Path must avoid a kind** — no path may pass through a given kind: ``` if not await client.path_exists(a, b, excluded_kinds=["LabReservedThing"]): self.log_error(message="No compliant path that avoids lab-reserved nodes") ``` **Path must traverse a kind** — every A→B path must pass through, for example, a firewall: ``` result = await client.traverse_paths(a, b) for path in result.paths: if not any(hop.node.kind == "SecurityFirewall" for hop in path.hops): self.log_error(message="Found a path that bypasses the firewall") ``` **Within N hops** — A and B must be no more than 3 hops apart: ``` result = await client.traverse_paths(a, b, max_depth=3) if not result.paths: self.log_error(message="A and B are more than 3 hops apart (or not connected)") ``` **Reach all required targets** — several required services must each be reachable from a device: ``` for service_id in dns_and_ntp_ids: if not await client.path_exists(device, service_id): self.log_error(message=f"{device.display_label} cannot reach required service {service_id}") ``` The patterns above use `path_exists` because they ask about a *specific* destination. When the check is about a *kind* of node rather than a specific one, use `reachable_nodes` instead. **Reach at least one node of a kind** — a device must be able to reach some DNS server: ``` result = await client.reachable_nodes(device, target_kinds=["NetworkDnsServer"], max_results=1) if result.count == 0: self.log_error(message=f"{device.display_label} cannot reach any DNS server") ``` **Redundancy by kind** — at least two NTP servers must be reachable: ``` result = await client.reachable_nodes(device, target_kinds=["NetworkNtpServer"]) if result.count < 2: self.log_error(message=f"{device.display_label} reaches only {result.count} NTP server(s); expected at least 2") ``` **Forbidden reach (blast radius / segmentation)** — a device must not reach any sensitive asset: ``` result = await client.reachable_nodes(device, target_kinds=["SecuritySensitiveAsset"], max_results=1) if result.count: self.log_error(message=f"{device.display_label} can reach a sensitive asset it should be isolated from") ``` **Report dependencies** — enumerate what a node depends on, with how far away each is: ``` result = await client.reachable_nodes(device, target_kinds=["InfraCircuit", "DcimCable"]) for dep in result.dependencies: self.log_info(message=f"depends on {dep.node.display_label} ({dep.node.kind}) at depth {dep.depth}") ``` ## Handling older servers[​](#handling-older-servers "Direct link to Handling older servers") ``` from infrahub_sdk.exceptions import VersionNotSupportedError try: result = await client.traverse_paths(source=src, destination=dst) except VersionNotSupportedError as exc: print(exc) # Graph path traversal requires Infrahub 1.10 or later. ``` note An empty result is **not** an error: when no path or no reachable node exists within the limits, `count` is `0` and the list is empty. A request for a node that does not exist raises a `GraphQLError`. --- # Installing infrahub-sdk The Infrahub SDK for Python is available on [PyPI](https://pypi.org/project/infrahub-sdk/) and can be installed using the pip package installer. It is recommended to install the SDK into a virtual environment. ``` python3 -m venv .venv source .venv/bin/activate pip install infrahub-sdk ``` ## Installing optional extras[​](#installing-optional-extras "Direct link to Installing optional extras") Extras can be installed as part of the Python SDK and are not installed by default. ### ctl[​](#ctl "Direct link to ctl") The `ctl` extra provides the `infrahubctl` command, which allows you to interact with an Infrahub instance. ``` pip install 'infrahub-sdk[ctl]' ``` ### tests[​](#tests "Direct link to tests") The `tests` extra provides all the components for the testing framework of Transforms, Queries and Checks. ``` pip install 'infrahub-sdk[tests]' ``` ### all[​](#all "Direct link to all") Installs `infrahub-sdk` together with all the extras. ``` pip install 'infrahub-sdk[all]' ``` --- # Using the object-storage The Python SDK can be used to interface with Infrahub's [object-storage](/artifact-file-storage/overview.md). ## Storing string objects to the object-storage[​](#storing-string-objects-to-the-object-storage "Direct link to Storing string objects to the object-storage") We can use the SDK to store string objects into the object-storage. ``` import json from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync() data = { "key": "value", "list": [ "item1", "item2", "item3", ] } response = client.object_store.upload(content=json.dumps(data)) print(response) ``` ## Retrieving objects from the object-storage[​](#retrieving-objects-from-the-object-storage "Direct link to Retrieving objects from the object-storage") Retrieving objects from the object-storage can be done using the object's identifier. In this step we will be using the identifier that we got as a response to the previous step. ``` import json from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync() identifier = "17d19a20-70f6-3e44-3341-c5124065cda9" response = client.object_store.get(identifier="17d19a20-70f6-3e44-3341-c5124065cda9") print(json.loads(response)) ``` ## Storing a file to the object-storage[​](#storing-a-file-to-the-object-storage "Direct link to Storing a file to the object-storage") You can store text files in the object-storage. We have to read the contents of the file into a string object, which we can then store in the object-storage. ``` import json from pathlib import Path from infrahub_sdk import InfrahubClientSync data = { "key": "value", "list": [ "item1", "item2", "item3", ] } json_file = Path("/tmp/file.json") json_file.write_text(json.dumps(data)) client = InfrahubClientSync() identifier = "17d19a20-70f6-3e44-3341-c5124065cda9" response = client.object_store.upload(content=json_file.read_text()) print(response) ``` --- # Overview Use Python's type system with the Infrahub SDK to catch schema mismatches while you write code, not when it runs. This guide shows how to type your SDK calls with Protocols, generate typed classes from your own schema, and generate Pydantic models from your GraphQL queries. What is Python Typing Python typing allows you to specify the expected data types of variables, function arguments, and return values to improve code clarity and catch bugs early. ``` # Basic type hints def percentage(num1: int, num2: int) -> float: return (num1 / num2) * 100 ``` ## Leveraging Python protocols[​](#leveraging-python-protocols "Direct link to Leveraging Python protocols") The Python SDK for Infrahub has been designed to automatically work with any schemas loaded into Infrahub. Internally, the Python SDK generates dynamic Python representations of your schemas. While this approach improves code readability, it presents challenges with type checking because each object has a different signature based on your schema. ### Without protocols[​](#without-protocols "Direct link to Without protocols") In the example below, type checkers like Mypy will typically complain about `blue_tag.description.value` because `description` is a dynamic parameter generated by the SDK. ``` # Type checker cannot verify the existence of 'description' blue_tag = client.get("BuiltinTag", name__value="blue") # blue_tag is of type InfrahubNode or InfrahubNodeSync blue_tag.description.value = "The blue tag" # Mypy: error: "InfrahubNode" has no attribute "description" blue_tag.save() ``` ### With protocols[​](#with-protocols "Direct link to With protocols") To provide strict type checking while maintaining platform extensibility, the Python SDK integrates with Python Protocols. For all core and internal models, the protocols are included in the SDK under `infrahub_sdk.protocols`. Whenever you need to specify the kind of object you're working with as a string, you can use the corresponding protocol instead. ``` from infrahub_sdk.protocols import BuiltinTag # Type checker can now verify all attributes blue_tag = client.get(BuiltinTag, name__value="blue") # blue_tag is of type BuiltinTag blue_tag.description.value = "The blue tag" # No type errors blue_tag.save() ``` Python Protocols Python Protocols, introduced in PEP 544, define a set of method and property signatures that a class must implement to be considered a match, enabling structural subtyping (also known as "duck typing" with static checks). They allow you to specify behavior without requiring inheritance, making code more flexible and type-safe. More information about Python Protocols can be found [here](https://typing.python.org/en/latest/spec/protocol.html) ## Generating custom protocols based on your schema[​](#generating-custom-protocols-based-on-your-schema "Direct link to Generating custom protocols based on your schema") You can generate Python Protocols for your own models using the `infrahubctl protocols` command. This supports both synchronous and asynchronous Python code. It's possible to provide the schema from a local directory or from an existing Infrahub Instance. * Existing Infrahub Instance * Local Directory ``` export INFRAHUB_ADDRESS=https://infrahub.example.com infrahubctl protocols --out lib/protocols.py --sync ``` ``` infrahubctl protocols --schemas schemas/tag.schema.yml --out lib/protocols.py ``` > When using a local directory, Protocols for Profiles and Object Templates won't be generated. ## Using custom protocols[​](#using-custom-protocols "Direct link to Using custom protocols") After generation, you can import and use your custom protocols as describe below. ``` from lib.protocols import MyOwnObject # Use your custom protocol my_object = client.get(MyOwnObject, name__value="example") ``` > if you don't have your own Python module, it's possible to use relative path by having the `protocols.py` in the same directory as your script/transform/generator ### Using protocols in generators and transforms[​](#using-protocols-in-generators-and-transforms "Direct link to Using protocols in generators and transforms") Protocols make the most difference in generators, transforms, and checks, where you traverse relationships and set attributes across many object kinds. Import the generated class, pass it as the `kind` argument, and use it as a type hint: ``` from .protocols import NetworkDevice # Passing the generated class as `kind` makes `filters()` return `list[NetworkDevice]`, # so `switch` is fully typed without annotating the variable yourself leaf_switches = await client.filters(kind=NetworkDevice, role__value="leaf", include=["interfaces"]) for switch in leaf_switches: switch.name.value # autocompleted, and checked against the schema ``` ### Keep protocols in sync with your schema[​](#keep-protocols-in-sync-with-your-schema "Direct link to Keep protocols in sync with your schema") The generated file is a build artifact: regenerate it whenever your schema changes, and commit the result. When the schema changes and you regenerate, your type checker reports every line that no longer matches the schema, so you find the mismatch while you write code instead of at runtime. Regenerate with the same command you used to create the file: ``` infrahubctl protocols --out lib/protocols.py ``` ## Generating Pydantic models from GraphQL queries[​](#generating-pydantic-models-from-graphql-queries "Direct link to Generating Pydantic models from GraphQL queries") When working with GraphQL queries, you can generate type-safe Pydantic models that correspond to your query return types. This provides excellent type safety and IDE support for your GraphQL operations. ### Why use generated return types?[​](#why-use-generated-return-types "Direct link to Why use generated return types?") Generated Pydantic models from GraphQL queries offer several important benefits: * **Type Safety**: Catch type errors during development time instead of at runtime * **IDE Support**: Get autocomplete, type hints, and better IntelliSense in your IDE * **Documentation**: Generated models serve as living documentation of your GraphQL API * **Validation**: Automatic validation of query responses against the expected schema ### Generating return types[​](#generating-return-types "Direct link to Generating return types") Use the `infrahubctl graphql generate-return-types` command to create Pydantic models from your GraphQL queries: ``` # Generate models for queries in a directory infrahubctl graphql generate-return-types queries/ # Generate models for specific query files infrahubctl graphql generate-return-types queries/get_tags.gql ``` > You can also export the GraphQL schema first using the `infrahubctl graphql export-schema` command. ### Example workflow[​](#example-workflow "Direct link to Example workflow") 1. **Create your GraphQL queries** in `.gql` files preferably in a directory (for example, `queries/`): ``` # queries/get_tags.gql query GetAllTags { BuiltinTag { edges { node { __typename name { value } } } } } ``` 2. **Export the GraphQL schema**: ``` infrahubctl graphql export-schema ``` 3. **Generate the Pydantic models**: ``` infrahubctl graphql generate-return-types queries/ ``` Query names Ensure each of your GraphQL queries has a unique name, as the generated Python files will be named based on these query names. Two queries with the same name will land in the same file, leading to potential overrides. 4. **Use the generated models** in your Python code ``` from .queries.get_tags import GetAllTagsQuery response = await client.execute_graphql(query=MY_QUERY) data = GetAllTagsQuery(**response) ``` --- # Querying data in Infrahub We can query data in 3 ways using the SDK: * querying a single node of a given kind, based on some filters, using the `get` method * querying multiple nodes of a given kind, based on some filters, using the `filters` method * querying all the nodes of a given kind, using the `all` method * querying with a GraphQL query, using the `execute_graphql` method ## Query filters[​](#query-filters "Direct link to Query filters") The `get` and `filters` query methods allow you to use filters. Filters specify on which attribute(s) or attributes of a relationship(s) the resulting node(s) should match. The available filters depend on the kind of Node you want to query and are dynamically generated from the schema. ### Discovering available filters[​](#discovering-available-filters "Direct link to Discovering available filters") The easiest way to discover the available filters for a kind of node is by opening the GraphQLi Explorer in the GraphQL sandbox. Under each GraphQL query you will find the available filters. ### Attribute filters[​](#attribute-filters "Direct link to Attribute filters") For every attribute the following filters will be generated (replace attribute with the name of the attribute): * `attribute__value`: filter for a single attribute value * `attribute__values`: (list) filter for multiple attribute values * `attribute__is_visible`: (boolean) filter for the `is_visible` property of an attribute * `attribute__is_protected`: (boolean) filter for the `is_protected` property of an attribute * `attribute__source__id`: filter for the `source` property of an attribute * `attribute__owner__id`: filter for the `owner` property of an attribute ### Relationship filters[​](#relationship-filters "Direct link to Relationship filters") For every attribute of a relationship of the node, the same filters will be generated. The name of the filter will be prefixed with the relationship name followed by 2 underscores `relationship__attribute__value`. ### Using filters[​](#using-filters "Direct link to Using filters") Filters can be passed as arguments to the `get` or `filters` method. * Async * Sync ``` tag = await client.get(kind="BuiltinTag", name__value="RED") ``` ``` tag = client.get(kind="BuiltinTag", name__value="RED") ``` ### Using multiple filters[​](#using-multiple-filters "Direct link to Using multiple filters") When you pass multiple filters as argument to the `get` or `filters` method, they will combined in a logical AND operation. The resulting nodes of your query will match on all the filters in that case. * Async * Sync ``` tags = await client.filters(kind="BuiltinTag", name__values=["RED", "BLUE"], name__is_protected=True) ``` ``` tags = client.filters(kind="BuiltinTag", name__values=["RED", "BLUE"], name__is_protected=True) ``` ## Querying a single node[​](#querying-a-single-node "Direct link to Querying a single node") You can query Infrahub for a single node of a particular kind, by using the `get` method and using 1 or more filters. * Async * Sync ``` tag = await client.get(kind="BuiltinTag", name__value="RED") ``` ``` tag = client.get(kind="BuiltinTag", name__value="RED") ``` ## Querying multiple nodes[​](#querying-multiple-nodes "Direct link to Querying multiple nodes") You can query Infrahub for multiple nodes of a particular kind, by using the `filters` method and using 1 or more filters. * Async * Sync ``` tags = client.filters(kind="BuiltinTag", name__values=["RED", "BLUE"]) ``` ``` tags = client.filters(kind="BuiltinTag", name__values=["RED", "BLUE"]) ``` ## Querying all nodes[​](#querying-all-nodes "Direct link to Querying all nodes") You can query Infrahub for all nodes of a particular kind, by using the `all` method. * Async * Sync ``` tags = await client.all(kind="BuiltinTag") ``` ``` tags = client.all(kind="BuiltinTag") ``` ## Querying with a GraphQL query[​](#querying-with-a-graphql-query "Direct link to Querying with a GraphQL query") In some scenarios it might be more convenient to query Infrahub using a GraphQL query, rather than using the builtin mechanisms in the SDK. An example might be finding all the devices connected to a given circuit. This would require us to execute multiple queries using the nodes provided by the SDK. However with a GraphQL query this can be achieved using only a single query. The downside of using this method, is that it will not construct Python objects for the resulting data. Instead the SDK will return a Python dictionary containing the deserialized JSON data returned by the GraphQL API. * Async * Sync ``` query = """query { BuiltinTag(name__values: ["RED", "BLUE"]) { edges { node { name { value } } } } }""" data = await client.execute_graphql(query=query) for tag in data["BuiltinTag"]["edges"]: print(tag["node"]["name"]["value"]) ``` ``` query = """query { BuiltinTag(name__values: ["RED", "BLUE"]) { edges { node { name { value } } } } }""" data = client.execute_graphql(query=query) for tag in data["BuiltinTag"]["edges"]: print(tag["node"]["name"]["value"]) ``` ## Attributes and relationships[​](#attributes-and-relationships "Direct link to Attributes and relationships") By default, the result of a query will include attributes, relationships of cardinality one and relationships of kind Attribute or Parent. Relationships that are included in a query will be automatically initialized which means the ID, type and display name of the peers will be included in the query. But the related node itself will not be included. To explore this in a bit more details, we are going to assume the following schema has been loaded into Infrahub ``` --- version: "1.0" nodes: - name: Device namespace: Test attributes: - name: name kind: Text unique: true optional: false relationships: - name: tags cardinality: many kind: Attribute peer: BuiltinTag optional: true - name: site cardinality: one kind: Attribute peer: TestSite optional: false - name: interfaces cardinality: many kind: Component peer: TestInterface optional: false - name: Site namespace: Test attributes: - name: name kind: Text unique: true optional: false relationships: - name: device cardinality: many kind: Attribute peer: TestDevice optional: true - name: Interface namespace: Test attributes: - name: name kind: Text unique: true optional: false relationships: - name: device cardinality: one kind: Parent peer: TestDevice optional: true ``` ### Attributes[​](#attributes "Direct link to Attributes") Attributes are included by default. * Async * Sync ``` device = await client.get(kind="BuiltinTag", name__value="atl1-edge1") print(device.name.value) ``` ``` device = client.get(kind="BuiltinTag", name__value="atl1-edge1") print(device.name.value) ``` ### Relationships of cardinality one[​](#relationships-of-cardinality-one "Direct link to Relationships of cardinality one") Relationships of cardinality one are included by default and will be initialized. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1") print(device.site.initialized) # True print(device.site.id, device.site.display_label, device.site.typename) print(device.site.peer) # the related node is not included ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1") print(device.site.initialized) # True print(device.site.id, device.site.display_label, device.site.typename) print(device.site.peer) # the related node is not included ``` Relationships of kind `Attribute`, `Parent` are included by default and will be initialized. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1") print(device.tags.initialized) # True print(device.tags.peers) for tag in device.tags.peers: print(tag.id, tag.display_label, tag.typename) ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1") print(device.tags.initialized) # True print(device.tags.peers) for tag in device.tags.peers: print(tag.id, tag.display_label, tag.typename) ``` ### Relationships of cardinality many are not included by default[​](#relationships-of-cardinality-many-are-not-included-by-default "Direct link to Relationships of cardinality many are not included by default") Relationships of cardinality many are not included by default. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1") print(device.interfaces.initialized) # False print(device.interfaces.peers) # empty list [] ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1") print(device.interfaces.initialized) # False print(device.interfaces.peers) # empty list [] ``` ### Including attributes and relationships[​](#including-attributes-and-relationships "Direct link to Including attributes and relationships") You can include attributes and relationships that are not retrieved as part of a query by default. The included relationships will be initialized and the related nodes (peers) will be initialized. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1", include=["interfaces"]) print(device.interfaces.initialized) # True print(device.interfaces.peers) for interface in device.interfaces.peers: print(interface.id, interface.display_label, interface.typename) ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1", include=["interfaces"]) print(device.interfaces.initialized) # True print(device.interfaces.peers) for interface in device.interfaces.peers: print(interface.id, interface.display_label, interface.typename) ``` ### Excluding attribute and relationships[​](#excluding-attribute-and-relationships "Direct link to Excluding attribute and relationships") You can exclude attributes and relationships that are retrieved with a query by default. This can be useful if you need to optimize or speed up a particular query. * Async * Sync ``` device = await client.get(kind="TestDevice", exclude=["site"]) print(device.site) # None ``` ``` device = client.get(kind="TestDevice", exclude=["site"]) print(device.site) # None ``` ### Fetching relationships manually[​](#fetching-relationships-manually "Direct link to Fetching relationships manually") The `fetch` method can be used to retrieve relationships, initialize them; and retrieving the related nodes manually. The `fetch` method can also be used on relationship that were previously initialized. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1") print(device.interfaces.initialized) # False await device.interfaces.fetch() print(device.interfaces.initialized) # True for interface in device.interfaces.peers: print(interface.id, interface.display_label, interface.typename, interface.peer.name.value) ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1") print(device.interfaces.initialized) # False device.interfaces.fetch() print(device.interfaces.initialized) # True for interface in device.interfaces.peers: print(interface.id, interface.display_label, interface.typename, interface.peer.name.value) ``` * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1", include=["interfaces"]) print(device.interfaces.initialized) # True await device.interfaces.fetch() for interface in device.interfaces.peers: print(interface.id, interface.display_label, interface.typename, interface.peer.name.value) ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1", include=["interfaces"]) print(device.interfaces.initialized) # True device.interfaces.fetch() for interface in device.interfaces.peers: print(interface.id, interface.display_label, interface.typename, interface.peer.name.value) ``` ### Prefetch relationships[​](#prefetch-relationships "Direct link to Prefetch relationships") Related nodes of a relationship can be retrieved, using the `prefetch_relationships` argument for the different `query` methods. But this requires the usage of the internal client store . More information can be found in the [Using the client store](/python-sdk/guides/store.md) guide. ## Query a node(s) in the past[​](#query-a-nodes-in-the-past "Direct link to Query a node(s) in the past") To query the state of a Node in the past, you can pass the `at` argument to all the query methods. The at argument accepts a `str`, `DateTime` or `Timestamp` object as value. Values of type `str` will be parsed using the [Pendulum](https://pendulum.eustace.io/) library. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1", at="10:10:10") ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1", at="10:10:10") ``` ### Properties of attributes and relationships[​](#properties-of-attributes-and-relationships "Direct link to Properties of attributes and relationships") By default, the [meta data or properties](/objects/metadata.md) of attributes and relationships are not included. We can include these properties using the `property` argument of the SDK client's `all`, `filters` or `get` method. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1", property=True) print(device.name.is_protected) print(device.name.source.display_label) ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1", property=True) print(device.name.protected) print(device.name.source.display_label) ``` ## Node metadata[​](#node-metadata "Direct link to Node metadata") Node metadata provides information about when a node was created or last updated, and by whom. This includes timestamps and references to the accounts that made the changes. ### Including node metadata in queries[​](#including-node-metadata-in-queries "Direct link to Including node metadata in queries") By default, node metadata is not included in query results. You can include it using the `include_metadata` argument of the SDK client's `all`, `filters`, or `get` method. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1", include_metadata=True) ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1", include_metadata=True) ``` ### Accessing node metadata[​](#accessing-node-metadata "Direct link to Accessing node metadata") Once metadata is included in the query, you can access it using the `get_node_metadata()` method. The metadata object contains the following fields: * `created_at`: Timestamp when the node was created * `created_by`: Reference to the account that created the node * `updated_at`: Timestamp when the node was last updated * `updated_by`: Reference to the account that last updated the node - Async - Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1", include_metadata=True) # Get the metadata object metadata = device.get_node_metadata() # Access creation metadata print(metadata.created_at) # e.g., "2024-01-15T10:30:00Z" print(metadata.created_by.display_label) # e.g., "admin" # Access update metadata print(metadata.updated_at) # e.g., "2024-01-20T14:45:00Z" print(metadata.updated_by.display_label) # e.g., "admin" ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1", include_metadata=True) # Get the metadata object metadata = device.get_node_metadata() # Access creation metadata print(metadata.created_at) # e.g., "2024-01-15T10:30:00Z" print(metadata.created_by.display_label) # e.g., "admin" # Access update metadata print(metadata.updated_at) # e.g., "2024-01-20T14:45:00Z" print(metadata.updated_by.display_label) # e.g., "admin" ``` The `created_by` and `updated_by` fields are `NodeProperty` objects that include: * `id`: The unique identifier of the account * `display_label`: A human-readable label for the account * `typename`: The GraphQL type name of the account ## Relationship metadata[​](#relationship-metadata "Direct link to Relationship metadata") When querying with `include_metadata=True`, you can also access metadata about relationship edges themselves. This tells you when a specific relationship (the connection between two nodes) was last modified and by whom. ### Accessing relationship metadata[​](#accessing-relationship-metadata "Direct link to Accessing relationship metadata") Use the `get_relationship_metadata()` method on a related node to access the relationship edge metadata. This is different from node metadata - it describes when the relationship itself was created or modified, not the connected node. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1", include_metadata=True) # For a cardinality-one relationship rel_metadata = device.site.get_relationship_metadata() if rel_metadata: print(rel_metadata.updated_at) # e.g., "2024-01-17T08:00:00Z" print(rel_metadata.updated_by.display_label) # e.g., "admin" # For a cardinality-many relationship for tag in device.tags.peers: rel_metadata = tag.get_relationship_metadata() if rel_metadata: print(f"Tag relationship updated at: {rel_metadata.updated_at}") print(f"Updated by: {rel_metadata.updated_by.display_label}") ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1", include_metadata=True) # For a cardinality-one relationship rel_metadata = device.site.get_relationship_metadata() if rel_metadata: print(rel_metadata.updated_at) # e.g., "2024-01-17T08:00:00Z" print(rel_metadata.updated_by.display_label) # e.g., "admin" # For a cardinality-many relationship for tag in device.tags.peers: rel_metadata = tag.get_relationship_metadata() if rel_metadata: print(f"Tag relationship updated at: {rel_metadata.updated_at}") print(f"Updated by: {rel_metadata.updated_by.display_label}") ``` The `RelationshipMetadata` object contains: * `updated_at`: Timestamp when the relationship was last updated * `updated_by`: Reference to the account that last updated the relationship (a `NodeProperty` object with `id`, `display_label`, and `typename`) note Relationship metadata tracks changes to the relationship edge itself (for example, when the relationship was created or when its properties were modified), not changes to the connected nodes. For node-level metadata, use `get_node_metadata()` on the node itself. ## Ordering query results[​](#ordering-query-results "Direct link to Ordering query results") You can control the order in which query results are returned using the `order` argument. Results can be ordered by attributes, related-node attributes, or object metadata such as creation and update timestamps. ### Ordering by fields[​](#ordering-by-fields "Direct link to Ordering by fields") Use the `Order` and `OrderByEntry` classes along with `OrderDirection` to order results by one or more fields. Each `OrderByEntry` takes a `field` and an optional `direction` (defaulting to `OrderDirection.ASC`). The entries are applied in order, so the first entry is the primary sort key. A `field` can reference: * an attribute, e.g. `name__value` * a related-node attribute, e.g. `owner__name__value` * node metadata, e.g. `node_metadata__created_at` or `node_metadata__updated_at` - Async - Sync ``` from infrahub_sdk.enums import OrderDirection from infrahub_sdk.types import Order, OrderByEntry # Get devices ordered by name (ascending is the default) devices = await client.all( kind="TestDevice", order=Order(by=[OrderByEntry(field="name__value")]) ) # Get devices ordered by creation time (oldest first) devices = await client.all( kind="TestDevice", order=Order(by=[OrderByEntry(field="node_metadata__created_at", direction=OrderDirection.ASC)]) ) # Get devices ordered by last update time (most recent first) devices = await client.all( kind="TestDevice", order=Order(by=[OrderByEntry(field="node_metadata__updated_at", direction=OrderDirection.DESC)]) ) # Order by name ascending, then by creation time descending devices = await client.all( kind="TestDevice", order=Order(by=[ OrderByEntry(field="name__value", direction=OrderDirection.ASC), OrderByEntry(field="node_metadata__created_at", direction=OrderDirection.DESC), ]) ) ``` ``` from infrahub_sdk.enums import OrderDirection from infrahub_sdk.types import Order, OrderByEntry # Get devices ordered by name (ascending is the default) devices = client.all( kind="TestDevice", order=Order(by=[OrderByEntry(field="name__value")]) ) # Get devices ordered by creation time (oldest first) devices = client.all( kind="TestDevice", order=Order(by=[OrderByEntry(field="node_metadata__created_at", direction=OrderDirection.ASC)]) ) # Get devices ordered by last update time (most recent first) devices = client.all( kind="TestDevice", order=Order(by=[OrderByEntry(field="node_metadata__updated_at", direction=OrderDirection.DESC)]) ) # Order by name ascending, then by creation time descending devices = client.all( kind="TestDevice", order=Order(by=[ OrderByEntry(field="name__value", direction=OrderDirection.ASC), OrderByEntry(field="node_metadata__created_at", direction=OrderDirection.DESC), ]) ) ``` The available order directions are: * `OrderDirection.ASC`: Ascending order (oldest/smallest first) * `OrderDirection.DESC`: Descending order (newest/largest first) note The `node_metadata` argument on `Order` is deprecated in favor of the `by` form shown above. It cannot be combined with `by` in the same `Order`. ### Disabling default ordering[​](#disabling-default-ordering "Direct link to Disabling default ordering") For performance optimization, you can disable the default ordering behavior entirely: * Async * Sync ``` from infrahub_sdk.types import Order # Disable ordering to improve query performance devices = await client.all(kind="TestDevice", order=Order(disable=True)) ``` ``` from infrahub_sdk.types import Order # Disable ordering to improve query performance devices = client.all(kind="TestDevice", order=Order(disable=True)) ``` ### Combining metadata and ordering[​](#combining-metadata-and-ordering "Direct link to Combining metadata and ordering") You can include metadata and order results in the same query: * Async * Sync ``` from infrahub_sdk.enums import OrderDirection from infrahub_sdk.types import NodeMetaOrder, Order # Get the 10 most recently updated devices with their metadata devices = await client.filters( kind="TestDevice", limit=10, include_metadata=True, order=Order(node_metadata=NodeMetaOrder(updated_at=OrderDirection.DESC)) ) for device in devices: metadata = device.get_node_metadata() print(f"{device.name.value} - Last updated: {metadata.updated_at}") ``` ``` from infrahub_sdk.enums import OrderDirection from infrahub_sdk.types import NodeMetaOrder, Order # Get the 10 most recently updated devices with their metadata devices = client.filters( kind="TestDevice", limit=10, include_metadata=True, order=Order(node_metadata=NodeMetaOrder(updated_at=OrderDirection.DESC)) ) for device in devices: metadata = device.get_node_metadata() print(f"{device.name.value} - Last updated: {metadata.updated_at}") ``` ## Query a node(s) in a different branch[​](#query-a-nodes-in-a-different-branch "Direct link to Query a node(s) in a different branch") If you want to query a node(s) in a different branch than the default branch with which the SDK client was initiated, then you can use the `branch` argument of the query methods. * Async * Sync ``` device = await client.get(kind="TestDevice", name__value="atl1-edge1", branch="refresh-site-atl1") ``` ``` device = client.get(kind="TestDevice", name__value="atl1-edge1", branch="refresh-site-atl1") ``` ## Query for a large amount of nodes[​](#query-for-a-large-amount-of-nodes "Direct link to Query for a large amount of nodes") When you have a query that matches a large amount of nodes, it could have an implication on the performance of the query. The SDK tries to optimize this process by using pagination, which happens transparently. However, even when using pagination, such queries could take a long time to complete. Using the `parallel` argument, we can enable concurrent retrieval of pages of data for large queries, which can significantly improve the resolution time of the query. The `parallel` argument is available for the SDK client's `filters` and `all` method. * Async * Sync ``` device = await client.all(kind="TestDevice", parallel=True) ``` ``` device = client.get(kind="TestDevice", parallel=True) ``` --- # Using Resource Managers The goal of this guide is to show you how to create a resource pool using the Python SDK, and how you can allocated resources with them. note The guide makes the assumption that we start with an Infrahub instance that doesn't have any data or schema loaded. ## Loading a schema[​](#loading-a-schema "Direct link to Loading a schema") Save the following schema in a file on your local system. The location or filename are not that important, but in this guide will be using `/tmp/schema.yml` ``` --- version: "1.0" nodes: - name: IPPrefix namespace: Ipam include_in_menu: false inherit_from: - "BuiltinIPPrefix" description: "IPv4 or IPv6 network" label: "IP Prefix" - name: IPAddress namespace: Ipam include_in_menu: false inherit_from: - "BuiltinIPAddress" description: "IP Address" label: "IP Address" - name: Device namespace: Infra description: "A Device" icon: "mdi:server" label: "Device" attributes: - name: name kind: Text label: Name optional: false relationships: - name: primary_ip label: "Primary IP Address" peer: IpamIPAddress optional: false kind: Attribute cardinality: one ``` Load the schema with the `infrahubctl` command. ❯ infrahubctl schema load /tmp/schema.yml ``` schema '/tmp/schema.yml' loaded successfully 1 schema processed in 6.846 seconds. ``` ## Creating an IP Prefix object[​](#creating-an-ip-prefix-object "Direct link to Creating an IP Prefix object") Next we will be creating an IP Prefix object, which the resource manager will use as a resource to allocate resources from. * Async * Sync ``` from infrahub_sdk import InfrahubClient client = await InfrahubClient() prefix = await client.create(kind="IpamIPPrefix", prefix="10.0.0.0/24", member_type="address") await prefix.save() ``` ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClient() prefix = client.create(kind="IpamIPPrefix", prefix="10.0.0.0/24", member_type="address") prefix.save() ``` ## Creating a resource manager[​](#creating-a-resource-manager "Direct link to Creating a resource manager") We can now create a resource manager of kind `CoreIPAddressPool`. The kind of the resource manager determines the kind of resource the manager will allocate. We will create a `CoreIPaddressPool` with the following properties: * Name: My IP address pool * Default Address Type: `IpamIPAddress` (the kind of the IP address node defined in our schema) * Default Prefix Size: 32 * Resources: 10.0.0.0/24 * IP Namespace: Namespace > Default - Async - Sync ``` pool = await client.create( kind="CoreIPAddressPool", name="My IP address pool", default_address_type="IpamIPAddress", default_prefix_size=32, resources=[prefix], is_pool=True, ip_namespace={"id": "default"} ) await pool.save() ``` ``` pool = client.create( kind="CoreIPAddressPool", name="My IP address pool", default_address_type="IpamIPAddress", default_prefix_size=32, resources=[prefix], is_pool=True, ip_namespace={"id": "default"} ) pool.save() ``` ## Allocating a resource out of the pool[​](#allocating-a-resource-out-of-the-pool "Direct link to Allocating a resource out of the pool") We can now start allocating resources out of the `CoreIPAddressPool` we created. We can use the resource manager to allocate resources out of a pool in 2 different ways: 1. Directly allocate a resource out of a pool. This is typically used when you need to allocate a resource that has no relation to other nodes. For example, allocating an IP address out of a pool that will be assigned to something that is not stored in Infrahub. 2. Allocate a resource out of a pool to a relationship of a node. For example, create a device and allocate an IP address out of a pool and assign it to the device ### Direct allocation of a resource[​](#direct-allocation-of-a-resource "Direct link to Direct allocation of a resource") The Python SDK provides 2 methods that perform direct allocations: * `allocate_next_ip_address` to allocate resources out of `CoreIPAddressPool` pools * `allocate_next_ip_prefix` to allocated resources out of `CoreIPPrefixPool` pools We will use `allocate_next_ip_address` to allocated an IP address out of the `CoreIPAddressPool` we created. * Async * Sync ``` ip_address = await client.allocate_next_ip_address( resource_pool=pool, data={"description": "my first allocated ip"} ) ``` ``` ip_address = client.allocate_next_ip_address( resource_pool=pool, data={"description": "my first allocated ip"} ) ``` We provided a data argument to method. This data argument is used to pass attributes and relationships that we want to set on the allocated resource. In this case we are setting to the description of the allocated IP Address. You can allocate resources in an idempotent way by passing an identifier argument to the allocation method. This identifier links the resource pool with the allocated resource allowing us to create idempotent allocation behavior. This is crucial when you want to allocate resources in an idempotent way using [generators](https://docs.infrahub.app/topics/generator). In this example we are executing the `allocate_next_ip_address` method 2 times, using the same identifier. * Async * Sync ``` ip_address2 = await client.allocate_next_ip_address(resource_pool=pool, identifier="my-allocated-ip") ip_address3 = await client.allocate_next_ip_address(resource_pool=pool, identifier="my-allocated-ip") assert ip_address2.id == ip_address3.id assert ip_address2.address.value == ip_address3.address.value ``` ``` ip_address2 = client.allocate_next_ip_address(resource_pool=pool, identifier="my-allocated-ip") ip_address3 = client.allocate_next_ip_address(resource_pool=pool, identifier="my-allocated-ip") assert ip_address2.id == ip_address3.id assert ip_address2.address.value == ip_address3.address.value ``` ### Allocating resources to a relationship of a node[​](#allocating-resources-to-a-relationship-of-a-node "Direct link to Allocating resources to a relationship of a node") Another way we can use resource managers is in situations where we create a node that has a relationship and we want to use a resource manager to allocate a new resource for that relationship. For example, we want to create a new device (or server) and assign an IP address to the device out of a pool. The Python SDK allows you to set the relationship of a Node to the resource pool that you want to allocate from. The resource will be allocated when you `save` the node. * Async * Sync ``` device = await client.create(kind="InfraDevice", name="dev-123", primary_ip=pool) await device.save() ``` ``` device = client.create(kind="InfraDevice", name="dev-123", primary_ip=pool) device.save() ``` When you allocated to a relationship of a node, the resource allocation happens in an idempotent way. An identifier is automatically assigned to the resource allocation in this case. ## Branch agnostic resource allocation[​](#branch-agnostic-resource-allocation "Direct link to Branch agnostic resource allocation") Resource managers have to allocate resources in a branch agnostic way. For example if we allocate a resource in a branch, then that resource should also be allocated in the main branch, even if the resource object does not yet exist in the main branch. Without this behavior we could allocate the resource multiple times, which should not be possible. Create a branch named `test` * Async * Sync ``` await client.branch.create("test") ``` ``` client.branch.create("test") ``` Allocate a new IP address in the `test` branch. * Async * Sync ``` ip_address4 = await client.allocate_next_ip_address( resource_pool=pool, branch="test" ) ``` ``` ip_address4 = client.allocate_next_ip_address( resource_pool=pool, branch="test" ) ``` Currently there is no dedicated functionality in the Python SDK to do this, so for now we have to use a GraphQL query. ``` query { InfrahubResourcePoolAllocated(pool_id: "", resource_id: "") { edges { node { display_label branch } } } IpamIPAddress { edges { node { display_label } } } } ``` Notice that we have one IP address allocated by the Resource manager in the test branch. The query in the main branch shows us this allocation, indicating that it has been allocated and the resource cannot be allocated again. However, the IP address does not exist itself within the main branch. ## CoreNumberPool and attribute allocation[​](#corenumberpool-and-attribute-allocation "Direct link to CoreNumberPool and attribute allocation") `CoreNumberPool` allocates integer values (such as VLAN IDs or AS numbers) directly to node attributes. The pool assigns the integer value at the moment the node is created on the server. ``` vlan = await client.create( kind="InfraVLAN", name="VLAN-100", vlan_id={"from_pool": {"id": pool_id}}, ) await vlan.save() ``` ### Limitation: `allow_upsert=True` with a pool-sourced HFID attribute[​](#limitation-allow_upserttrue-with-a-pool-sourced-hfid-attribute "Direct link to limitation-allow_upserttrue-with-a-pool-sourced-hfid-attribute") `CoreNumberPool` assigns the integer value at server creation time, and a new value on each creation. When a node's human-friendly identifier (HFID) includes a pool-sourced attribute, the HFID is never stable, so an upsert can never match an existing node by it. Rather than updating, each `save(allow_upsert=True)` would create another node. The SDK blocks this up front to avoid silently duplicating data. warning Calling `save(allow_upsert=True)` on a node whose HFID contains a `CoreNumberPool`-sourced attribute raises `ValidationError` before any network call is made. ``` # Schema has human_friendly_id: ["vlan_id__value"] vlan = await client.create( kind="InfraVLAN", name="VLAN-100", vlan_id={"from_pool": {"id": pool_id}}, ) # This raises ValidationError - vlan_id is pool-sourced and in the HFID, so the upsert can't be idempotent await vlan.save(allow_upsert=True) ``` **Alternatives:** * **Look up and reuse** - find the existing node by a stable attribute or relationship and reuse it, otherwise create it once. This is the idempotent pattern to use in generators: ``` existing = await client.filters(kind="InfraVLAN", service__ids=[service.id]) vlan = existing[0] if existing else await client.create( kind="InfraVLAN", name="VLAN-100", vlan_id={"from_pool": {"id": pool_id}}, ) await vlan.save() # creates on first run, pool assigns vlan_id; reuses afterwards ``` * **Explicit id** - if you already know the node's UUID, set `node.id` before saving. The upsert will use the UUID directly and skip HFID lookup: ``` vlan.id = "known-uuid" await vlan.save(allow_upsert=True) # matches on the id directly, no HFID lookup needed ``` * **Deterministic identifier** - if possible, design your schema so the HFID uses a non-pool attribute (for example, a human-assigned `name`) and keep `vlan_id` out of the HFID. --- # Using the client store The client in the SDK contains a store that is used to store objects in a local cache. The store is mainly used for the internal working of the SDK. It is used to create relations between objects that might not yet exist in the database, or to store relations for objects that were retrieved from the database, amongst other things. The store stores objects that we are retrieving from Infrahub using the different query methods. This allows to not have to keep references to objects throughout scripts, or avoids situations where we have to re-execute queries. Objects are stored in the following scenario: * The resulting objects from using the SDK client's `get`, `filters` or `all` methods - Async - Sync ``` tag = await client.get(kind="BuiltinTag", name__value="RED") tag_in_store = client.store.get(key=tag.id) ``` ``` tag = client.get(kind="BuiltinTag", name__value="RED") tag_in_store = client.store.get(key=tag.id) ``` * The resulting related objects for objects retrieved using the SDK client's query methods, when we use the `prefetch_relationships` argument. - Async - Sync ``` device = await client.get(kind="InfraDevice", name__value="atl1-edge1", prefetch_relationships=True) site = client.store.get(key=device.site.id) ``` ``` device = client.get(kind="InfraDevice", name__value="atl1-edge1", prefetch_relationships=True) site = client.store.get(key=device.site.id) ``` * The related objects of a object's relationship when the `fetch` method is used - Async - Sync ``` device = await client.get(kind="InfraDevice", name__value="atl1-edge1") await device.site.fetch() site = client.store.get(key=device.site.id) ``` ``` device = client.get(kind="InfraDevice", name__value="atl1-edge1") device.site.fetch() site = client.store.get(key=device.site.id) ``` * Objects that get created using the SDK - Async - Sync ``` tag = await client.create("BuiltinTag", name="BLACK") await tag.save() tag_in_store = client.store.get(key=tag.id) ``` ``` tag = client.create("BuiltinTag", name="BLACK") tag.save() tag_in_store = client.store.get(key=tag.id) ``` ## Retrieving objects from the store[​](#retrieving-objects-from-the-store "Direct link to Retrieving objects from the store") You can retrieve objects from the store using their `id` or `hfid`. When using the `hfid`, we also have to provide the `kind` of the object that we want to retrieve. * Async * Sync ``` tag = await client.get("BuiltinTag", name__value="BLACK") tag_in_store = client.store.get(key=tag.id) tag == tag_in_store tag = await client.get("BuiltinTag", name__value="BLACK") tag_in_store = client.store.get(key=tag.hfid, kind="BuiltinTag") tag == tag_in_store ``` ``` tag = client.get("BuiltinTag", name__value="BLACK") tag_in_store = client.store.get(key=tag.id) tag == tag_in_store tag = client.get("BuiltinTag", name__value="BLACK") tag_in_store = client.store.get(key=tag.hfid, kind="BuiltinTag") tag == tag_in_store ``` ## Manually storing objects in the store[​](#manually-storing-objects-in-the-store "Direct link to Manually storing objects in the store") You can store objects in the store manually using the `set` method. This has the advantage that you can choose a key that you want to use to reference the object in the store, besides the `id` or `hfid`. For example, we could use the name attribute value of the node as the key. * Async * Sync ``` tag = await client.get(kind="BuiltinTag", name__value="RED", populate_store=False) client.store.set(key=tag.name.value, node=tag) tag_in_store = client.store.get(key=tag.name.value) tag_in_store = client.store.get(key=tag.id) tag_in_store = client.store.get(key=tag.hfid, kind="BuiltinTag") ``` ``` tag = client.get(kind="BuiltinTag", name__value="RED", populate_store=False) client.store.set(key=tag.name.value, node=tag) tag_in_store = client.store.get(key=tag.name.value) tag_in_store = client.store.get(key=tag.id) tag_in_store = client.store.get(key=tag.hfid, kind="BuiltinTag") ``` ## Disable storing objects in the store using the different query methods[​](#disable-storing-objects-in-the-store-using-the-different-query-methods "Direct link to Disable storing objects in the store using the different query methods") In some scenarios it might not be desirable to automatically store the retrieved objects in the store, when using the SDK client's different query methods. In this case you can set the `populate_store` argument to `False`. * Async * Sync ``` tag = await client.get(kind="BuiltinTag", name__value="RED", populate_store=False) ``` ``` tag = client.get(kind="BuiltinTag", name__value="RED", populate_store=False) ``` --- # Using the client tracking mode The Python SDK provides a feature known as *Tracking Mode*. This mode allows for the aggregation and tracking of operations performed during a session, enhancing efficiency and data management. ## Overview[​](#overview "Direct link to Overview") Tracking mode is designed for scenarios where multiple operations or mutations are performed in sequence, and you want to consolidate or track these operations for efficiency or auditing purposes. It is ideal for scenarios requiring precise control and repeatability, such as idempotent scripting. Under the hood, tracking mode leverages a `CoreStandardGroup` object to aggregate and track these operations. For a detailed exploration of tracking mode applications and how to use it for idempotent scripting, visit the [tracking topic](/python-sdk/topics/tracking.md). ## Automatic tracking[​](#automatic-tracking "Direct link to Automatic tracking") The Infrahub Python SDK offers a streamlined process for managing the tracking session lifecycle using the context manager. This method is available for both asynchronous and synchronous clients, ensuring all operations within the context are efficiently tracked without manual intervention. ### Utilizing the context manager[​](#utilizing-the-context-manager "Direct link to Utilizing the context manager") With the `start_tracking` method, you initiate tracking mode as you enter the context and automatically conclude the tracking session as you exit. This guarantees that all operations performed within the context are tracked under the specified session, simplifying tracking management. * Async * Sync ``` # Auto-manage tracking session with async context manager async with client.start_tracking(identifier="my_tracking_session", params=params, delete_unused_nodes=True) as session: # Tracked operations node = await session.create(kind="MyNodeKind", data={"name": "Example"}) await node.save() # Optionally, add related Nodes and Groups to the context await session.group_context.add_related_nodes([another_node.id]) await session.group_context.add_related_groups([group.id]) ``` ``` # Auto-manage tracking session with sync context manager with client.start_tracking(identifier="my_tracking_session", params=params, delete_unused_nodes=True) as session: # Tracked operations node = session.create(kind="MyNodeKind", data={"name": "Example"}) node.save() # Optionally, add related Nodes and Groups to the context session.group_context.add_related_nodes([another_node.id]) session.group_context.add_related_groups([group.id]) ``` info The context manager feature elegantly handles the start and conclusion of tracking sessions, making your code cleaner and less prone to errors related to manual tracking management. ## Manual tracking[​](#manual-tracking "Direct link to Manual tracking") Manual tracking involves explicitly starting, managing, and concluding sessions, offering fine-grained control over the tracking process. ### Setting up[​](#setting-up "Direct link to Setting up") First, ensure the Python SDK is installed and configured to communicate with your Infrahub instance. Then, let's start by setting up our client and enabling tracking mode. Once tracking mode is enabled, it's important to note that only the objects that are being saved (created or updated) will be automatically tracked under the specified session identifier. Objects that are merely queried and not modified or saved will not be added to the tracking group. This behavior ensures that the tracking group specifically reflects changes made during the session, providing a clear audit trail of modifications. * Async * Sync ``` from infrahub_sdk.client import InfrahubClient client = InfrahubClient(address="http://localhost:8000") await client.start_tracking(identifier="my_tracking_session") node = await client.create(kind="MyNodeKind", data={"name": "Example"}) await node.save() ``` ``` from infrahub_sdk.client_sync import InfrahubClientSync client = InfrahubClientSync(address="http://localhost:8000") client.start_tracking(identifier="my_tracking_session") node = client.create(kind="MyNodeKind", data={"name": "Example"}) node.save() ``` #### Tracking parameters[​](#tracking-parameters "Direct link to Tracking parameters") When enabling tracking mode with the `start_tracking` method, you can customize the tracking session with the following parameters: | Parameter | Description | | ------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | **identifier** | Unique string to identify the session, used for correlating operations and logs. Defaults to "python-sdk" if not specified. | | **params** | Optional dictionary for extra context, enabling fine-grained control over tracking. | | **delete\_unused\_nodes** | Boolean indicating if nodes not referenced should be automatically deleted, helping maintain a clean state. | | **group\_type** | Type of group object for tracking, default is `CoreStandardGroup`, customizable for specific grouping logic. | These parameters provide flexibility, enabling detailed auditing, efficient data management, and support for idempotent operations. #### Advanced tracking with parameters[​](#advanced-tracking-with-parameters "Direct link to Advanced tracking with parameters") In addition to the basic tracking mode functionalities, the Infrahub Python SDK allows for more granular control over the tracking sessions through parameters. These parameters can be beneficial for categorizing, filtering, or adding metadata to the groups created or updated during the tracking session. When starting a tracking session with `start_tracking,` you can pass a dictionary of parameters to define further the context and characteristics of the group being tracked. This allows for dynamic grouping based on runtime data, user inputs, or other operational metrics. * Async * Sync ``` params = { "data_source": "external", "analysis_type": "ipam_data" } await client.start_tracking( identifier="weekly_analysis", params=params, delete_unused_nodes=True ) ``` ``` params = { "data_source": "external", "analysis_type": "ipam_data" } client.start_tracking( identifier="weekly_analysis", params=params, delete_unused_nodes=True ) ``` ### Ending a tracking session and updating tracking information[​](#ending-a-tracking-session-and-updating-tracking-information "Direct link to Ending a tracking session and updating tracking information") This step involves creating or updating (using upsert) the `CoreStandardGroup` used to store all the Nodes and Groups used during execution. * Async * Sync ``` from infrahub_sdk.client import InfrahubClient client = InfrahubClient(address="http://localhost:8000") await client.start_tracking(identifier="my_tracking_session") node = await client.create(kind="MyNodeKind", data={"name": "Example"}) await node.save() # Update tracking information for async client await client.group_context.update_group() ``` ``` from infrahub_sdk.client_sync import InfrahubClientSync client = InfrahubClientSync(address="http://localhost:8000") client.start_tracking(identifier="my_tracking_session") node = client.create(kind="MyNodeKind", data={"name": "Example"}) node.save() # Update tracking information for sync client client.group_context.update_group() ``` ## Retrieving and manipulating groups[​](#retrieving-and-manipulating-groups "Direct link to Retrieving and manipulating groups") After setting up a tracking session with specific parameters, you may need to retrieve the group associated with this session for further manipulation. This involves fetching the group, accessing its members, and executing specific logic based on the member types. Here's how to retrieve a group based on an identifier and parameters and interact with its members: * Async * Sync ``` # Set the context properties to match the tracking session await client.set_context_properties(identifier="my_tracking_session", params=params) # Retrieve the group associated with the specified identifier and parameters group = await client.group_context.get_group(store_peers=True) # Check if the group exists if group: # Access previous members of the group, if any if client.group_context.previous_members: for member in client.group_context.previous_members: # Fetch the object from the store based on the member's type and ID obj = client.store.get(kind=member._typename, key=member.id) # Perform operations based on the member's type pass ``` ``` # Set the context properties to match the tracking session client.set_context_properties(identifier="my_tracking_session", params=params) # Retrieve the group associated with the specified identifier and parameters group = client.group_context.get_group(store_peers=True) # Check if the group exists if group: # Access previous members of the group, if any if client.group_context.previous_members: for member in client.group_context.previous_members: # Fetch the object from the store based on the member's type and ID obj = client.store.get(kind=member._typename, key=member.id) # Perform operations based on the member's type pass ``` --- # Infrahub Python SDK The Infrahub Python SDK is a client library for interacting with [Infrahub](https://docs.infrahub.app/) programmatically. It handles authentication, query construction, and data serialization so you can work with infrastructure data using native Python objects instead of raw API calls. ## What you can do with it[​](#what-you-can-do-with-it "Direct link to What you can do with it") The SDK covers three main use cases: * **Automate inside Infrahub** — Write [transforms](https://docs.infrahub.app/topics/transformation), [generators](https://docs.infrahub.app/topics/generator), and [checks](https://docs.infrahub.app/topics/check) that run as part of Infrahub's pipeline. * **Integrate with external systems** — Query and sync data between Infrahub and your existing tools. * **Build custom applications** — Use Infrahub as a data backend for your own Python projects. Under the hood, tools like [`infrahubctl`](/infrahubctl/infrahubctl.md) and the [Infrahub Ansible collection](https://docs.infrahub.app/ansible) both use this SDK. ## A quick look[​](#a-quick-look "Direct link to A quick look") * Async * Sync ``` import asyncio from infrahub_sdk import InfrahubClient async def main(): client = InfrahubClient() # Create a new tag and save it tag = await client.create( kind="BuiltinTag", name="staging", description="Resources in the staging environment", ) await tag.save() # Query all tags tags = await client.all(kind="BuiltinTag") for tag in tags: print(f"{tag.name.value} — {tag.description.value}") asyncio.run(main()) ``` ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync() # Create a new tag and save it tag = client.create( kind="BuiltinTag", name="staging", description="Resources in the staging environment", ) tag.save() # Query all tags tags = client.all(kind="BuiltinTag") for tag in tags: print(f"{tag.name.value} — {tag.description.value}") ``` Both async and sync clients expose the same API. Choose async for applications that benefit from concurrent I/O (transforms, large-scale sync scripts) and sync for straightforward scripting. ## When to use the SDK instead of direct API calls[​](#when-to-use-the-sdk-instead-of-direct-api-calls "Direct link to When to use the SDK instead of direct API calls") | Concern | Raw API | SDK | | -------------- | ------------------------------------ | -------------------------------------------------- | | Authentication | Manual token management, JWT refresh | Handled automatically | | Querying | Build GraphQL queries by hand | `client.get()`, `client.all()`, `client.filters()` | | Mutations | Construct and POST GraphQL mutations | `node.save()`, `node.delete()` | | Concurrency | Roll your own async batching | Built-in `batch` with concurrency control | | Typing | Maintain type definitions manually | Schema-driven type export using protocols | tip The SDK removes the boilerplate so you can focus on the logic that matters to your infrastructure. ## Key capabilities[​](#key-capabilities "Direct link to Key capabilities") * **CRUD operations** — Create, read, update, and delete any node type defined in your schema. * **Batch execution** — Group multiple queries into a batch with configurable concurrency limits. * **Tracking** — Tag operations with identifiers for auditing and idempotent updates. * **Store** — Cache retrieved nodes locally to reduce redundant queries. * **GraphQL escape hatch** — Run arbitrary GraphQL queries when the high-level API doesn't cover your use case. ## Video walkthrough[​](#video-walkthrough "Direct link to Video walkthrough") ## Installation[​](#installation "Direct link to Installation") * uv * pip ``` uv add infrahub-sdk ``` Extras are available for additional functionality: ``` uv add 'infrahub-sdk[ctl]' # Adds the infrahubctl CLI uv add 'infrahub-sdk[tests]' # Adds the testing framework for transforms and checks uv add 'infrahub-sdk[all]' # Everything ``` ``` pip install infrahub-sdk ``` Extras are available for additional functionality: ``` pip install 'infrahub-sdk[ctl]' # Adds the infrahubctl CLI pip install 'infrahub-sdk[tests]' # Adds the testing framework for transforms and checks pip install 'infrahub-sdk[all]' # Everything ``` [Installation guide/python-sdk/guides/installation](/python-sdk/guides/installation.md) ## Next steps[​](#next-steps "Direct link to Next steps") * **[Hello world example](/python-sdk/guides/client.md#hello-world-example)** — Your first client connection and query. * **[Create and configure a client](/python-sdk/guides/client.md)** — Set up authentication, proxy settings, and client options. * **[Query data](/python-sdk/guides/query_data.md)** — Retrieve nodes with filters and GraphQL. * **[Create, update, and delete nodes](/python-sdk/guides/create_update_delete.md)** — Manage infrastructure data programmatically. * **[Work with branches](/python-sdk/guides/branches.md)** — Use Infrahub's branch workflow from Python. * **[Batch operations](/python-sdk/guides/batch.md)** — Optimize performance for bulk operations. * **[Client configuration reference](/python-sdk/reference/config.md)** — All available configuration options. * **[Compatibility matrix](/python-sdk/reference/compatibility.md)** * **[Python SDK Release Notes](https://github.com/opsmill/infrahub-sdk-python/releases)** --- # Compatibility matrix This page documents which versions of the Infrahub Python SDK are compatible with each version of Infrahub. ## Version mapping[​](#version-mapping "Direct link to Version mapping") Each Infrahub release pins a specific SDK version. Using the matching SDK version ensures full compatibility. Newer patch releases of the SDK within the same minor version are generally safe to use. | Infrahub | Required SDK | Release date | | -------- | ------------ | -------------- | | 1.11.x | >= 1.23.0 | August 2026 | | 1.10.x | >= 1.22.0 | June 2026 | | 1.9.x | >= 1.20.0 | April 2026 | | 1.8.x | >= 1.19.0 | March 2026 | | 1.7.x | >= 1.18.1 | January 2026 | | 1.6.x | >= 1.16.0 | December 2025 | | 1.5.x | >= 1.15.0 | November 2025 | | 1.4.x | >= 1.13.5 | August 2025 | | 1.3.x | >= 1.13.0 | June 2025 | | 1.2.x | >= 1.8.0 | March 2025 | | 1.1.x | >= 1.3.0 | December 2024 | | 1.0.x | >= 1.0.0 | October 2024 | | 0.16.x | >= 0.13.1 | September 2024 | ## Detailed release mapping[​](#detailed-release-mapping "Direct link to Detailed release mapping") The table below shows the exact SDK version pinned to each Infrahub release. | Infrahub | SDK version | Infrahub release date | | -------- | ----------- | --------------------- | | 1.11.2 | 1.23.1 | 2026-09-03 | | 1.11.1 | 1.23.1 | 2026-08-31 | | 1.11.0 | 1.23.0 | 2026-08-19 | | 1.10.10 | 1.22.3 | 2026-08-31 | | 1.10.9 | 1.22.3 | 2026-08-19 | | 1.10.8 | 1.22.2 | 2026-08-14 | | 1.10.7 | 1.22.2 | 2026-08-11 | | 1.10.6 | 1.22.2 | 2026-07-28 | | 1.10.5 | 1.22.1 | 2026-07-15 | | 1.10.4 | 1.22.1 | 2026-07-13 | | 1.10.3 | 1.22.1 | 2026-07-08 | | 1.10.2 | 1.22.0 | 2026-07-03 | | 1.10.1 | 1.22.0 | 2026-07-01 | | 1.10.0 | 1.22.0 | 2026-06-23 | | 1.9.10 | 1.20.1 | 2026-07-07 | | 1.9.9 | 1.20.1 | 2026-06-23 | | 1.9.8 | 1.20.1 | 2026-06-09 | | 1.9.7 | 1.20.1 | 2026-06-03 | | 1.9.6 | 1.20.1 | 2026-05-20 | | 1.9.5 | 1.20.0 | 2026-05-18 | | 1.9.4 | 1.20.0 | 2026-05-13 | | 1.9.3 | 1.20.0 | 2026-05-05 | | 1.9.2 | 1.20.0 | 2026-04-30 | | 1.9.1 | 1.20.0 | 2026-04-29 | | 1.9.0 | 1.20.0 | 2026-04-24 | | 1.8.7 | 1.19.0 | 2026-06-04 | | 1.8.6 | 1.19.0 | 2026-04-21 | | 1.8.5 | 1.19.0 | 2026-04-17 | | 1.8.4 | 1.19.0 | 2026-04-02 | | 1.8.3 | 1.19.0 | 2026-03-31 | | 1.8.2 | 1.19.0 | 2026-03-25 | | 1.8.1 | 1.19.0 | 2026-03-19 | | 1.8.0 | 1.19.0 | 2026-03-16 | | 1.7.7 | 1.18.1 | 2026-03-12 | | 1.7.6 | 1.18.1 | 2026-02-25 | | 1.7.5 | 1.18.1 | 2026-02-24 | | 1.7.4 | 1.18.1 | 2026-02-03 | | 1.7.3 | 1.18.1 | 2026-01-28 | | 1.7.2 | 1.18.1 | 2026-01-27 | | 1.7.1 | 1.18.1 | 2026-01-12 | | 1.7.0 | 1.18.1 | 2026-01-09 | | 1.6.3 | 1.17.0 | 2026-01-07 | | 1.6.2 | 1.17.0 | 2025-12-22 | | 1.6.1 | 1.17.0 | 2025-12-11 | | 1.6.0 | 1.16.0 | 2025-12-01 | | 1.5.5 | 1.15.1 | 2025-12-22 | | 1.5.4 | 1.15.1 | 2025-12-16 | | 1.5.3 | 1.15.1 | 2025-11-24 | | 1.5.2 | 1.15.1 | 2025-11-18 | | 1.5.1 | 1.15.1 | 2025-11-13 | | 1.5.0 | 1.15.0 | 2025-11-10 | | 1.4.13 | 1.13.5 | 2025-11-06 | | 1.4.12 | 1.13.5 | 2025-10-23 | | 1.4.11 | 1.13.5 | 2025-10-17 | | 1.4.10 | 1.13.5 | 2025-10-01 | | 1.4.9 | 1.13.5 | 2025-09-26 | | 1.4.8 | 1.13.5 | 2025-09-23 | | 1.4.7 | 1.13.5 | 2025-09-16 | | 1.4.6 | 1.13.5 | 2025-09-10 | | 1.4.5 | 1.14.0 | 2025-09-08 | | 1.4.4 | 1.13.5 | 2025-09-03 | | 1.4.3 | 1.13.5 | 2025-08-29 | | 1.4.2 | 1.13.5 | 2025-08-28 | | 1.4.1 | 1.13.5 | 2025-08-27 | | 1.4.0 | 1.13.5 | 2025-08-26 | | 1.3.9 | 1.13.5 | 2025-09-08 | | 1.3.8 | 1.13.5 | 2025-08-26 | | 1.3.7 | 1.13.5 | 2025-08-14 | | 1.3.6 | 1.13.5 | 2025-08-11 | | 1.3.5 | 1.13.5 | 2025-08-05 | | 1.3.4 | 1.13.5 | 2025-07-24 | | 1.3.3 | 1.13.3 | 2025-07-15 | | 1.3.2 | 1.13.3 | 2025-06-30 | | 1.3.1 | 1.13.2 | 2025-06-27 | | 1.3.0 | 1.13.0 | 2025-06-12 | | 1.2.12 | 1.12.1 | 2025-06-03 | | 1.2.11 | 1.12.1 | 2025-05-23 | | 1.2.10 | 1.12.1 | 2025-05-14 | | 1.2.9 | 1.12.0 | 2025-05-07 | | 1.2.8 | 1.12.0 | 2025-05-01 | | 1.2.7 | 1.11.1 | 2025-04-28 | | 1.2.6 | 1.10.2 | 2025-04-18 | | 1.2.5 | 1.10.2 | 2025-04-12 | | 1.2.4 | 1.10.1 | 2025-04-04 | | 1.2.3 | 1.10.0 | 2025-04-01 | | 1.2.2 | 1.9.2 | 2025-03-28 | | 1.2.1 | 1.9.1 | 2025-03-26 | | 1.2.0 | 1.8.0 | 2025-03-21 | | 1.1.10 | 1.7.2 | 2025-04-01 | | 1.1.9 | 1.7.2 | 2025-03-17 | | 1.1.8 | 1.7.2 | 2025-03-08 | | 1.1.7 | 1.7.0 | 2025-02-18 | | 1.1.6 | 1.7.1 | 2025-01-31 | | 1.1.5 | 1.7.0 | 2025-01-24 | | 1.1.4 | 1.6.1 | 2025-01-17 | | 1.1.3 | 1.6.0 | 2025-01-16 | | 1.1.2 | 1.5.0 | 2025-01-09 | | 1.1.1 | 1.4.0 | 2025-01-06 | | 1.1.0 | 1.3.0 | 2024-12-30 | | 1.0.10 | 1.1.0 | 2024-12-20 | | 1.0.9 | 1.1.0 | 2024-12-13 | | 1.0.8 | 1.1.0 | 2024-12-03 | | 1.0.7 | 1.0.1 | 2024-11-20 | | 1.0.6 | 1.0.1 | 2024-11-18 | | 1.0.5 | 1.0.1 | 2024-11-15 | | 1.0.4 | 1.0.1 | 2024-11-13 | | 1.0.3 | 1.0.0 | 2024-11-08 | | 1.0.2 | 1.0.0 | 2024-11-06 | | 1.0.1 | 1.0.0 | 2024-10-31 | | 0.16.4 | 0.14.0 | 2024-10-17 | | 0.16.3 | 0.14.0 | 2024-10-10 | | 0.16.2 | 0.13.1 | 2024-10-01 | | 0.16.1 | 0.13.1 | 2024-09-25 | ## Python version support[​](#python-version-support "Direct link to Python version support") | SDK version | Python versions | | --------------- | ---------------------------- | | >= 1.17.0 | 3.10, 3.11, 3.12, 3.13, 3.14 | | 1.16.0 | 3.10, 3.11, 3.12, 3.13 | | 1.13.0 - 1.15.x | 3.9, 3.10, 3.11, 3.12, 3.13 | note The Infrahub server requires Python 3.12 or later. The SDK supports older Python versions so it can be used from environments that do not run the server itself. ## Feature-specific requirements[​](#feature-specific-requirements "Direct link to Feature-specific requirements") Some SDK features require a minimum Infrahub version: | Feature | Minimum SDK | Minimum Infrahub | | ------------------------- | ----------- | ---------------- | | infrahubctl branch report | 1.19.0 | 1.7 | | FileObject support | 1.19.0 | 1.8 | | NumberPool support | 1.13.0 | 1.3 | ## General guidance[​](#general-guidance "Direct link to General guidance") * **Use the SDK version that matches your Infrahub release.** The version mapping table above shows which SDK version was tested and shipped with each Infrahub release. * **Patch upgrades within a minor SDK version are safe.** For example, if your Infrahub version pins SDK 1.18.0, upgrading to 1.18.1 is safe. * **Upgrading to a newer minor SDK version may work** but is not guaranteed. New SDK features may depend on server-side changes that are not present in older Infrahub versions. --- # Python SDK Configuration The Python SDK (Async or Sync) client can be configured using an instance of the `Config` class. * Async * Sync ``` from infrahub_sdk import Config, InfrahubClient config = Config(address="http://infrahub:8080", api_token="123-xyz-invalid-token") client = InfrahubClient(config=config) ``` ``` from infrahub_sdk import Config, InfrahubClientSync config = Config(address="http://infrahub:8080", api_token="123-xyz-invalid-token") client = InfrahubClientSync(config=config) ``` The following settings can be defined in the `Config` class ## address[​](#address "Direct link to address") **Description**: The URL to use when connecting to Infrahub.
**Type**: `string`
**Default value**:
**Environment variable**: `INFRAHUB_ADDRESS`
## api\_token[​](#api_token "Direct link to api_token") **Description**: API token for authentication against Infrahub.
**Type**: `string`
**Environment variable**: `INFRAHUB_API_TOKEN`
## echo\_graphql\_queries[​](#echo_graphql_queries "Direct link to echo_graphql_queries") **Description**: If set the GraphQL query and variables will be echoed to the screen
**Type**: `boolean`
**Default value**: False
**Environment variable**: `INFRAHUB_ECHO_GRAPHQL_QUERIES`
## username[​](#username "Direct link to username") **Description**: Username for accessing Infrahub
**Type**: `string`
**Environment variable**: `INFRAHUB_USERNAME`
## password[​](#password "Direct link to password") **Description**: Password for accessing Infrahub
**Type**: `string`
**Environment variable**: `INFRAHUB_PASSWORD`
## default\_branch[​](#default_branch "Direct link to default_branch") **Description**: Default branch to target if not specified for each request.
**Type**: `string`
**Default value**: main
**Environment variable**: `INFRAHUB_DEFAULT_BRANCH`
## default\_branch\_from\_git[​](#default_branch_from_git "Direct link to default_branch_from_git") **Description**: Indicates if the default Infrahub branch to target should come from the active branch in the local Git repository.
**Type**: `boolean`
**Default value**: False
**Environment variable**: `INFRAHUB_DEFAULT_BRANCH_FROM_GIT`
## identifier[​](#identifier "Direct link to identifier") **Description**: Tracker identifier
**Type**: `string`
**Environment variable**: `INFRAHUB_IDENTIFIER`
## insert\_tracker[​](#insert_tracker "Direct link to insert_tracker") **Description**: Insert a tracker on queries to the server
**Type**: `boolean`
**Default value**: False
**Environment variable**: `INFRAHUB_INSERT_TRACKER`
## max\_concurrent\_execution[​](#max_concurrent_execution "Direct link to max_concurrent_execution") **Description**: Max concurrent execution in batch mode
**Type**: `integer`
**Default value**: 5
**Environment variable**: `INFRAHUB_MAX_CONCURRENT_EXECUTION`
## mode[​](#mode "Direct link to mode") **Description**: Default mode for the client
**Type**: `object`
**Environment variable**: `INFRAHUB_MODE`
## pagination\_size[​](#pagination_size "Direct link to pagination_size") **Description**: Page size for queries to the server
**Type**: `integer`
**Default value**: 50
**Environment variable**: `INFRAHUB_PAGINATION_SIZE`
## priority[​](#priority "Direct link to priority") **Description**: Default request priority emitted as the X-Priority header on every request; one of high|medium|low (case-insensitive). When unset, no header is sent.
**Type**: `object`
**Environment variable**: `INFRAHUB_PRIORITY`
## retry\_delay[​](#retry_delay "Direct link to retry_delay") **Description**: Base delay in seconds before retrying a request that failed with a transient error. The delay doubles after every attempt, with jitter, up to the maximum retry delay.
**Type**: `integer`
**Default value**: 5
**Environment variable**: `INFRAHUB_RETRY_DELAY`
## retry\_max\_delay[​](#retry_max_delay "Direct link to retry_max_delay") **Description**: Maximum delay in seconds between two retries of a request that failed with a transient error.
**Type**: `integer`
**Default value**: 60
**Environment variable**: `INFRAHUB_RETRY_MAX_DELAY`
## retry\_on\_failure[​](#retry_on_failure "Direct link to retry_on_failure") **Description**: Retry requests that fail with a transient error: connection error, timeout, an HTTP status listed in the retry status codes, or a GraphQL error the server flags with one of those statuses. Other errors are never retried. The maximum retry duration controls how long to keep retrying.
**Type**: `boolean`
**Default value**: False
**Environment variable**: `INFRAHUB_RETRY_ON_FAILURE`
## retry\_status\_codes[​](#retry_status_codes "Direct link to retry_status_codes") **Description**: HTTP status codes treated as transient when retrying on failure is enabled. Also matched against the HTTP status reported in GraphQL error extensions. 500 is included because Infrahub reports some transient database errors without further classification; remove it to fail fast on them.
**Type**: `array`
**Default value**: \[500, 502, 503, 504]
**Environment variable**: `INFRAHUB_RETRY_STATUS_CODES`
## rate\_limit\_retry\_enabled[​](#rate_limit_retry_enabled "Direct link to rate_limit_retry_enabled") **Description**: Retry requests that receive HTTP 429 using backoff. Set False to disable.
**Type**: `boolean`
**Default value**: True
**Environment variable**: `INFRAHUB_RATE_LIMIT_RETRY_ENABLED`
## rate\_limit\_max\_retries[​](#rate_limit_max_retries "Direct link to rate_limit_max_retries") **Description**: Maximum number of retries after the initial attempt when receiving HTTP 429.
**Type**: `integer`
**Default value**: 10
**Environment variable**: `INFRAHUB_RATE_LIMIT_MAX_RETRIES`
## rate\_limit\_backoff\_base[​](#rate_limit_backoff_base "Direct link to rate_limit_backoff_base") **Description**: Base interval in seconds for exponential backoff between 429 retries.
**Type**: `number`
**Default value**: 0.5
**Environment variable**: `INFRAHUB_RATE_LIMIT_BACKOFF_BASE`
## rate\_limit\_backoff\_max[​](#rate_limit_backoff_max "Direct link to rate_limit_backoff_max") **Description**: Maximum wait in seconds for any single 429 retry (also clamps Retry-After).
**Type**: `number`
**Default value**: 60.0
**Environment variable**: `INFRAHUB_RATE_LIMIT_BACKOFF_MAX`
## max\_retry\_duration[​](#max_retry_duration "Direct link to max_retry_duration") **Description**: Maximum number of seconds to keep retrying a request that fails with transient errors when retrying on failure is enabled. Set to 0 to retry indefinitely.
**Type**: `integer`
**Default value**: 300
**Environment variable**: `INFRAHUB_MAX_RETRY_DURATION`
## schema\_converge\_timeout[​](#schema_converge_timeout "Direct link to schema_converge_timeout") **Description**: Number of seconds to wait for schema to have converged
**Type**: `integer`
**Default value**: 60
**Environment variable**: `INFRAHUB_SCHEMA_CONVERGE_TIMEOUT`
## timeout[​](#timeout "Direct link to timeout") **Description**: Default request timeout in seconds, applied to the read, write and pool phases of a request.
**Type**: `integer`
**Default value**: 60
**Environment variable**: `INFRAHUB_TIMEOUT`
## connect\_timeout[​](#connect_timeout "Direct link to connect_timeout") **Description**: Timeout in seconds for establishing the TCP/TLS connection to Infrahub. Kept short so an unreachable address fails fast; it never exceeds the request timeout.
**Type**: `integer`
**Default value**: 10
**Environment variable**: `INFRAHUB_CONNECT_TIMEOUT`
## transport[​](#transport "Direct link to transport") **Description**: Set an alternate transport using a predefined option
**Type**: `object`
**Environment variable**: `INFRAHUB_TRANSPORT`
## proxy[​](#proxy "Direct link to proxy") **Description**: Proxy address
**Type**: `string`
**Environment variable**: `INFRAHUB_PROXY`
## proxy\_mounts[​](#proxy_mounts "Direct link to proxy_mounts") **Description**: Proxy mounts configuration
**Type**: `object`
**Environment variable**: `INFRAHUB_PROXY_MOUNTS`
## marketplace\_url[​](#marketplace_url "Direct link to marketplace_url") **Description**: Base URL for the Infrahub Marketplace.
**Type**: `string`
**Default value**:
**Environment variable**: `INFRAHUB_MARKETPLACE_URL`
## update\_group\_context[​](#update_group_context "Direct link to update_group_context") **Description**: Update GraphQL query groups
**Type**: `boolean`
**Default value**: False
**Environment variable**: `INFRAHUB_UPDATE_GROUP_CONTEXT`
## tls\_insecure[​](#tls_insecure "Direct link to tls_insecure") **Description**:
Indicates if TLS certificates are verified. Enabling this option will disable: CA verification, expiry date verification, hostname verification). Can be useful to test with self-signed certificates.
**Type**: `boolean`
**Default value**: False
**Environment variable**: `INFRAHUB_TLS_INSECURE`
## tls\_ca\_file[​](#tls_ca_file "Direct link to tls_ca_file") **Description**: File path to CA cert or bundle in PEM format
**Type**: `string`
**Environment variable**: `INFRAHUB_TLS_CA_FILE`
## recorder[​](#recorder "Direct link to recorder") **Property**: recorder
**Description**: Select builtin recorder for later replay.
**Type**: `RecorderType`
**Default value**: RecorderType.NONE
## custom\_recorder[​](#custom_recorder "Direct link to custom_recorder") **Property**: custom\_recorder
**Description**: Provides a way to record responses from the Infrahub API
**Type**: `Recorder` (protocol)
**Default value**: NoRecorder.default
## requester[​](#requester "Direct link to requester") **Property**: requester
**Type**: `AsyncRequester`
**Default value**: None
## sync\_requester[​](#sync_requester "Direct link to sync_requester") **Property**: sync\_requester
**Type**: `SyncRequester`
**Default value**: None
--- # Python SDK Templating Filters can be used when defining [computed attributes](https://docs.infrahub.app/guides/computed-attributes) or [Jinja2 Transforms](https://docs.infrahub.app/guides/jinja2-transform) within Infrahub. ## Execution contexts[​](#execution-contexts "Direct link to Execution contexts") Filters are restricted based on the execution context in which a template is rendered: * **CORE** — Computed attributes evaluated inside the Infrahub API server. Only fully trusted filters (no I/O, no side effects) are allowed. * **WORKER** — Jinja2 transforms executed on Prefect background workers. Trusted filters and Infrahub client-dependent filters are allowed. * **LOCAL** — Local CLI rendering and unrestricted usage. All filters are allowed. The `validate()` method on `Jinja2Template` accepts an optional `context` parameter to enforce these restrictions: ``` from infrahub_sdk.template import Jinja2Template from infrahub_sdk.template.filters import ExecutionContext template = Jinja2Template(template="{{ sid | artifact_content }}") # Raises JinjaTemplateOperationViolationError — blocked in CORE template.validate(context=ExecutionContext.CORE) # Passes — allowed in WORKER template.validate(context=ExecutionContext.WORKER) ``` For backward compatibility, `validate(restricted=True)` maps to `CORE` and `validate(restricted=False)` maps to `LOCAL`. ## Builtin Jinja2 filters[​](#builtin-jinja2-filters "Direct link to Builtin Jinja2 filters") The following filters are [shipped with Jinja2](https://jinja.palletsprojects.com/en/stable/templates/#list-of-builtin-filters) and enabled within Infrahub. | Name | CORE | WORKER | LOCAL | | -------------- | ---- | ------ | ----- | | abs | ✅ | ✅ | ✅ | | attr | ❌ | ✅ | ✅ | | batch | ❌ | ✅ | ✅ | | capitalize | ✅ | ✅ | ✅ | | center | ✅ | ✅ | ✅ | | count | ✅ | ✅ | ✅ | | d | ✅ | ✅ | ✅ | | default | ✅ | ✅ | ✅ | | dictsort | ❌ | ✅ | ✅ | | e | ✅ | ✅ | ✅ | | escape | ✅ | ✅ | ✅ | | filesizeformat | ✅ | ✅ | ✅ | | first | ✅ | ✅ | ✅ | | float | ✅ | ✅ | ✅ | | forceescape | ✅ | ✅ | ✅ | | format | ✅ | ✅ | ✅ | | groupby | ❌ | ✅ | ✅ | | indent | ✅ | ✅ | ✅ | | int | ✅ | ✅ | ✅ | | items | ❌ | ✅ | ✅ | | join | ✅ | ✅ | ✅ | | last | ✅ | ✅ | ✅ | | length | ✅ | ✅ | ✅ | | list | ✅ | ✅ | ✅ | | lower | ✅ | ✅ | ✅ | | map | ❌ | ✅ | ✅ | | max | ✅ | ✅ | ✅ | | min | ✅ | ✅ | ✅ | | pprint | ❌ | ✅ | ✅ | | random | ❌ | ✅ | ✅ | | reject | ❌ | ✅ | ✅ | | rejectattr | ❌ | ✅ | ✅ | | replace | ✅ | ✅ | ✅ | | reverse | ✅ | ✅ | ✅ | | round | ✅ | ✅ | ✅ | | safe | ❌ | ✅ | ✅ | | select | ❌ | ✅ | ✅ | | selectattr | ❌ | ✅ | ✅ | | slice | ✅ | ✅ | ✅ | | sort | ❌ | ✅ | ✅ | | string | ✅ | ✅ | ✅ | | striptags | ✅ | ✅ | ✅ | | sum | ✅ | ✅ | ✅ | | title | ✅ | ✅ | ✅ | | tojson | ❌ | ✅ | ✅ | | trim | ✅ | ✅ | ✅ | | truncate | ✅ | ✅ | ✅ | | unique | ❌ | ✅ | ✅ | | upper | ✅ | ✅ | ✅ | | urlencode | ✅ | ✅ | ✅ | | urlize | ❌ | ✅ | ✅ | | wordcount | ✅ | ✅ | ✅ | | wordwrap | ✅ | ✅ | ✅ | | xmlattr | ❌ | ✅ | ✅ | ## Netutils filters[​](#netutils-filters "Direct link to Netutils filters") The following Jinja2 filters from [Netutils](https://netutils.readthedocs.io) are included within Infrahub. | Name | CORE | WORKER | LOCAL | | ---------------------------------- | ---- | ------ | ----- | | abbreviated\_interface\_name | ✅ | ✅ | ✅ | | abbreviated\_interface\_name\_list | ✅ | ✅ | ✅ | | asn\_to\_int | ✅ | ✅ | ✅ | | bits\_to\_name | ✅ | ✅ | ✅ | | bytes\_to\_name | ✅ | ✅ | ✅ | | canonical\_interface\_name | ✅ | ✅ | ✅ | | canonical\_interface\_name\_list | ✅ | ✅ | ✅ | | cidr\_to\_netmask | ✅ | ✅ | ✅ | | cidr\_to\_netmaskv6 | ✅ | ✅ | ✅ | | clean\_config | ✅ | ✅ | ✅ | | compare\_version\_loose | ✅ | ✅ | ✅ | | compare\_version\_strict | ✅ | ✅ | ✅ | | config\_compliance | ✅ | ✅ | ✅ | | config\_section\_not\_parsed | ✅ | ✅ | ✅ | | delimiter\_change | ✅ | ✅ | ✅ | | diff\_network\_config | ✅ | ✅ | ✅ | | feature\_compliance | ✅ | ✅ | ✅ | | find\_unordered\_cfg\_lines | ✅ | ✅ | ✅ | | fqdn\_to\_ip | ❌ | ❌ | ✅ | | get\_all\_host | ❌ | ❌ | ✅ | | get\_broadcast\_address | ✅ | ✅ | ✅ | | get\_first\_usable | ✅ | ✅ | ✅ | | get\_ips\_sorted | ✅ | ✅ | ✅ | | get\_nist\_urls | ✅ | ✅ | ✅ | | get\_nist\_vendor\_platform\_urls | ✅ | ✅ | ✅ | | get\_oui | ✅ | ✅ | ✅ | | get\_peer\_ip | ✅ | ✅ | ✅ | | get\_range\_ips | ✅ | ✅ | ✅ | | get\_upgrade\_path | ✅ | ✅ | ✅ | | get\_usable\_range | ✅ | ✅ | ✅ | | hash\_data | ✅ | ✅ | ✅ | | int\_to\_asdot | ✅ | ✅ | ✅ | | interface\_range\_compress | ✅ | ✅ | ✅ | | interface\_range\_expansion | ✅ | ✅ | ✅ | | ip\_addition | ✅ | ✅ | ✅ | | ip\_subtract | ✅ | ✅ | ✅ | | ip\_to\_bin | ✅ | ✅ | ✅ | | ip\_to\_hex | ✅ | ✅ | ✅ | | ipaddress\_address | ✅ | ✅ | ✅ | | ipaddress\_interface | ✅ | ✅ | ✅ | | ipaddress\_network | ✅ | ✅ | ✅ | | is\_classful | ✅ | ✅ | ✅ | | is\_fqdn\_resolvable | ❌ | ❌ | ✅ | | is\_ip | ✅ | ✅ | ✅ | | is\_ip\_range | ✅ | ✅ | ✅ | | is\_ip\_within | ✅ | ✅ | ✅ | | is\_netmask | ✅ | ✅ | ✅ | | is\_network | ✅ | ✅ | ✅ | | is\_reversible\_wildcardmask | ✅ | ✅ | ✅ | | is\_valid\_mac | ✅ | ✅ | ✅ | | longest\_prefix\_match | ✅ | ✅ | ✅ | | mac\_normalize | ✅ | ✅ | ✅ | | mac\_to\_format | ✅ | ✅ | ✅ | | mac\_to\_int | ✅ | ✅ | ✅ | | mac\_type | ✅ | ✅ | ✅ | | name\_to\_bits | ✅ | ✅ | ✅ | | name\_to\_bytes | ✅ | ✅ | ✅ | | name\_to\_name | ✅ | ✅ | ✅ | | netmask\_to\_cidr | ✅ | ✅ | ✅ | | netmask\_to\_wildcardmask | ✅ | ✅ | ✅ | | normalise\_delimiter\_caret\_c | ✅ | ✅ | ✅ | | paloalto\_panos\_brace\_to\_set | ✅ | ✅ | ✅ | | paloalto\_panos\_clean\_newlines | ✅ | ✅ | ✅ | | regex\_findall | ❌ | ❌ | ✅ | | regex\_match | ❌ | ❌ | ✅ | | regex\_search | ❌ | ❌ | ✅ | | regex\_split | ❌ | ❌ | ✅ | | regex\_sub | ❌ | ❌ | ✅ | | sanitize\_config | ✅ | ✅ | ✅ | | section\_config | ✅ | ✅ | ✅ | | sort\_interface\_list | ✅ | ✅ | ✅ | | split\_interface | ✅ | ✅ | ✅ | | uptime\_seconds\_to\_string | ✅ | ✅ | ✅ | | uptime\_string\_to\_seconds | ✅ | ✅ | ✅ | | version\_metadata | ✅ | ✅ | ✅ | | vlanconfig\_to\_list | ✅ | ✅ | ✅ | | vlanlist\_to\_config | ✅ | ✅ | ✅ | | wildcardmask\_to\_netmask | ✅ | ✅ | ✅ | ## Infrahub filters[​](#infrahub-filters "Direct link to Infrahub filters") These filters are provided by the Infrahub SDK for artifact and file object content composition. | Name | CORE | WORKER | LOCAL | | ----------------------------- | ---- | ------ | ----- | | `artifact_content` | ❌ | ✅ | ❌ | | `file_object_content` | ❌ | ✅ | ❌ | | `file_object_content_by_hfid` | ❌ | ✅ | ❌ | | `file_object_content_by_id` | ❌ | ✅ | ❌ | | `from_json` | ✅ | ✅ | ✅ | | `from_yaml` | ✅ | ✅ | ✅ | ### Usage examples[​](#usage-examples "Direct link to Usage examples") **Inline artifact content by `storage_id`:** ``` {{ artifact.node.storage_id.value | artifact_content }} ``` **Inline file object content:** ``` {# By storage_id #} {{ file_object.storage_id.value | file_object_content }} {# By node UUID #} {{ file_object.id | file_object_content_by_id }} {# By Human-Friendly ID #} {{ hfid_components | file_object_content_by_hfid(kind="NetworkCircuitContract") }} ``` **Parse structured content with chaining:** ``` {# JSON artifact → access parsed fields #} {% set config = artifact.node.storage_id.value | artifact_content | from_json %} interface {{ config.interface_name }} ip address {{ config.ip_address }} {# YAML artifact → iterate parsed data #} {% set config = artifact.node.storage_id.value | artifact_content | from_yaml %} {% for route in config.static_routes %} ip route {{ route.prefix }} {{ route.next_hop }} {% endfor %} ``` Client-dependent filters (`artifact_content`, `file_object_content`, `file_object_content_by_id`, `file_object_content_by_hfid`) require an `InfrahubClient` to be passed to `Jinja2Template`: ``` from infrahub_sdk.template import Jinja2Template # At construction time template = Jinja2Template(template=my_template, client=client) # Or via deferred injection template = Jinja2Template(template=my_template) template.set_client(client) ``` ## Known issues[​](#known-issues "Direct link to Known issues") ### Unable to combine the map and sort filters ()[​](#unable-to-combine-the-map-and-sort-filters-httpsgithubcompalletsjinjaissues2081 "Direct link to unable-to-combine-the-map-and-sort-filters-httpsgithubcompalletsjinjaissues2081") When using the `map` filter with the `sort` filter, you may encounter the following error: ``` TypeError: 'async_generator' object is not iterable ``` **As a workaround you can use the `list` filter between `map` and `sort` filter.** --- # `infrahub_sdk.client` ## Classes[​](#classes "Direct link to Classes") ### `InfrahubClient`[​](#infrahubclient "Direct link to infrahubclient") GraphQL Client to interact with Infrahub. **Methods:** #### `get`[​](#get "Direct link to get") ``` get(self, kind: type[SchemaType], raise_when_missing: Literal[False], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> SchemaType | None ``` Show 6 other overloads #### `get`[​](#get-1 "Direct link to get-1") ``` get(self, kind: type[SchemaType], raise_when_missing: Literal[True], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> SchemaType ``` #### `get`[​](#get-2 "Direct link to get-2") ``` get(self, kind: type[SchemaType], raise_when_missing: bool = ..., at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> SchemaType ``` #### `get`[​](#get-3 "Direct link to get-3") ``` get(self, kind: str, raise_when_missing: Literal[False], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> InfrahubNode | None ``` #### `get`[​](#get-4 "Direct link to get-4") ``` get(self, kind: str, raise_when_missing: Literal[True], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> InfrahubNode ``` #### `get`[​](#get-5 "Direct link to get-5") ``` get(self, kind: str, raise_when_missing: bool = ..., at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> InfrahubNode ``` #### `get`[​](#get-6 "Direct link to get-6") ``` get(self, kind: str | type[SchemaType], raise_when_missing: bool = True, at: Timestamp | None = None, branch: str | None = None, timeout: int | None = None, id: str | None = None, hfid: list[str] | None = None, include: list[str] | None = None, exclude: list[str] | None = None, populate_store: bool = True, fragment: bool = False, prefetch_relationships: bool = False, property: bool = False, include_metadata: bool = False, query_name: str | None = None, priority: Priority | None = None, **kwargs: Any) -> InfrahubNode | SchemaType | None ``` #### `delete`[​](#delete "Direct link to delete") ``` delete(self, kind: str | type[SchemaType], id: str, branch: str | None = None) -> None ``` #### `create`[​](#create "Direct link to create") ``` create(self, kind: str | type[SchemaType], data: dict | None = None, branch: str | None = None, timeout: int | None = None, **kwargs: Any) -> InfrahubNode | SchemaType ``` Show 2 other overloads #### `create`[​](#create-1 "Direct link to create-1") ``` create(self, kind: str, data: dict | None = ..., branch: str | None = ..., **kwargs: Any) -> InfrahubNode ``` #### `create`[​](#create-2 "Direct link to create-2") ``` create(self, kind: type[SchemaType], data: dict | None = ..., branch: str | None = ..., **kwargs: Any) -> SchemaType ``` #### `get_version`[​](#get_version "Direct link to get_version") ``` get_version(self) -> str ``` Return the Infrahub version. #### `get_server_information`[​](#get_server_information "Direct link to get_server_information") ``` get_server_information(self) -> ServerInfo ``` Return the Infrahub server information (version and deployment ID). #### `get_user`[​](#get_user "Direct link to get_user") ``` get_user(self) -> dict ``` Return user information. #### `get_user_permissions`[​](#get_user_permissions "Direct link to get_user_permissions") ``` get_user_permissions(self) -> dict ``` Return user permissions. #### `count`[​](#count "Direct link to count") ``` count(self, kind: str | type[SchemaType], at: Timestamp | None = None, branch: str | None = None, timeout: int | None = None, partial_match: bool = False, query_name: str | None = None, priority: Priority | None = None, **kwargs: Any) -> int ``` Return the number of nodes of a given kind. #### `traverse_paths`[​](#traverse_paths "Direct link to traverse_paths") ``` traverse_paths(self, source: str | InfrahubNode, destination: str | InfrahubNode) -> PathTraversalResult ``` Find the shortest path(s) between two nodes in the graph. Kind filters (`kind_filter`, `excluded_kinds`, `included_kinds`) accept kind-name strings and/or generated protocol classes. `relationship_filter` matches schema relationship identifiers (for example `dcimconnector__dcimendpoint`), not the per-side names shown in the result. Requires Infrahub 1.10 or later. **Args:** * `source`: Node to start from, as a UUID string or an `InfrahubNode` instance. * `destination`: Node to reach, as a UUID string or an `InfrahubNode` instance. * `max_depth`: Maximum number of relationship hops to explore. * `max_paths`: Maximum number of paths to return. * `kind_filter`: Only traverse through nodes of these kinds. * `relationship_filter`: Only traverse through these schema relationship identifiers. * `excluded_namespaces`: Schema namespaces to exclude from traversal. * `excluded_kinds`: Node kinds to exclude from traversal. * `included_kinds`: Node kinds to re-include when otherwise excluded by default. * `shortest_paths_only`: When True (the server default), only return the shortest path(s); when False, return all loopless paths (exhaustive mode). * `branch`: Name of the branch to query from. Defaults to default\_branch. * `at`: Time of the query. Defaults to now. * `timeout`: Overrides the default GraphQL timeout, in seconds. **Raises:** * `VersionNotSupportedError`: If the server does not support graph traversal (pre-1.10). * `GraphQLError`: When the GraphQL response contains errors (e.g. unknown node). #### `path_exists`[​](#path_exists "Direct link to path_exists") ``` path_exists(self, source: str | InfrahubNode, destination: str | InfrahubNode) -> bool ``` Return whether at least one path connects `source` to `destination`. Convenience wrapper around :meth:`traverse_paths` for checks: it requests a single path (the cheapest way to answer "is there a path?") and returns `True` if one was found. Accepts the same source/destination and filter arguments as `traverse_paths`. Requires Infrahub 1.10 or later. **Args:** * `source`: Node to start from, as a UUID string or an `InfrahubNode` instance. * `destination`: Node to reach, as a UUID string or an `InfrahubNode` instance. * `max_depth`: Maximum number of relationship hops to explore. * `kind_filter`: Only traverse through nodes of these kinds. * `relationship_filter`: Only traverse through these schema relationship identifiers. * `excluded_namespaces`: Schema namespaces to exclude from traversal. * `excluded_kinds`: Node kinds to exclude from traversal. * `included_kinds`: Node kinds to re-include when otherwise excluded by default. * `branch`: Name of the branch to query from. Defaults to default\_branch. * `at`: Time of the query. Defaults to now. * `timeout`: Overrides the default GraphQL timeout, in seconds. **Raises:** * `VersionNotSupportedError`: If the server does not support graph traversal (pre-1.10). * `GraphQLError`: When the GraphQL response contains errors (e.g. unknown node). #### `reachable_nodes`[​](#reachable_nodes "Direct link to reachable_nodes") ``` reachable_nodes(self, source: str | InfrahubNode, target_kinds: list[str | type[SchemaType]]) -> ReachableNodesResult ``` Find all nodes of the given kinds reachable from a source node. `target_kinds` accepts kind-name strings and/or generated protocol classes. Requires Infrahub 1.10 or later. **Args:** * `source`: Node to start from, as a UUID string or an `InfrahubNode` instance. * `target_kinds`: Kinds of nodes to look for, as kind-name strings or protocol classes. * `max_depth`: Maximum number of relationship hops to explore. * `max_results`: Maximum number of reachable nodes to return. * `max_paths`: Maximum number of paths to compute per reachable node. * `shortest_paths_only`: When True, only return the shortest path(s) to each node. * `branch`: Name of the branch to query from. Defaults to default\_branch. * `at`: Time of the query. Defaults to now. * `timeout`: Overrides the default GraphQL timeout, in seconds. **Raises:** * `VersionNotSupportedError`: If the server does not support graph traversal (pre-1.10). * `GraphQLError`: When the GraphQL response contains errors (e.g. unknown node). #### `all`[​](#all "Direct link to all") ``` all(self, kind: type[SchemaType], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., populate_store: bool = ..., offset: int | None = ..., limit: int | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., parallel: bool = ..., order: Order | None = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ...) -> list[SchemaType] ``` Show 2 other overloads #### `all`[​](#all-1 "Direct link to all-1") ``` all(self, kind: str, at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., populate_store: bool = ..., offset: int | None = ..., limit: int | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., parallel: bool = ..., order: Order | None = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ...) -> list[InfrahubNode] ``` #### `all`[​](#all-2 "Direct link to all-2") ``` all(self, kind: str | type[SchemaType], at: Timestamp | None = None, branch: str | None = None, timeout: int | None = None, populate_store: bool = True, offset: int | None = None, limit: int | None = None, include: list[str] | None = None, exclude: list[str] | None = None, fragment: bool = False, prefetch_relationships: bool = False, property: bool = False, parallel: bool = False, order: Order | None = None, include_metadata: bool = False, query_name: str | None = None, priority: Priority | None = None) -> list[InfrahubNode] | list[SchemaType] ``` Retrieve all nodes of a given kind. **Args:** * `kind`: kind of the nodes to query * `at`: Time of the query. Defaults to Now. * `branch`: Name of the branch to query from. Defaults to default\_branch. * `populate_store`: Flag to indicate whether to populate the store with the retrieved nodes. * `timeout`: Overrides default timeout used when querying the GraphQL API. Specified in seconds. * `offset`: The offset for pagination. * `limit`: The limit for pagination. * `include`: List of attributes or relationships to include in the query. * `exclude`: List of attributes or relationships to exclude from the query. * `fragment`: Flag to use GraphQL fragments for generic schemas. * `prefetch_relationships`: Flag to indicate whether to pre-fetch related node data. * `parallel`: Whether to use parallel processing for the query. * `order`: Ordering related options. Setting `disable=True` enhances performances. * `include_metadata`: If True, includes node\_metadata and relationship\_metadata in the query. * `query_name`: If provided is used as the GraphQL operation name else All\_\ is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for these requests only. When None, the client default (if any) is used. **Returns:** * list\[InfrahubNode]: List of Nodes #### `filters`[​](#filters "Direct link to filters") ``` filters(self, kind: type[SchemaType], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., populate_store: bool = ..., offset: int | None = ..., limit: int | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., fragment: bool = ..., prefetch_relationships: bool = ..., partial_match: bool = ..., property: bool = ..., parallel: bool = ..., order: Order | None = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> list[SchemaType] ``` Show 2 other overloads #### `filters`[​](#filters-1 "Direct link to filters-1") ``` filters(self, kind: str, at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., populate_store: bool = ..., offset: int | None = ..., limit: int | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., fragment: bool = ..., prefetch_relationships: bool = ..., partial_match: bool = ..., property: bool = ..., parallel: bool = ..., order: Order | None = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> list[InfrahubNode] ``` #### `filters`[​](#filters-2 "Direct link to filters-2") ``` filters(self, kind: str | type[SchemaType], at: Timestamp | None = None, branch: str | None = None, timeout: int | None = None, populate_store: bool = True, offset: int | None = None, limit: int | None = None, include: list[str] | None = None, exclude: list[str] | None = None, fragment: bool = False, prefetch_relationships: bool = False, partial_match: bool = False, property: bool = False, parallel: bool = False, order: Order | None = None, include_metadata: bool = False, query_name: str | None = None, priority: Priority | None = None, **kwargs: Any) -> list[InfrahubNode] | list[SchemaType] ``` Retrieve nodes of a given kind based on provided filters. **Args:** * `kind`: kind of the nodes to query * `at`: Time of the query. Defaults to Now. * `branch`: Name of the branch to query from. Defaults to default\_branch. * `timeout`: Overrides default timeout used when querying the GraphQL API. Specified in seconds. * `populate_store`: Flag to indicate whether to populate the store with the retrieved nodes. * `offset`: The offset for pagination. * `limit`: The limit for pagination. * `include`: List of attributes or relationships to include in the query. * `exclude`: List of attributes or relationships to exclude from the query. * `fragment`: Flag to use GraphQL fragments for generic schemas. * `prefetch_relationships`: Flag to indicate whether to pre-fetch related node data. * `partial_match`: Allow partial match of filter criteria for the query. * `parallel`: Whether to use parallel processing for the query. * `order`: Ordering related options. Setting `disable=True` enhances performances. * `include_metadata`: If True, includes node\_metadata and relationship\_metadata in the query. * `query_name`: If provided is used as the GraphQL operation name else Filters\_\ is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for these requests only. When None, the client default (if any) is used. * `**kwargs`: Additional filter criteria for the query. **Returns:** * list\[InfrahubNode]: List of Nodes that match the given filters. #### `clone`[​](#clone "Direct link to clone") ``` clone(self, branch: str | None = None) -> InfrahubClient ``` Return a cloned version of the client using the same configuration. #### `execute_graphql`[​](#execute_graphql "Direct link to execute_graphql") ``` execute_graphql(self, query: str, variables: dict | None = None, branch_name: str | None = None, at: str | Timestamp | None = None, timeout: int | None = None, tracker: str | None = None, operation_name: str | None = None, priority: Priority | None = None) -> dict ``` Execute a GraphQL query (or mutation). If retry\_on\_failure is True, transient failures (connection errors, timeouts, transient HTTP statuses and GraphQL errors the server flags as transient) are retried until max\_retry\_duration is exhausted, or indefinitely when max\_retry\_duration is 0. **Args:** * `query`: GraphQL Query to execute, can be a query or a mutation * `variables`: Variables to pass along with the GraphQL query. Defaults to None. * `branch_name`: Name of the branch on which the query will be executed. Defaults to None. * `at`: Time when the query should be executed. Defaults to None. * `timeout`: Timeout in second for the query. Defaults to None. * `operation_name`: GraphQL operation name, sent as `operationName` in the request payload so tracing/observability tools can identify the operation. Defaults to None. * `priority`: Per-request priority emitted as the X-Priority header. Overrides the client-wide default for this request only. When None, the client default (if any) is used. **Returns:** * The GraphQL data payload (response\["data"]). **Raises:** * `GraphQLError`: When the GraphQL response contains errors that are not transient, or transient ones once the retry budget is exhausted. * `ServerNotReachableError`: If the server is not reachable, after exhausting retries when enabled. * `ServerNotResponsiveError`: If the server does not answer before the timeout, after exhausting retries. * `AuthenticationError`: If the server returns a 401 or 403 response. * `URLNotFoundError`: If the server returns a 404 response. #### `refresh_login`[​](#refresh_login "Direct link to refresh_login") ``` refresh_login(self) -> None ``` #### `login`[​](#login "Direct link to login") ``` login(self, refresh: bool = False) -> None ``` #### `query_gql_query`[​](#query_gql_query "Direct link to query_gql_query") ``` query_gql_query(self, name: str, variables: dict | None = None, update_group: bool = False, subscribers: list[str] | None = None, params: dict | None = None, branch_name: str | None = None, at: str | None = None, timeout: int | None = None, tracker: str | None = None) -> dict ``` #### `create_diff`[​](#create_diff "Direct link to create_diff") ``` create_diff(self, branch: str, name: str, from_time: datetime, to_time: datetime, wait_until_completion: bool = True, priority: Priority | None = None) -> bool | str ``` #### `get_diff_summary`[​](#get_diff_summary "Direct link to get_diff_summary") ``` get_diff_summary(self, branch: str, name: str | None = None, from_time: datetime | None = None, to_time: datetime | None = None, timeout: int | None = None, tracker: str | None = None, priority: Priority | None = None) -> list[NodeDiff] ``` #### `get_diff_tree`[​](#get_diff_tree "Direct link to get_diff_tree") ``` get_diff_tree(self, branch: str, name: str | None = None, from_time: datetime | None = None, to_time: datetime | None = None, timeout: int | None = None, tracker: str | None = None, priority: Priority | None = None) -> DiffTreeData | None ``` Get complete diff tree with metadata and nodes. Returns None if no diff exists. **Raises:** * `ValueError`: If `from_time` is later than `to_time`. #### `allocate_next_ip_address`[​](#allocate_next_ip_address "Direct link to allocate_next_ip_address") ``` allocate_next_ip_address(self, resource_pool: CoreNode, kind: type[SchemaType], identifier: str | None = ..., prefix_length: int | None = ..., address_type: str | None = ..., data: dict[str, Any] | None = ..., branch: str | None = ..., timeout: int | None = ..., tracker: str | None = ...) -> SchemaType | None ``` Show 2 other overloads #### `allocate_next_ip_address`[​](#allocate_next_ip_address-1 "Direct link to allocate_next_ip_address-1") ``` allocate_next_ip_address(self, resource_pool: CoreNode, kind: None = ..., identifier: str | None = ..., prefix_length: int | None = ..., address_type: str | None = ..., data: dict[str, Any] | None = ..., branch: str | None = ..., timeout: int | None = ..., tracker: str | None = ...) -> CoreNode | None ``` #### `allocate_next_ip_address`[​](#allocate_next_ip_address-2 "Direct link to allocate_next_ip_address-2") ``` allocate_next_ip_address(self, resource_pool: CoreNode, kind: type[SchemaType] | None = None, identifier: str | None = None, prefix_length: int | None = None, address_type: str | None = None, data: dict[str, Any] | None = None, branch: str | None = None, timeout: int | None = None, tracker: str | None = None) -> CoreNode | SchemaType | None ``` Allocate a new IP address by using the provided resource pool. **Args:** * `resource_pool`: Node corresponding to the pool to allocate resources from. * `identifier`: Value to perform idempotent allocation, the same resource will be returned for a given identifier. * `prefix_length`: Length of the prefix to set on the address to allocate. * `address_type`: Kind of the address to allocate. * `data`: A key/value map to use to set attributes values on the allocated address. * `branch`: Name of the branch to allocate from. Defaults to default\_branch. * `timeout`: Flag to indicate whether to populate the store with the retrieved nodes. * `tracker`: The offset for pagination. **Returns:** * Node corresponding to the allocated resource. **Raises:** * `ValueError`: If `resource_pool` is not a `CoreIPAddressPool`. #### `allocate_next_ip_prefix`[​](#allocate_next_ip_prefix "Direct link to allocate_next_ip_prefix") ``` allocate_next_ip_prefix(self, resource_pool: CoreNode, kind: type[SchemaType], identifier: str | None = ..., prefix_length: int | None = ..., member_type: str | None = ..., prefix_type: str | None = ..., data: dict[str, Any] | None = ..., branch: str | None = ..., timeout: int | None = ..., tracker: str | None = ...) -> SchemaType | None ``` Show 2 other overloads #### `allocate_next_ip_prefix`[​](#allocate_next_ip_prefix-1 "Direct link to allocate_next_ip_prefix-1") ``` allocate_next_ip_prefix(self, resource_pool: CoreNode, kind: None = ..., identifier: str | None = ..., prefix_length: int | None = ..., member_type: str | None = ..., prefix_type: str | None = ..., data: dict[str, Any] | None = ..., branch: str | None = ..., timeout: int | None = ..., tracker: str | None = ...) -> CoreNode | None ``` #### `allocate_next_ip_prefix`[​](#allocate_next_ip_prefix-2 "Direct link to allocate_next_ip_prefix-2") ``` allocate_next_ip_prefix(self, resource_pool: CoreNode, kind: type[SchemaType] | None = None, identifier: str | None = None, prefix_length: int | None = None, member_type: str | None = None, prefix_type: str | None = None, data: dict[str, Any] | None = None, branch: str | None = None, timeout: int | None = None, tracker: str | None = None) -> CoreNode | SchemaType | None ``` Allocate a new IP prefix by using the provided resource pool. **Args:** * `resource_pool`: Node corresponding to the pool to allocate resources from. * `identifier`: Value to perform idempotent allocation, the same resource will be returned for a given identifier. * `prefix_length`: Length of the prefix to allocate. * `member_type`: Member type of the prefix to allocate. * `prefix_type`: Kind of the prefix to allocate. * `data`: A key/value map to use to set attributes values on the allocated prefix. * `branch`: Name of the branch to allocate from. Defaults to default\_branch. * `timeout`: Flag to indicate whether to populate the store with the retrieved nodes. * `tracker`: The offset for pagination. **Returns:** * Node corresponding to the allocated resource. **Raises:** * `ValueError`: If `resource_pool` is not a `CoreIPPrefixPool`. #### `create_batch`[​](#create_batch "Direct link to create_batch") ``` create_batch(self, return_exceptions: bool = False) -> InfrahubBatch ``` #### `get_list_repositories`[​](#get_list_repositories "Direct link to get_list_repositories") ``` get_list_repositories(self, branches: dict[str, BranchData] | None = None, kind: str = 'CoreGenericRepository') -> dict[str, RepositoryData] ``` #### `repository_update_commit`[​](#repository_update_commit "Direct link to repository_update_commit") ``` repository_update_commit(self, branch_name: str, repository_id: str, commit: str, is_read_only: bool = False) -> bool ``` #### `convert_object_type`[​](#convert_object_type "Direct link to convert_object_type") ``` convert_object_type(self, node_id: str, target_kind: str, branch: str | None = None, fields_mapping: dict[str, ConversionFieldInput] | None = None) -> InfrahubNode ``` Convert a given node to another kind on a given branch. `fields_mapping` keys are target fields names and its values indicate how to fill in these fields. Any mandatory field not having an equivalent field in the source kind should be specified in this mapping. See for more information. ### `InfrahubClientSync`[​](#infrahubclientsync "Direct link to infrahubclientsync") **Methods:** #### `get`[​](#get-7 "Direct link to get-7") ``` get(self, kind: type[SchemaTypeSync], raise_when_missing: Literal[False], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> SchemaTypeSync | None ``` Show 6 other overloads #### `get`[​](#get-8 "Direct link to get-8") ``` get(self, kind: type[SchemaTypeSync], raise_when_missing: Literal[True], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> SchemaTypeSync ``` #### `get`[​](#get-9 "Direct link to get-9") ``` get(self, kind: type[SchemaTypeSync], raise_when_missing: bool = ..., at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> SchemaTypeSync ``` #### `get`[​](#get-10 "Direct link to get-10") ``` get(self, kind: str, raise_when_missing: Literal[False], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> InfrahubNodeSync | None ``` #### `get`[​](#get-11 "Direct link to get-11") ``` get(self, kind: str, raise_when_missing: Literal[True], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> InfrahubNodeSync ``` #### `get`[​](#get-12 "Direct link to get-12") ``` get(self, kind: str, raise_when_missing: bool = ..., at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., id: str | None = ..., hfid: list[str] | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., populate_store: bool = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> InfrahubNodeSync ``` #### `get`[​](#get-13 "Direct link to get-13") ``` get(self, kind: str | type[SchemaTypeSync], raise_when_missing: bool = True, at: Timestamp | None = None, branch: str | None = None, timeout: int | None = None, id: str | None = None, hfid: list[str] | None = None, include: list[str] | None = None, exclude: list[str] | None = None, populate_store: bool = True, fragment: bool = False, prefetch_relationships: bool = False, property: bool = False, include_metadata: bool = False, query_name: str | None = None, priority: Priority | None = None, **kwargs: Any) -> InfrahubNodeSync | SchemaTypeSync | None ``` #### `delete`[​](#delete-1 "Direct link to delete-1") ``` delete(self, kind: str | type[SchemaTypeSync], id: str, branch: str | None = None) -> None ``` #### `create`[​](#create-3 "Direct link to create-3") ``` create(self, kind: str | type[SchemaTypeSync], data: dict | None = None, branch: str | None = None, timeout: int | None = None, **kwargs: Any) -> InfrahubNodeSync | SchemaTypeSync ``` Show 2 other overloads #### `create`[​](#create-4 "Direct link to create-4") ``` create(self, kind: str, data: dict | None = ..., branch: str | None = ..., **kwargs: Any) -> InfrahubNodeSync ``` #### `create`[​](#create-5 "Direct link to create-5") ``` create(self, kind: type[SchemaTypeSync], data: dict | None = ..., branch: str | None = ..., **kwargs: Any) -> SchemaTypeSync ``` #### `get_version`[​](#get_version-1 "Direct link to get_version-1") ``` get_version(self) -> str ``` Return the Infrahub version. #### `get_server_information`[​](#get_server_information-1 "Direct link to get_server_information-1") ``` get_server_information(self) -> ServerInfo ``` Return the Infrahub server information (version and deployment ID). #### `get_user`[​](#get_user-1 "Direct link to get_user-1") ``` get_user(self) -> dict ``` Return user information. #### `get_user_permissions`[​](#get_user_permissions-1 "Direct link to get_user_permissions-1") ``` get_user_permissions(self) -> dict ``` Return user permissions. #### `clone`[​](#clone-1 "Direct link to clone-1") ``` clone(self, branch: str | None = None) -> InfrahubClientSync ``` Return a cloned version of the client using the same configuration. #### `execute_graphql`[​](#execute_graphql-1 "Direct link to execute_graphql-1") ``` execute_graphql(self, query: str, variables: dict | None = None, branch_name: str | None = None, at: str | Timestamp | None = None, timeout: int | None = None, tracker: str | None = None, operation_name: str | None = None, priority: Priority | None = None) -> dict ``` Execute a GraphQL query (or mutation). If retry\_on\_failure is True, transient failures (connection errors, timeouts, transient HTTP statuses and GraphQL errors the server flags as transient) are retried until max\_retry\_duration is exhausted, or indefinitely when max\_retry\_duration is 0. **Args:** * `query`: GraphQL Query to execute, can be a query or a mutation * `variables`: Variables to pass along with the GraphQL query. Defaults to None. * `branch_name`: Name of the branch on which the query will be executed. Defaults to None. * `at`: Time when the query should be executed. Defaults to None. * `timeout`: Timeout in second for the query. Defaults to None. * `operation_name`: GraphQL operation name, sent as `operationName` in the request payload so tracing/observability tools can identify the operation. Defaults to None. * `priority`: Per-request priority emitted as the X-Priority header. Overrides the client-wide default for this request only. When None, the client default (if any) is used. **Returns:** * The GraphQL data payload (`response["data"]`). **Raises:** * `GraphQLError`: When the GraphQL response contains errors that are not transient, or transient ones once the retry budget is exhausted. * `ServerNotReachableError`: If the server is not reachable, after exhausting retries when enabled. * `ServerNotResponsiveError`: If the server does not answer before the timeout, after exhausting retries. * `AuthenticationError`: If the server returns a 401 or 403 response. * `URLNotFoundError`: If the server returns a 404 response. #### `count`[​](#count-1 "Direct link to count-1") ``` count(self, kind: str | type[SchemaType], at: Timestamp | None = None, branch: str | None = None, timeout: int | None = None, partial_match: bool = False, query_name: str | None = None, priority: Priority | None = None, **kwargs: Any) -> int ``` Return the number of nodes of a given kind. #### `traverse_paths`[​](#traverse_paths-1 "Direct link to traverse_paths-1") ``` traverse_paths(self, source: str | InfrahubNodeSync, destination: str | InfrahubNodeSync) -> PathTraversalResult ``` Find the shortest path(s) between two nodes in the graph. Kind filters (`kind_filter`, `excluded_kinds`, `included_kinds`) accept kind-name strings and/or generated protocol classes. `relationship_filter` matches schema relationship identifiers (for example `dcimconnector__dcimendpoint`), not the per-side names shown in the result. Requires Infrahub 1.10 or later. **Args:** * `source`: Node to start from, as a UUID string or an `InfrahubNode` instance. * `destination`: Node to reach, as a UUID string or an `InfrahubNode` instance. * `max_depth`: Maximum number of relationship hops to explore. * `max_paths`: Maximum number of paths to return. * `kind_filter`: Only traverse through nodes of these kinds. * `relationship_filter`: Only traverse through these schema relationship identifiers. * `excluded_namespaces`: Schema namespaces to exclude from traversal. * `excluded_kinds`: Node kinds to exclude from traversal. * `included_kinds`: Node kinds to re-include when otherwise excluded by default. * `shortest_paths_only`: When True (the server default), only return the shortest path(s); when False, return all loopless paths (exhaustive mode). * `branch`: Name of the branch to query from. Defaults to default\_branch. * `at`: Time of the query. Defaults to now. * `timeout`: Overrides the default GraphQL timeout, in seconds. **Raises:** * `VersionNotSupportedError`: If the server does not support graph traversal (pre-1.10). * `GraphQLError`: When the GraphQL response contains errors (e.g. unknown node). #### `path_exists`[​](#path_exists-1 "Direct link to path_exists-1") ``` path_exists(self, source: str | InfrahubNodeSync, destination: str | InfrahubNodeSync) -> bool ``` Return whether at least one path connects `source` to `destination`. Convenience wrapper around :meth:`traverse_paths` for checks: it requests a single path (the cheapest way to answer "is there a path?") and returns `True` if one was found. Accepts the same source/destination and filter arguments as `traverse_paths`. Requires Infrahub 1.10 or later. **Args:** * `source`: Node to start from, as a UUID string or an `InfrahubNode` instance. * `destination`: Node to reach, as a UUID string or an `InfrahubNode` instance. * `max_depth`: Maximum number of relationship hops to explore. * `kind_filter`: Only traverse through nodes of these kinds. * `relationship_filter`: Only traverse through these schema relationship identifiers. * `excluded_namespaces`: Schema namespaces to exclude from traversal. * `excluded_kinds`: Node kinds to exclude from traversal. * `included_kinds`: Node kinds to re-include when otherwise excluded by default. * `branch`: Name of the branch to query from. Defaults to default\_branch. * `at`: Time of the query. Defaults to now. * `timeout`: Overrides the default GraphQL timeout, in seconds. **Raises:** * `VersionNotSupportedError`: If the server does not support graph traversal (pre-1.10). * `GraphQLError`: When the GraphQL response contains errors (e.g. unknown node). #### `reachable_nodes`[​](#reachable_nodes-1 "Direct link to reachable_nodes-1") ``` reachable_nodes(self, source: str | InfrahubNodeSync, target_kinds: list[str | type[SchemaTypeSync]]) -> ReachableNodesResult ``` Find all nodes of the given kinds reachable from a source node. `target_kinds` accepts kind-name strings and/or generated protocol classes. Requires Infrahub 1.10 or later. **Args:** * `source`: Node to start from, as a UUID string or an `InfrahubNode` instance. * `target_kinds`: Kinds of nodes to look for, as kind-name strings or protocol classes. * `max_depth`: Maximum number of relationship hops to explore. * `max_results`: Maximum number of reachable nodes to return. * `max_paths`: Maximum number of paths to compute per reachable node. * `shortest_paths_only`: When True, only return the shortest path(s) to each node. * `branch`: Name of the branch to query from. Defaults to default\_branch. * `at`: Time of the query. Defaults to now. * `timeout`: Overrides the default GraphQL timeout, in seconds. **Raises:** * `VersionNotSupportedError`: If the server does not support graph traversal (pre-1.10). * `GraphQLError`: When the GraphQL response contains errors (e.g. unknown node). #### `all`[​](#all-3 "Direct link to all-3") ``` all(self, kind: type[SchemaTypeSync], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., populate_store: bool = ..., offset: int | None = ..., limit: int | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., parallel: bool = ..., order: Order | None = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ...) -> list[SchemaTypeSync] ``` Show 2 other overloads #### `all`[​](#all-4 "Direct link to all-4") ``` all(self, kind: str, at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., populate_store: bool = ..., offset: int | None = ..., limit: int | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., fragment: bool = ..., prefetch_relationships: bool = ..., property: bool = ..., parallel: bool = ..., order: Order | None = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ...) -> list[InfrahubNodeSync] ``` #### `all`[​](#all-5 "Direct link to all-5") ``` all(self, kind: str | type[SchemaTypeSync], at: Timestamp | None = None, branch: str | None = None, timeout: int | None = None, populate_store: bool = True, offset: int | None = None, limit: int | None = None, include: list[str] | None = None, exclude: list[str] | None = None, fragment: bool = False, prefetch_relationships: bool = False, property: bool = False, parallel: bool = False, order: Order | None = None, include_metadata: bool = False, query_name: str | None = None, priority: Priority | None = None) -> list[InfrahubNodeSync] | list[SchemaTypeSync] ``` Retrieve all nodes of a given kind. **Args:** * `kind`: kind of the nodes to query * `at`: Time of the query. Defaults to Now. * `branch`: Name of the branch to query from. Defaults to default\_branch. * `timeout`: Overrides default timeout used when querying the GraphQL API. Specified in seconds. * `populate_store`: Flag to indicate whether to populate the store with the retrieved nodes. * `offset`: The offset for pagination. * `limit`: The limit for pagination. * `include`: List of attributes or relationships to include in the query. * `exclude`: List of attributes or relationships to exclude from the query. * `fragment`: Flag to use GraphQL fragments for generic schemas. * `prefetch_relationships`: Flag to indicate whether to pre-fetch related node data. * `parallel`: Whether to use parallel processing for the query. * `order`: Ordering related options. Setting `disable=True` enhances performances. * `include_metadata`: If True, includes node\_metadata and relationship\_metadata in the query. * `query_name`: If provided is used as the GraphQL operation name else All\_\ is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for these requests only. When None, the client default (if any) is used. **Returns:** * list\[InfrahubNodeSync]: List of Nodes #### `filters`[​](#filters-3 "Direct link to filters-3") ``` filters(self, kind: type[SchemaTypeSync], at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., populate_store: bool = ..., offset: int | None = ..., limit: int | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., fragment: bool = ..., prefetch_relationships: bool = ..., partial_match: bool = ..., property: bool = ..., parallel: bool = ..., order: Order | None = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> list[SchemaTypeSync] ``` Show 2 other overloads #### `filters`[​](#filters-4 "Direct link to filters-4") ``` filters(self, kind: str, at: Timestamp | None = ..., branch: str | None = ..., timeout: int | None = ..., populate_store: bool = ..., offset: int | None = ..., limit: int | None = ..., include: list[str] | None = ..., exclude: list[str] | None = ..., fragment: bool = ..., prefetch_relationships: bool = ..., partial_match: bool = ..., property: bool = ..., parallel: bool = ..., order: Order | None = ..., include_metadata: bool = ..., query_name: str | None = ..., priority: Priority | None = ..., **kwargs: Any) -> list[InfrahubNodeSync] ``` #### `filters`[​](#filters-5 "Direct link to filters-5") ``` filters(self, kind: str | type[SchemaTypeSync], at: Timestamp | None = None, branch: str | None = None, timeout: int | None = None, populate_store: bool = True, offset: int | None = None, limit: int | None = None, include: list[str] | None = None, exclude: list[str] | None = None, fragment: bool = False, prefetch_relationships: bool = False, partial_match: bool = False, property: bool = False, parallel: bool = False, order: Order | None = None, include_metadata: bool = False, query_name: str | None = None, priority: Priority | None = None, **kwargs: Any) -> list[InfrahubNodeSync] | list[SchemaTypeSync] ``` Retrieve nodes of a given kind based on provided filters. **Args:** * `kind`: kind of the nodes to query * `at`: Time of the query. Defaults to Now. * `branch`: Name of the branch to query from. Defaults to default\_branch. * `timeout`: Overrides default timeout used when querying the GraphQL API. Specified in seconds. * `populate_store`: Flag to indicate whether to populate the store with the retrieved nodes. * `offset`: The offset for pagination. * `limit`: The limit for pagination. * `include`: List of attributes or relationships to include in the query. * `exclude`: List of attributes or relationships to exclude from the query. * `fragment`: Flag to use GraphQL fragments for generic schemas. * `prefetch_relationships`: Flag to indicate whether to pre-fetch related node data. * `partial_match`: Allow partial match of filter criteria for the query. * `parallel`: Whether to use parallel processing for the query. * `order`: Ordering related options. Setting `disable=True` enhances performances. * `include_metadata`: If True, includes node\_metadata and relationship\_metadata in the query. * `query_name`: If provided is used as the GraphQL operation name else Filters\_\ is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for these requests only. When None, the client default (if any) is used. * `**kwargs`: Additional filter criteria for the query. **Returns:** * list\[InfrahubNodeSync]: List of Nodes that match the given filters. #### `create_batch`[​](#create_batch-1 "Direct link to create_batch-1") ``` create_batch(self, return_exceptions: bool = False) -> InfrahubBatchSync ``` Create a batch to execute multiple queries concurrently. Executing the batch will be performed using a thread pool, meaning it cannot guarantee the execution order. It is not recommended to use such batch to manipulate objects that depend on each others. #### `get_list_repositories`[​](#get_list_repositories-1 "Direct link to get_list_repositories-1") ``` get_list_repositories(self, branches: dict[str, BranchData] | None = None, kind: str = 'CoreGenericRepository') -> dict[str, RepositoryData] ``` #### `query_gql_query`[​](#query_gql_query-1 "Direct link to query_gql_query-1") ``` query_gql_query(self, name: str, variables: dict | None = None, update_group: bool = False, subscribers: list[str] | None = None, params: dict | None = None, branch_name: str | None = None, at: str | None = None, timeout: int | None = None, tracker: str | None = None) -> dict ``` #### `create_diff`[​](#create_diff-1 "Direct link to create_diff-1") ``` create_diff(self, branch: str, name: str, from_time: datetime, to_time: datetime, wait_until_completion: bool = True, priority: Priority | None = None) -> bool | str ``` #### `get_diff_summary`[​](#get_diff_summary-1 "Direct link to get_diff_summary-1") ``` get_diff_summary(self, branch: str, name: str | None = None, from_time: datetime | None = None, to_time: datetime | None = None, timeout: int | None = None, tracker: str | None = None, priority: Priority | None = None) -> list[NodeDiff] ``` #### `get_diff_tree`[​](#get_diff_tree-1 "Direct link to get_diff_tree-1") ``` get_diff_tree(self, branch: str, name: str | None = None, from_time: datetime | None = None, to_time: datetime | None = None, timeout: int | None = None, tracker: str | None = None, priority: Priority | None = None) -> DiffTreeData | None ``` Get complete diff tree with metadata and nodes. Returns None if no diff exists. **Raises:** * `ValueError`: If `from_time` is later than `to_time`. #### `allocate_next_ip_address`[​](#allocate_next_ip_address-3 "Direct link to allocate_next_ip_address-3") ``` allocate_next_ip_address(self, resource_pool: CoreNodeSync, kind: type[SchemaTypeSync], identifier: str | None = ..., prefix_length: int | None = ..., address_type: str | None = ..., data: dict[str, Any] | None = ..., branch: str | None = ..., timeout: int | None = ..., tracker: str | None = ...) -> SchemaTypeSync | None ``` Show 2 other overloads #### `allocate_next_ip_address`[​](#allocate_next_ip_address-4 "Direct link to allocate_next_ip_address-4") ``` allocate_next_ip_address(self, resource_pool: CoreNodeSync, kind: None = ..., identifier: str | None = ..., prefix_length: int | None = ..., address_type: str | None = ..., data: dict[str, Any] | None = ..., branch: str | None = ..., timeout: int | None = ..., tracker: str | None = ...) -> CoreNodeSync | None ``` #### `allocate_next_ip_address`[​](#allocate_next_ip_address-5 "Direct link to allocate_next_ip_address-5") ``` allocate_next_ip_address(self, resource_pool: CoreNodeSync, kind: type[SchemaTypeSync] | None = None, identifier: str | None = None, prefix_length: int | None = None, address_type: str | None = None, data: dict[str, Any] | None = None, branch: str | None = None, timeout: int | None = None, tracker: str | None = None) -> CoreNodeSync | SchemaTypeSync | None ``` Allocate a new IP address by using the provided resource pool. **Args:** * `resource_pool`: Node corresponding to the pool to allocate resources from. * `identifier`: Value to perform idempotent allocation, the same resource will be returned for a given identifier. * `prefix_length`: Length of the prefix to set on the address to allocate. * `address_type`: Kind of the address to allocate. * `data`: A key/value map to use to set attributes values on the allocated address. * `branch`: Name of the branch to allocate from. Defaults to default\_branch. * `timeout`: Flag to indicate whether to populate the store with the retrieved nodes. * `tracker`: The offset for pagination. **Returns:** * Node corresponding to the allocated resource. **Raises:** * `ValueError`: If `resource_pool` is not a `CoreIPAddressPool`. #### `allocate_next_ip_prefix`[​](#allocate_next_ip_prefix-3 "Direct link to allocate_next_ip_prefix-3") ``` allocate_next_ip_prefix(self, resource_pool: CoreNodeSync, kind: type[SchemaTypeSync], identifier: str | None = ..., prefix_length: int | None = ..., member_type: str | None = ..., prefix_type: str | None = ..., data: dict[str, Any] | None = ..., branch: str | None = ..., timeout: int | None = ..., tracker: str | None = ...) -> SchemaTypeSync | None ``` Show 2 other overloads #### `allocate_next_ip_prefix`[​](#allocate_next_ip_prefix-4 "Direct link to allocate_next_ip_prefix-4") ``` allocate_next_ip_prefix(self, resource_pool: CoreNodeSync, kind: None = ..., identifier: str | None = ..., prefix_length: int | None = ..., member_type: str | None = ..., prefix_type: str | None = ..., data: dict[str, Any] | None = ..., branch: str | None = ..., timeout: int | None = ..., tracker: str | None = ...) -> CoreNodeSync | None ``` #### `allocate_next_ip_prefix`[​](#allocate_next_ip_prefix-5 "Direct link to allocate_next_ip_prefix-5") ``` allocate_next_ip_prefix(self, resource_pool: CoreNodeSync, kind: type[SchemaTypeSync] | None = None, identifier: str | None = None, prefix_length: int | None = None, member_type: str | None = None, prefix_type: str | None = None, data: dict[str, Any] | None = None, branch: str | None = None, timeout: int | None = None, tracker: str | None = None) -> CoreNodeSync | SchemaTypeSync | None ``` Allocate a new IP prefix by using the provided resource pool. **Args:** * `resource_pool`: Node corresponding to the pool to allocate resources from. * `identifier`: Value to perform idempotent allocation, the same resource will be returned for a given identifier. * `prefix_length`: Length of the prefix to allocate. * `member_type`: Member type of the prefix to allocate. * `prefix_type`: Kind of the prefix to allocate. * `data`: A key/value map to use to set attributes values on the allocated prefix. * `branch`: Name of the branch to allocate from. Defaults to default\_branch. * `timeout`: Flag to indicate whether to populate the store with the retrieved nodes. * `tracker`: The offset for pagination. **Returns:** * Node corresponding to the allocated resource. **Raises:** * `ValueError`: If `resource_pool` is not a `CoreIPPrefixPool`. #### `repository_update_commit`[​](#repository_update_commit-1 "Direct link to repository_update_commit-1") ``` repository_update_commit(self, branch_name: str, repository_id: str, commit: str, is_read_only: bool = False) -> bool ``` #### `refresh_login`[​](#refresh_login-1 "Direct link to refresh_login-1") ``` refresh_login(self) -> None ``` #### `login`[​](#login-1 "Direct link to login-1") ``` login(self, refresh: bool = False) -> None ``` #### `convert_object_type`[​](#convert_object_type-1 "Direct link to convert_object_type-1") ``` convert_object_type(self, node_id: str, target_kind: str, branch: str | None = None, fields_mapping: dict[str, ConversionFieldInput] | None = None) -> InfrahubNodeSync ``` Convert a given node to another kind on a given branch. `fields_mapping` keys are target fields names and its values indicate how to fill in these fields. Any mandatory field not having an equivalent field in the source kind should be specified in this mapping. See for more information. ### `ProcessRelationsNode`[​](#processrelationsnode "Direct link to processrelationsnode") ### `ProxyConfig`[​](#proxyconfig "Direct link to proxyconfig") ### `ProxyConfigSync`[​](#proxyconfigsync "Direct link to proxyconfigsync") ### `ProcessRelationsNodeSync`[​](#processrelationsnodesync "Direct link to processrelationsnodesync") ### `BaseClient`[​](#baseclient "Direct link to baseclient") Base class for InfrahubClient and InfrahubClientSync. **Methods:** #### `retry_on_failure`[​](#retry_on_failure "Direct link to retry_on_failure") ``` retry_on_failure(self) -> bool ``` Whether transient failures are retried. Can be toggled at runtime, e.g. by a long-running generator. #### `retry_on_failure`[​](#retry_on_failure-1 "Direct link to retry_on_failure-1") ``` retry_on_failure(self, value: bool) -> None ``` #### `retry_delay`[​](#retry_delay "Direct link to retry_delay") ``` retry_delay(self) -> float ``` Base delay in seconds between retries of a transient failure; doubles per attempt up to retry\_max\_delay. #### `retry_delay`[​](#retry_delay-1 "Direct link to retry_delay-1") ``` retry_delay(self, value: float) -> None ``` #### `request_context`[​](#request_context "Direct link to request_context") ``` request_context(self) -> RequestContext | None ``` #### `request_context`[​](#request_context-1 "Direct link to request_context-1") ``` request_context(self, request_context: RequestContext) -> None ``` #### `start_tracking`[​](#start_tracking "Direct link to start_tracking") ``` start_tracking(self, identifier: str | None = None, params: dict[str, Any] | None = None, delete_unused_nodes: bool = False, group_type: str | None = None, group_params: dict[str, Any] | None = None, branch: str | None = None) -> Self ``` #### `set_context_properties`[​](#set_context_properties "Direct link to set_context_properties") ``` set_context_properties(self, identifier: str, params: dict[str, str] | None = None, delete_unused_nodes: bool = True, reset: bool = True, group_type: str | None = None, group_params: dict[str, Any] | None = None, branch: str | None = None) -> None ``` ## Functions[​](#functions "Direct link to Functions") ### `handle_relogin`[​](#handle_relogin "Direct link to handle_relogin") ``` handle_relogin(func: Callable[..., Coroutine[Any, Any, httpx.Response]]) -> Callable[..., Coroutine[Any, Any, httpx.Response]] ``` ### `handle_relogin_sync`[​](#handle_relogin_sync "Direct link to handle_relogin_sync") ``` handle_relogin_sync(func: Callable[..., httpx.Response]) -> Callable[..., httpx.Response] ``` ### `get_kind_as_string`[​](#get_kind_as_string "Direct link to get_kind_as_string") ``` get_kind_as_string(kind: str | type[SchemaType | SchemaTypeSync]) -> str ``` --- # `infrahub_sdk.graph_traversal.models` Pydantic models for the Infrahub graph-traversal queries (Infrahub 1.10+). These mirror the server GraphQL types for `InfrahubPathTraversal` and `InfrahubReachableNodes`. The server returns snake\_case field names, so the Python attributes map directly without aliasing. Models ignore unknown fields so additive server changes do not break parsing. ## Classes[​](#classes "Direct link to Classes") ### `GraphTraversalModel`[​](#graphtraversalmodel "Direct link to graphtraversalmodel") Base for all traversal models: tolerate unknown/extra server fields. ### `PathNode`[​](#pathnode "Direct link to pathnode") Identity of a node encountered during a traversal. This is a lightweight identity (no attributes or relationships). Use :meth:`fetch` to resolve it into the full SDK node when needed. **Methods:** #### `fetch`[​](#fetch "Direct link to fetch") ``` fetch(self, timeout: int | None = None) -> Any ``` Resolve this node into the full SDK node. On an async client you await the return value (`await node.fetch()`); on a sync client it returns the node directly. The result is added to the client store, so fetching the same id again is served from the store. **Raises:** * `Error`: If this node is not bound to a client (for example, constructed manually). ### `PathRelationship`[​](#pathrelationship "Direct link to pathrelationship") A relationship (edge) traversed between two nodes. ### `PathHop`[​](#pathhop "Direct link to pathhop") A single step in a path: the node visited and the relationship used to reach it. `relationship` is `None` for the source-anchored first hop. ### `Path`[​](#path "Direct link to path") One route between two nodes, as an ordered list of hops. ### `PathTraversalResult`[​](#pathtraversalresult "Direct link to pathtraversalresult") Result of :meth:`InfrahubClient.traverse_paths`. ### `ReachableNode`[​](#reachablenode "Direct link to reachablenode") A node reachable from the source, with the path used to reach it. ### `ReachableNodesResult`[​](#reachablenodesresult "Direct link to reachablenodesresult") Result of :meth:`InfrahubClient.reachable_nodes`. --- # `infrahub_sdk.graph_traversal.query` GraphQL query strings and variable builders for graph traversal (Infrahub 1.10+). Both server queries accept a single complex input object passed as the GraphQL variable `$data`. The input field names are snake\_case on the wire, matching the SDK keyword arguments, so the variable dict is built directly with unset optional fields omitted (the server applies its own defaults). ## Functions[​](#functions "Direct link to Functions") ### `is_unknown_field_error`[​](#is_unknown_field_error "Direct link to is_unknown_field_error") ``` is_unknown_field_error(errors: list[dict[str, Any]], field_name: str) -> bool ``` Return True if the GraphQL errors indicate `field_name` is an unknown query field. Used to detect a pre-1.10 server that lacks the traversal queries, so the SDK can raise a clear version error instead of surfacing an opaque validation failure. The server's own runtime errors (such as "Source node not found") do not match. ### `build_path_traversal_input`[​](#build_path_traversal_input "Direct link to build_path_traversal_input") ``` build_path_traversal_input(source_id: str, destination_id: str) -> dict[str, Any] ``` Build the `PathTraversalInput` variable, omitting unset optional fields. ### `build_reachable_nodes_input`[​](#build_reachable_nodes_input "Direct link to build_reachable_nodes_input") ``` build_reachable_nodes_input(source_id: str, target_kinds: list[str]) -> dict[str, Any] ``` Build the `ReachableNodesInput` variable, omitting unset optional fields. --- # `infrahub_sdk.node.node` ## Classes[​](#classes "Direct link to Classes") ### `InfrahubNode`[​](#infrahubnode "Direct link to infrahubnode") Asynchronous Infrahub node bound to an :class:`InfrahubClient`. Provides full CRUD against the backend (:meth:`save`, :meth:`create`, :meth:`update`, :meth:`delete`) along with relationship traversal (:meth:`get_flat_value`, :meth:`extract`), feature-gated artifact and resource-pool helpers (:meth:`artifact_generate`, :meth:`get_pool_allocated_resources`), and file upload or download for nodes inheriting from `CoreFileObject`. Attributes and relationships defined on the schema are exposed as instance attributes via attribute-style access (`node.name.value`, `node.parent`). **Methods:** #### `from_graphql`[​](#from_graphql "Direct link to from_graphql") ``` from_graphql(cls, client: InfrahubClient, branch: str, data: dict, schema: MainSchemaTypesAPI | None = None, timeout: int | None = None) -> Self ``` Build an :class:`InfrahubNode` from a raw GraphQL response. When no `schema` is provided, the node kind is read from `__typename` in the payload and the schema is fetched from the client. **Args:** * `client`: The client used to interact with the backend. * `branch`: The branch the node belongs to. * `data`: The GraphQL payload describing the node. * `schema`: Pre-fetched schema for the node kind. Skips the schema lookup when provided. * `timeout`: Overrides the default timeout used when fetching the schema. Specified in seconds. **Returns:** * The hydrated node instance. **Raises:** * `ValueError`: If `__typename` is missing from `data` and no `schema` was provided. #### `generate`[​](#generate "Direct link to generate") ``` generate(self, nodes: list[str] | None = None) -> None ``` Trigger artifact generation for this artifact definition. Only available on nodes whose kind is `CoreArtifactDefinition`. **Args:** * `nodes`: The IDs of target nodes to generate artifacts for. When omitted, generation runs for all targets matched by the definition. **Raises:** * `FeatureNotSupportedError`: If this node is not a `CoreArtifactDefinition`. #### `artifact_generate`[​](#artifact_generate "Direct link to artifact_generate") ``` artifact_generate(self, name: str) -> None ``` Regenerate a named artifact targeting this node. Looks up the `CoreArtifact` named `name` for this node, then calls :meth:`generate` on the related definition with this artifact's ID. **Args:** * `name`: The name of the artifact to regenerate. **Raises:** * `FeatureNotSupportedError`: If this node does not inherit from `CoreArtifactTarget`. #### `artifact_fetch`[​](#artifact_fetch "Direct link to artifact_fetch") ``` artifact_fetch(self, name: str) -> str | dict[str, Any] ``` Fetch the stored content of a named artifact for this node. **Args:** * `name`: The name of the artifact to fetch. **Returns:** * str | dict\[str, Any]: The artifact content. Returns a parsed object for * JSON-typed artifacts and a string for text-typed artifacts. **Raises:** * `FeatureNotSupportedError`: If this node does not inherit from `CoreArtifactTarget`. #### `download_file`[​](#download_file "Direct link to download_file") ``` download_file(self, dest: None = None, skip_if_unchanged: bool = ...) -> bytes ``` Show 2 other overloads #### `download_file`[​](#download_file-1 "Direct link to download_file-1") ``` download_file(self, dest: Path, skip_if_unchanged: bool = ...) -> int ``` #### `download_file`[​](#download_file-2 "Direct link to download_file-2") ``` download_file(self, dest: Path | None = None, skip_if_unchanged: bool = False) -> bytes | int ``` Download the file content from this FileObject node. This method is only available for nodes that inherit from CoreFileObject. The node must have been saved (have an id) before calling this method. **Args:** * `dest`: Optional destination path. If provided, the file will be streamed directly to this path (memory-efficient for large files) and the number of bytes written will be returned. If not provided, the file content will be returned as bytes. * `skip_if_unchanged`: When `True`, compute the SHA-1 of the file at `dest` (which must be provided) and compare against the node's `checksum` attribute. If they match, return `0` without hitting the network. The `checksum` is the value loaded when this node was fetched — a later server-side change to the file will not be detected unless the caller re-fetches the node first. **Returns:** * If `dest` is None: The file content as bytes. * If `dest` is provided: The number of bytes written to the file. * If `skip_if_unchanged=True` and the local file matches the server checksum: `0`. **Raises:** * `FeatureNotSupportedError`: If this node doesn't inherit from CoreFileObject. * `ValueError`: If the node hasn't been saved yet, file not found, or `skip_if_unchanged=True` was passed without a `dest`. * `AuthenticationError`: If authentication fails. **Examples:** ``` >>> # Download to memory >>> content = await contract.download_file() >>> # Stream to file (memory-efficient for large files) >>> bytes_written = await contract.download_file(dest=Path("/tmp/contract.pdf")) >>> # Skip download if local file already matches server checksum >>> bytes_written = await contract.download_file( ... dest=Path("/tmp/contract.pdf"), skip_if_unchanged=True ... ) ``` #### `matches_local_checksum`[​](#matches_local_checksum "Direct link to matches_local_checksum") ``` matches_local_checksum(self, source: bytes | Path | BinaryIO) -> bool ``` Return True if `source`'s SHA-1 matches this node's server checksum. Only available for nodes inheriting from `CoreFileObject`. Callers that want to branch on the comparison without invoking a transfer should use this primitive instead of reading `node.checksum.value` and hashing `source` themselves, so the hashing convention stays centralised in the SDK. The comparison is against the `checksum` attribute as loaded when this node was retrieved from the server. If the server's file has been replaced since the node was fetched, this method will not see that change — re-fetch the node to refresh the checksum before comparing. **Args:** * `source`: Local content to hash and compare. Accepts the same shapes as :func:`infrahub_sdk.file_handler.sha1_of_source`. **Returns:** * True if the local digest equals the server's stored checksum. **Raises:** * `FeatureNotSupportedError`: Node is not a `CoreFileObject`. * `ValueError`: Node has no server-side checksum yet (unsaved or file never attached). #### `upload_if_changed`[​](#upload_if_changed "Direct link to upload_if_changed") ``` upload_if_changed(self, source: bytes | Path | BinaryIO, name: str | None = None) -> UploadResult ``` Upload `source` only if its SHA-1 differs from the server checksum. Composes :meth:`matches_local_checksum` with :meth:`upload_from_path` (or :meth:`upload_from_bytes`) and :meth:`save`. For unsaved nodes or nodes that have no prior server-side file, the upload is always performed — there is nothing to compare against. Idempotency is content-only: when the local SHA-1 matches the server checksum the upload is skipped even if `name` differs from the server-side filename. Use a regular :meth:`upload_from_path` / :meth:`save` round-trip if you need to rename without changing content. **Args:** * `source`: Content to upload. `bytes` and `BinaryIO` sources must supply `name`; for a `Path` the filename is derived from `source.name` when `name` is omitted. * `name`: Filename to use on the server. Required for `bytes` / `BinaryIO` sources. **Returns:** * class:`UploadResult` with `was_uploaded=False` (skipped) or * `was_uploaded=True` (transfer occurred), and the resulting server * checksum (`None` only when no server checksum was available * after the operation). **Raises:** * `FeatureNotSupportedError`: Node is not a `CoreFileObject`. * `ValueError`: `source` is `bytes` or `BinaryIO` and no `name` was supplied. #### `delete`[​](#delete "Direct link to delete") ``` delete(self, timeout: int | None = None, request_context: RequestContext | None = None, priority: Priority | None = None) -> None ``` Delete this node on the backend. **Args:** * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `request_context`: Request-level context passed through to the mutation. When omitted, the client's request context is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for this request only. #### `save`[​](#save "Direct link to save") ``` save(self, allow_upsert: bool = False, update_group_context: bool | None = None, timeout: int | None = None, request_context: RequestContext | None = None, priority: Priority | None = None) -> None ``` Persist this node to the backend, creating or updating it as appropriate. New nodes are created (or upserted when `allow_upsert` is set), and existing nodes are updated with only the modified fields. After a successful save, the node is added to the client store and, when applicable, to the active group context for tracking. **Args:** * `allow_upsert`: When `True`, an existing node is upserted instead of failing with a duplicate. Defaults to `False`. * `update_group_context`: Whether to update the group context with this node. When `None` and the client is in tracking mode, defaults to `True`. * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `request_context`: Request-level context passed through to the mutation. When omitted, the client's request context is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for this request only. #### `generate_query_data`[​](#generate_query_data "Direct link to generate_query_data") ``` generate_query_data(self, filters: dict[str, Any] | None = None, offset: int | str | None = None, limit: int | str | None = None, include: list[str] | None = None, exclude: list[str] | None = None, fragment: bool = False, prefetch_relationships: bool = False, partial_match: bool = False, property: bool = False, order: Order | None = None, include_metadata: bool = False) -> dict[str, Any | dict] ``` Generate the full GraphQL query payload for this node kind. The returned dict combines :meth:`generate_query_data_init` with :meth:`generate_query_data_node`. When the node is a generic and `fragment` is `True`, `...on Kind` fragments are added for every implementing kind so the relevant attributes are returned alongside the generic fields. **Args:** * `filters`: Filters to apply to the query. * `offset`: Pagination offset, either a literal value or a GraphQL variable placeholder such as `"$offset"`. * `limit`: Pagination limit, either a literal value or a GraphQL variable placeholder such as `"$limit"`. * `include`: Attributes or relationships to include. * `exclude`: Attributes or relationships to exclude. * `fragment`: When `True` and the schema is a generic, emit `...on Kind` fragments for each implementing kind. Defaults to `False`. * `prefetch_relationships`: When `True`, pre-fetch related node data instead of returning only their identifiers. Defaults to `False`. * `partial_match`: When `True`, allow partial matches on filter criteria. Defaults to `False`. * `property`: When `True`, include attribute and relationship properties (`source`, `owner`, `is_protected`, ...). Defaults to `False`. * `order`: Ordering options to apply to the query. * `include_metadata`: When `True`, include `node_metadata` and `relationship_metadata` in the result. Defaults to `False`. **Returns:** * dict\[str, Any | dict]: A query payload keyed by the node kind, ready to be * rendered as GraphQL. #### `generate_query_data_node`[​](#generate_query_data_node "Direct link to generate_query_data_node") ``` generate_query_data_node(self, include: list[str] | None = None, exclude: list[str] | None = None, inherited: bool = True, insert_alias: bool = False, prefetch_relationships: bool = False, property: bool = False, include_metadata: bool = False) -> dict[str, Any | dict] ``` Generate the node part of a GraphQL Query with attributes and nodes. **Args:** * `include`: List of attributes or relationships to include. Defaults to None. * `exclude`: List of attributes or relationships to exclude. Defaults to None. * `inherited`: Indicated of the attributes and the relationships inherited from generics should be included as well. Defaults to True. * `insert_alias`: If True, inserts aliases in the query for each attribute or relationship. * `prefetch_relationships`: If True, pre-fetches relationship data as part of the query. * `include_metadata`: If True, includes node\_metadata and relationship\_metadata in the query. **Returns:** * dict\[str, Union\[Any, Dict]]: GraphQL query in dictionary format #### `add_relationships`[​](#add_relationships "Direct link to add_relationships") ``` add_relationships(self, relation_to_update: str, related_nodes: list[str]) -> None ``` Add peers to a cardinality-many relationship through a dedicated mutation. Unlike :meth:`save`, this method targets a single relationship and only adds peers, leaving every other field untouched. **Args:** * `relation_to_update`: The name of the relationship to update. * `related_nodes`: The IDs of the peers to add. #### `remove_relationships`[​](#remove_relationships "Direct link to remove_relationships") ``` remove_relationships(self, relation_to_update: str, related_nodes: list[str]) -> None ``` Remove peers from a cardinality-many relationship through a dedicated mutation. Unlike :meth:`save`, this method targets a single relationship and only removes the listed peers, leaving every other field untouched. **Args:** * `relation_to_update`: The name of the relationship to update. * `related_nodes`: The IDs of the peers to remove. #### `create`[​](#create "Direct link to create") ``` create(self, allow_upsert: bool = False, timeout: int | None = None, request_context: RequestContext | None = None, priority: Priority | None = None) -> None ``` Create this node on the backend. For nodes inheriting from `CoreFileObject`, the file content set with :meth:`upload_from_path` or :meth:`upload_from_bytes` is uploaded as part of the mutation and cleared from the node afterward. Prefer :meth:`save` over calling `create()` directly so existing-vs-new logic is handled for you. **Args:** * `allow_upsert`: When `True`, the operation upserts instead of erroring on a duplicate. Defaults to `False`. * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `request_context`: Request-level context passed through to the mutation. When omitted, the client's request context is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for this request only. **Raises:** * `ValueError`: If this is a file-object node and no file content has been set. #### `update`[​](#update "Direct link to update") ``` update(self, do_full_update: bool = False, timeout: int | None = None, request_context: RequestContext | None = None, priority: Priority | None = None) -> None ``` Update this node on the backend. By default only the modified attributes and relationships are sent so the server can compute a minimal diff. Setting `do_full_update` re-sends every field even when unchanged, which is useful when forcing relationship reconciliation. Prefer :meth:`save` over calling `update()` directly so existing-vs-new logic is handled for you. **Args:** * `do_full_update`: When `True`, send every field even when unmodified. Defaults to `False`. * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `request_context`: Request-level context passed through to the mutation. When omitted, the client's request context is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for this request only. #### `get_pool_allocated_resources`[​](#get_pool_allocated_resources "Direct link to get_pool_allocated_resources") ``` get_pool_allocated_resources(self, resource: InfrahubNode) -> list[InfrahubNode] ``` Fetch all nodes that were allocated for the pool and a given resource. **Args:** * `resource`: The resource from which the nodes were allocated. **Returns:** * list\[InfrahubNode]: The allocated nodes. **Raises:** * `ValueError`: If the node is not a resource pool. #### `get_pool_resources_utilization`[​](#get_pool_resources_utilization "Direct link to get_pool_resources_utilization") ``` get_pool_resources_utilization(self) -> list[dict[str, Any]] ``` Fetch the utilization of each resource for the pool. **Returns:** * list\[dict\[str, Any]]: A list containing the allocation numbers for each resource of the pool. **Raises:** * `ValueError`: If the node is not a resource pool. #### `get_flat_value`[​](#get_flat_value "Direct link to get_flat_value") ``` get_flat_value(self, key: str, separator: str = '__') -> Any ``` Resolve a value addressed by a flat key over this node and its related nodes. Walks attributes on this node, descending through cardinality-one relationships (which are fetched on demand) until the final component is reached. Each relationship hop incurs a backend call, so this is intended for ad-hoc lookups rather than bulk traversal. **Args:** * `key`: The flat key to resolve (for example `"name__value"` or `"site__name__value"`). * `separator`: Component separator in `key`. Defaults to `"__"`. **Returns:** * The resolved value. **Raises:** * `ValueError`: If a component does not match an attribute or relationship, or if a relationship hop targets a non cardinality-one relationship. **Examples:** name\_\_value module.object.value #### `extract`[​](#extract "Direct link to extract") ``` extract(self, params: dict[str, str]) -> dict[str, Any] ``` Extract several values addressed by flat keys into a labeled dict. Each value in `params` is resolved with :meth:`get_flat_value`, and the corresponding key is preserved as the output label. **Args:** * `params`: A mapping of output label to flat key to resolve. **Returns:** * dict\[str, Any]: The resolved values keyed by their output label. ### `InfrahubNodeSync`[​](#infrahubnodesync "Direct link to infrahubnodesync") Synchronous Infrahub node bound to an :class:`InfrahubClientSync`. Synchronous counterpart of :class:`InfrahubNode`. Provides full CRUD against the backend (:meth:`save`, :meth:`create`, :meth:`update`, :meth:`delete`) along with relationship traversal (:meth:`get_flat_value`, :meth:`extract`), feature-gated artifact and resource-pool helpers (:meth:`artifact_generate`, :meth:`get_pool_allocated_resources`), and file upload or download for nodes inheriting from `CoreFileObject`. Attributes and relationships defined on the schema are exposed as instance attributes via attribute-style access (`node.name.value`, `node.parent`). **Methods:** #### `from_graphql`[​](#from_graphql-1 "Direct link to from_graphql-1") ``` from_graphql(cls, client: InfrahubClientSync, branch: str, data: dict, schema: MainSchemaTypesAPI | None = None, timeout: int | None = None) -> Self ``` Build an :class:`InfrahubNodeSync` from a raw GraphQL response. When no `schema` is provided, the node kind is read from `__typename` in the payload and the schema is fetched from the client. **Args:** * `client`: The client used to interact with the backend. * `branch`: The branch the node belongs to. * `data`: The GraphQL payload describing the node. * `schema`: Pre-fetched schema for the node kind. Skips the schema lookup when provided. * `timeout`: Overrides the default timeout used when fetching the schema. Specified in seconds. **Returns:** * The hydrated node instance. **Raises:** * `ValueError`: If `__typename` is missing from `data` and no `schema` was provided. #### `generate`[​](#generate-1 "Direct link to generate-1") ``` generate(self, nodes: list[str] | None = None) -> None ``` Trigger artifact generation for this artifact definition. Only available on nodes whose kind is `CoreArtifactDefinition`. **Args:** * `nodes`: The IDs of target nodes to generate artifacts for. When omitted, generation runs for all targets matched by the definition. **Raises:** * `FeatureNotSupportedError`: If this node is not a `CoreArtifactDefinition`. #### `artifact_generate`[​](#artifact_generate-1 "Direct link to artifact_generate-1") ``` artifact_generate(self, name: str) -> None ``` Regenerate a named artifact targeting this node. Looks up the `CoreArtifact` named `name` for this node, then calls :meth:`generate` on the related definition with this artifact's ID. **Args:** * `name`: The name of the artifact to regenerate. **Raises:** * `FeatureNotSupportedError`: If this node does not inherit from `CoreArtifactTarget`. #### `artifact_fetch`[​](#artifact_fetch-1 "Direct link to artifact_fetch-1") ``` artifact_fetch(self, name: str) -> str | dict[str, Any] ``` Fetch the stored content of a named artifact for this node. **Args:** * `name`: The name of the artifact to fetch. **Returns:** * str | dict\[str, Any]: The artifact content. Returns a parsed object for * JSON-typed artifacts and a string for text-typed artifacts. **Raises:** * `FeatureNotSupportedError`: If this node does not inherit from `CoreArtifactTarget`. #### `download_file`[​](#download_file-3 "Direct link to download_file-3") ``` download_file(self, dest: None = None, skip_if_unchanged: bool = ...) -> bytes ``` Show 2 other overloads #### `download_file`[​](#download_file-4 "Direct link to download_file-4") ``` download_file(self, dest: Path, skip_if_unchanged: bool = ...) -> int ``` #### `download_file`[​](#download_file-5 "Direct link to download_file-5") ``` download_file(self, dest: Path | None = None, skip_if_unchanged: bool = False) -> bytes | int ``` Download the file content from this FileObject node. This method is only available for nodes that inherit from CoreFileObject. The node must have been saved (have an id) before calling this method. **Args:** * `dest`: Optional destination path. If provided, the file will be streamed directly to this path (memory-efficient for large files) and the number of bytes written will be returned. If not provided, the file content will be returned as bytes. * `skip_if_unchanged`: When `True`, compute the SHA-1 of the file at `dest` (which must be provided) and compare against the node's `checksum` attribute. If they match, return `0` without hitting the network. The `checksum` is the value loaded when this node was fetched — a later server-side change to the file will not be detected unless the caller re-fetches the node first. **Returns:** * If `dest` is None: The file content as bytes. * If `dest` is provided: The number of bytes written to the file. * If `skip_if_unchanged=True` and the local file matches the server checksum: `0`. **Raises:** * `FeatureNotSupportedError`: If this node doesn't inherit from CoreFileObject. * `ValueError`: If the node hasn't been saved yet, file not found, or `skip_if_unchanged=True` was passed without a `dest`. * `AuthenticationError`: If authentication fails. **Examples:** ``` >>> # Download to memory >>> content = contract.download_file() >>> # Stream to file (memory-efficient for large files) >>> bytes_written = contract.download_file(dest=Path("/tmp/contract.pdf")) >>> # Skip download if local file already matches server checksum >>> bytes_written = contract.download_file( ... dest=Path("/tmp/contract.pdf"), skip_if_unchanged=True ... ) ``` #### `matches_local_checksum`[​](#matches_local_checksum-1 "Direct link to matches_local_checksum-1") ``` matches_local_checksum(self, source: bytes | Path | BinaryIO) -> bool ``` Return True if `source`'s SHA-1 matches this node's server checksum. Only available for nodes inheriting from `CoreFileObject`. Callers that want to branch on the comparison without invoking a transfer should use this primitive instead of reading `node.checksum.value` and hashing `source` themselves, so the hashing convention stays centralised in the SDK. The comparison is against the `checksum` attribute as loaded when this node was retrieved from the server. If the server's file has been replaced since the node was fetched, this method will not see that change — re-fetch the node to refresh the checksum before comparing. **Args:** * `source`: Local content to hash and compare. Accepts the same shapes as :func:`infrahub_sdk.file_handler.sha1_of_source`. **Returns:** * True if the local digest equals the server's stored checksum. **Raises:** * `FeatureNotSupportedError`: Node is not a `CoreFileObject`. * `ValueError`: Node has no server-side checksum yet (unsaved or file never attached). #### `upload_if_changed`[​](#upload_if_changed-1 "Direct link to upload_if_changed-1") ``` upload_if_changed(self, source: bytes | Path | BinaryIO, name: str | None = None) -> UploadResult ``` Upload `source` only if its SHA-1 differs from the server checksum. Composes :meth:`matches_local_checksum` with :meth:`upload_from_path` (or :meth:`upload_from_bytes`) and :meth:`save`. For unsaved nodes or nodes that have no prior server-side file, the upload is always performed — there is nothing to compare against. Idempotency is content-only: when the local SHA-1 matches the server checksum the upload is skipped even if `name` differs from the server-side filename. Use a regular :meth:`upload_from_path` / :meth:`save` round-trip if you need to rename without changing content. **Args:** * `source`: Content to upload. `bytes` and `BinaryIO` sources must supply `name`; for a `Path` the filename is derived from `source.name` when `name` is omitted. * `name`: Filename to use on the server. Required for `bytes` / `BinaryIO` sources. **Returns:** * class:`UploadResult` with `was_uploaded=False` (skipped) or * `was_uploaded=True` (transfer occurred), and the resulting server * checksum (`None` only when no server checksum was available * after the operation). **Raises:** * `FeatureNotSupportedError`: Node is not a `CoreFileObject`. * `ValueError`: `source` is `bytes` or `BinaryIO` and no `name` was supplied. #### `delete`[​](#delete-1 "Direct link to delete-1") ``` delete(self, timeout: int | None = None, request_context: RequestContext | None = None, priority: Priority | None = None) -> None ``` Delete this node on the backend. **Args:** * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `request_context`: Request-level context passed through to the mutation. When omitted, the client's request context is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for this request only. #### `save`[​](#save-1 "Direct link to save-1") ``` save(self, allow_upsert: bool = False, update_group_context: bool | None = None, timeout: int | None = None, request_context: RequestContext | None = None, priority: Priority | None = None) -> None ``` Persist this node to the backend, creating or updating it as appropriate. New nodes are created (or upserted when `allow_upsert` is set), and existing nodes are updated with only the modified fields. After a successful save, the node is added to the client store and, when applicable, to the active group context for tracking. **Args:** * `allow_upsert`: When `True`, an existing node is upserted instead of failing with a duplicate. Defaults to `False`. * `update_group_context`: Whether to update the group context with this node. When `None` and the client is in tracking mode, defaults to `True`. * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `request_context`: Request-level context passed through to the mutation. When omitted, the client's request context is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for this request only. #### `generate_query_data`[​](#generate_query_data-1 "Direct link to generate_query_data-1") ``` generate_query_data(self, filters: dict[str, Any] | None = None, offset: int | str | None = None, limit: int | str | None = None, include: list[str] | None = None, exclude: list[str] | None = None, fragment: bool = False, prefetch_relationships: bool = False, partial_match: bool = False, property: bool = False, order: Order | None = None, include_metadata: bool = False) -> dict[str, Any | dict] ``` Generate the full GraphQL query payload for this node kind. The returned dict combines :meth:`generate_query_data_init` with :meth:`generate_query_data_node`. When the node is a generic and `fragment` is `True`, `...on Kind` fragments are added for every implementing kind so the relevant attributes are returned alongside the generic fields. **Args:** * `filters`: Filters to apply to the query. * `offset`: Pagination offset, either a literal value or a GraphQL variable placeholder such as `"$offset"`. * `limit`: Pagination limit, either a literal value or a GraphQL variable placeholder such as `"$limit"`. * `include`: Attributes or relationships to include. * `exclude`: Attributes or relationships to exclude. * `fragment`: When `True` and the schema is a generic, emit `...on Kind` fragments for each implementing kind. Defaults to `False`. * `prefetch_relationships`: When `True`, pre-fetch related node data instead of returning only their identifiers. Defaults to `False`. * `partial_match`: When `True`, allow partial matches on filter criteria. Defaults to `False`. * `property`: When `True`, include attribute and relationship properties (`source`, `owner`, `is_protected`, ...). Defaults to `False`. * `order`: Ordering options to apply to the query. * `include_metadata`: When `True`, include `node_metadata` and `relationship_metadata` in the result. Defaults to `False`. **Returns:** * dict\[str, Any | dict]: A query payload keyed by the node kind, ready to be * rendered as GraphQL. #### `generate_query_data_node`[​](#generate_query_data_node-1 "Direct link to generate_query_data_node-1") ``` generate_query_data_node(self, include: list[str] | None = None, exclude: list[str] | None = None, inherited: bool = True, insert_alias: bool = False, prefetch_relationships: bool = False, property: bool = False, include_metadata: bool = False) -> dict[str, Any | dict] ``` Generate the node part of a GraphQL Query with attributes and nodes. **Args:** * `include`: List of attributes or relationships to include. Defaults to None. * `exclude`: List of attributes or relationships to exclude. Defaults to None. * `inherited`: Indicated of the attributes and the relationships inherited from generics should be included as well. Defaults to True. * `insert_alias`: If True, inserts aliases in the query for each attribute or relationship. * `prefetch_relationships`: If True, pre-fetches relationship data as part of the query. * `include_metadata`: If True, includes node\_metadata and relationship\_metadata in the query. **Returns:** * dict\[str, Union\[Any, Dict]]: GraphQL query in dictionary format #### `add_relationships`[​](#add_relationships-1 "Direct link to add_relationships-1") ``` add_relationships(self, relation_to_update: str, related_nodes: list[str]) -> None ``` Add peers to a cardinality-many relationship through a dedicated mutation. Unlike :meth:`save`, this method targets a single relationship and only adds peers, leaving every other field untouched. **Args:** * `relation_to_update`: The name of the relationship to update. * `related_nodes`: The IDs of the peers to add. #### `remove_relationships`[​](#remove_relationships-1 "Direct link to remove_relationships-1") ``` remove_relationships(self, relation_to_update: str, related_nodes: list[str]) -> None ``` Remove peers from a cardinality-many relationship through a dedicated mutation. Unlike :meth:`save`, this method targets a single relationship and only removes the listed peers, leaving every other field untouched. **Args:** * `relation_to_update`: The name of the relationship to update. * `related_nodes`: The IDs of the peers to remove. #### `create`[​](#create-1 "Direct link to create-1") ``` create(self, allow_upsert: bool = False, timeout: int | None = None, request_context: RequestContext | None = None, priority: Priority | None = None) -> None ``` Create this node on the backend. For nodes inheriting from `CoreFileObject`, the file content set with :meth:`upload_from_path` or :meth:`upload_from_bytes` is uploaded as part of the mutation and cleared from the node afterward. Prefer :meth:`save` over calling `create()` directly so existing-vs-new logic is handled for you. **Args:** * `allow_upsert`: When `True`, the operation upserts instead of erroring on a duplicate. Defaults to `False`. * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `request_context`: Request-level context passed through to the mutation. When omitted, the client's request context is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for this request only. **Raises:** * `ValueError`: If this is a file-object node and no file content has been set. #### `update`[​](#update-1 "Direct link to update-1") ``` update(self, do_full_update: bool = False, timeout: int | None = None, request_context: RequestContext | None = None, priority: Priority | None = None) -> None ``` Update this node on the backend. By default only the modified attributes and relationships are sent so the server can compute a minimal diff. Setting `do_full_update` re-sends every field even when unchanged, which is useful when forcing relationship reconciliation. Prefer :meth:`save` over calling `update()` directly so existing-vs-new logic is handled for you. **Args:** * `do_full_update`: When `True`, send every field even when unmodified. Defaults to `False`. * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `request_context`: Request-level context passed through to the mutation. When omitted, the client's request context is used. * `priority`: Per-request priority emitted as the X-Priority header, overriding the client default for this request only. #### `get_pool_allocated_resources`[​](#get_pool_allocated_resources-1 "Direct link to get_pool_allocated_resources-1") ``` get_pool_allocated_resources(self, resource: InfrahubNodeSync) -> list[InfrahubNodeSync] ``` Fetch all nodes that were allocated for the pool and a given resource. **Args:** * `resource`: The resource from which the nodes were allocated. **Returns:** * list\[InfrahubNodeSync]: The allocated nodes. **Raises:** * `ValueError`: If the node is not a resource pool. #### `get_pool_resources_utilization`[​](#get_pool_resources_utilization-1 "Direct link to get_pool_resources_utilization-1") ``` get_pool_resources_utilization(self) -> list[dict[str, Any]] ``` Fetch the utilization of each resource for the pool. **Returns:** * list\[dict\[str, Any]]: A list containing the allocation numbers for each resource of the pool. **Raises:** * `ValueError`: If the node is not a resource pool. #### `get_flat_value`[​](#get_flat_value-1 "Direct link to get_flat_value-1") ``` get_flat_value(self, key: str, separator: str = '__') -> Any ``` Resolve a value addressed by a flat key over this node and its related nodes. Walks attributes on this node, descending through cardinality-one relationships (which are fetched on demand) until the final component is reached. Each relationship hop incurs a backend call, so this is intended for ad-hoc lookups rather than bulk traversal. **Args:** * `key`: The flat key to resolve (for example `"name__value"` or `"site__name__value"`). * `separator`: Component separator in `key`. Defaults to `"__"`. **Returns:** * The resolved value. **Raises:** * `ValueError`: If a component does not match an attribute or relationship, or if a relationship hop targets a non cardinality-one relationship. **Examples:** name\_\_value module.object.value #### `extract`[​](#extract-1 "Direct link to extract-1") ``` extract(self, params: dict[str, str]) -> dict[str, Any] ``` Extract several values addressed by flat keys into a labeled dict. Each value in `params` is resolved with :meth:`get_flat_value`, and the corresponding key is preserved as the output label. **Args:** * `params`: A mapping of output label to flat key to resolve. **Returns:** * dict\[str, Any]: The resolved values keyed by their output label. ### `UploadResult`[​](#uploadresult "Direct link to uploadresult") Outcome of an idempotent upload attempt. Returned by :meth:`InfrahubNode.upload_if_changed` and its sync twin. `was_uploaded` tells the caller whether a network transfer actually happened; `checksum` carries the SHA-1 of the content held on the server after the operation — on skip paths that is the server's pre-existing value, on upload paths it is the locally-computed SHA-1 used as a proxy (which matches what a standard CoreFileObject server stores, since the server computes SHA-1 of received bytes). `None` only when no server checksum was available (either the node was unsaved and nothing was transferred, or the save returned no checksum value). The comparison used by `upload_if_changed` reads the node's `checksum` attribute, which was populated when the node was fetched via `client.get(...)`. A server-side change to the file between the fetch and the call will not be detected unless the caller re-fetches the node first. ### `InfrahubNodeBase`[​](#infrahubnodebase "Direct link to infrahubnodebase") Base class for :class:`InfrahubNode` and :class:`InfrahubNodeSync`. Owns the schema-driven state shared between the async and sync clients: attributes, relationships, identity (`id`, `hfid`), node metadata, and the helpers that turn the in-memory state into GraphQL query and mutation payloads. This class is not meant to be instantiated directly; use :class:`InfrahubNode` or :class:`InfrahubNodeSync` instead. **Attributes:** * `id`: The unique identifier of the node, when known. * `display_label`: Human-readable label of the node. * `typename`: The GraphQL `__typename` of the node. **Methods:** #### `get_branch`[​](#get_branch "Direct link to get_branch") ``` get_branch(self) -> str ``` Return the branch this node is bound to. **Returns:** * The name of the branch. #### `get_path_value`[​](#get_path_value "Direct link to get_path_value") ``` get_path_value(self, path: str) -> Any ``` Resolve a value addressed by a dunder-separated path on this node. The path can target an attribute (`name__value`, `name__source`), a cardinality-one related node (`parent`), an attribute of that related node (`parent__name__value`), or a property of one of its attributes (`parent__name__source`). **Args:** * `path`: A path with components separated by `__`. **Returns:** * The resolved value, or `None` when any path component cannot be * resolved (for example, an unfetched related node not present in the store). #### `get_human_friendly_id`[​](#get_human_friendly_id "Direct link to get_human_friendly_id") ``` get_human_friendly_id(self) -> list[str] | None ``` Compute the human-friendly ID for this node from its schema. The HFID is composed of the values addressed by the schema's `human_friendly_id` paths. When any component cannot be resolved, the HFID is considered invalid and `None` is returned. **Returns:** * list\[str] | None: The HFID as a list of stringified components, or `None` * when the schema does not define an HFID or a component is missing. #### `get_human_friendly_id_as_string`[​](#get_human_friendly_id_as_string "Direct link to get_human_friendly_id_as_string") ``` get_human_friendly_id_as_string(self, include_kind: bool = False) -> str | None ``` Return the human-friendly ID joined into a single string. **Args:** * `include_kind`: When `True`, the node kind is prepended as the first component of the resulting string. Defaults to `False`. **Returns:** * str | None: The HFID joined with the HFID separator, or `None` when no * HFID is available. #### `hfid`[​](#hfid "Direct link to hfid") ``` hfid(self) -> list[str] | None ``` Return the human-friendly ID of this node as a list of components. **Returns:** * list\[str] | None: The HFID components, or `None` when unavailable. #### `hfid_str`[​](#hfid_str "Direct link to hfid_str") ``` hfid_str(self) -> str | None ``` Return the human-friendly ID of this node as a string, including the kind prefix. **Returns:** * str | None: The HFID as `Kind__part1__part2`, or `None` when unavailable. #### `get_node_metadata`[​](#get_node_metadata "Direct link to get_node_metadata") ``` get_node_metadata(self) -> NodeMetadata | None ``` Return the node metadata (`created_at`, `created_by`, `updated_at`, `updated_by`). The metadata is populated only when the parent query was executed with `include_metadata=True`. **Returns:** * NodeMetadata | None: The node metadata if fetched, otherwise `None`. #### `get_kind`[​](#get_kind "Direct link to get_kind") ``` get_kind(self) -> str ``` Return the schema kind of this node. **Returns:** * The schema kind (for example `"CoreAccount"`). #### `get_all_kinds`[​](#get_all_kinds "Direct link to get_all_kinds") ``` get_all_kinds(self) -> list[str] ``` Return this node's kind plus all generic kinds it inherits from. **Returns:** * list\[str]: The node's own kind followed by the inherited kinds, in the order * declared on the schema. #### `is_ip_prefix`[​](#is_ip_prefix "Direct link to is_ip_prefix") ``` is_ip_prefix(self) -> bool ``` Return whether this node represents an IP prefix. **Returns:** * `True` when the node kind is `BuiltinIPPrefix` or inherits from it. #### `is_ip_address`[​](#is_ip_address "Direct link to is_ip_address") ``` is_ip_address(self) -> bool ``` Return whether this node represents an IP address. **Returns:** * `True` when the node kind is `BuiltinIPAddress` or inherits from it. #### `is_resource_pool`[​](#is_resource_pool "Direct link to is_resource_pool") ``` is_resource_pool(self) -> bool ``` Return whether this node is a resource pool. **Returns:** * `True` when the node inherits from `CoreResourcePool`. #### `is_file_object`[​](#is_file_object "Direct link to is_file_object") ``` is_file_object(self) -> bool ``` Return whether this node inherits from `CoreFileObject` and supports file uploads. **Returns:** * `True` when file upload/download operations are supported on this node. #### `upload_from_path`[​](#upload_from_path "Direct link to upload_from_path") ``` upload_from_path(self, path: Path) -> None ``` Set a file from disk to be uploaded when saving this FileObject node. The file will be streamed during upload, avoiding loading the entire file into memory. **Args:** * `path`: Path to the file on disk. **Raises:** * `FeatureNotSupportedError`: If this node doesn't inherit from CoreFileObject. #### `upload_from_bytes`[​](#upload_from_bytes "Direct link to upload_from_bytes") ``` upload_from_bytes(self, content: bytes | BinaryIO, name: str) -> None ``` Set content to be uploaded when saving this FileObject node. The content can be provided as bytes or a file-like object. Using BinaryIO is recommended for large content to stream during upload. **Args:** * `content`: The file content as bytes or a file-like object. * `name`: The filename to use for the uploaded file. **Raises:** * `FeatureNotSupportedError`: If this node doesn't inherit from CoreFileObject. **Examples:** ``` >>> # Using bytes (for small files) >>> node.upload_from_bytes(content=b"file content", name="example.txt") >>> # Using file-like object (for large files) >>> with open("/path/to/file.bin", "rb") as f: ... node.upload_from_bytes(content=f, name="file.bin") ``` #### `clear_file`[​](#clear_file "Direct link to clear_file") ``` clear_file(self) -> None ``` Clear any pending file content. #### `get_raw_graphql_data`[​](#get_raw_graphql_data "Direct link to get_raw_graphql_data") ``` get_raw_graphql_data(self) -> dict | None ``` Return the raw GraphQL payload used to build this node. **Returns:** * dict | None: The original GraphQL data, or `None` when the node was * constructed without payload (for example, a brand-new node). #### `generate_query_data_init`[​](#generate_query_data_init "Direct link to generate_query_data_init") ``` generate_query_data_init(self, filters: dict[str, Any] | None = None, offset: int | str | None = None, limit: int | str | None = None, include: list[str] | None = None, exclude: list[str] | None = None, partial_match: bool = False, order: Order | None = None, include_metadata: bool = False) -> dict[str, Any | dict] ``` Build the top-level `count`/`edges` skeleton of a GraphQL query for this kind. The returned dict is the outer structure consumed by :meth:`generate_query_data`; it carries the `@filters` block and the empty `edges.node` placeholder that will later be filled by the caller. **Args:** * `filters`: Filters to apply to the query. * `offset`: Pagination offset, either a literal value or a GraphQL variable placeholder such as `"$offset"`. * `limit`: Pagination limit, either a literal value or a GraphQL variable placeholder such as `"$limit"`. * `include`: Attributes or relationships to include. * `exclude`: Attributes or relationships to exclude. * `partial_match`: When `True`, allow partial matches on filter criteria. Defaults to `False`. * `order`: Ordering options to apply to the query. * `include_metadata`: When `True`, include `node_metadata` in the result. Defaults to `False`. **Returns:** * dict\[str, Any | dict]: The query skeleton ready to be combined with node-level * attributes and relationships. **Raises:** * `ValueError`: If the same name appears in both `include` and `exclude`. --- # `infrahub_sdk.node.attribute` ## Classes[​](#classes "Direct link to Classes") ### `Attribute`[​](#attribute "Direct link to attribute") Represents an attribute of a Node, including its schema, value, and properties. An `Attribute` wraps a single attribute on an :class:`InfrahubNode`. It tracks the current value, the metadata properties (`source`, `owner`, `is_protected`, ...), and whether the value has been mutated since the node was loaded. Mutation tracking is used by `InfrahubNode.update()` to send only the changed fields to the API. **Attributes:** * `name`: The name of the attribute. * `id`: The unique identifier of the attribute, when known. * `value`: The current attribute value. Setting this marks the attribute as mutated. * `value_has_been_mutated`: True when `value` has been assigned after construction. * `is_default`: True when the value comes from the schema default. * `is_from_profile`: True when the value is inherited from a profile. * `is_inherited`: True when the attribute is inherited from a generic. * `is_protected`: True when the attribute is protected from modification. * `updated_at`: ISO-8601 timestamp of the most recent update. * `source`: The node that supplied this attribute value. * `owner`: The node that owns this attribute. * `updated_by`: The account that performed the most recent update. **Methods:** #### `value`[​](#value "Direct link to value") ``` value(self) -> Any ``` #### `value`[​](#value-1 "Direct link to value-1") ``` value(self, value: Any) -> None ``` #### `is_from_pool_attribute`[​](#is_from_pool_attribute "Direct link to is_from_pool_attribute") ``` is_from_pool_attribute(self) -> bool ``` Check whether this attribute's value is sourced from a resource pool. **Returns:** * True if the attribute value is a resource pool node or was explicitly allocated from a pool. #### `is_unresolved_pool_attribute`[​](#is_unresolved_pool_attribute "Direct link to is_unresolved_pool_attribute") ``` is_unresolved_pool_attribute(self) -> bool ``` Return True when pool-backed but no concrete scalar value is available yet. A pool-backed attribute is unresolved when: * its value is a pool node object (the pool reference itself, not an allocated scalar), or * its value is None and the from\_pool allocation dict is set. An attribute whose \_from\_pool dict is set but whose value has already been populated with the allocated scalar (e.g. after a prior save) is considered resolved. --- # `infrahub_sdk.node.constants` *This module is empty or contains only private/internal implementations.* --- # `infrahub_sdk.node.metadata` ## Classes[​](#classes "Direct link to Classes") ### `NodeMetadata`[​](#nodemetadata "Direct link to nodemetadata") Represents metadata about a node (created\_at, created\_by, updated\_at, updated\_by). Populated from the `node_metadata` GraphQL block when `include_metadata=True` is passed to a query. The `*_by` fields point to the user who created or last updated the node, exposed as :class:`NodeProperty` references. **Attributes:** * `created_at`: ISO-8601 timestamp of node creation. * `created_by`: The account that created the node. * `updated_at`: ISO-8601 timestamp of the most recent update. * `updated_by`: The account that performed the most recent update. ### `RelationshipMetadata`[​](#relationshipmetadata "Direct link to relationshipmetadata") Represents metadata about a relationship edge (updated\_at, updated\_by). Populated from the `relationship_metadata` GraphQL block when `include_metadata=True` is passed to a query. Unlike :class:`NodeMetadata`, this only carries update info because the creation timestamp of an edge is not tracked separately from its peer node. **Attributes:** * `updated_at`: ISO-8601 timestamp of the most recent edge update. * `updated_by`: The account that performed the most recent edge update. --- # `infrahub_sdk.node.parsers` ## Functions[​](#functions "Direct link to Functions") ### `parse_human_friendly_id`[​](#parse_human_friendly_id "Direct link to parse_human_friendly_id") ``` parse_human_friendly_id(hfid: str | list[str]) -> tuple[str | None, list[str]] ``` Parse a human-friendly ID into a kind and an identifier. Accepts the HFID either as a separator-joined string (`"Kind__part1__part2"`) or as a list of components. When a string is provided, the first component is treated as the node kind only when more than one component is present. **Args:** * `hfid`: The HFID to parse, either as a separator-joined string or as a list of components. **Returns:** * tuple\[str | None, list\[str]]: A tuple of `(kind, identifier_components)`. `kind` is * `None` when no kind prefix is present (single-component string or list input). **Raises:** * `ValueError`: If `hfid` is neither a string nor a list. --- # `infrahub_sdk.node.property` ## Classes[​](#classes "Direct link to Classes") ### `NodeProperty`[​](#nodeproperty "Direct link to nodeproperty") Represents a property of a node, typically used for metadata like display labels. A `NodeProperty` is a lightweight pointer to another node, used to expose attribute and relationship metadata such as `source`, `owner`, `created_by`, or `updated_by` without loading the full peer node. **Attributes:** * `id`: The identifier of the referenced node. * `display_label`: A human-readable label for the referenced node. * `typename`: The GraphQL `__typename` of the referenced node. --- # `infrahub_sdk.node.related_node` ## Classes[​](#classes "Direct link to Classes") ### `RelatedNodeBase`[​](#relatednodebase "Direct link to relatednodebase") Base class for representing a related node in a relationship. A `RelatedNodeBase` is the peer end of a cardinality-one relationship. It carries the lightweight identification of the peer (`id`, `hfid`, `typename`, ...) along with the relationship-edge properties (`source`, `owner`, `is_protected`, ...). The full peer node is fetched lazily through :meth:`RelatedNode.fetch` / :meth:`RelatedNodeSync.fetch`. **Attributes:** * `schema`: The schema describing the relationship. * `name`: The name of the relationship slot on the parent node. * `updated_at`: ISO-8601 timestamp of the most recent edge update. **Methods:** #### `id`[​](#id "Direct link to id") ``` id(self) -> str | None ``` Return the parsed peer id without triggering a store lookup. Returns None when the response carried only hfid\_str (no id, no peer) — in that case .peer.id would resolve through the store and yield a non-None id, so .id and .peer.id are NOT interchangeable. **Returns:** * str | None: The peer node ID, or `None` when neither the peer nor an ID is set. #### `hfid`[​](#hfid "Direct link to hfid") ``` hfid(self) -> list[Any] | None ``` Return the human-friendly ID of the related node. **Returns:** * list\[Any] | None: The peer HFID as a list of components, or `None` when not set. #### `hfid_str`[​](#hfid_str "Direct link to hfid_str") ``` hfid_str(self) -> str | None ``` Return the human-friendly ID of the related node as a separator-joined string. The returned string includes the kind prefix and is therefore suitable as a key for the client store. **Returns:** * str | None: The peer HFID joined with the HFID separator, or `None` when * unavailable (no resolved peer or missing HFID). #### `is_resource_pool`[​](#is_resource_pool "Direct link to is_resource_pool") ``` is_resource_pool(self) -> bool ``` Return whether the related node is a resource pool. **Returns:** * `True` when the resolved peer inherits from `CoreResourcePool`. #### `initialized`[​](#initialized "Direct link to initialized") ``` initialized(self) -> bool ``` Return whether this related node has an identifier. **Returns:** * `True` when an ID or HFID is known and the relationship can be referenced. #### `display_label`[​](#display_label "Direct link to display_label") ``` display_label(self) -> str | None ``` Return the human-readable label of the related node. **Returns:** * str | None: The peer display label, or `None` when not provided. #### `typename`[​](#typename "Direct link to typename") ``` typename(self) -> str | None ``` Return the GraphQL `__typename` of the related node. **Returns:** * str | None: The peer typename, or `None` when not provided. #### `kind`[​](#kind "Direct link to kind") ``` kind(self) -> str | None ``` Return the schema kind of the related node. **Returns:** * str | None: The peer schema kind, or `None` when not provided. #### `is_from_profile`[​](#is_from_profile "Direct link to is_from_profile") ``` is_from_profile(self) -> bool ``` Return whether this relationship was set from a profile. A relationship is considered profile-sourced when the typename of its `source` property starts with the profile kind prefix. **Returns:** * `True` when the relationship's source is a profile node. #### `get_relationship_metadata`[​](#get_relationship_metadata "Direct link to get_relationship_metadata") ``` get_relationship_metadata(self) -> RelationshipMetadata | None ``` Return the relationship-edge metadata (`updated_at`, `updated_by`). The metadata is populated only when the parent query was executed with `include_metadata=True`. **Returns:** * RelationshipMetadata | None: The edge metadata if fetched, otherwise `None`. ### `RelatedNode`[​](#relatednode "Direct link to relatednode") Asynchronous related node bound to an :class:`InfrahubClient`. Extends :class:`RelatedNodeBase` with the ability to lazily resolve the peer node: :meth:`fetch` retrieves the full peer from the backend, :meth:`get` returns it from the local cache or the client store, and :attr:`peer` is a convenience accessor around :meth:`get`. **Methods:** #### `fetch`[​](#fetch "Direct link to fetch") ``` fetch(self, timeout: int | None = None, priority: Priority | None = None) -> None ``` Fetch the full peer node from the backend and cache it on this object. After `fetch()` completes, attribute and relationship access on the peer is available via :attr:`peer` or :meth:`get`. **Args:** * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `priority`: Override the client-wide request priority for this fetch. When None, the client default is used. **Raises:** * `Error`: If neither `id` nor `typename` is set on this related node. #### `peer`[​](#peer "Direct link to peer") ``` peer(self) -> PeerT ``` Return the resolved peer node. This is a convenience accessor for :meth:`get`; the peer must already have been fetched or stored in the client store. **Returns:** * The resolved peer node. #### `get`[​](#get "Direct link to get") ``` get(self) -> PeerT ``` Return the resolved peer node from cache or the client store. Lookup order: 1. The peer cached locally after a successful :meth:`fetch`. 2. The client store keyed by `id` and `typename`. 3. The client store keyed by `hfid_str`. When resolving via `hfid_str` the returned node has a non-None id even when this `RelatedNode`'s `.id` is None — that is the case in which `.peer.id` and `.id` diverge. **Returns:** * The resolved peer node. **Raises:** * `ValueError`: If neither an ID nor an HFID is available to look up the peer. ### `RelatedNodeSync`[​](#relatednodesync "Direct link to relatednodesync") Synchronous related node bound to an :class:`InfrahubClientSync`. Synchronous counterpart of :class:`RelatedNode`. Extends :class:`RelatedNodeBase` with the ability to lazily resolve the peer node: :meth:`fetch` retrieves the full peer from the backend, :meth:`get` returns it from the local cache or the client store, and :attr:`peer` is a convenience accessor around :meth:`get`. **Methods:** #### `fetch`[​](#fetch-1 "Direct link to fetch-1") ``` fetch(self, timeout: int | None = None, priority: Priority | None = None) -> None ``` Fetch the full peer node from the backend and cache it on this object. After `fetch()` completes, attribute and relationship access on the peer is available via :attr:`peer` or :meth:`get`. **Args:** * `timeout`: Overrides the default timeout used when querying the GraphQL API. Specified in seconds. * `priority`: Override the client-wide request priority for this fetch. When None, the client default is used. **Raises:** * `Error`: If neither `id` nor `typename` is set on this related node. #### `peer`[​](#peer-1 "Direct link to peer-1") ``` peer(self) -> PeerTSync ``` Return the resolved peer node. This is a convenience accessor for :meth:`get`; the peer must already have been fetched or stored in the client store. **Returns:** * The resolved peer node. #### `get`[​](#get-1 "Direct link to get-1") ``` get(self) -> PeerTSync ``` Return the resolved peer node from cache or the client store. Lookup order: 1. The peer cached locally after a successful :meth:`fetch`. 2. The client store keyed by `id` and `typename`. 3. The client store keyed by `hfid_str`. When resolving via `hfid_str` the returned node has a non-None id even when this `RelatedNode`'s `.id` is None — that is the case in which `.peer.id` and `.id` diverge. **Returns:** * The resolved peer node. **Raises:** * `ValueError`: If neither an ID nor an HFID is available to look up the peer. ### `RelationshipAttribute`[​](#relationshipattribute "Direct link to relationshipattribute") Typing descriptor for a cardinality-one relationship on a generated protocol. It reads back as `RelatedNode[PeerT]` (so `.peer` keeps the peer type) but accepts assignment of an id string, an HFID, a peer node, or `None` — mirroring the runtime `InfrahubNode.__setattr__` behaviour, which wraps the assigned value in a `RelatedNode`. This type only appears in generated protocols (it is never instantiated at runtime), so it exists purely to give `node.rel` separate read and assignment types under a type checker. ### `RelationshipAttributeSync`[​](#relationshipattributesync "Direct link to relationshipattributesync") Synchronous counterpart of :class:`RelationshipAttribute`. --- # `infrahub_sdk.node.relationship` ## Classes[​](#classes "Direct link to Classes") ### `RelationshipManagerBase`[​](#relationshipmanagerbase "Direct link to relationshipmanagerbase") Base class for :class:`RelationshipManager` and :class:`RelationshipManagerSync`. A `RelationshipManagerBase` exposes a cardinality-many relationship as a list of peers along with helpers to add, remove, or extend the set. Relationship managers are initialized lazily: until :meth:`fetch` (on the async/sync subclasses) is called, the members are not loaded and editing is not allowed. **Attributes:** * `name`: The name of the relationship slot on the parent node. * `schema`: The schema describing the relationship. * `branch`: The branch the relationship is bound to. * `peers`: The current peer set. * `initialized`: `True` once the manager has been populated with data. **Methods:** #### `peer_ids`[​](#peer_ids "Direct link to peer_ids") ``` peer_ids(self) -> list[str] ``` Return the IDs of all peers that have one. **Returns:** * list\[str]: The IDs of the peers, in insertion order. #### `peer_hfids`[​](#peer_hfids "Direct link to peer_hfids") ``` peer_hfids(self) -> list[list[Any]] ``` Return the HFIDs of all peers that have one. **Returns:** * list\[list\[Any]]: The HFIDs of the peers as lists of components, in insertion order. #### `peer_hfids_str`[​](#peer_hfids_str "Direct link to peer_hfids_str") ``` peer_hfids_str(self) -> list[str] ``` Return the HFIDs of all peers as separator-joined strings. **Returns:** * list\[str]: The HFIDs of the peers as `Kind__part1__part2` strings. #### `has_update`[​](#has_update "Direct link to has_update") ``` has_update(self) -> bool ``` Return whether the peer set has been modified since initialization. **Returns:** * `True` after a successful :meth:`add`, :meth:`extend`, or :meth:`remove`. #### `is_from_profile`[​](#is_from_profile "Direct link to is_from_profile") ``` is_from_profile(self) -> bool ``` Return whether this relationship was set from a profile. The relationship is considered profile-sourced only when every peer is itself sourced from a profile. **Returns:** * `True` when at least one peer exists and all peers are from a profile. ### `RelationshipManager`[​](#relationshipmanager "Direct link to relationshipmanager") Asynchronous manager for a cardinality-many relationship. Extends :class:`RelationshipManagerBase` with the ability to populate and edit the peer set against an :class:`InfrahubClient`: :meth:`fetch` resolves every peer in a parallel batch and :meth:`add`, :meth:`extend`, and :meth:`remove` mutate the peer list in memory. Peers are exposed as :class:`RelatedNode` instances and can be accessed by index via `manager[i]`. **Methods:** #### `fetch`[​](#fetch "Direct link to fetch") ``` fetch(self) -> None ``` Populate the peer set and resolve every peer to a full node. When the manager is not yet initialized, the parent node is re-queried with this relationship included so the peer list can be populated. The peers are then fetched in a parallel batch grouped by kind and stored in the client store. **Raises:** * `Error`: If any peer is missing an `id` or `typename` and cannot be resolved. #### `add`[​](#add "Direct link to add") ``` add(self, data: str | RelatedNode | dict) -> None ``` Add a new peer to this relationship. The new peer is only added when its ID or HFID is not already present; duplicate adds are silently ignored. **Args:** * `data`: The peer to add. Accepts an ID string, an existing :class:`RelatedNode`, or a dict describing the peer (with `id` or `hfid` keys, plus optional relationship properties). **Raises:** * `UninitializedError`: If :meth:`fetch` has not been called on this manager yet. #### `extend`[​](#extend "Direct link to extend") ``` extend(self, data: Iterable[str | RelatedNode | dict]) -> None ``` Add new peers to this relationship. This is a convenience wrapper that calls :meth:`add` for every item in `data`. Items already present (by ID or HFID) are silently ignored. **Args:** * `data`: The peers to add, in any of the formats accepted by :meth:`add`. **Raises:** * `UninitializedError`: If :meth:`fetch` has not been called on this manager yet. #### `remove`[​](#remove "Direct link to remove") ``` remove(self, data: str | RelatedNode | dict) -> None ``` Remove a peer from this relationship. The peer to remove is matched first by ID, then by HFID. When no match is found, the call is a no-op. **Args:** * `data`: The peer to remove. Accepts an ID string, an existing :class:`RelatedNode`, or a dict describing the peer. **Raises:** * `UninitializedError`: If :meth:`fetch` has not been called on this manager yet. * `IndexError`: If the internal peer index is inconsistent with the lookup result. ### `RelationshipManagerSync`[​](#relationshipmanagersync "Direct link to relationshipmanagersync") Synchronous manager for a cardinality-many relationship. Synchronous counterpart of :class:`RelationshipManager`. Extends :class:`RelationshipManagerBase` with the ability to populate and edit the peer set against an :class:`InfrahubClientSync`: :meth:`fetch` resolves every peer in a parallel batch and :meth:`add`, :meth:`extend`, and :meth:`remove` mutate the peer list in memory. Peers are exposed as :class:`RelatedNodeSync` instances and can be accessed by index via `manager[i]`. **Methods:** #### `fetch`[​](#fetch-1 "Direct link to fetch-1") ``` fetch(self) -> None ``` Populate the peer set and resolve every peer to a full node. When the manager is not yet initialized, the parent node is re-queried with this relationship included so the peer list can be populated. The peers are then fetched in a parallel batch grouped by kind and stored in the client store. **Raises:** * `Error`: If any peer is missing an `id` or `typename` and cannot be resolved. #### `add`[​](#add-1 "Direct link to add-1") ``` add(self, data: str | RelatedNodeSync | dict) -> None ``` Add a new peer to this relationship. The new peer is only added when its ID or HFID is not already present; duplicate adds are silently ignored. **Args:** * `data`: The peer to add. Accepts an ID string, an existing :class:`RelatedNodeSync`, or a dict describing the peer (with `id` or `hfid` keys, plus optional relationship properties). **Raises:** * `UninitializedError`: If :meth:`fetch` has not been called on this manager yet. #### `extend`[​](#extend-1 "Direct link to extend-1") ``` extend(self, data: Iterable[str | RelatedNodeSync | dict]) -> None ``` Add new peers to this relationship. This is a convenience wrapper that calls :meth:`add` for every item in `data`. Items already present (by ID or HFID) are silently ignored. **Args:** * `data`: The peers to add, in any of the formats accepted by :meth:`add`. **Raises:** * `UninitializedError`: If :meth:`fetch` has not been called on this manager yet. #### `remove`[​](#remove-1 "Direct link to remove-1") ``` remove(self, data: str | RelatedNodeSync | dict) -> None ``` Remove a peer from this relationship. The peer to remove is matched first by ID, then by HFID. When no match is found, the call is a no-op. **Args:** * `data`: The peer to remove. Accepts an ID string, an existing :class:`RelatedNodeSync`, or a dict describing the peer. **Raises:** * `UninitializedError`: If :meth:`fetch` has not been called on this manager yet. * `IndexError`: If the internal peer index is inconsistent with the lookup result. --- # Manage data with Object files ## Introduction[​](#introduction "Direct link to Introduction") An Object file is a YAML file that allows you to manage data to be loaded in Infrahub based on your own custom schema. It provides a declarative way to define and manage resources in your Infrahub instance. Object files work well for models that don't change too often and/or that need to be tracked in Git. Examples include: Groups, tags, Users, etc. Below is an example of an Object file that defines tags (`BuiltinTag`). ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: BuiltinTag data: - name: Blue - name: Yellow - name: Red ``` Object files are meant to be used in an idempotent way and as such they work better for models with a Human Friendly ID (HFID) defined. An HFID is a unique identifier that makes it easier to reference objects across different files and operations. ## Load Object files into Infrahub[​](#load-object-files-into-infrahub "Direct link to Load Object files into Infrahub") Object files can be loaded into Infrahub using the `infrahubctl object load` command. ``` infrahubctl object load ``` Multiple object files can be loaded at once by specifying the path to multiple files or by specifying a directory. The `object load` command will create/update the objects using an `Upsert` operation. All objects previously loaded will NOT be deleted in the Infrahub instance. Also, if some objects present in different files are identical and dependent on each other, the `object load` command will NOT calculate the dependencies between the objects and as such it's the responsibility of the users to execute the command in the right order. > Object files can also be loaded into Infrahub when using external Git repositories. To see how to do this, please refer to the [.infrahub.yml](https://docs.infrahub.app/topics/infrahub-yml) documentation. ### Validate the format of object files[​](#validate-the-format-of-object-files "Direct link to Validate the format of object files") The object file can be validated using the `infrahubctl object validate` command. ``` infrahubctl object validate ``` ## Object file format[​](#object-file-format "Direct link to Object file format") All object files must start with the following format, all other formats will be automatically ignored. Each file is intended for one specific top level kind, but one file can include multiple nested objects of any kind. The kind of the top level object must be defined in spec/kind. ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: parameters: expand_range: # Optional, defaults to false data: - [...] ``` > Multiple documents in a single YAML file are also supported, each document will be loaded separately. Documents are separated by `---` ### Data processing parameters[​](#data-processing-parameters "Direct link to Data processing parameters") The `parameters` field controls how the data in the object file is processed before loading into Infrahub: | Parameter | Description | Default | | -------------- | -------------------------------------------------------------------------------------------------------------- | ------- | | `expand_range` | When set to `true`, range patterns (for example, `[1-5]`) in string fields are expanded into multiple objects. | `false` | When `expand_range` is not specified, it defaults to `false`. ### Relationship of cardinality one[​](#relationship-of-cardinality-one "Direct link to Relationship of cardinality one") A relationship of cardinality one can either reference an existing node via its HFID or create a new node if it doesn't exist. In the example below, both `site` and `primary_ip` are relationships of cardinality one. ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: InfraDevice data: - name: edge01 site: "Paris" # Reference existing node via its HFID primary_ip: # Nested object, will be created if it doesn't exist data: address: "192.168.1.1" ``` ### Relationship of cardinality many[​](#relationship-of-cardinality-many "Direct link to Relationship of cardinality many") A relationship of cardinality many can reference existing nodes via their HFID or define nested objects. #### Existing nodes referenced by their HFID[​](#existing-nodes-referenced-by-their-hfid "Direct link to Existing nodes referenced by their HFID") Existing nodes can be referenced by their HFID in string format or in list format. In the example below, both `best_friends` and `tags` are relationships of cardinality many. > An HFID is composed of a single value, it's possible to use a string instead of a list ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: TestingPerson data: - name: Mike Johnson height: 175 best_friends: # Relationship of cardinality many that references existing nodes based on their HFID - [Jane Smith, Max] - [Sarah Williams, Charlie] tags: - Veterinarian # Existing Node referenced by its HFID in string format - [Breeder] # Existing Node referenced by its HFID in list format ``` #### Nested objects[​](#nested-objects "Direct link to Nested objects") When defining nested objects, the node will be automatically created if it doesn't exist and if the relationship between the parent object and the nested object exists, it will be automatically inserted. For example, in the example below, the `owner` of a `TestingDog` doesn't need to be specified because it will be automatically inserted. Two different syntax are supported: * A dictionary with multiple values under data * A list of objects ##### Nested objects as a dictionary[​](#nested-objects-as-a-dictionary "Direct link to Nested objects as a dictionary") In the example below, `tags` is a relationship of cardinality many that is defined as a dictionary with multiple values under data. > The kind is optional here because there is only one option possible (not a generic) ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: TestingPerson data: - name: Alex Thompson tags: data: - name: dog-lover description: "Dog Lover" - name: veterinarian description: "Veterinarian" ``` This format works well when all objects are of the same kind and when all objects are using the same properties. For more complex cases, the list of objects format is more flexible. ##### Nested objects as a list of objects[​](#nested-objects-as-a-list-of-objects "Direct link to Nested objects as a list of objects") In the example below, `animals` is a relationship of cardinality many that is defined as a list of objects. Each object must contain a `data` key and each object can also define a specific `kind`. > If the kind is not specified, it will be inferred from schema ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: TestingPerson data: - name: Alex Thompson height: 180 animals: - kind: TestingDog data: name: Max weight: 25 breed: Golden Retriever color: "#FFD700" - kind: TestingCat data: name: Mimi breed: Persian ``` ### Support for metadata[​](#support-for-metadata "Direct link to Support for metadata") Metadata support is planned for future releases. Currently, the Object file does not support metadata on attributes or relationships. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Common issues[​](#common-issues "Direct link to Common issues") 1. **Objects not being created**: Ensure that the YAML syntax is correct and that the file follows the required format. 2. **Dependency errors**: When objects depend on each other, load them in the correct order (dependencies first). 3. **Validation errors**: Use the `infrahubctl object validate` command to check for syntax errors before loading. ### Best practices[​](#best-practices "Direct link to Best practices") 1. Use Human Friendly IDs (HFIDs) for all objects to ensure consistent referencing. 2. Keep object files organized by model type or purpose. 3. Validate object files before loading them into production environments. 4. Use comments in your YAML files to document complex relationships or dependencies. ## Range expansion in object files[​](#range-expansion-in-object-files "Direct link to Range expansion in object files") The Infrahub Python SDK supports **range expansion** for string fields in object files when the `parameters > expand_range` is set to `true`. This feature allows you to specify a range pattern (for example, `[1-5]`) in any string value, and the SDK will automatically expand it into multiple objects during validation and processing. ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: BuiltinLocation parameters: expand_range: true # Enable range expansion data: - name: AMS[1-3] type: Country ``` ### How range expansion works[​](#how-range-expansion-works "Direct link to How range expansion works") * Any string field containing a pattern like `[1-5]`, `[10-15]`, or `[1,3,5]` will be expanded into multiple objects. * If multiple fields in the same object use range expansion, **all expanded lists must have the same length**. If not, validation will fail. * The expansion is performed before validation and processing, so all downstream logic works on the expanded data. ### Examples[​](#examples "Direct link to Examples") #### Single field expansion[​](#single-field-expansion "Direct link to Single field expansion") ``` spec: kind: BuiltinLocation parameters: expand_range: true data: - name: AMS[1-3] type: Country ``` This will expand to: ``` - name: AMS1 type: Country - name: AMS2 type: Country - name: AMS3 type: Country ``` #### Multiple field expansion (matching lengths)[​](#multiple-field-expansion-matching-lengths "Direct link to Multiple field expansion (matching lengths)") ``` spec: kind: BuiltinLocation parameters: expand_range: true data: - name: AMS[1-3] description: Datacenter [A-C] type: Country ``` This will expand to: ``` - name: AMS1 description: Datacenter A type: Country - name: AMS2 description: Datacenter B type: Country - name: AMS3 description: Datacenter C type: Country ``` #### Error: mismatched range lengths[​](#error-mismatched-range-lengths "Direct link to Error: mismatched range lengths") If you use ranges of different lengths in multiple fields: ``` spec: kind: BuiltinLocation parameters: expand_range: true data: - name: AMS[1-3] description: "Datacenter [10-15]" type: Country ``` This will **fail validation** with an error like: ``` Range expansion mismatch: fields expanded to different lengths: [3, 6] ``` ### Notes[​](#notes "Direct link to Notes") * Range expansion is supported for any string field in the `data` section. * If no range pattern is present, the field is left unchanged. * If expansion fails for any field, validation will fail with an error message. --- # Understanding tracking in the Python SDK ## Introduction[​](#introduction "Direct link to Introduction") Tracking mode in the Python SDK is an advanced functionality designed to optimize and streamline data operations. ## The essence of tracking[​](#the-essence-of-tracking "Direct link to The essence of tracking") ### What is tracking?[​](#what-is-tracking "Direct link to What is tracking?") Tracking mode in the Python SDK is aggregates operations, such as creations, updates, and deletions, performed during a session into a `CoreStandardGroup` object. This aggregation facilitates efficient management and auditing of data mutations. ### Why it matters?[​](#why-it-matters "Direct link to Why it matters?") In complex workflows where multiple operations occur sequentially, tracking ensures that these actions are consolidated, leading to easier auditing. This is particularly crucial in scenarios where operations must be repeatable without side effects — a core principle of idempotency. ## Under the hood[​](#under-the-hood "Direct link to Under the hood") At the heart of the tracking feature lies the `CoreStandardGroup` object. This object acts as a container, grouping related operations to provide a unified view of the changes made during a session. This grouping is not just for organizational purposes; it plays a vital role in enabling idempotent operations. ### Automatic vs. manual tracking[​](#automatic-vs-manual-tracking "Direct link to Automatic vs. manual tracking") The SDK offers two modes of operation for tracking: automatic and manual: * Automatic tracking is facilitated through the use of a context manager, simplifying session management and ensuring that all operations within the context are tracked. * Manual tracking, on the other hand, provides granular control, allowing developers to start and end sessions explicitly and manage group memberships as needed. ## Achieving idempotency[​](#achieving-idempotency "Direct link to Achieving idempotency") ### Idempotency explained[​](#idempotency-explained "Direct link to Idempotency explained") Idempotency is a property of certain operations that ensures the operation can be performed multiple times without changing the result beyond the initial application. This concept is critical in ensuring that scripts and operations are safe to retry, leading to more robust and reliable systems. ### Tracking in idempotency[​](#tracking-in-idempotency "Direct link to Tracking in idempotency") The tracking feature of the Python SDK shines in scenarios requiring idempotency. By grouping operations within a `CoreStandardGroup`, the SDK allows for operations to be rolled back or repeated without adverse effects, ensuring that the state of the system remains consistent. ## Practical applications[​](#practical-applications "Direct link to Practical applications") ### Use cases[​](#use-cases "Direct link to Use cases") * **Auditing:** Tracking provides a clear audit trail of operations performed during a session. * **Error Handling:** In case of errors, tracking enables precise identification and rollback of changes. ### Implementing an idempotent script[​](#implementing-an-idempotent-script "Direct link to Implementing an idempotent script") An idempotent script using the Python SDK might involve the following steps: 1. **Start a Tracking Session:** Begin with a unique identifier and any relevant parameters. 2. **Perform Operations:** Execute create, update, or delete operations as required. 3. **End the Session:** Conclude the tracking session, optionally updating the group context to reflect the changes made. 4. **Handle Errors:** In case of errors, use the tracking information to rollback or correct the state. For more detailed guidance on implementing tracking in your scripts, refer to the [tracking guide](/python-sdk/guides/tracking.md). --- # Schema library for Infrahub Welcome to the Schema Library for Infrahub! This repository offers a collection of schemas designed to streamline and standardize infrastructure-related data structures. warning This project is currently a collection of examples intended to serve as inspiration. Please note that it is in an experimental phase and may undergo significant changes. ## Use case[​](#use-case "Direct link to Use case") Infrahub ships without built-in schemas, giving you complete freedom to define your own. However, starting from scratch can be overwhelming, especially when many infrastructures share common models. That's where the [Schema Library](https://github.com/opsmill/schema-library), maintained by OpsMill and the community, comes in. **The library provides a curated collection of practical, ready-to-use schemas designed to reflect real-world infrastructure needs. Use them out of the box, or treat them as a foundation: copy, adapt, and extend each schema to align precisely with your organization.** ## Project structure[​](#project-structure "Direct link to Project structure") This project is divided into three main parts: * **Base**: The foundational layer required for most extensions. It must be loaded before adding extensions. * **Extensions**: Generic schema components for managing infrastructure. Extensions may have dependencies on each other. * **Experimental**: Schema components that are not yet fully supported. ## Getting started[​](#getting-started "Direct link to Getting started") ### Clone the repository[​](#clone-the-repository "Direct link to Clone the repository") ``` git clone https://github.com/opsmill/schema-library.git cd schema-library ``` note One of Infrahub's key strengths is its flexibility. **We encourage you to copy any schemas you find useful into your own repository** and tailor them to your specific needs, ensuring your schema aligns perfectly with your requirements. ### How to load schemas[​](#how-to-load-schemas "Direct link to How to load schemas") There are two main ways to load schemas into Infrahub, depending on where you are in your Infrahub journey: * **Quick View**: To quickly view a schema, you can use `infrahubctl`. Please reference the [`infrahubctl` Guide](https://docs.infrahub.app/infrahubctl) for further instructions. * **Controlled Integration**: For a more organized and unified approach, you can connect a Git repository. Follow [this guide](https://docs.infrahub.app/guides/repository) to connect a Git repository. note For proof of concept or development purposes, using `infrahubctl` is the preferred approach. When running Infrahub in production, you might need the extra control provided by Git integration to load schemas. ### Load schemas[​](#load-schemas "Direct link to Load schemas") important You need to load the `base` schema before loading any extensions. Extensions may have dependencies on each other, so you'll need to load them in the correct order. ``` # Load the base infrahubctl schema load base # Load an extension infrahubctl schema load extensions/location_minimal # Load another extension infrahubctl schema load extensions/vlan ``` ## Available schemas[​](#available-schemas "Direct link to Available schemas") This list provides an overview of the schemas available in this repository. Each schema is designed to address specific infrastructure needs, and you can find detailed documentation for each one by clicking its name. ### base[​](#base "Direct link to base") | Name | Description | | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **[DCIM](/schema-library/reference/dcim.md)** | Base DCIM schema: device types, platforms, devices, and physical and virtual interfaces, plus the endpoint and connector generics that cables, circuits and cross-connects build on. | | **[IPAM](/schema-library/reference/ipam.md)** | Base IPAM schema: IP prefixes and IP addresses, plus the scope generics a prefix or a VLAN group can be attached to. | | **[Location](/schema-library/reference/location.md)** | Base Location schema: the generic every location tier inherits from, and the hosting generic that lets devices and other objects be placed at a location. Concrete tiers such as Site come from the location extensions. | | **[Organization](/schema-library/reference/organization.md)** | Base Organization schema: the generic organization plus Manufacturer and Provider nodes. | ### experimental[​](#experimental "Direct link to experimental") | Name | Description | | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **[Azure](/schema-library/reference/azure.md)** | This schema extension introduces cloud support for Microsoft Azure. | | **[Circuit Service](/schema-library/reference/circuit_service.md)** | This schema extension contains model coming on top of circuit to capture a single service shared across multiple circuits. For example you have a MPLS network supported by a provider connecting multiple locations: - One single CircuitService would be needed to store MPLS related information (e.g. service id, provider ...) - On each site we would create a circuit connecting on one side our device and the CircuitService on the other side | | **[Firewall Policer](/schema-library/reference/firewall_policer.md)** | This schema extension contains models for traffic policers and the classes and rules that reference them, so you can describe rate limiting alongside the interfaces it is applied to. | | **[InfiniBand Fabric Hardware](/schema-library/reference/infiniband.md)** | This schema extension models the hardware of an InfiniBand fabric: switches with their configuration profile (AAA, role, split-ready, SSH and CLI settings), switch and management interfaces with lane width, split and speed options, rail-optimized scalable units (RSU) and the fabric that groups them. It does not model the InfiniBand protocol itself (subnet manager, partitions, routing). | | **[Location Extended](/schema-library/reference/location_extended.md)** | This schema extension is the most detailed when it comes to location, you'll find all the layers you can think of. It defines its own hierarchical Location.Rack, incompatible with the flat one extensions/rack defines, so load one or the other, not both. | | **[Modules Linecards](/schema-library/reference/modules_linecards.md)** | This schema extension allows you to capture Linecard related information like the version. You can insert the Linecard into a Dcim Physical Device and leverage the Linecard type model. The Linecard can accept PIC to help configure PORT information like breakout-capabilities and configurations. | | **[Modules Routing Engine](/schema-library/reference/modules_routing_engine.md)** | This schema extension allows you to capture Routing Engine related information like the version. You can insert the Routing Engine into a Dcim Physical Device and leverage the Routing Engine type model. | | **[Optical Transport](/schema-library/reference/optical_transport.md)** | Comprehensive optical transport network schemas for DWDM/WDM systems (ADVA FSP 3000 and similar platforms). Covers four layers: wavelength (ITU-T G.694.1 grid, optical bands, DWDM channels), topology (logical optical nodes, passive multiplexers, fiber links), equipment (transponder/amplifier/ROADM modules, ROADM degrees, WSS cross-connects), and service (end-to-end optical services, optical paths, path segments). Not designed to be loaded together with extensions/optical\_multiplexer. | | **[QoS](/schema-library/reference/qos.md)** | This schema extension contains models for Quality of Service (QoS) | | **[Security](/schema-library/reference/security.md)** | This schema extension contains models for implementing detailed security. | | **[Topology](/schema-library/reference/topology.md)** | A schema for defining and managing network topology, strategies, and services. | | **[VLAN Translation](/schema-library/reference/vlan-translation.md)** | This schema extension is based on Juniper VLAN MAP, and not yet test out for other vendors. | ### extensions[​](#extensions "Direct link to extensions") | Name | Description | | ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **[Cable](/schema-library/reference/cable.md)** | This schema extension contains a basic Cable model allowing you to connect two endpoints. | | **[Circuit](/schema-library/reference/circuit.md)** | This schema extension adds Circuit and Circuit Endpoint nodes and links them to your infrastructure. A circuit could be a fiber connecting two sites, with one endpoint on each site. | | **[Circuit Contract](/schema-library/reference/circuit_contract.md)** | This schema extension adds a Circuit Contract node to capture the service agreement behind a circuit with its provider. | | **[Cluster](/schema-library/reference/cluster.md)** | This schema extension contains the foundations to capture clusters. With this one in place you can unlock various cluster flavours (a hosting cluster able to host VMs, a firewall cluster built with specific appliances ...). | | **[Compute](/schema-library/reference/compute.md)** | This schema extension captures physical servers and virtual machines, with a generic compute unit both inherit from and a generic for anything able to host VMs. Pair it with extensions/hosting\_cluster to group compute units into clusters. | | **[Cross-Connect](/schema-library/reference/cross_connect.md)** | This schema extension adds a Cross-Connect node. Think of it as a cable operated by a provider: attach it to a location, then connect endpoints to it (e.g. the rear interface of a patch panel or a circuit endpoint). | | **[Device Module](/schema-library/reference/device_module.md)** | This schema extension lets you capture module bays on a DcimPhysicalDevice and the modules installed into them (e.g. fan trays, line cards). A device can have any number of module bays, and each bay can optionally hold one module. It ships a ready-to-use Module and Module Type pair, plus the generic Module and Module Type you can build your own flavours on: see extensions/device\_module\_psu, or the experimental modules\_linecards and modules\_routing\_engine extensions. | | **[Hosting Cluster](/schema-library/reference/hosting_cluster.md)** | A generic cluster built from compute units (e.g. servers) that can host virtual machines. | | **[Interface Breakout](/schema-library/reference/interface_breakout.md)** | This schema extension introduces relationships to support breakout interfaces, enabling you to document the breakout of a physical interface into smaller physical interfaces. | | **[Internet Exchange](/schema-library/reference/peering_ixp.md)** | This schema extension adds an Internet Exchange (IXP) node and an IXP Connection node linking your devices to an exchange, and ties IXP connections to the BGP sessions established over them. | | **[IPAM Aggregate](/schema-library/reference/ipam_aggregate.md)** | This schema extension captures the top-level IPv4 and IPv6 blocks assigned by a Regional Internet Registry (RIR). It adds an Aggregate node for the block and an RIR node for the registry or private entity that manages the address space. | | **[LAG](/schema-library/reference/lag.md)** | This schema extension includes models for Link Aggregation Groups (LAGs), enabling you to link physical interfaces as building blocks of your LAG interface. It can be used in standard networking environments as well as in compute scenarios, such as capturing bond interfaces. | | **[Location Minimal](/schema-library/reference/location_minimal.md)** | This schema extension provides a self-contained Region -> Country -> Metro -> Site hierarchy for storing location data, with the Site carrying facility, physical address, timezone and status. Its Site node is the same as the one in extensions/location\_site, so the two can be loaded together. A location name is unique across every tier, so a single-country deployment should enter the hierarchy at Country, with Region as the national node, rather than repeating a country under several regions. | | **[Location Site](/schema-library/reference/location_site.md)** | This schema extension introduces a Site node with facility, physical address, timezone and status, for deployments that want a flat list of sites without a hierarchy. It is the same Site node as in extensions/location\_minimal, which adds Region, Country and Metro tiers above it. | | **[MLAG](/schema-library/reference/mlag.md)** | This schema extension contains the foundations to capture Multi-Chassis Link Aggregation Groups (MLAG). It comes on top of the LAG extension. In this implementation, a MLAG interface is essentially a LAG interface but linked to a MLAG domain (instead of a device). The MLAG domain regroups devices together (usually 2) and is built over LAG interfaces used as peer-link between the devices. MLAG interfaces defined at the MLAG domain level are then spread across all devices in the domain. This is a deliberately minimal implementation of MLAG, meant to blend with models you already have. For example, in a data center fabric you might already have a LeafGroup or similar concept: have it inherit from GenericMlagDomain and add the relationships and attributes you need. Not covered yet: the layer 3 overlay for MLAG interfaces (loopback, peer address ...) and surfacing MLAG interfaces on each device in the domain. | | **[Module Port](/schema-library/reference/module_port.md)** | This schema extension adds module ports: the ports a module provides, as declared by its module type - what NetBox module-type definitions list under `interfaces`, `console-ports` and `power-ports`. These are deliberately not DcimInterface objects. DcimInterface.device is a mandatory Parent, and Infrahub requires the relationships used in a uniqueness constraint to be mandatory, so an interface cannot hang off a module instead of a device. A DcimModulePort is a declaration parented by the module, carrying the port name, its category (interface, console, power, front, rear), the NetBox type slug, and power draw. NOTE: port names keep NetBox's `{module}` bay-position token verbatim, because a template is not bound to a bay. Substituting it and creating the real device interfaces is a generator step once the module is installed. | | **[Optical Multiplexer](/schema-library/reference/optical_multiplexer.md)** | This schema extension models optical add-drop multiplexers (OADM) and the wavelength division multiplexing (WDM) channels they carry, for both CWDM and DWDM. It adds an Optical Multiplexer device with front and rear interfaces, a WDM Channel node holding channel number, wavelength and frequency, and a WDM Transceiver flavour of the transceiver model tuned to a channel. Some vendors configure tunable optics by wavelength or frequency rather than channel number; the WDM Channel node gives you a single entry in Infrahub for all three. Not designed to be loaded together with experimental/optical\_transport, which covers the same domain in more depth. | | **[Patch Panel](/schema-library/reference/patch_panel.md)** | This schema extension allows you to capture patch panel related information like rear and front interfaces and the mapping between them. You can insert the patch panel into a rack and leverage the device type model. Cassettes and other inserts are tracked as regular device modules in module bays, through extensions/device\_module. The front and rear interfaces accept all sorts of connectors, so you can plug cables, circuits and cross-connects into them. | | **[Physical Disk](/schema-library/reference/physical_disk.md)** | Simple schema allowing you to capture physical disk information for inventory and lifecycle management. This extension works with any kind of device: apply the DcimDeviceWithPhysicalDisks generic to a model to enable disk tracking. You might also link disks to a location, for instance to capture spares. | | **[PSU Module](/schema-library/reference/device_module_psu.md)** | This schema extension adds a PSU (Power Supply Unit) flavour on top of the generic Module and Module Type from extensions/device\_module, so you can track power supplies installed in a device's module bays with PSU-specific attributes such as wattage and hot-swap capability. | | **[QinQ](/schema-library/reference/qinq.md)** | This schema extension adds the Service VLAN and Customer VLAN nodes needed to model QinQ on top of the VLAN extension. | | **[Rack](/schema-library/reference/rack.md)** | This schema extension introduces a Rack node representing a physical equipment rack installed at a site. Racks are placed at a LocationSite, which comes from extensions/location\_site or extensions/location\_minimal, and can host devices and other equipment. experimental/location\_extended defines its own Rack node, so do not load both. | | **[Routing](/schema-library/reference/routing.md)** | This schema extension contains the Routing Protocol generic that every protocol extension builds on. The idea is one protocol instance per IpamVRF and DcimDevice pair. | | **[Routing Aggregate](/schema-library/reference/routing_aggregate.md)** | This schema extension adds an Aggregate Route node: a summary route, with its destination prefix and discard flag, announced by a routing protocol instance. | | **[Routing BGP](/schema-library/reference/routing_bgp.md)** | This schema extension adds Autonomous System, BGP Peer Group and BGP Session nodes, and links autonomous systems to devices and organizations. | | **[Routing BGP Community](/schema-library/reference/routing_bgp_community.md)** | This schema extension adds a BGP Community node to reference from routing policies. | | **[Routing BGP Route Reflector](/schema-library/reference/routing_bgp_rr.md)** | This schema extension adds a Route Reflector Cluster node on top of the BGP extension and links it to BGP peer groups. | | **[Routing OSPF](/schema-library/reference/routing_ospf.md)** | This schema extension adds OSPF and OSPF Interface nodes to model an OSPF instance per device and VRF and the interfaces it runs on. | | **[Routing PIM](/schema-library/reference/routing_pim.md)** | This schema extension adds PIM (Protocol Independent Multicast) and PIM Interface nodes to model a PIM instance per device and VRF and the interfaces it runs on. | | **[Routing Policies](/schema-library/reference/routing_policies.md)** | This schema extension contains the Routing Policy generic. Each routing protocol extension subclasses it with its own policy flavour. | | **[Routing Policies Aggregate](/schema-library/reference/routing_policies_aggregate.md)** | This schema extension adds an Aggregate Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on Aggregate Route with relationships to it. | | **[Routing Policies BGP](/schema-library/reference/routing_policies_bgp.md)** | This schema extension adds a BGP Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on BGP Peer Group and BGP Session with relationships to it. | | **[Routing Policies OSPF](/schema-library/reference/routing_policies_ospf.md)** | This schema extension adds an OSPF Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on OSPF with relationships to it. | | **[Routing Policies PIM](/schema-library/reference/routing_policies_pim.md)** | This schema extension adds a PIM Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on PIM with relationships to it. | | **[SNMP](/schema-library/reference/snmp.md)** | This schema extension contains models for SNMP Communities and SNMP Clients. A community links to the devices it is configured on; any further wiring, to a tenant for instance, is left to you. | | **[Tenancy](/schema-library/reference/tenancy.md)** | This schema extension introduces a Tenant node that can own devices, IP prefixes and addresses, and hosting locations. See the comment at the bottom of tenancy.yml for an example of how to extend tenancy onto optional extension nodes such as DcimCircuit. | | **[Transceiver](/schema-library/reference/transceiver.md)** | This schema extension models pluggable transceivers across form factors (SFP, SFP+, QSFP, QSFP28, QSFP-DD, OSFP, CFP, XFP and more), with a Standard Transceiver and a Bidirectional Transceiver flavour built on a shared generic. A transceiver is either plugged into a physical interface or stored as a spare at a location, and can be linked to its manufacturer. Limitations: there is no validation between type, form factor, protocol and distance; any transceiver can be plugged into any physical interface; and nothing prevents a transceiver from being linked to both an interface and a spare location. | | **[VLAN](/schema-library/reference/vlan.md)** | This schema extension adds VLAN and VLAN Group nodes, with a generic VLAN to build on, and links VLANs to prefixes and layer 2 interfaces. | | **[VRF](/schema-library/reference/vrf.md)** | This schema extension adds VRF and Route Target nodes and links prefixes and IP addresses to a VRF. | | **[VRRP](/schema-library/reference/vrrp.md)** | This schema extension adds VRRP Group and VRRP Interface nodes, linking a virtual IP address to the layer 3 interfaces that take part in the group. | ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") * **`Unable to load the schema: xxx unable to find the generic yyy` - Missing requirements** This error indicates that the schema you are trying to load has dependencies on other schemas that are not present in your Infrahub instance. success To resolve this, ensure that you have loaded all required schemas before attempting to load the one in question. You can find the list of dependencies in the schema documentation. --- # Contributing We welcome contributions and feedback! Please open an issue or submit a pull request to suggest additions, improvements, or to report bugs. ## Adding a new schema[​](#adding-a-new-schema "Direct link to Adding a new schema") To add a new schema: * Create a directory in either `experimental` or `extension` with the schema name (ensure the name is unique). * Add a `.yml` file using the [Infrahub schema format](https://docs.infrahub.app/reference/schema). * Update the `.metadata.yml` file to include your schema, providing its `name`, `description`, and `dependencies`. * To verify integration, run `invoke schemas.load-all-schemas` to load the entire schema library in your local Infrahub instance. To add documentation for your schema: * Run `invoke docs.generate` to generate documentation files. * Then run `invoke docs.build` and `invoke docs.serve` to build and serve the documentation locally. ## Documentation[​](#documentation "Direct link to Documentation") Most documentation is generated automatically: * The `docs.py` task generates documentation. * `home.mdx` is generated from the `_templates/home_page.j2` template. * Each file in the `reference` directory is generated from the `_templates/schema_reference.j2` template. important To modify the documentation, edit the templates in the `_templates` directory. After making changes, run `invoke docs.generate` to update the documentation. --- # Azure This schema extension introduces cloud support for Microsoft Azure. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * **Attribution:** [Rowan Coleman](https://www.linkedin.com/in/rowan-coleman-6a147156/) ## Nodes[​](#nodes "Direct link to Nodes") ### Location[​](#location "Direct link to Location") * **Namespace:** Azure #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | ### Tenant[​](#tenant "Direct link to Tenant") * **Namespace:** Azure #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | | tenant\_id | | Text | | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | ----------------- | -------- | ----------- | --------- | | subscriptions | AzureSubscription | | many | Component | ### Subscription[​](#subscription "Direct link to Subscription") * **Namespace:** Azure #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | | subscription\_id | | Text | | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------- | ------------------ | -------- | ----------- | --------- | | tenant | AzureTenant | False | one | Parent | | resourcegroups | AzureResourceGroup | | many | Component | ### ResourceGroup[​](#resourcegroup "Direct link to ResourceGroup") * **Namespace:** Azure #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ----------------- | -------- | ----------- | --------- | | location | AzureLocation | | one | Attribute | | subscription | AzureSubscription | False | one | Parent | ### VirtualNetwork[​](#virtualnetwork "Direct link to VirtualNetwork") * **Label:** Virtual Networks * **Namespace:** Azure * **Inherit From:** AzureResource #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------- | ------------------------- | -------- | ----------- | --------- | | address\_space | BuiltinIPPrefix | | many | Attribute | | subnets | AzureVirtualNetworkSubnet | | many | Component | ### VirtualNetworkSubnet[​](#virtualnetworksubnet "Direct link to VirtualNetworkSubnet") * **Label:** Subnets * **Namespace:** Azure #### Attributes[​](#attributes-4 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------------- | ------------------- | -------- | ----------- | --------- | | virtualnetwork | AzureVirtualNetwork | False | one | Parent | | address\_prefixes | BuiltinIPPrefix | | many | Attribute | ## Generics[​](#generics "Direct link to Generics") ### Resource[​](#resource "Direct link to Resource") * **Label:** Azure * **Namespace:** Azure #### Attributes[​](#attributes-5 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | #### Relationships[​](#relationships-5 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | ------------------ | -------- | ----------- | --------- | | location | AzureLocation | | one | Attribute | | resourcegroup | AzureResourceGroup | False | one | Parent | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: Resource namespace: Azure label: Azure include_in_menu: true attributes: - name: name kind: Text relationships: - name: location cardinality: one kind: Attribute peer: AzureLocation - name: resourcegroup cardinality: one peer: AzureResourceGroup kind: Parent optional: false nodes: - name: Location namespace: Azure menu_placement: AzureResource display_label: name__value attributes: - name: name kind: Text - name: Tenant namespace: Azure menu_placement: AzureResource display_label: '{{ name__value }} (Tenant ID: {{ tenant_id__value }})' attributes: - name: name kind: Text - name: tenant_id kind: Text relationships: - name: subscriptions cardinality: many peer: AzureSubscription kind: Component - name: Subscription namespace: Azure menu_placement: AzureResource display_label: '{{ name__value }} (Subscription ID: {{ subscription_id__value }})' attributes: - name: name kind: Text - name: subscription_id kind: Text relationships: - name: tenant cardinality: one peer: AzureTenant kind: Parent optional: false - name: resourcegroups cardinality: many peer: AzureResourceGroup kind: Component - name: ResourceGroup namespace: Azure menu_placement: AzureResource display_label: name__value attributes: - name: name kind: Text relationships: - name: location cardinality: one kind: Attribute peer: AzureLocation - name: subscription cardinality: one peer: AzureSubscription kind: Parent optional: false - name: VirtualNetwork label: Virtual Networks namespace: Azure menu_placement: AzureResource display_label: name__value inherit_from: - AzureResource relationships: - name: address_space cardinality: many kind: Attribute peer: BuiltinIPPrefix - name: subnets cardinality: many kind: Component peer: AzureVirtualNetworkSubnet - name: VirtualNetworkSubnet label: Subnets namespace: Azure menu_placement: AzureResource include_in_menu: true display_label: name__value attributes: - name: name kind: Text relationships: - name: virtualnetwork cardinality: one peer: AzureVirtualNetwork kind: Parent optional: false - name: address_prefixes cardinality: many kind: Attribute peer: BuiltinIPPrefix ``` --- # Cable This schema extension contains a basic Cable model allowing you to connect two endpoints. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Cable[​](#cable "Direct link to Cable") * **Label:** Cable * **Description:** Physical cable connecting two endpoints * **Namespace:** Dcim * **Icon:** mdi :cable-data * **Human Friendly ID:** label\_\_value * **Inherit From:** DcimConnector #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | -------- | -------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | status | | Dropdown | False | connected | connected, planned | | cable\_type | | Dropdown | True | | cat3, cat5, cat5e, cat6, cat6a, cat7, cat7a, cat8, mmf\_om1, mmf\_om2, mmf\_om3, mmf\_om4, mmf\_om5, smf\_os1, smf\_os2, dac\_passive, dac\_active, aoc, coaxial, mrj21\_trunk, power, usb | | color | | Dropdown | True | | dark\_red, red, pink, rose, fuchsia, purple, dark\_purple, indigo, blue, light\_blue, cyan, teal, aqua, dark\_green, green, light\_green, lime, yellow, amber, orange, dark\_orange, brown, light\_grey, grey, dark\_grey, black | | label | | Text | True | | | | length | | Number | True | | | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Cable namespace: Dcim description: Physical cable connecting two endpoints label: Cable icon: mdi:cable-data include_in_menu: true menu_placement: DcimDevice inherit_from: - DcimConnector human_friendly_id: - label__value order_by: - label__value display_label: label__value attributes: - name: status kind: Dropdown default_value: connected choices: - name: connected label: Connected description: Fully operational and currently connected color: '#00d25b' - name: planned label: Planned description: In the process of being set up color: '#2196f3' optional: false order_weight: 1200 - name: cable_type kind: Dropdown choices: - name: cat3 label: CAT3 description: Category 3 twisted pair cable. color: '#99ccff' - name: cat5 label: CAT5 description: Category 5 twisted pair cable. color: '#99ccff' - name: cat5e label: CAT5e description: Enhanced Category 5 cable. color: '#99ccff' - name: cat6 label: CAT6 description: Category 6 twisted pair cable. color: '#99ccff' - name: cat6a label: CAT6A description: Augmented Category 6 cable. color: '#99ccff' - name: cat7 label: CAT7 description: Category 7 twisted pair cable. color: '#99ccff' - name: cat7a label: CAT7A description: Augmented Category 7 cable. color: '#99ccff' - name: cat8 label: CAT8 description: Category 8 twisted pair cable. color: '#99ccff' - name: mmf_om1 label: Multimode Fiber OM1 description: "Multimode fiber (62.5/125 \xB5m)." color: '#ff9933' - name: mmf_om2 label: Multimode Fiber OM2 description: "Multimode fiber (50/125 \xB5m)." color: '#ff9933' - name: mmf_om3 label: Multimode Fiber OM3 description: "Optimized multimode fiber (50/125 \xB5m)." color: '#ff9933' - name: mmf_om4 label: Multimode Fiber OM4 description: "Enhanced multimode fiber (50/125 \xB5m)." color: '#ff9933' - name: mmf_om5 label: Multimode Fiber OM5 description: "Wideband multimode fiber (50/125 \xB5m)." color: '#ff9933' - name: smf_os1 label: Singlemode Fiber OS1 description: Standard singlemode fiber. color: '#ffcc00' - name: smf_os2 label: Singlemode Fiber OS2 description: Improved singlemode fiber. color: '#ffcc00' - name: dac_passive label: Passive DAC description: Direct Attach Copper (Passive). color: '#a0a0a0' - name: dac_active label: Active DAC description: Direct Attach Copper (Active). color: '#a0a0a0' - name: aoc label: Active Optical Cable description: Active Optical Cable. color: '#a0a0a0' - name: coaxial label: Coaxial description: Coaxial cable for broadband. color: '#000000' - name: mrj21_trunk label: MRJ21 Trunk description: High-density copper cable. color: '#000000' - name: power label: Power description: Power cables for electrical connections. color: '#000000' - name: usb label: USB description: Universal Serial Bus cable. color: '#000000' optional: true order_weight: 1100 - name: color kind: Dropdown choices: - name: dark_red label: Dark Red color: '#aa1409' - name: red label: Red color: '#f44336' - name: pink label: Pink color: '#e91e63' - name: rose label: Rose color: '#ffe4e1' - name: fuchsia label: Fuchsia color: '#ff66ff' - name: purple label: Purple color: '#9c27b0' - name: dark_purple label: Dark Purple color: '#673ab7' - name: indigo label: Indigo color: '#3f51b5' - name: blue label: Blue color: '#2196f3' - name: light_blue label: Light Blue color: '#03a9f4' - name: cyan label: Cyan color: '#00bcd4' - name: teal label: Teal color: '#009688' - name: aqua label: Aqua color: '#00ffff' - name: dark_green label: Dark Green color: '#2f6a31' - name: green label: Green color: '#4caf50' - name: light_green label: Light Green color: '#8bc34a' - name: lime label: Lime color: '#cddc39' - name: yellow label: Yellow color: '#ffeb3b' - name: amber label: Amber color: '#ffc107' - name: orange label: Orange color: '#ff9800' - name: dark_orange label: Dark Orange color: '#ff5722' - name: brown label: Brown color: '#795548' - name: light_grey label: Light Grey color: '#c0c0c0' - name: grey label: Grey color: '#9e9e9e' - name: dark_grey label: Dark Grey color: '#607d8b' - name: black label: Black color: '#111111' optional: true order_weight: 1300 - name: label kind: Text unique: true optional: true order_weight: 1000 - name: length kind: Number label: Length (in cm) optional: true order_weight: 1350 ``` --- # Circuit This schema extension adds Circuit and Circuit Endpoint nodes and links them to your infrastructure. A circuit could be a fiber connecting two sites, with one endpoint on each site. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Circuit[​](#circuit "Direct link to Circuit") * **Label:** Circuit * **Description:** A Circuit represent service operated by a provider. * **Namespace:** Dcim * **Icon:** mdi :cable-data * **Human Friendly ID:** circuit\_id\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------- | ------------------------------ | -------- | -------- | -------------- | -------------------------------------------------------------- | | circuit\_id | | Text | False | | | | description | | Text | True | | | | commit\_rate | | Number | True | | | | circuit\_type | Specifies the type of circuit. | Dropdown | True | | internet\_access, point\_to\_point, peering, dark\_fiber, mpls | | status | | Dropdown | False | active | active, provisioning, maintenance, drained | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | -------------------- | -------- | ----------- | --------- | | provider | OrganizationProvider | False | one | Attribute | | endpoints | DcimCircuitEndpoint | True | many | Component | ### CircuitEndpoint[​](#circuitendpoint "Direct link to CircuitEndpoint") * **Label:** Circuit Endpoint * **Description:** One of the two terminations (A or Z) of a circuit, at the location where the provider hands the circuit off. * **Namespace:** Dcim * **Icon:** mdi :ethernet * **Uniqueness Constraints:** * circuit, side\_\_value * **Human Friendly ID:** circuit\_\_circuit\_id\_\_value, name\_\_value * **Inherit From:** DcimEndpoint #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | -------- | -------- | -------------- | ------------------------------------------ | | side | | Dropdown | False | | a, z | | name | | Text | False | | | | status | | Dropdown | False | active | active, provisioning, maintenance, drained | | description | | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | --------------- | -------- | ----------- | --------- | | circuit | DcimCircuit | False | one | Parent | | location | LocationHosting | False | one | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### OrganizationProvider[​](#organizationprovider "Direct link to OrganizationProvider") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | ----------- | -------- | ----------- | ------- | | circuits | DcimCircuit | True | many | Generic | ### LocationHosting[​](#locationhosting "Direct link to LocationHosting") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------ | ------------------- | -------- | ----------- | ------- | | circuit\_endpoints | DcimCircuitEndpoint | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Circuit namespace: Dcim description: A Circuit represent service operated by a provider. label: Circuit icon: mdi:cable-data include_in_menu: true human_friendly_id: - circuit_id__value order_by: - circuit_id__value display_label: circuit_id__value attributes: - name: circuit_id kind: Text unique: true optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 - name: commit_rate kind: Number label: Commit Rate (Kbps) optional: true order_weight: 1400 - name: circuit_type kind: Dropdown choices: - name: internet_access label: Internet Access description: Broadband or dedicated internet access circuit. color: '#1e90ff' - name: point_to_point label: Point to Point description: Dedicated layer 2 or layer 3 link between two sites. color: '#a0c878' - name: peering label: Peering description: Connection to another network for exchange of traffic. color: '#20b2aa' - name: dark_fiber label: Dark Fiber description: Leased, unlit fiber managed and operated by the customer. color: '#555555' - name: mpls label: MPLS description: Multi-Protocol Label Switching circuit for QoS-based routing. color: '#7f00ff' optional: true description: Specifies the type of circuit. order_weight: 1300 - name: status kind: Dropdown default_value: active choices: - name: active label: Active description: Fully operational and currently in service. color: '#00d25b' - name: provisioning label: Provisioning description: In the process of being set up and configured. color: '#f0ad4e' - name: maintenance label: Maintenance description: Undergoing routine maintenance or repairs. color: '#ff9800' - name: drained label: Drained description: Temporarily taken out of service. color: '#9e9e9e' optional: false order_weight: 1200 relationships: - name: provider peer: OrganizationProvider kind: Attribute cardinality: one optional: false identifier: circuit__provider order_weight: 1500 - name: endpoints peer: DcimCircuitEndpoint kind: Component cardinality: many optional: true identifier: circuit__endpoint order_weight: 1600 - name: CircuitEndpoint namespace: Dcim description: One of the two terminations (A or Z) of a circuit, at the location where the provider hands the circuit off. label: Circuit Endpoint icon: mdi:ethernet include_in_menu: true menu_placement: DcimCircuit inherit_from: - DcimEndpoint human_friendly_id: - circuit__circuit_id__value - name__value order_by: - name__value display_label: name__value uniqueness_constraints: - - circuit - side__value attributes: - name: side kind: Dropdown label: Side choices: - name: a label: A description: A-side of the circuit. - name: z label: Z description: Z-side of the circuit. optional: false order_weight: 950 - name: name kind: Text read_only: true computed_attribute: kind: Jinja2 jinja2_template: '{{ circuit__circuit_id__value }}_{{ side__value | upper }}' optional: false order_weight: 1000 - name: status kind: Dropdown default_value: active choices: - name: active label: Active description: Fully operational and currently in service. color: '#00d25b' - name: provisioning label: Provisioning description: In the process of being set up and configured. color: '#f0ad4e' - name: maintenance label: Maintenance description: Undergoing routine maintenance or repairs. color: '#ff9800' - name: drained label: Drained description: Temporarily taken out of service. color: '#9e9e9e' optional: false order_weight: 1200 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: circuit peer: DcimCircuit kind: Parent cardinality: one optional: false identifier: circuit__endpoint order_weight: 900 - name: location peer: LocationHosting label: Location kind: Attribute cardinality: one optional: false identifier: circuit_endpoint__location order_weight: 1500 extensions: nodes: - kind: OrganizationProvider relationships: - name: circuits peer: DcimCircuit kind: Generic cardinality: many optional: true identifier: circuit__provider order_weight: 1300 - kind: LocationHosting relationships: - name: circuit_endpoints peer: DcimCircuitEndpoint kind: Generic cardinality: many optional: true identifier: circuit_endpoint__location order_weight: 1700 ``` --- # Circuit Contract This schema extension adds a Circuit Contract node to capture the service agreement behind a circuit with its provider. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/circuit](/schema-library/reference/circuit.md) ## Nodes[​](#nodes "Direct link to Nodes") ### CircuitContract[​](#circuitcontract "Direct link to CircuitContract") * **Label:** Circuit Contract * **Description:** A contract document associated with a circuit service, including terms, costs, and validity period. * **Namespace:** Dcim * **Icon:** mdi :file-sign * **Human Friendly ID:** name\_\_value * **Inherit From:** CoreFileObject #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------------- | ----------- | -------- | -------- | -------------- | ------------- | | name | | Text | False | | | | contract\_start | | DateTime | False | | | | contract\_end | | DateTime | False | | | | monthly\_cost | | Number | True | | | | currency | | Dropdown | True | | usd, eur, gbp | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------- | ----------- | -------- | ----------- | --------- | | circuit | DcimCircuit | False | one | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### DcimCircuit[​](#dcimcircuit "Direct link to DcimCircuit") #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | ------------------- | -------- | ----------- | ------- | | contract | DcimCircuitContract | True | one | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: CircuitContract namespace: Dcim description: A contract document associated with a circuit service, including terms, costs, and validity period. label: Circuit Contract icon: mdi:file-sign include_in_menu: true menu_placement: DcimCircuit inherit_from: - CoreFileObject human_friendly_id: - name__value order_by: - contract_end__value display_label: Contract (Circuit `{{ circuit__circuit_id__value }}`) attributes: - name: name kind: Text read_only: true computed_attribute: kind: Jinja2 jinja2_template: contract_{{ circuit__circuit_id__value }} optional: false order_weight: 1000 - name: contract_start kind: DateTime optional: false order_weight: 1200 - name: contract_end kind: DateTime optional: false order_weight: 1300 - name: monthly_cost kind: Number optional: true order_weight: 1400 - name: currency kind: Dropdown choices: - name: usd label: USD description: United States Dollar - name: eur label: EUR description: Euro - name: gbp label: GBP description: British Pound Sterling optional: true order_weight: 1450 relationships: - name: circuit peer: DcimCircuit kind: Attribute cardinality: one optional: false identifier: circuit__contract order_weight: 950 extensions: nodes: - kind: DcimCircuit relationships: - name: contract peer: DcimCircuitContract kind: Generic cardinality: one optional: true identifier: circuit__contract order_weight: 1550 ``` --- # Circuit Service This schema extension contains model coming on top of circuit to capture a single service shared across multiple circuits. For example you have a MPLS network supported by a provider connecting multiple locations: * One single CircuitService would be needed to store MPLS related information (e.g. service id, provider ...) * On each site we would create a circuit connecting on one side our device and the CircuitService on the other side ## Details[​](#details "Direct link to Details") * **Dependencies:** * [extensions/circuit](/schema-library/reference/circuit.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Service[​](#service "Direct link to Service") * **Label:** Circuit Service * **Description:** Represent the boundary of a provider network, the details of which are unknown or unimportant * **Namespace:** Circuit * **Icon:** mdi :cloud * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | | service\_id | | Text | True | | | | description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------ | -------------------- | -------- | ----------- | --------- | | provider | OrganizationProvider | False | one | Attribute | | circuit\_endpoints | DcimCircuitEndpoint | True | many | Component | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### OrganizationProvider[​](#organizationprovider "Direct link to OrganizationProvider") #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------------- | -------------- | -------- | ----------- | ---- | | circuit\_services | CircuitService | True | many | | ### DcimCircuitEndpoint[​](#dcimcircuitendpoint "Direct link to DcimCircuitEndpoint") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | -------------- | -------- | ----------- | ---- | | circuit\_service | CircuitService | True | one | | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Service namespace: Circuit description: Represent the boundary of a provider network, the details of which are unknown or unimportant label: Circuit Service icon: mdi:cloud menu_placement: DcimCircuit human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true - name: service_id kind: Text optional: true - name: description kind: Text optional: true relationships: - name: provider peer: OrganizationProvider optional: false cardinality: one kind: Attribute - name: circuit_endpoints peer: DcimCircuitEndpoint optional: true cardinality: many kind: Component extensions: nodes: - kind: OrganizationProvider relationships: - name: circuit_services peer: CircuitService cardinality: many optional: true - kind: DcimCircuitEndpoint relationships: - name: circuit_service peer: CircuitService cardinality: one optional: true ``` --- # Cluster This schema extension contains the foundations to capture clusters. With this one in place you can unlock various cluster flavours (a hosting cluster able to host VMs, a firewall cluster built with specific appliances ...). ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/compute](/schema-library/reference/compute.md) ## Generics[​](#generics "Direct link to Generics") ### Generic[​](#generic "Direct link to Generic") * **Description:** A cluster of machines hosting services or other machines. * **Namespace:** Cluster * **Icon:** mdi :dots-hexagon * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | -------------------- | ---- | -------- | -------------- | ------- | | name | Name of the cluster. | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | --------------- | -------- | ----------- | --------- | | locations | LocationGeneric | False | many | Attribute | | tags | BuiltinTag | True | many | Attribute | ### GenericComputeUnitNodes[​](#genericcomputeunitnodes "Direct link to GenericComputeUnitNodes") * **Description:** A generic to apply on clusters that can be built out of generic compute units. * **Namespace:** Cluster #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | ------------------ | -------- | ----------- | --------- | | nodes | ComputeGenericUnit | True | many | Component | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### ComputeGenericUnit[​](#computegenericunit "Direct link to ComputeGenericUnit") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------- | ------------------------------ | -------- | ----------- | ------- | | worker\_in\_cluster | ClusterGenericComputeUnitNodes | True | one | Generic | ### LocationGeneric[​](#locationgeneric "Direct link to LocationGeneric") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | -------------- | -------- | ----------- | --------- | | clusters | ClusterGeneric | True | many | Component | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: Generic namespace: Cluster description: A cluster of machines hosting services or other machines. icon: mdi:dots-hexagon include_in_menu: false human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false description: Name of the cluster. order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: locations peer: LocationGeneric label: Locations kind: Attribute cardinality: many optional: false order_weight: 1400 - name: tags peer: BuiltinTag kind: Attribute cardinality: many optional: true order_weight: 2000 - name: GenericComputeUnitNodes namespace: Cluster description: A generic to apply on clusters that can be built out of generic compute units. include_in_menu: false relationships: - name: nodes peer: ComputeGenericUnit label: Nodes kind: Component cardinality: many optional: true identifier: worker_in_cluster order_weight: 1500 extensions: nodes: - kind: ComputeGenericUnit relationships: - name: worker_in_cluster peer: ClusterGenericComputeUnitNodes label: Worker in cluster kind: Generic cardinality: one optional: true identifier: worker_in_cluster description: This device is a worker node of the specified cluster. order_weight: 1900 - kind: LocationGeneric relationships: - name: clusters peer: ClusterGeneric label: Clusters kind: Component cardinality: many optional: true description: All clusters available on that location. order_weight: 1650 ``` --- # Compute This schema extension captures physical servers and virtual machines, with a generic compute unit both inherit from and a generic for anything able to host VMs. Pair it with extensions/hosting\_cluster to group compute units into clusters. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### PhysicalServer[​](#physicalserver "Direct link to PhysicalServer") * **Label:** Physical Server * **Description:** A physical server with fixed resources and specific hardware characteristics. * **Namespace:** Compute * **Icon:** mdi :server * **Inherit From:** ComputeGenericUnit, DcimGenericDevice, DcimPhysicalDevice, VirtualizationHostVirtualMachine #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------ | ----------- | -------- | -------- | -------------- | ------------------------------------------ | | status | | Dropdown | False | active | active, provisioning, maintenance, drained | ### VirtualMachine[​](#virtualmachine "Direct link to VirtualMachine") * **Label:** Virtual Machine * **Description:** A virtual machine hosted on a server or a cluster. * **Namespace:** Virtualization * **Icon:** carbon :virtual-machine * **Inherit From:** ComputeGenericUnit, DcimGenericDevice #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------ | -------------------------------------------- | -------- | -------- | -------------- | -------------------- | | role | Role of the virtual machine. | Dropdown | True | | application, storage | | vcpu | Number of CPU cores assigned to the VM. | Number | True | | | | memory | Amount of memory (in GB) assigned to the VM. | Number | True | | | | disk | Disk space (in GB) assigned to the VM. | Number | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | -------------------------------- | -------- | ----------- | --------- | | host | VirtualizationHostVirtualMachine | False | one | Attribute | ## Generics[​](#generics "Direct link to Generics") ### GenericUnit[​](#genericunit "Direct link to GenericUnit") * **Label:** Compute * **Description:** A generic unit that can compute (e.g. server, vm ...). * **Namespace:** Compute ### HostVirtualMachine[​](#hostvirtualmachine "Direct link to HostVirtualMachine") * **Description:** A generic unit that can host VM * **Namespace:** Virtualization #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------------- | ---------------------------- | -------- | ----------- | --------- | | virtual\_machines | VirtualizationVirtualMachine | True | many | Component | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: GenericUnit namespace: Compute description: A generic unit that can compute (e.g. server, vm ...). label: Compute include_in_menu: true - name: HostVirtualMachine namespace: Virtualization description: A generic unit that can host VM include_in_menu: false relationships: - name: virtual_machines peer: VirtualizationVirtualMachine kind: Component cardinality: many optional: true identifier: host__virtual_machine order_weight: 1450 nodes: - name: PhysicalServer namespace: Compute description: A physical server with fixed resources and specific hardware characteristics. label: Physical Server icon: mdi:server include_in_menu: true menu_placement: ComputeGenericUnit inherit_from: - ComputeGenericUnit - DcimGenericDevice - DcimPhysicalDevice - VirtualizationHostVirtualMachine attributes: - name: status kind: Dropdown default_value: active choices: - name: active label: Active description: Fully operational and currently in service. color: '#00d25b' - name: provisioning label: Provisioning description: In the process of being set up and configured. color: '#f0ad4e' - name: maintenance label: Maintenance description: Undergoing routine maintenance or repairs. color: '#ff9800' - name: drained label: Drained description: Temporarily taken out of service. color: '#9e9e9e' optional: false order_weight: 1200 - name: VirtualMachine namespace: Virtualization description: A virtual machine hosted on a server or a cluster. label: Virtual Machine icon: carbon:virtual-machine include_in_menu: true menu_placement: ComputeGenericUnit inherit_from: - ComputeGenericUnit - DcimGenericDevice attributes: - name: role kind: Dropdown choices: - name: application label: Application color: '#7f7fff' - name: storage label: Storage color: '#bf7fbf' optional: true description: Role of the virtual machine. order_weight: 1300 - name: vcpu kind: Number optional: true description: Number of CPU cores assigned to the VM. order_weight: 1400 - name: memory kind: Number optional: true description: Amount of memory (in GB) assigned to the VM. order_weight: 1450 - name: disk kind: Number optional: true description: Disk space (in GB) assigned to the VM. order_weight: 1500 relationships: - name: host peer: VirtualizationHostVirtualMachine kind: Attribute cardinality: one optional: false identifier: host__virtual_machine order_weight: 1350 ``` --- # Cross-Connect This schema extension adds a Cross-Connect node. Think of it as a cable operated by a provider: attach it to a location, then connect endpoints to it (e.g. the rear interface of a patch panel or a circuit endpoint). ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### CrossConnect[​](#crossconnect "Direct link to CrossConnect") * **Label:** Cross-Connect * **Description:** Cross-connect between different endpoints within a datacenter. * **Namespace:** Dcim * **Icon:** streamline :arrow-crossover-right-solid * **Uniqueness Constraints:** * identifier\_\_value, provider * **Human Friendly ID:** provider\_\_name\_\_value, identifier\_\_value * **Inherit From:** DcimConnector #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | -------- | -------- | -------------- | ---------------------------- | | identifier | | Text | False | | | | description | | Text | True | | | | status | | Dropdown | False | planned | connected, planned, reserved | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | -------------------- | -------- | ----------- | --------- | | location | LocationHosting | False | one | Attribute | | provider | OrganizationProvider | False | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: CrossConnect namespace: Dcim description: Cross-connect between different endpoints within a datacenter. label: Cross-Connect icon: streamline:arrow-crossover-right-solid include_in_menu: true inherit_from: - DcimConnector human_friendly_id: - provider__name__value - identifier__value order_by: - provider__name__value display_label: '{{ identifier__value }} (Provider: {{ provider__name__value }})' uniqueness_constraints: - - identifier__value - provider attributes: - name: identifier kind: Text optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 - name: status kind: Dropdown default_value: planned choices: - name: connected label: Connected description: Fully operational and currently connected color: '#00d25b' - name: planned label: Planned description: In the process of being set up color: '#2196f3' - name: reserved label: Reserved description: Fully connected and reserved for a future use color: '#4d90fe' optional: false order_weight: 1200 relationships: - name: location peer: LocationHosting label: Location kind: Attribute cardinality: one optional: false order_weight: 1400 - name: provider peer: OrganizationProvider kind: Attribute cardinality: one optional: false order_weight: 1300 ``` --- # DCIM Base DCIM schema: device types, platforms, devices, and physical and virtual interfaces, plus the endpoint and connector generics that cables, circuits and cross-connects build on. ## Details[​](#details "Direct link to Details") * **Dependencies:** No dependencies ## Nodes[​](#nodes "Direct link to Nodes") ### DeviceType[​](#devicetype "Direct link to DeviceType") * **Label:** Device Type * **Description:** A model of device * **Namespace:** Dcim * **Icon:** mdi :poll * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------ | ----------- | ------- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | | part\_number | | Text | True | | | | height | | Number | False | 1 | | | full\_depth | | Boolean | False | True | | | weight | | Number | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ------------------------ | -------- | ----------- | --------- | | manufacturer | OrganizationManufacturer | False | one | Attribute | ### Platform[​](#platform "Direct link to Platform") * **Label:** Platform * **Description:** A Platform represent the type of software running on a device. * **Namespace:** Dcim * **Icon:** mdi :application-cog-outline * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ------------------------ | -------- | ----------- | --------- | | devices | DcimGenericDevice | True | many | Generic | | manufacturer | OrganizationManufacturer | True | one | Attribute | ### Device[​](#device "Direct link to Device") * **Label:** Network Device * **Description:** A configurable network device including routers, switches... * **Namespace:** Dcim * **Icon:** clarity :network-switch-solid * **Inherit From:** CoreArtifactTarget, DcimGenericDevice, DcimPhysicalDevice #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------ | ----------- | -------- | -------- | -------------- | ------------------------------------------------------- | | status | | Dropdown | False | active | active, reserved, provisioning, maintenance, deprecated | | role | | Dropdown | True | | core, edge, cpe, spine, leaf, tor | ### Physical[​](#physical "Direct link to Physical") * **Label:** Physical Interface * **Description:** Physical network port on a device * **Namespace:** Interface * **Inherit From:** DcimInterface, InterfaceLayer2, InterfaceLayer3, DcimEndpoint, InterfaceHasSubInterface ### Virtual[​](#virtual "Direct link to Virtual") * **Label:** Virtual Interface * **Description:** Virtual interface like VLAN or Loopback * **Namespace:** Interface * **Inherit From:** DcimInterface, InterfaceLayer2, InterfaceLayer3 #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------------- | ------------------------ | -------- | ----------- | --------- | | parent\_interface | InterfaceHasSubInterface | True | one | Attribute | ## Generics[​](#generics "Direct link to Generics") ### GenericDevice[​](#genericdevice "Direct link to GenericDevice") * **Label:** Device * **Description:** Generic Device object. * **Namespace:** Dcim * **Icon:** mdi :server * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | ------------- | -------- | ----------- | --------- | | interfaces | DcimInterface | True | many | Component | | tags | BuiltinTag | True | many | Attribute | | primary\_address | IpamIPAddress | True | one | Attribute | | platform | DcimPlatform | True | one | Attribute | ### PhysicalDevice[​](#physicaldevice "Direct link to PhysicalDevice") * **Description:** Generic holding attributes and relationships relevant for physical device. * **Namespace:** Dcim #### Attributes[​](#attributes-4 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ------------------------------------------------ | -------- | -------- | -------------- | ----------- | | name | | Text | False | | | | position | Lowest unit. | Number | True | | | | serial\_number | | Text | True | | | | rack\_face | On which face of the rack the device is mounted. | Dropdown | False | front | front, rear | #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | --------------- | -------- | ----------- | --------- | | device\_type | DcimDeviceType | True | one | Attribute | | location | LocationHosting | False | one | Attribute | ### Endpoint[​](#endpoint "Direct link to Endpoint") * **Description:** Generic Endpoint to receive a connector. * **Namespace:** Dcim #### Relationships[​](#relationships-5 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | ------------- | -------- | ----------- | --------- | | connector | DcimConnector | True | one | Attribute | ### Connector[​](#connector "Direct link to Connector") * **Description:** Generic Connector to link two endpoints together. * **Namespace:** Dcim #### Relationships[​](#relationships-6 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------------- | ------------ | -------- | ----------- | --------- | | connected\_endpoints | DcimEndpoint | True | many | Attribute | ### Interface[​](#interface "Direct link to Interface") * **Label:** Interface * **Description:** Generic Network Interface * **Namespace:** Dcim * **Icon:** mdi :ethernet * **Uniqueness Constraints:** * device, name\_\_value * **Human Friendly ID:** device\_\_name\_\_value, name\_\_value #### Attributes[​](#attributes-5 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------------------------------------------------------------- | -------- | -------- | -------------- | ----------------------------------------------------- | | name | Name of the interface | Text | False | | | | description | A brief description of the interface | Text | True | | | | status | The status of the interface | Dropdown | False | active | provisioning, free, active, maintenance, disabled | | role | The role of the interface in the network | Dropdown | True | | core, customer, access, management, peering, upstream | | mtu | Maximum transmission unit, in bytes (IP payload, not frame size). | Number | True | 1500 | | #### Relationships[​](#relationships-7 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | ----------------- | -------- | ----------- | ------ | | device | DcimGenericDevice | False | one | Parent | ### Layer2[​](#layer2 "Direct link to Layer2") * **Label:** Layer 2 Interface * **Description:** Layer 2 specific attributes for network interfaces * **Namespace:** Interface #### Attributes[​](#attributes-6 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------ | ----------------------------- | ---------- | -------- | -------------- | ------------------------- | | mac\_address | | MacAddress | True | | | | l2\_mode | Layer 2 mode of the interface | Dropdown | True | | access, trunk, trunk\_all | ### Layer3[​](#layer3 "Direct link to Layer3") * **Label:** Layer 3 Interface * **Description:** Layer 3 specific attributes for network interfaces * **Namespace:** Interface #### Relationships[​](#relationships-8 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | ------------- | -------- | ----------- | --------- | | ip\_addresses | IpamIPAddress | True | many | Attribute | ### HasSubInterface[​](#hassubinterface "Direct link to HasSubInterface") * **Description:** A generic interface that can have sub-interfaces * **Namespace:** Interface #### Relationships[​](#relationships-9 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ---------------- | -------- | ----------- | --------- | | sub\_interfaces | InterfaceVirtual | True | many | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: GenericDevice namespace: Dcim description: Generic Device object. label: Device icon: mdi:server include_in_menu: false human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: interfaces peer: DcimInterface kind: Component cardinality: many optional: true identifier: device__interface order_weight: 1650 - name: tags peer: BuiltinTag kind: Attribute cardinality: many optional: true order_weight: 2000 - name: primary_address peer: IpamIPAddress label: Primary IP Address kind: Attribute cardinality: one optional: true identifier: device__primary_address order_weight: 1700 - name: platform peer: DcimPlatform kind: Attribute cardinality: one optional: true identifier: device__platform order_weight: 1250 - name: PhysicalDevice namespace: Dcim description: Generic holding attributes and relationships relevant for physical device. include_in_menu: false attributes: - name: name kind: Text unique: true optional: false allow_override: any order_weight: 1000 - name: position kind: Number label: Position (U) optional: true description: Lowest unit. order_weight: 1550 - name: serial_number kind: Text optional: true order_weight: 1400 - name: rack_face kind: Dropdown label: Rack Face default_value: front choices: - name: front label: Front description: Device mounted on the front face of the rack. - name: rear label: Rear description: Device mounted on the rear face of the rack. optional: false description: On which face of the rack the device is mounted. order_weight: 1600 relationships: - name: device_type peer: DcimDeviceType kind: Attribute cardinality: one optional: true order_weight: 1150 - name: location peer: LocationHosting label: Location kind: Attribute cardinality: one optional: false identifier: physical_device__location order_weight: 1500 - name: Endpoint namespace: Dcim description: Generic Endpoint to receive a connector. include_in_menu: false relationships: - name: connector peer: DcimConnector kind: Attribute cardinality: one optional: true identifier: connector__endpoint order_weight: 1600 - name: Connector namespace: Dcim description: Generic Connector to link two endpoints together. include_in_menu: false relationships: - name: connected_endpoints peer: DcimEndpoint kind: Attribute cardinality: many optional: true identifier: connector__endpoint order_weight: 1500 - name: Interface namespace: Dcim description: Generic Network Interface label: Interface icon: mdi:ethernet include_in_menu: true menu_placement: DcimDevice human_friendly_id: - device__name__value - name__value order_by: - device__name__value - name__value display_label: name__value uniqueness_constraints: - - device - name__value attributes: - name: name kind: Text optional: false description: Name of the interface order_weight: 1000 - name: description kind: Text optional: true description: A brief description of the interface order_weight: 1100 - name: status kind: Dropdown default_value: active choices: - name: provisioning label: Provisioning description: Interface is being provisioned. color: '#f0ad4e' - name: free label: Free description: Interface is unused. color: '#c8e6c9' - name: active label: Active description: Interface is active and operational. color: '#00d25b' - name: maintenance label: Maintenance description: Interface is under maintenance. color: '#ff9800' - name: disabled label: Disabled description: Interface has been disabled. color: '#6c757d' optional: false description: The status of the interface order_weight: 1200 - name: role kind: Dropdown choices: - name: core label: Core Interface description: Central part of the network. color: '#A9CCE3' - name: customer label: Customer Interface description: Interface dedicated to customer connections. color: '#D2B4DE' - name: access label: Access Interfaces description: Interface connecting endpoint devices. color: '#B4E0DC' - name: management label: Management Interface description: Interface dedicated to device management. color: '#E3DAC9' - name: peering label: Peering Interface description: Interface dedicated to peering with other networks. color: '#C4B7E6' - name: upstream label: Upstream Interface description: Interface dedicated to upstream traffic between networks. color: '#B2D4E6' optional: true description: The role of the interface in the network order_weight: 1300 - name: mtu kind: Number label: MTU default_value: 1500 optional: true description: Maximum transmission unit, in bytes (IP payload, not frame size). order_weight: 1350 relationships: - name: device peer: DcimGenericDevice kind: Parent cardinality: one optional: false identifier: device__interface order_weight: 900 - name: Layer2 namespace: Interface description: Layer 2 specific attributes for network interfaces label: Layer 2 Interface include_in_menu: false attributes: - name: mac_address kind: MacAddress label: MAC Address optional: true order_weight: 1400 - name: l2_mode kind: Dropdown label: Layer2 Mode choices: - name: access label: Access description: Access mode - name: trunk label: Trunk description: Trunk mode - name: trunk_all label: Trunk (All) description: Trunk all mode optional: true description: Layer 2 mode of the interface order_weight: 1450 - name: Layer3 namespace: Interface description: Layer 3 specific attributes for network interfaces label: Layer 3 Interface include_in_menu: false relationships: - name: ip_addresses peer: IpamIPAddress label: IP Addresses kind: Attribute cardinality: many optional: true identifier: ip_address__interface description: List of IP addresses associated with the interface order_weight: 1150 - name: HasSubInterface namespace: Interface description: A generic interface that can have sub-interfaces include_in_menu: false relationships: - name: sub_interfaces peer: InterfaceVirtual label: Sub-interface(s) kind: Attribute cardinality: many optional: true identifier: sub__interface description: Sub-interfaces of this interface order_weight: 1750 nodes: - name: DeviceType namespace: Dcim description: A model of device label: Device Type icon: mdi:poll include_in_menu: true menu_placement: DcimDevice human_friendly_id: - name__value order_by: - manufacturer__name__value - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 - name: part_number kind: Text label: Part Number optional: true order_weight: 1200 - name: height kind: Number label: Height (U) default_value: 1 optional: false order_weight: 1400 - name: full_depth kind: Boolean label: Full Depth default_value: true optional: false order_weight: 1500 - name: weight kind: Number label: Weight (kg) optional: true order_weight: 1600 relationships: - name: manufacturer peer: OrganizationManufacturer kind: Attribute cardinality: one optional: false identifier: manufacturer__device_type order_weight: 1250 - name: Platform namespace: Dcim description: A Platform represent the type of software running on a device. label: Platform icon: mdi:application-cog-outline include_in_menu: true menu_placement: DcimDevice human_friendly_id: - name__value order_by: - manufacturer__name__value - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: devices peer: DcimGenericDevice kind: Generic cardinality: many optional: true identifier: device__platform order_weight: 1350 - name: manufacturer peer: OrganizationManufacturer kind: Attribute cardinality: one optional: true identifier: manufacturer__platform order_weight: 1300 - name: Device namespace: Dcim description: A configurable network device including routers, switches... label: Network Device icon: clarity:network-switch-solid include_in_menu: true inherit_from: - CoreArtifactTarget - DcimGenericDevice - DcimPhysicalDevice generate_template: true attributes: - name: status kind: Dropdown default_value: active choices: - name: active label: Active description: Fully operational and currently in service. color: '#00d25b' - name: reserved label: Reserved description: Reserved for future use, not yet active. color: '#4d90fe' - name: provisioning label: Provisioning description: In the process of being set up and configured. color: '#f0ad4e' - name: maintenance label: Maintenance description: Undergoing routine maintenance or repairs. color: '#ff9800' - name: deprecated label: Deprecated description: No longer in use and scheduled for removal. color: '#e04040' optional: false order_weight: 1200 - name: role kind: Dropdown choices: - name: core label: Core Router description: Central part of the network. color: '#7f7fff' - name: edge label: Edge Router description: Network boundary with external networks. color: '#bf7fbf' - name: cpe label: Customer Premise Equipment description: Devices located at the customer's premises. color: '#bf7f7f' - name: spine label: Spine Router description: Aggregation router part of a Fabric. color: '#aeeeee' - name: leaf label: Leaf Switch description: Top of Rack part of a Fabric. color: '#e6e6fa' - name: tor label: Tor Switch description: Tor switch part of a Fabric. color: '#e8e7fd' optional: true order_weight: 1300 - name: Physical namespace: Interface description: Physical network port on a device label: Physical Interface include_in_menu: false inherit_from: - DcimInterface - InterfaceLayer2 - InterfaceLayer3 - DcimEndpoint - InterfaceHasSubInterface - name: Virtual namespace: Interface description: Virtual interface like VLAN or Loopback label: Virtual Interface include_in_menu: false inherit_from: - DcimInterface - InterfaceLayer2 - InterfaceLayer3 relationships: - name: parent_interface peer: InterfaceHasSubInterface kind: Attribute cardinality: one optional: true identifier: sub__interface description: Parent interface to which this sub-interface belongs order_weight: 1600 ``` --- # Device Module This schema extension lets you capture module bays on a DcimPhysicalDevice and the modules installed into them (e.g. fan trays, line cards). A device can have any number of module bays, and each bay can optionally hold one module. It ships a ready-to-use Module and Module Type pair, plus the generic Module and Module Type you can build your own flavours on: see extensions/device\_module\_psu, or the experimental modules\_linecards and modules\_routing\_engine extensions. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### ModuleBay[​](#modulebay "Direct link to ModuleBay") * **Label:** Module Bay * **Description:** A physical bay on a device that can receive a module. * **Namespace:** Dcim * **Icon:** mdi :tray * **Uniqueness Constraints:** * device, name\_\_value * **Human Friendly ID:** computed\_name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ------------------------------------------------------------------------------------ | -------- | -------- | -------------- | ------------------------------------------ | | computed\_name | Name computed from the device and bay name. | Text | False | | | | name | Name of the bay, e.g. 'slot 1' or 'psu 1'. | Text | False | | | | position | Position of the bay within the device, e.g. '1', 'F3' or 'PSU-2'. | Text | True | | | | bay\_label | What the bay is for, for example 'Supervisor' or 'Line Card'. | Text | True | | | | description | | Text | True | | | | role | The role of the module bay, indicating the type of module it is intended to receive. | Dropdown | True | | supervisor, line\_card, power\_supply, fan | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------------- | ------------------ | -------- | ----------- | --------- | | device | DcimPhysicalDevice | False | one | Parent | | installed\_module | DcimGenericModule | True | one | Attribute | ### Module[​](#module "Direct link to Module") * **Label:** Module * **Description:** Standard module installed in a device. * **Namespace:** Dcim * **Icon:** mdi :expansion-card * **Inherit From:** DcimGenericModule #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | -------------------------------------------------- | ---- | -------- | -------------- | ------- | | computed\_name | Name computed from the module bay and module type. | Text | False | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | -------------- | -------- | ----------- | --------- | | module\_type | DcimModuleType | False | one | Attribute | ### ModuleType[​](#moduletype "Direct link to ModuleType") * **Label:** Module Type * **Description:** Standard module type, defining common specifications for a Module. * **Namespace:** Dcim * **Icon:** mdi :database-cog * **Inherit From:** DcimGenericModuleType ## Generics[​](#generics "Direct link to Generics") ### GenericModule[​](#genericmodule "Direct link to GenericModule") * **Label:** Module * **Description:** A generic module, such as a Linecard or Routing Engine, installed in a module bay. * **Namespace:** Dcim * **Icon:** mdi :expansion-card * **Human Friendly ID:** computed\_name\_\_value #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ----------------------------------------------------------------------- | -------- | -------- | -------------- | ---------------------------------------------- | | computed\_name | Name computed from the module bay; concrete module kinds may refine it. | Text | False | | | | serial\_number | Serial number of the module, if known. | Text | True | | | | description | | Text | True | | | | status | Operational status of the module. | Dropdown | False | active | provisioning, active, maintenance, failed, rma | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------- | ------------- | -------- | ----------- | --------- | | module\_bay | DcimModuleBay | False | one | Attribute | ### GenericModuleType[​](#genericmoduletype "Direct link to GenericModuleType") * **Label:** Module Type * **Description:** A generic module type, with common specifications like part number and manufacturer. * **Namespace:** Dcim * **Icon:** mdi :database-cog * **Uniqueness Constraints:** * name\_\_value, manufacturer * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------- | ------------------------------- | ------ | -------- | -------------- | ------- | | name | Name of the module type. | Text | False | | | | description | Description of the module type. | Text | True | | | | part\_number | Part number of the module. | Text | True | | | | weight\_grams | Weight of the module in grams. | Number | True | | | #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ------------------------ | -------- | ----------- | --------- | | manufacturer | OrganizationManufacturer | False | one | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### DcimPhysicalDevice[​](#dcimphysicaldevice "Direct link to DcimPhysicalDevice") #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ------------- | -------- | ----------- | --------- | | module\_bays | DcimModuleBay | True | many | Component | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: GenericModule namespace: Dcim description: A generic module, such as a Linecard or Routing Engine, installed in a module bay. label: Module icon: mdi:expansion-card include_in_menu: true human_friendly_id: - computed_name__value display_label: computed_name__value attributes: - name: computed_name kind: Text unique: true read_only: true computed_attribute: kind: Jinja2 jinja2_template: '{{ module_bay__computed_name__value }}' optional: false description: Name computed from the module bay; concrete module kinds may refine it. allow_override: any order_weight: 1000 - name: serial_number kind: Text optional: true description: Serial number of the module, if known. order_weight: 1400 - name: description kind: Text optional: true order_weight: 1100 - name: status kind: Dropdown default_value: active choices: - name: provisioning label: Provisioning description: Module is being installed and provisioned. color: '#f0ad4e' - name: active label: Active description: Module is installed and operational. color: '#00d25b' - name: maintenance label: Maintenance description: Module is under maintenance. color: '#ff9800' - name: failed label: Failed description: Module is still installed but is not operational. color: '#e04040' - name: rma label: RMA description: Module has failed and is being returned to the manufacturer. color: '#6c757d' optional: false description: Operational status of the module. order_weight: 1200 relationships: - name: module_bay peer: DcimModuleBay label: Module Bay kind: Attribute cardinality: one optional: false identifier: module_bay__module order_weight: 900 - name: GenericModuleType namespace: Dcim description: A generic module type, with common specifications like part number and manufacturer. label: Module Type icon: mdi:database-cog include_in_menu: true menu_placement: DcimGenericModule human_friendly_id: - name__value order_by: - manufacturer__name__value - name__value display_label: name__value uniqueness_constraints: - - name__value - manufacturer attributes: - name: name kind: Text unique: true optional: false description: Name of the module type. order_weight: 1000 - name: description kind: Text optional: true description: Description of the module type. order_weight: 1100 - name: part_number kind: Text label: Part Number optional: true description: Part number of the module. order_weight: 1200 - name: weight_grams label: Weight (g) kind: Number optional: true description: Weight of the module in grams. order_weight: 1300 relationships: - name: manufacturer peer: OrganizationManufacturer kind: Attribute cardinality: one optional: false identifier: manufacturer__moduletype description: Manufacturer of the module type. order_weight: 1250 nodes: - name: ModuleBay namespace: Dcim description: A physical bay on a device that can receive a module. label: Module Bay icon: mdi:tray include_in_menu: true menu_placement: DcimDevice human_friendly_id: - computed_name__value order_by: - device__name__value - name__value display_label: computed_name__value uniqueness_constraints: - - device - name__value attributes: - name: computed_name kind: Text unique: true read_only: true computed_attribute: kind: Jinja2 jinja2_template: '{{ device__name__value }} > {{ name__value }}' optional: false description: Name computed from the device and bay name. order_weight: 800 - name: name kind: Text optional: false description: Name of the bay, e.g. 'slot 1' or 'psu 1'. order_weight: 1000 - name: position kind: Text optional: true description: Position of the bay within the device, e.g. '1', 'F3' or 'PSU-2'. order_weight: 1050 - name: bay_label label: Bay Label kind: Text optional: true description: What the bay is for, for example 'Supervisor' or 'Line Card'. order_weight: 1060 - name: description kind: Text optional: true order_weight: 1100 - name: role kind: Dropdown choices: - name: supervisor label: Supervisor description: Bay intended for a supervisor / routing engine module. color: '#A9CCE3' - name: line_card label: Line Card description: Bay intended for a line card module. color: '#B4E0DC' - name: power_supply label: Power Supply description: Bay intended for a power supply module. color: '#E3DAC9' - name: fan label: Fan description: Bay intended for a fan module. color: '#D2B4DE' optional: true description: The role of the module bay, indicating the type of module it is intended to receive. order_weight: 1300 relationships: - name: device peer: DcimPhysicalDevice label: Device kind: Parent cardinality: one optional: false identifier: device__module_bays order_weight: 900 - name: installed_module peer: DcimGenericModule label: Installed Module kind: Attribute cardinality: one optional: true identifier: module_bay__module order_weight: 1200 - name: Module namespace: Dcim description: Standard module installed in a device. label: Module icon: mdi:expansion-card include_in_menu: false inherit_from: - DcimGenericModule attributes: - name: computed_name kind: Text unique: true read_only: true computed_attribute: kind: Jinja2 jinja2_template: '{{ module_bay__computed_name__value }} ({{ module_type__name__value }})' optional: false description: Name computed from the module bay and module type. order_weight: 1000 relationships: - name: module_type peer: DcimModuleType kind: Attribute cardinality: one optional: false order_weight: 950 - name: ModuleType namespace: Dcim description: Standard module type, defining common specifications for a Module. label: Module Type icon: mdi:database-cog include_in_menu: false inherit_from: - DcimGenericModuleType extensions: nodes: - kind: DcimPhysicalDevice relationships: - name: module_bays peer: DcimModuleBay label: Module Bays kind: Component cardinality: many optional: true identifier: device__module_bays description: Module bays available on this device. order_weight: 1750 ``` --- # PSU Module This schema extension adds a PSU (Power Supply Unit) flavour on top of the generic Module and Module Type from extensions/device\_module, so you can track power supplies installed in a device's module bays with PSU-specific attributes such as wattage and hot-swap capability. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/device\_module](/schema-library/reference/device_module.md) ## Nodes[​](#nodes "Direct link to Nodes") ### PSUModule[​](#psumodule "Direct link to PSUModule") * **Label:** PSU Module * **Description:** Power Supply Unit (PSU) installed in a device. * **Namespace:** Dcim * **Icon:** mdi :power-plug * **Inherit From:** DcimGenericModule #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | -------------------------------------------------- | ---- | -------- | -------------- | ------- | | computed\_name | Name computed from the module bay and module type. | Text | False | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ----------------- | -------- | ----------- | --------- | | module\_type | DcimPSUModuleType | False | one | Attribute | ### PSUModuleType[​](#psumoduletype "Direct link to PSUModuleType") * **Label:** PSU Module Type * **Description:** PSU module type, defining common specifications such as wattage and hot-swap capability. * **Namespace:** Dcim * **Icon:** mdi :database-cog * **Inherit From:** DcimGenericModuleType #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ----------------------------------------------------------------- | ------- | -------- | -------------- | ------- | | wattage | Wattage in watts. | Number | True | | | | hot\_swappable | Whether the PSU can be replaced without powering down the device. | Boolean | False | False | | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: PSUModule namespace: Dcim description: Power Supply Unit (PSU) installed in a device. label: PSU Module icon: mdi:power-plug include_in_menu: false inherit_from: - DcimGenericModule attributes: - name: computed_name kind: Text unique: true read_only: true computed_attribute: kind: Jinja2 jinja2_template: '{{ module_bay__computed_name__value }} ({{ module_type__name__value }})' optional: false description: Name computed from the module bay and module type. order_weight: 1000 relationships: - name: module_type peer: DcimPSUModuleType kind: Attribute cardinality: one optional: false order_weight: 950 - name: PSUModuleType namespace: Dcim description: PSU module type, defining common specifications such as wattage and hot-swap capability. label: PSU Module Type icon: mdi:database-cog include_in_menu: false inherit_from: - DcimGenericModuleType attributes: - name: wattage kind: Number label: Wattage (W) optional: true parameters: min_value: 1 description: Wattage in watts. order_weight: 1300 - name: hot_swappable kind: Boolean label: Hot-Swappable default_value: false optional: false description: Whether the PSU can be replaced without powering down the device. order_weight: 1350 ``` --- # Firewall Policer This schema extension contains models for traffic policers and the classes and rules that reference them, so you can describe rate limiting alongside the interfaces it is applied to. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Policer[​](#policer "Direct link to Policer") * **Label:** Network Policer * **Description:** A generic policer configuration. * **Namespace:** Security * **Icon:** mdi :car-speed-limiter * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------ | ----------------------------------------------- | -------- | -------- | -------------- | ------------------------------------------------------------------------------- | | name | Unique name of the policer. | Text | | | | | description | | Text | True | | | | policer\_type | Type of policer. | Dropdown | True | | bandwidth-policer, interface-policer, shared-policer, hierarchical-policer | | bandwidth\_limit | Bandwidth limit for the policer. | Dropdown | True | | 500k, 2125k, 5250k, 10m, 20m, 30m, 50m, 75m, 100m, 200m, 300m, 1000m | | pps\_limit | Packets per second (PPS) limit for the policer. | Dropdown | True | | 500pps, 1000pps, 5000pps | | burst\_size\_limit | Burst size limit for the policer. | Dropdown | True | | 50k, 100k, 128k, 256k, 512k, 1m, 1500k, 2m, 3m, 4m, 8m, 12m, 37m, 40m, 1000000k | | packet\_burst | Packet burst size for the policer. | Dropdown | True | | 1k, 5k, 10k | | action | Action to take when limits are exceeded. | Dropdown | | | discard, drop, accept | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Policer namespace: Security label: Network Policer icon: mdi:car-speed-limiter description: A generic policer configuration. display_label: name__value order_by: - name__value human_friendly_id: - name__value attributes: - name: name kind: Text label: Name description: Unique name of the policer. order_weight: 1000 unique: true - name: description kind: Text optional: true order_weight: 1100 - name: policer_type kind: Dropdown description: Type of policer. label: Policer Type optional: true order_weight: 1150 choices: - name: bandwidth-policer label: Bandwidth Policer description: Policer that limits bandwidth on interfaces. color: '#C3E0E5' - name: interface-policer label: Interface Policer description: Policer applied to interfaces for rate-limiting traffic. color: '#D1E7E1' - name: shared-policer label: Shared Policer description: Policer with shared bandwidth across multiple links. color: '#A5C9C7' - name: hierarchical-policer label: Hierarchical Policer description: Policer applied in a hierarchical manner (e.g., parent-child relationships). color: '#B1E0D9' - name: bandwidth_limit kind: Dropdown description: Bandwidth limit for the policer. label: Bandwidth Limit optional: true order_weight: 1200 choices: - name: 500k label: 500 Kbps description: Bandwidth limit of 500 Kbps. color: '#A9CCE3' - name: 2125k label: 2,125 Kbps description: Bandwidth limit of 2,125 Kbps. color: '#AED6F1' - name: 5250k label: 5,250 Kbps description: Bandwidth limit of 5,250 Kbps. color: '#B4DDED' - name: 10m label: 10 Mbps description: Bandwidth limit of 10 Mbps. color: '#C2E2F3' - name: 20m label: 20 Mbps description: Bandwidth limit of 20 Mbps. color: '#D0E7F8' - name: 30m label: 30 Mbps description: Bandwidth limit of 30 Mbps. color: '#E0ECF9' - name: 50m label: 50 Mbps description: Bandwidth limit of 50 Mbps. color: '#AFC7F2' - name: 75m label: 75 Mbps description: Bandwidth limit of 75 Mbps. color: '#E8F3FD' - name: 100m label: 100 Mbps description: Bandwidth limit of 100 Mbps. color: '#F0F9FF' - name: 200m label: 200 Mbps description: Bandwidth limit of 200 Mbps. color: '#D1E6F9' - name: 300m label: 300 Mbps description: Bandwidth limit of 300 Mbps. color: '#EAF2FC' - name: 1000m label: 1 Gbps description: Bandwidth limit of 1 Gbps. color: '#E6E6FA' - name: pps_limit kind: Dropdown description: Packets per second (PPS) limit for the policer. label: PPS Limit optional: true order_weight: 1250 choices: - name: 500pps label: 500 PPS description: PPS limit of 500. color: '#E0BBE4' - name: 1000pps label: 1,000 PPS description: PPS limit of 1,000. color: '#D4A5E4' - name: 5000pps label: 5,000 PPS description: PPS limit of 5,000. color: '#C89BE4' - name: burst_size_limit kind: Dropdown description: Burst size limit for the policer. label: Burst Size Limit optional: true order_weight: 1300 choices: - name: 50k label: 50 KB description: Burst size limit of 50 KB. color: '#CDEACC' - name: 100k label: 100 KB description: Burst size limit of 100 KB. color: '#B3E2A8' - name: 128k label: 128 KB description: Burst size limit of 128 KB. color: '#A3D89E' - name: 256k label: 256 KB description: Burst size limit of 256 KB. color: '#92CF91' - name: 512k label: 512 KB description: Burst size limit of 512 KB. color: '#88C786' - name: 1m label: 1 MB description: Burst size limit of 1 MB. color: '#7FCF79' - name: 1500k label: 1.5 MB description: Burst size limit of 1.5 MB. color: '#77C46B' - name: 2m label: 2 MB description: Burst size limit of 2 MB. color: '#63A17E' - name: 3m label: 3 MB description: Burst size limit of 3 MB. color: '#8FD19E' - name: 4m label: 4 MB description: Burst size limit of 4 MB. color: '#70B961' - name: 8m label: 8 MB description: Burst size limit of 8 MB. color: '#6BAD57' - name: 12m label: 12 MB description: Burst size limit of 12 MB. color: '#63A14C' - name: 37m label: 37 MB description: Burst size limit of 37 MB. color: '#56A56C' - name: 40m label: 40 MB description: Burst size limit of 40 MB. color: '#5F9742' - name: 1000000k label: 1 GB description: Burst size limit of 1 GB. color: '#599D4A' - name: packet_burst kind: Dropdown description: Packet burst size for the policer. label: Packet Burst optional: true order_weight: 1350 choices: - name: 1k label: 1,000 packets description: Packet burst size of 1,000 packets. color: '#FFE4E1' - name: 5k label: 5,000 packets description: Packet burst size of 5,000 packets. color: '#FFFACD' - name: 10k label: 10,000 packets description: Packet burst size of 10,000 packets. color: '#FFF0F5' - name: action kind: Dropdown description: Action to take when limits are exceeded. label: Action order_weight: 1400 choices: - name: discard label: Discard description: Discard the packet. color: '#F4CCCC' - name: drop label: Drop description: Drop the packet. color: '#FAD7A0' - name: accept label: Accept description: Accept the packet. color: '#CDEACC' ``` --- # Hosting Cluster A generic cluster built from compute units (e.g. servers) that can host virtual machines. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/cluster](/schema-library/reference/cluster.md) * [extensions/compute](/schema-library/reference/compute.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Hosting[​](#hosting "Direct link to Hosting") * **Label:** Hosting Cluster * **Description:** A cluster hosting virtual machines. * **Namespace:** Cluster * **Icon:** mdi :dots-hexagon * **Inherit From:** ClusterGeneric, VirtualizationHostVirtualMachine, ClusterGenericComputeUnitNodes #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------- | --------------------------------------------------- | -------- | -------- | -------------- | ------------------------------------------------------------------------- | | technology | Underlying virtualisation or clustering technology. | Dropdown | False | | vmware, kvm, hyper\_v, xen, nutanix, openstack, kubernetes, public\_cloud | | status | | Dropdown | False | active | active, provisioning, maintenance, drained | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Hosting namespace: Cluster description: A cluster hosting virtual machines. label: Hosting Cluster icon: mdi:dots-hexagon include_in_menu: true menu_placement: ComputeGenericUnit inherit_from: - ClusterGeneric - VirtualizationHostVirtualMachine - ClusterGenericComputeUnitNodes attributes: - name: technology kind: Dropdown label: Technology choices: - name: vmware label: VMware vSphere color: '#3d8600' - name: kvm label: KVM / QEMU color: '#0082e2' - name: hyper_v label: Hyper-V color: '#00adef' - name: xen label: Xen / XenServer color: '#f5821f' - name: nutanix label: Nutanix AHV color: '#024da1' - name: openstack label: OpenStack color: '#ed1944' - name: kubernetes label: Kubernetes color: '#326ce5' - name: public_cloud label: Public Cloud color: '#888888' optional: false description: Underlying virtualisation or clustering technology. order_weight: 1300 - name: status kind: Dropdown default_value: active choices: - name: active label: Active description: Fully operational and currently in service. color: '#00d25b' - name: provisioning label: Provisioning description: In the process of being set up and configured. color: '#f0ad4e' - name: maintenance label: Maintenance description: Undergoing routine maintenance or repairs. color: '#ff9800' - name: drained label: Drained description: Temporarily taken out of service. color: '#9e9e9e' optional: false order_weight: 1200 ``` --- # InfiniBand Fabric Hardware This schema extension models the hardware of an InfiniBand fabric: switches with their configuration profile (AAA, role, split-ready, SSH and CLI settings), switch and management interfaces with lane width, split and speed options, rail-optimized scalable units (RSU) and the fabric that groups them. It does not model the InfiniBand protocol itself (subnet manager, partitions, routing). ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/compute](/schema-library/reference/compute.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Switch[​](#switch "Direct link to Switch") * **Label:** InfiniBand Switch * **Description:** InfiniBand Switch * **Namespace:** Infiniband * **Icon:** mdi :server * **Inherit From:** CoreArtifactTarget #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------------------------- | ------------------------------------------------------- | -------- | -------- | -------------------------------- | ------------- | | name | | Text | | | | | description | | Text | True | | | | os\_version | | Text | True | | | | rsu\_rail\_id | InfiniBand RSU Rail ID (numeric) assigned to the switch | Number | True | | | | aaa\_authentication\_model | AAA authentication model to be used by the switch | Dropdown | False | local | tacacs, local | | aaa\_accounting\_model | AAA accounting model to be used by the switch | Dropdown | False | local | tacacs, local | | role | Role of the switch in the InfiniBand network | Dropdown | | | spine, leaf | | split\_ready | Enable Split-Ready profile for switch | Boolean | | False | | | ipv6\_enable | Enable IPv6 on the switch | Boolean | | False | | | cli\_prefix\_modes | Enable CLI prefix modes | Boolean | | True | | | xml\_gateway | Enable XML Gateway | Boolean | | False | | | ssh\_server\_security\_strict | Enable SSH Server Security Strict | Boolean | | False | | | banner | Banner to be displayed on login | Text | | NVIDIA MLNX-OS Switch Management | | | password\_hardening | Enable Password Hardening | Boolean | | False | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ----------------------------- | -------- | ----------- | --------- | | interfaces | InfinibandSwitchInterface | True | many | Component | | mgmt\_interface | InfinibandSwitchMgmtInterface | True | one | Component | | rsu | InfinibandRSU | True | one | Attribute | ### SwitchInterface[​](#switchinterface "Direct link to SwitchInterface") * **Label:** InfiniBand Switch Interface * **Description:** InfiniBand Switch Interface * **Namespace:** Infiniband * **Icon:** mdi :ethernet * **Uniqueness Constraints:** * switch, name\_\_value * **Human Friendly ID:** switch\_\_name\_\_value, name\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------------------------- | ----------------------------------------------------------------------------------------------- | -------- | -------- | -------------- | ------------------------- | | name | | Text | | | | | description | | Text | True | | | | speed | | Number | | | | | enabled | | Boolean | | True | | | width | Width value sets supported lane options for the interface | Number | True | 7 | | | port\_type | Enable interface to be split X times (requires Split-Ready profile to be enabled on the switch) | Number | True | | | | port\_type\_force | Force the configured port type setting; use in conjunction with Port Type | Boolean | | False | | | operational\_virtual\_lanes | Number of operational virtual lanes for an interface | Number | | 8 | | | mtu | Maximum Transmission Unit (bytes) | Number | | | | | speed\_forced | Force the configured speed setting(s); use in conjunction with Speed options | Boolean | | False | | | sfp\_type | Type of SFP module used in the interface | Text | True | | | | role | Role of the interface in the InfiniBand network | Dropdown | True | | endhost, uplink, reserved | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | -------------------------------- | -------- | ----------- | --------- | | speed\_option | InfinibandSwitchIntfSpeedOptions | True | many | Attribute | | switch | InfinibandSwitch | False | one | Parent | ### SwitchMgmtInterface[​](#switchmgmtinterface "Direct link to SwitchMgmtInterface") * **Label:** InfiniBand Mgmt Interface * **Description:** InfiniBand Switch Management Interface * **Namespace:** Infiniband * **Icon:** mdi :ethernet * **Uniqueness Constraints:** * switch, name\_\_value * **Human Friendly ID:** switch\_\_name\_\_value, name\_\_value #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ---------------------------------------- | ------- | -------- | -------------- | ------- | | name | | Text | | | | | description | | Text | True | | | | speed | | Number | | | | | mtu | | Number | | 1500 | | | enabled | | Boolean | | True | | | dhcp | Enable DHCP for the management interface | Boolean | | True | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | ---------------- | -------- | ----------- | ------ | | switch | InfinibandSwitch | False | one | Parent | | ipv4\_address | IpamIPAddress | True | one | | ### SwitchIntfSpeedOptions[​](#switchintfspeedoptions "Direct link to SwitchIntfSpeedOptions") * **Label:** InfiniBand Interface Speed Options * **Description:** InfiniBand Interface Speed Options * **Namespace:** Infiniband * **Icon:** mdi :ethernet * **Human Friendly ID:** description\_\_value #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ------------------------------------------------ | -------- | -------- | -------------- | ---------------------------- | | speed | Speed of the interface | Dropdown | | | SDR, NDR, QDR, FDR, EDR, HDR | | description | Description of the speed option (must be unique) | Text | | | | #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | ------------------------- | -------- | ----------- | ------ | | switch | InfinibandSwitchInterface | False | one | Parent | ### RSU[​](#rsu "Direct link to RSU") * **Label:** InfiniBand RSU * **Description:** InfiniBand Rail-Optimized Scalable Unit * **Namespace:** Infiniband * **Icon:** mdi :network * **Uniqueness Constraints:** * identifier\_\_value #### Attributes[​](#attributes-4 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------- | ---------------------------------------------------------- | ------ | -------- | -------------- | ------- | | identifier | InfiniBand RSU Identifier (A-Z: a unique character string) | Text | | | | | size | Number of InfiniBand Leaf Switches (Rails) in the RSU | Number | True | | | #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------- | ------------------------- | -------- | ----------- | --------- | | switches | InfinibandSwitch | True | many | Attribute | | interfaces | InfinibandSwitchInterface | True | many | Attribute | ### Fabric[​](#fabric "Direct link to Fabric") * **Label:** InfiniBand Network Fabric * **Description:** InfiniBand Network Fabric * **Namespace:** Infiniband * **Icon:** mdi :network * **Human Friendly ID:** name\_\_value * **Inherit From:** CoreArtifactTarget #### Attributes[​](#attributes-5 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ------------------------------ | ---- | -------- | -------------- | ------- | | name | InfiniBand Network Fabric Name | Text | | | | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Switch namespace: Infiniband description: InfiniBand Switch label: InfiniBand Switch icon: mdi:server inherit_from: - CoreArtifactTarget order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true order_weight: 1000 - name: description kind: Text optional: true order_weight: 2000 - name: os_version kind: Text optional: true order_weight: 2200 - name: rsu_rail_id kind: Number label: RSU Rail ID description: InfiniBand RSU Rail ID (numeric) assigned to the switch optional: true - name: aaa_authentication_model kind: Dropdown label: AAA Authentication Model description: AAA authentication model to be used by the switch optional: false default_value: local choices: - name: tacacs label: TACACS+ description: Default AAA authentication will use TACACS+ server color: '#7fbf71' - name: local label: local description: Default AAA authentication will use local AAA configuration color: '#7fbf72' - name: aaa_accounting_model kind: Dropdown label: AAA Accounting Model description: AAA accounting model to be used by the switch default_value: local optional: false choices: - name: tacacs label: TACACS+ description: Default AAA accounting will use TACACS+ server color: '#7fbf71' - name: local label: local description: Default AAA accounting will use local AAA configuration color: '#ffff7f' - name: role kind: Dropdown label: Switch Role description: Role of the switch in the InfiniBand network choices: - name: spine label: Spine description: Spine Switch color: '#7fbf7f' - name: leaf label: Leaf description: Leaf Switch color: '#ffff7f' - name: split_ready kind: Boolean label: Split Ready description: Enable Split-Ready profile for switch default_value: false - name: ipv6_enable kind: Boolean label: IPv6 Enable description: Enable IPv6 on the switch default_value: false - name: cli_prefix_modes kind: Boolean label: CLI Prefix Modes description: Enable CLI prefix modes default_value: true - name: xml_gateway kind: Boolean label: XML Gateway description: Enable XML Gateway default_value: false - name: ssh_server_security_strict kind: Boolean label: SSH Server Security Strict description: Enable SSH Server Security Strict default_value: false - name: banner kind: Text label: Banner description: Banner to be displayed on login default_value: NVIDIA MLNX-OS Switch Management - name: password_hardening kind: Boolean label: Password Hardening description: Enable Password Hardening default_value: false relationships: - name: interfaces kind: Component cardinality: many peer: InfinibandSwitchInterface optional: true - name: mgmt_interface kind: Component cardinality: one peer: InfinibandSwitchMgmtInterface optional: true - name: rsu peer: InfinibandRSU optional: true cardinality: one kind: Attribute - name: SwitchInterface namespace: Infiniband description: InfiniBand Switch Interface label: InfiniBand Switch Interface icon: mdi:ethernet human_friendly_id: - switch__name__value - name__value order_by: - name__value display_label: name__value uniqueness_constraints: - - switch - name__value attributes: - name: name kind: Text order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 - name: speed kind: Number order_weight: 1400 - name: enabled kind: Boolean default_value: true order_weight: 1200 - name: width kind: Number label: Width description: Width value sets supported lane options for the interface optional: true default_value: 7 enum: - 1 - 3 - 5 - 7 - name: port_type kind: Number label: 'Port Type: Split' description: Enable interface to be split X times (requires Split-Ready profile to be enabled on the switch) optional: true enum: - 2 - name: port_type_force kind: Boolean label: Port Type Force description: Force the configured port type setting; use in conjunction with Port Type default_value: false - name: operational_virtual_lanes kind: Number label: Operational Virtual Lanes description: Number of operational virtual lanes for an interface default_value: 8 enum: - 1 - 2 - 4 - 8 - name: mtu kind: Number label: MTU (bytes) description: Maximum Transmission Unit (bytes) enum: - 256 - 512 - 1024 - 2048 - 4096 - name: speed_forced kind: Boolean label: Speed Forced description: Force the configured speed setting(s); use in conjunction with Speed options default_value: false - name: sfp_type kind: Text label: SFP Type description: Type of SFP module used in the interface optional: true - name: role kind: Dropdown label: Interface Role description: Role of the interface in the InfiniBand network optional: true choices: - name: endhost label: Network Device Endhost Port description: InfiniBand Switch Interface <> Server HCA color: '#98b2d1' - name: uplink label: Uplink description: InfiniBand Switch Interface <> InfiniBand Switch Interface color: '#93e9be' - name: reserved label: Reserved description: Reserved for future use color: '#d3d3d3' relationships: - name: speed_option peer: InfinibandSwitchIntfSpeedOptions optional: true cardinality: many kind: Attribute max_count: 6 - name: switch peer: InfinibandSwitch optional: false cardinality: one kind: Parent - name: SwitchMgmtInterface namespace: Infiniband description: InfiniBand Switch Management Interface label: InfiniBand Mgmt Interface icon: mdi:ethernet human_friendly_id: - switch__name__value - name__value order_by: - name__value display_label: name__value uniqueness_constraints: - - switch - name__value attributes: - name: name kind: Text order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 - name: speed kind: Number order_weight: 1400 - name: mtu label: MTU default_value: 1500 kind: Number order_weight: 1500 - name: enabled kind: Boolean default_value: true order_weight: 1200 - name: dhcp kind: Boolean label: DHCP description: Enable DHCP for the management interface default_value: true relationships: - name: switch peer: InfinibandSwitch optional: false cardinality: one kind: Parent - name: ipv4_address peer: IpamIPAddress optional: true cardinality: one - name: SwitchIntfSpeedOptions namespace: Infiniband description: InfiniBand Interface Speed Options label: InfiniBand Interface Speed Options icon: mdi:ethernet human_friendly_id: - description__value order_by: - speed__value display_label: speed__value attributes: - name: speed kind: Dropdown label: Speed description: Speed of the interface choices: - name: SDR label: SDR - name: NDR label: NDR - name: QDR label: QDR - name: FDR label: FDR - name: EDR label: EDR - name: HDR label: HDR - name: description kind: Text label: Description description: Description of the speed option (must be unique) unique: true relationships: - name: switch peer: InfinibandSwitchInterface optional: false cardinality: one kind: Parent - name: RSU namespace: Infiniband description: InfiniBand Rail-Optimized Scalable Unit label: InfiniBand RSU icon: mdi:network order_by: - identifier__value display_label: identifier__value uniqueness_constraints: - - identifier__value attributes: - name: identifier kind: Text label: Identifier description: 'InfiniBand RSU Identifier (A-Z: a unique character string)' regex: '[A-Z]' - name: size kind: Number label: Size description: Number of InfiniBand Leaf Switches (Rails) in the RSU optional: true relationships: - name: switches peer: InfinibandSwitch optional: true cardinality: many kind: Attribute - name: interfaces peer: InfinibandSwitchInterface optional: true cardinality: many kind: Attribute - name: Fabric namespace: Infiniband description: InfiniBand Network Fabric label: InfiniBand Network Fabric human_friendly_id: - name__value icon: mdi:network inherit_from: - CoreArtifactTarget order_by: - name__value display_label: name__value attributes: - name: name kind: Text label: Name description: InfiniBand Network Fabric Name unique: true ``` --- # Interface Breakout This schema extension introduces relationships to support breakout interfaces, enabling you to document the breakout of a physical interface into smaller physical interfaces. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### InterfacePhysical[​](#interfacephysical "Direct link to InterfacePhysical") #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------------- | -------------------------------------------------- | ------- | -------- | -------------- | ------- | | breakout\_capability | Indicates if the port supports breakout capability | Boolean | False | False | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------------------- | ----------------- | -------- | ----------- | --------- | | breakout\_child\_interfaces | InterfacePhysical | True | many | Attribute | | breakout\_parent\_interface | InterfacePhysical | True | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' extensions: nodes: - kind: InterfacePhysical attributes: - name: breakout_capability kind: Boolean default_value: false optional: false description: Indicates if the port supports breakout capability order_weight: 1800 relationships: - name: breakout_child_interfaces peer: InterfacePhysical label: Breakout child interface(s) kind: Attribute cardinality: many optional: true identifier: physical__breakout direction: outbound description: Interfaces resulting from the breakout order_weight: 1850 - name: breakout_parent_interface peer: InterfacePhysical label: Breakout parent interface kind: Attribute cardinality: one optional: true identifier: physical__breakout direction: inbound description: Interface from which breakout is created order_weight: 1900 ``` --- # IPAM Base IPAM schema: IP prefixes and IP addresses, plus the scope generics a prefix or a VLAN group can be attached to. ## Details[​](#details "Direct link to Details") * **Dependencies:** No dependencies ## Nodes[​](#nodes "Direct link to Nodes") ### Prefix[​](#prefix "Direct link to Prefix") * **Label:** Prefix * **Description:** IPv4 or IPv6 network (with mask) * **Namespace:** Ipam * **Icon:** mdi :ip-network * **Uniqueness Constraints:** * prefix\_\_value, ip\_namespace * **Human Friendly ID:** ip\_namespace\_\_name\_\_value, prefix\_\_value * **Inherit From:** BuiltinIPPrefix #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | -------- | -------- | -------------- | ------------------------------------ | | status | | Dropdown | False | active | active, reserved, deprecated | | description | | Text | True | | | | role | | Dropdown | True | | management, link, customer, backbone | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | --------------- | -------- | ----------- | --------- | | scope | IpamPrefixScope | True | one | Attribute | ### IPAddress[​](#ipaddress "Direct link to IPAddress") * **Label:** IP Address * **Description:** IP Address * **Namespace:** Ipam * **Icon:** mdi :ip * **Uniqueness Constraints:** * address\_\_value, ip\_namespace * **Human Friendly ID:** ip\_namespace\_\_name\_\_value, address\_\_value * **Inherit From:** BuiltinIPAddress #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | -------- | -------- | -------------- | ---------------------------- | | description | | Text | True | | | | status | | Dropdown | False | active | active, reserved, deprecated | | role | | Dropdown | True | | loopback, secondary, anycast | | fqdn | | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------- | ----------------- | -------- | ----------- | --------- | | primary\_of | DcimGenericDevice | True | many | Attribute | | interface | InterfaceLayer3 | True | one | Attribute | ## Generics[​](#generics "Direct link to Generics") ### VLANGroupScope[​](#vlangroupscope "Direct link to VLANGroupScope") * **Label:** VLAN Group Scope * **Description:** Mixin for objects that can scope a VLAN group (e.g. Region, Site). The reverse relationship is added by extensions/vlan. * **Namespace:** Ipam ### PrefixScope[​](#prefixscope "Direct link to PrefixScope") * **Label:** Prefix Scope * **Description:** Mixin for objects that can scope IP prefixes (e.g. Region, Site). * **Namespace:** Ipam #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | ---------- | -------- | ----------- | ------- | | prefixes | IpamPrefix | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: VLANGroupScope namespace: Ipam description: Mixin for objects that can scope a VLAN group (e.g. Region, Site). The reverse relationship is added by extensions/vlan. label: VLAN Group Scope include_in_menu: false - name: PrefixScope namespace: Ipam description: Mixin for objects that can scope IP prefixes (e.g. Region, Site). label: Prefix Scope include_in_menu: false relationships: - name: prefixes peer: IpamPrefix kind: Generic cardinality: many optional: true identifier: prefix__prefix_scope order_weight: 1550 nodes: - name: Prefix namespace: Ipam description: IPv4 or IPv6 network (with mask) label: Prefix icon: mdi:ip-network include_in_menu: false inherit_from: - BuiltinIPPrefix human_friendly_id: - ip_namespace__name__value - prefix__value order_by: - prefix__value display_label: prefix__value uniqueness_constraints: - - prefix__value - ip_namespace attributes: - name: status kind: Dropdown default_value: active choices: - name: active label: Active color: '#00d25b' - name: reserved label: Reserved color: '#4d90fe' - name: deprecated label: Deprecated color: '#e04040' optional: false order_weight: 1200 - name: description kind: Text optional: true order_weight: 1100 - name: role kind: Dropdown choices: - name: management label: Management description: Out-of-band management network. color: '#AEC6CF' - name: link label: Link description: Point-to-point links between devices. color: '#FDFD96' - name: customer label: Customer description: Prefixes assigned to customers. color: '#77DD77' - name: backbone label: Backbone description: Core network infrastructure prefixes. color: '#9678B6' optional: true order_weight: 1300 relationships: - name: scope peer: IpamPrefixScope kind: Attribute cardinality: one optional: true identifier: prefix__prefix_scope order_weight: 1500 - name: IPAddress namespace: Ipam description: IP Address label: IP Address icon: mdi:ip include_in_menu: false inherit_from: - BuiltinIPAddress human_friendly_id: - ip_namespace__name__value - address__value order_by: - address__value display_label: address__value uniqueness_constraints: - - address__value - ip_namespace attributes: - name: description kind: Text optional: true order_weight: 1100 - name: status kind: Dropdown default_value: active choices: - name: active label: Active color: '#00d25b' - name: reserved label: Reserved color: '#4d90fe' - name: deprecated label: Deprecated color: '#e04040' optional: false order_weight: 1200 - name: role kind: Dropdown choices: - name: loopback label: Loopback color: '#B0A8B9' - name: secondary label: Secondary color: '#AEC6CF' - name: anycast label: Anycast color: '#FDFD96' optional: true order_weight: 1300 - name: fqdn kind: Text label: FQDN regex: (?=^.{1,253}$)(^(((?!-)[a-zA-Z0-9-]{1,63}(? :ip-network * **Uniqueness Constraints:** * prefix\_\_value * **Human Friendly ID:** prefix\_\_value * **Inherit From:** BuiltinIPPrefix #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | -------- | -------- | -------------- | ------- | | description | | Text | True | | | | date\_added | | DateTime | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | --------------- | -------- | ----------- | --------- | | rir | OrganizationRIR | False | one | Attribute | ### RIR[​](#rir "Direct link to RIR") * **Label:** RIR * **Description:** A Regional Internet Registry (ARIN, RIPE, APNIC, LACNIC, AFRINIC) or a private entity managing private address space. * **Namespace:** Organization * **Icon:** mdi :certificate * **Inherit From:** OrganizationGeneric #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------- | ----------------------------------------------------------------------- | ------- | -------- | -------------- | ------- | | private | Indicates this RIR manages private (RFC 1918 / RFC 4193) address space. | Boolean | False | False | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------- | ------------- | -------- | ----------- | ------- | | aggregates | IpamAggregate | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Aggregate namespace: Ipam description: An aggregate is a top-level IPv4 or IPv6 prefix assigned by a RIR. label: Aggregate icon: mdi:ip-network include_in_menu: true menu_placement: BuiltinIPAM inherit_from: - BuiltinIPPrefix human_friendly_id: - prefix__value order_by: - prefix__value display_label: prefix__value uniqueness_constraints: - - prefix__value attributes: - name: description kind: Text optional: true order_weight: 1100 - name: date_added kind: DateTime label: Date Added optional: true order_weight: 1200 relationships: - name: rir peer: OrganizationRIR label: RIR kind: Attribute cardinality: one optional: false identifier: aggregate__rir order_weight: 1000 - name: RIR namespace: Organization description: A Regional Internet Registry (ARIN, RIPE, APNIC, LACNIC, AFRINIC) or a private entity managing private address space. label: RIR icon: mdi:certificate include_in_menu: true menu_placement: OrganizationGeneric inherit_from: - OrganizationGeneric attributes: - name: private kind: Boolean label: Private default_value: false optional: false description: Indicates this RIR manages private (RFC 1918 / RFC 4193) address space. order_weight: 1300 relationships: - name: aggregates peer: IpamAggregate kind: Generic cardinality: many optional: true identifier: aggregate__rir order_weight: 1400 ``` --- # LAG This schema extension includes models for Link Aggregation Groups (LAGs), enabling you to link physical interfaces as building blocks of your LAG interface. It can be used in standard networking environments as well as in compute scenarios, such as capturing bond interfaces. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Lag[​](#lag "Direct link to Lag") * **Label:** LAG Interface * **Description:** LAG interface * **Namespace:** Interface * **Inherit From:** DcimInterface, InterfaceLayer2, InterfaceLayer3, InterfaceHasSubInterface, GenericInterfaceBundle #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------- | -------------------------------------- | -------- | -------- | -------------- | ------------------------- | | lacp\_rate | LACP rate for the aggregated interface | Dropdown | False | fast | slow, fast | | lacp\_mode | LACP mode for the aggregated interface | Dropdown | False | active | active, passive, disabled | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ----------------- | -------- | ----------- | --------- | | bundle\_members | InterfacePhysical | True | many | Attribute | ## Generics[​](#generics "Direct link to Generics") ### InterfaceBundle[​](#interfacebundle "Direct link to InterfaceBundle") * **Label:** Generic Interface Bundle * **Description:** Generic holding the members of an interface bundle such as a LAG. * **Namespace:** Generic #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ------------------------------------------ | ------ | -------- | -------------- | ------- | | name | Name of the interface | Text | False | | | | bundle\_number | Bundle number for the aggregated interface | Number | False | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ----------------- | -------- | ----------- | --------- | | bundle\_members | InterfacePhysical | True | many | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### InterfacePhysical[​](#interfacephysical "Direct link to InterfacePhysical") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | ---------------------- | -------- | ----------- | --------- | | bundle | GenericInterfaceBundle | True | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: InterfaceBundle namespace: Generic description: Generic holding the members of an interface bundle such as a LAG. label: Generic Interface Bundle include_in_menu: false attributes: - name: name kind: Text optional: false description: Name of the interface order_weight: 1000 - name: bundle_number kind: Number optional: false description: Bundle number for the aggregated interface order_weight: 1250 relationships: - name: bundle_members peer: InterfacePhysical label: Member(s) kind: Attribute cardinality: many optional: true identifier: interface__bundle description: Physical Interfaces that are members of this aggregate order_weight: 1800 nodes: - name: Lag namespace: Interface description: LAG interface label: LAG Interface include_in_menu: false inherit_from: - DcimInterface - InterfaceLayer2 - InterfaceLayer3 - InterfaceHasSubInterface - GenericInterfaceBundle attributes: - name: lacp_rate kind: Dropdown label: LACP Rate default_value: fast choices: - name: slow label: Slow color: '#E6E6FA' - name: fast label: Fast color: '#E6E6FA' optional: false description: LACP rate for the aggregated interface order_weight: 1850 - name: lacp_mode kind: Dropdown label: LACP Mode default_value: active choices: - name: active label: Active color: '#E6E6FA' - name: passive label: Passive color: '#E6E6FA' - name: disabled label: Disabled color: '#E6E6FA' optional: false description: LACP mode for the aggregated interface order_weight: 1900 relationships: - name: bundle_members peer: InterfacePhysical label: Member(s) kind: Attribute cardinality: many optional: true identifier: interface__bundle common_parent: device description: Physical Interfaces that are members of this aggregate order_weight: 1800 extensions: nodes: - kind: InterfacePhysical relationships: - name: bundle peer: GenericInterfaceBundle label: Interface Bundle kind: Attribute cardinality: one optional: true identifier: interface__bundle description: Interface Bundle using this Physical Interface order_weight: 1950 ``` --- # Location Base Location schema: the generic every location tier inherits from, and the hosting generic that lets devices and other objects be placed at a location. Concrete tiers such as Site come from the location extensions. ## Details[​](#details "Direct link to Details") * **Dependencies:** No dependencies ## Generics[​](#generics "Direct link to Generics") ### Generic[​](#generic "Direct link to Generic") * **Label:** Location * **Description:** A location in the estate, such as a region, a country or a site. * **Namespace:** Location * **Icon:** mingcute :location-line * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | ---------- | -------- | ----------- | --------- | | tags | BuiltinTag | True | many | Attribute | ### Hosting[​](#hosting "Direct link to Hosting") * **Description:** Location directly hosting device and services. * **Namespace:** Location * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------- | ------------------ | -------- | ----------- | ------- | | devices | DcimPhysicalDevice | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: Generic namespace: Location description: A location in the estate, such as a region, a country or a site. label: Location icon: mingcute:location-line include_in_menu: true hierarchical: true human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: tags peer: BuiltinTag kind: Attribute cardinality: many optional: true order_weight: 2000 - name: Hosting namespace: Location description: Location directly hosting device and services. include_in_menu: false human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 relationships: - name: devices peer: DcimPhysicalDevice label: Devices kind: Generic cardinality: many optional: true identifier: physical_device__location order_weight: 1500 ``` --- # Location Extended This schema extension is the most detailed when it comes to location, you'll find all the layers you can think of. It defines its own hierarchical Location.Rack, incompatible with the flat one extensions/rack defines, so load one or the other, not both. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Continent[​](#continent "Direct link to Continent") * **Label:** Continent * **Namespace:** Location * **Icon:** jam :world * **Inherit From:** LocationGeneric ### Country[​](#country "Direct link to Country") * **Label:** Country * **Namespace:** Location * **Icon:** gis :search-country * **Inherit From:** LocationGeneric ### Region[​](#region "Direct link to Region") * **Label:** Region * **Namespace:** Location * **Icon:** carbon :cics-region-target * **Inherit From:** LocationGeneric ### Metro[​](#metro "Direct link to Metro") * **Label:** Metro * **Namespace:** Location * **Icon:** healthicons :city * **Inherit From:** LocationGeneric ### Building[​](#building "Direct link to Building") * **Label:** Building * **Namespace:** Location * **Icon:** ri :building-line * **Inherit From:** LocationGeneric #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------------- | ----------- | ---- | -------- | -------------- | ------- | | facility\_id | | Text | True | | | | physical\_address | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | ------------------- | -------- | ----------- | ---- | | owner | OrganizationGeneric | True | one | | ### Floor[​](#floor "Direct link to Floor") * **Label:** Floor * **Namespace:** Location * **Icon:** mdi :home-floor-0 * **Inherit From:** LocationGeneric ### Suite[​](#suite "Direct link to Suite") * **Label:** Suite * **Namespace:** Location * **Icon:** game-icons :cage * **Inherit From:** LocationGeneric #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------ | ----------- | ---- | -------- | -------------- | ------- | | facility\_id | | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | ------------------- | -------- | ----------- | ---- | | owner | OrganizationGeneric | True | one | | ### Rack[​](#rack "Direct link to Rack") * **Label:** Rack * **Namespace:** Location * **Icon:** clarity :rack-server-line * **Human Friendly ID:** name\_\_value * **Inherit From:** LocationGeneric #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------ | ----------- | ---- | -------- | -------------- | ------- | | facility\_id | | Text | True | | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | ------------------- | -------- | ----------- | ---- | | owner | OrganizationGeneric | True | one | | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### OrganizationProvider[​](#organizationprovider "Direct link to OrganizationProvider") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | ---------------- | -------- | ----------- | ---- | | location | LocationBuilding | True | many | | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Continent namespace: Location inherit_from: - LocationGeneric include_in_menu: true menu_placement: LocationGeneric label: Continent display_label: name__value parent: '' children: LocationCountry icon: jam:world - name: Country namespace: Location inherit_from: - LocationGeneric include_in_menu: true menu_placement: LocationGeneric label: Country display_label: name__value parent: LocationContinent children: LocationRegion icon: gis:search-country - name: Region namespace: Location inherit_from: - LocationGeneric include_in_menu: true menu_placement: LocationGeneric label: Region display_label: name__value parent: LocationCountry children: LocationMetro icon: carbon:cics-region-target - name: Metro namespace: Location inherit_from: - LocationGeneric include_in_menu: true menu_placement: LocationGeneric label: Metro display_label: name__value parent: LocationRegion children: LocationBuilding icon: healthicons:city - name: Building namespace: Location inherit_from: - LocationGeneric include_in_menu: true menu_placement: LocationGeneric label: Building display_label: name__value parent: LocationMetro children: LocationFloor icon: ri:building-line attributes: - name: facility_id kind: Text unique: false optional: true order_weight: 1100 - name: physical_address kind: Text unique: false optional: true order_weight: 1500 relationships: - name: owner peer: OrganizationGeneric optional: true cardinality: one - name: Floor namespace: Location inherit_from: - LocationGeneric include_in_menu: true menu_placement: LocationGeneric label: Floor display_label: name__value parent: LocationBuilding icon: mdi:home-floor-0 children: LocationSuite - name: Suite namespace: Location inherit_from: - LocationGeneric include_in_menu: true menu_placement: LocationGeneric label: Suite display_label: name__value parent: LocationFloor children: LocationRack icon: game-icons:cage attributes: - name: facility_id kind: Text unique: false optional: true order_weight: 1100 relationships: - name: owner peer: OrganizationGeneric optional: true cardinality: one - name: Rack namespace: Location inherit_from: - LocationGeneric include_in_menu: true menu_placement: LocationGeneric label: Rack display_label: name__value human_friendly_id: - name__value parent: LocationSuite children: '' icon: clarity:rack-server-line attributes: - name: facility_id kind: Text unique: false optional: true order_weight: 1100 relationships: - name: owner peer: OrganizationGeneric optional: true cardinality: one extensions: nodes: - kind: OrganizationProvider relationships: - name: location peer: LocationBuilding cardinality: many optional: true ``` --- # Location Minimal This schema extension provides a self-contained Region -> Country -> Metro -> Site hierarchy for storing location data, with the Site carrying facility, physical address, timezone and status. Its Site node is the same as the one in extensions/location\_site, so the two can be loaded together. A location name is unique across every tier, so a single-country deployment should enter the hierarchy at Country, with Region as the national node, rather than repeating a country under several regions. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Region[​](#region "Direct link to Region") * **Label:** Region * **Description:** A geographical region grouping countries and sites. * **Namespace:** Location * **Icon:** carbon :cics-region-target * **Inherit From:** LocationGeneric, IpamPrefixScope ### Country[​](#country "Direct link to Country") * **Label:** Country * **Description:** A country, sitting between a region and the metros it contains. * **Namespace:** Location * **Icon:** gis :search-country * **Inherit From:** LocationGeneric ### Metro[​](#metro "Direct link to Metro") * **Label:** Metro * **Description:** A metropolitan area grouping the sites of one city or urban region. * **Namespace:** Location * **Icon:** healthicons :city * **Inherit From:** LocationGeneric ### Site[​](#site "Direct link to Site") * **Label:** Site * **Description:** A physical location hosting equipment, such as a data center or an office. * **Namespace:** Location * **Icon:** ri :building-line * **Inherit From:** LocationGeneric, LocationHosting, IpamPrefixScope, IpamVLANGroupScope #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------------- | ----------- | -------- | -------- | -------------- | ----------------------------------------------------- | | status | | Dropdown | False | active | active, planned, staging, decommissioning, deprecated | | facility | | Text | True | | | | physical\_address | | Text | True | | | | timezone | | Text | True | | | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Region namespace: Location description: A geographical region grouping countries and sites. label: Region icon: carbon:cics-region-target include_in_menu: true menu_placement: LocationGeneric inherit_from: - LocationGeneric - IpamPrefixScope parent: '' children: LocationCountry - name: Country namespace: Location description: A country, sitting between a region and the metros it contains. label: Country icon: gis:search-country include_in_menu: true menu_placement: LocationGeneric inherit_from: - LocationGeneric parent: LocationRegion children: LocationMetro - name: Metro namespace: Location description: A metropolitan area grouping the sites of one city or urban region. label: Metro icon: healthicons:city include_in_menu: true menu_placement: LocationGeneric inherit_from: - LocationGeneric parent: LocationCountry children: LocationSite - name: Site namespace: Location description: A physical location hosting equipment, such as a data center or an office. label: Site icon: ri:building-line include_in_menu: true menu_placement: LocationGeneric inherit_from: - LocationGeneric - LocationHosting - IpamPrefixScope - IpamVLANGroupScope parent: LocationMetro display_label: name__value attributes: - name: status kind: Dropdown default_value: active choices: - name: active label: Active color: '#00d25b' - name: planned label: Planned color: '#2196f3' - name: staging label: Staging color: '#f0ad4e' - name: decommissioning label: Decommissioning color: '#ff9800' - name: deprecated label: Deprecated color: '#6c757d' optional: false order_weight: 1200 - name: facility kind: Text unique: false optional: true order_weight: 1150 - name: physical_address kind: Text unique: false optional: true order_weight: 1400 - name: timezone kind: Text optional: true order_weight: 1300 ``` --- # Location Site This schema extension introduces a Site node with facility, physical address, timezone and status, for deployments that want a flat list of sites without a hierarchy. It is the same Site node as in extensions/location\_minimal, which adds Region, Country and Metro tiers above it. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Site[​](#site "Direct link to Site") * **Label:** Site * **Description:** A physical location hosting equipment, such as a data center or an office. * **Namespace:** Location * **Icon:** ri :building-line * **Inherit From:** LocationGeneric, LocationHosting, IpamPrefixScope, IpamVLANGroupScope #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------------- | ----------- | -------- | -------- | -------------- | ----------------------------------------------------- | | status | | Dropdown | False | active | active, planned, staging, decommissioning, deprecated | | facility | | Text | True | | | | physical\_address | | Text | True | | | | timezone | | Text | True | | | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Site namespace: Location description: A physical location hosting equipment, such as a data center or an office. label: Site icon: ri:building-line include_in_menu: true menu_placement: LocationGeneric inherit_from: - LocationGeneric - LocationHosting - IpamPrefixScope - IpamVLANGroupScope display_label: name__value attributes: - name: status kind: Dropdown default_value: active choices: - name: active label: Active color: '#00d25b' - name: planned label: Planned color: '#2196f3' - name: staging label: Staging color: '#f0ad4e' - name: decommissioning label: Decommissioning color: '#ff9800' - name: deprecated label: Deprecated color: '#6c757d' optional: false order_weight: 1200 - name: facility kind: Text unique: false optional: true order_weight: 1150 - name: physical_address kind: Text unique: false optional: true order_weight: 1400 - name: timezone kind: Text optional: true order_weight: 1300 ``` --- # MLAG This schema extension contains the foundations to capture Multi-Chassis Link Aggregation Groups (MLAG). It comes on top of the LAG extension. In this implementation, a MLAG interface is essentially a LAG interface but linked to a MLAG domain (instead of a device). The MLAG domain regroups devices together (usually 2) and is built over LAG interfaces used as peer-link between the devices. MLAG interfaces defined at the MLAG domain level are then spread across all devices in the domain. This is a deliberately minimal implementation of MLAG, meant to blend with models you already have. For example, in a data center fabric you might already have a LeafGroup or similar concept: have it inherit from GenericMlagDomain and add the relationships and attributes you need. Not covered yet: the layer 3 overlay for MLAG interfaces (loopback, peer address ...) and surfacing MLAG interfaces on each device in the domain. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/lag](/schema-library/reference/lag.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Interface[​](#interface "Direct link to Interface") * **Label:** MLAG Interface * **Description:** Multi-Chassis Link Aggregation Group Interface * **Namespace:** Mlag * **Uniqueness Constraints:** * mlag\_domain, name\_\_value * **Human Friendly ID:** mlag\_domain\_\_domain\_id\_\_value, name\_\_value * **Inherit From:** InterfaceLayer2, GenericInterfaceBundle #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ---------------------------------------- | -------- | -------- | -------------- | ------------------------------------------------------------------ | | name | Name of the interface | Text | False | | | | description | A brief description of the interface | Text | True | | | | status | The status of the interface | Dropdown | False | active | provisioning, free, active, maintenance, disabled, deleted, outage | | role | The role of the interface in the network | Dropdown | True | | server, router | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ----------------- | -------- | ----------- | ------ | | mlag\_domain | GenericMlagDomain | False | one | Parent | ### Domain[​](#domain "Direct link to Domain") * **Label:** MLAG Domain * **Description:** MLAG (Multi-Chassis Link Aggregation Group) Domain * **Namespace:** Mlag * **Icon:** mingcute :cloud-line * **Inherit From:** GenericMlagDomain #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | ---------- | -------- | ----------- | ------- | | peers | DcimDevice | False | many | Generic | ## Generics[​](#generics "Direct link to Generics") ### MlagDomain[​](#mlagdomain "Direct link to MlagDomain") * **Label:** MLAG Domain * **Description:** Generic holding the peer links and interfaces shared by the members of an MLAG domain. * **Namespace:** Generic * **Human Friendly ID:** domain\_id\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------- | ------------------------------------------------------------------ | ------ | -------- | -------------- | ------- | | domain\_id | Identifier for the MLAG domain | Text | False | | | | reload\_delay | Time in seconds to wait before bringing up the MLAG after a reload | Number | False | 300 | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | ------------- | -------- | ----------- | --------- | | peer\_links | InterfaceLag | True | many | Attribute | | mlag\_interfaces | MlagInterface | True | many | Component | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### InterfaceLag[​](#interfacelag "Direct link to InterfaceLag") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | ---------- | -------- | ----------- | --------- | | mlag\_peer\_link | MlagDomain | True | one | Attribute | ### DcimDevice[​](#dcimdevice "Direct link to DcimDevice") #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ---------- | -------- | ----------- | --------- | | mlag\_domain | MlagDomain | True | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: MlagDomain namespace: Generic description: Generic holding the peer links and interfaces shared by the members of an MLAG domain. label: MLAG Domain include_in_menu: false human_friendly_id: - domain_id__value display_label: domain_id__value attributes: - name: domain_id kind: Text label: MLAG Domain ID unique: true optional: false description: Identifier for the MLAG domain order_weight: 1000 - name: reload_delay kind: Number label: Reload Delay default_value: 300 optional: false description: Time in seconds to wait before bringing up the MLAG after a reload order_weight: 1650 relationships: - name: peer_links peer: InterfaceLag label: Peer Links kind: Attribute cardinality: many optional: true description: LAG interfaces used as peer-links between the MLAG peers order_weight: 1500 - name: mlag_interfaces peer: MlagInterface label: MLAG Interface(s) kind: Component cardinality: many optional: true description: Bundle interfaces that are spread across all devices in this MLAG domain. order_weight: 1550 nodes: - name: Interface namespace: Mlag description: Multi-Chassis Link Aggregation Group Interface label: MLAG Interface include_in_menu: false inherit_from: - InterfaceLayer2 - GenericInterfaceBundle human_friendly_id: - mlag_domain__domain_id__value - name__value order_by: - name__value display_label: name__value uniqueness_constraints: - - mlag_domain - name__value attributes: - name: name kind: Text optional: false description: Name of the interface order_weight: 1000 - name: description kind: Text optional: true description: A brief description of the interface order_weight: 1100 - name: status kind: Dropdown default_value: active choices: - name: provisioning label: Provisioning description: Interface is being provisioned. color: '#A9DFBF' - name: free label: Free description: Interface is unused. color: '#CDEACC' - name: active label: Active description: Interface is active and operational. color: '#00d25b' - name: maintenance label: Maintenance description: Interface is under maintenance. color: '#FFF2CC' - name: disabled label: Disabled description: Interface has been disabled. color: '#D3D3D3' - name: deleted label: Deleted description: Interface has been deleted. color: '#FAD7A0' - name: outage label: Outage description: Interface is currently experiencing an outage. color: '#F4CCCC' optional: false description: The status of the interface order_weight: 1200 - name: role kind: Dropdown choices: - name: server label: Server Interface description: Interface connecting servers color: '#A9DFBF' - name: router label: Router Interface description: Interface connecting routers color: '#B2D4E6' optional: true description: The role of the interface in the network order_weight: 1300 relationships: - name: mlag_domain peer: GenericMlagDomain label: MLAG Domain kind: Parent cardinality: one optional: false description: MLAG Domain this MLAG Interface belongs to order_weight: 900 - name: Domain namespace: Mlag description: MLAG (Multi-Chassis Link Aggregation Group) Domain label: MLAG Domain icon: mingcute:cloud-line include_in_menu: true inherit_from: - GenericMlagDomain relationships: - name: peers peer: DcimDevice kind: Generic cardinality: many optional: false identifier: device__mlag_domain min_count: 2 max_count: 2 order_weight: 950 extensions: nodes: - kind: InterfaceLag relationships: - name: mlag_peer_link peer: MlagDomain label: MLAG Peer Link kind: Attribute cardinality: one optional: true description: MLAG Domain this LAG is used as peer link order_weight: 1950 - kind: DcimDevice relationships: - name: mlag_domain peer: MlagDomain label: MLAG Domain kind: Attribute cardinality: one optional: true identifier: device__mlag_domain description: MLAG Domain this device belongs to order_weight: 1900 ``` --- # Module Port This schema extension adds module ports: the ports a module provides, as declared by its module type - what NetBox module-type definitions list under `interfaces`, `console-ports` and `power-ports`. These are deliberately not DcimInterface objects. DcimInterface.device is a mandatory Parent, and Infrahub requires the relationships used in a uniqueness constraint to be mandatory, so an interface cannot hang off a module instead of a device. A DcimModulePort is a declaration parented by the module, carrying the port name, its category (interface, console, power, front, rear), the NetBox type slug, and power draw. NOTE: port names keep NetBox's `{module}` bay-position token verbatim, because a template is not bound to a bay. Substituting it and creating the real device interfaces is a generator step once the module is installed. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/device\_module](/schema-library/reference/device_module.md) ## Nodes[​](#nodes "Direct link to Nodes") ### ModulePort[​](#moduleport "Direct link to ModulePort") * **Label:** Module Port * **Description:** A port provided by a module, as declared by its module type. * **Namespace:** Dcim * **Icon:** mdi :ethernet * **Uniqueness Constraints:** * module, name\_\_value * **Human Friendly ID:** module\_\_computed\_name\_\_value, name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------- | -------------------------------------------------------------------- | -------- | -------- | -------------- | -------------------------------------- | | name | Port name as declared by the module type, e.g. `Ethernet{module}/1`. | Text | False | | | | category | Which NetBox component list this port came from. | Dropdown | | interface | interface, console, power, front, rear | | port\_type | NetBox type slug, e.g. '1000base-t', 'rj-45', 'iec-60320-c14'. | Text | True | | | | mgmt\_only | Whether the port is reserved for out-of-band management. | Boolean | | False | | | maximum\_draw | Maximum power draw, for power ports. | Number | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | ----------------- | -------- | ----------- | ------ | | module | DcimGenericModule | False | one | Parent | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### DcimGenericModule[​](#dcimgenericmodule "Direct link to DcimGenericModule") #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | -------------- | -------- | ----------- | --------- | | ports | DcimModulePort | True | many | Component | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: ModulePort namespace: Dcim label: Module Port description: A port provided by a module, as declared by its module type. icon: mdi:ethernet include_in_menu: false menu_placement: DcimGenericModule display_label: name__value order_by: - module__computed_name__value - name__value human_friendly_id: - module__computed_name__value - name__value uniqueness_constraints: - - module - name__value attributes: - name: name kind: Text optional: false description: Port name as declared by the module type, e.g. `Ethernet{module}/1`. order_weight: 1000 - name: category kind: Dropdown description: Which NetBox component list this port came from. default_value: interface order_weight: 1100 choices: - name: interface label: Interface description: A network interface. color: '#A9CCE3' - name: console label: Console Port description: A console port. color: '#E2D4C6' - name: power label: Power Port description: A power inlet. color: '#F4CCCC' - name: front label: Front Port description: A front-facing pass-through port. color: '#D2B4DE' - name: rear label: Rear Port description: A rear-facing pass-through port. color: '#B4E0DC' - name: port_type label: Port Type kind: Text optional: true description: NetBox type slug, e.g. '1000base-t', 'rj-45', 'iec-60320-c14'. order_weight: 1200 - name: mgmt_only label: Management Only kind: Boolean default_value: false description: Whether the port is reserved for out-of-band management. order_weight: 1300 - name: maximum_draw label: Maximum Draw (W) kind: Number optional: true description: Maximum power draw, for power ports. order_weight: 1400 relationships: - name: module peer: DcimGenericModule identifier: module__port optional: false cardinality: one kind: Parent order_weight: 1050 extensions: nodes: - kind: DcimGenericModule relationships: - name: ports peer: DcimModulePort identifier: module__port optional: true cardinality: many kind: Component order_weight: 1600 ``` --- # Modules Linecards This schema extension allows you to capture Linecard related information like the version. You can insert the Linecard into a Dcim Physical Device and leverage the Linecard type model. The Linecard can accept PIC to help configure PORT information like breakout-capabilities and configurations. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/device\_module](/schema-library/reference/device_module.md) ## Nodes[​](#nodes "Direct link to Nodes") ### LinecardType[​](#linecardtype "Direct link to LinecardType") * **Label:** Linecard Type * **Description:** Linecard Type information, detailing specifications such as part number and manufacturer. * **Namespace:** Device * **Icon:** mdi :poll * **Inherit From:** DcimGenericModuleType #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | -------------- | -------- | ----------- | ------- | | linecards | DeviceLinecard | | many | Generic | ### Linecard[​](#linecard "Direct link to Linecard") * **Label:** Linecard * **Description:** A Linecard installed in a device, specifying slot, power status, and functionalities. * **Namespace:** Device * **Icon:** bi :pci-card * **Inherit From:** DcimGenericModule #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------ | ----------------------------------------------------------------- | ------- | -------- | -------------- | ------- | | slot | The slot number where the Linecard is installed within the device | Number | True | | | | bng\_enabled | BNG activated or deactivated on the Linecard | Boolean | True | False | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------- | ------------------ | -------- | ----------- | --------- | | linecard\_type | DeviceLinecardType | False | one | Attribute | | pics | DevicePic | True | many | Attribute | ### Pic[​](#pic "Direct link to Pic") * **Label:** PIC * **Description:** Physical Interface Card (PIC) installed in the Linecard, containing multiple ports. * **Namespace:** Device * **Icon:** mdi :memory * **Uniqueness Constraints:** * linecard, slot\_\_value * **Human Friendly ID:** serial\_number\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ------------------------------------------ | ------ | -------- | -------------- | ------- | | serial\_number | Unique serial number of the PIC. | Text | | | | | slot | Slot number of the PIC within the Linecard | Number | | | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | -------------- | -------- | ----------- | --------- | | linecard | DeviceLinecard | False | one | Parent | | ports | DevicePort | True | many | Component | ### Port[​](#port "Direct link to Port") * **Label:** Port * **Description:** A network port on a PIC, specifying speed and port number. * **Namespace:** Device * **Icon:** mdi :ethernet * **Uniqueness Constraints:** * pic, port\_number\_\_value * **Human Friendly ID:** pic\_\_serial\_number\_\_value, port\_number\_\_value #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------ | ---------------------- | -------- | -------- | -------------- | --------- | | port\_number | Port number on the PIC | Number | | | | | speed | Speed of the port | Dropdown | | | 10g, 100g | #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | --------- | -------- | ----------- | ------ | | pic | DevicePic | False | one | Parent | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: LinecardType namespace: Device menu_placement: DeviceLinecard description: Linecard Type information, detailing specifications such as part number and manufacturer. icon: mdi:poll label: Linecard Type inherit_from: - DcimGenericModuleType relationships: - name: linecards peer: DeviceLinecard cardinality: many kind: Generic description: Linecards of this type. - name: Linecard namespace: Device description: A Linecard installed in a device, specifying slot, power status, and functionalities. label: Linecard icon: bi:pci-card menu_placement: DcimGenericModule generate_template: true inherit_from: - DcimGenericModule attributes: - name: slot kind: Number optional: true description: The slot number where the Linecard is installed within the device order_weight: 1050 - name: bng_enabled label: BNG Enabled description: BNG activated or deactivated on the Linecard kind: Boolean optional: true default_value: false order_weight: 1400 relationships: - name: linecard_type label: Linecard Type peer: DeviceLinecardType optional: false cardinality: one kind: Attribute order_weight: 1150 - name: pics label: PICs peer: DevicePic optional: true cardinality: many kind: Attribute order_weight: 1500 - name: Pic namespace: Device description: Physical Interface Card (PIC) installed in the Linecard, containing multiple ports. label: PIC menu_placement: DeviceLinecard icon: mdi:memory uniqueness_constraints: - - linecard - slot__value human_friendly_id: - serial_number__value display_label: serial_number__value order_by: - linecard__serial_number__value attributes: - name: serial_number kind: Text unique: true description: Unique serial number of the PIC. order_weight: 1000 - name: slot kind: Number description: Slot number of the PIC within the Linecard order_weight: 1200 relationships: - name: linecard label: Linecard peer: DeviceLinecard identifier: linecard__pics optional: false cardinality: one kind: Parent order_weight: 1000 - name: ports label: Ports peer: DevicePort optional: true cardinality: many kind: Component order_weight: 1100 - name: Port namespace: Device description: A network port on a PIC, specifying speed and port number. label: Port menu_placement: DeviceLinecard icon: mdi:ethernet uniqueness_constraints: - - pic - port_number__value human_friendly_id: - pic__serial_number__value - port_number__value display_label: port_number__value order_by: - pic__serial_number__value attributes: - name: port_number kind: Number description: Port number on the PIC order_weight: 1100 - name: speed kind: Dropdown description: Speed of the port choices: - name: 10g label: 10Gbps description: 10 Gigabit per second color: '#A9CCE3' - name: 100g label: 100Gbps description: 100 Gigabit per second color: '#9fbdf2' order_weight: 1200 relationships: - name: pic label: PIC peer: DevicePic optional: false cardinality: one kind: Parent order_weight: 1000 ``` --- # Modules Routing Engine This schema extension allows you to capture Routing Engine related information like the version. You can insert the Routing Engine into a Dcim Physical Device and leverage the Routing Engine type model. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/device\_module](/schema-library/reference/device_module.md) ## Nodes[​](#nodes "Direct link to Nodes") ### RoutingEngineType[​](#routingenginetype "Direct link to RoutingEngineType") * **Label:** Routing Engine Type * **Description:** Routing Engine Type information, detailing specifications such as part number and manufacturer. * **Namespace:** Device * **Inherit From:** DcimGenericModuleType #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | ------------------- | -------- | ----------- | ------- | | routing\_engines | DeviceRoutingEngine | | many | Generic | ### RoutingEngine[​](#routingengine "Direct link to RoutingEngine") * **Label:** Routing Engine * **Description:** A Routing Engine (RE) installed in a device, responsible for routing functionalities. * **Namespace:** Device * **Icon:** mdi :cpu-64-bit * **Inherit From:** DcimGenericModule #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------- | ----------------------------------------------------------------------- | ------ | -------- | -------------- | ------- | | slot | The slot number where the Routing Engine is installed within the device | Number | | | | | version | Firmware version of the Routing Engine. | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------------- | ----------------------- | -------- | ----------- | --------- | | routing\_engine\_type | DeviceRoutingEngineType | False | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: RoutingEngineType namespace: Device description: Routing Engine Type information, detailing specifications such as part number and manufacturer. label: Routing Engine Type inherit_from: - DcimGenericModuleType relationships: - name: routing_engines peer: DeviceRoutingEngine cardinality: many kind: Generic description: Routing engines of this type. - name: RoutingEngine namespace: Device description: A Routing Engine (RE) installed in a device, responsible for routing functionalities. label: Routing Engine icon: mdi:cpu-64-bit inherit_from: - DcimGenericModule attributes: - name: slot kind: Number description: The slot number where the Routing Engine is installed within the device order_weight: 1100 - name: version kind: Text label: Version description: Firmware version of the Routing Engine. optional: true order_weight: 1200 relationships: - name: routing_engine_type label: RE Type peer: DeviceRoutingEngineType optional: false cardinality: one kind: Attribute order_weight: 1150 ``` --- # Optical Multiplexer This schema extension models optical add-drop multiplexers (OADM) and the wavelength division multiplexing (WDM) channels they carry, for both CWDM and DWDM. It adds an Optical Multiplexer device with front and rear interfaces, a WDM Channel node holding channel number, wavelength and frequency, and a WDM Transceiver flavour of the transceiver model tuned to a channel. Some vendors configure tunable optics by wavelength or frequency rather than channel number; the WDM Channel node gives you a single entry in Infrahub for all three. Not designed to be loaded together with experimental/optical\_transport, which covers the same domain in more depth. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/transceiver](/schema-library/reference/transceiver.md) ## Nodes[​](#nodes "Direct link to Nodes") ### OpticalMultiplexer[​](#opticalmultiplexer "Direct link to OpticalMultiplexer") * **Label:** Optical Multiplexer * **Description:** An OADM (Optical Add Drop Multiplexer) supporting various WDM (Wavelength Division Multiplexing) technologies. * **Namespace:** Dcim * **Icon:** mdi :transit-connection-variant * **Human Friendly ID:** name\_\_value * **Inherit From:** DcimPhysicalDevice #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ---------------------------------------- | -------- | -------- | -------------- | ---------- | | name | | Text | False | | | | wdm\_type | Type of WDM technology (e.g. CWDM, DWDM) | Dropdown | False | dwdm | cwdm, dwdm | | description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------------- | ---------------------- | -------- | ----------- | --------- | | front\_interfaces | DcimOadmFrontInterface | True | many | Component | | rear\_interface | DcimOadmRearInterface | True | one | Component | ### OadmFrontInterface[​](#oadmfrontinterface "Direct link to OadmFrontInterface") * **Label:** Optical Multiplexer Front Interfaces * **Description:** Client-side interface of an optical add-drop multiplexer, carrying a single channel. * **Namespace:** Dcim * **Uniqueness Constraints:** * optical\_multiplexer, name\_\_value * **Human Friendly ID:** optical\_multiplexer\_\_name\_\_value, name\_\_value * **Inherit From:** DcimEndpoint, DcimGenericOadmInterface #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------------- | ---------------------- | -------- | ----------- | --------- | | optical\_multiplexer | DcimOpticalMultiplexer | False | one | Parent | | channels | DcimWdmChannel | True | many | Attribute | ### OadmRearInterface[​](#oadmrearinterface "Direct link to OadmRearInterface") * **Label:** Optical Multiplexer Rear Interfaces * **Description:** Line-side interface of an optical add-drop multiplexer, carrying the multiplexed signal. * **Namespace:** Dcim * **Uniqueness Constraints:** * optical\_multiplexer, name\_\_value * **Human Friendly ID:** optical\_multiplexer\_\_name\_\_value, name\_\_value * **Inherit From:** DcimEndpoint, DcimGenericOadmInterface #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------------- | ---------------------- | -------- | ----------- | ------ | | optical\_multiplexer | DcimOpticalMultiplexer | False | one | Parent | ### WdmChannel[​](#wdmchannel "Direct link to WdmChannel") * **Label:** WDM Channel * **Description:** A WDM channel with its wavelength and frequency. * **Namespace:** Dcim * **Icon:** game-icons :laser-warning * **Uniqueness Constraints:** * frequency\_\_value, wavelength\_\_value, channel\_\_value, wdm\_type\_\_value * channel\_\_value, wdm\_type\_\_value * **Human Friendly ID:** wdm\_type\_\_value, channel\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------- | ---------------------------------------- | -------- | -------- | -------------- | ---------- | | channel | WDM channel number. | Number | False | | | | wdm\_type | Type of WDM technology (e.g. CWDM, DWDM) | Dropdown | False | dwdm | cwdm, dwdm | | wavelength | Wavelength of the channel in nm. | Text | False | | | | frequency | Frequency of the channel in GHz. | Text | False | | | ### WdmTransceiver[​](#wdmtransceiver "Direct link to WdmTransceiver") * **Label:** WDM Transceiver * **Description:** Transceiver tuned to a Wavelength Division Multiplexing channel. * **Namespace:** Dcim * **Icon:** mdi :laser-pointer * **Inherit From:** DcimGenericTransceiver #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------- | ---------------------------------------- | -------- | -------- | -------------- | ---------- | | wdm\_type | Type of WDM technology (e.g. CWDM, DWDM) | Dropdown | False | dwdm | cwdm, dwdm | #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | -------------- | -------- | ----------- | --------- | | wdm\_channel | DcimWdmChannel | False | one | Attribute | ## Generics[​](#generics "Direct link to Generics") ### GenericOadmInterface[​](#genericoadminterface "Direct link to GenericOadmInterface") * **Label:** Optical Multiplexer Interfaces * **Description:** Generic interface of an optical add-drop multiplexer, front or rear. * **Namespace:** Dcim * **Icon:** mdi :ethernet #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------------- | ----------- | -------- | -------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | name | | Text | False | | | | description | | Text | True | | | | connector\_type | | Dropdown | False | | fc, lc, lc\_pc, lc\_upc, lc\_apc, lsh, lsh\_pc, lsh\_upc, lsh\_apc, lx\_5, lx\_5\_pc, lx\_5\_upc, lx\_5\_apc, mpo, mtrj, sc, sc\_pc, sc\_upc, sc\_apc, st, cs, sn, sma\_905, sma\_906, urm\_p2, urm\_p4, urm\_p8, splice | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: GenericOadmInterface namespace: Dcim description: Generic interface of an optical add-drop multiplexer, front or rear. label: Optical Multiplexer Interfaces icon: mdi:ethernet include_in_menu: true menu_placement: DcimOpticalMultiplexer attributes: - name: name kind: Text optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 - name: connector_type kind: Dropdown choices: - name: fc label: FC description: Standardized fiber optic connector used primarily in datacom and telecom applications. - name: lc label: LC description: Compact fiber optic connector with a push-pull mechanism. - name: lc_pc label: LC/PC description: Polished LC connector providing physical contact (PC). - name: lc_upc label: LC/UPC description: Ultra-Physical Contact (UPC) variant of the LC connector with enhanced polish. - name: lc_apc label: LC/APC description: Angled Physical Contact (APC) version of the LC connector with a slanted fiber end-face. - name: lsh label: LSH description: European fiber optic connector offering high durability. - name: lsh_pc label: LSH/PC description: Physical Contact version of LSH with standard polish. - name: lsh_upc label: LSH/UPC description: Ultra-Physical Contact variant of LSH, minimizing return loss with a superior polish. - name: lsh_apc label: LSH/APC description: Angled Physical Contact version of LSH, designed to reduce back reflections. - name: lx_5 label: LX.5 description: Miniaturized fiber optic connector similar to LC but with an additional shutter mechanism. - name: lx_5_pc label: LX.5/PC description: Physical Contact version of LX.5. - name: lx_5_upc label: LX.5/UPC description: Ultra-Physical Contact variant of LX.5. - name: lx_5_apc label: LX.5/APC description: Angled Physical Contact version of LX.5. - name: mpo label: MPO description: Multi-fiber Push-On connector typically used in data centers for high-speed applications. - name: mtrj label: MTRJ description: Male-to-female fiber optic connector with two fibers. - name: sc label: SC description: Square fiber optic connector with push-pull lock. - name: sc_pc label: SC/PC description: Physical Contact SC connector with a polished end-face. - name: sc_upc label: SC/UPC description: Ultra-Physical Contact variant of SC. - name: sc_apc label: SC/APC description: Angled Physical Contact version of SC. - name: st label: ST description: Bayonet-style fiber optic connector primarily used in industrial and military applications. - name: cs label: CS description: Compact connector with a high-density duplex configuration. - name: sn label: SN description: Small-form connector with dual fibers. - name: sma_905 label: SMA 905 description: Stainless steel fiber optic connector. - name: sma_906 label: SMA 906 description: Variant of SMA 905 with similar durability, frequently used in high-vibration settings. - name: urm_p2 label: URM-P2 description: Specialized fiber optic connector for industrial and harsh environments. - name: urm_p4 label: URM-P4 description: Similar to URM-P2 but designed for higher performance. - name: urm_p8 label: URM-P8 description: Enhanced version of URM connectors with higher protection. - name: splice label: Splice description: Permanent fiber connection method where two fiber ends are fused. optional: false order_weight: 1200 nodes: - name: OpticalMultiplexer namespace: Dcim description: An OADM (Optical Add Drop Multiplexer) supporting various WDM (Wavelength Division Multiplexing) technologies. label: Optical Multiplexer icon: mdi:transit-connection-variant include_in_menu: true inherit_from: - DcimPhysicalDevice human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: wdm_type kind: Dropdown default_value: dwdm choices: - name: cwdm label: CWDM (Coarse Wavelength Division Multiplexing) description: Supports multiple wavelengths for communication up to 70km. color: '#0099cc' - name: dwdm label: DWDM (Dense Wavelength Division Multiplexing) description: Supports dense wavelengths and amplification for long-distance communication. color: '#9933cc' optional: false description: Type of WDM technology (e.g. CWDM, DWDM) order_weight: 1300 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: front_interfaces peer: DcimOadmFrontInterface kind: Component cardinality: many optional: true identifier: optical_multiplexer__front_interfaces order_weight: 1350 - name: rear_interface peer: DcimOadmRearInterface kind: Component cardinality: one optional: true identifier: optical_multiplexer__rear_interface order_weight: 1450 - name: OadmFrontInterface namespace: Dcim description: Client-side interface of an optical add-drop multiplexer, carrying a single channel. label: Optical Multiplexer Front Interfaces include_in_menu: true menu_placement: DcimGenericOadmInterface inherit_from: - DcimEndpoint - DcimGenericOadmInterface human_friendly_id: - optical_multiplexer__name__value - name__value order_by: - optical_multiplexer__name__value - name__value display_label: '{{ optical_multiplexer__name__value }} > {{ name__value }}' uniqueness_constraints: - - optical_multiplexer - name__value relationships: - name: optical_multiplexer peer: DcimOpticalMultiplexer kind: Parent cardinality: one optional: false identifier: optical_multiplexer__front_interfaces order_weight: 900 - name: channels peer: DcimWdmChannel kind: Attribute cardinality: many optional: true identifier: oadm_interface__channels order_weight: 1300 - name: OadmRearInterface namespace: Dcim description: Line-side interface of an optical add-drop multiplexer, carrying the multiplexed signal. label: Optical Multiplexer Rear Interfaces include_in_menu: true menu_placement: DcimGenericOadmInterface inherit_from: - DcimEndpoint - DcimGenericOadmInterface human_friendly_id: - optical_multiplexer__name__value - name__value order_by: - optical_multiplexer__name__value - name__value display_label: '{{ optical_multiplexer__name__value }} > {{ name__value }}' uniqueness_constraints: - - optical_multiplexer - name__value relationships: - name: optical_multiplexer peer: DcimOpticalMultiplexer kind: Parent cardinality: one optional: false identifier: optical_multiplexer__rear_interface order_weight: 900 - name: WdmChannel namespace: Dcim description: A WDM channel with its wavelength and frequency. label: WDM Channel icon: game-icons:laser-warning include_in_menu: true human_friendly_id: - wdm_type__value - channel__value order_by: - wdm_type__value - channel__value display_label: '{{ wdm_type__value }} Channel {{ channel__value }}' uniqueness_constraints: - - frequency__value - wavelength__value - channel__value - wdm_type__value - - channel__value - wdm_type__value attributes: - name: channel kind: Number optional: false description: WDM channel number. order_weight: 1000 - name: wdm_type kind: Dropdown default_value: dwdm choices: - name: cwdm label: CWDM (Coarse Wavelength Division Multiplexing) description: Supports multiple wavelengths for communication up to 70km. color: '#0099cc' - name: dwdm label: DWDM (Dense Wavelength Division Multiplexing) description: Supports dense wavelengths and amplification for long-distance communication. color: '#9933cc' optional: false description: Type of WDM technology (e.g. CWDM, DWDM) order_weight: 1100 - name: wavelength kind: Text label: Wavelength (nm) optional: false description: Wavelength of the channel in nm. order_weight: 1200 - name: frequency kind: Text label: Frequency (GHz) optional: false description: Frequency of the channel in GHz. order_weight: 1300 - name: WdmTransceiver namespace: Dcim description: Transceiver tuned to a Wavelength Division Multiplexing channel. label: WDM Transceiver icon: mdi:laser-pointer include_in_menu: true menu_placement: DcimGenericTransceiver inherit_from: - DcimGenericTransceiver attributes: - name: wdm_type kind: Dropdown default_value: dwdm choices: - name: cwdm label: CWDM (Coarse Wavelength Division Multiplexing) description: Supports multiple wavelengths for communication up to 70km. color: '#0099cc' - name: dwdm label: DWDM (Dense Wavelength Division Multiplexing) description: Supports dense wavelengths and amplification for long-distance communication. color: '#9933cc' optional: false description: Type of WDM technology (e.g. CWDM, DWDM) order_weight: 1150 relationships: - name: wdm_channel peer: DcimWdmChannel label: WDM Channel kind: Attribute cardinality: one optional: false identifier: wdm_transceiver__channel order_weight: 1250 ``` --- # Optical Transport Comprehensive optical transport network schemas for DWDM/WDM systems (ADVA FSP 3000 and similar platforms). Covers four layers: wavelength (ITU-T G.694.1 grid, optical bands, DWDM channels), topology (logical optical nodes, passive multiplexers, fiber links), equipment (transponder/amplifier/ROADM modules, ROADM degrees, WSS cross-connects), and service (end-to-end optical services, optical paths, path segments). Not designed to be loaded together with extensions/optical\_multiplexer. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/device\_module](/schema-library/reference/device_module.md) * [extensions/cable](/schema-library/reference/cable.md) ## Nodes[​](#nodes "Direct link to Nodes") ### ITUGrid[​](#itugrid "Direct link to ITUGrid") * **Label:** ITU-T DWDM Grid * **Description:** ITU-T G.694.1 frequency grid standards for DWDM * **Namespace:** Dcim * **Icon:** mdi :grid * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------------- | ----------------------------------------------------- | ------ | -------- | -------------- | ------- | | name | Grid name (e.g., 50GHz, 100GHz) | Text | False | | | | channel\_spacing\_ghz | Channel spacing in GHz (50, 100, etc.) | Number | False | | | | reference\_frequency\_thz | Reference frequency in THz (e.g., '193.1' for 1550nm) | Text | False | | | | description | Grid description and use cases | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | --------------- | -------- | ----------- | --------- | | bands | DcimOpticalBand | True | many | Component | ### OpticalBand[​](#opticalband "Direct link to OpticalBand") * **Label:** Optical Band * **Description:** Optical frequency band (C-Band, L-Band, S-Band) * **Namespace:** Dcim * **Icon:** mdi :sine-wave * **Uniqueness Constraints:** * grid, band\_name\_\_value * **Human Friendly ID:** grid\_\_name\_\_value, band\_name\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------------------- | ---------------------------------------------------- | -------- | -------- | -------------- | ---------------------------------- | | band\_name | Optical band designation | Dropdown | False | | c\_band, l\_band, s\_band, o\_band | | start\_frequency\_thz | Band start frequency in THz (e.g., '191.0') | Text | False | | | | end\_frequency\_thz | Band end frequency in THz (e.g., '196.1') | Text | False | | | | start\_wavelength\_nm | Band start wavelength in nm (e.g., '1530.0') | Text | True | | | | end\_wavelength\_nm | Band end wavelength in nm (e.g., '1565.0') | Text | True | | | | computed\_name | Auto-generated band identifier (e.g., 50GHz-C\_BAND) | Text | False | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | --------------- | -------- | ----------- | --------- | | grid | DcimITUGrid | False | one | Parent | | channels | DcimDWDMChannel | True | many | Component | ### DWDMChannel[​](#dwdmchannel "Direct link to DWDMChannel") * **Label:** DWDM Channel * **Description:** Individual DWDM wavelength/channel (CH20, CH21, etc.) * **Namespace:** Dcim * **Icon:** mdi :wave * **Uniqueness Constraints:** * band, channel\_number\_\_value * channel\_name\_\_value * channel\_number\_\_value, frequency\_thz\_\_value, wavelength\_nm\_\_value * **Human Friendly ID:** channel\_name\_\_value #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------- | ------------------------------------------------------------------------------------- | ------ | -------- | -------------- | ------- | | channel\_name | ITU DWDM Channel identifier (computed as CH + channel number, e.g., CH20, CH21, CH40) | Text | False | | | | adva\_channel\_name | ADVA-specific channel name (e.g., D02, DC1, D32) | Text | True | | | | channel\_number | ITU channel number (e.g., 20, 21, 60) | Number | False | | | | frequency\_thz | Center frequency in THz (e.g., '192.1') | Text | False | | | | wavelength\_nm | Center wavelength in nm (e.g., '1560.61') | Text | False | | | | description | Channel notes and usage | Text | True | | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | ------------------------ | -------- | ----------- | --------- | | band | DcimOpticalBand | False | one | Parent | | mux\_assignments | DcimMuxChannelAssignment | True | many | Component | | mappings | DcimChannelMapping | True | many | Generic | ### MuxChannelAssignment[​](#muxchannelassignment "Direct link to MuxChannelAssignment") * **Label:** Mux Channel Assignment * **Description:** Maps DWDM channel to multiplexer port (equipment configuration) * **Namespace:** Dcim * **Icon:** mdi :cable-data * **Uniqueness Constraints:** * optical\_device, port\_number\_\_value * optical\_device, channel * **Human Friendly ID:** optical\_device\_\_computed\_name\_\_value, port\_number\_\_value #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ------------------------------------------------ | -------- | -------- | -------------- | ------------------------------------ | | port\_number | Physical port number on multiplexer (e.g., 1-96) | Number | False | | | | tx\_power\_dbm | Configured transmit power in dBm (e.g., '0.0') | Text | True | | | | status | Channel assignment status | Dropdown | False | configured | configured, active, disabled, failed | | computed\_name | Auto-generated assignment identifier | Text | False | | | #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ----------------- | -------- | ----------- | --------- | | channel | DcimDWDMChannel | False | one | Attribute | | optical\_device | DcimOpticalDevice | False | one | Parent | ### FiberMapping[​](#fibermapping "Direct link to FiberMapping") * **Label:** Fiber Mapping * **Description:** Long-haul fiber link between degrees on DIFFERENT ROADMs/sites (kilometers distance) * **Namespace:** Dcim * **Icon:** mdi :fiber-optic * **Uniqueness Constraints:** * fiber\_link, channel * **Human Friendly ID:** fiber\_link\_\_link\_id\_\_value, channel\_\_channel\_name\_\_value * **Inherit From:** DcimChannelMapping #### Attributes[​](#attributes-4 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------ | -------------------------------------------------- | -------- | -------- | -------------- | ------------------------------------------------- | | allocation\_status | Allocation status | Dropdown | False | reserved | reserved, allocated, in\_use, maintenance, failed | | tx\_power\_dbm | Measured transmit power in dBm (e.g., '0.5') | Text | True | | | | rx\_power\_dbm | Measured receive power in dBm (e.g., '-2.3') | Text | True | | | | osnr\_db | Optical Signal-to-Noise Ratio in dB (e.g., '25.5') | Text | True | | | | allocated\_date | Date channel was allocated | DateTime | True | | | | computed\_name | Auto-generated mapping identifier | Text | False | | | | description | Additional fiber mapping details | Text | True | | | #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------- | ------------- | -------- | ----------- | --------- | | fiber\_link | DcimFiberLink | False | one | Attribute | ### OpticalNode[​](#opticalnode "Direct link to OpticalNode") * **Label:** Optical Node * **Description:** Logical network endpoint in optical topology (graph node, not physical device) * **Namespace:** Dcim * **Icon:** mdi :lan * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value * **Inherit From:** DcimOpticalDevice #### Attributes[​](#attributes-5 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ----------------------------------------------------- | -------- | -------- | -------------- | --------------------------------- | | computed\_name | Computed name for display | Text | False | | | | name | Logical node name (e.g., GENESIS-OL-10, SKYLINE-OL-1) | Text | False | | | | node\_type | Topology role of this node | Dropdown | False | | endpoint, intermediate, branching | | description | Node notes and function | Text | True | | | #### Relationships[​](#relationships-5 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------------- | --------------------- | -------- | ----------- | --------- | | device | DcimPhysicalDevice | True | one | Attribute | | transponder\_modules | DcimTransponderModule | True | many | Attribute | | links | DcimFiberLink | True | many | Component | ### PassiveMultiplexer[​](#passivemultiplexer "Direct link to PassiveMultiplexer") * **Label:** Passive Multiplexer * **Description:** Passive CWDM/DWDM multiplexer (fixed port count, no modules) * **Namespace:** Dcim * **Icon:** mdi :resistor-nodes * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value * **Inherit From:** DcimPhysicalDevice #### Attributes[​](#attributes-6 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ------------------------------------------- | -------- | -------- | -------------- | ---------- | | computed\_name | Computed name for display | Text | False | | | | name | Passive mux name (e.g., CHI-PMUX-01) | Text | False | | | | port\_count | Number of fixed ports (e.g., 8, 16, 32, 40) | Number | False | | | | mux\_type | Type of passive multiplexer | Dropdown | False | | cwdm, dwdm | | description | Device notes | Text | True | | | ### FiberLink[​](#fiberlink "Direct link to FiberLink") * **Label:** Fiber Link * **Description:** Fiber connecting two optical nodes (graph edge) * **Namespace:** Dcim * **Icon:** mdi :cable-data * **Uniqueness Constraints:** * link\_id\_\_value * **Human Friendly ID:** link\_id\_\_value #### Attributes[​](#attributes-7 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------ | ----------------------------------------------- | -------- | -------- | -------------- | --------------------------------------------- | | link\_id | Unique link identifier (e.g., LINK-001) | Text | False | | | | distance\_km | Physical distance in kilometers (e.g., '920.5') | Text | True | | | | attenuation\_db | Total fiber attenuation in dB (e.g., '18.4') | Text | True | | | | status | Operational status of fiber link | Dropdown | False | planned | planned, active, standby, maintenance, failed | | commissioned\_date | Date link was commissioned | DateTime | True | | | | description | Additional link details | Text | True | | | #### Relationships[​](#relationships-6 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | ---------------- | -------- | ----------- | --------- | | endpoints | DcimOpticalNode | False | many | Attribute | | mappings | DcimFiberMapping | True | many | Component | ### OpticalModuleType[​](#opticalmoduletype "Direct link to OpticalModuleType") * **Label:** Optical Module Type * **Description:** Categorization of optical modules by function * **Namespace:** Dcim * **Icon:** mdi :expansion-card * **Inherit From:** DcimGenericModuleType #### Attributes[​](#attributes-8 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | --------------------------------- | -------- | -------- | -------------- | ------------------------------------------------------------------------ | | name | Module type name | Text | False | | | | description | Module type description | Text | True | | | | module\_category | Functional category of the module | Dropdown | False | | transponder, multiplexer, amplifier, monitoring, control, power, cooling | ### TransponderModule[​](#transpondermodule "Direct link to TransponderModule") * **Label:** Transponder Module * **Description:** Optical-electrical-optical conversion module (100G, 400G coherent) installed in chassis * **Namespace:** Dcim * **Icon:** mdi :chip * **Uniqueness Constraints:** * computed\_name\_\_value * **Human Friendly ID:** computed\_name\_\_value * **Inherit From:** DcimGenericModule #### Attributes[​](#attributes-9 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------ | ------------------------------------------------------- | -------- | -------- | -------------- | ---------------------------------------- | | computed\_name | Computed module name (e.g., dc1-leaf1 > slot 1-TPD-1/1) | Text | False | | | | slot\_number | Slot number in chassis (e.g., '1/1', '1/2', '2/1') | Text | False | | | | capacity\_gbps | Line rate capacity in Gbps (e.g., 100, 200, 400) | Number | False | | | | modulation\_format | Modulation format for optical transmission | Dropdown | False | | dp\_qpsk, dp\_16qam, dp\_8qam, dp\_64qam | | tunable\_range | Wavelength tuning capability | Dropdown | False | | c\_band, c\_l\_band, fixed | | fec\_type | Forward error correction type | Dropdown | True | | sd\_fec, hd\_fec, c\_fec, o\_fec | | client\_interface | Client-side interface (e.g., 100GE, 400GE, OTU4) | Text | True | | | | line\_interface | Line-side interface (e.g., CFP2-DCO, QSFP28, QSFP-DD) | Text | True | | | #### Relationships[​](#relationships-7 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------------------- | --------------- | -------- | ----------- | --------- | | tuned\_channel | DcimDWDMChannel | True | one | Attribute | | connected\_to\_optical\_node | DcimOpticalNode | True | one | Attribute | ### OpticalMultiplexerModule[​](#opticalmultiplexermodule "Direct link to OpticalMultiplexerModule") * **Label:** Optical Multiplexer Module * **Description:** Multiplexer/Demultiplexer/ROADM module for wavelength management installed in chassis * **Namespace:** Dcim * **Icon:** mdi :router-network * **Human Friendly ID:** computed\_name\_\_value * **Inherit From:** DcimGenericModule #### Attributes[​](#attributes-10 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------------- | ------------------------------------------------------------------------ | -------- | -------- | -------------- | ---------------------------------------------- | | computed\_name | Computed module name (e.g., dc1-leaf1 > slot 1-MUX-1/3) | Text | False | | | | slot\_number | Slot number in chassis (e.g., '1/3', '1/4', '2/3') | Text | False | | | | mux\_type | Type of multiplexer technology | Dropdown | False | | passive\_mux, passive\_demux, oadm, roadm, wss | | channel\_capacity | Number of supported DWDM channels (e.g., 40, 80, 96) | Number | False | | | | technology | Underlying multiplexer technology | Dropdown | True | | thin\_film\_filter, awg, wss\_lcos, wss\_mems | | degree\_count | Number of ROADM degrees (e.g., 2, 4, 8, 16) | Number | True | | | | colorless | Colorless add/drop capability (any channel on any port) | Boolean | True | False | | | directionless | Directionless add/drop capability (any port to any degree) | Boolean | True | False | | | contentionless | Contentionless add/drop (CDC - Colorless, Directionless, Contentionless) | Boolean | True | False | | #### Relationships[​](#relationships-8 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | ------------------ | -------- | ----------- | --------- | | degrees | DcimRoadmDegree | True | many | Component | | mappings | DcimChannelMapping | True | many | Attribute | ### RoadmDegree[​](#roadmdegree "Direct link to RoadmDegree") * **Label:** ROADM Degree * **Description:** ROADM degree with line port (OL-1, OL-2, etc.) * **Namespace:** Dcim * **Icon:** mdi :lan-connect * **Uniqueness Constraints:** * roadm, degree\_number\_\_value * roadm, line\_port\_\_value * **Human Friendly ID:** computed\_name\_\_value #### Attributes[​](#attributes-11 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------------- | ---------------------------------------------- | -------- | -------- | -------------- | ---------------------------------------- | | computed\_name | Computed name (e.g., ROADM-SITE-A-OL-1) | Text | False | | | | degree\_number | Degree number (1-based, e.g., 1, 2, 3, 4) | Number | False | | | | line\_port | Line port designation (e.g., OL-1, OL-2, OL-7) | Text | False | | | | direction | Directional designation of degree | Dropdown | False | | north, south, east, west, express, local | | degree\_type | Type of ROADM degree | Dropdown | False | | line, express, add\_drop | | wavelength\_capacity | Number of wavelengths supported on this degree | Number | True | | | | description | Additional degree details | Text | True | | | #### Relationships[​](#relationships-9 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------------- | ---------------------------- | -------- | ----------- | --------- | | roadm | DcimOpticalMultiplexerModule | False | one | Parent | | connected\_fiber | DcimFiberLink | True | one | Attribute | | channel\_mappings | DcimChannelMapping | True | many | Attribute | ### WSSConnect[​](#wssconnect "Direct link to WSSConnect") * **Label:** WSS Connect * **Description:** Internal WSS cross-connect between degrees on SAME ROADM device (0m distance) * **Namespace:** Dcim * **Icon:** mdi :swap-horizontal * **Uniqueness Constraints:** * roadm, channel * **Human Friendly ID:** computed\_name\_\_value * **Inherit From:** DcimChannelMapping #### Attributes[​](#attributes-12 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | -------------------------------- | ---- | -------- | -------------- | ------- | | computed\_name | Computed connection ID | Text | False | | | | description | Additional cross-connect details | Text | True | | | #### Relationships[​](#relationships-10 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | ---------------------------- | -------- | ----------- | --------- | | roadm | DcimOpticalMultiplexerModule | False | one | Attribute | ### CableMapping[​](#cablemapping "Direct link to CableMapping") * **Label:** Cable Mapping * **Description:** Local patch cable between degrees on DIFFERENT ROADMs (meters distance, same site) * **Namespace:** Dcim * **Icon:** mdi :cable-data * **Uniqueness Constraints:** * cable, channel * **Human Friendly ID:** computed\_name\_\_value * **Inherit From:** DcimChannelMapping #### Attributes[​](#attributes-13 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | -------------------------------------------- | ---- | -------- | -------------- | ------- | | computed\_name | Computed mapping name (e.g., CABLE-123-CH58) | Text | False | | | | description | Additional cable mapping details | Text | True | | | #### Relationships[​](#relationships-11 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | --------- | -------- | ----------- | --------- | | cable | DcimCable | False | one | Attribute | ### OpticalAmplifierModule[​](#opticalamplifiermodule "Direct link to OpticalAmplifierModule") * **Label:** Optical Amplifier Module * **Description:** Optical signal amplifier module (EDFA, Raman, VGC) installed in chassis * **Namespace:** Dcim * **Icon:** mdi :amplifier * **Human Friendly ID:** computed\_name\_\_value * **Inherit From:** DcimGenericModule #### Attributes[​](#attributes-14 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------------------- | ------------------------------------------------------- | -------- | -------- | -------------- | ----------------------------- | | computed\_name | Computed module name (e.g., dc1-leaf1 > slot 1-AMP-1/5) | Text | False | | | | slot\_number | Slot number in chassis (e.g., '1/5', '1/6', '2/5') | Text | False | | | | amplifier\_type | Type of optical amplifier | Dropdown | False | | edfa, raman, soa, vgc, hybrid | | stage | Amplifier stage position | Dropdown | False | | booster, inline, preamplifier | | gain\_db | Typical gain in dB (e.g., '20.0', '17.5') | Text | True | | | | max\_output\_power\_dbm | Maximum output power in dBm (e.g., '17.0', '23.0') | Text | True | | | | noise\_figure\_db | Noise figure in dB (e.g., '5.5', '4.8') | Text | True | | | | operating\_band | Operating wavelength band | Dropdown | False | | c\_band, l\_band, c\_l\_band | #### Relationships[​](#relationships-12 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ------------- | -------- | ----------- | --------- | | amplified\_link | DcimFiberLink | True | one | Attribute | ### OpticalService[​](#opticalservice "Direct link to OpticalService") * **Label:** Optical Service * **Description:** End-to-end optical transport service (customer circuit) spanning multiple sites * **Namespace:** Dcim * **Icon:** mdi :transit-connection-variant * **Uniqueness Constraints:** * service\_id\_\_value * **Human Friendly ID:** service\_id\_\_value #### Attributes[​](#attributes-15 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------------------- | ----------------------------------------------------------- | -------- | -------- | -------------- | ------------------------------------------------- | | service\_id | Unique service identifier | Text | False | | | | service\_name | Customer service name (e.g., 'ATC 10G - Albion to Arco #2') | Text | False | | | | service\_type | Type of optical service | Dropdown | False | | wavelength, transport, ip\_transit | | bandwidth | Service bandwidth (e.g., '10G', '100G', '400G') | Text | False | | | | status | Service operational status | Dropdown | False | planned | planned, provisioned, active, maintenance, failed | | customer\_circuit\_id | Customer's circuit ID (e.g., '99/OKFS/000029//SYG') | Text | True | | | | provision\_date | Date service was provisioned | DateTime | True | | | | description | Additional service details | Text | True | | | #### Relationships[​](#relationships-13 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | --------------------- | -------- | ----------- | --------- | | transponders | DcimTransponderModule | False | many | Attribute | | channel | DcimDWDMChannel | False | one | Attribute | | primary\_path | DcimOpticalPath | False | one | Component | | backup\_path | DcimOpticalPath | True | one | Component | ### OpticalPath[​](#opticalpath "Direct link to OpticalPath") * **Label:** Optical Path * **Description:** Ordered sequence of segments forming an optical path through the network * **Namespace:** Dcim * **Icon:** mdi :map-marker-path * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-16 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------- | --------------------------------------- | -------- | -------- | -------------- | ------------------------ | | name | Path name (e.g., MS-W17D5-primary) | Text | False | | | | path\_type | Type of path (primary, backup, express) | Dropdown | False | | primary, backup, express | | total\_distance\_km | Total path distance in kilometers | Text | True | | | | total\_loss\_db | Total optical loss in dB | Text | True | | | | hop\_count | Number of ROADM hops in path | Number | True | | | | is\_active | Whether this path is currently active | Boolean | False | True | | #### Relationships[​](#relationships-14 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------------ | ------------------ | -------- | ----------- | --------- | | segments | DcimPathSegment | True | many | Component | | is\_primary\_of\_service | DcimOpticalService | True | one | Attribute | | is\_backup\_of\_service | DcimOpticalService | True | one | Attribute | ### PathSegment[​](#pathsegment "Direct link to PathSegment") * **Label:** Path Segment * **Description:** One hop in optical path - references channel mapping (fiber, cable, or cross-connect) * **Namespace:** Dcim * **Icon:** mdi :ray-start-arrow * **Uniqueness Constraints:** * path, segment\_order\_\_value * **Human Friendly ID:** computed\_name\_\_value #### Attributes[​](#attributes-17 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | --------------------------------------------------- | -------- | -------- | -------------- | --------------------------------------- | | computed\_name | Computed segment name (e.g., MS-W17D5-primary-SEG1) | Text | False | | | | segment\_order | Order of this segment in the path (1, 2, 3...) | Number | False | | | | segment\_type | Type of segment | Dropdown | False | | fiber, cross\_connect, cable, add, drop | | loss\_db | Optical loss for this segment in dB | Text | True | | | #### Relationships[​](#relationships-15 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | ------------------ | -------- | ----------- | --------- | | path | DcimOpticalPath | False | one | Parent | | channel\_mapping | DcimChannelMapping | True | one | Attribute | ## Generics[​](#generics "Direct link to Generics") ### ChannelMapping[​](#channelmapping "Direct link to ChannelMapping") * **Label:** Channel Mapping * **Description:** Generic base for any channel allocation/routing (fiber, cable, or internal cross-connect) * **Namespace:** Dcim #### Attributes[​](#attributes-18 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ---------------------------------------- | -------- | -------- | -------------- | ------- | | is\_active | Whether this mapping is currently active | Boolean | False | True | | | configured\_date | Date this mapping was configured | DateTime | True | | | #### Relationships[​](#relationships-16 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------- | --------------- | -------- | ----------- | --------- | | segment | DcimPathSegment | False | one | Parent | | channel | DcimDWDMChannel | False | one | Attribute | | degrees | DcimRoadmDegree | False | many | Attribute | ### OpticalDevice[​](#opticaldevice "Direct link to OpticalDevice") * **Label:** Optical Multiplexer Device * **Description:** Generic interface for any device that can multiplex DWDM channels (active or passive) * **Namespace:** Dcim * **Uniqueness Constraints:** * computed\_name\_\_value #### Attributes[​](#attributes-19 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ----------- | ---- | -------- | -------------- | ------- | | computed\_name | | Text | False | | | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### DcimPhysicalDevice[​](#dcimphysicaldevice "Direct link to DcimPhysicalDevice") #### Relationships[​](#relationships-17 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | --------------- | -------- | ----------- | ------- | | optical\_node | DcimOpticalNode | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: ChannelMapping namespace: Dcim description: Generic base for any channel allocation/routing (fiber, cable, or internal cross-connect) label: Channel Mapping include_in_menu: false attributes: - name: is_active kind: Boolean optional: false default_value: true description: Whether this mapping is currently active - name: configured_date kind: DateTime optional: true description: Date this mapping was configured relationships: - name: segment peer: DcimPathSegment cardinality: one kind: Parent optional: false identifier: channel_mapping__path_segments description: Optical Path segment associated with this mapping - name: channel peer: DcimDWDMChannel cardinality: one kind: Attribute optional: false identifier: dwdm_channel__mappings description: DWDM channel being mapped/routed - name: degrees peer: DcimRoadmDegree cardinality: many max_count: 2 kind: Attribute optional: false identifier: roadm_degree__channel_mappings description: Two ROADM degrees involved in this mapping (bidirectional) - name: OpticalDevice namespace: Dcim description: Generic interface for any device that can multiplex DWDM channels (active or passive) label: Optical Multiplexer Device include_in_menu: false uniqueness_constraints: - - computed_name__value attributes: - name: computed_name kind: Text optional: false nodes: - name: ITUGrid namespace: Dcim include_in_menu: true label: ITU-T DWDM Grid description: ITU-T G.694.1 frequency grid standards for DWDM icon: mdi:grid display_label: name__value human_friendly_id: - name__value uniqueness_constraints: - - name__value attributes: - name: name kind: Text optional: false description: Grid name (e.g., 50GHz, 100GHz) - name: channel_spacing_ghz kind: Number optional: false description: Channel spacing in GHz (50, 100, etc.) - name: reference_frequency_thz kind: Text optional: false description: Reference frequency in THz (e.g., '193.1' for 1550nm) - name: description kind: Text optional: true description: Grid description and use cases relationships: - name: bands peer: DcimOpticalBand cardinality: many kind: Component optional: true description: Optical bands using this grid - name: OpticalBand namespace: Dcim include_in_menu: true label: Optical Band description: Optical frequency band (C-Band, L-Band, S-Band) icon: mdi:sine-wave display_label: computed_name__value human_friendly_id: - grid__name__value - band_name__value uniqueness_constraints: - - grid - band_name__value attributes: - name: band_name kind: Dropdown optional: false choices: - name: c_band label: C-Band description: Conventional Band (1530-1565 nm) color: '#ff6b6b' - name: l_band label: L-Band description: Long Band (1565-1625 nm) color: '#4ecdc4' - name: s_band label: S-Band description: Short Band (1460-1530 nm) color: '#95e1d3' - name: o_band label: O-Band description: Original Band (1260-1360 nm) color: '#f38181' description: Optical band designation - name: start_frequency_thz kind: Text optional: false description: Band start frequency in THz (e.g., '191.0') - name: end_frequency_thz kind: Text optional: false description: Band end frequency in THz (e.g., '196.1') - name: start_wavelength_nm kind: Text optional: true description: Band start wavelength in nm (e.g., '1530.0') - name: end_wavelength_nm kind: Text optional: true description: Band end wavelength in nm (e.g., '1565.0') - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ grid__name__value }}-{{ band_name__value|upper }}' description: Auto-generated band identifier (e.g., 50GHz-C_BAND) relationships: - name: grid peer: DcimITUGrid cardinality: one kind: Parent optional: false description: Parent ITU grid - name: channels peer: DcimDWDMChannel cardinality: many kind: Component optional: true description: DWDM channels in this band - name: DWDMChannel namespace: Dcim include_in_menu: true label: DWDM Channel description: Individual DWDM wavelength/channel (CH20, CH21, etc.) icon: mdi:wave display_label: channel_name__value human_friendly_id: - channel_name__value order_by: - band__computed_name__value - channel_number__value uniqueness_constraints: - - band - channel_number__value - - channel_name__value - - channel_number__value - frequency_thz__value - wavelength_nm__value attributes: - name: channel_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: CH{{ channel_number__value }} description: ITU DWDM Channel identifier (computed as CH + channel number, e.g., CH20, CH21, CH40) - name: adva_channel_name kind: Text optional: true description: ADVA-specific channel name (e.g., D02, DC1, D32) - name: channel_number kind: Number optional: false description: ITU channel number (e.g., 20, 21, 60) - name: frequency_thz kind: Text optional: false description: Center frequency in THz (e.g., '192.1') - name: wavelength_nm kind: Text optional: false description: Center wavelength in nm (e.g., '1560.61') - name: description kind: Text optional: true description: Channel notes and usage relationships: - name: band peer: DcimOpticalBand cardinality: one kind: Parent optional: false description: Parent optical band - name: mux_assignments peer: DcimMuxChannelAssignment cardinality: many kind: Component optional: true description: Mux port assignments for this channel - name: mappings peer: DcimChannelMapping cardinality: many kind: Generic optional: true description: Fiber allocations for this channel - name: MuxChannelAssignment namespace: Dcim include_in_menu: true label: Mux Channel Assignment description: Maps DWDM channel to multiplexer port (equipment configuration) icon: mdi:cable-data display_label: computed_name__value human_friendly_id: - optical_device__computed_name__value - port_number__value uniqueness_constraints: - - optical_device - port_number__value - - optical_device - channel attributes: - name: port_number kind: Number optional: false description: Physical port number on multiplexer (e.g., 1-96) - name: tx_power_dbm kind: Text optional: true description: Configured transmit power in dBm (e.g., '0.0') - name: status kind: Dropdown optional: false default_value: configured choices: - name: configured label: Configured description: Channel is configured on port color: '#f5a623' - name: active label: Active description: Channel is actively transmitting color: '#00cc00' - name: disabled label: Disabled description: Channel is disabled color: '#cccccc' - name: failed label: Failed description: Channel has failed color: '#cc0000' description: Channel assignment status - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ optical_device__computed_name__value }}-P{{ ''{:02d}''.format(port_number__value) }}-{{ channel__channel_name__value }}' description: Auto-generated assignment identifier relationships: - name: channel peer: DcimDWDMChannel cardinality: one kind: Attribute optional: false description: DWDM channel assigned to this port - name: optical_device peer: DcimOpticalDevice cardinality: one kind: Parent optional: false identifier: mux_channel_assignments description: Multiplexer device (OpticalNode or PassiveMultiplexer) - name: FiberMapping namespace: Dcim inherit_from: - DcimChannelMapping include_in_menu: true label: Fiber Mapping description: Long-haul fiber link between degrees on DIFFERENT ROADMs/sites (kilometers distance) icon: mdi:fiber-optic display_label: computed_name__value human_friendly_id: - fiber_link__link_id__value - channel__channel_name__value uniqueness_constraints: - - fiber_link - channel attributes: - name: allocation_status kind: Dropdown optional: false default_value: reserved choices: - name: reserved label: Reserved description: Channel reserved but not in use color: '#f5a623' - name: allocated label: Allocated description: Channel allocated to service color: '#0099cc' - name: in_use label: In Use description: Channel actively carrying traffic color: '#00cc00' - name: maintenance label: Maintenance description: Channel under maintenance color: '#ff9800' - name: failed label: Failed description: Channel has failed color: '#cc0000' description: Allocation status - name: tx_power_dbm kind: Text optional: true description: Measured transmit power in dBm (e.g., '0.5') - name: rx_power_dbm kind: Text optional: true description: Measured receive power in dBm (e.g., '-2.3') - name: osnr_db kind: Text optional: true description: Optical Signal-to-Noise Ratio in dB (e.g., '25.5') - name: allocated_date kind: DateTime optional: true description: Date channel was allocated - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ fiber_link__link_id__value }}-{{ channel__channel_name__value }}' description: Auto-generated mapping identifier - name: description kind: Text optional: true description: Additional fiber mapping details relationships: - name: fiber_link peer: DcimFiberLink cardinality: one kind: Attribute optional: false identifier: fiber_channel_mappings description: Long-haul fiber link connecting the degrees - name: OpticalNode namespace: Dcim include_in_menu: true label: Optical Node description: Logical network endpoint in optical topology (graph node, not physical device) icon: mdi:lan inherit_from: - DcimOpticalDevice display_label: computed_name__value human_friendly_id: - name__value uniqueness_constraints: - - name__value attributes: - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ name__value }}' description: Computed name for display - name: name kind: Text optional: false description: Logical node name (e.g., GENESIS-OL-10, SKYLINE-OL-1) - name: node_type kind: Dropdown optional: false choices: - name: endpoint label: Endpoint description: Edge node (traffic originates/terminates) color: '#2196f3' - name: intermediate label: Intermediate description: Pass-through node (ROADM only) color: '#9c27b0' - name: branching label: Branching description: Multiple fiber paths meet (hub) color: '#ff9800' description: Topology role of this node - name: description kind: Text optional: true description: Node notes and function relationships: - name: device peer: DcimPhysicalDevice cardinality: one kind: Attribute optional: true identifier: optical_nodes__physical_device description: Physical chassis that implements this logical node - name: transponder_modules peer: DcimTransponderModule cardinality: many kind: Attribute optional: true identifier: optical_node__transponder_modules description: Transponder modules connected to this optical node (for cross-connects) - name: links peer: DcimFiberLink cardinality: many kind: Component optional: true identifier: optical_node__links description: Fiber links connected to this node - name: PassiveMultiplexer namespace: Dcim include_in_menu: true label: Passive Multiplexer description: Passive CWDM/DWDM multiplexer (fixed port count, no modules) icon: mdi:resistor-nodes inherit_from: - DcimPhysicalDevice display_label: computed_name__value human_friendly_id: - name__value uniqueness_constraints: - - name__value attributes: - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ name__value }}' description: Computed name for display - name: name kind: Text optional: false description: Passive mux name (e.g., CHI-PMUX-01) - name: port_count kind: Number optional: false description: Number of fixed ports (e.g., 8, 16, 32, 40) - name: mux_type kind: Dropdown optional: false choices: - name: cwdm label: CWDM description: Coarse Wavelength Division Multiplexing color: '#4a90e2' - name: dwdm label: DWDM description: Dense Wavelength Division Multiplexing color: '#9933cc' description: Type of passive multiplexer - name: description kind: Text optional: true description: Device notes - name: FiberLink namespace: Dcim include_in_menu: true label: Fiber Link description: Fiber connecting two optical nodes (graph edge) icon: mdi:cable-data display_label: link_id__value human_friendly_id: - link_id__value uniqueness_constraints: - - link_id__value attributes: - name: link_id kind: Text optional: false description: Unique link identifier (e.g., LINK-001) - name: distance_km kind: Text optional: true description: Physical distance in kilometers (e.g., '920.5') - name: attenuation_db kind: Text optional: true description: Total fiber attenuation in dB (e.g., '18.4') - name: status kind: Dropdown optional: false default_value: planned choices: - name: planned label: Planned description: Link is planned but not yet installed color: '#95a5a6' - name: active label: Active description: Link is in service color: '#00cc00' - name: standby label: Standby description: Link is installed but not active color: '#f5a623' - name: maintenance label: Maintenance description: Link is under maintenance color: '#ff9800' - name: failed label: Failed description: Link has failed color: '#cc0000' description: Operational status of fiber link - name: commissioned_date kind: DateTime optional: true description: Date link was commissioned - name: description kind: Text optional: true description: Additional link details relationships: - name: endpoints peer: DcimOpticalNode identifier: optical_node__links cardinality: many kind: Attribute optional: false max_count: 2 description: Port-level endpoints participating in this link - name: mappings peer: DcimFiberMapping cardinality: many kind: Component identifier: fiber_channel_mappings optional: true description: DWDM channels allocated on this link - name: OpticalModuleType namespace: Dcim include_in_menu: true label: Optical Module Type description: Categorization of optical modules by function icon: mdi:expansion-card inherit_from: - DcimGenericModuleType attributes: - name: name kind: Text optional: false description: Module type name - name: description kind: Text optional: true description: Module type description - name: module_category kind: Dropdown optional: false description: Functional category of the module choices: - name: transponder label: Transponder description: Optical-electrical-optical conversion color: '#2196f3' - name: multiplexer label: Multiplexer/ROADM description: Wavelength multiplexing and switching color: '#9c27b0' - name: amplifier label: Amplifier description: Optical signal amplification color: '#ff9800' - name: monitoring label: Monitoring description: Optical performance monitoring color: '#4caf50' - name: control label: Control description: System control and management color: '#607d8b' - name: power label: Power Supply description: Power management modules color: '#795548' - name: cooling label: Cooling description: Thermal management color: '#00bcd4' - name: TransponderModule namespace: Dcim include_in_menu: true label: Transponder Module description: Optical-electrical-optical conversion module (100G, 400G coherent) installed in chassis icon: mdi:chip inherit_from: - DcimGenericModule display_label: computed_name__value uniqueness_constraints: - - computed_name__value human_friendly_id: - computed_name__value attributes: - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ module_bay__computed_name__value }}-TPD-{{ slot_number__value }}' description: Computed module name (e.g., dc1-leaf1 > slot 1-TPD-1/1) - name: slot_number kind: Text optional: false description: Slot number in chassis (e.g., '1/1', '1/2', '2/1') - name: capacity_gbps kind: Number optional: false description: Line rate capacity in Gbps (e.g., 100, 200, 400) - name: modulation_format kind: Dropdown optional: false choices: - name: dp_qpsk label: DP-QPSK description: Dual-polarization QPSK (100G) color: '#2196f3' - name: dp_16qam label: DP-16QAM description: Dual-polarization 16QAM (200G) color: '#9c27b0' - name: dp_8qam label: DP-8QAM description: Dual-polarization 8QAM (150G) color: '#ff9800' - name: dp_64qam label: DP-64QAM description: Dual-polarization 64QAM (400G) color: '#4caf50' description: Modulation format for optical transmission - name: tunable_range kind: Dropdown optional: false choices: - name: c_band label: C-Band (1530-1565nm) description: Full C-Band tunable color: '#9933cc' - name: c_l_band label: C+L-Band (1530-1625nm) description: Extended C+L-Band tunable color: '#cc33cc' - name: fixed label: Fixed Wavelength description: Non-tunable, fixed wavelength color: '#95a5a6' description: Wavelength tuning capability - name: fec_type kind: Dropdown optional: true choices: - name: sd_fec label: SD-FEC description: Soft-decision FEC (7% overhead) color: '#2196f3' - name: hd_fec label: HD-FEC description: Hard-decision FEC (25% overhead) color: '#ff9800' - name: c_fec label: C-FEC description: Concatenated FEC color: '#4caf50' - name: o_fec label: O-FEC description: OpenFEC color: '#9c27b0' description: Forward error correction type - name: client_interface kind: Text optional: true description: Client-side interface (e.g., 100GE, 400GE, OTU4) - name: line_interface kind: Text optional: true description: Line-side interface (e.g., CFP2-DCO, QSFP28, QSFP-DD) relationships: - name: tuned_channel peer: DcimDWDMChannel cardinality: one kind: Attribute optional: true identifier: dwdm_channel__transponder_modules description: DWDM channel currently tuned/configured - name: connected_to_optical_node peer: DcimOpticalNode cardinality: one kind: Attribute optional: true identifier: optical_node__transponder_modules description: Optical node this transponder module connects to (for cross-connects) - name: OpticalMultiplexerModule namespace: Dcim include_in_menu: true label: Optical Multiplexer Module description: Multiplexer/Demultiplexer/ROADM module for wavelength management installed in chassis icon: mdi:router-network inherit_from: - DcimGenericModule display_label: computed_name__value human_friendly_id: - computed_name__value attributes: - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ module_bay__computed_name__value }}-MUX-{{ slot_number__value }}' description: Computed module name (e.g., dc1-leaf1 > slot 1-MUX-1/3) - name: slot_number kind: Text optional: false description: Slot number in chassis (e.g., '1/3', '1/4', '2/3') - name: mux_type kind: Dropdown optional: false choices: - name: passive_mux label: Passive Mux description: Passive wavelength multiplexer (fixed) color: '#95a5a6' - name: passive_demux label: Passive Demux description: Passive wavelength demultiplexer (fixed) color: '#7f8c8d' - name: oadm label: OADM description: Optical Add-Drop Multiplexer color: '#3498db' - name: roadm label: ROADM description: Reconfigurable Optical Add-Drop Multiplexer color: '#9b59b6' - name: wss label: WSS description: Wavelength Selective Switch color: '#e74c3c' description: Type of multiplexer technology - name: channel_capacity kind: Number optional: false description: Number of supported DWDM channels (e.g., 40, 80, 96) - name: technology kind: Dropdown optional: true choices: - name: thin_film_filter label: Thin Film Filter description: Passive thin film filter technology color: '#95a5a6' - name: awg label: AWG description: Arrayed Waveguide Grating color: '#3498db' - name: wss_lcos label: WSS (LCOS) description: Wavelength Selective Switch (Liquid Crystal on Silicon) color: '#9b59b6' - name: wss_mems label: WSS (MEMS) description: Wavelength Selective Switch (Micro-Electro-Mechanical Systems) color: '#e74c3c' description: Underlying multiplexer technology - name: degree_count kind: Number optional: true description: Number of ROADM degrees (e.g., 2, 4, 8, 16) - name: colorless kind: Boolean optional: true default_value: false description: Colorless add/drop capability (any channel on any port) - name: directionless kind: Boolean optional: true default_value: false description: Directionless add/drop capability (any port to any degree) - name: contentionless kind: Boolean optional: true default_value: false description: Contentionless add/drop (CDC - Colorless, Directionless, Contentionless) relationships: - name: degrees peer: DcimRoadmDegree cardinality: many kind: Component optional: true identifier: optical_multiplexer_module__degrees description: ROADM degrees (line ports) on this multiplexer module - name: mappings peer: DcimChannelMapping cardinality: many kind: Attribute optional: true identifier: roadm_module__channel_mappings description: Channel mappings (DirectConnect, CableMapping, FiberMapping) associated with this module - name: RoadmDegree namespace: Dcim include_in_menu: true label: ROADM Degree description: ROADM degree with line port (OL-1, OL-2, etc.) icon: mdi:lan-connect display_label: computed_name__value human_friendly_id: - computed_name__value uniqueness_constraints: - - roadm - degree_number__value - - roadm - line_port__value attributes: - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ roadm__computed_name__value }}-{{ line_port__value }}' description: Computed name (e.g., ROADM-SITE-A-OL-1) - name: degree_number kind: Number optional: false description: Degree number (1-based, e.g., 1, 2, 3, 4) - name: line_port kind: Text optional: false description: Line port designation (e.g., OL-1, OL-2, OL-7) - name: direction kind: Dropdown optional: false choices: - name: north label: North description: Northbound direction color: '#3498db' - name: south label: South description: Southbound direction color: '#e74c3c' - name: east label: East description: Eastbound direction color: '#2ecc71' - name: west label: West description: Westbound direction color: '#f39c12' - name: express label: Express description: Express port (bypass) color: '#9b59b6' - name: local label: Local description: Local add/drop color: '#95a5a6' description: Directional designation of degree - name: degree_type kind: Dropdown optional: false choices: - name: line label: Line description: Line port (connects to fiber link) color: '#2196f3' - name: express label: Express description: Express port (R-x ports, bypass) color: '#9c27b0' - name: add_drop label: Add/Drop description: Local add/drop for transponders color: '#4caf50' description: Type of ROADM degree - name: wavelength_capacity kind: Number optional: true description: Number of wavelengths supported on this degree - name: description kind: Text optional: true description: Additional degree details relationships: - name: roadm peer: DcimOpticalMultiplexerModule cardinality: one kind: Parent optional: false identifier: optical_multiplexer_module__degrees description: Parent ROADM module containing this degree - name: connected_fiber peer: DcimFiberLink cardinality: one kind: Attribute optional: true identifier: fiber_link__roadm_degree description: Fiber link connected to this line port - name: channel_mappings peer: DcimChannelMapping cardinality: many kind: Attribute optional: true identifier: roadm_degree__channel_mappings description: Channel mappings involving this degree - name: WSSConnect namespace: Dcim inherit_from: - DcimChannelMapping include_in_menu: true label: WSS Connect description: Internal WSS cross-connect between degrees on SAME ROADM device (0m distance) icon: mdi:swap-horizontal display_label: computed_name__value human_friendly_id: - computed_name__value uniqueness_constraints: - - roadm - channel attributes: - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ roadm__computed_name__value }}-{{ channel__channel_name__value }}' description: Computed connection ID - name: description kind: Text optional: true description: Additional cross-connect details relationships: - name: roadm peer: DcimOpticalMultiplexerModule cardinality: one kind: Attribute optional: false identifier: roadm_module__channel_mappings description: Parent ROADM device - name: CableMapping namespace: Dcim inherit_from: - DcimChannelMapping include_in_menu: true label: Cable Mapping description: Local patch cable between degrees on DIFFERENT ROADMs (meters distance, same site) icon: mdi:cable-data display_label: computed_name__value human_friendly_id: - computed_name__value uniqueness_constraints: - - cable - channel attributes: - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ cable__label__value }}-{{ channel__channel_name__value }}' description: Computed mapping name (e.g., CABLE-123-CH58) - name: description kind: Text optional: true description: Additional cable mapping details relationships: - name: cable peer: DcimCable cardinality: one kind: Attribute optional: false identifier: cable__channel_mappings description: Physical patch cable connecting the degrees - name: OpticalAmplifierModule namespace: Dcim include_in_menu: true label: Optical Amplifier Module description: Optical signal amplifier module (EDFA, Raman, VGC) installed in chassis icon: mdi:amplifier inherit_from: - DcimGenericModule display_label: computed_name__value human_friendly_id: - computed_name__value attributes: - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ module_bay__computed_name__value }}-AMP-{{ slot_number__value }}' description: Computed module name (e.g., dc1-leaf1 > slot 1-AMP-1/5) - name: slot_number kind: Text optional: false description: Slot number in chassis (e.g., '1/5', '1/6', '2/5') - name: amplifier_type kind: Dropdown optional: false choices: - name: edfa label: EDFA description: Erbium-Doped Fiber Amplifier color: '#2196f3' - name: raman label: Raman description: Raman Amplifier (distributed) color: '#9c27b0' - name: soa label: SOA description: Semiconductor Optical Amplifier color: '#ff9800' - name: vgc label: VGC description: Variable Gain Controller color: '#4caf50' - name: hybrid label: Hybrid description: Hybrid amplifier (e.g., EDFA + Raman) color: '#607d8b' description: Type of optical amplifier - name: stage kind: Dropdown optional: false choices: - name: booster label: Booster description: Power amplifier (transmit side) color: '#e74c3c' - name: inline label: Inline description: In-line amplifier (along fiber span) color: '#3498db' - name: preamplifier label: Pre-Amplifier description: Low-noise amplifier (receive side) color: '#2ecc71' description: Amplifier stage position - name: gain_db kind: Text optional: true description: Typical gain in dB (e.g., '20.0', '17.5') - name: max_output_power_dbm kind: Text optional: true description: Maximum output power in dBm (e.g., '17.0', '23.0') - name: noise_figure_db kind: Text optional: true description: Noise figure in dB (e.g., '5.5', '4.8') - name: operating_band kind: Dropdown optional: false choices: - name: c_band label: C-Band (1530-1565nm) description: C-Band operation color: '#9933cc' - name: l_band label: L-Band (1565-1625nm) description: L-Band operation color: '#cc33cc' - name: c_l_band label: C+L-Band description: Dual-band operation color: '#ff33cc' description: Operating wavelength band relationships: - name: amplified_link peer: DcimFiberLink cardinality: one kind: Attribute optional: true identifier: fiber_link__amplifier_modules description: Fiber link being amplified (for inline amplifiers) - name: OpticalService namespace: Dcim include_in_menu: true label: Optical Service description: End-to-end optical transport service (customer circuit) spanning multiple sites icon: mdi:transit-connection-variant display_label: service_name__value human_friendly_id: - service_id__value uniqueness_constraints: - - service_id__value order_by: - service_id__value attributes: - name: service_id kind: Text optional: false description: Unique service identifier - name: service_name kind: Text optional: false description: 'Customer service name (e.g., ''ATC 10G - Albion to Arco #2'')' - name: service_type kind: Dropdown optional: false choices: - name: wavelength label: Wavelength Service description: Dedicated wavelength end-to-end color: '#9c27b0' - name: transport label: Transport Service description: Layer 1 optical transport color: '#2196f3' - name: ip_transit label: IP Transit description: Layer 3 IP service over optical color: '#4caf50' description: Type of optical service - name: bandwidth kind: Text optional: false description: Service bandwidth (e.g., '10G', '100G', '400G') - name: status kind: Dropdown optional: false default_value: planned choices: - name: planned label: Planned description: Service planned but not provisioned color: '#95a5a6' - name: provisioned label: Provisioned description: Service configured but not active color: '#f5a623' - name: active label: Active description: Service in production color: '#00cc00' - name: maintenance label: Maintenance description: Service under maintenance color: '#ff9800' - name: failed label: Failed description: Service down color: '#cc0000' description: Service operational status - name: customer_circuit_id kind: Text optional: true description: Customer's circuit ID (e.g., '99/OKFS/000029//SYG') - name: provision_date kind: DateTime optional: true description: Date service was provisioned - name: description kind: Text optional: true description: Additional service details relationships: - name: transponders peer: DcimTransponderModule cardinality: many min_count: 2 max_count: 2 kind: Attribute optional: false identifier: transponder_module__optical_services description: Two transponders (endpoints) for this service (bidirectional) - name: channel peer: DcimDWDMChannel cardinality: one kind: Attribute optional: false identifier: dwdm_channel__services description: DWDM channel used for this service - name: primary_path peer: DcimOpticalPath cardinality: one kind: Component optional: false identifier: optical_service__primary_path description: Primary optical path for this service - name: backup_path peer: DcimOpticalPath cardinality: one kind: Component optional: true identifier: optical_service__backup_path description: Backup/protection path for this service - name: OpticalPath namespace: Dcim include_in_menu: true label: Optical Path description: Ordered sequence of segments forming an optical path through the network icon: mdi:map-marker-path display_label: name__value human_friendly_id: - name__value attributes: - name: name kind: Text optional: false unique: true description: Path name (e.g., MS-W17D5-primary) - name: path_type kind: Dropdown optional: false choices: - name: primary label: Primary description: Primary working path color: '#2196f3' - name: backup label: Backup description: Backup/protection path color: '#ff9800' - name: express label: Express description: Express bypass path color: '#9c27b0' description: Type of path (primary, backup, express) - name: total_distance_km kind: Text optional: true description: Total path distance in kilometers - name: total_loss_db kind: Text optional: true description: Total optical loss in dB - name: hop_count kind: Number optional: true description: Number of ROADM hops in path - name: is_active kind: Boolean optional: false default_value: true description: Whether this path is currently active relationships: - name: segments peer: DcimPathSegment identifier: optical_path__segments cardinality: many kind: Component optional: true description: Ordered segments forming this path - name: is_primary_of_service peer: DcimOpticalService identifier: optical_service__primary_path cardinality: one kind: Attribute optional: true description: Optical service using this path - name: is_backup_of_service peer: DcimOpticalService identifier: optical_service__backup_path cardinality: one kind: Attribute optional: true description: Optical service using this path as backup - name: PathSegment namespace: Dcim include_in_menu: true label: Path Segment description: One hop in optical path - references channel mapping (fiber, cable, or cross-connect) icon: mdi:ray-start-arrow display_label: computed_name__value human_friendly_id: - computed_name__value uniqueness_constraints: - - path - segment_order__value attributes: - name: computed_name kind: Text read_only: true optional: false computed_attribute: kind: Jinja2 jinja2_template: '{{ path__name__value }}-SEG{{ segment_order__value }}' description: Computed segment name (e.g., MS-W17D5-primary-SEG1) - name: segment_order kind: Number optional: false description: Order of this segment in the path (1, 2, 3...) - name: segment_type kind: Dropdown optional: false choices: - name: fiber label: Fiber description: Segment traverses a long-haul fiber link color: '#2196f3' - name: cross_connect label: Cross-Connect description: Segment crosses through ROADM degrees (internal WSS) color: '#9c27b0' - name: cable label: Cable description: Segment traverses a local patch cable color: '#ff9800' - name: add label: Add description: Service added at this point (A-end) color: '#4caf50' - name: drop label: Drop description: Service dropped at this point (Z-end) color: '#ff5722' description: Type of segment - name: loss_db kind: Text optional: true description: Optical loss for this segment in dB relationships: - name: path peer: DcimOpticalPath cardinality: one kind: Parent optional: false identifier: optical_path__segments description: Parent path containing this segment - name: channel_mapping peer: DcimChannelMapping cardinality: one kind: Attribute on_delete: cascade optional: true identifier: channel_mapping__path_segments description: Channel mapping (WSS Connect, Cable Mapping, or Fiber Mapping) extensions: nodes: - kind: DcimPhysicalDevice relationships: - name: optical_node peer: DcimOpticalNode kind: Generic optional: true cardinality: many identifier: optical_nodes__physical_device ``` --- # Organization Base Organization schema: the generic organization plus Manufacturer and Provider nodes. ## Details[​](#details "Direct link to Details") * **Dependencies:** No dependencies ## Nodes[​](#nodes "Direct link to Nodes") ### Manufacturer[​](#manufacturer "Direct link to Manufacturer") * **Description:** Device Manufacturer * **Namespace:** Organization * **Icon:** mdi :domain * **Inherit From:** OrganizationGeneric #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | -------------- | -------- | ----------- | --------- | | device\_types | DcimDeviceType | True | many | Attribute | | platforms | DcimPlatform | True | many | Attribute | ### Provider[​](#provider "Direct link to Provider") * **Description:** Circuit or Location Provider * **Namespace:** Organization * **Icon:** mdi :domain * **Inherit From:** OrganizationGeneric ## Generics[​](#generics "Direct link to Generics") ### Generic[​](#generic "Direct link to Generic") * **Label:** Organization * **Description:** An organization represents a legal entity, such as a company. * **Namespace:** Organization * **Icon:** mdi :domain * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | ---------- | -------- | ----------- | --------- | | tags | BuiltinTag | True | many | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: Generic namespace: Organization description: An organization represents a legal entity, such as a company. label: Organization icon: mdi:domain include_in_menu: true human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: tags peer: BuiltinTag kind: Attribute cardinality: many optional: true order_weight: 2000 nodes: - name: Manufacturer namespace: Organization description: Device Manufacturer icon: mdi:domain include_in_menu: true menu_placement: OrganizationGeneric inherit_from: - OrganizationGeneric relationships: - name: device_types peer: DcimDeviceType kind: Attribute cardinality: many optional: true identifier: manufacturer__device_type order_weight: 1300 - name: platforms peer: DcimPlatform kind: Attribute cardinality: many optional: true identifier: manufacturer__platform order_weight: 1400 - name: Provider namespace: Organization description: Circuit or Location Provider icon: mdi:domain include_in_menu: true menu_placement: OrganizationGeneric inherit_from: - OrganizationGeneric ``` --- # Patch Panel This schema extension allows you to capture patch panel related information like rear and front interfaces and the mapping between them. You can insert the patch panel into a rack and leverage the device type model. Cassettes and other inserts are tracked as regular device modules in module bays, through extensions/device\_module. The front and rear interfaces accept all sorts of connectors, so you can plug cables, circuits and cross-connects into them. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/device\_module](/schema-library/reference/device_module.md) ## Nodes[​](#nodes "Direct link to Nodes") ### PatchPanel[​](#patchpanel "Direct link to PatchPanel") * **Label:** Patch Panel * **Description:** A Patch Panel used for managing network cable connections in a data center or telecom setup. * **Namespace:** Dcim * **Icon:** ic :round-device-hub * **Human Friendly ID:** name\_\_value * **Inherit From:** DcimPhysicalDevice #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ------------------------------------------------------------------------- | ------ | -------- | -------------- | ------- | | name | | Text | False | | | | module\_capacity | The maximum number of modules that can be housed within this patch panel. | Number | True | | | | description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------------- | ---------------------------- | -------- | ----------- | --------- | | front\_interfaces | DcimFrontPatchPanelInterface | True | many | Component | | rear\_interfaces | DcimRearPatchPanelInterface | True | many | Component | ### FrontPatchPanelInterface[​](#frontpatchpanelinterface "Direct link to FrontPatchPanelInterface") * **Label:** Patch Panel Front Interfaces * **Description:** Front interface of a patch panel, where equipment is patched in. * **Namespace:** Dcim * **Uniqueness Constraints:** * patch\_panel, name\_\_value * **Human Friendly ID:** patch\_panel\_\_name\_\_value, name\_\_value * **Inherit From:** DcimEndpoint, DcimGenericPatchPanelInterface #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------------------- | --------------------------- | -------- | ----------- | --------- | | corresponding\_front\_rear | DcimRearPatchPanelInterface | True | one | Attribute | | patch\_panel | DcimPatchPanel | False | one | Parent | ### RearPatchPanelInterface[​](#rearpatchpanelinterface "Direct link to RearPatchPanelInterface") * **Label:** Patch Panel Rear Interfaces * **Description:** Rear interface of a patch panel, terminating the permanent cabling. * **Namespace:** Dcim * **Uniqueness Constraints:** * patch\_panel, name\_\_value * **Human Friendly ID:** patch\_panel\_\_name\_\_value, name\_\_value * **Inherit From:** DcimEndpoint, DcimGenericPatchPanelInterface #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------------------- | ---------------------------- | -------- | ----------- | --------- | | corresponding\_front\_rear | DcimFrontPatchPanelInterface | True | many | Attribute | | patch\_panel | DcimPatchPanel | False | one | Parent | ## Generics[​](#generics "Direct link to Generics") ### GenericPatchPanelInterface[​](#genericpatchpanelinterface "Direct link to GenericPatchPanelInterface") * **Label:** Patch Panel Interfaces * **Description:** Generic interface of a patch panel, front or rear. * **Namespace:** Dcim * **Icon:** mdi :ethernet #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------------- | ----------- | -------- | -------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | name | | Text | False | | | | description | | Text | True | | | | connector\_type | | Dropdown | False | | fc, lc, lc\_pc, lc\_upc, lc\_apc, lsh, lsh\_pc, lsh\_upc, lsh\_apc, lx\_5, lx\_5\_pc, lx\_5\_upc, lx\_5\_apc, mpo, mtrj, sc, sc\_pc, sc\_upc, sc\_apc, st, cs, sn, sma\_905, sma\_906, urm\_p2, urm\_p4, urm\_p8, splice | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: GenericPatchPanelInterface namespace: Dcim description: Generic interface of a patch panel, front or rear. label: Patch Panel Interfaces icon: mdi:ethernet include_in_menu: true menu_placement: DcimPatchPanel attributes: - name: name kind: Text optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 - name: connector_type kind: Dropdown choices: - name: fc label: FC description: Standardized fiber optic connector used primarily in datacom and telecom applications. - name: lc label: LC description: Compact fiber optic connector with a push-pull mechanism. - name: lc_pc label: LC/PC description: Polished LC connector providing physical contact (PC). - name: lc_upc label: LC/UPC description: Ultra-Physical Contact (UPC) variant of the LC connector with enhanced polish. - name: lc_apc label: LC/APC description: Angled Physical Contact (APC) version of the LC connector with a slanted fiber end-face. - name: lsh label: LSH description: European fiber optic connector offering high durability. - name: lsh_pc label: LSH/PC description: Physical Contact version of LSH with standard polish. - name: lsh_upc label: LSH/UPC description: Ultra-Physical Contact variant of LSH, minimizing return loss with a superior polish. - name: lsh_apc label: LSH/APC description: Angled Physical Contact version of LSH, designed to reduce back reflections. - name: lx_5 label: LX.5 description: Miniaturized fiber optic connector similar to LC but with an additional shutter mechanism. - name: lx_5_pc label: LX.5/PC description: Physical Contact version of LX.5. - name: lx_5_upc label: LX.5/UPC description: Ultra-Physical Contact variant of LX.5. - name: lx_5_apc label: LX.5/APC description: Angled Physical Contact version of LX.5. - name: mpo label: MPO description: Multi-fiber Push-On connector typically used in data centers for high-speed applications. - name: mtrj label: MTRJ description: Male-to-female fiber optic connector with two fibers. - name: sc label: SC description: Square fiber optic connector with push-pull lock. - name: sc_pc label: SC/PC description: Physical Contact SC connector with a polished end-face. - name: sc_upc label: SC/UPC description: Ultra-Physical Contact variant of SC. - name: sc_apc label: SC/APC description: Angled Physical Contact version of SC. - name: st label: ST description: Bayonet-style fiber optic connector primarily used in industrial and military applications. - name: cs label: CS description: Compact connector with a high-density duplex configuration. - name: sn label: SN description: Small-form connector with dual fibers. - name: sma_905 label: SMA 905 description: Stainless steel fiber optic connector. - name: sma_906 label: SMA 906 description: Variant of SMA 905 with similar durability, frequently used in high-vibration settings. - name: urm_p2 label: URM-P2 description: Specialized fiber optic connector for industrial and harsh environments. - name: urm_p4 label: URM-P4 description: Similar to URM-P2 but designed for higher performance. - name: urm_p8 label: URM-P8 description: Enhanced version of URM connectors with higher protection. - name: splice label: Splice description: Permanent fiber connection method where two fiber ends are fused. optional: false order_weight: 1200 nodes: - name: PatchPanel namespace: Dcim description: A Patch Panel used for managing network cable connections in a data center or telecom setup. label: Patch Panel icon: ic:round-device-hub include_in_menu: true inherit_from: - DcimPhysicalDevice human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: module_capacity kind: Number label: Module Capacity optional: true description: The maximum number of modules that can be housed within this patch panel. order_weight: 1300 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: front_interfaces peer: DcimFrontPatchPanelInterface kind: Component cardinality: many optional: true identifier: patch_panel__front_interfaces order_weight: 1350 - name: rear_interfaces peer: DcimRearPatchPanelInterface kind: Component cardinality: many optional: true identifier: patch_panel__rear_interfaces order_weight: 1450 - name: FrontPatchPanelInterface namespace: Dcim description: Front interface of a patch panel, where equipment is patched in. label: Patch Panel Front Interfaces include_in_menu: true menu_placement: DcimGenericPatchPanelInterface inherit_from: - DcimEndpoint - DcimGenericPatchPanelInterface human_friendly_id: - patch_panel__name__value - name__value order_by: - corresponding_front_rear__name__value - name__value display_label: '{{ patch_panel__name__value }} > {{ name__value }}' uniqueness_constraints: - - patch_panel - name__value relationships: - name: corresponding_front_rear peer: DcimRearPatchPanelInterface label: Corresponding rear interface kind: Attribute cardinality: one optional: true order_weight: 1300 - name: patch_panel peer: DcimPatchPanel kind: Parent cardinality: one optional: false identifier: patch_panel__front_interfaces order_weight: 900 - name: RearPatchPanelInterface namespace: Dcim description: Rear interface of a patch panel, terminating the permanent cabling. label: Patch Panel Rear Interfaces include_in_menu: true menu_placement: DcimGenericPatchPanelInterface inherit_from: - DcimEndpoint - DcimGenericPatchPanelInterface human_friendly_id: - patch_panel__name__value - name__value order_by: - patch_panel__name__value - name__value display_label: '{{ patch_panel__name__value }} > {{ name__value }}' uniqueness_constraints: - - patch_panel - name__value relationships: - name: corresponding_front_rear peer: DcimFrontPatchPanelInterface label: Corresponding front interfaces kind: Attribute cardinality: many optional: true order_weight: 1300 - name: patch_panel peer: DcimPatchPanel kind: Parent cardinality: one optional: false identifier: patch_panel__rear_interfaces order_weight: 900 ``` --- # Internet Exchange This schema extension adds an Internet Exchange (IXP) node and an IXP Connection node linking your devices to an exchange, and ties IXP connections to the BGP sessions established over them. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing\_bgp](/schema-library/reference/routing_bgp.md) * [extensions/routing\_bgp\_community](/schema-library/reference/routing_bgp_community.md) * [extensions/routing\_policies\_bgp](/schema-library/reference/routing_policies_bgp.md) ## Nodes[​](#nodes "Direct link to Nodes") ### IXP[​](#ixp "Direct link to IXP") * **Label:** Internet Exchange * **Description:** An Internet Exchange Point (IXP) for peering * **Namespace:** Peering * **Icon:** mdi :network * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ------------------------------------------------ | -------- | -------- | -------------- | ----------------- | | name | Name of the Internet Exchange | Text | False | | | | description | An optional description of the Internet Exchange | Text | True | | | | status | | Dropdown | False | enabled | enabled, disabled | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | ----------------------- | -------- | ----------- | --------- | | connections | PeeringIXPConnection | True | many | Component | | local\_as | RoutingAutonomousSystem | True | one | Attribute | | import\_policies | RoutingPolicyBGP | True | many | Generic | | export\_policies | RoutingPolicyBGP | True | many | Generic | | bgp\_communities | RoutingBGPCommunity | True | many | Generic | | tags | BuiltinTag | True | many | Attribute | ### IXPConnection[​](#ixpconnection "Direct link to IXPConnection") * **Label:** IXP Connection * **Description:** A connection to an Internet Exchange Point (IXP) * **Namespace:** Peering * **Icon:** mdi :lan-connect * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------- | ------------------------------------------ | ---------- | -------- | -------------- | ------------------------------------------------------------------- | | name | Name of the IXP Connection | Text | False | | | | description | Description of the IXP Connection | Text | True | | | | peeringdb\_netixlan | PeeringDB ID for the IXP connection | Number | True | | | | status | | Dropdown | True | enabled | enabled, pre\_maintenance, maintenance, post\_maintenance, disabled | | vlan | VLAN ID for the connection | Number | True | | | | mac\_address | MAC address associated with the connection | MacAddress | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------------- | ----------------- | -------- | ----------- | --------- | | ipv6\_address | IpamIPAddress | True | one | Attribute | | ipv4\_address | IpamIPAddress | True | one | Attribute | | internet\_exchange\_point | PeeringIXP | False | one | Parent | | router | DcimDevice | True | one | Attribute | | bgp\_sessions | RoutingBGPSession | True | many | Component | | tags | BuiltinTag | True | many | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### RoutingBGPSession[​](#routingbgpsession "Direct link to RoutingBGPSession") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | -------------------- | -------- | ----------- | --------- | | ixp\_connection | PeeringIXPConnection | True | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: IXP namespace: Peering description: An Internet Exchange Point (IXP) for peering label: Internet Exchange icon: mdi:network include_in_menu: true human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false description: Name of the Internet Exchange order_weight: 1000 - name: description kind: Text optional: true description: An optional description of the Internet Exchange order_weight: 1100 - name: status kind: Dropdown default_value: enabled choices: - name: enabled label: Enabled description: Internet Exchange is active and operational color: '#00d25b' - name: disabled label: Disabled description: Internet Exchange is not operational color: '#b0bec5' optional: false order_weight: 1200 relationships: - name: connections peer: PeeringIXPConnection kind: Component cardinality: many optional: true description: IXP connections associated with this Internet Exchange order_weight: 1500 - name: local_as peer: RoutingAutonomousSystem kind: Attribute cardinality: one optional: true order_weight: 1300 - name: import_policies peer: RoutingPolicyBGP label: Import Routing Policies kind: Generic cardinality: many optional: true identifier: ixp__import_bgppolicies description: Import routing policies applied to every session on this internet exchange point order_weight: 1600 - name: export_policies peer: RoutingPolicyBGP label: Export Routing Policies kind: Generic cardinality: many optional: true identifier: ixp__export_bgppolicies description: Export routing policies applied to every session on this internet exchange point order_weight: 1650 - name: bgp_communities peer: RoutingBGPCommunity label: BGP Communities kind: Generic cardinality: many optional: true description: BGP communities associated with this internet exchange point order_weight: 1700 - name: tags peer: BuiltinTag kind: Attribute cardinality: many optional: true order_weight: 2000 - name: IXPConnection namespace: Peering description: A connection to an Internet Exchange Point (IXP) label: IXP Connection icon: mdi:lan-connect include_in_menu: true menu_placement: PeeringIXP human_friendly_id: - name__value order_by: - name__value display_label: name__value uniqueness_constraints: - - name__value attributes: - name: name kind: Text unique: true optional: false description: Name of the IXP Connection order_weight: 1000 - name: description kind: Text optional: true description: Description of the IXP Connection order_weight: 1100 - name: peeringdb_netixlan kind: Number label: PeeringDB Netixlan optional: true description: PeeringDB ID for the IXP connection order_weight: 1150 - name: status kind: Dropdown default_value: enabled choices: - name: enabled label: Enabled description: The system is fully operational and functioning as expected. color: '#00d25b' - name: pre_maintenance label: Pre-Maintenance description: Preparation stage before performing maintenance tasks. color: '#f4a261' - name: maintenance label: Maintenance description: Currently undergoing maintenance tasks. color: '#e9c46a' - name: post_maintenance label: Post-Maintenance description: Final checks and verifications after maintenance tasks. color: '#f0e5de' - name: disabled label: Disabled description: The system is not operational and cannot be used. color: '#b0bec5' optional: true order_weight: 1200 - name: vlan kind: Number optional: true description: VLAN ID for the connection order_weight: 1300 - name: mac_address kind: MacAddress label: MAC Address optional: true description: MAC address associated with the connection order_weight: 1350 relationships: - name: ipv6_address peer: IpamIPAddress label: IPv6 Address kind: Attribute cardinality: one optional: true identifier: ixpconn__ipv6_address description: IPv6 address assigned to the connection order_weight: 1400 - name: ipv4_address peer: IpamIPAddress label: IPv4 Address kind: Attribute cardinality: one optional: true identifier: ixpconn__ipv4_address description: IPv4 address assigned to the connection order_weight: 1375 - name: internet_exchange_point peer: PeeringIXP label: IXP kind: Parent cardinality: one optional: false description: The Internet Exchange Point this connection is part of order_weight: 900 - name: router peer: DcimDevice kind: Attribute cardinality: one optional: true description: The router this IXP connection is connected to order_weight: 1500 - name: bgp_sessions peer: RoutingBGPSession label: BGP Sessions kind: Component cardinality: many optional: true identifier: ixpconn__bgpsessions on_delete: no-action description: BGP sessions established over this IXP connection order_weight: 1550 - name: tags peer: BuiltinTag kind: Attribute cardinality: many optional: true order_weight: 2000 extensions: nodes: - kind: RoutingBGPSession relationships: - name: ixp_connection peer: PeeringIXPConnection label: IXP Connection kind: Attribute cardinality: one optional: true identifier: ixpconn__bgpsessions description: The IXP connection this BGP session is established over order_weight: 1800 ``` --- # Physical Disk Simple schema allowing you to capture physical disk information for inventory and lifecycle management. This extension works with any kind of device: apply the DcimDeviceWithPhysicalDisks generic to a model to enable disk tracking. You might also link disks to a location, for instance to capture spares. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### PhysicalDisk[​](#physicaldisk "Direct link to PhysicalDisk") * **Label:** Physical Disk * **Description:** Physical Disk * **Namespace:** Dcim * **Icon:** carbon :vmdk-disk * **Uniqueness Constraints:** * device, name\_\_value * **Human Friendly ID:** device\_\_name\_\_value, name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ------------------------------------------- | -------- | -------- | -------------- | ----------------------------------------------- | | name | | Text | False | | | | disk\_type | Specifies the type of disk | Dropdown | True | | ssd, nvme, hdd, hybrid | | status | Lifecycle status of the hardware component. | Dropdown | False | | in\_inventory, active, decommissioned, disposed | | size | Disk capacity (in GB). | Number | False | | | | serial\_number | Serial number of the disk | Text | True | | | | description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | --------------------------- | -------- | ----------- | ------ | | device | DcimDeviceWithPhysicalDisks | False | one | Parent | ## Generics[​](#generics "Direct link to Generics") ### DeviceWithPhysicalDisks[​](#devicewithphysicaldisks "Direct link to DeviceWithPhysicalDisks") * **Description:** Generic that hold relationship toward physical disks. To apply on device that can have physical disks. * **Namespace:** Dcim #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ---------------- | -------- | ----------- | --------- | | physical\_disks | DcimPhysicalDisk | True | many | Component | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: DeviceWithPhysicalDisks namespace: Dcim description: Generic that hold relationship toward physical disks. To apply on device that can have physical disks. include_in_menu: false attributes: - name: name kind: Text unique: true optional: false allow_override: any order_weight: 1000 relationships: - name: physical_disks peer: DcimPhysicalDisk kind: Component cardinality: many optional: true order_weight: 1950 nodes: - name: PhysicalDisk namespace: Dcim description: Physical Disk label: Physical Disk icon: carbon:vmdk-disk include_in_menu: true human_friendly_id: - device__name__value - name__value order_by: - name__value display_label: name__value uniqueness_constraints: - - device - name__value attributes: - name: name kind: Text optional: false order_weight: 1000 - name: disk_type kind: Dropdown choices: - name: ssd label: SSD description: Solid State Drive color: '#a6c1ff' - name: nvme label: NVMe description: Non-Volatile Memory Express color: '#ffbf80' - name: hdd label: HDD description: Hard Disk Drive color: '#80c7a6' - name: hybrid label: Hybrid description: Hybrid Drive color: '#ffcc80' optional: true description: Specifies the type of disk order_weight: 1300 - name: status kind: Dropdown choices: - name: in_inventory label: In Inventory description: The disk is newly acquired and held in inventory. color: '#6c757d' - name: active label: Active description: The disk is currently in use within the infrastructure. color: '#00d25b' - name: decommissioned label: Decommissioned description: The disk is retired from active use but still stored for potential reuse or auditing. color: '#17a2b8' - name: disposed label: Disposed description: The disk has been securely erased and disposed of. color: '#dc3545' optional: false description: Lifecycle status of the hardware component. order_weight: 1200 - name: size kind: Number label: Size (GB) optional: false description: Disk capacity (in GB). order_weight: 1400 - name: serial_number kind: Text unique: true optional: true description: Serial number of the disk order_weight: 1450 - name: description kind: Text unique: false optional: true order_weight: 1100 relationships: - name: device peer: DcimDeviceWithPhysicalDisks kind: Parent cardinality: one optional: false order_weight: 900 ``` --- # QinQ This schema extension adds the Service VLAN and Customer VLAN nodes needed to model QinQ on top of the VLAN extension. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/vlan](/schema-library/reference/vlan.md) ## Nodes[​](#nodes "Direct link to Nodes") ### SVLAN[​](#svlan "Direct link to SVLAN") * **Label:** S-VLAN * **Description:** A VLAN used to encapsulate multiple CVLANs. * **Namespace:** Ipam * **Uniqueness Constraints:** * vlan\_id\_\_value, vlan\_group * **Inherit From:** IpamGenericVLAN #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | --------- | -------- | ----------- | --------- | | cvlan | IpamCVLAN | True | many | Attribute | ### CVLAN[​](#cvlan "Direct link to CVLAN") * **Label:** C-VLAN * **Description:** A VLAN assigned to customer traffic, encapsulated within an S-VLAN. * **Namespace:** Ipam * **Uniqueness Constraints:** * vlan\_id\_\_value, svlan * **Human Friendly ID:** svlan\_\_name\_\_value, vlan\_id\_\_value * **Inherit From:** IpamGenericVLAN #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | --------- | -------- | ----------- | --------- | | svlan | IpamSVLAN | False | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: SVLAN namespace: Ipam description: A VLAN used to encapsulate multiple CVLANs. label: S-VLAN include_in_menu: false inherit_from: - IpamGenericVLAN uniqueness_constraints: - - vlan_id__value - vlan_group relationships: - name: cvlan peer: IpamCVLAN label: Customer vlan(s) kind: Attribute cardinality: many optional: true identifier: vlan__qinq direction: outbound order_weight: 1600 - name: CVLAN namespace: Ipam description: A VLAN assigned to customer traffic, encapsulated within an S-VLAN. label: C-VLAN include_in_menu: false inherit_from: - IpamGenericVLAN human_friendly_id: - svlan__name__value - vlan_id__value display_label: '{{ svlan__name__value }}-{{ vlan_id__value }}' uniqueness_constraints: - - vlan_id__value - svlan attributes: - name: name kind: Text read_only: true computed_attribute: kind: Jinja2 jinja2_template: '{{ svlan__name__value }}-{{ vlan_id__value }}' optional: true order_weight: 1000 relationships: - name: svlan peer: IpamSVLAN label: Supplier vlan kind: Attribute cardinality: one optional: false identifier: vlan__qinq direction: inbound order_weight: 1600 ``` --- # QoS This schema extension contains models for Quality of Service (QoS) ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### ForwardingClass[​](#forwardingclass "Direct link to ForwardingClass") * **Label:** Forwarding Class * **Description:** Represents a forwarding class in QoS with distinct loss priorities. * **Namespace:** Qos * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------------------- | ------------------------------------------- | ---- | -------- | -------------- | ------- | | name | Name of the forwarding class. | Text | | | | | high\_loss\_priority\_code | List of code points for high loss priority. | List | True | | | | low\_loss\_priority\_code | List of code points for low loss priority. | List | True | | | ### ClassOfService[​](#classofservice "Direct link to ClassOfService") * **Label:** Class of Service * **Description:** Defines a Class of Service configuration. * **Namespace:** Qos * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------------------------- | ---- | -------- | -------------- | ------- | | name | Name of the Class of Service. | Text | | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------------------- | ------------------------ | -------- | ----------- | ---- | | traffic\_control\_profiles | QosTrafficControlProfile | True | many | | ### TrafficControlProfile[​](#trafficcontrolprofile "Direct link to TrafficControlProfile") * **Label:** Traffic Control Profile * **Description:** Defines a traffic control profile with an active/inactive state. * **Namespace:** Qos * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------ | -------------------------------------------------------- | -------- | -------- | -------------- | ---------------- | | name | Name of the traffic control profile. | Text | | | | | status | Status of the traffic control profile (active/inactive). | Dropdown | | inactive | active, inactive | ### Classifier[​](#classifier "Direct link to Classifier") * **Label:** Classifier * **Description:** Represents a classifier mapping DSCP or EXP values to forwarding classes. * **Namespace:** Qos * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ------------------------------------- | -------- | -------- | -------------- | -------------------- | | name | Name of the classifier. | Text | | | | | classifier\_type | Type of classifier (DSCP, EXP, etc.). | Dropdown | | | dscp, exp, dscp-ipv6 | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------- | ------------------ | -------- | ----------- | ---- | | forwarding\_classes | QosForwardingClass | True | many | | ### Scheduler[​](#scheduler "Direct link to Scheduler") * **Label:** Scheduler * **Description:** Represents a scheduler configuration. * **Namespace:** Qos * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-4 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | -------------------------------- | -------- | -------- | -------------- | ---------------------- | | name | Name of the scheduler. | Text | | | | | transmit\_rate | Transmit rate in percentage. | Number | | | | | buffer\_size | Buffer size in percentage. | Number | | | | | priority | Priority of the scheduler. | Dropdown | True | | low, high, strict-high | | excess\_priority | Excess priority when applicable. | Dropdown | True | | low, high | ### SchedulerMap[​](#schedulermap "Direct link to SchedulerMap") * **Label:** Scheduler Map * **Description:** Defines mappings of schedulers to forwarding classes. * **Namespace:** Qos * **Uniqueness Constraints:** * name\_\_value * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-5 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | -------------------------- | ---- | -------- | -------------- | ------- | | name | Name of the scheduler map. | Text | | | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------- | ------------------ | -------- | ----------- | ---- | | schedulers | QosScheduler | True | many | | | forwarding\_classes | QosForwardingClass | True | many | | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: ForwardingClass namespace: Qos label: Forwarding Class description: Represents a forwarding class in QoS with distinct loss priorities. display_label: name__value order_by: - name__value human_friendly_id: - name__value uniqueness_constraints: - - name__value menu_placement: QosClassOfService attributes: - name: name kind: Text description: Name of the forwarding class. unique: true order_weight: 1000 - name: high_loss_priority_code kind: List description: List of code points for high loss priority. optional: true order_weight: 1200 - name: low_loss_priority_code kind: List description: List of code points for low loss priority. optional: true order_weight: 1300 - name: ClassOfService namespace: Qos label: Class of Service description: Defines a Class of Service configuration. display_label: name__value order_by: - name__value human_friendly_id: - name__value uniqueness_constraints: - - name__value attributes: - name: name kind: Text description: Name of the Class of Service. unique: true order_weight: 1000 relationships: - name: traffic_control_profiles peer: QosTrafficControlProfile description: List of traffic control profiles. cardinality: many optional: true order_weight: 1200 - name: TrafficControlProfile namespace: Qos label: Traffic Control Profile description: Defines a traffic control profile with an active/inactive state. display_label: name__value order_by: - name__value human_friendly_id: - name__value uniqueness_constraints: - - name__value menu_placement: QosClassOfService attributes: - name: name kind: Text description: Name of the traffic control profile. unique: true order_weight: 1000 - name: status kind: Dropdown description: Status of the traffic control profile (active/inactive). choices: - name: active label: Active - name: inactive label: Inactive default_value: inactive order_weight: 1200 - name: Classifier namespace: Qos label: Classifier description: Represents a classifier mapping DSCP or EXP values to forwarding classes. display_label: name__value order_by: - name__value human_friendly_id: - name__value uniqueness_constraints: - - name__value menu_placement: QosClassOfService attributes: - name: name kind: Text description: Name of the classifier. unique: true order_weight: 1000 - name: classifier_type kind: Dropdown description: Type of classifier (DSCP, EXP, etc.). choices: - name: dscp label: DSCP Classifier - name: exp label: EXP Classifier - name: dscp-ipv6 label: DSCP-IPv6 Classifier order_weight: 1200 relationships: - name: forwarding_classes peer: QosForwardingClass description: List of forwarding classes defined in the classifier. cardinality: many optional: true order_weight: 1300 - name: Scheduler namespace: Qos label: Scheduler description: Represents a scheduler configuration. display_label: name__value order_by: - name__value human_friendly_id: - name__value uniqueness_constraints: - - name__value menu_placement: QosClassOfService attributes: - name: name kind: Text description: Name of the scheduler. unique: true order_weight: 1000 - name: transmit_rate label: Transmit Rate (%) kind: Number description: Transmit rate in percentage. order_weight: 1200 - name: buffer_size label: Buffer Size (%) kind: Number description: Buffer size in percentage. order_weight: 1300 - name: priority kind: Dropdown description: Priority of the scheduler. choices: - name: low label: Low Priority - name: high label: High Priority - name: strict-high label: Strict High Priority optional: true order_weight: 1400 - name: excess_priority kind: Dropdown description: Excess priority when applicable. choices: - name: low label: Low Excess Priority - name: high label: High Excess Priority optional: true order_weight: 1500 - name: SchedulerMap namespace: Qos label: Scheduler Map description: Defines mappings of schedulers to forwarding classes. display_label: name__value order_by: - name__value human_friendly_id: - name__value uniqueness_constraints: - - name__value menu_placement: QosClassOfService attributes: - name: name kind: Text description: Name of the scheduler map. unique: true order_weight: 1000 relationships: - name: schedulers peer: QosScheduler description: List of schedulers defined in the map. cardinality: many optional: true order_weight: 1200 - name: forwarding_classes peer: QosForwardingClass description: List of forwarding classes associated with schedulers. cardinality: many optional: true order_weight: 1300 ``` --- # Rack This schema extension introduces a Rack node representing a physical equipment rack installed at a site. Racks are placed at a LocationSite, which comes from extensions/location\_site or extensions/location\_minimal, and can host devices and other equipment. experimental/location\_extended defines its own Rack node, so do not load both. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/location\_site](/schema-library/reference/location_site.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Rack[​](#rack "Direct link to Rack") * **Label:** Rack * **Description:** A physical equipment rack installed at a site. * **Namespace:** Location * **Icon:** clarity :rack-server-line * **Uniqueness Constraints:** * site, name\_\_value * **Human Friendly ID:** site\_\_name\_\_value, name\_\_value * **Inherit From:** LocationHosting #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | ----------- | -------- | -------- | -------------- | ----------------------------------------------------------------- | | name | | Text | False | | | | facility | | Text | True | | | | status | | Dropdown | False | active | active, available, reserved, planned, decommissioning, deprecated | | height | | Number | False | 42 | | | serial\_number | | Text | True | | | | asset\_tag | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | ------------ | -------- | ----------- | --------- | | site | LocationSite | False | one | Parent | | tags | BuiltinTag | True | many | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### LocationSite[​](#locationsite "Direct link to LocationSite") #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | ------------ | -------- | ----------- | --------- | | racks | LocationRack | True | many | Component | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Rack namespace: Location description: A physical equipment rack installed at a site. label: Rack icon: clarity:rack-server-line include_in_menu: true menu_placement: LocationGeneric inherit_from: - LocationHosting human_friendly_id: - site__name__value - name__value order_by: - name__value display_label: name__value uniqueness_constraints: - - site - name__value attributes: - name: name kind: Text unique: false optional: false order_weight: 1000 - name: facility kind: Text optional: true order_weight: 1150 - name: status kind: Dropdown default_value: active choices: - name: active label: Active description: Rack is installed and in use. color: '#00d25b' - name: available label: Available description: Rack is installed and has capacity for new equipment. color: '#00bcd4' - name: reserved label: Reserved description: Rack is allocated but not yet in use. color: '#4d90fe' - name: planned label: Planned description: Rack is planned but not yet installed. color: '#2196f3' - name: decommissioning label: Decommissioning description: Rack is being phased out and emptied. color: '#ff9800' - name: deprecated label: Deprecated description: Rack has been removed from service. color: '#6c757d' optional: false order_weight: 1200 - name: height kind: Number label: Height (U) default_value: 42 optional: false order_weight: 1300 - name: serial_number kind: Text optional: true order_weight: 1350 - name: asset_tag kind: Text optional: true order_weight: 1400 relationships: - name: site peer: LocationSite kind: Parent cardinality: one optional: false identifier: site__racks order_weight: 900 - name: tags peer: BuiltinTag kind: Attribute cardinality: many optional: true order_weight: 2000 extensions: nodes: - kind: LocationSite relationships: - name: racks peer: LocationRack kind: Component cardinality: many optional: true identifier: site__racks order_weight: 1850 ``` --- # Routing This schema extension contains the Routing Protocol generic that every protocol extension builds on. The idea is one protocol instance per IpamVRF and DcimDevice pair. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/vrf](/schema-library/reference/vrf.md) ## Generics[​](#generics "Direct link to Generics") ### Protocol[​](#protocol "Direct link to Protocol") * **Label:** Routing Protocol * **Description:** Generic protocol model for routing protocols * **Namespace:** Routing * **Icon:** carbon :router #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ------------------------------------- | -------- | -------- | -------------- | ------------------------- | | description | Description of the protocol | Text | False | | | | status | Status of the Protocol Configuration. | Dropdown | False | | active, disabled, deleted | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | ---------- | -------- | ----------- | --------- | | device | DcimDevice | False | one | Parent | | vrf | IpamVRF | False | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: Protocol namespace: Routing description: Generic protocol model for routing protocols label: Routing Protocol icon: carbon:router include_in_menu: true attributes: - name: description kind: Text unique: true optional: false description: Description of the protocol order_weight: 1100 - name: status kind: Dropdown choices: - name: active label: Active description: Configuration is active and operational. color: '#00d25b' - name: disabled label: Disabled description: Configuration has been disabled. color: '#D3D3D3' - name: deleted label: Deleted description: Configuration has been deleted. color: '#FAD7A0' optional: false description: Status of the Protocol Configuration. order_weight: 1200 relationships: - name: device peer: DcimDevice kind: Parent cardinality: one optional: false order_weight: 900 - name: vrf peer: IpamVRF label: VRF kind: Attribute cardinality: one optional: false order_weight: 1050 ``` --- # Routing Aggregate This schema extension adds an Aggregate Route node: a summary route, with its destination prefix and discard flag, announced by a routing protocol instance. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing](/schema-library/reference/routing.md) ## Nodes[​](#nodes "Direct link to Nodes") ### AggregateRoute[​](#aggregateroute "Direct link to AggregateRoute") * **Label:** Aggregate Routes * **Description:** Aggregate Protocol with action and BGP communities * **Namespace:** Routing * **Icon:** grommet-icons :aggregate * **Inherit From:** RoutingProtocol #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ----------- | ------- | -------- | -------------- | ------- | | discard | | Boolean | True | False | | | import\_policies | | Text | True | | | | export\_policies | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------- | ---------- | -------- | ----------- | --------- | | destination | IpamPrefix | True | | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: AggregateRoute namespace: Routing description: Aggregate Protocol with action and BGP communities label: Aggregate Routes icon: grommet-icons:aggregate include_in_menu: true menu_placement: RoutingProtocol inherit_from: - RoutingProtocol attributes: - name: discard kind: Boolean label: Discard default_value: false optional: true order_weight: 1350 - name: import_policies kind: Text optional: true order_weight: 1400 - name: export_policies kind: Text optional: true order_weight: 1450 relationships: - name: destination peer: IpamPrefix kind: Attribute optional: true description: Destination network aggregated by this route. order_weight: 1300 ``` --- # Routing BGP This schema extension adds Autonomous System, BGP Peer Group and BGP Session nodes, and links autonomous systems to devices and organizations. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing](/schema-library/reference/routing.md) ## Nodes[​](#nodes "Direct link to Nodes") ### AutonomousSystem[​](#autonomoussystem "Direct link to AutonomousSystem") * **Label:** Autonomous System * **Description:** An Autonomous System (AS) is a set of Internet routable IP prefixes belonging to a network * **Namespace:** Routing * **Icon:** mdi :bank-circle-outline * **Human Friendly ID:** asn\_\_value, name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ------------------------------------ | ------ | -------- | -------------- | ------- | | name | Name of the Autonomous System | Text | False | | | | asn | Autonomous System Number | Number | False | | | | description | Description of the Autonomous System | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ------------------- | -------- | ----------- | --------- | | organization | OrganizationGeneric | False | one | Attribute | | location | LocationGeneric | True | one | Attribute | | devices | DcimDevice | True | many | Attribute | ### BGPPeerGroup[​](#bgppeergroup "Direct link to BGPPeerGroup") * **Label:** BGP Peer Group * **Description:** A BGP Peer Group is used to regroup parameters that are shared across multiple peers * **Namespace:** Routing * **Icon:** mdi :view-grid-plus-outline * **Human Friendly ID:** name\_\_value, description\_\_value * **Inherit From:** RoutingProtocol #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ---------------------------------------------------------------------------- | -------- | -------- | -------------- | ---------- | | name | Name of the BGP Group | Text | False | | | | import\_policies | | Text | True | | | | export\_policies | | Text | True | | | | maximum\_routes | Maximum routes for the BGP Group. | Number | True | | | | local\_pref | Force Local Pref for this BGP Peer Group. | Number | True | | | | send\_community | Whether to send community attributes. | Checkbox | True | | | | address\_family | The address family for the routing policy indicating the type of IP address. | Dropdown | False | ipv4 | ipv4, ipv6 | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------- | ----------------------- | -------- | ----------- | --------- | | local\_as | RoutingAutonomousSystem | True | one | Attribute | | remote\_as | RoutingAutonomousSystem | True | one | Attribute | ### BGPSession[​](#bgpsession "Direct link to BGPSession") * **Label:** BGP Session * **Description:** A BGP Session represent a point to point connection between two routers * **Namespace:** Routing * **Icon:** mdi :router * **Inherit From:** RoutingProtocol #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ------------------------------------------- | -------- | -------- | -------------- | --------------------------- | | import\_policies | | Text | True | | | | export\_policies | | Text | True | | | | session\_type | Type of BGP Session | Dropdown | False | | external, internal | | role | Role of the BGP Session | Dropdown | True | | backbone, upstream, peering | | local\_pref | Force Local Pref for this BGP Peer Session. | Number | True | | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | ----------------------- | -------- | ----------- | --------- | | local\_as | RoutingAutonomousSystem | True | one | Attribute | | remote\_as | RoutingAutonomousSystem | True | one | Attribute | | local\_ip | IpamIPAddress | True | one | Attribute | | remote\_ip | IpamIPAddress | True | one | Attribute | | device | DcimDevice | True | one | Generic | | peer\_group | RoutingBGPPeerGroup | True | one | Attribute | | peer\_session | RoutingBGPSession | True | one | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### DcimGenericDevice[​](#dcimgenericdevice "Direct link to DcimGenericDevice") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | ----------------------- | -------- | ----------- | --------- | | asn | RoutingAutonomousSystem | True | one | Attribute | ### OrganizationGeneric[​](#organizationgeneric "Direct link to OrganizationGeneric") #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | ----------------------- | -------- | ----------- | ------- | | asn | RoutingAutonomousSystem | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: AutonomousSystem namespace: Routing description: An Autonomous System (AS) is a set of Internet routable IP prefixes belonging to a network label: Autonomous System icon: mdi:bank-circle-outline include_in_menu: true menu_placement: RoutingBGPSession human_friendly_id: - asn__value - name__value order_by: - asn__value - name__value display_label: 'AS {{ asn__value }}: {{ name__value }}' attributes: - name: name kind: Text unique: true optional: false description: Name of the Autonomous System order_weight: 1000 - name: asn kind: Number unique: true optional: false description: Autonomous System Number order_weight: 1050 - name: description kind: Text optional: true description: Description of the Autonomous System order_weight: 1100 relationships: - name: organization peer: OrganizationGeneric kind: Attribute cardinality: one optional: false order_weight: 1200 - name: location peer: LocationGeneric kind: Attribute cardinality: one optional: true order_weight: 1300 - name: devices peer: DcimDevice kind: Attribute cardinality: many optional: true order_weight: 1400 - name: BGPPeerGroup namespace: Routing description: A BGP Peer Group is used to regroup parameters that are shared across multiple peers label: BGP Peer Group icon: mdi:view-grid-plus-outline include_in_menu: true menu_placement: RoutingBGPSession inherit_from: - RoutingProtocol human_friendly_id: - name__value - description__value order_by: - name__value display_label: '{{ name__value }} - {{ description__value }}' attributes: - name: name kind: Text unique: true optional: false description: Name of the BGP Group order_weight: 1000 - name: import_policies kind: Text optional: true order_weight: 1300 - name: export_policies kind: Text optional: true order_weight: 1350 - name: maximum_routes kind: Number parameters: min_value: 0 optional: true description: Maximum routes for the BGP Group. order_weight: 1400 - name: local_pref kind: Number parameters: min_value: 0 optional: true description: Force Local Pref for this BGP Peer Group. order_weight: 1450 - name: send_community kind: Checkbox optional: true description: Whether to send community attributes. order_weight: 1500 - name: address_family kind: Dropdown default_value: ipv4 choices: - name: ipv4 label: IPv4 description: Policy applies to IPv4 addresses. color: '#E6E6FA' - name: ipv6 label: IPv6 description: Policy applies to IPv6 addresses. color: '#E6E6FA' optional: false description: The address family for the routing policy indicating the type of IP address. order_weight: 1250 relationships: - name: local_as peer: RoutingAutonomousSystem kind: Attribute cardinality: one optional: true identifier: bgppeergroup__local_as order_weight: 1550 - name: remote_as peer: RoutingAutonomousSystem kind: Attribute cardinality: one optional: true identifier: bgppeergroup__remote_as order_weight: 1600 - name: BGPSession namespace: Routing description: A BGP Session represent a point to point connection between two routers label: BGP Session icon: mdi:router include_in_menu: true inherit_from: - RoutingProtocol order_by: - remote_as__asn__value display_label: description__value attributes: - name: import_policies kind: Text optional: true order_weight: 1700 - name: export_policies kind: Text optional: true order_weight: 1750 - name: session_type kind: Dropdown choices: - name: external label: External description: Session established with a router in a different autonomous system. - name: internal label: Internal description: Session established with a router in the same autonomous system. optional: false description: Type of BGP Session order_weight: 1250 - name: role kind: Dropdown choices: - name: backbone label: Backbone description: Provide main data routes. color: '#E6E6FA' - name: upstream label: Upstream description: Connect to Internet service provider. color: '#E6E6FA' - name: peering label: Peering description: Connect with other networks via IX. color: '#E6E6FA' optional: true description: Role of the BGP Session order_weight: 1300 - name: local_pref kind: Number parameters: min_value: 0 optional: true description: Force Local Pref for this BGP Peer Session. order_weight: 1650 relationships: - name: local_as peer: RoutingAutonomousSystem kind: Attribute cardinality: one optional: true identifier: bgpsession__local_as order_weight: 1350 - name: remote_as peer: RoutingAutonomousSystem kind: Attribute cardinality: one optional: true identifier: bgpsession__remote_as order_weight: 1400 - name: local_ip peer: IpamIPAddress kind: Attribute cardinality: one optional: true identifier: bgpsession__local_ip order_weight: 1450 - name: remote_ip peer: IpamIPAddress kind: Attribute cardinality: one optional: true identifier: bgpsession__remote_ip order_weight: 1500 - name: device peer: DcimDevice kind: Generic cardinality: one optional: true order_weight: 900 - name: peer_group peer: RoutingBGPPeerGroup kind: Attribute cardinality: one optional: true order_weight: 1550 - name: peer_session peer: RoutingBGPSession kind: Attribute cardinality: one optional: true order_weight: 1600 extensions: nodes: - kind: DcimGenericDevice relationships: - name: asn peer: RoutingAutonomousSystem kind: Attribute cardinality: one optional: true order_weight: 1800 - kind: OrganizationGeneric relationships: - name: asn peer: RoutingAutonomousSystem label: Autonomous System kind: Generic cardinality: many optional: true order_weight: 1700 ``` --- # Routing BGP Community This schema extension adds a BGP Community node to reference from routing policies. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing\_policies](/schema-library/reference/routing_policies.md) ## Nodes[​](#nodes "Direct link to Nodes") ### BGPCommunity[​](#bgpcommunity "Direct link to BGPCommunity") * **Label:** BGP Community * **Description:** Defines a BGP community. * **Namespace:** Routing * **Icon:** iconoir :community * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------------------------------------------------------- | ---- | -------- | -------------- | ------- | | name | The name of the BGP community. | Text | False | | | | description | An optional description of the BGP community. | Text | True | | | | community | The value of the BGP community (RFC1997, RFC4360, RFC8092). | Text | False | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ------------- | -------- | ----------- | --------- | | routing\_policy | RoutingPolicy | True | many | Generic | | tags | BuiltinTag | True | many | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: BGPCommunity namespace: Routing description: Defines a BGP community. label: BGP Community icon: iconoir:community include_in_menu: true human_friendly_id: - name__value order_by: - name__value display_label: '{{ name__value }} ({{ community__value }})' attributes: - name: name kind: Text unique: true optional: false description: The name of the BGP community. order_weight: 1000 - name: description kind: Text optional: true description: An optional description of the BGP community. order_weight: 1100 - name: community kind: Text unique: true optional: false description: The value of the BGP community (RFC1997, RFC4360, RFC8092). order_weight: 1200 relationships: - name: routing_policy peer: RoutingPolicy label: Routing Policies kind: Generic cardinality: many optional: true description: The BGP Policies using this BGP Community. order_weight: 1300 - name: tags peer: BuiltinTag kind: Attribute cardinality: many optional: true order_weight: 2000 ``` --- # Routing BGP Route Reflector This schema extension adds a Route Reflector Cluster node on top of the BGP extension and links it to BGP peer groups. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing\_bgp](/schema-library/reference/routing_bgp.md) ## Nodes[​](#nodes "Direct link to Nodes") ### BGPRRCluster[​](#bgprrcluster "Direct link to BGPRRCluster") * **Label:** Route Reflector Cluster * **Description:** A Route Reflector (RR) Cluster used for grouping internal peers * **Namespace:** Routing * **Icon:** mdi :router-network * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | --------------------------------------------------- | ---- | -------- | -------------- | ------- | | name | Name of the Route Reflector Cluster | Text | False | | | | description | Optional description of the Route Reflector Cluster | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ------------------- | -------- | ----------- | --------- | | cluster\_id | IpamIPAddress | False | one | Attribute | | peer\_groups | RoutingBGPPeerGroup | True | many | Generic | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### RoutingBGPPeerGroup[​](#routingbgppeergroup "Direct link to RoutingBGPPeerGroup") #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------- | ------------------- | -------- | ----------- | --------- | | rr\_cluster | RoutingBGPRRCluster | True | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: BGPRRCluster namespace: Routing description: A Route Reflector (RR) Cluster used for grouping internal peers label: Route Reflector Cluster icon: mdi:router-network include_in_menu: true menu_placement: RoutingBGPSession human_friendly_id: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false description: Name of the Route Reflector Cluster order_weight: 1000 - name: description kind: Text optional: true description: Optional description of the Route Reflector Cluster order_weight: 1100 relationships: - name: cluster_id peer: IpamIPAddress label: Cluster ID kind: Attribute cardinality: one optional: false description: Cluster ID represented as a reference to an IP Address order_weight: 1200 - name: peer_groups peer: RoutingBGPPeerGroup label: BGP Peer Groups kind: Generic cardinality: many optional: true order_weight: 1300 extensions: nodes: - kind: RoutingBGPPeerGroup relationships: - name: rr_cluster peer: RoutingBGPRRCluster label: RR Cluster kind: Attribute cardinality: one optional: true order_weight: 1700 ``` --- # Routing OSPF This schema extension adds OSPF and OSPF Interface nodes to model an OSPF instance per device and VRF and the interfaces it runs on. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing](/schema-library/reference/routing.md) ## Nodes[​](#nodes "Direct link to Nodes") ### OSPF[​](#ospf "Direct link to OSPF") * **Label:** OSPF * **Description:** OSPF (Open Shortest Path First) instance on a Virtual Router. * **Namespace:** Routing * **Icon:** mdi :network-outline * **Uniqueness Constraints:** * device, vrf, version\_\_value * **Human Friendly ID:** device\_\_name\_\_value, vrf\_\_name\_\_value, version\_\_value * **Inherit From:** RoutingProtocol #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------------- | ------------------------------------------------ | -------- | -------- | -------------- | ------------ | | reference\_bandwidth | Reference bandwidth for OSPF instance (in Mbps). | Number | True | 1000 | | | version | Version of the OSPF protocol. | Dropdown | False | ospf | ospf, ospfv3 | | import\_policies | | Text | True | | | | export\_policies | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | -------------------- | -------- | ----------- | --------- | | router\_id | IpamIPAddress | True | one | Attribute | | ospf\_interfaces | RoutingOSPFInterface | True | many | Component | ### OSPFInterface[​](#ospfinterface "Direct link to OSPFInterface") * **Label:** OSPF Interface * **Description:** Pivot table linking OSPF configuration to an interface. * **Namespace:** Routing * **Icon:** mdi :ethernet * **Uniqueness Constraints:** * ospf, interface * **Human Friendly ID:** description\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------------- | ------------------------------------------------------------------------------------------- | -------- | -------- | -------------- | ------------------------------- | | description | Description of the OSPF interface. | Text | False | | | | metric | OSPF metric for the interface. | Number | True | | | | mode | Mode of the OSPF interface. | Dropdown | False | normal | normal, passive, peer\_to\_peer | | authentication\_key | Shared secret used to authenticate and secure routing messages between neighboring routers. | Password | True | | | | authentication\_mode | | Dropdown | True | | md5, sha1 | | area | OSPF area associated with the interface. | Text | False | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | --------------- | -------- | ----------- | --------- | | ospf | RoutingOSPF | False | one | Parent | | interface | InterfaceLayer3 | False | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: OSPF namespace: Routing description: OSPF (Open Shortest Path First) instance on a Virtual Router. label: OSPF icon: mdi:network-outline include_in_menu: true menu_placement: RoutingProtocol inherit_from: - RoutingProtocol human_friendly_id: - device__name__value - vrf__name__value - version__value order_by: - router_id__address__value - vrf__name__value - device__name__value display_label: description__value uniqueness_constraints: - - device - vrf - version__value attributes: - name: reference_bandwidth kind: Number default_value: 1000 optional: true description: Reference bandwidth for OSPF instance (in Mbps). order_weight: 1350 - name: version kind: Dropdown default_value: ospf choices: - name: ospf label: OSPFv2 description: Open Shortest Path First version 2. color: '#E6E6FA' - name: ospfv3 label: OSPFv3 description: Open Shortest Path First version 3. color: '#E6E6FA' optional: false description: Version of the OSPF protocol. order_weight: 1300 - name: import_policies kind: Text optional: true order_weight: 1450 - name: export_policies kind: Text optional: true order_weight: 1500 relationships: - name: router_id peer: IpamIPAddress kind: Attribute cardinality: one optional: true order_weight: 1400 - name: ospf_interfaces peer: RoutingOSPFInterface label: OSPF Interfaces kind: Component cardinality: many optional: true identifier: ospf__ospfinterfaces order_weight: 1550 - name: OSPFInterface namespace: Routing description: Pivot table linking OSPF configuration to an interface. label: OSPF Interface icon: mdi:ethernet include_in_menu: true menu_placement: RoutingOSPF human_friendly_id: - description__value order_by: - description__value display_label: description__value uniqueness_constraints: - - ospf - interface attributes: - name: description kind: Text unique: true optional: false description: Description of the OSPF interface. order_weight: 1100 - name: metric kind: Number optional: true description: OSPF metric for the interface. order_weight: 1400 - name: mode kind: Dropdown default_value: normal choices: - name: normal label: Normal description: Standard OSPF interface mode. color: '#E6E6FA' - name: passive label: Passive description: Interface will not send OSPF hello packets. color: '#E6E6FA' - name: peer_to_peer label: Peer-to-Peer description: OSPF peer-to-peer interface mode. color: '#E6E6FA' optional: false description: Mode of the OSPF interface. order_weight: 1300 - name: authentication_key kind: Password optional: true description: Shared secret used to authenticate and secure routing messages between neighboring routers. order_weight: 1250 - name: authentication_mode kind: Dropdown choices: - name: md5 label: MD5 color: '#E6E6FA' - name: sha1 label: SHA1 color: '#E6E6FA' optional: true order_weight: 1225 - name: area kind: Text optional: false description: OSPF area associated with the interface. order_weight: 1200 relationships: - name: ospf peer: RoutingOSPF label: OSPF kind: Parent cardinality: one optional: false identifier: ospf__ospfinterfaces order_weight: 900 - name: interface peer: InterfaceLayer3 kind: Attribute cardinality: one optional: false order_weight: 1500 ``` --- # Routing PIM This schema extension adds PIM (Protocol Independent Multicast) and PIM Interface nodes to model a PIM instance per device and VRF and the interfaces it runs on. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing](/schema-library/reference/routing.md) ## Nodes[​](#nodes "Direct link to Nodes") ### PIM[​](#pim "Direct link to PIM") * **Label:** PIM * **Description:** Protocol Independent Multicast (PIM) instance on a Virtual Router. * **Namespace:** Routing * **Icon:** mdi :network-outline * **Uniqueness Constraints:** * device, vrf * **Human Friendly ID:** device\_\_name\_\_value, vrf\_\_name\_\_value * **Inherit From:** RoutingProtocol #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | --------------------------- | ------ | -------- | -------------- | ------- | | dr\_priority | Designated Router priority. | Number | True | 1 | | | import\_policies | | Text | True | | | | export\_policies | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ------------------- | -------- | ----------- | --------- | | rp\_address | IpamIPAddress | True | one | Attribute | | pim\_interfaces | RoutingPIMInterface | True | many | Component | ### PIMInterface[​](#piminterface "Direct link to PIMInterface") * **Label:** PIM Interface * **Description:** Interface configuration for PIM. * **Namespace:** Routing * **Icon:** mdi :ethernet * **Uniqueness Constraints:** * pim, interface * **Human Friendly ID:** description\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------------- | ------------------------------------------------------ | -------- | -------- | -------------- | ---------------------------- | | description | Description of the PIM interface. | Text | False | | | | pim\_mode | PIM mode used for multicast routing on this interface. | Dropdown | False | | sparse, dense, bidirectional | | hello\_interval | Interval for PIM hello messages (in seconds). | Number | True | 30 | | | dr\_priority | Designated Router priority on the interface. | Number | True | 1 | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | --------------- | -------- | ----------- | --------- | | pim | RoutingPIM | False | one | Parent | | interface | InterfaceLayer3 | False | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: PIM namespace: Routing description: Protocol Independent Multicast (PIM) instance on a Virtual Router. label: PIM icon: mdi:network-outline include_in_menu: true menu_placement: RoutingProtocol inherit_from: - RoutingProtocol human_friendly_id: - device__name__value - vrf__name__value order_by: - vrf__name__value - device__name__value display_label: description__value uniqueness_constraints: - - device - vrf attributes: - name: dr_priority kind: Number default_value: 1 optional: true description: Designated Router priority. order_weight: 1250 - name: import_policies kind: Text optional: true order_weight: 1300 - name: export_policies kind: Text optional: true order_weight: 1350 relationships: - name: rp_address peer: IpamIPAddress kind: Attribute cardinality: one optional: true description: Rendezvous Point (RP) address for PIM. order_weight: 1400 - name: pim_interfaces peer: RoutingPIMInterface label: PIM Interfaces kind: Component cardinality: many optional: true identifier: pim__piminterfaces order_weight: 1450 - name: PIMInterface namespace: Routing description: Interface configuration for PIM. label: PIM Interface icon: mdi:ethernet include_in_menu: true menu_placement: RoutingPIM human_friendly_id: - description__value order_by: - description__value display_label: description__value uniqueness_constraints: - - pim - interface attributes: - name: description kind: Text unique: true optional: false description: Description of the PIM interface. order_weight: 1100 - name: pim_mode kind: Dropdown choices: - name: sparse label: Sparse Mode description: Sparse Mode for efficient multicast forwarding. color: '#E6E6FA' - name: dense label: Dense Mode description: Dense Mode for heavy multicast traffic. color: '#E6E6FA' - name: bidirectional label: Bidirectional Mode description: Bidirectional PIM for efficient traffic forwarding. color: '#E6E6FA' optional: false description: PIM mode used for multicast routing on this interface. order_weight: 1150 - name: hello_interval kind: Number default_value: 30 optional: true description: Interval for PIM hello messages (in seconds). order_weight: 1300 - name: dr_priority kind: Number default_value: 1 optional: true description: Designated Router priority on the interface. order_weight: 1250 relationships: - name: pim peer: RoutingPIM label: PIM kind: Parent cardinality: one optional: false identifier: pim__piminterfaces order_weight: 900 - name: interface peer: InterfaceLayer3 kind: Attribute cardinality: one optional: false order_weight: 1200 ``` --- # Routing Policies This schema extension contains the Routing Policy generic. Each routing protocol extension subclasses it with its own policy flavour. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Generics[​](#generics "Direct link to Generics") ### Policy[​](#policy "Direct link to Policy") * **Label:** Routing Policy * **Description:** Policy defining the rules for routing traffic in a network. * **Namespace:** Routing * **Icon:** carbon :deployment-policy * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------------- | -------------------------------------------------------------------------------- | -------- | -------- | -------------- | ------------------------------------------------------ | | name | The name of the routing policy. | Text | False | | | | description | An optional description of the routing policy. | Text | True | | | | policy\_type | The type of routing policy which specifies the direction of route advertisement. | Dropdown | False | | import\_policy, export\_policy, import\_export\_policy | | weight | Priority of the routing policy. The higher the number, the higher the priority. | Number | True | 1000 | | | address\_family | The address family for the routing policy indicating the type of IP address. | Dropdown | False | all | ipv4, ipv6, all | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: Policy namespace: Routing description: Policy defining the rules for routing traffic in a network. label: Routing Policy icon: carbon:deployment-policy include_in_menu: true human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false description: The name of the routing policy. order_weight: 1000 - name: description kind: Text optional: true description: An optional description of the routing policy. order_weight: 1100 - name: policy_type kind: Dropdown label: Type choices: - name: import_policy label: Import description: Policy for incoming routes. color: '#E6E6FA' - name: export_policy label: Export description: Policy for outgoing routes. color: '#E6E6FA' - name: import_export_policy label: Import + Export description: Policy for both incoming and outgoing routes. color: '#E6E6FA' optional: false description: The type of routing policy which specifies the direction of route advertisement. order_weight: 1200 - name: weight kind: Number default_value: 1000 optional: true description: Priority of the routing policy. The higher the number, the higher the priority. order_weight: 1400 - name: address_family kind: Dropdown default_value: all choices: - name: ipv4 label: IPv4 description: Policy applies to IPv4 addresses. color: '#E6E6FA' - name: ipv6 label: IPv6 description: Policy applies to IPv6 addresses. color: '#E6E6FA' - name: all label: All description: Policy applies to both IPv4 and IPv6 addresses. color: '#E6E6FA' optional: false description: The address family for the routing policy indicating the type of IP address. order_weight: 1150 ``` --- # Routing Policies Aggregate This schema extension adds an Aggregate Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on Aggregate Route with relationships to it. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing\_policies](/schema-library/reference/routing_policies.md) * [extensions/routing\_aggregate](/schema-library/reference/routing_aggregate.md) ## Nodes[​](#nodes "Direct link to Nodes") ### PolicyAggregate[​](#policyaggregate "Direct link to PolicyAggregate") * **Label:** Aggregate Routing Policies * **Description:** A routing policy for Aggregate. * **Namespace:** Routing * **Icon:** carbon :deployment-policy * **Inherit From:** RoutingPolicy ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: PolicyAggregate namespace: Routing description: A routing policy for Aggregate. label: Aggregate Routing Policies icon: carbon:deployment-policy include_in_menu: true menu_placement: RoutingPolicy inherit_from: - RoutingPolicy ``` --- # Routing Policies BGP This schema extension adds a BGP Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on BGP Peer Group and BGP Session with relationships to it. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing\_policies](/schema-library/reference/routing_policies.md) * [extensions/routing\_bgp](/schema-library/reference/routing_bgp.md) ## Nodes[​](#nodes "Direct link to Nodes") ### PolicyBGP[​](#policybgp "Direct link to PolicyBGP") * **Label:** BGP Routing Policies * **Description:** A routing policy for BGP. * **Namespace:** Routing * **Icon:** carbon :deployment-policy * **Inherit From:** RoutingPolicy ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### RoutingBGPPeerGroup[​](#routingbgppeergroup "Direct link to RoutingBGPPeerGroup") #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ----------- | ---- | -------- | -------------- | ------- | | import\_policies | | Text | | | | | export\_policies | | Text | | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------------- | ---------------- | -------- | ----------- | ------- | | import\_routing\_policies | RoutingPolicyBGP | True | many | Generic | | export\_routing\_policies | RoutingPolicyBGP | True | many | Generic | ### RoutingBGPSession[​](#routingbgpsession "Direct link to RoutingBGPSession") #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ----------- | ---- | -------- | -------------- | ------- | | import\_policies | | Text | | | | | export\_policies | | Text | | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------------- | ---------------- | -------- | ----------- | ------- | | import\_routing\_policies | RoutingPolicyBGP | True | many | Generic | | export\_routing\_policies | RoutingPolicyBGP | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: PolicyBGP namespace: Routing description: A routing policy for BGP. label: BGP Routing Policies icon: carbon:deployment-policy include_in_menu: true menu_placement: RoutingPolicy inherit_from: - RoutingPolicy extensions: nodes: - kind: RoutingBGPPeerGroup attributes: - name: import_policies kind: Text state: absent - name: export_policies kind: Text state: absent relationships: - name: import_routing_policies peer: RoutingPolicyBGP label: Import Routing Policies kind: Generic cardinality: many optional: true identifier: bgppeergroup__import_policies description: The routing-policies used by this instance for import. order_weight: 1750 - name: export_routing_policies peer: RoutingPolicyBGP label: Export Routing Policies kind: Generic cardinality: many optional: true identifier: bgppeergroup__export_policies description: The routing-policies used by this instance for export. order_weight: 1800 - kind: RoutingBGPSession attributes: - name: import_policies kind: Text state: absent - name: export_policies kind: Text state: absent relationships: - name: import_routing_policies peer: RoutingPolicyBGP label: Import Routing Policies kind: Generic cardinality: many optional: true identifier: bgpsession__import_policies description: The routing-policies used by this instance for import. order_weight: 1850 - name: export_routing_policies peer: RoutingPolicyBGP label: Export Routing Policies kind: Generic cardinality: many optional: true identifier: bgpsession__export_policies description: The routing-policies used by this instance for export. order_weight: 1900 ``` --- # Routing Policies OSPF This schema extension adds an OSPF Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on OSPF with relationships to it. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing\_policies](/schema-library/reference/routing_policies.md) * [extensions/routing\_ospf](/schema-library/reference/routing_ospf.md) ## Nodes[​](#nodes "Direct link to Nodes") ### PolicyOSPF[​](#policyospf "Direct link to PolicyOSPF") * **Label:** OSPF Routing Policies * **Description:** A routing policy for OSPF. * **Namespace:** Routing * **Icon:** carbon :deployment-policy * **Inherit From:** RoutingPolicy ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### RoutingOSPF[​](#routingospf "Direct link to RoutingOSPF") #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ----------- | ---- | -------- | -------------- | ------- | | import\_policies | | Text | | | | | export\_policies | | Text | | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------------- | ----------------- | -------- | ----------- | ------- | | import\_routing\_policies | RoutingPolicyOSPF | True | many | Generic | | export\_routing\_policies | RoutingPolicyOSPF | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: PolicyOSPF namespace: Routing description: A routing policy for OSPF. label: OSPF Routing Policies icon: carbon:deployment-policy include_in_menu: true menu_placement: RoutingPolicy inherit_from: - RoutingPolicy extensions: nodes: - kind: RoutingOSPF attributes: - name: import_policies kind: Text state: absent - name: export_policies kind: Text state: absent relationships: - name: import_routing_policies peer: RoutingPolicyOSPF label: Import Routing Policies kind: Generic cardinality: many optional: true identifier: ospf__import_policies description: The routing-policies used by this instance for import. order_weight: 1600 - name: export_routing_policies peer: RoutingPolicyOSPF label: Export Routing Policies kind: Generic cardinality: many optional: true identifier: ospf__export_policies description: The routing-policies used by this instance for export. order_weight: 1650 ``` --- # Routing Policies PIM This schema extension adds a PIM Routing Policy node, a flavour of the Routing Policy generic, and replaces the free-text import and export policies on PIM with relationships to it. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/routing\_policies](/schema-library/reference/routing_policies.md) * [extensions/routing\_pim](/schema-library/reference/routing_pim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### PolicyPIM[​](#policypim "Direct link to PolicyPIM") * **Label:** PIM Routing Policies * **Description:** A routing policy for PIM. * **Namespace:** Routing * **Icon:** carbon :deployment-policy * **Inherit From:** RoutingPolicy ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### RoutingPIM[​](#routingpim "Direct link to RoutingPIM") #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------------- | ----------- | ---- | -------- | -------------- | ------- | | import\_policies | | Text | | | | | export\_policies | | Text | | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------------- | ---------------- | -------- | ----------- | ------- | | import\_routing\_policies | RoutingPolicyPIM | True | many | Generic | | export\_routing\_policies | RoutingPolicyPIM | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: PolicyPIM namespace: Routing description: A routing policy for PIM. label: PIM Routing Policies icon: carbon:deployment-policy include_in_menu: true menu_placement: RoutingPolicy inherit_from: - RoutingPolicy extensions: nodes: - kind: RoutingPIM attributes: - name: import_policies kind: Text state: absent - name: export_policies kind: Text state: absent relationships: - name: import_routing_policies peer: RoutingPolicyPIM label: Import Routing Policies kind: Generic cardinality: many optional: true identifier: pim__import_policies description: The routing-policies used by this instance for import. order_weight: 1500 - name: export_routing_policies peer: RoutingPolicyPIM label: Export Routing Policies kind: Generic cardinality: many optional: true identifier: pim__export_policies description: The routing-policies used by this instance for export. order_weight: 1550 ``` --- # Security This schema extension contains models for implementing detailed security. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Zone[​](#zone "Direct link to Zone") * **Label:** Security zone * **Description:** Security zones * **Namespace:** Security * **Icon:** game-icons :fire-zone * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------- | ------------------------- | -------- | ----------- | --------- | | interfaces | SecurityFirewallInterface | True | | Attribute | ### IPAMIPAddress[​](#ipamipaddress "Direct link to IPAMIPAddress") * **Label:** IPAM IP Address * **Description:** Infrahub IPv4/6 address * **Namespace:** Security * **Icon:** mdi :ip-outline * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericAddress #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | description | | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------- | ------------- | -------- | ----------- | --------- | | ip\_address | IpamIPAddress | False | one | Attribute | ### IPAMIPPrefix[​](#ipamipprefix "Direct link to IPAMIPPrefix") * **Label:** IPAM IP Prefix * **Description:** Infrahub IPv4/6 prefix * **Namespace:** Security * **Icon:** mdi :ip-network-outline * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericAddress #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | description | | Text | True | | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------- | ---------- | -------- | ----------- | --------- | | ip\_prefix | IpamPrefix | False | one | Attribute | ### IPAddress[​](#ipaddress "Direct link to IPAddress") * **Label:** IP Address * **Description:** IPv4/6 address * **Namespace:** Security * **Icon:** mdi :ip-outline * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericAddress #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ------ | -------- | -------------- | ------- | | address | | IPHost | | | | | description | | Text | True | | | ### Prefix[​](#prefix "Direct link to Prefix") * **Label:** Prefix * **Description:** IPv4/6 prefix * **Namespace:** Security * **Icon:** mdi :ip-network-outline * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericAddress #### Attributes[​](#attributes-4 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | --------- | -------- | -------------- | ------- | | prefix | | IPNetwork | False | | | | description | | Text | True | | | ### IPRange[​](#iprange "Direct link to IPRange") * **Label:** IP Range * **Description:** IPv4/6 Range * **Namespace:** Security * **Icon:** mdi :ip-outline * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericAddress #### Attributes[​](#attributes-5 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----- | ----------- | ------ | -------- | -------------- | ------- | | start | | IPHost | False | | | | end | | IPHost | False | | | ### FQDN[​](#fqdn "Direct link to FQDN") * **Label:** FQDN * **Description:** Full Qualified Domain Name * **Namespace:** Security * **Icon:** eos-icons :dns * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericAddress #### Attributes[​](#attributes-6 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | fqdn | | Text | False | | | ### AddressGroup[​](#addressgroup "Direct link to AddressGroup") * **Label:** Address Group * **Description:** Group of addresses * **Namespace:** Security * **Icon:** material-symbols :menu-book-outline-rounded * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericAddressGroup ### IPProtocol[​](#ipprotocol "Direct link to IPProtocol") * **Label:** IP Protocols * **Description:** IP protocol * **Namespace:** Security * **Icon:** mdi :protocol * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericService #### Attributes[​](#attributes-7 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------- | ----------- | ------ | -------- | -------------- | ------- | | protocol | | Number | True | | | ### Service[​](#service "Direct link to Service") * **Label:** Service * **Description:** Service * **Namespace:** Security * **Icon:** eos-icons :application-outlined * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericService #### Attributes[​](#attributes-8 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ------ | -------- | -------------- | ------- | | port | | Number | | | | #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ------------------ | -------- | ----------- | --------- | | ip\_protocol | SecurityIPProtocol | True | one | Attribute | ### ServiceRange[​](#servicerange "Direct link to ServiceRange") * **Label:** Service range * **Description:** Service range * **Namespace:** Security * **Icon:** eos-icons :application-outlined * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericService #### Attributes[​](#attributes-9 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----- | ----------- | ------ | -------- | -------------- | ------- | | start | | Number | False | | | | end | | Number | False | | | #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ------------------ | -------- | ----------- | --------- | | ip\_protocol | SecurityIPProtocol | False | one | Attribute | ### ServiceGroup[​](#servicegroup "Direct link to ServiceGroup") * **Label:** Service group * **Description:** Group of services * **Namespace:** Security * **Icon:** material-symbols :menu-book-outline-rounded * **Human Friendly ID:** name\_\_value * **Inherit From:** SecurityGenericServiceGroup ### Policy[​](#policy "Direct link to Policy") * **Label:** Security Policy * **Namespace:** Security * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-10 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships-5 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | ------------------ | -------- | ----------- | --------- | | rules | SecurityPolicyRule | | many | Component | | location\_target | LocationGeneric | True | one | Attribute | | device\_target | SecurityFirewall | True | one | Attribute | ### PolicyRule[​](#policyrule "Direct link to PolicyRule") * **Label:** Policy rule * **Description:** Policy rule * **Namespace:** Security * **Icon:** material-symbols :policy * **Uniqueness Constraints:** * index\_\_value, source\_zone, destination\_zone, policy #### Attributes[​](#attributes-11 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------ | ----------- | ------- | -------- | -------------- | ------- | | index | | Number | False | | | | name | | Text | False | | | | action | | Text | False | permit | | | log | | Boolean | True | False | | #### Relationships[​](#relationships-6 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------------------- | --------------------------- | -------- | ----------- | --------- | | policy | SecurityPolicy | False | one | Attribute | | source\_zone | SecurityZone | False | one | Attribute | | destination\_zone | SecurityZone | False | one | Attribute | | source\_address | SecurityGenericAddress | True | many | Attribute | | source\_groups | SecurityGenericAddressGroup | True | many | Attribute | | source\_services | SecurityGenericService | True | many | Attribute | | source\_service\_groups | SecurityGenericServiceGroup | True | many | Attribute | | destination\_address | SecurityGenericAddress | True | many | Attribute | | destination\_groups | SecurityGenericAddressGroup | True | many | Attribute | | destination\_services | SecurityGenericService | True | many | Attribute | | destination\_service\_groups | SecurityGenericServiceGroup | True | many | Attribute | ### Firewall[​](#firewall "Direct link to Firewall") * **Namespace:** Security * **Icon:** mdi :firewall * **Human Friendly ID:** name\_\_value * **Inherit From:** DcimGenericDevice, DcimPhysicalDevice, CoreArtifactTarget, SecurityPolicyAssignment #### Attributes[​](#attributes-12 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | -------- | -------- | -------------- | -------------- | | role | | Dropdown | True | | edge\_firewall | #### Relationships[​](#relationships-7 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | -------------- | -------- | ----------- | --------- | | policy | SecurityPolicy | | one | Attribute | ### RenderedPolicyRule[​](#renderedpolicyrule "Direct link to RenderedPolicyRule") * **Label:** Policy rule * **Description:** Policy rule * **Namespace:** Security #### Attributes[​](#attributes-13 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------ | ----------- | ------- | -------- | -------------- | ------- | | index | | Number | False | | | | name | | Text | False | | | | action | | Text | False | permit | | | log | | Boolean | True | False | | #### Relationships[​](#relationships-8 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------------------- | --------------------------- | -------- | ----------- | --------- | | source\_policy | SecurityPolicy | False | one | Attribute | | source\_zone | SecurityZone | False | one | Attribute | | destination\_zone | SecurityZone | False | one | Attribute | | source\_address | SecurityGenericAddress | True | many | Attribute | | source\_groups | SecurityGenericAddressGroup | True | many | Attribute | | source\_services | SecurityGenericService | True | many | Attribute | | source\_service\_groups | SecurityGenericServiceGroup | True | many | Attribute | | destination\_address | SecurityGenericAddress | True | many | Attribute | | destination\_groups | SecurityGenericAddressGroup | True | many | Attribute | | destination\_services | SecurityGenericService | True | many | Attribute | | destination\_service\_groups | SecurityGenericServiceGroup | True | many | Attribute | ### FirewallInterface[​](#firewallinterface "Direct link to FirewallInterface") * **Label:** Firewall Interface * **Namespace:** Security * **Icon:** mdi :ethernet * **Inherit From:** DcimInterface, DcimEndpoint #### Relationships[​](#relationships-9 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------- | ------------- | -------- | ----------- | --------- | | ip\_addresses | IpamIPAddress | True | many | Component | | security\_zone | SecurityZone | False | one | Attribute | ## Generics[​](#generics "Direct link to Generics") ### PolicyAssignment[​](#policyassignment "Direct link to PolicyAssignment") * **Label:** Security Policy * **Namespace:** Security #### Relationships[​](#relationships-10 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | -------------------------- | -------- | ----------- | --------- | | rules | SecurityRenderedPolicyRule | True | many | Component | ### GenericAddressGroup[​](#genericaddressgroup "Direct link to GenericAddressGroup") * **Namespace:** Security #### Attributes[​](#attributes-14 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships-11 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | ---------------------- | -------- | ----------- | --------- | | addresses | SecurityGenericAddress | True | many | Component | ### GenericAddress[​](#genericaddress "Direct link to GenericAddress") * **Namespace:** Security #### Attributes[​](#attributes-15 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | #### Relationships[​](#relationships-12 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | --------------------------- | -------- | ----------- | ---- | | address\_groups | SecurityGenericAddressGroup | True | many | | ### GenericServiceGroup[​](#genericservicegroup "Direct link to GenericServiceGroup") * **Namespace:** Security #### Attributes[​](#attributes-16 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships-13 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | ---------------------- | -------- | ----------- | --------- | | services | SecurityGenericService | True | many | Component | ### GenericService[​](#genericservice "Direct link to GenericService") * **Namespace:** Security #### Attributes[​](#attributes-17 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships-14 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | --------------------------- | -------- | ----------- | ---- | | service\_groups | SecurityGenericServiceGroup | True | many | | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### LocationGeneric[​](#locationgeneric "Direct link to LocationGeneric") #### Relationships[​](#relationships-15 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | -------------- | -------- | ----------- | --------- | | policy | SecurityPolicy | | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: PolicyAssignment namespace: Security label: Security Policy include_in_menu: false relationships: - name: rules label: Policy peer: SecurityRenderedPolicyRule kind: Component cardinality: many optional: true - name: GenericAddressGroup namespace: Security include_in_menu: false display_label: name__value hierarchical: true attributes: - name: name kind: Text label: Name optional: false unique: true - name: description label: Description kind: Text optional: true relationships: - name: addresses peer: SecurityGenericAddress cardinality: many kind: Component optional: true - name: GenericAddress namespace: Security display_label: name__value include_in_menu: false attributes: - name: name kind: Text optional: false unique: true relationships: - name: address_groups label: Address Groups peer: SecurityGenericAddressGroup cardinality: many optional: true - name: GenericServiceGroup namespace: Security include_in_menu: false hierarchical: true display_label: name__value attributes: - name: name label: Name kind: Text optional: false - name: description label: Description kind: Text optional: true relationships: - name: services peer: SecurityGenericService label: Services cardinality: many kind: Component optional: true - name: GenericService include_in_menu: false namespace: Security display_label: name__value attributes: - name: name label: Name kind: Text optional: false - name: description kind: Text label: Description optional: true relationships: - name: service_groups label: Service Groups peer: SecurityGenericServiceGroup cardinality: many optional: true nodes: - name: Zone namespace: Security menu_placement: SecurityPolicy human_friendly_id: - name__value display_label: name__value include_in_menu: true icon: game-icons:fire-zone description: Security zones label: Security zone attributes: - name: name kind: Text unique: true optional: false relationships: - name: interfaces label: Interfaces kind: Attribute optional: true peer: SecurityFirewallInterface - name: IPAMIPAddress namespace: Security menu_placement: SecurityPolicy include_in_menu: true icon: mdi:ip-outline description: Infrahub IPv4/6 address label: IPAM IP Address human_friendly_id: - name__value inherit_from: - SecurityGenericAddress attributes: - name: description kind: Text optional: true relationships: - name: ip_address peer: IpamIPAddress cardinality: one kind: Attribute optional: false - name: IPAMIPPrefix namespace: Security menu_placement: SecurityPolicy include_in_menu: true icon: mdi:ip-network-outline description: Infrahub IPv4/6 prefix label: IPAM IP Prefix human_friendly_id: - name__value inherit_from: - SecurityGenericAddress attributes: - name: description kind: Text optional: true relationships: - name: ip_prefix peer: IpamPrefix cardinality: one kind: Attribute optional: false - name: IPAddress namespace: Security menu_placement: SecurityPolicy include_in_menu: true description: IPv4/6 address human_friendly_id: - name__value label: IP Address icon: mdi:ip-outline inherit_from: - SecurityGenericAddress order_by: - address__value display_label: '{{ name__value }} ({{ address__value }})' attributes: - name: address kind: IPHost - name: description kind: Text optional: true - name: Prefix namespace: Security menu_placement: SecurityPolicy include_in_menu: true icon: mdi:ip-network-outline description: IPv4/6 prefix label: Prefix human_friendly_id: - name__value inherit_from: - SecurityGenericAddress order_by: - name__value display_label: '{{ name__value }} ({{ prefix__value }})' attributes: - name: prefix kind: IPNetwork optional: false unique: true - name: description kind: Text optional: true - name: IPRange namespace: Security menu_placement: SecurityPolicy include_in_menu: true icon: mdi:ip-outline description: IPv4/6 Range label: IP Range human_friendly_id: - name__value display_label: name__value inherit_from: - SecurityGenericAddress order_by: - name__value attributes: - name: start label: Start IP Address kind: IPHost optional: false - name: end label: End IP Address kind: IPHost optional: false - name: FQDN namespace: Security description: Full Qualified Domain Name include_in_menu: true icon: eos-icons:dns menu_placement: SecurityPolicy label: FQDN human_friendly_id: - name__value display_label: name__value inherit_from: - SecurityGenericAddress order_by: - name__value - fqdn__value attributes: - name: fqdn label: FQDN kind: Text optional: false regex: (?=^.{1,253}$)(^(((?!-)[a-zA-Z0-9-]{1,63}(? :community * **Human Friendly ID:** name\_\_value * **Inherit From:** SnmpCommunity #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------------- | ----------- | -------- | -------- | -------------- | ----------------------- | | community\_string | | Password | False | | | | access | | Dropdown | False | | read\_only, read\_write | ### CommunityV3[​](#communityv3 "Direct link to CommunityV3") * **Label:** SNMP v3 * **Description:** SNMP version 3 configuration with enhanced security. * **Namespace:** Snmp * **Icon:** iconoir :community * **Human Friendly ID:** name\_\_value * **Inherit From:** SnmpCommunity #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------------- | ----------- | -------- | -------- | -------------- | ---------------------------------------------- | | username | | Text | False | | | | auth\_protocol | | Dropdown | False | | none, md5, sha | | auth\_password | | Password | True | | | | privacy\_protocol | | Dropdown | False | | none, des, aes | | privacy\_password | | Password | True | | | | security\_level | | Dropdown | False | | no\_auth\_no\_priv, auth\_no\_priv, auth\_priv | ### Client[​](#client "Direct link to Client") * **Label:** SNMP Client * **Description:** Represents an SNMP client that interacts with SNMP Community. * **Namespace:** Snmp * **Icon:** ph :user-list-light * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | client\_description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | ------------- | -------- | ----------- | --------- | | ip\_addresses | IpamIPAddress | True | many | Attribute | | communities | SnmpCommunity | True | many | Attribute | ## Generics[​](#generics "Direct link to Generics") ### Community[​](#community "Direct link to Community") * **Label:** SNMP Community * **Description:** Generic model for SNMP community configurations. * **Namespace:** Snmp * **Icon:** iconoir :community * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------- | ----------------- | -------- | ----------- | --------- | | clients | SnmpClient | True | many | Component | | devices | DcimGenericDevice | True | many | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: Community namespace: Snmp description: Generic model for SNMP community configurations. label: SNMP Community icon: iconoir:community include_in_menu: true human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: clients peer: SnmpClient kind: Component cardinality: many optional: true order_weight: 1900 - name: devices peer: DcimGenericDevice label: Devices kind: Attribute cardinality: many optional: true identifier: snmp_community__device description: Devices this community is configured on. order_weight: 1800 nodes: - name: CommunityV2 namespace: Snmp description: SNMP v1/v2c community configuration. label: SNMP v1/v2c icon: iconoir:community include_in_menu: true menu_placement: SnmpCommunity inherit_from: - SnmpCommunity human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: community_string kind: Password optional: false order_weight: 1300 - name: access kind: Dropdown choices: - name: read_only label: Read-Only description: Read-only access. - name: read_write label: Read-Write description: Read-write access. optional: false order_weight: 1200 - name: CommunityV3 namespace: Snmp description: SNMP version 3 configuration with enhanced security. label: SNMP v3 icon: iconoir:community include_in_menu: true menu_placement: SnmpCommunity inherit_from: - SnmpCommunity human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: username kind: Text optional: false order_weight: 1300 - name: auth_protocol kind: Dropdown label: Authentication Protocol choices: - name: none label: None description: No authentication. - name: md5 label: MD5 description: MD5 authentication. - name: sha label: SHA description: SHA authentication. optional: false order_weight: 1400 - name: auth_password kind: Password label: Authentication Password optional: true order_weight: 1500 - name: privacy_protocol kind: Dropdown label: Privacy Protocol choices: - name: none label: None description: No privacy. - name: des label: DES description: DES encryption. - name: aes label: AES description: AES encryption. optional: false order_weight: 1600 - name: privacy_password kind: Password label: Privacy Password optional: true order_weight: 1700 - name: security_level kind: Dropdown label: Security Level choices: - name: no_auth_no_priv label: NoAuthNoPriv description: No authentication and no privacy. - name: auth_no_priv label: AuthNoPriv description: Authentication but no privacy. - name: auth_priv label: AuthPriv description: Both authentication and privacy. optional: false order_weight: 1200 - name: Client namespace: Snmp description: Represents an SNMP client that interacts with SNMP Community. label: SNMP Client icon: ph:user-list-light include_in_menu: true menu_placement: SnmpCommunity human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: client_description kind: Text label: Description optional: true order_weight: 1100 relationships: - name: ip_addresses peer: IpamIPAddress label: IP Addresses kind: Attribute cardinality: many optional: true description: Authorized SNMP client address. order_weight: 1150 - name: communities peer: SnmpCommunity kind: Attribute cardinality: many optional: true order_weight: 1200 ``` --- # Tenancy This schema extension introduces a Tenant node that can own devices, IP prefixes and addresses, and hosting locations. See the comment at the bottom of tenancy.yml for an example of how to extend tenancy onto optional extension nodes such as DcimCircuit. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Tenant[​](#tenant "Direct link to Tenant") * **Label:** Tenant * **Description:** A tenant is the owner of the corresponding devices, addressing, and locations. * **Namespace:** Organization * **Icon:** mdi :domain * **Inherit From:** OrganizationGeneric #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | ----------------- | -------- | ----------- | ------- | | devices | DcimGenericDevice | True | many | Generic | | prefixes | IpamPrefix | True | many | Generic | | addresses | IpamIPAddress | True | many | Generic | | locations | LocationHosting | True | many | Generic | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### DcimGenericDevice[​](#dcimgenericdevice "Direct link to DcimGenericDevice") #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | ------------------ | -------- | ----------- | --------- | | tenant | OrganizationTenant | True | one | Attribute | ### IpamPrefix[​](#ipamprefix "Direct link to IpamPrefix") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | ------------------ | -------- | ----------- | --------- | | tenant | OrganizationTenant | True | one | Attribute | ### IpamIPAddress[​](#ipamipaddress "Direct link to IpamIPAddress") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | ------------------ | -------- | ----------- | --------- | | tenant | OrganizationTenant | True | one | Attribute | ### LocationHosting[​](#locationhosting "Direct link to LocationHosting") #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------ | ------------------ | -------- | ----------- | --------- | | tenant | OrganizationTenant | True | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: Tenant namespace: Organization description: A tenant is the owner of the corresponding devices, addressing, and locations. label: Tenant icon: mdi:domain include_in_menu: true menu_placement: OrganizationGeneric inherit_from: - OrganizationGeneric relationships: - name: devices peer: DcimGenericDevice label: Devices kind: Generic cardinality: many optional: true identifier: tenant__device order_weight: 1300 - name: prefixes peer: IpamPrefix label: Prefixes kind: Generic cardinality: many optional: true identifier: tenant__prefix order_weight: 1350 - name: addresses peer: IpamIPAddress label: Addresses kind: Generic cardinality: many optional: true identifier: tenant__ip_address order_weight: 1400 - name: locations peer: LocationHosting label: Locations kind: Generic cardinality: many optional: true identifier: tenant__location order_weight: 1450 extensions: nodes: - kind: DcimGenericDevice relationships: - name: tenant peer: OrganizationTenant kind: Attribute cardinality: one optional: true identifier: tenant__device order_weight: 1850 - kind: IpamPrefix relationships: - name: tenant peer: OrganizationTenant kind: Attribute cardinality: one optional: true identifier: tenant__prefix order_weight: 1400 - kind: IpamIPAddress relationships: - name: tenant peer: OrganizationTenant kind: Attribute cardinality: one optional: true identifier: tenant__ip_address order_weight: 1350 - kind: LocationHosting relationships: - name: tenant peer: OrganizationTenant kind: Attribute cardinality: one optional: true identifier: tenant__location order_weight: 1800 ``` --- # Topology A schema for defining and managing network topology, strategies, and services. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) * [extensions/vlan](/schema-library/reference/vlan.md) ## Nodes[​](#nodes "Direct link to Nodes") ### Topology[​](#topology "Direct link to Topology") * **Label:** Topology * **Description:** A Topology represents the entire network pod. * **Namespace:** Topology * **Icon:** carbon :network-3 * **Human Friendly ID:** name\_\_value * **Inherit From:** CoreArtifactTarget #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | | description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------------- | ----------------------- | -------- | ----------- | --------- | | elements | TopologyGenericElement | True | many | Component | | strategy | TopologyNetworkStrategy | True | one | Component | | location | LocationGeneric | True | one | Attribute | | devices | DcimGenericDevice | True | many | Component | | network\_services | TopologyNetworkService | True | many | Component | ### MPLSStrategy[​](#mplsstrategy "Direct link to MPLSStrategy") * **Label:** MPLS Strategy * **Description:** Specific strategy attributes for MPLS. * **Namespace:** Topology * **Icon:** eos-icons :neural-network * **Human Friendly ID:** name\_\_value * **Inherit From:** TopologyNetworkStrategy #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------- | ----------- | -------- | -------- | -------------- | --------------------------- | | underlay | | Dropdown | | | ospf, isis, bgp | | overlay | | Dropdown | | | ldp, rsvp, segment\_routing | ### EVPNStrategy[​](#evpnstrategy "Direct link to EVPNStrategy") * **Label:** EVPN Strategy * **Description:** Specific strategy attributes for EVPN. * **Namespace:** Topology * **Icon:** carbon :load-balancer-network * **Human Friendly ID:** name\_\_value * **Inherit From:** TopologyNetworkStrategy #### Attributes[​](#attributes-2 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------- | ----------- | -------- | -------- | -------------- | ---------------- | | underlay | | Dropdown | | | ebgp, ospf, isis | | overlay | | Dropdown | | | ebgp, ibgp | ### PhysicalElement[​](#physicalelement "Direct link to PhysicalElement") * **Label:** Physical Topology Element * **Description:** Physical aspect of topology elements. * **Namespace:** Topology * **Icon:** carbon :network-3-reference * **Human Friendly ID:** name\_\_value * **Inherit From:** TopologyGenericElement #### Attributes[​](#attributes-3 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------- | ----------- | -------- | -------- | -------------- | ------------------------------------------------------------------- | | mtu | | Number | | 1500 | | | border | | Boolean | | False | | | mlag\_support | | Boolean | | False | | | device\_role | | Dropdown | | | spine, leaf, pe\_router, p\_router, route\_reflector, cpe, firewall | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | -------------- | -------- | ----------- | --------- | | device\_type | DcimDeviceType | True | one | Attribute | ### DhcpOption[​](#dhcpoption "Direct link to DhcpOption") * **Label:** Dhcp Option * **Description:** Represents a configurable option within a Dhcp server. * **Namespace:** Network * **Icon:** gis :globe-options * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-4 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------ | ----------- | ------ | -------- | -------------- | ------- | | name | | Text | | | | | description | | Text | True | | | | option\_code | | Number | | | | | content | | Text | | | | ### DhcpServer[​](#dhcpserver "Direct link to DhcpServer") * **Label:** Dhcp Server * **Description:** Represents a Dhcp server in the network. * **Namespace:** Network * **Icon:** eos-icons :ip * **Human Friendly ID:** name\_\_value * **Inherit From:** NetworkManagementServer #### Attributes[​](#attributes-5 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | lease\_time | | Text | | | | #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | ----------------- | -------- | ----------- | --------- | | dhcp\_options | NetworkDhcpOption | True | many | Component | ### NameServer[​](#nameserver "Direct link to NameServer") * **Label:** DNS Server * **Description:** Represents a DNS server in the network. * **Namespace:** Network * **Icon:** eos-icons :dns * **Human Friendly ID:** name\_\_value * **Inherit From:** NetworkManagementServer ### NTPServer[​](#ntpserver "Direct link to NTPServer") * **Label:** NTP Server * **Description:** Represents a NTP server in the network. * **Namespace:** Network * **Icon:** iconoir :time-zone * **Human Friendly ID:** name\_\_value * **Inherit From:** NetworkManagementServer ### NetworkServiceIdentifier[​](#networkserviceidentifier "Direct link to NetworkServiceIdentifier") * **Label:** Network Service Identifier * **Description:** Generic model for different types of identifiers used in network services. * **Namespace:** Topology * **Icon:** mdi :identifier * **Human Friendly ID:** identifier\_\_value #### Attributes[​](#attributes-6 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ---------- | ----------- | ------ | -------- | -------------- | ------- | | identifier | | Number | | | | #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------- | ---------------------- | -------- | ----------- | ---- | | service | TopologyNetworkService | True | one | | ### NetworkService[​](#networkservice "Direct link to NetworkService") * **Label:** Network Service * **Description:** Network services attached to a Topology. * **Namespace:** Topology * **Icon:** carbon :ibm-cloud-internet-services * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-7 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | | description | | Text | | | | | service\_type | | Text | | | | #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------- | -------------------------------- | -------- | ----------- | --------- | | identifier | TopologyNetworkServiceIdentifier | False | one | Attribute | | topology | TopologyTopology | False | one | Parent | | vlan | IpamVLAN | True | one | Component | | prefix | IpamPrefix | True | one | Component | ## Generics[​](#generics "Direct link to Generics") ### ManagementServer[​](#managementserver "Direct link to ManagementServer") * **Label:** Network Management Servers * **Description:** Generic model for network management server (dns, ntp, and dhcp). * **Namespace:** Network * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-8 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | -------- | -------- | -------------- | ------------------------------------------ | | name | | Text | | | | | description | | Text | True | | | | status | | Dropdown | | | active, provisioning, maintenance, drained | #### Relationships[​](#relationships-5 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | --------------- | -------- | ----------- | --------- | | location | LocationGeneric | True | many | | | ip\_addresses | IpamIPAddress | True | many | Component | ### GenericElement[​](#genericelement "Direct link to GenericElement") * **Label:** Generic Topology Element * **Description:** Base model for elements * **Namespace:** Topology * **Icon:** carbon :network-3-reference * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-9 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ------ | -------- | -------------- | ------- | | name | | Text | | | | | description | | Text | True | | | | quantity | | Number | | | | #### Relationships[​](#relationships-6 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | ---------------- | -------- | ----------- | ------ | | topology | TopologyTopology | False | one | Parent | ### NetworkStrategy[​](#networkstrategy "Direct link to NetworkStrategy") * **Label:** Network Strategy * **Description:** Generic model for network strategies (underlays and overlays). * **Namespace:** Topology * **Icon:** iconoir :strategy * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-10 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | | | | | description | | Text | True | | | #### Relationships[​](#relationships-7 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | ---------------- | -------- | ----------- | ---- | | topology | TopologyTopology | True | many | | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### DcimGenericDevice[​](#dcimgenericdevice "Direct link to DcimGenericDevice") #### Relationships[​](#relationships-8 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------- | ---------------- | -------- | ----------- | ---- | | topology | TopologyTopology | True | one | | ### IpamPrefix[​](#ipamprefix "Direct link to IpamPrefix") #### Relationships[​](#relationships-9 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | ---------------------- | -------- | ----------- | ---- | | network\_service | TopologyNetworkService | True | one | | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: ManagementServer namespace: Network description: Generic model for network management server (dns, ntp, and dhcp). label: Network Management Servers human_friendly_id: - name__value display_label: name__value order_by: - name__value include_in_menu: true attributes: - name: name kind: Text order_weight: 1000 - name: description kind: Text order_weight: 1100 optional: true - name: status kind: Dropdown choices: - name: active label: Active description: Fully operational and currently in service. color: '#7fbf7f' - name: provisioning label: Provisioning description: In the process of being set up and configured. color: '#ffff7f' - name: maintenance label: Maintenance description: Undergoing routine maintenance or repairs. color: '#ffd27f' - name: drained label: Drained description: Temporarily taken out of service. color: '#bfbfbf' relationships: - name: location peer: LocationGeneric optional: true cardinality: many - name: ip_addresses peer: IpamIPAddress optional: true cardinality: many kind: Component - name: GenericElement namespace: Topology description: Base model for elements label: Generic Topology Element icon: carbon:network-3-reference human_friendly_id: - name__value display_label: name__value order_by: - name__value include_in_menu: false menu_placement: TopologyTopology attributes: - name: name kind: Text order_weight: 1000 - name: description kind: Text order_weight: 1100 optional: true - name: quantity kind: Number order_weight: 1200 relationships: - name: topology peer: TopologyTopology cardinality: one kind: Parent optional: false - name: NetworkStrategy namespace: Topology description: Generic model for network strategies (underlays and overlays). label: Network Strategy icon: iconoir:strategy human_friendly_id: - name__value display_label: name__value order_by: - name__value include_in_menu: true menu_placement: TopologyTopology attributes: - name: name kind: Text order_weight: 1000 - name: description kind: Text order_weight: 1100 optional: true relationships: - name: topology peer: TopologyTopology optional: true cardinality: many nodes: - name: Topology namespace: Topology description: A Topology represents the entire network pod. label: Topology icon: carbon:network-3 human_friendly_id: - name__value display_label: name__value order_by: - name__value inherit_from: - CoreArtifactTarget attributes: - name: name kind: Text unique: true order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: elements peer: TopologyGenericElement optional: true cardinality: many kind: Component - name: strategy peer: TopologyNetworkStrategy optional: true cardinality: one kind: Component - name: location peer: LocationGeneric optional: true cardinality: one kind: Attribute - name: devices peer: DcimGenericDevice optional: true cardinality: many kind: Component - name: network_services peer: TopologyNetworkService optional: true cardinality: many kind: Component - name: MPLSStrategy namespace: Topology description: Specific strategy attributes for MPLS. label: MPLS Strategy icon: eos-icons:neural-network inherit_from: - TopologyNetworkStrategy menu_placement: TopologyNetworkStrategy include_in_menu: false human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: underlay kind: Dropdown choices: - name: ospf label: OSPF - name: isis label: ISIS - name: bgp label: BGP - name: overlay kind: Dropdown choices: - name: ldp label: LDP - name: rsvp label: RSVP - name: segment_routing label: Segment Routing - name: EVPNStrategy namespace: Topology description: Specific strategy attributes for EVPN. label: EVPN Strategy icon: carbon:load-balancer-network inherit_from: - TopologyNetworkStrategy menu_placement: TopologyNetworkStrategy include_in_menu: false human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: underlay kind: Dropdown choices: - name: ebgp label: EBGP - name: ospf label: OSPF - name: isis label: ISIS - name: overlay kind: Dropdown choices: - name: ebgp label: EBGP - name: ibgp label: IBGP - name: PhysicalElement namespace: Topology description: Physical aspect of topology elements. label: Physical Topology Element icon: carbon:network-3-reference menu_placement: TopologyTopology human_friendly_id: - name__value order_by: - name__value display_label: name__value inherit_from: - TopologyGenericElement attributes: - name: mtu kind: Number default_value: 1500 order_weight: 3000 - name: border kind: Boolean label: Is a Topology Border default_value: false order_weight: 2400 - name: mlag_support kind: Boolean label: MLAG Support default_value: false order_weight: 2500 - name: device_role label: Role kind: Dropdown choices: - name: spine color: '#ffb3ba' - name: leaf color: '#ffdfba' - name: pe_router color: '#baffc9' - name: p_router color: '#bae1ff' - name: route_reflector color: '#ffbaba' - name: cpe color: '#f2bae1' - name: firewall color: '#c5a3ff' order_weight: 1300 relationships: - name: device_type label: Type peer: DcimDeviceType optional: true cardinality: one kind: Attribute order_weight: 1400 - name: DhcpOption namespace: Network description: Represents a configurable option within a Dhcp server. label: Dhcp Option icon: gis:globe-options menu_placement: NetworkManagementServer include_in_menu: false human_friendly_id: - name__value display_label: name__value order_by: - name__value attributes: - name: name kind: Text order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 - name: option_code kind: Number order_weight: 2000 - name: content kind: Text order_weight: 2100 - name: DhcpServer namespace: Network description: Represents a Dhcp server in the network. label: Dhcp Server icon: eos-icons:ip menu_placement: NetworkManagementServer include_in_menu: false human_friendly_id: - name__value display_label: name__value order_by: - name__value inherit_from: - NetworkManagementServer attributes: - name: lease_time kind: Text order_weight: 2100 relationships: - name: dhcp_options peer: NetworkDhcpOption optional: true cardinality: many kind: Component - name: NameServer namespace: Network description: Represents a DNS server in the network. label: DNS Server icon: eos-icons:dns menu_placement: NetworkManagementServer human_friendly_id: - name__value display_label: name__value order_by: - name__value inherit_from: - NetworkManagementServer - name: NTPServer namespace: Network description: Represents a NTP server in the network. label: NTP Server icon: iconoir:time-zone menu_placement: NetworkManagementServer human_friendly_id: - name__value display_label: name__value order_by: - name__value inherit_from: - NetworkManagementServer - name: NetworkServiceIdentifier namespace: Topology description: Generic model for different types of identifiers used in network services. label: Network Service Identifier icon: mdi:identifier menu_placement: TopologyNetworkService human_friendly_id: - identifier__value order_by: - identifier__value display_label: identifier__value attributes: - name: identifier kind: Number relationships: - name: service label: Network Service peer: TopologyNetworkService cardinality: one optional: true - name: NetworkService namespace: Topology description: Network services attached to a Topology. label: Network Service icon: carbon:ibm-cloud-internet-services human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name label: Service Name kind: Text order_weight: 1000 - name: description kind: Text order_weight: 1100 - name: service_type kind: Text enum: - Layer2 - Layer3 order_weight: 1200 relationships: - name: identifier peer: TopologyNetworkServiceIdentifier cardinality: one kind: Attribute optional: false - name: topology peer: TopologyTopology cardinality: one kind: Parent optional: false - name: vlan peer: IpamVLAN optional: true cardinality: one kind: Component order_weight: 1500 - name: prefix peer: IpamPrefix optional: true cardinality: one kind: Component order_weight: 1400 extensions: nodes: - kind: DcimGenericDevice relationships: - name: topology peer: TopologyTopology optional: true cardinality: one - kind: IpamPrefix relationships: - name: network_service label: Network Service peer: TopologyNetworkService optional: true cardinality: one order_weight: 1400 ``` --- # Transceiver This schema extension models pluggable transceivers across form factors (SFP, SFP+, QSFP, QSFP28, QSFP-DD, OSFP, CFP, XFP and more), with a Standard Transceiver and a Bidirectional Transceiver flavour built on a shared generic. A transceiver is either plugged into a physical interface or stored as a spare at a location, and can be linked to its manufacturer. Limitations: there is no validation between type, form factor, protocol and distance; any transceiver can be plugged into any physical interface; and nothing prevents a transceiver from being linked to both an interface and a spare location. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### StandardTransceiver[​](#standardtransceiver "Direct link to StandardTransceiver") * **Label:** Standard Transceiver * **Description:** Standard transceiver for common types (e.g. LR, SR, T). * **Namespace:** Dcim * **Icon:** mdi :gpu * **Inherit From:** DcimGenericTransceiver ### BidiTransceiver[​](#biditransceiver "Direct link to BidiTransceiver") * **Label:** Bidirectional Transceiver * **Description:** Bidirectional transceiver using two wavelengths for single-fiber operation. * **Namespace:** Dcim * **Icon:** lineicons :arrow-both-direction-vertical-1 * **Inherit From:** DcimGenericTransceiver #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------------- | -------------------------- | ------ | -------- | -------------- | ------- | | wavelength\_tx | Transmit wavelength in nm. | Number | False | | | | wavelength\_rx | Receive wavelength in nm. | Number | False | | | ## Generics[​](#generics "Direct link to Generics") ### GenericTransceiver[​](#generictransceiver "Direct link to GenericTransceiver") * **Label:** Transceiver * **Description:** Generic base for all pluggable transceivers (SFP, QSFP, OSFP, CFP and similar form factors). * **Namespace:** Dcim * **Icon:** mdi :gpu #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------------- | -------------------------------------------- | -------- | -------- | -------------- | --------------------------------------------------------------------------------------------- | | serial\_number | | Text | True | | | | transceiver\_type | Type of transceiver, such as LR, SR, T. | Dropdown | False | | lr, sr, lrm, t, sr4, lr4, zr, er, dac, aoc | | status | | Dropdown | False | plugged | plugged, spare, decommissioned | | form\_factor | The physical form factor of the transceiver. | Dropdown | False | | sfp, sfp\_plus, qsfp, qsfp\_plus, qsfp28, qsfp\_dd, cfp, cfp2, cfp4, xfp, sfp56, qsfp56, osfp | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------------- | ------------------------ | -------- | ----------- | --------- | | interface | InterfacePhysical | True | one | Attribute | | spare\_location | LocationHosting | True | one | Attribute | | manufacturer | OrganizationManufacturer | True | one | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### InterfacePhysical[​](#interfacephysical "Direct link to InterfacePhysical") #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------------- | ---------------------- | -------- | ----------- | ------- | | plugged\_transceiver | DcimGenericTransceiver | True | one | Generic | ### LocationHosting[​](#locationhosting "Direct link to LocationHosting") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------------- | ---------------------- | -------- | ----------- | ------- | | spare\_transceivers | DcimGenericTransceiver | True | many | Generic | ### OrganizationManufacturer[​](#organizationmanufacturer "Direct link to OrganizationManufacturer") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ---------------------- | -------- | ----------- | ------- | | transceivers | DcimGenericTransceiver | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: GenericTransceiver namespace: Dcim description: Generic base for all pluggable transceivers (SFP, QSFP, OSFP, CFP and similar form factors). label: Transceiver icon: mdi:gpu include_in_menu: true display_label: '{{ form_factor__value }} {{ transceiver_type__value }}' attributes: - name: serial_number kind: Text optional: true order_weight: 1400 - name: transceiver_type kind: Dropdown choices: - name: lr label: LR (Long Reach) description: Long Reach transceiver, suitable for long-distance fiber connections. color: '#009933' - name: sr label: SR (Short Reach) description: Short Reach transceiver, typically used for short-distance fiber. color: '#cc66ff' - name: lrm label: LRM (Long Reach Multimode) description: Transceiver for multimode fiber over longer distances. color: '#3366ff' - name: t label: T (Twisted Pair) description: Copper-based transceiver, often used for twisted pair Ethernet. color: '#ff9900' - name: sr4 label: SR4 (Short Range 4-lane) description: Short Range 4-lane transceiver, used for parallel optics. color: '#6666ff' - name: lr4 label: LR4 (Long Range 4-lane) description: Long Range 4-lane transceiver, typically used for longer distances over parallel optics. color: '#336699' - name: zr label: ZR (Extended Reach) description: Extended Reach transceiver, suitable for distances up to 80km. color: '#cc3300' - name: er label: ER (Extended Reach) description: Extended Reach transceiver, typically used for 40km distances. color: '#ff6600' - name: dac label: DAC (Direct Attach Copper) description: Direct Attach Copper, used for short connections over copper. color: '#b35900' - name: aoc label: AOC (Active Optical Cable) description: Active Optical Cable, used for short connections over fiber. color: '#6699ff' optional: false description: Type of transceiver, such as LR, SR, T. order_weight: 1100 - name: status kind: Dropdown default_value: plugged choices: - name: plugged label: Plugged description: Plugged into a device's interface. color: '#7fbf7f' - name: spare label: Spare description: Stored somewhere as a spare. color: '#ffff7f' - name: decommissioned label: Decommissioned description: Decommissioned might be broken or not used anymore. color: '#ffd27f' optional: false order_weight: 1200 - name: form_factor kind: Dropdown choices: - name: sfp label: SFP color: '#009933' - name: sfp_plus label: SFP+ color: '#cc66ff' - name: qsfp label: QSFP color: '#6666ff' - name: qsfp_plus label: QSFP+ color: '#3366ff' - name: qsfp28 label: QSFP28 color: '#336699' - name: qsfp_dd label: QSFP-DD color: '#ff9900' - name: cfp label: CFP color: '#cc3300' - name: cfp2 label: CFP2 color: '#ff6600' - name: cfp4 label: CFP4 color: '#b35900' - name: xfp label: XFP color: '#6699ff' - name: sfp56 label: SFP56 color: '#9966ff' - name: qsfp56 label: QSFP56 color: '#9933cc' - name: osfp label: OSFP color: '#0099cc' optional: false description: The physical form factor of the transceiver. order_weight: 1000 relationships: - name: interface peer: InterfacePhysical kind: Attribute cardinality: one optional: true identifier: transceiver__interface order_weight: 1300 - name: spare_location peer: LocationHosting kind: Attribute cardinality: one optional: true order_weight: 1500 - name: manufacturer peer: OrganizationManufacturer kind: Attribute cardinality: one optional: true order_weight: 1350 nodes: - name: StandardTransceiver namespace: Dcim description: Standard transceiver for common types (e.g. LR, SR, T). label: Standard Transceiver icon: mdi:gpu include_in_menu: true menu_placement: DcimGenericTransceiver inherit_from: - DcimGenericTransceiver - name: BidiTransceiver namespace: Dcim description: Bidirectional transceiver using two wavelengths for single-fiber operation. label: Bidirectional Transceiver icon: lineicons:arrow-both-direction-vertical-1 include_in_menu: true menu_placement: DcimGenericTransceiver inherit_from: - DcimGenericTransceiver attributes: - name: wavelength_tx kind: Number label: Transmit Wavelength (nm) optional: false description: Transmit wavelength in nm. order_weight: 1175 - name: wavelength_rx kind: Number label: Receive Wavelength (nm) optional: false description: Receive wavelength in nm. order_weight: 1150 extensions: nodes: - kind: InterfacePhysical relationships: - name: plugged_transceiver peer: DcimGenericTransceiver kind: Generic cardinality: one optional: true identifier: transceiver__interface order_weight: 1700 - kind: LocationHosting relationships: - name: spare_transceivers peer: DcimGenericTransceiver kind: Generic cardinality: many optional: true order_weight: 1750 - kind: OrganizationManufacturer relationships: - name: transceivers peer: DcimGenericTransceiver label: Transceivers kind: Generic cardinality: many optional: true order_weight: 1500 ``` --- # VLAN This schema extension adds VLAN and VLAN Group nodes, with a generic VLAN to build on, and links VLANs to prefixes and layer 2 interfaces. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### VLAN[​](#vlan "Direct link to VLAN") * **Label:** VLAN * **Description:** A plain VLAN, with no QinQ role of its own. * **Namespace:** Ipam * **Uniqueness Constraints:** * vlan\_id\_\_value, vlan\_group * **Inherit From:** IpamGenericVLAN ### VLANGroup[​](#vlangroup "Direct link to VLANGroup") * **Label:** VLAN Group * **Description:** A VLAN group scopes VLANs to a domain (site, rack, etc.). * **Namespace:** Ipam * **Icon:** mdi :domain-switch * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----- | ------------------ | -------- | ----------- | --------- | | scope | IpamVLANGroupScope | True | one | Attribute | | vlans | IpamGenericVLAN | True | many | Component | ## Generics[​](#generics "Direct link to Generics") ### GenericVLAN[​](#genericvlan "Direct link to GenericVLAN") * **Label:** VLAN * **Description:** A VLAN is an isolated layer two domain. * **Namespace:** Ipam * **Icon:** mdi :lan-pending * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | -------- | -------- | -------------- | ------------------------------------------------------- | | name | | Text | False | | | | description | | Text | True | | | | vlan\_id | | Number | False | | | | status | | Dropdown | False | active | active, reserved, provisioning, maintenance, deprecated | | role | | Dropdown | True | | server, management, user | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------- | ------------- | -------- | ----------- | --------- | | prefixes | IpamPrefix | True | many | Attribute | | vlan\_group | IpamVLANGroup | False | one | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### IpamVLANGroupScope[​](#ipamvlangroupscope "Direct link to IpamVLANGroupScope") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------ | ------------- | -------- | ----------- | --------- | | vlan\_groups | IpamVLANGroup | True | many | Attribute | ### IpamPrefix[​](#ipamprefix "Direct link to IpamPrefix") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | --------------- | -------- | ----------- | --------- | | vlan | IpamGenericVLAN | True | one | Attribute | ### InterfaceLayer2[​](#interfacelayer2 "Direct link to InterfaceLayer2") #### Relationships[​](#relationships-4 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | -------------- | --------------- | -------- | ----------- | ------- | | untagged\_vlan | IpamGenericVLAN | True | one | Generic | | tagged\_vlan | IpamGenericVLAN | True | many | Generic | ## Code[​](#code "Direct link to Code") ``` version: '1.0' generics: - name: GenericVLAN namespace: Ipam description: A VLAN is an isolated layer two domain. label: VLAN icon: mdi:lan-pending include_in_menu: true human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 - name: vlan_id kind: Number optional: false order_weight: 1050 - name: status kind: Dropdown default_value: active choices: - name: active label: Active description: Fully operational and currently in service. color: '#00d25b' - name: reserved label: Reserved description: Reserved for future use, not yet active. color: '#4d90fe' - name: provisioning label: Provisioning description: In the process of being set up and configured. color: '#f0ad4e' - name: maintenance label: Maintenance description: Undergoing routine maintenance or repairs. color: '#ff9800' - name: deprecated label: Deprecated description: No longer in use and scheduled for removal. color: '#e04040' optional: false order_weight: 1200 - name: role kind: Dropdown choices: - name: server label: Server description: Dedicated systems for managing networked resources. color: '#c4bed7' - name: management label: Management description: Network segments for administrative and control tasks. color: '#9af1e1' - name: user label: User description: Segments designed for end-user access and activities. color: '#a0b78d' optional: true order_weight: 1300 relationships: - name: prefixes peer: IpamPrefix kind: Attribute cardinality: many optional: true identifier: vlan__prefix order_weight: 1500 - name: vlan_group peer: IpamVLANGroup label: VLAN Group kind: Attribute cardinality: one optional: false identifier: vlan_group__vlan order_weight: 1400 nodes: - name: VLAN namespace: Ipam description: A plain VLAN, with no QinQ role of its own. label: VLAN include_in_menu: false inherit_from: - IpamGenericVLAN uniqueness_constraints: - - vlan_id__value - vlan_group - name: VLANGroup namespace: Ipam description: A VLAN group scopes VLANs to a domain (site, rack, etc.). label: VLAN Group icon: mdi:domain-switch include_in_menu: true menu_placement: IpamGenericVLAN human_friendly_id: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: scope peer: IpamVLANGroupScope kind: Attribute cardinality: one optional: true identifier: vlan_group__scope order_weight: 900 - name: vlans peer: IpamGenericVLAN kind: Component cardinality: many optional: true identifier: vlan_group__vlan order_weight: 1500 extensions: nodes: - kind: IpamVLANGroupScope relationships: - name: vlan_groups peer: IpamVLANGroup label: VLAN Groups kind: Attribute cardinality: many optional: true identifier: vlan_group__scope order_weight: 1600 - kind: IpamPrefix relationships: - name: vlan peer: IpamGenericVLAN kind: Attribute cardinality: one optional: true identifier: vlan__prefix order_weight: 1450 - kind: InterfaceLayer2 relationships: - name: untagged_vlan peer: IpamGenericVLAN label: Untagged VLAN kind: Generic cardinality: one optional: true identifier: interface_l2__untagged_vlan order_weight: 1500 - name: tagged_vlan peer: IpamGenericVLAN label: Tagged VLANs kind: Generic cardinality: many optional: true identifier: interface_l2__tagged_vlan order_weight: 1550 ``` --- # VLAN Translation This schema extension is based on Juniper VLAN MAP, and not yet test out for other vendors. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### MapInOut[​](#mapinout "Direct link to MapInOut") * **Label:** VLAN Map In/Out * **Description:** VLAN Mapping for In/Out operations * **Namespace:** Network * **Icon:** ph :swap * **Uniqueness Constraints:** * interface, direction\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ------------------------ | ------------------------------------------------------- | -------- | -------- | -------------- | ------------------------------------------------------------------------ | | direction | Direction of the mapping | Dropdown | True | | input, output | | operation | Operation type | Dropdown | True | | pop, pop\_pop, pop\_swap, push, push\_push, swap, swap\_push, swap\_swap | | vlan\_id\_swap | VLAN ID to swap to during SWAP operations | Number | True | | | | inner\_vlan\_id | Inner VLAN ID for operations involving double VLAN tags | Number | True | | | | inner\_tag\_protocol\_id | Inner tag protocol ID (TPID) | Number | True | | | | tag\_protocol\_id | Tag protocol ID (TPID) for outer VLAN operations | Number | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | --------- | ------------- | -------- | ----------- | ------ | | interface | DcimInterface | False | one | Parent | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### DcimInterface[​](#dciminterface "Direct link to DcimInterface") #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ------------- | --------------- | -------- | ----------- | --------- | | network\_maps | NetworkMapInOut | | many | Component | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: MapInOut namespace: Network label: VLAN Map In/Out icon: ph:swap menu_placement: DcimInterface include_in_menu: true description: VLAN Mapping for In/Out operations order_by: - interface__name__value - direction__value - operation__value uniqueness_constraints: - - interface - direction__value attributes: - name: direction kind: Dropdown description: Direction of the mapping label: Map Direction order_weight: 1050 choices: - name: input label: Input description: Input direction color: '#D2B4DE' - name: output label: Output description: Output direction color: '#A9CCE3' optional: true - name: operation kind: Dropdown description: Operation type label: Map Operation order_weight: 1100 choices: - name: pop label: POP description: Single POP operation color: '#B2D4E6' - name: pop_pop label: POP-POP description: Double POP operation color: '#AED6F1' - name: pop_swap label: POP-SWAP description: POP then SWAP operation color: '#A9DFBF' - name: push label: PUSH description: Single PUSH operation color: '#CDEACC' - name: push_push label: PUSH-PUSH description: Double PUSH operation color: '#9FA8DA' - name: swap label: SWAP description: Single SWAP operation color: '#D2B4DE' - name: swap_push label: SWAP-PUSH description: SWAP then PUSH operation color: '#C4B7E6' - name: swap_swap label: SWAP-SWAP description: Double SWAP operation color: '#CBC3E3' optional: true - name: vlan_id_swap label: VLAN ID Swap kind: Number description: VLAN ID to swap to during SWAP operations optional: true order_weight: 1200 - name: inner_vlan_id label: Inner VLAN ID kind: Number description: Inner VLAN ID for operations involving double VLAN tags optional: true order_weight: 1300 - name: inner_tag_protocol_id label: Inner Tag Protocol ID kind: Number description: Inner tag protocol ID (TPID) optional: true order_weight: 1400 - name: tag_protocol_id label: Tag Protocol ID kind: Number description: Tag protocol ID (TPID) for outer VLAN operations optional: true order_weight: 1500 relationships: - name: interface kind: Parent peer: DcimInterface description: Interface to which the Input/Output VLAN mapping is applied cardinality: one optional: false label: Interface order_weight: 1000 extensions: nodes: - kind: DcimInterface relationships: - name: network_maps kind: Component peer: NetworkMapInOut description: Interface Input/Output VLAN mapping cardinality: many label: Input/Output MAP order_weight: 1600 ``` --- # VRF This schema extension adds VRF and Route Target nodes and links prefixes and IP addresses to a VRF. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### VRF[​](#vrf "Direct link to VRF") * **Label:** VRF * **Description:** A VRF is an isolated layer three domain. * **Namespace:** Ipam * **Icon:** mdi :router * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | --------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------- | -------- | -------------- | ------- | | name | | Text | False | | | | vrf\_rd | | Text | True | | | | enforce\_unique | Declares the intent that prefixes and addresses be unique in this VRF. Infrahub does not enforce it; use a check or generator. | Boolean | True | True | | | description | | Text | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------- | --------------- | -------- | ----------- | --------- | | import\_rt | IpamRouteTarget | True | many | Attribute | | export\_rt | IpamRouteTarget | True | many | Attribute | ### RouteTarget[​](#routetarget "Direct link to RouteTarget") * **Label:** Route Target * **Description:** Route Target (RFC 4360) * **Namespace:** Ipam * **Icon:** mdi :target * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | ----------- | ----------- | ---- | -------- | -------------- | ------- | | name | | Text | False | | | | description | | Text | True | | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------- | ------- | -------- | ----------- | ------- | | import\_vrf | IpamVRF | True | many | Generic | | export\_vrf | IpamVRF | True | many | Generic | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### IpamPrefix[​](#ipamprefix "Direct link to IpamPrefix") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | ------- | -------- | ----------- | --------- | | vrf | IpamVRF | True | one | Attribute | ### IpamIPAddress[​](#ipamipaddress "Direct link to IpamIPAddress") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | ------- | -------- | ----------- | --------- | | vrf | IpamVRF | True | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: VRF namespace: Ipam description: A VRF is an isolated layer three domain. label: VRF icon: mdi:router include_in_menu: true human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: vrf_rd kind: Text label: Route Distinguisher optional: true order_weight: 1050 - name: enforce_unique kind: Boolean label: Enforce Unique default_value: true optional: true description: Declares the intent that prefixes and addresses be unique in this VRF. Infrahub does not enforce it; use a check or generator. order_weight: 1200 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: import_rt peer: IpamRouteTarget label: Import Targets kind: Attribute cardinality: many optional: true identifier: vrf__import order_weight: 1300 - name: export_rt peer: IpamRouteTarget label: Export Targets kind: Attribute cardinality: many optional: true identifier: vrf__export order_weight: 1350 - name: RouteTarget namespace: Ipam description: Route Target (RFC 4360) label: Route Target icon: mdi:target include_in_menu: true menu_placement: IpamVRF human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text unique: true optional: false order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: import_vrf peer: IpamVRF label: Imported by VRFs kind: Generic cardinality: many optional: true identifier: vrf__import order_weight: 1300 - name: export_vrf peer: IpamVRF label: Exported by VRFs kind: Generic cardinality: many optional: true identifier: vrf__export order_weight: 1350 extensions: nodes: - kind: IpamPrefix relationships: - name: vrf peer: IpamVRF label: VRF kind: Attribute cardinality: one optional: true identifier: prefix__vrf order_weight: 1150 - kind: IpamIPAddress relationships: - name: vrf peer: IpamVRF label: VRF kind: Attribute cardinality: one optional: true identifier: ip_address__vrf order_weight: 1150 ``` --- # VRRP This schema extension adds VRRP Group and VRRP Interface nodes, linking a virtual IP address to the layer 3 interfaces that take part in the group. ## Details[​](#details "Direct link to Details") * **Dependencies:** * [base](/schema-library/reference/dcim.md) ## Nodes[​](#nodes "Direct link to Nodes") ### VRRPGroup[​](#vrrpgroup "Direct link to VRRPGroup") * **Label:** VRRP Group * **Description:** VRRP Group configuration * **Namespace:** Network * **Icon:** fluent :virtual-network-20-filled * **Human Friendly ID:** name\_\_value #### Attributes[​](#attributes "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------- | ----------------------------------------------------- | -------------- | -------- | -------------- | ------- | | name | Unique name of the entry | Text | False | | | | vrid | Virtual Router ID (VRID, should be between 1 and 255) | Number | False | | | | password | VRRP Password/Key | HashedPassword | True | | | #### Relationships[​](#relationships "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---------------- | -------------------- | -------- | ----------- | --------- | | ip\_addresses | IpamIPAddress | True | many | Attribute | | vrrp\_interfaces | NetworkVRRPInterface | True | many | Component | ### VRRPInterface[​](#vrrpinterface "Direct link to VRRPInterface") * **Label:** VRRP Interface * **Description:** VRRP Interface configuration * **Namespace:** Network * **Icon:** carbon :interface-usage * **Uniqueness Constraints:** * vrrp\_group, interface #### Attributes[​](#attributes-1 "Direct link to Attributes") | name | description | kind | optional | default\_value | choices | | -------- | ------------------------------------------- | ------ | -------- | -------------- | ------- | | priority | VRRP Priority (Should be between 1 and 255) | Number | False | 100 | | #### Relationships[​](#relationships-1 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ----------- | ---------------- | -------- | ----------- | --------- | | vrrp\_group | NetworkVRRPGroup | False | one | Attribute | | interface | InterfaceLayer3 | False | one | Attribute | ## Extensions[​](#extensions "Direct link to Extensions") note In this context "extensions" refer to modifications or additions to the existing schema, such as adding new attributes, relationships, or other schema elements. ### InterfaceLayer3[​](#interfacelayer3 "Direct link to InterfaceLayer3") #### Relationships[​](#relationships-2 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | -------------------- | -------- | ----------- | --------- | | vrrp | NetworkVRRPInterface | True | one | Component | ### IpamIPAddress[​](#ipamipaddress "Direct link to IpamIPAddress") #### Relationships[​](#relationships-3 "Direct link to Relationships") | name | peer | optional | cardinality | kind | | ---- | ---------------- | -------- | ----------- | --------- | | vrrp | NetworkVRRPGroup | True | one | Attribute | ## Code[​](#code "Direct link to Code") ``` version: '1.0' nodes: - name: VRRPGroup namespace: Network description: VRRP Group configuration label: VRRP Group icon: fluent:virtual-network-20-filled include_in_menu: true human_friendly_id: - name__value order_by: - name__value display_label: name__value attributes: - name: name kind: Text label: Name unique: true optional: false description: Unique name of the entry order_weight: 1000 - name: vrid kind: Number label: VRID parameters: min_value: 1 max_value: 255 optional: false description: Virtual Router ID (VRID, should be between 1 and 255) order_weight: 1100 - name: password kind: HashedPassword label: Password optional: true description: VRRP Password/Key order_weight: 1400 relationships: - name: ip_addresses peer: IpamIPAddress label: IP Addresses kind: Attribute cardinality: many optional: true on_delete: cascade description: VRRP IP (v4 or v6) order_weight: 1200 - name: vrrp_interfaces peer: NetworkVRRPInterface label: VRRP Interfaces kind: Component cardinality: many optional: true on_delete: cascade order_weight: 1300 - name: VRRPInterface namespace: Network description: VRRP Interface configuration label: VRRP Interface icon: carbon:interface-usage include_in_menu: true menu_placement: NetworkVRRPGroup order_by: - priority__value uniqueness_constraints: - - vrrp_group - interface attributes: - name: priority kind: Number label: VRRP Priority default_value: 100 parameters: min_value: 1 max_value: 255 optional: false description: VRRP Priority (Should be between 1 and 255) order_weight: 1100 relationships: - name: vrrp_group peer: NetworkVRRPGroup label: VRRP Group kind: Attribute cardinality: one optional: false description: VRRP Group order_weight: 1200 - name: interface peer: InterfaceLayer3 label: Interface kind: Attribute cardinality: one optional: false description: Interface L3 order_weight: 1300 extensions: nodes: - kind: InterfaceLayer3 relationships: - name: vrrp peer: NetworkVRRPInterface label: VRRP kind: Component cardinality: one optional: true description: VRRP Interface Configuration order_weight: 1650 - kind: IpamIPAddress relationships: - name: vrrp peer: NetworkVRRPGroup label: VRRP Group kind: Attribute cardinality: one optional: true description: Part of VRRP Group order_weight: 1700 ``` --- # Infrahub Skills Infrahub Skills is an open-source AI skills package that gives your AI coding assistant built-in knowledge of Infrahub's data model, conventions, and workflow patterns. Instead of learning the platform from documentation alone, describe what you want to build in plain language and the skills produce valid, best-practice Infrahub resources — schemas, object data, generators, transforms, checks, and menus — ready to load into a running instance. The skills cover the full Infrahub development lifecycle. Each skill embeds the rules, examples, and references the AI needs for a specific task, and loads only the context required — so the assistant stays focused and accurate without exhausting its context window. ## What you can do with it[​](#what-you-can-do-with-it "Direct link to What you can do with it") ### Design and build your data model[​](#design-and-build-your-data-model "Direct link to Design and build your data model") Describe your data model in plain language and the skills produce a valid Infrahub schema — node types, attribute kinds, relationships, generics, display properties, profiles (shared default values), and object templates (cloneable structures) — without requiring manual study of the schema format. Naming conventions, relationship identifiers, and attribute type rules are applied at the point of generation. ### Build generators, transforms, and checks[​](#build-generators-transforms-and-checks "Direct link to Build generators, transforms, and checks") Create Python generators that auto-create infrastructure objects from design definitions. Write Python or Jinja2 transforms that convert Infrahub data to device configs, reports, or other formats. Build validation checks that run in proposed change pipelines — with Infrahub-specific conventions handled automatically. ### Populate and manage data[​](#populate-and-manage-data "Direct link to Populate and manage data") Produce YAML data files for infrastructure objects — devices, locations, organizations, interfaces — with correct file structure, value mappings, and load order, including assigning profiles for shared defaults and creating objects from templates. Generate batches of objects from descriptions or seed data for new instances. ### Query and analyze a live instance[​](#query-and-analyze-a-live-instance "Direct link to Query and analyze a live instance") The Data Analyzer skill connects to a running Infrahub instance via MCP and answers operational questions directly: cross-node correlation, drift detection, blast-radius analysis, and data quality audits — without writing GraphQL queries manually. ### Audit your repository[​](#audit-your-repository "Direct link to Audit your repository") Run a comprehensive audit of an Infrahub repository against all best practices. The Repo Auditor checks schema files, object files, Python components, `.infrahub.yml` registration, and project structure — then reports what to fix and why. ### Customize the UI[​](#customize-the-ui "Direct link to Customize the UI") Design custom navigation menus for the Infrahub web interface — sidebar structure, section groupings, icons, and ordering — from a plain language description. ## Three ways to work[​](#three-ways-to-work "Direct link to Three ways to work") ### Direct mode[​](#direct-mode "Direct link to Direct mode") For well-scoped, single-skill tasks. Describe what you need and the AI matches it to the right skill automatically. Best for adding an attribute, creating a check, populating a batch of objects, or any task that maps to a single skill. ### Spec-Driven Development (SDD)[​](#spec-driven-development-sdd "Direct link to Spec-Driven Development (SDD)") For complex or multi-part builds. The AI reasons through requirements with you first — capturing what needs to be built, validating the approach against Infrahub conventions, and laying out a task plan for review. Nothing is generated until you approve the plan. Best for designing new schema domains, building generator chains, or any task that involves design decisions across multiple skills. See [Spec-Driven Development](/skills/spec-driven-development.md) for details. ### End-to-end execution[​](#end-to-end-execution "Direct link to End-to-end execution") For full lifecycle workflows. The AI generates files, loads them with `infrahubctl`, validates the result, and iterates if something fails — all in a single session. ## Skills included[​](#skills-included "Direct link to Skills included") | Skill | What it does | | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- | | [Schema Manager](/skills/skills-reference/managing-schemas.md) | Creates and validates Infrahub schema YAML from natural language | | [Object Manager](/skills/skills-reference/managing-objects.md) | Produces YAML data files for infrastructure objects | | [Check Manager](/skills/skills-reference/managing-checks.md) | Builds Python validation checks for proposed change pipelines | | [Generator Manager](/skills/skills-reference/managing-generators.md) | Creates design-driven generators that auto-create infrastructure objects | | [Transform Manager](/skills/skills-reference/managing-transforms.md) | Builds Python or Jinja2 transforms for config generation and data export | | [Menu Manager](/skills/skills-reference/managing-menus.md) | Designs custom navigation menus for the Infrahub web UI | | [Data Analyzer](/skills/skills-reference/analyzing-data.md) | Queries and analyzes live Infrahub data via MCP | | [Repo Auditor](/skills/skills-reference/auditing-repo.md) | Audits a repository against Infrahub best practices | | [Data Importer](/skills/skills-reference/importing-data.md) | Converts CSV/TSV inputs into Infrahub object YAML and loads onto a fresh branch | | [Diagnostics Collector](/skills/skills-reference/collecting-diagnostics.md) | Guides `infrahub-collect` to gather a support bundle when Infrahub is misbehaving, and reviews it before hand-off | | [Issue Reporter](/skills/skills-reference/reporting-issues.md) | Routes a bug or feature request to the right Infrahub-ecosystem repository and prepares a sanitized draft for your review | | [Skill Gap Reporter](/skills/skills-reference/reporting-skill-gaps.md) | Turns friction with an Infrahub skill into a reviewed issue with a proposed rule change, filed through the Issue Reporter | | [Concept Tutor](/skills/skills-reference/teaching-concepts.md) | Teaches Infrahub concepts through the learner's own repo and instance, with verified exercises and progress tracking | | [Diagnostics Analyzer](/skills/skills-reference/analyzing-diagnostics.md) | Turns a collected diagnostic bundle into a triage report, correlating errors into incidents and matching them against known Infrahub issues | | [NetBox Device Type Converter](/skills/skills-reference/converting-netbox-device-types.md) | Converts NetBox device-type and module-type definitions into Infrahub Object Templates via a mapping-profile-driven converter | A shared reference library (`infrahub-common`) provides GraphQL query syntax, `.infrahub.yml` configuration format, YAML structure conventions, and the Profiles vs Object Templates distinction to all skills. ## Compatibility[​](#compatibility "Direct link to Compatibility") Infrahub Skills works with any AI tool that supports the Agent Skills format or reads custom context files from the project directory: * **Claude Code** — full support including auto-detection and direct skill invocation * **GitHub Copilot** — passive skill activation from project context * **Cursor** — passive skill activation from project context * **Windsurf** — passive skill activation from project context * **Amp, Cline, Codex** — passive skill activation from project context See [Installation & Setup](/skills/installation-setup.md) for tool-specific configuration. ## Next steps[​](#next-steps "Direct link to Next steps") * **Get started** — [Installation & Setup](/skills/installation-setup.md) * **Understand the architecture** — [How It Works](/skills/how-it-works.md) * **Plan a complex build** — [Spec-Driven Development](/skills/spec-driven-development.md) * **Explore individual skills** — [Skills Reference](/skills/skills-reference/managing-schemas.md) --- # How It Works Infrahub Skills gives an AI assistant domain-specific knowledge about the Infrahub platform. Each skill is a structured set of Markdown files — rules, examples, and references — that the AI reads at the point it's needed. The assistant matches each user request to the relevant skill automatically, loads only the context required for that task, and applies the rules embedded in the skill. ## What a skill is[​](#what-a-skill-is "Direct link to What a skill is") A skill is a directory containing a `SKILL.md` entry point, modular rule files, examples, and references. Skills are designed to load progressively so they don't exhaust the AI's context window: * **Metadata** (always loaded, \~100 words) — the name and description in `SKILL.md` frontmatter. This is the triggering mechanism: if the description matches the user's request, the AI activates the skill and reads the full `SKILL.md`. * **SKILL.md body** (loaded on activation) — overview, workflow steps, and pointers to supporting files. Tells the AI what to do and where to look next. * **Supporting files** (loaded on demand) — individual rule files organized by category, `examples.md`, `reference.md`. The AI reads only the files relevant to the current task. ``` skills/infrahub-managing-schemas/ ├── SKILL.md # Entry point — overview and workflow ├── examples.md # Concrete schema examples ├── reference.md # Attribute types, relationship kinds ├── validation.md # Schema validation rules └── rules/ ├── naming.md # Naming convention rules ├── relationships.md # Relationship identifier rules ├── generics.md # Generic/hierarchy rules └── ... ``` ## How skills are triggered[​](#how-skills-are-triggered "Direct link to How skills are triggered") The AI infers whether a skill is relevant by matching the user's request against each skill's description. When a user asks "create a schema for VLAN management," the AI recognizes this matches the Schema Manager skill, activates it, and reads its `SKILL.md` to understand how to proceed. Users can also invoke skills directly by name in Claude Code: ``` /infrahub:managing-schemas /infrahub:managing-objects /infrahub:managing-checks ``` For tools other than Claude Code, skill activation is passive — the AI finds skill files through its normal file context and applies them when the request is relevant. No explicit invocation is needed. ## Auto-detection (Claude Code plugin)[​](#auto-detection-claude-code-plugin "Direct link to Auto-detection (Claude Code plugin)") When installed as a Claude Code plugin, a `SessionStart` hook runs on project open. The hook checks for Infrahub project markers: * `.infrahub.yml` * `infrahub.toml` * Schema files containing `version: "1.0"` with `nodes:` or `generics:` keys If markers are found, the hook outputs context that tells the AI which skills are available and how to use them — before the user types anything. With other AI tools (npx or manual copy), discovery happens because the skill files are present in the project directory. The AI finds them through its normal file context when the user's request is relevant. ## The infrahub-common library[​](#the-infrahub-common-library "Direct link to The infrahub-common library") A shared set of references available to all skills. Skills reference these files rather than duplicating the content, which keeps rules consistent across the entire package. Contents: * **GraphQL query syntax** — how to structure queries for Infrahub's API * **`.infrahub.yml` configuration format** — how to register checks, generators, transforms, and artifact definitions * **YAML structure** — correct structure for schema and object definition files * **Profiles and Object Templates** — when to use shared default values (profiles) vs. cloneable object structures (templates), and their mechanics * **Shared rules** — git integration patterns, protocol/generated file handling, Python environment detection, and display label caching ## Context efficiency[​](#context-efficiency "Direct link to Context efficiency") Each skill is self-contained but references shared resources by file path rather than embedding them. When the AI activates a skill, it loads: 1. The `SKILL.md` body (\~500–1000 words) 2. Only the rule files relevant to the current task 3. Shared references from `infrahub-common` as needed This progressive loading means a simple task like "add an attribute to this node" loads far less context than a complex task like "design a new schema domain with generics and hierarchies." The AI manages this automatically. ## Supported AI tools[​](#supported-ai-tools "Direct link to Supported AI tools") | AI Tool | Installation Method | Auto-detection | Direct Skill Invocation | | -------------------- | ---------------------------------- | ------------------ | ---------------------------------------- | | Claude Code (plugin) | `/plugin install infrahub@opsmill` | Yes (session hook) | Yes (`/infrahub:managing-schemas`, etc.) | | Claude Code (npx) | `npx skills add` | No | Yes | | GitHub Copilot | `npx skills add` | No | No — passive only | | Cursor | `npx skills add` | No | No — passive only | | Windsurf | `npx skills add` | No | No — passive only | | Amp | `npx skills add` | No | No — passive only | | Cline | `npx skills add` | No | No — passive only | | Codex | `npx skills add` | No | No — passive only | **Auto-detection** means the tool automatically recognizes an Infrahub project and activates skills without the user explicitly asking. This is a Claude Code plugin feature via the session start hook. **Direct skill invocation** means the user can call a skill by name (e.g., `/infrahub:managing-schemas`). This is a Claude Code feature. All other tools are passive — they read skill content as context when the request is relevant. For passive tools, describe what you want and the AI applies the right skill automatically. --- # Installation & Setup Infrahub Skills can be installed into any AI tool that supports skills or custom context files. Installation takes one command for most setups — install into a specific Infrahub repository to work in that project, or install globally to use the skills across all your projects. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * **AI coding assistant** — any tool that supports the Agent Skills format or reads custom context files: Claude Code, GitHub Copilot, Cursor, Windsurf, Amp, Cline, Codex, and others * **Infrahub instance** — version 1.7 or later (earlier versions may work but are not tested). See the [Infrahub installation docs](https://docs.infrahub.app/) for setup instructions * **infrahubctl** — required for loading generated schemas, objects, and running generators. See the [infrahubctl docs](https://docs.infrahub.app/infrahubctl) * **Infrahub MCP server** (optional) — required only for the [Data Analyzer](/skills/skills-reference/analyzing-data.md) skill, beneficial but not required for other skills. See the [MCP server docs](https://docs.infrahub.app/mcp) ## Installation[​](#installation "Direct link to Installation") * npx (Recommended) * Claude Code Plugin * Manual Copy The `npx` installer detects which AI tools you have configured and installs the skills in the correct format for each one automatically. ``` npx skills add opsmill/infrahub-skills ``` The installer determines whether to install the skills in the current project directory or globally. If installing per-project, run the command from your Infrahub project root. Install from the Claude Code plugin marketplace for a global installation that works across all Infrahub projects: ``` /plugin marketplace add opsmill/claude-marketplace /plugin install infrahub@opsmill ``` The plugin auto-detects Infrahub projects on session start by looking for `.infrahub.yml`, `infrahub.toml`, or schema file markers in the project directory. For any tool that reads files from the project directory: ``` git clone https://github.com/opsmill/infrahub-skills.git cp -r infrahub-skills/skills /path/to/your-project/ rm -rf infrahub-skills ``` warning Always include `skills/infrahub-common/` — it contains shared references that all skills depend on. Omitting it will cause skills to produce incomplete or incorrect output. ## Tool-specific configuration[​](#tool-specific-configuration "Direct link to Tool-specific configuration") After installing via npx or manual copy, some tools benefit from a small configuration file to activate the skills more reliably. These tools will use the skills without these files, but adding them improves consistency. ### GitHub Copilot[​](#github-copilot "Direct link to GitHub Copilot") Create `.github/instructions/infrahub.instructions.md` in your project root: ``` --- applyTo: "**" --- Use the skills in the `skills/` directory when working on Infrahub schemas, objects, checks, generators, transforms, or menus. Each skill's SKILL.md file describes when and how to use it. ``` ### Cursor[​](#cursor "Direct link to Cursor") Create `.cursor/rules/infrahub.mdc` in your project root: ``` --- description: Infrahub development skills for schemas, objects, checks, generators, transforms, and menus globs: "**/*.{yml,yaml,py,gql,j2}" alwaysApply: false --- Use the skills in the `skills/` directory when working on Infrahub resources. Each skill's SKILL.md file describes when and how to use it. ``` ### Windsurf[​](#windsurf "Direct link to Windsurf") Windsurf reads Markdown files from the project automatically. Optionally, reference the skills from `.windsurfrules`: ``` For Infrahub development tasks, use the skills in the `skills/` directory. Each skill's SKILL.md file describes when and how to use it. ``` ### Claude Code (plugin)[​](#claude-code-plugin "Direct link to Claude Code (plugin)") No additional configuration required after plugin install. The session hook fires on project open and provides context about available skills automatically. ## MCP server setup (Data Analyzer skill)[​](#mcp-server-setup-data-analyzer-skill "Direct link to MCP server setup (Data Analyzer skill)") The [Data Analyzer](/skills/skills-reference/analyzing-data.md) skill uses the Infrahub MCP server to query live data from a running instance. All other skills work without it. See the [Infrahub MCP server docs](https://docs.infrahub.app/mcp) for setup instructions. tip Without MCP, the AI can still query Infrahub data by constructing GraphQL API calls directly. The interaction is more verbose, but functional. ## Verifying the installation[​](#verifying-the-installation "Direct link to Verifying the installation") 1. **Check the files are present** — confirm that `skills/*/SKILL.md` files exist in your project directory (or globally, depending on your install method). 2. **Test skill activation** — open the project in your AI tool and ask: > What Infrahub skills do you have available? The AI should list the installed skills. 3. **Test skill output** — ask the AI to create a simple schema: > Create a Device node with a name and status attribute. If the skills are active, the output should: * Start with `version: "1.0"` * Use `kind: Dropdown` for the status field with choice objects * Include `human_friendly_id` and `display_label` Without the skills, most AI assistants produce a simpler or incorrect schema that misses these conventions. ## Upgrading[​](#upgrading "Direct link to Upgrading") * npx * Claude Code Plugin * Manual ``` npx skills update ``` ``` /plugin update infrahub@opsmill ``` Pull the latest from GitHub and re-copy: ``` git clone https://github.com/opsmill/infrahub-skills.git cp -r infrahub-skills/skills /path/to/your-project/ rm -rf infrahub-skills ``` --- ## [📄️Release 1.2.8](/skills/release-notes/release-1_2_8.md) [Four new skills — NetBox device-type conversion, concept tutoring, diagnostic-bundle triage, and skill-gap reporting — plus verified generic and cardinality semantics and a corrected infrahubctl command surface.](/skills/release-notes/release-1_2_8.md) --- # Release 0.0.1 | Release Number | 0.0.1 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Feature | | Release Date | April 9th, 2026 | | Tag | [v0.0.1](https://github.com/opsmill/infrahub-skills/releases/tag/v0.0.1) | ## Release summary[​](#release-summary "Direct link to Release summary") Describe a schema, object, check, generator, transform, or menu in plain language and get valid Infrahub resources without first mastering the platform's data model. Infrahub Skills is an open-source skills package for AI coding assistants, and this first release ships eight skills across the build, query, and audit lifecycle. ### Build Infrahub resources from natural language[​](#build-infrahub-resources-from-natural-language "Direct link to Build Infrahub resources from natural language") Eight skills cover the Infrahub development lifecycle, one per task, each with the rules, examples, and references for that task. * Generate schemas, objects, checks, generators, transforms, and menus, one skill per task — [managing-schemas](/skills/skills-reference/managing-schemas.md), [managing-objects](/skills/skills-reference/managing-objects.md), and the rest. * Query live data with [analyzing-data](/skills/skills-reference/analyzing-data.md), and audit a repository against Infrahub conventions with [auditing-repo](/skills/skills-reference/auditing-repo.md), which checks details that span files, such as whether query names match between Python and `.infrahub.yml`. * Apply conventions verified against the server: a relationship to a generic needs an explicit `kind:` and `data:` wrapper, `regex` / `min_length` / `max_length` move into a `parameters` block, and generated SDK protocol files are never edited manually. ### Let the right skill activate automatically inside an Infrahub repository[​](#let-the-right-skill-activate-automatically-inside-an-infrahub-repository "Direct link to Let the right skill activate automatically inside an Infrahub repository") Open an Infrahub project and the matching skill activates for the task at hand. Installed as a Claude Code plugin, a session-start hook detects the project automatically; with other tools, the skills are present in the project directory. * A `SessionStart` hook detects an Infrahub project from `.infrahub.yml`, `infrahub.toml`, or a schema file, then points the assistant at the matching skill. * Each skill description carries explicit `TRIGGER when:` and `DO NOT TRIGGER when:` patterns, so the matching skill activates for a request. * [analyzing-data](/skills/skills-reference/analyzing-data.md) and [auditing-repo](/skills/skills-reference/auditing-repo.md) run in a forked context, so their large reports run separately from the main conversation. ### Work on large schema and generator tasks without exhausting context[​](#work-on-large-schema-and-generator-tasks-without-exhausting-context "Direct link to Work on large schema and generator tasks without exhausting context") Only the guidance the current step needs is loaded, leaving room in the context for the work itself. * Skill metadata decides whether a skill applies; the `SKILL.md` body supplies the workflow; rule and reference files load only when a step needs them. * Each skill includes a step-by-step `## Workflow` that names the exact rule to read at each step. ## Minor changes[​](#minor-changes "Direct link to Minor changes") **Testing** * A skillgrade evaluation suite with deterministic Python graders checks each skill's behavior, including guards against tests that would pass on empty output. **Developer experience** * A shared rule protects generated SDK protocol files from manual edits. * `infrahubctl info` connectivity and version checks (with uv and Poetry environment detection) validate output against a running server. * CI automation: version auto-bump from PR labels, release-drafter notes, and a documentation build check. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * Eight skills: managing-schemas, managing-objects, managing-checks, managing-generators, managing-transforms, managing-menus, auditing-repo, and analyzing-data ([#2](https://github.com/opsmill/infrahub-skills/pull/2), [#6](https://github.com/opsmill/infrahub-skills/pull/6), [#7](https://github.com/opsmill/infrahub-skills/pull/7)). * Project detection via a `SessionStart` hook ([#1](https://github.com/opsmill/infrahub-skills/pull/1)), and the skillgrade evaluation harness ([#13](https://github.com/opsmill/infrahub-skills/pull/13)). * Apache 2.0 license, with setup guidance for Claude Code, GitHub Copilot, and Cursor ([#3](https://github.com/opsmill/infrahub-skills/pull/3)). * CI automation: auto-bump, release-drafter, and documentation build ([#40](https://github.com/opsmill/infrahub-skills/pull/40)). See the [full list of merged pull requests](https://github.com/opsmill/infrahub-skills/releases/tag/v0.0.1). ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data --- # Release 1.2.0 | Release Number | 1.2.0 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Maintenance | | Release Date | April 10th, 2026 | | Tag | [v1.2.0](https://github.com/opsmill/infrahub-skills/releases/tag/v1.2.0) | ## Release summary[​](#release-summary "Direct link to Release summary") The package version moves from `0.0.1` to `1.2.0` ahead of the automated release tooling. No skill changes. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Changed[​](#changed "Direct link to Changed") * Plugin version bumped from `0.0.1` to `1.2.0`, with version metadata synced across every `SKILL.md`, before release-drafter and the auto-bump workflow manage versioning. ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data --- # Release 1.2.1 | Release Number | 1.2.1 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Improvement | | Release Date | April 16th, 2026 | | Tag | [v1.2.1](https://github.com/opsmill/infrahub-skills/releases/tag/v1.2.1) | ## Release summary[​](#release-summary "Direct link to Release summary") Browse the installation guide, how-it-works, and per-skill reference on a documentation site, now available alongside the repository instead of only from inside an AI assistant. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * A documentation site covering installation, how the skills work, and a per-skill reference ([#33](https://github.com/opsmill/infrahub-skills/pull/33)). ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data --- # Release 1.2.2 | Release Number | 1.2.2 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Improvement | | Release Date | May 11th, 2026 | | Tag | [v1.2.2](https://github.com/opsmill/infrahub-skills/releases/tag/v1.2.2) | ## Release summary[​](#release-summary "Direct link to Release summary") Model schema patterns common in production OpsMill repositories so they pass Infrahub's validators on the first load. You can now express constructs that often fail validation — computed attributes, branch behavior, delete handling, menu placement, artifact targets, and object templates — and define relationships and register checks the way the server-side validator expects. ### Model schema patterns common in production repositories[​](#model-schema-patterns-common-in-production-repositories "Direct link to Model schema patterns common in production repositories") When you model a schema with one of these constructs, you no longer have to infer the shape Infrahub expects — you get it from the managing-schemas skill, with compliant and non-compliant examples. * Define an attribute whose value is computed from other fields with a Jinja2 expression (`computed_attribute`, paired with `read_only: true`), instead of storing it directly. * Mark an attribute `branch: agnostic` so its value applies across every branch, or leave it branch-aware (the default) so changes stay scoped to a branch until merge. * Set a relationship's `on_delete` to `cascade` or `no-action` to control whether deleting a node also deletes its related nodes. * Control where a node appears in the sidebar with `menu_placement`, instead of relying on the default position. * Extend `CoreArtifactTarget` so a node can be rendered against by artifact definitions, and set `generate_template: true` to create nodes from a reusable template. * Define a `kind: Parent` relationship as `optional: false` and `cardinality: one`, with at most one per node, so it matches the server-side validator. ## Bug fixes[​](#bug-fixes "Direct link to Bug fixes") * `check_definitions` does not accept a `query:` field, unlike `generator_definitions`. The query binds from the Python class's `query` attribute, which must match a `name` under the top-level `queries:` section; the configuration model forbids extra keys, so a stray `query:` makes the repository configuration fail validation. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * Six schema rules from production repositories: computed attributes, branch behavior, relationship `on_delete`, menu placement, `CoreArtifactTarget`, and object templates ([#41](https://github.com/opsmill/infrahub-skills/pull/41)). ### Changed[​](#changed "Direct link to Changed") * `kind: Parent` constraints tightened to match the server-side validator ([#41](https://github.com/opsmill/infrahub-skills/pull/41)). ### Fixed[​](#fixed "Direct link to Fixed") * Clarified that `check_definitions` has no `query:` field ([#41](https://github.com/opsmill/infrahub-skills/pull/41)). ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data --- # Release 1.2.3 | Release Number | 1.2.3 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Improvement | | Release Date | May 11th, 2026 | | Tag | [v1.2.3](https://github.com/opsmill/infrahub-skills/releases/tag/v1.2.3) | ## Release summary[​](#release-summary "Direct link to Release summary") Every `managing-*` skill now appears after a fresh install. A non-standard frontmatter field had hidden six of them from Claude Code's invocable list. ## Bug fixes[​](#bug-fixes "Direct link to Bug fixes") * Removed the non-standard `paths:` frontmatter from the six `managing-*` skills. Claude Code's loader treated `paths:` as unsupported and silently hid those skills — `/infrahub-managing-schemas` returned "Unknown skill" after a fresh install. All six now appear and invoke. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Fixed[​](#fixed "Direct link to Fixed") * Removed unsupported `paths:` frontmatter that hid the `managing-*` skills ([#44](https://github.com/opsmill/infrahub-skills/pull/44)). ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data --- # Release 1.2.4 | Release Number | 1.2.4 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Improvement | | Release Date | May 11th, 2026 | | Tag | [v1.2.4](https://github.com/opsmill/infrahub-skills/releases/tag/v1.2.4) | ## Release summary[​](#release-summary "Direct link to Release summary") Ask analyzing-data a question and get an answer to that question. When the question was omitted, it had answered a built-in example instead. ## Bug fixes[​](#bug-fixes "Direct link to Bug fixes") * [analyzing-data](/skills/skills-reference/analyzing-data.md) now requires the question as `args`. It runs in a forked context with no view of the parent conversation; without `args` it had used the example question built into the skill. It now stops with a re-invoke instruction instead of answering a different question. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Fixed[​](#fixed "Direct link to Fixed") * Require the question as `args` for analyzing-data ([#45](https://github.com/opsmill/infrahub-skills/pull/45)). ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data --- # Release 1.2.5 | Release Number | 1.2.5 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Feature | | Release Date | May 26th, 2026 | | Tag | [v1.2.5](https://github.com/opsmill/infrahub-skills/releases/tag/v1.2.5) | ## Release summary[​](#release-summary "Direct link to Release summary") You can now report Infrahub bugs and feature requests without first working out which repository owns them or manually redacting your terminal output. ### Report bugs and feature requests across the Infrahub ecosystem[​](#report-bugs-and-feature-requests-across-the-infrahub-ecosystem "Direct link to Report bugs and feature requests across the Infrahub ecosystem") Report an issue from your AI assistant with the reporting-issues skill, and it is routed to the right repository, filled into that repository's template, and cleared of sensitive data before submission. * Route a report to the correct repository in the Infrahub ecosystem, defaulting to `opsmill/infrahub` when the target is unclear. * Run a deterministic sanitization pass that replaces IPs, internal hostnames, URLs, and filesystem paths, and stops to ask you when it detects a token or database connection string. * Stop at a review step instead of submitting automatically — choose `gh`, the MCP server, or manual submission. ## Minor changes[​](#minor-changes "Direct link to Minor changes") **Documentation** * Infrahub schema string-length limits documented in the schema skill ([#48](https://github.com/opsmill/infrahub-skills/pull/48)). ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * `infrahub-reporting-issues` skill ([#47](https://github.com/opsmill/infrahub-skills/pull/47)). ### Changed[​](#changed "Direct link to Changed") * Dropped the stale `CHANGELOG.md`; release-drafter now owns release notes ([#50](https://github.com/opsmill/infrahub-skills/pull/50)). ### Fixed[​](#fixed "Direct link to Fixed") * Typo in the skills update instructions ([#51](https://github.com/opsmill/infrahub-skills/pull/51)). ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data * infrahub-reporting-issues --- # Release 1.2.6 | Release Number | 1.2.6 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Feature | | Release Date | June 15th, 2026 | | Tag | [v1.2.6](https://github.com/opsmill/infrahub-skills/releases/tag/v1.2.6) | ## Release summary[​](#release-summary "Direct link to Release summary") After upgrading, data changes target a dedicated branch by default, you can model file attachments with `CoreFileObject`, and you can collect a redacted diagnostic bundle to share when you escalate an issue to OpsMill. ### Keep data changes on a branch you can review or roll back[​](#keep-data-changes-on-a-branch-you-can-review-or-roll-back "Direct link to Keep data changes on a branch you can review or roll back") Data changes now target a dedicated branch by default instead of the default branch, so you can review them through a proposed change or discard the branch instead of reverting each object manually. * Direct every write path — `infrahubctl object load`, the MCP `infrahub_create` / `infrahub_update` / `infrahub_delete` tools, `infrahubctl generator run`, and the Python SDK — to a dedicated branch instead of the default branch (`main` by convention, configurable per deployment). * A shared rule in `infrahub-common` covers when a direct write to the default branch is and is not appropriate, so the assistant defaults to a branch even when a request asks to write directly. ### Model file attachments with `CoreFileObject`[​](#model-file-attachments-with-corefileobject "Direct link to model-file-attachments-with-corefileobject") Store documents and binary files — diagrams, certificates, contracts, rack photos — on Infrahub nodes with the managing-schemas skill, using the built-in `CoreFileObject` generic instead of a text attribute that holds a URL or path. * Make a node a file-bearing entity by inheriting `CoreFileObject`; Infrahub adds a `file:` argument to the node's create, update, and upsert mutations so a file can be uploaded against it. * An audit rule flags reserved-attribute collisions (`file_name`, `file_size`, `file_type`, `checksum`, `storage_id`) and the antipattern of storing a file as a URL or path in a `kind: Text` attribute. ### Collect a redacted diagnostic bundle to share with OpsMill[​](#collect-a-redacted-diagnostic-bundle-to-share-with-opsmill "Direct link to Collect a redacted diagnostic bundle to share with OpsMill") Assemble a redacted, manifest-described diagnostic bundle from a local deployment with the infrahub-collecting-diagnostics skill when you escalate an issue to OpsMill. * Collect from Docker Compose or Kubernetes deployments, including multi-replica setups where one replica's missing logs can hide a bug, and read instance state through `infrahubctl` only. * Apply tiered redaction before producing the bundle, and describe its contents in a `manifest.yml`. ## Bug fixes[​](#bug-fixes "Direct link to Bug fixes") * Fixed integration-layer gaps found in a live build: union-typed GraphQL queries missing inline fragments, relationship references passed as bare strings or mis-packed lists, natural-key collisions on create, and artifact regeneration treated as synchronous (managing-generators, managing-transforms). * Corrected owner and source metadata semantics: source records lineage only, while `owner` together with `is_protected` controls write access. * Verified content-type allowlists, `extensions:` scope, `__typename` resolution, and the Jinja2 filter sandbox against Infrahub source or a running server. ## Minor changes[​](#minor-changes "Direct link to Minor changes") **Documentation** * Resources testing-framework documentation added to the check and transform skills. **Developer experience** * "Information Priority" guidance moved to a shared rule inherited by every skill. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * `infrahub-collecting-diagnostics` skill ([#54](https://github.com/opsmill/infrahub-skills/pull/54)). * `CoreFileObject` schema and audit rules ([#55](https://github.com/opsmill/infrahub-skills/pull/55)). ### Changed[​](#changed "Direct link to Changed") * Branch-default across object loads, MCP CRUD, generators, and the SDK ([#56](https://github.com/opsmill/infrahub-skills/pull/56)). * "Information Priority" moved to a shared rule ([#57](https://github.com/opsmill/infrahub-skills/pull/57)). ### Fixed[​](#fixed "Direct link to Fixed") * Integration-layer fixes from a live build ([#49](https://github.com/opsmill/infrahub-skills/pull/49), [#53](https://github.com/opsmill/infrahub-skills/pull/53)). * Owner and source metadata semantics corrected ([#27](https://github.com/opsmill/infrahub-skills/pull/27)). * Testing-framework documentation added to the check and transform skills ([#18](https://github.com/opsmill/infrahub-skills/pull/18)). ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data * infrahub-reporting-issues * infrahub-collecting-diagnostics --- # Release 1.2.7 | Release Number | 1.2.7 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Feature | | Release Date | July 27th, 2026 | | Tag | [v1.2.7](https://github.com/opsmill/infrahub-skills/releases/tag/v1.2.7) | ## Release summary[​](#release-summary "Direct link to Release summary") After upgrading, you can import CSV or TSV exports as Infrahub objects on a reviewable branch, model shared defaults and repeated structure with Profiles and Object Templates, and collect a diagnostic bundle by running the `infrahub-collect` tool instead of assembling one by hand. ### Import CSV and TSV exports onto a reviewable branch[​](#import-csv-and-tsv-exports-onto-a-reviewable-branch "Direct link to Import CSV and TSV exports onto a reviewable branch") Turn a spreadsheet export into Infrahub objects with the new [importing-data](/skills/skills-reference/importing-data.md) skill: it converts the input into object YAML that `infrahubctl object load` accepts and loads it onto a fresh branch you can review through a proposed change. * Import a single file, a folder of files, an explicit list of paths, or one large denormalized sheet, splitting the denormalized sheet across multiple kinds. * Map columns against the live schema only: a column with no schema home stops the import and lists the offending columns, and the schema decision is routed to managing-schemas — the skill never edits the schema and never writes to the default branch. ### Model shared defaults and repeated structure with Profiles and Object Templates[​](#model-shared-defaults-and-repeated-structure-with-profiles-and-object-templates "Direct link to Model shared defaults and repeated structure with Profiles and Object Templates") Enable and use Infrahub Profiles and Object Templates across the schema, data, and audit skills. The two features are easy to conflate, so the guidance leads with the distinction and keeps it consistent everywhere: Profiles share default values, Object Templates clone structure. * Enable the features on a node with `generate_profile` and `generate_template` using [managing-schemas](/skills/skills-reference/managing-schemas.md), including how Object Templates handle `Component` children and `member_of_groups_for_instances` propagation, and which fields a Profile cannot carry (unique and human-friendly-ID attributes; values are static only). * Assign `profiles:` on objects with the documented override precedence, create objects from an `object_template`, and author the `Template` objects themselves with [managing-objects](/skills/skills-reference/managing-objects.md). * Four new audit rules flag a profile used where a plain schema default would do, a generator that rebuilds what a template could stamp out, an enabled `generate_*` flag no data uses, and templates and profiles used in each other's role — each paired with a creation-time nudge in the authoring skills. ### Find custom Python that a cheaper layer could replace[​](#find-custom-python-that-a-cheaper-layer-could-replace "Direct link to Find custom Python that a cheaper layer could replace") The [auditing-repo](/skills/skills-reference/auditing-repo.md) skill now grades checks, transforms, and generators against a cost-to-fix ladder — drop it, reuse a built-in primitive, use a schema feature, traverse a relationship, use Jinja2, use GraphQL with a thin assert, and only then minimum Python — so an audit shows where a schema feature or a query could replace custom code. * YAGNI findings are advisory — the code works and the pipeline passes — so they never exceed MEDIUM severity; HIGH and CRITICAL stay reserved for broken references, silent failures, and deprecated fields. A one-line or structural fix (inherit a built-in, move data to YAML, add a schema constraint) grades MEDIUM; a larger rewrite where the Python is more defensible grades LOW. * "Before writing Python" guidance in the checks, transforms, and generators skills steers a new artifact toward the cheaper layer before the code is written, not only at audit time. ### Collect a diagnostic bundle by running infrahub-collect[​](#collect-a-diagnostic-bundle-by-running-infrahub-collect "Direct link to Collect a diagnostic bundle by running infrahub-collect") The [collecting-diagnostics](/skills/skills-reference/collecting-diagnostics.md) skill now installs and drives the released [`infrahub-collect`](https://docs.infrahub.app/backup/guides/install-collect) tool, which performs deployment detection, every-replica log collection, key-name redaction, and the bundle manifest natively — instead of walking you through assembling the bundle by hand. * Install and verify the binary, detect the Docker Compose or Kubernetes environment, and run `infrahub-collect create` with flags matched to the reported symptom. * Authenticate through your existing Docker or kubectl access — the collection no longer asks for an Infrahub API token. * Review the bundle contents before sharing: the skill keeps that human gate the tool cannot perform, then hands off to OpsMill support or cross-links reporting-issues when the hand-off becomes a public issue. ### Get documentation-backed answers for tasks the skills do not cover[​](#get-documentation-backed-answers-for-tasks-the-skills-do-not-cover "Direct link to Get documentation-backed answers for tasks the skills do not cover") When a request is Infrahub-related but no loaded skill's rules cover it — deleting nodes, git-integration semantics — the assistant now looks the answer up in the official documentation instead of answering from training recall. * The fallback finds the matching page through the docs' `llms.txt` index, reads that page's Markdown twin, and answers with a citation and a caveat that the point is outside the skill's tested rules. * A version-matched skill rule always wins over the documentation, and on a network error or a missing page the assistant flags the answer as unverified. ### Build generators that reuse query results instead of requesting each object[​](#build-generators-that-reuse-query-results-instead-of-requesting-each-object "Direct link to Build generators that reuse query results instead of requesting each object") Build a generator with the managing-generators skill so that, when its query already returns the related objects it needs, the generated code builds them from that response instead of fetching each one again. * Create related objects from the data already in the query response with `from_graphql`, instead of calling `client.get` once per object. * Follow the query-coverage requirement so the response carries everything those objects need: `__typename`, `id`, and every attribute the generator changes. * Use the before/after example in [managing-generators](/skills/skills-reference/managing-generators.md) to refactor an existing generator. ## Bug fixes[​](#bug-fixes "Direct link to Bug fixes") * The analyzing-data skill now references the Infrahub MCP server's v1.1.7 tool surface (`get_nodes`, `search_nodes`, `query_graphql`, `node_upsert`, and the session-branch tools) instead of tool names that current servers no longer expose ([#68](https://github.com/opsmill/infrahub-skills/pull/68)). ## Minor changes[​](#minor-changes "Direct link to Minor changes") **Documentation** * Curated release notes for every release from v0.0.1 onward published on the docs site ([#62](https://github.com/opsmill/infrahub-skills/pull/62)). **Maintenance** * Docs-site npm dependencies patched for all Dependabot-reported vulnerabilities ([#59](https://github.com/opsmill/infrahub-skills/pull/59)). ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * `infrahub-importing-data` skill ([#63](https://github.com/opsmill/infrahub-skills/pull/63)). * Profiles and Object Templates coverage across managing-schemas, managing-objects, auditing-repo, and a shared reference ([#71](https://github.com/opsmill/infrahub-skills/pull/71)). * YAGNI / cost-to-fix audit rules and creation-time nudges ([#64](https://github.com/opsmill/infrahub-skills/pull/64)). * `llms.txt` documentation-fallback procedure in `infrahub-common` ([#73](https://github.com/opsmill/infrahub-skills/pull/73)). * `from_graphql` rule for managing-generators ([#58](https://github.com/opsmill/infrahub-skills/pull/58)). * Curated release notes on the docs site ([#62](https://github.com/opsmill/infrahub-skills/pull/62)). ### Changed[​](#changed "Direct link to Changed") * `infrahub-collecting-diagnostics` rewritten around the `infrahub-collect` tool ([#72](https://github.com/opsmill/infrahub-skills/pull/72)). ### Fixed[​](#fixed "Direct link to Fixed") * analyzing-data MCP tool references updated to the v1.1.7 surface ([#68](https://github.com/opsmill/infrahub-skills/pull/68)). * Dependabot-reported npm vulnerabilities in the docs site ([#59](https://github.com/opsmill/infrahub-skills/pull/59)). ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data * infrahub-reporting-issues * infrahub-collecting-diagnostics * infrahub-importing-data --- # Release 1.2.8 | Release Number | 1.2.8 | | -------------- | ------------------------------------------------------------------------ | | Release Type | Feature | | Release Date | September 10th, 2026 | | Tag | [v1.2.8](https://github.com/opsmill/infrahub-skills/releases/tag/v1.2.8) | ## Release summary[​](#release-summary "Direct link to Release summary") After upgrading, you can convert NetBox device-type definitions into Infrahub Object Templates, learn Infrahub concepts through your own repository and instance, triage a diagnostic bundle before escalating it, and report a skill's own missing guidance as a reviewed issue. The schema, check, transform, and generator guidance also changed: what a declaration on a generic imposes on every inheriting kind is now stated explicitly, and every `infrahubctl` command the skills print has been checked against the CLI. What to expect after upgrading Most of what changed is what the skills know rather than how you invoke them, so existing schema, object, check, transform, and generator work continues unchanged. One item needs an edit on your side. * Refresh any vendored copy of these skills; a stale copy may still print `infrahubctl` commands that do not exist. See [Upgrade notes](#upgrade-notes) for full details. ### Convert NetBox device-type definitions into Infrahub Object Templates[​](#convert-netbox-device-type-definitions-into-infrahub-object-templates "Direct link to Convert NetBox device-type definitions into Infrahub Object Templates") Turn NetBox device-type and module-type files — the [netbox-community/devicetype-library](https://github.com/netbox-community/devicetype-library) format — into Infrahub Object Templates with the new [converting-netbox-device-types](/skills/skills-reference/converting-netbox-device-types.md) skill. A bundled Python converter reads a mapping profile describing your target schema and writes manufacturers, device types, module types, and component templates as object YAML, alongside a coverage report naming everything it could not map ([#80](https://github.com/opsmill/infrahub-skills/pull/80)). * Convert a single file, a folder, or a mixed tree of device types and module types in one pass. The two families are told apart by `slug`, which every device type carries and no module type does, so a mixed tree does not have to be split up first. Output files are numbered `01_manufacturers` through `05_module_templates` so `infrahubctl object load` runs them in dependency order. * Point the converter at your own schema with a mapping profile rather than editing the script. Every Infrahub kind, attribute, and relationship name is read from a YAML profile, so a custom schema needs a new profile instead of a forked converter. Three profiles ship: `schema-library.yml`, `schema-library-modules.yml`, and an annotated `_template.yml`. * Read the Markdown coverage report before loading anything. Anything the profile cannot map is skipped and named rather than dropped silently, and against the stock `schema-library.yml` profile no published device type converts losslessly, because schema-library models no equivalent node for most NetBox component lists. Turn each report line into a schema change with the worked YAML in `extending-your-schema.md` — the skill offers the change rather than making it, since enabling `generate_template` is a migration against your source of truth. * Create the real interfaces behind a `{module}` token with the bundled module-port generator. The token cannot be resolved at conversion time, so a template carries it and the generator resolves it per installed module after the templates are loaded. ### Learn Infrahub concepts through your own repository and instance[​](#learn-infrahub-concepts-through-your-own-repository-and-instance "Direct link to Learn Infrahub concepts through your own repository and instance") Work through Infrahub concepts — foundations, schema, objects, GraphQL, branches, repository integration, proposed changes, checks, transforms, generators, and menus — with the new [teaching-concepts](/skills/skills-reference/teaching-concepts.md) skill. It asks what you already know before it explains anything, and teaches from your own repository files and read-only instance queries rather than from generic examples ([#118](https://github.com/opsmill/infrahub-skills/pull/118)). * Resume a session instead of restarting it. Progress, lessons, and exercises are recorded in a `.infrahub-learning/` workspace, and each lesson is written to `lessons/.md` in a fixed Probe, Explain, Exercise, Check shape that doubles as the durable learning record. * Follow a dependency-ordered map of eleven concepts. Each carries its own probe questions, exercise specification, verification method, documentation anchors, and a graduation pointer to the skill that does the same work for real. * Attempt an exercise knowing it is solvable. The reference solution is written and verified — through offline schema validation, a read-only query, or an `infrahubctl` run — before the exercise is shown, and a hint ladder runs before any solution is revealed. * Keep your instance safe while learning. Read-only access is the default, authoring exercises are done in local files, and instance writes need an explicit opt-in, run only in a `learning-*` branch, are never merged, and are always deleted. Nothing writes to the default branch. ### Triage a diagnostic bundle before escalating it[​](#triage-a-diagnostic-bundle-before-escalating-it "Direct link to Triage a diagnostic bundle before escalating it") Read a bundle collected by [collecting-diagnostics](/skills/skills-reference/collecting-diagnostics.md) yourself with the new [analyzing-diagnostics](/skills/skills-reference/analyzing-diagnostics.md) skill. It reads the manifest first, sweeps every service's logs for tracebacks, ERROR and CRITICAL lines, OOM kills, and `*.previous.log` restart evidence, then reports correlated incidents with root causes separated from cascades rather than a flat list of errors ([#76](https://github.com/opsmill/infrahub-skills/pull/76)). * Find out whether your crash is already known, and possibly already fixed, before anyone files an issue or waits on support. Findings are matched against existing `opsmill/infrahub` issues using stable search keys — exception class plus normalized message — with volatile tokens such as branch names, UUIDs, and hostnames stripped so a known issue actually matches. A match then resolves against the version the bundle reports running: a newer fix version means upgrade, and already running the fix means an unconfirmed match or a possible regression. * Read every finding against evidence you can check. Each claim cites a bundle path with a quoted excerpt, the report opens with the deployment context — running version, topology, replica counts — and what the bundle cannot answer is reported as an open question mapped to the `infrahub-collect create` flags a next bundle would need. * Search targeted failure patterns instead of generic ones. A known-failure-patterns reference covers Prefect background services disabled by edited Helm values, lock deadlocks after crashed merges, stale RUNNING tasks, and storage-persistence and multi-replica shared-storage misconfigurations, each mapped to where its evidence sits in the bundle. * Keep the analysis read-only toward your deployment. It never files anything: filing hands off to reporting-issues, a second collection hands back to collecting-diagnostics, and both diagnostics skills now frame `--include-backup` as a last resort, with a caution about restored backups carrying live git repository connections. ### Report a skill's missing or wrong guidance as a reviewed issue[​](#report-a-skills-missing-or-wrong-guidance-as-a-reviewed-issue "Direct link to Report a skill's missing or wrong guidance as a reviewed issue") When an Infrahub skill's guidance is missing or wrong, the failure is quiet — extra requests and repeated corrections, never a crash. Diagnose it with the new [reporting-skill-gaps](/skills/skills-reference/reporting-skill-gaps.md) skill, which works out which rule file is at fault and what it should say instead, then hands the draft to [reporting-issues](/skills/skills-reference/reporting-issues.md) for everything GitHub-facing ([#83](https://github.com/opsmill/infrahub-skills/pull/83), [#119](https://github.com/opsmill/infrahub-skills/pull/119)). * Rest a report on evidence rather than an impression. Four probes run strongest first: a verifier that failed and later passed, a coverage read of the skill's `rules/` naming the file or its absence, the difference between the rejected and the accepted artifact, and retry counts. A draft resting only on retry counters is refused, and the refusal names the probe that came up empty. * See the report classified before it is filed. A case a rule covers where the guidance still produced the wrong answer is a bug; a case no rule covers where the documentation supplied the answer is a feature request; a case no rule covers where the documentation did not is a documentation gap. The discriminator is what happened after the escape to documentation, not whether documentation existed. * Read which revision of the guidance failed from the report header, which records the skills-plugin version alongside the Infrahub SDK and Infrahub versions. All three come from one `infrahubctl info` call, and `unknown` and `n/a` are accepted because offline work never reaches a server. * Keep diagnosis and filing separate. This skill never files and never names a target repository — reporting-issues resolves that from the report type against the registry it already owns — and the tracker search it ran is passed across, so the search is not repeated and answered differently. ### Know what a generic imposes on every kind that inherits it[​](#know-what-a-generic-imposes-on-every-kind-that-inherits-it "Direct link to Know what a generic imposes on every kind that inherits it") Declaring something on a generic behaves differently from declaring it on a kind, and until now [managing-schemas](/skills/skills-reference/managing-schemas.md) described the syntax without the semantics. Model with generics knowing what each declaration imposes on every inheriting kind: every rule below quotes the load-time error its violation produces, proven against Infrahub's in-memory schema validator rather than inferred from reading code ([#85](https://github.com/opsmill/infrahub-skills/pull/85), [#111](https://github.com/opsmill/infrahub-skills/pull/111), [#112](https://github.com/opsmill/infrahub-skills/pull/112), [#113](https://github.com/opsmill/infrahub-skills/pull/113)). * Scope a `uniqueness_constraints` entry before you declare it. A constraint on a generic is enforced across every kind that inherits it, and a concrete kind cannot narrow what it inherits — declaring its own constraints only adds a kind-scoped check while the generic's still runs. That decides a migration: on the generic, old and new instances collide and deletion becomes an ordering precondition; on the concrete kinds they coexist and the migration stays reversible. * Meet the three preconditions a relationship named in a `uniqueness_constraints` entry has to satisfy — `optional: false`, `cardinality: one`, and the bare relationship name with no peer-attribute path — each with its verbatim error string and a loadable example. A `human_friendly_id` is compiled into a uniqueness constraint too, so an optional relationship reached by an HFID path fails schema load with a `uniqueness_constraints:` error that names a field you never wrote. * Override a `Dropdown` attribute on an implementer without widening it by accident. A shorter `choices` list loads cleanly, and so does a longer one carrying an invented choice, so a concrete kind can hold a value the generic says is impossible and a query over the generic can return it. Guard it with a test comparing each restated list against the generic's, or with `allow_override: none` on the generic's attribute, which rejects any override and trades away per-kind defaults. * Treat a generic's implementer set as an interface its consumers already read. A brand-new kind inheriting a generic produces no migration and no constraint validation, so every offline gate stays green while queries, groups, menus, and constraints declared on the generic include the new kind. An offline test over the schema YAML asserting the expected implementer set is the gate a contributor can break, and it runs in under a second. * Order results within what a generic can resolve. `order_by` on a generic resolves against that generic's own declarations only, and `node_metadata__created_at` and `node_metadata__updated_at` stay valid even on a generic that declares no attributes at all. The error may name an implementer rather than the generic, because the value is copied down to every kind that declares none of its own — so when a kind whose file contains no `order_by` is named, look at its generics. ### Change a relationship's cardinality without breaking the queries that read it[​](#change-a-relationships-cardinality-without-breaking-the-queries-that-read-it "Direct link to Change a relationship's cardinality without breaking the queries that read it") Setting `cardinality: one`, hitting a write-time failure on the second object, and widening the relationship is a common sequence, and its second half breaks every stored query that selects that relationship. Both halves are now documented across the schema and query rules, read from Infrahub v1.10.8 source with every schema outcome reproduced by running that checkout's validator ([#114](https://github.com/opsmill/infrahub-skills/pull/114)). * Widen the side that actually holds the cap. Your own `cardinality: one` limits what you hold; how many objects may point at a peer is set by that peer's reciprocal declaration on the same identifier, and where the peer declares nothing there is no inbound cap at all. Widening only your own side loads cleanly and leaves the write-time failure exactly where it was. * Rewrite the query the change invalidates. `one` generates `NestedEdged`, selected as `{ node { … } }`; `many` generates `NestedPaginated`, selected as `{ edges { node { … } } }` with `count`. Widening leaves a stored query selecting `{ node … }` against a now-paginated field, which returns `Cannot query field 'node' on type 'NestedPaginated'` — and that type name appears in no schema file, `.gql` file, or documentation page, so the error is hard to connect to the cardinality edit that caused it. * Express a genuine cap of one with `cardinality: one`. `max_count: 1` on a cardinality-many relationship is rejected outright. A hierarchical kind is the exception to all of this: its generated fields ignore what you declared, with `parent` always node-shaped and `children`, `ancestors`, and `descendants` always edges-shaped. ### Reuse a schema the marketplace already publishes[​](#reuse-a-schema-the-marketplace-already-publishes "Direct link to Reuse a schema the marketplace already publishes") Check whether the Infrahub marketplace already publishes a domain before modelling it. A workflow step in managing-schemas checks the catalog first, a shared reference in `infrahub-common` gives every skill the same view of it, and an audit rule in [auditing-repo](/skills/skills-reference/auditing-repo.md) flags a whole domain hand-rolled from scratch when the marketplace publishes it ([#69](https://github.com/opsmill/infrahub-skills/pull/69), [#113](https://github.com/opsmill/infrahub-skills/pull/113)). * Search the whole catalog for any domain with `infrahubctl marketplace list`, `search`, and `show`, and fetch with `infrahubctl marketplace get`, including collections with `-c`. The catalog API covers the discovery the CLI does not expose, and `--marketplace-url` points at an internal mirror; reachability never blocks schema work, since the guidance falls back to a custom schema built on built-in primitives. * Tell the three kinds of availability apart before building on one. Platform core kinds (`Core*`, `Builtin*`, `Ipam*`) exist on a clean instance; marketplace-published kinds do not, and become a dependency your repository carries, loads first, and records a version for; locally defined kinds are yours. Check with `infrahubctl schema list` on the cleanest instance you have rather than the development machine that has accumulated whatever was loaded during experiments — a kind appearing in an example is evidence of a sensible shape, not evidence it exists on your instance. * Evaluate a published file per generic rather than per file. The cost of a candidate is its transitive peer set, so a sibling with expensive peers is no reason to reject a cheap candidate in the same file, and rejecting per file is how a published shape gets reinvented. Taking a subset also loses the `marketplace get` update path, so the rule prefers the whole file when the cost is comparable, and asks for a provenance comment recording the identifier, the version, what was taken, and why the rest was excluded. Before adopting a generic from a published file, read what its implementer set imposes on you in [Know what a generic imposes on every kind that inherits it](#know-what-a-generic-imposes-on-every-kind-that-inherits-it). ### Write checks, transforms, and generators against verified behavior[​](#write-checks-transforms-and-generators-against-verified-behavior "Direct link to Write checks, transforms, and generators against verified behavior") Field reports kept describing one class of problem: a mechanism described correctly in one place and contradicted everywhere an example appeared. The checks, transforms, and generators skills now carry the behavior each pattern depends on, verified against Infrahub source, the SDK, and the runtime image ([#81](https://github.com/opsmill/infrahub-skills/pull/81), [#89](https://github.com/opsmill/infrahub-skills/pull/89), [#115](https://github.com/opsmill/infrahub-skills/pull/115), [#116](https://github.com/opsmill/infrahub-skills/pull/116), [#117](https://github.com/opsmill/infrahub-skills/pull/117)). * Declare a `watch` block on every `python_transforms` and `generator_definitions` entry, including `files: []` when there is nothing beyond the entry point. Infrahub never scans the imports, so with no key it folds the commit id into the definition's fingerprint and the definition re-renders or re-runs on every commit; a present key is your assertion that the list is complete. `jinja2_transforms` are the exception, because their closure comes from parsing the template. No `watch` is noisy but safe, and a wrong `watch` is silent — declaring anything switches off the conservative fallback, and an incomplete list then under-regenerates with no error anywhere. * Validate a node against a related node's state in a [check](/skills/skills-reference/managing-checks.md) by traversing the relationship in the query itself. A check runs its query once and `validate()` receives that single payload with no lazy fetch, so a query selecting only the child has no parent state left to test. The rule covers unwrapping each hop null-safely, surfacing an unresolvable related node as a violation rather than a silent skip, and exempting the root of a containment chain, which has no parent. * Read GraphQL failures from the response body, not the status code. The server hard-codes `status_code=200` on every response whose document executed and puts constraint violations, unknown fields, and permission denials in the body's `errors` array. `self.client.execute_graphql` raises `GraphQLError` on those, so a check using the SDK client is not exposed; the trap is raw `httpx` against `/graphql` branching on the status code. Test the failing path — a check that fails open cannot be told apart from a passing check by observation. * Constrain a path traversal in a [generator](/skills/skills-reference/managing-generators.md) with `relationship_filter`, which takes schema relationship identifiers such as `device__interface` rather than the per-side names that appear in the result. `kind_filter` is a whitelist of node kinds and still lets a shared catalog node of an allowed kind bridge unrelated subgraphs; `included_kinds` re-includes kinds excluded by default and is not a whitelist. Check `truncated_at_depth` in the result, since a non-null value means the search ran out of budget and the answer is incomplete, and note that `shortest_paths_only` defaults to `true` and drops longer routes through the same intermediate objects. * Write a group membership from the side whose peer can be resolved by name. `CoreGroup.members` peers `CoreNode`, which has no attributes, no `default_filter`, and no `human_friendly_id`, so a member name has nothing to resolve against; `member_of_groups` peers `CoreGroup`, which does have `default_filter: name__value`. The general rule: a relationship whose peer is a bare generic cannot be resolved by name in object data. A generator's target group must also exist and be non-empty, because an existing but empty group dispatches nothing and reports no error. * Emit a non-text artifact from a [transform](/skills/skills-reference/managing-transforms.md). A complete `image/svg+xml` example now sits alongside the content-type table — query, transform, `python_transforms` registration, `artifact_definitions` entry with its target group, and the `CoreArtifactTarget` requirement — and the serialization rule is stated as a matrix rather than a return-type column: only `application/json` and `application/yaml` special-case a returned dict, so returning a dict for `text/csv` or `image/svg+xml` writes a Python repr into the artifact body with no error and no warning. Returning nothing is the one case that fails loudly. ### Audit a repository without changing your working tree[​](#audit-a-repository-without-changing-your-working-tree "Direct link to Audit a repository without changing your working tree") An audit is most useful on a tree holding uncommitted work, which is exactly the tree where a write cannot be undone. Run an audit with auditing-repo and it now reads without writing: a CRITICAL conduct rule constrains the auditor rather than the repository, and Phase 0 of the audit procedure states it before the first check runs ([#109](https://github.com/opsmill/infrahub-skills/pull/109)). * Get your uncommitted work back unchanged. In a reported failure, an audit had stashed the tree, run a generator with `--check`, popped the stash, and then run `git checkout -- objects/`, destroying another agent's regenerated files while reporting its findings correctly and never mentioning that it had modified anything. The rule names cleanup itself as the hazard, including cleanup that undoes the auditor's own side effect. * Read the audit's own account of what it did. The report format now carries the tree's condition at audit time and whether the audit modified it, and an honest "check not performed" finding is available for a check that cannot be run without a write — the compliance checks read the list of commands the audit ran rather than its prose, so disclosing what you did not run counts as compliance. * Compare against committed state with read-only git: `git show :`, `git diff -- `, `git cat-file`, and `git ls-tree` answer every "what does this look like as committed?" question without touching the tree. * See where a bare-string `kind` could be a typed protocol class. A new LOW-severity rule at step 7 of the cost-to-fix ladder flags non-trivial schema-coupled Python that passes `client.create` or `client.get` a `kind="Foo"` string, or hand-builds dict payloads, when a generated protocol class exists for that kind. Generate the classes with `infrahubctl protocols`, and a schema change becomes a type error on the exact line instead of a runtime failure in the pipeline. The rule carries its own limits: protocols type attributes, not relationship peers, and trivial one-off scripts are not flagged. It sits at the far end of the same cost-to-fix ladder whose first step is [Reuse a schema the marketplace already publishes](#reuse-a-schema-the-marketplace-already-publishes) ([#79](https://github.com/opsmill/infrahub-skills/pull/79)). ## Bug fixes[​](#bug-fixes "Direct link to Bug fixes") * Every `infrahubctl` command the skills print is now a real one. A sweep found 11 invalid invocations — `schema validate`, `check run `, `transform run `, `generator run `, `generator list`, and `import load` — of which only two had been reported. The offline gate is `infrahubctl schema format --check`, now documented in managing-schemas ([#74](https://github.com/opsmill/infrahub-skills/pull/74)), and the server-side check is `infrahubctl schema check`. `check`, `generator`, and `transform` take their target as a positional argument, so `check run mycheck` had looked for a check literally named `run`, and there is no `import` or `export` command group, since the LDJSON tools are the top-level `load` and `dump`. Every form was verified against `infrahub-sdk` 1.23.1, and a continuous-integration check now gates every invocation the repository prints, including the ones in graders and eval rubrics ([#110](https://github.com/opsmill/infrahub-skills/pull/110)). * `infrahubctl render` cannot serve a Python transform, and the message it returns for one — *"Unable to find \ in repository config file"* — reads like an unregistered transform rather than the wrong command, so a reader concludes the dry-run gate is unavailable and skips it. `render` resolves Jinja2 transforms and `transform` resolves Python ones; the rule now gives both and says which serves which section, plus the two details that each cost a failed run: a required query variable must be typed manually when run locally, because the artifact definition binds it from the target in the pipeline, and neither command reaches a check or a generator ([#110](https://github.com/opsmill/infrahub-skills/pull/110)). * `infrahubctl info` reports a green status with no token, because the user lookup is skipped entirely when neither a token nor a username is set while the server information call succeeds anonymously. A green result therefore does not prove write authorization. The connectivity rule now carries that three-way asymmetry and gives a create-and-delete branch write probe rather than a branch listing, which is a read ([#110](https://github.com/opsmill/infrahub-skills/pull/110)). * The `infrahubctl --version` flag does not exist and errors with "No such option". `infrahubctl info` prints `SDK Version` and `Infrahub Version` together. Three places were wrong, including the shared connectivity rule every skill inherits, which showed output labels that appear nowhere in the real output — so grepping for them found nothing ([#119](https://github.com/opsmill/infrahub-skills/pull/119)). * Retrofitting or changing a relationship `identifier` on a relationship that already exists in an instance produces `'not_supported': None` from `infrahubctl schema check`, and nothing explained it. It is a schema-update immutability error, not an invalid `kind`/`cardinality` pairing: `identifier`, `direction`, `branch`, and `hierarchical` are the relationship fields that cannot be updated, while `kind`, `cardinality`, and `optional` can. Any `kind`/`cardinality` pairing is valid given a shared identifier and a fresh kind. Both the error and its fix are now in the relationship rules and the common-errors lookup table ([#75](https://github.com/opsmill/infrahub-skills/pull/75)). ## Minor changes[​](#minor-changes "Direct link to Minor changes") **Developer experience** * `AGENTS.md` links the `dev/` guides instead of `@`-importing them, so each guide loads only when a task needs it; always-loaded project memory drops from about 50 KB to about 4.5 KB per session ([#77](https://github.com/opsmill/infrahub-skills/pull/77)). * `eval.yaml` loads in `skillgrade` again. Seven grader-less `csv-import-*` tasks had failed configuration validation before any task could run, which blocked all of them for every skill ([#126](https://github.com/opsmill/infrahub-skills/pull/126)). ## Upgrade notes[​](#upgrade-notes "Direct link to Upgrade notes") ### Vendored copies of the skills[​](#vendored-copies-of-the-skills "Direct link to Vendored copies of the skills") If: you copied these skills into a repository rather than installing the plugin. Then: refresh the copy. The invalid command forms corrected here — among them `infrahubctl protocols generate --async`, which is really `infrahubctl protocols` — had already propagated into vendored copies, so a stale copy keeps printing commands that do not exist. Notes: no action is needed if you install the plugin. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * `infrahub-converting-netbox-device-types` skill, with a bundled converter, three mapping profiles, and a module-port generator ([#80](https://github.com/opsmill/infrahub-skills/pull/80)). * `infrahub-teaching-concepts` skill: twelve tutor rules, a dependency-ordered concept map, and a lesson protocol ([#118](https://github.com/opsmill/infrahub-skills/pull/118)). * `infrahub-analyzing-diagnostics` skill for diagnostic-bundle triage ([#76](https://github.com/opsmill/infrahub-skills/pull/76)). * `infrahub-reporting-skill-gaps` skill, filing through `infrahub-reporting-issues` ([#83](https://github.com/opsmill/infrahub-skills/pull/83)). * Marketplace visibility across the schema, objects, generators, and audit skills, plus the `yagni-reuse-existing-marketplace-schema` audit rule ([#69](https://github.com/opsmill/infrahub-skills/pull/69)). * `yagni-untyped-python-vs-generated-protocols` audit rule and generated-protocol adoption guidance ([#79](https://github.com/opsmill/infrahub-skills/pull/79)). * `watch` dependency declarations in `infrahub-common`, managing-transforms, and managing-generators ([#89](https://github.com/opsmill/infrahub-skills/pull/89)). * Read-only audit conduct rule and Phase 0 of the audit procedure ([#109](https://github.com/opsmill/infrahub-skills/pull/109)). * `uniqueness_constraints` semantics, including inheritance scope, the null-collision case, and the relationship preconditions ([#111](https://github.com/opsmill/infrahub-skills/pull/111), [#85](https://github.com/opsmill/infrahub-skills/pull/85)). * Generic inheritance rules for `order_by` scope, `Dropdown` overrides, and relationship peer kinds ([#112](https://github.com/opsmill/infrahub-skills/pull/112)). * Schema reuse rules and the generic-membership interface rule ([#113](https://github.com/opsmill/infrahub-skills/pull/113)). * Cardinality consequences across the schema and GraphQL query rules ([#114](https://github.com/opsmill/infrahub-skills/pull/114)). * `image/svg+xml` artifact example and the artifact serialization matrix ([#115](https://github.com/opsmill/infrahub-skills/pull/115)). * GraphQL error-surface and shared-module rules for managing-checks ([#116](https://github.com/opsmill/infrahub-skills/pull/116)). * Relationship-traversal validation rule for managing-checks ([#81](https://github.com/opsmill/infrahub-skills/pull/81)). * Path traversal, group membership, and shared-object ownership rules for managing-generators ([#117](https://github.com/opsmill/infrahub-skills/pull/117)). * `infrahubctl schema format` documented in managing-schemas ([#74](https://github.com/opsmill/infrahub-skills/pull/74)). * Infrahub SDK and Infrahub versions in the skill-gap report header ([#119](https://github.com/opsmill/infrahub-skills/pull/119)). * `python-lint` job with a pinned ruff rule set ([#128](https://github.com/opsmill/infrahub-skills/pull/128)). * Python test suite in continuous integration ([#80](https://github.com/opsmill/infrahub-skills/pull/80)). ### Changed[​](#changed "Direct link to Changed") * `AGENTS.md` links the `dev/` guides instead of `@`-importing them ([#77](https://github.com/opsmill/infrahub-skills/pull/77)). * Review lessons from pull requests #85 through #117 routed into the `dev/` guides ([#120](https://github.com/opsmill/infrahub-skills/pull/120)). * Evidence discovery in reporting-skill-gaps made assistant-agnostic, and the docs-site listing extended to the Issue Reporter and Skill Gap Reporter ([#119](https://github.com/opsmill/infrahub-skills/pull/119)). ### Fixed[​](#fixed "Direct link to Fixed") * 11 invalid `infrahubctl` invocations, with a continuous-integration gate on every printed invocation ([#110](https://github.com/opsmill/infrahub-skills/pull/110)). * `infrahubctl --version` references replaced with `infrahubctl info` ([#119](https://github.com/opsmill/infrahub-skills/pull/119)). * Shared-identifier relationship rules and the `not_supported` schema-check error explained ([#75](https://github.com/opsmill/infrahub-skills/pull/75)). * Seven grader-less eval tasks that stopped `skillgrade` loading `eval.yaml` ([#126](https://github.com/opsmill/infrahub-skills/pull/126)). * 14 ruff findings on `main` ([#128](https://github.com/opsmill/infrahub-skills/pull/128)). * A fenced-block marker in an eval prompt that desynchronized the CLI invocation sweep ([#130](https://github.com/opsmill/infrahub-skills/pull/130)). * `uv.lock` synced to the project version, removing a stray diff from unrelated pull requests ([#121](https://github.com/opsmill/infrahub-skills/pull/121)). * direnv files ignored ([#82](https://github.com/opsmill/infrahub-skills/pull/82)). ## Skills included[​](#skills-included "Direct link to Skills included") * infrahub-managing-schemas * infrahub-managing-objects * infrahub-managing-checks * infrahub-managing-generators * infrahub-managing-transforms * infrahub-managing-menus * infrahub-auditing-repo * infrahub-analyzing-data * infrahub-reporting-issues * infrahub-reporting-skill-gaps * infrahub-collecting-diagnostics * infrahub-analyzing-diagnostics * infrahub-importing-data * infrahub-teaching-concepts * infrahub-converting-netbox-device-types --- # Data Analyzer The Data Analyzer skill queries and analyzes data from a live Infrahub instance using the Infrahub MCP server. It answers operational questions that span multiple node types — correlating data, detecting drift, tracing service impact, auditing data quality — without requiring the user to write GraphQL queries manually. ## Requirements[​](#requirements "Direct link to Requirements") The Data Analyzer skill requires the **Infrahub MCP server** to be configured and connected to your AI tool. See the [MCP server docs](https://docs.infrahub.app/mcp) for setup instructions. tip Without MCP, the AI can still query Infrahub data by constructing GraphQL API calls directly. The interaction is more verbose and less intuitive, but functional. ## When to use[​](#when-to-use "Direct link to When to use") * Answering ad-hoc operational questions about live infrastructure data * Cross-referencing two or more node types to find relationships or gaps * Investigating the blast radius of a change before executing it * Auditing data quality (missing fields, stale records, naming violations) * Exploring schema structure and data before writing a generator or check * Producing one-time or on-demand reports ## Types of analysis[​](#types-of-analysis "Direct link to Types of analysis") | Analysis type | Example question | | ----------------------- | ---------------------------------------------------------------------------------- | | **Compliance** | "Are all devices following the naming convention `--`?" | | **Service impact** | "Which services are hosted on devices in Rack A-03?" | | **Maintenance windows** | "Which devices are in a maintenance window, and what BGP sessions depend on them?" | | **Drift detection** | "Which realized devices differ from their topology design object?" | | **Capacity** | "Which racks are over 80% utilized?" | | **Inventory gaps** | "Which devices have no platform or OS version recorded?" | | **Change impact** | "What BGP sessions, services, and IP allocations depend on this prefix?" | ## How it works[​](#how-it-works "Direct link to How it works") The skill uses MCP tools (`infrahub_query`, `infrahub_list_schema`, `infrahub_get`) to fetch data from the running instance. For multi-step analysis, queries run sequentially or in combination — data is joined in the AI's context, not in the database. The skill reports findings with counts, lists of affected objects, and suggested next steps. ## Data Analyzer vs. Check Manager vs. Transform Manager[​](#data-analyzer-vs-check-manager-vs-transform-manager "Direct link to Data Analyzer vs. Check Manager vs. Transform Manager") | Use Data Analyzer when... | Use Check Manager when... | Use Transform Manager when... | | --------------------------------- | ------------------------------------ | ------------------------------------------------- | | Answering a one-time question | Enforcing a rule in the pipeline | Producing a repeatable scheduled report | | Interactive, exploratory analysis | Blocking changes that violate policy | Generating artifacts (configs, exports) | | No pipeline integration needed | Automated enforcement required | Output needs to be stored as a versioned artifact | --- # Diagnostics Analyzer The Diagnostics Analyzer reads a bundle the [Diagnostics Collector](/skills/skills-reference/collecting-diagnostics.md) already produced and turns its logs into a triage report. It reads the manifest before any log, sweeps every service for tracebacks and other error signals, correlates related errors into incidents with root causes separated from cascades, and searches existing `opsmill/infrahub` issues so you learn whether your crash is already known — without touching the running deployment, applying a fix, or filing anything on your behalf. ## When to use[​](#when-to-use "Direct link to When to use") * You have a bundle in hand and want to know what it says before involving support * Tracebacks in the logs need interpreting * Deciding whether a crash is a known Infrahub issue, and whether a newer version already fixes it * A container or worker keeps restarting and you need the pre-restart cause * A performance symptom, where a `--benchmark` collection needs reading alongside the logs ## What it produces[​](#what-it-produces "Direct link to What it produces") A findings report grounded in bundle evidence: * A deployment-context header — running Infrahub version, topology (Docker Compose or Kubernetes), replica counts — because the version is what turns a matched issue into a conclusion rather than a coincidence * One section per incident, with severity, the evidence as bundle file paths plus quoted excerpts, and the reasoning that grouped those signals into one incident * Matching `opsmill/infrahub` issues with title, state, and URL, searched across open and closed * Open questions, including whether the symptom reproduces on demand, each mapped to the `infrahub-collect create` flags a next bundle would need to answer it ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "I collected a bundle — can you tell me what's wrong?" * "Analyze these Infrahub logs" * "What do the tracebacks in the bundle mean?" * "Is this crash a known Infrahub issue?" * "Why did the task-worker keep restarting?" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **You name the bundle** — the skill asks for the path instead of scanning for one, because a machine often holds several bundles and picking the wrong one produces a confident report about the wrong incident * **Manifest before any log** — `bundle_information.json` records what was collected and what failed, and a service whose logs could not be collected is often the service that is down, so collection failures are findings rather than gaps in the evidence * **Every service, not just the server** — the sweep covers each directory under `bundle/logs/` for tracebacks, ERROR and CRITICAL lines, panics, OOM kills, and connection failures, and treats any `*.previous.log` as restart evidence whose tail usually holds the crash cause * **Incidents, not error lists** — signals are grouped by timestamp and causal chain, so a database OOM followed by server connection errors is reported as one incident with a named root rather than two problems * **Stable search keys** — a key is built from the exception class, the normalized message, and the innermost Infrahub frame, with branch names, UUIDs, and hostnames stripped, so a known issue actually matches * **Both halves of the tracker** — the issue search covers open and closed issues, because a closed match is the one that tells you which version includes the fix * **Evidence per finding** — every claim cites a bundle path and a quoted excerpt, and what the bundle cannot answer is stated as an open question rather than left out * **Read-only** — the analysis reads an already-collected bundle and runs issue searches; it never touches the running instance, restarts nothing, edits no configuration, and creates no issue ## Diagnostics Collector vs. Diagnostics Analyzer[​](#diagnostics-collector-vs-diagnostics-analyzer "Direct link to Diagnostics Collector vs. Diagnostics Analyzer") | Use the Diagnostics Collector when... | Use the Diagnostics Analyzer when... | | -------------------------------------------------------------- | ---------------------------------------------------------------------------- | | Infrahub is misbehaving and no bundle exists yet | A bundle already exists, or you have its contents | | You need the right `infrahub-collect` command for your symptom | You need to know what the collected logs actually say | | Output is a support bundle on disk | Output is a findings report citing that bundle | | The next step is sharing it with OpsMill | The next step is an upgrade, a comment on an existing issue, or a new report | ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | ----------------------------------------------------------------- | ------------------------------------------------------------------------------- | | Reading only the server's logs | Sweeps every service directory under `bundle/logs/` | | Treating a log that could not be collected as a gap in the bundle | Reports it as a finding — that service is often the one that failed | | Reporting every error line as its own problem | Correlates signals into incidents and names the root | | Searching a traceback verbatim | Strips branch names, UUIDs, and hostnames first, so the search matches | | Limiting the issue search to open issues | Searches open and closed, since a closed match names the fixing version | | Restarting a container to test a theory | Recommends it in the report and applies nothing | | Filing the bug from here | Hands off to the [Issue Reporter](/skills/skills-reference/reporting-issues.md) | tip Benchmark data exists only if the bundle was created with `infrahub-collect create --benchmark`. For a performance symptom without it, the report asks for a new bundle created with that flag rather than guessing from logs alone — the single-CPU score and the storage IOPS of the Neo4j and PostgreSQL volumes are often what decide whether the cause is software or an undersized host. note No bundle yet? Start with the [Diagnostics Collector](/skills/skills-reference/collecting-diagnostics.md). This skill will not scrape `docker compose logs` or `kubectl logs` as a substitute, because a hand-scraped set of logs misses the replica coverage, the previous-container logs, and the manifest that the analysis depends on. --- # Repo Auditor The Repo Auditor performs a comprehensive audit of an Infrahub repository against all Infrahub best practices. It checks schema files, object files, Python components (checks, generators, transforms), `.infrahub.yml` registration, and project structure — then generates a report identifying issues and explaining what to fix and why. ## When to use[​](#when-to-use "Direct link to When to use") * Before loading a new schema into a production instance * After onboarding to verify the repository follows Infrahub conventions * When inheriting or taking over an existing Infrahub implementation * As a periodic quality check on an active repository * Before opening a pull request that touches schema or automation files ## What it checks[​](#what-it-checks "Direct link to What it checks") ### Project structure[​](#project-structure "Direct link to Project structure") * Required files present (`.infrahub.yml`, schema files) * Directory conventions followed * File naming patterns (numeric prefixes on object files, snake\_case on Python files) ### Schema[​](#schema "Direct link to Schema") * Naming conventions (namespace, node name, attribute name lengths and casing) * Attribute types (no deprecated `String`, correct use of `Dropdown`, `Text`, `Number`, etc.) * Relationship identifiers (both sides match) * Relationship peer references (full namespace+name) * `human_friendly_id` present on all nodes * `display_label` present and syntactically correct * Hierarchy configuration (correct `parent`/`children` setup for hierarchical generics) ### Object files[​](#object-files "Direct link to Object files") * Correct `apiVersion`, `kind: Object`, `spec` structure * Value mapping (correct use of `value:`, dropdown names vs. labels) * Load order (no forward references) * Relationship references match existing node `human_friendly_id` values ### Python components[​](#python-components "Direct link to Python components") * **`InfrahubCheck`** — `validate()` signature, `log_error()` vs `log_info()` usage, error collection pattern * **`InfrahubGenerator`** — `allow_upsert=True`, `delete_unused_nodes=True`, async `generate()` method * **`InfrahubTransform`** — correct return types, `data` variable usage in Jinja2 templates ### Cross-references and registration[​](#cross-references-and-registration "Direct link to Cross-references and registration") * Every Python component referenced in `.infrahub.yml` exists as a file * Every `.gql` query referenced in Python classes exists as a file * Query names match between `.gql` files and Python class attributes * Artifact definitions reference existing transforms ### Best practices[​](#best-practices "Direct link to Best practices") * `display_label` caching patterns for nodes with parent relationships * Python environment detection configuration * Git integration setup * `watch` dependency declarations on Python transforms and generators — a missing key re-fingerprints the definition on every commit, and an incomplete list silently under-regenerates ### YAGNI / cost-to-fix[​](#yagni--cost-to-fix "Direct link to YAGNI / cost-to-fix") Beyond correctness, the auditor flags places where a from-scratch schema, Python, or denormalized data is doing work a cheaper option — an off-the-shelf marketplace schema or a built-in Infrahub layer — already handles — for example: * Any domain hand-rolled from scratch when the [Infrahub Marketplace](https://marketplace.infrahub.app/) already publishes it — the auditor flags common marketplace-published domains (DCIM, location, organization, circuits, cabling) via an offline signature heuristic, so reuse them with `infrahubctl marketplace get /` and `inherit_from` instead of redefining the model * A check that re-implements a uniqueness, presence, or allowed-values rule the schema can enforce as a constraint * A Python transform that only formats strings, where a Jinja2 template would do * A generator that hardcodes data belonging in `objects/`, reinvents a built-in IPAM/VLAN primitive, or hand-rolls subnet/IP/VLAN/port allocation instead of using a resource pool * A Profile carrying a single never-varying value that an attribute `default_value` would cover, or a `generate_profile`/`generate_template` flag enabled with no consumer in the repo * A generator that stamps a fixed structure an Object Template could clone, or an Object Template used to push shared values (or a Profile used to clone structure) — the wrong tool for the job These findings are advisory — the code works as-is, but the cheaper option is less to maintain and stays aligned with the platform. Each one names the specific layer to move to, and they are ranked by how cheap the fix is: reusing a published marketplace schema is the cheapest win of all and ranks first, above a one-line schema change, which ranks above a full rewrite. ## What it produces[​](#what-it-produces "Direct link to What it produces") A structured audit report listing: * **Issue** — what was found * **Severity** — error, warning, or info * **Location** — file and line where possible * **Recommended fix** — what to change and why Issues are grouped by category so the most critical problems are addressed first. ## Acting on the results[​](#acting-on-the-results "Direct link to Acting on the results") | Severity | Meaning | Action | | ----------- | ------------------------------------------------------ | ------------------------------------- | | **Error** | Will cause load failures or runtime errors | Must fix before loading into Infrahub | | **Warning** | Convention violation that may cause confusing behavior | Should fix | | **Info** | Suggestion for improvement | Optional | Cost-to-fix (YAGNI) findings are always advisory — they never block a load. They are listed cheapest-fix-first so you can pick off the easy wins, and each carries the specific replacement to apply. The Repo Auditor can be asked to fix issues directly after reporting them — describe which issues to address and the AI applies the relevant skill to make the corrections. --- # Diagnostics Collector The Diagnostics Collector guides you through producing a support bundle with OpsMill's [`infrahub-collect`](https://docs.infrahub.app/backup/guides/install-collect) tool when Infrahub is misbehaving. It installs and verifies the binary, detects your Docker Compose or Kubernetes deployment, runs `infrahub-collect create` with the flags that match your symptom, walks you through reviewing the bundle before you share it, and hands off — without diagnosing root cause or filing anything on your behalf. ## When to use[​](#when-to-use "Direct link to When to use") * Infrahub is broken, failing, erroring, or crashing and you need to collect logs for OpsMill support * Something went wrong after an upgrade (stuck branch, `NEED_UPGRADE_REBASE`) * A container is in `CrashLoopBackOff`, the API is returning 500s, or a worker was OOM-killed * Slow UI, slow diff, or slow/failing database operations * Preparing a hand-off for an OpsMill expert ## What it produces[​](#what-it-produces "Direct link to What it produces") * The correct `infrahub-collect` command sequence for your deployment: install/verify → `environment detect` → `create` * Symptom-appropriate flags (e.g. `--benchmark` for performance, `--include-queries` for database issues, `--include-backup` for reproduction) * A review-before-sharing checklist — the tool masks only key names, so you scan the bundle for anything else sensitive before it leaves your machine * A hand-off to OpsMill support, cross-linking the Issue Reporter if you also want to file a public issue The bundle itself is written by `infrahub-collect` (by default under `./infrahub_bundles/`), with logs from every replica, container/service state, version and config, and a `bundle_information.json` manifest. ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Infrahub is throwing 500s — help me collect logs to send to OpsMill" * "A task-worker keeps getting OOM-killed on our Kubernetes deployment; gather a diagnostic bundle" * "Database queries are slow on our Docker Compose setup — collect what support needs to diagnose it" * "My proposed-change pipeline is stuck and the repo won't load schemas; put together a bundle for OpsMill" * "Something broke after the upgrade — collect diagnostics" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **The tool does the collecting** — deployment detection, every-replica logs, key-name redaction, the manifest, and the bundle layout are all `infrahub-collect`'s job; the skill never hand-rolls them * **Read-only** — collection runs against your existing Docker / kubectl access with no writes and no scale changes; on Kubernetes it needs only `pods/log` and `pods/exec` * **No API token** — `infrahub-collect` authenticates through your local Docker/kubectl access, not an Infrahub API token * **Review before sharing** — masking covers only keys named `password`/`secret`/`token`/`key`; the skill stops for you to scan `bundle/logs/` and `bundle/server/` for internal hostnames, customer data, and secrets stored under other key names * **Symptom-driven flags** — start with plain `infrahub-collect create` and add a flag only when the symptom calls for it, rather than piling on every flag * **Hints, not diagnoses** — the skill produces a bundle for an expert; it does not claim a root cause * **Cross-link, don't duplicate** — for filing a public GitHub issue it hands off to the Issue Reporter instead of reimplementing that routing ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | Hand-collecting logs and config by hand | Runs `infrahub-collect`, which captures every replica and previous-container logs consistently | | Trusting the automatic masking as complete | Stops for a review-before-sharing pass, because masking is key-name-only | | Piling every flag onto `create` | Adds `--benchmark` / `--include-queries` / `--include-backup` only when the symptom matches | | Sending an Infrahub API token to collect state | Uses existing Docker/kubectl access; no token is involved | | Discarding a partial bundle on a degraded deployment | Sends the partial bundle as-is — `create` records collector failures in the manifest and still exits cleanly | | Filing a GitHub issue from here | Hands off to the Issue Reporter for that | ## Installing and running[​](#installing-and-running "Direct link to Installing and running") The skill produces these commands. Install and verify the binary, confirm detection, then create the bundle: ``` # Install the binary for your OS/architecture curl https://infrahub.opsmill.io/ops/$(uname -s)/$(uname -m)/infrahub-collect -o infrahub-collect chmod +x infrahub-collect sudo mv infrahub-collect /usr/local/bin/ # optional # Verify infrahub-collect version # Confirm the tool sees your deployment infrahub-collect environment detect # Create the bundle (add symptom flags as needed) infrahub-collect create infrahub-collect create --benchmark # performance / OOM infrahub-collect create --include-queries # slow/failing database queries ``` warning `infrahub-collect` masks values only when the key name contains `password`, `secret`, `token`, or `key`. Logs, database query logs (`--include-queries`), and secrets stored under other key names are collected as-is. Review the bundle before sharing it. tip Collector failures on a degraded deployment are expected — a stopped container can't answer status queries. `create` still exits successfully and records each failure in the manifest; send the partial bundle as-is. See the [troubleshooting-bundle guide](https://docs.infrahub.app/backup/guides/collect-troubleshooting-bundle) for the full workflow. --- # NetBox Device Type Converter The NetBox Device Type Converter turns NetBox device-type and module-type YAML into Infrahub Object Templates, so you create a device with its ports already in place from one reusable definition. The input format is [netbox-community/devicetype-library](https://github.com/netbox-community/devicetype-library), which you can browse at the [NetBox Data Exchange](https://netboxlabs.com/ndx/). A bundled Python script does the conversion, driven by a mapping profile that describes your target schema. Point it at the right schema and read what it reports back — without editing your schema for you or guessing a field name the profile does not define. ## When to use[​](#when-to-use "Direct link to When to use") * Seeding a new Infrahub instance with vendor device models from the NetBox library * Converting a handful of device types for a specific project or lab topology * Re-running a conversion after the upstream library or your local schema changed * Building the mapping profile that binds NetBox fields to a custom Infrahub schema Not for syncing a **live** NetBox instance into Infrahub — that is [infrahub-sync](https://docs.infrahub.app/sync/), a separate product. This skill converts the static YAML definitions. ## What it produces[​](#what-it-produces "Direct link to What it produces") One NetBox file becomes several Infrahub artifacts, because Infrahub splits what NetBox keeps together: | NetBox | Infrahub | Why | | -------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------ | | `manufacturer: Cisco` | `OrganizationManufacturer` object | Manufacturers are first-class objects | | `model`, `part_number`, `u_height`, `weight` | `DcimDeviceType` object | Templates hold no model data | | `interfaces:` and other component lists | `TemplateDcimDevice` plus component templates | The reusable blueprint | | `module-types/*.yaml` | `DcimModuleType` object, optionally a module template | A second input family, told apart by the absence of a `slug` | The files are numbered in dependency order, so loading them in sequence satisfies each file's references: * `01_manufacturers.yml` * `02_device_types.yml` * `03_device_templates.yml` * `04_module_types.yml` * `05_module_templates.yml` Empty files are not written, so a device-types-only run produces just the first three. A Markdown coverage report names everything the profile could not map. It is written to the `--report` path, which is resolved on its own rather than inside the output directory, and defaults to `coverage-report.md` in the current directory. ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Convert these Cisco device types into Infrahub object templates" * "Seed my instance with device models from the NetBox devicetype-library" * "Build a mapping profile for my custom DCIM schema" * "Re-run the device-type conversion — we upgraded schema-library" * "Convert the module types too, including the line cards" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **`generate_template: true` comes first** — `Template*` kinds exist only where a node declares it, so the skill checks the target schema rather than assuming either way; this is the most common reason a conversion loads nothing * **Names come from the profile, never guessed** — every Infrahub kind, attribute, and relationship name is read from a YAML mapping profile, so a custom schema needs a new profile rather than a forked script * **Competing fields declare precedence** — NetBox defines more free-text fields than most schemas have room for, so the profile names which field falls back to which; the loser is reported as shadowed, and an undeclared collision is refused rather than letting mapping order decide it silently * **Shared relationships accumulate** — where two NetBox component lists map to the same Infrahub relationship, both are kept; assigning rather than accumulating would silently erase the first list * **Integers only** — Infrahub has no float or decimal attribute kind, so every numeric transform emits a whole number * **Nothing is dropped silently** — anything the profile cannot map is skipped and named in the coverage report, which closes with an explanation of why each gap exists * **Template names are slug-based and parent-namespaced** — so names stay unique across the whole library rather than colliding between vendors * **The skill offers schema changes; it does not make them** — enabling `generate_template` is a migration against your source of truth, so it is proposed for you to apply ## Running the converter[​](#running-the-converter "Direct link to Running the converter") The skill normally runs the converter for you. To run it by hand, resolve the two `skills/...` paths against wherever your installation put the skill directory. The command below assumes the [manual copy](/skills/installation-setup.md), which puts it under your project root; the `npx` installer and the Claude Code plugin place it elsewhere. ``` python skills/infrahub-converting-netbox-device-types/scripts/netbox_to_infrahub_templates.py \ devicetype-library/device-types/Cisco/ \ --mapping skills/infrahub-converting-netbox-device-types/scripts/mappings/schema-library.yml \ --output-dir ./generated \ --report ./generated/coverage-report.md ``` Point it at a single file, a folder, or a mixed tree of device types and module types — the two families are told apart automatically, so a mixed tree converts in one pass. The skill includes three mapping profiles: `schema-library.yml`, `schema-library-modules.yml`, and an annotated `_template.yml` to copy for a custom schema. ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | ------------------------------------------- | ------------------------------------------------------------------------------- | | Assuming `Template*` kinds already exist | Checks the target schema for `generate_template: true` before converting | | Hand-transcribing a 48-port switch | Runs the script, which is exact and repeatable across thousands of device types | | Editing the script for a custom schema | Writes a new mapping profile, leaving the script untouched | | Reading a clean run as a lossless one | Points you at the coverage report, which names what did not convert | | Loading the output files in arbitrary order | Numbers them `01`–`05` in dependency order | | Expecting `{module}` port names to resolve | Explains that the bay position is known only once a module is installed | | Reaching for this to sync a live NetBox | Sends you to infrahub-sync, which is a different product | warning Read the coverage report before loading anything. Against the stock `schema-library.yml` profile most NetBox component lists have no equivalent node, so a report full of unmapped entries is the normal path rather than an error — it is the list of schema changes a lossless conversion would need. tip Module port names contain NetBox's `{module}` token, which no conversion can resolve, because the bay position is known only once a module is installed. Once the schema-library module extensions are loaded, a bundled generator resolves the token per installed module and creates the real device interfaces. --- # Data Importer The Data Importer turns CSV or TSV inputs into Infrahub object YAML and loads it onto a fresh branch. It introspects the live schema, maps columns to attributes by heuristic plus an up-front interview, splits denormalized inputs across the right kinds with the correct load order, and fails closed when a column has no schema home — without ever proposing a schema edit. ## When to use[​](#when-to-use "Direct link to When to use") * Importing infrastructure data from a CSV or TSV export (NetBox, vendor inventories, spreadsheets) * Loading a denormalized one-big-sheet export and splitting it across the right Infrahub kinds * Bulk-loading initial data into a new Infrahub instance from a folder of CSVs * Re-importing edits to existing rows by HFID (upsert) * Importing data with lineage tagging via `source` / `owner` / `is_protected` ## What it produces[​](#what-it-produces "Direct link to What it produces") * Numbered object YAML files (`01_manufacturers.yml`, `02_sites.yml`, `10_devices.yml`, …) that conform to the Object Manager envelope (`apiVersion: infrahub.app/v1`, `kind: Object`, `spec.kind`, `spec.data`) * A fresh import branch created via `infrahubctl branch create` before any validate or load * A confirmed mapping plan from the up-front interview, locked before the first file is written * Local self-check against Object Manager rules, followed by server-side `infrahubctl object validate`, followed by `object load` — all scoped to the import branch * Optional `source` / `owner` / `is_protected` metadata stamping when the user opts in ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Import `devices.csv` into Infrahub on a fresh branch" * "Convert this denormalized CSV with manufacturer, location, and device columns into separate object files in the right load order" * "Load these three CSVs (manufacturers, sites, devices) and resolve the references between them" * "Import an interface CSV with `eth0..eth47` ranges and collapse them with `expand_range: true`" * "Import `assets.csv` and stamp every value with `source: csv-import-20260622`" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **Schema is read-only** — the skill never proposes attribute additions, dropdown choice additions, or any schema edit; an unmappable column triggers a fail-closed report and routes to the Schema Manager * **Up-front interview** — every ambiguity (unmapped dropdown cells, denormalization splits, branch name, lineage opt-in) is batched into one question round before any file is written * **Branch-first** — runs `infrahubctl branch create ` before `object validate` or `object load`; the import never lands on the default branch * **Local self-check before server validate** — the emission is walked against the Object Manager rules locally first, so shape errors don't cost a branch that has to be discarded * **Dropdown label to choice name** — UI-facing labels in the CSV (e.g., `Active`) are translated to the schema's choice name (e.g., `active`) using a label-to-name lookup * **Reference shape from HFID** — relationship references emit as a scalar or a positional list based on the target node's `human_friendly_id` length, read directly from the schema * **Range collapse for interfaces** — contiguous sequences like `eth0..eth47` collapse to `eth[0-47]` with `parameters.expand_range: true` when all sibling columns match across the range * **Numbered load order** — files use `NN_` prefixes so referents load before referrers, and dependent kinds are emitted to higher-numbered files ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------ | | Dropping unmapped columns silently | Fails closed with a structured report of unmappable columns and the kinds checked | | Emitting dropdown labels instead of choice names | Builds a label→name table from the schema and emits the name | | Wrong reference shape (scalar vs list) | Reads the target's `human_friendly_id` length and emits the matching shape | | Repeated parent rows treated as duplicates | Detects the denormalization and asks whether to nest as component children or split into separate kinds | | Emitting interface rows as 48 literal entries | Detects contiguous sequences with identical sibling columns and emits a single range row plus `expand_range: true` | | Loading to the default branch | Creates a dedicated import branch and validates + loads there | | Trying to resume a partial load | Discards the branch and re-runs with a fresh branch name (`object load` is not transactional across files) | ## Validating and loading[​](#validating-and-loading "Direct link to Validating and loading") The skill produces these commands as part of the workflow. Run them in order on the import branch: ``` # Create the dedicated import branch (after the local self-check passes) infrahubctl branch create csv-import-20260622-1430 # Validate the emission against the branch infrahubctl object validate ./output_dir/ --branch csv-import-20260622-1430 # Load on success infrahubctl object load ./output_dir/ --branch csv-import-20260622-1430 # If anything fails partway, discard the branch and re-run with a fresh name infrahubctl branch delete csv-import-20260622-1430 ``` warning The skill never edits the schema. If a CSV column has no attribute or relationship to map to, the skill stops with a list of the unmappable columns and the kinds it checked. Resolve the gap with the [Schema Manager](/skills/skills-reference/managing-schemas.md), then re-run. tip For Excel inputs, export each sheet to CSV first — the skill covers CSV and TSV only. For LDJSON dumps from `infrahubctl export dump`, use `infrahubctl import load` instead; that's a different format and a different tool. --- # Check Manager The Check Manager produces Python validation checks that run in Infrahub's proposed change pipeline. Each check consists of three components: a GraphQL query file, a Python class inheriting from `InfrahubCheck`, and a `.infrahub.yml` registration entry. The skill generates all three and ensures they are correctly wired together. ## When to use[​](#when-to-use "Direct link to When to use") * Enforcing naming conventions on devices, interfaces, or other nodes when a change is proposed * Validating that required relationships are populated (e.g., every device has a primary IP) * Blocking changes that would violate design intent (e.g., a BGP session referencing a decommissioned device) * Any validation that should block a proposed change from merging ## What it produces[​](#what-it-produces "Direct link to What it produces") Three files that must be used together: 1. **GraphQL query file** (`.gql`) — fetches the data the check needs. Named to match the `query` attribute in the Python class. 2. **Python class** (`InfrahubCheck` subclass) — implements the `validate()` method. Uses `self.log_error()` to flag violations (which blocks the change) and `self.log_info()` for informational messages (which does not block). 3. **`.infrahub.yml` entry** — registers the check under `check_definitions` with the class name, query name, and any parameters. ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Create a check that validates every device has a primary IP assigned" * "Write a check that enforces the naming convention `--` for all device names" * "Create a check that blocks any proposed change that removes a BGP session from an active device" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **Three-component structure** — all three files are always generated together; missing any one causes the check to fail at runtime * **`log_error()` vs `log_info()`** — `log_error()` blocks the proposed change; `log_info()` does not. The skill uses the correct one based on the intent described * **GraphQL query naming** — the query name in the `.gql` file must match the `query` class attribute exactly * **Parameter mapping** — if the check accepts parameters (e.g., to parameterize a naming pattern), the parameters are declared in both the Python class and the `.infrahub.yml` registration * **Error collection before logging** — errors are collected across all objects first, then logged together, rather than short-circuiting on the first error ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | --------------------------------------------------- | ---------------------------------------------------- | | Using `log_error()` when `log_info()` was intended | Matches the log method to the described intent | | Query name mismatch between `.gql` and Python class | Uses the same name in both | | Missing `.infrahub.yml` registration | Generates the registration entry alongside the check | | Short-circuiting on first error | Collects all errors before logging | ## Running checks[​](#running-checks "Direct link to Running checks") Checks run automatically in the proposed change pipeline when a change is submitted to Infrahub. To test a check locally: ``` infrahubctl check run ``` --- # Generator Manager The Generator Manager produces Python generators that query Infrahub for design objects and automatically create or update the infrastructure objects derived from them. Generators are idempotent: re-running them updates existing objects rather than creating duplicates. The skill generates the Python class, GraphQL query, and `.infrahub.yml` registration. ## When to use[​](#when-to-use "Direct link to When to use") * Auto-creating devices, interfaces, and IP addresses from a fabric design object * Generating BGP sessions from a topology design * Creating any set of objects that should be derived from a design definition and kept in sync with it * Any scenario where "given this design, create these objects" needs to run repeatedly as the design evolves ## What it produces[​](#what-it-produces "Direct link to What it produces") Three components: 1. **Target group** — a `CoreGeneratorGroup` that defines which design objects trigger the generator 2. **GraphQL query** (`.gql`) — fetches the design object(s) the generator will act on 3. **Python class** (`InfrahubGenerator` subclass) — implements the `async generate()` method, uses `self.client.create()` with `allow_upsert=True` The `.infrahub.yml` entry tying them together carries a `watch` dependency declaration alongside them. ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Create a generator that takes a fabric design object and creates all spine and leaf devices with their loopback interfaces" * "Write a generator that creates BGP sessions between all spine-leaf pairs defined in a topology design" * "Generate IP address assignments for all interfaces based on a design's IP allocation scheme" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **`allow_upsert=True`** — always set on `self.client.create()` calls; without it, re-running the generator creates duplicates instead of updating existing objects * **`delete_unused_nodes=True`** — enables automatic cleanup of objects that were created by the generator but are no longer in the design; prevents orphaned objects from accumulating over time * **Target group** — generator must reference a `CoreGeneratorGroup`; this is what connects design objects to the generator execution * **Async `generate()` method** — the method signature must be `async def generate(self, data):` * **GraphQL query naming** — query name must match the `query` class attribute * **`watch` dependencies** — the `generator_definitions` entry always carries a `watch` block naming every first-party module the generator imports, sibling query models included (`files: []` when there are none); imports are never analyzed, so without the key the generator re-runs on every commit, and with an incomplete list a change to a shared helper silently stops re-running it ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | | Missing `allow_upsert=True` | Always includes it — prevents duplicate creation on re-run | | Missing `delete_unused_nodes=True` | Always includes it — prevents orphaned objects | | Sync `generate()` method | Uses `async def generate(self, data):` | | Missing target group configuration | Generates the `CoreGeneratorGroup` definition | | Generator that only stamps a fixed structure with constant values | Steers toward an Object Template (`generate_template`) to clone instead — the structure lives in data, not Python | ## Running generators[​](#running-generators "Direct link to Running generators") Generators run automatically when a design object in their target group changes. To run a generator manually: ``` infrahubctl generator run ``` --- # Menu Manager The Menu Manager produces YAML menu definition files that customize the Infrahub web UI sidebar. Menus can be flat or deeply nested, with group headers, icons, and ordering. The skill generates syntactically correct menu YAML and handles the nesting structure that is easy to get wrong by hand. ## When to use[​](#when-to-use "Direct link to When to use") * Replacing Infrahub's auto-generated sidebar navigation with a custom structure * Grouping schema nodes into logical sections (e.g., "IP Management", "Network Topology", "Inventory") * Adding nested sub-menus and section headers * Controlling the ordering of menu items ## What it produces[​](#what-it-produces "Direct link to What it produces") * YAML file with `apiVersion`, `kind: Menu`, and a `spec.data` block containing the menu tree * Loaded into Infrahub alongside schema and object files ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Create a menu with sections for Network Topology, IP Management, and Inventory — each with relevant schema nodes nested underneath" * "Build a flat menu that lists Device, Interface, Site, and Rack in that order" * "Add a nested sub-menu under Network Topology for BGP with entries for BGP Sessions and BGP Peers" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **`spec.data` wrapping** — the top-level menu items must be in `spec.data`; a common mistake is writing `spec:` followed by a list directly * **`children.data` wrapping** — nested items must use `children.data:` not `children:` directly; this is the most common nesting mistake * **`kind` vs `path`** — each item uses either `kind:` (links to a schema node's list view) or `path:` (links to a custom URL), never both * **Group headers** — section headers with no link (no `kind` or `path`) are valid and used for visual grouping * **Icon format** — icons use the Material Design Icons format with `mdi:` prefix (e.g., `mdi:server`, `mdi:ip-network`) * **`order_weight`** — integer controlling item display order; lower numbers appear first ## Loading[​](#loading "Direct link to Loading") Load a menu file into a running Infrahub instance: ``` infrahubctl menu load ``` --- # Object Manager The Object Manager produces YAML data files for Infrahub infrastructure objects — devices, locations, organizations, interfaces, modules, and any custom node type. It applies the correct file structure, value mappings, and load order so files are ready to load into a running Infrahub instance. ## When to use[​](#when-to-use "Direct link to When to use") * Creating initial data files for a new Infrahub instance * Populating objects from a spreadsheet, CSV, or description * Creating a batch of devices, interfaces, or IP addresses with consistent structure * Generating test or demo data for a schema * Adding new infrastructure objects to an existing instance ## What it produces[​](#what-it-produces "Direct link to What it produces") * YAML files with `apiVersion`, `kind: Object`, and a `spec` block * Each file covers one or more objects of the same node type * Supports multi-document YAML files (multiple objects in one file separated by `---`) * Supports `expand_range: true` for sequential interface/object generation (e.g., `eth0` through `eth47`) * Output respects load order: schemas must be loaded before objects ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Create object files for 3 spine switches and 6 leaf switches with loopback interfaces" * "Populate 5 locations: 2 regions, 2 sites within each region, and a rack in each site" * "Generate 48 interfaces on a device named `spine-01` using expand\_range" * "Create organization objects for Cisco, Juniper, and Arista as manufacturers" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **File structure** — `apiVersion`, `kind: Object`, `spec` with `nodes` list; missing any of these causes load failure * **Value mapping** — Text values wrapped in `value:` key; Dropdown fields use the choice name not the label; relationship references use the node's `human_friendly_id` value * **Children and hierarchy** — child objects (e.g., interfaces on a device) use the correct nesting structure with `kind:` specified * **Load order** — objects referencing other objects (e.g., interfaces referencing a device) must be in the right file or the right order within a multi-document file * **Numeric file prefixes** — files are named with numeric prefixes (e.g., `01_organizations.yml`, `02_locations.yml`) to enforce load order * **Profiles and templates** — assigns shared defaults via a `profiles:` list (explicit values override), creates objects from a template via `object_template`, and authors template objects under the generated `Template` kind — with the referenced profile or template loaded first ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | ------------------------------------------------------------ | -------------------------------------------------------- | | Missing `value:` wrapper on text attributes | Wraps all text values in `value:` | | Using Dropdown label instead of name | Uses the choice name (e.g., `"active"` not `"Active"`) | | Relationship references that don't match `human_friendly_id` | Uses the correct identifier value | | Incorrect nesting for child objects | Applies the correct `children` structure with `kind:` | | Load order violations | Orders files and documents so dependencies are satisfied | ## Loading[​](#loading "Direct link to Loading") Load object files into a running Infrahub instance: ``` infrahubctl object load ``` warning Load schemas before objects. Objects that reference node types not yet in the schema will fail to load. --- # Schema Manager The Schema Manager produces valid Infrahub schema YAML from natural language descriptions. It applies Infrahub's naming conventions, relationship rules, attribute type requirements, and display property patterns automatically — without requiring manual study of the schema format. ## When to use[​](#when-to-use "Direct link to When to use") * Reusing an existing schema from the Infrahub Marketplace instead of modelling any domain from scratch (the whole catalog is checked, not just common domains) * Creating a new schema node from a description * Adding attributes or relationships to an existing node * Designing a generic (abstract base type) with inheriting nodes * Creating hierarchical node types (location trees, organizational hierarchies) * Modifying an existing schema to add new fields or fix convention violations * Validating a schema against best practices before loading ## What it produces[​](#what-it-produces "Direct link to What it produces") * Schema YAML file starting with `version: "1.0"` containing `nodes:` and/or `generics:` * Each node includes: namespace, name, attributes (with kinds and defaults), relationships (with identifiers, cardinality, peer references), `human_friendly_id`, `display_label`, and `order_weight` * Output is ready to validate with `infrahubctl schema check` and load with `infrahubctl schema load` ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Create a schema for VLAN management with an id attribute, a name, a status dropdown, and a role field" * "Add a `contract_start_date` attribute to `InfraCircuit`" * "Design a location hierarchy with Region, Site, and Rack as hierarchical generics" * "Create a Device generic with Platform and Role as dropdowns, and model specific device types (Router, Switch, Firewall) inheriting from it" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **Naming conventions** — namespace is PascalCase and meaningful (e.g., `Infra`, `Ipam`, `Dcim`); node names are PascalCase; attribute names are snake\_case with minimum 3 characters * **Attribute types** — uses `Text` (not deprecated `String`); `Dropdown` for status/role fields with choices defined; `Number`, `Boolean`, `DateTime` as appropriate * **Relationship identifiers** — both sides of every relationship share the same identifier string; mismatched identifiers are the most common source of schema errors * **Relationship peer references** — uses full namespace+name (e.g., `IpamVlanGroup`) not short names * **human\_friendly\_id** — included for every node so objects are identifiable in the UI and through the API * **display\_label** — Jinja2 template set for every node for dropdown and reference rendering * **order\_weight** — integer set on attributes to control display order in the UI * **Hierarchy setup** — for hierarchical generics: `hierarchical: true`, `parent` and `children` relationships configured correctly * **Profiles (`generate_profile`)** — enabled when the same attribute or relationship values recur across many objects and should stay in sync; a single value that never varies stays an attribute `default_value` instead * **Object Templates (`generate_template`)** — enabled to clone a node together with its component children as a reusable starting structure; kept distinct from Profiles (which share live values, not structure) ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | ----------------------------------------- | ---------------------------------------------------------------- | | Using `String` instead of `Text` | Uses `Text` — `String` is deprecated | | Attribute names shorter than 3 characters | Uses descriptive names (e.g., `vlan_id` not `id`) | | `display_labels` (plural) | Uses `display_label` (singular) | | Mismatched relationship identifiers | Sets the same identifier on both sides | | Short kind references in peers | Uses full namespace+name (e.g., `IpamVlanGroup` not `VlanGroup`) | | Missing `human_friendly_id` | Includes it on every node | ## Reusing published schemas[​](#reusing-published-schemas "Direct link to Reusing published schemas") Before modelling **any** domain from scratch, the skill searches the **entire** [Infrahub Marketplace](https://marketplace.infrahub.app/) for a published schema and reuses it rather than re-deriving the model — so if a schema already exists, no modelling is needed. The marketplace publishes far more than the obvious domains: DCIM, location, organization, circuits, cabling, routing (BGP, OSPF, PIM), compute and clusters, security and firewall policy, VLAN translation, cross-connects, and many more. The whole catalog is checked every time, not a fixed shortlist. Fetch a match with the CLI and `inherit_from` the result, adding only site-specific attributes: ``` infrahubctl marketplace get / ``` Discovery is CLI-first: `infrahubctl marketplace list`, `search `, and `show /` browse the whole published catalog (paging through every result by default), and `infrahubctl marketplace get` fetches the match. Reuse only from the marketplace, not from GitHub. In airgapped environments this is a fallback chain, not a blocker — point `--marketplace-url` (or `INFRAHUB_MARKETPLACE_URL`) at an internal mirror, and if none is reachable the skill proceeds with a custom schema (still preferring built-in primitives). ## Validating and loading[​](#validating-and-loading "Direct link to Validating and loading") Validate a generated schema before loading it into Infrahub: ``` infrahubctl schema check ``` Load the schema into a running instance: ``` infrahubctl schema load ``` tip Schema changes on an existing instance may require a migration. See the [schema topic guide](https://docs.infrahub.app/topics/schema) for details on how Infrahub handles schema evolution. --- # Transform Manager The Transform Manager produces data transforms that convert Infrahub data into other formats — device configurations, JSON exports, CSV reports, and more. Transforms can be implemented as Python classes, Jinja2 templates, or a hybrid of both. The skill generates the transform code, GraphQL query, optional Jinja2 templates, and `.infrahub.yml` registration. ## When to use[​](#when-to-use "Direct link to When to use") * Generating device configuration files from Infrahub data * Exporting infrastructure data to JSON, CSV, or other formats * Producing human-readable reports from graph data * Any scenario where Infrahub data needs to be rendered into a different representation ## Transform types[​](#transform-types "Direct link to Transform types") ### Python transform[​](#python-transform "Direct link to Python transform") Use when the output requires logic: conditional rendering, calculations, data reshaping. * Implements `InfrahubTransform` class with a `transform()` method * Returns a `dict` for JSON output or a `str` for text output ### Jinja2 transform[​](#jinja2-transform "Direct link to Jinja2 transform") Use when the output is primarily a template with variable substitution: device configs, structured text files. * Template file receives the GraphQL query response in a `data` variable * Supports `netutils` filters for network-specific operations (IP address manipulation, interface normalization) ### Hybrid (Python + Jinja2)[​](#hybrid-python--jinja2 "Direct link to Hybrid (Python + Jinja2)") Use when data needs preprocessing before rendering. * Python class prepares and reshapes the data, Jinja2 template handles rendering * Supports platform-specific templates (e.g., different configs per vendor) using `FileSystemLoader` ## What it produces[​](#what-it-produces "Direct link to What it produces") * GraphQL query file (`.gql`) — fetches the data the transform needs * Python transform class and/or Jinja2 template file(s) * `.infrahub.yml` entry under `python_transforms` or `jinja2_transforms`, including its `watch` dependency declaration * `artifact_definitions` entry if the transform produces versioned artifacts ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Create a Jinja2 transform that produces Cisco IOS configuration for each device, pulling interface and BGP data from Infrahub" * "Write a Python transform that exports all IP addresses to a CSV file" * "Build a hybrid transform that generates vendor-specific configs — Cisco IOS for routers, Juniper JunOS for switches" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **Return type** — Python transform must return `dict` for JSON output or `str` for text; wrong return type causes rendering failure * **`data` variable** — Jinja2 templates always receive the GraphQL response as `data`; accessing the wrong key is a common mistake * **Artifact definitions** — if the transform output should be stored as a versioned artifact in Infrahub, an `artifact_definitions` entry is required in `.infrahub.yml`; the skill generates this when appropriate * **`netutils` availability** — `netutils` filters are available in Jinja2 templates; the skill uses them where appropriate for IP and interface operations * **`watch` dependencies** — a Python transform always carries a `watch` block naming every first-party module it imports and every file it reads at runtime (`files: []` when there are none); Infrahub never analyzes imports, so without the key the artifacts re-render on every commit, and with an incomplete list they silently go stale. A Jinja2 transform needs one only for a reference its parser cannot follow, such as a computed `{% include %}` ## Running transforms[​](#running-transforms "Direct link to Running transforms") Transforms run as part of the artifact generation pipeline or on demand: ``` infrahubctl transform run ``` --- # Issue Reporter The Issue Reporter files a bug report or feature request against the correct repository in the Infrahub ecosystem. There are 11 candidate repos and most people do not know which one owns their problem, so the skill reads your working directory for cues, proposes a target, searches for duplicates, and drafts an issue that matches that repo's intake form. It never auto-submits: it stops at a review gate and submits only after you approve both the content and the method. ## When to use[​](#when-to-use "Direct link to When to use") * You think you have found a bug in Infrahub or one of its components * You want to request a feature and do not know where it belongs * You are not sure which repo owns the problem you are seeing * You want an issue drafted properly before it goes on a public tracker Do not use it while you are still debugging locally with no intention of filing. Filing is what separates this skill from ordinary troubleshooting. For friction with an Infrahub *skill's* own guidance, use the [Skill Gap Reporter](/skills/skills-reference/reporting-skill-gaps.md) instead, which drafts the report and hands it here. ## What it produces[​](#what-it-produces "Direct link to What it produces") * A routing decision with a one-sentence rationale, presented for you to confirm or correct * A duplicate search across open and closed issues, with the top matches shown to you * Either a new issue or a comment on the existing one that already covers your problem * For bugs, a minimal environment section: product versions, OS and architecture, runtime version * A body rendered against the target repo's own issue template when it has one * The issue or comment URL after submission ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "I think I found a bug in Infrahub" * "How do I report this issue?" * "I'd like to request a feature for the Ansible collection" * "Where should I file this?" * "This isn't working, I want to open a ticket" ## Where issues get routed[​](#where-issues-get-routed "Direct link to Where issues get routed") | Repo | Owns | | ----------------------------- | -------------------------------------- | | `opsmill/infrahub` | The platform, and the default fallback | | `opsmill/infrahub-sdk-python` | The Python SDK | | `opsmill/infrahub-vscode` | The VS Code extension | | `opsmill/infrahub-ansible` | The Ansible collection | | `opsmill/nornir-infrahub` | The Nornir plugin | | `opsmill/infrahub-helm` | The Helm chart | | `opsmill/infrahub-mcp` | The MCP server | | `opsmill/schema-library` | The schema library | | `opsmill/infrahub-backup` | The backup CLI | | `opsmill/infrahub-sync` | The sync tool | | `opsmill/infrahub-skills` | This plugin | **When routing is ambiguous, it defaults to `opsmill/infrahub`.** The main repo can re-route an incoming issue to a sub-repo far more easily than a sub-repo can send it back, so erring toward the platform is the cheaper mistake. ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **Never submit without approval** — the review gate shows you the target repo, the final title, the full body, and whether this is a new issue or a comment, before anything is posted * **Default to main when unsure** — ambiguous cues, equally matching repos, or your own uncertainty all route to `opsmill/infrahub` * **Search before filing** — open and closed issues both, with a second pass using synonyms when the first returns nothing * **Comment instead of duplicating** — when a match exists, the skill either complements the issue with genuinely new information or records a short "another hit" so maintainers can gauge frequency * **Minimal environment info** — versions, OS, and architecture only; no file paths, no project structure, no verbatim logs, no env dumps, no `docker compose ps` output * **Unknown beats guessed** — a version that cannot be determined cleanly is marked `unknown` rather than inferred by running broader discovery commands * **Support questions are not issues** — "how do I do X?" is routed to Discord or GitHub Discussions ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | -------------------------------------------------- | ------------------------------------------------------------- | | Filing in a sub-repo when the symptom is ambiguous | Files in `opsmill/infrahub` and lets maintainers re-route | | Opening a duplicate | Adds a comment to the existing issue with the new information | | Pasting full logs or `docker compose` output | Collects versions, OS, and architecture, nothing more | | Filing a support question as a bug | Points you to Discord or Discussions | | Guessing a version number | Marks it `unknown` | | Submitting before you have read it | Stops at the review gate until you explicitly approve | ## Finding your versions[​](#finding-your-versions "Direct link to Finding your versions") For bugs, the skill collects the relevant version. These are the commands it uses: | Component | Command | | ------------------ | ----------------------------------------------------------------- | | Infrahub server | `infrahubctl --version`, or the `image:` tag in your compose file | | Python SDK | `pip show infrahub-sdk` | | Ansible collection | `ansible-galaxy collection list opsmill.infrahub` | | Nornir plugin | `pip show nornir-infrahub` | | Helm chart | `helm list -n ` | | infrahub-sync | `infrahub-sync --version` | | infrahub-backup | `infrahub-backup --version` | | MCP server | MCP client config, or `pip show infrahub-mcp` | | VS Code extension | Extensions panel in VS Code | | Skills plugin | `version` field in `.claude-plugin/plugin.json` | ## Submitting[​](#submitting "Direct link to Submitting") Once you approve the content, the skill asks how you want to submit. All three methods work for a new issue or a comment on an existing one: ``` # New issue gh issue create --repo --title "..." --body "..." # Comment on an existing issue gh issue comment --repo --body "..." ``` A GitHub MCP server is used instead when you have one, and manual mode prints copy-paste-ready markdown plus the right URL. warning Issue bodies are public and cannot be retracted. Error messages carry file paths, hostnames, and node kinds that identify your infrastructure, so the skill paraphrases rather than pasting. Read the draft at the review gate with that in mind. tip If Infrahub is broken rather than misbehaving in a specific, reportable way, the [Diagnostics Collector](/skills/skills-reference/collecting-diagnostics.md) gathers a redacted bundle for OpsMill support instead. The two are complementary: a public issue for a reproducible defect, a private bundle for a broken deployment. --- # Skill Gap Reporter The Skill Gap Reporter turns friction with an Infrahub skill into a reviewed GitHub issue against [`opsmill/infrahub-skills`](https://github.com/opsmill/infrahub-skills). Infrahub skills fail quietly: a missing or unclear rule does not crash, it produces extra round trips and repeated nudges from you until the model finds the answer on its own. This skill works out which rule was missing, wrong, or unclear, checks the tracker for an existing report, and drafts a proposed rule change. It never files anything itself: it hands the redacted draft to the [Issue Reporter](/skills/skills-reference/reporting-issues.md), which shows it to you and gets explicit approval before any submission. ## When to use[​](#when-to-use "Direct link to When to use") * A skill took far more attempts than it should have, and you want the friction reported * You want to know why a skill kept getting the same thing wrong * A skill's guidance is missing something and you want the gap fixed upstream * You accepted an offer to report friction at the end of a session Do not use it for a bug in Infrahub itself, the SDK, or any other `opsmill/infrahub-*` product. That belongs to the Issue Reporter. It is also not for ordinary work that finished without friction. That offer can also arrive on its own. With the [infrahub-speckit](https://github.com/opsmill/infrahub-speckit) extension installed, Spec Kit's `after_implement` hook checks every finished `/speckit.implement` cycle and prints a one-line offer when it finds evidence of a gap; replying routes into this skill. The hook detects, it never drafts or files. See [Skill-gap detection after implement](/skills/spec-driven-development.md#skill-gap-detection-after-implement). ## What it produces[​](#what-it-produces "Direct link to What it produces") * A named artifact: the rule file that should have prevented the friction, or a plain statement that no rule covers the topic * A proposed rule change, so a maintainer does not have to reverse-engineer the fix from a symptom * A comment on the existing issue, or a new issue carrying an explicit confidence label * A redacted body, with anything identifying your infrastructure removed before the draft leaves the skill * A hand-off to the Issue Reporter, which owns the review gate and submission ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "That took way too many tries, can you report the friction?" * "Report skill friction." * "Why did the schema skill keep getting relationship cardinality wrong?" * "We hit this same gap last week too, please file it." ## How it spots the friction[​](#how-it-spots-the-friction "Direct link to How it spots the friction") The current conversation is always the first and usually the only source. A past session's transcript under `~/.claude/projects/` is read only when you point at one, and its absence never blocks a report. Three things count as friction: | Signal | What it looks like | | ----------------- | ----------------------------------------------------------------------------------------------------------------------- | | Failed tool calls | `tool_result` blocks carrying `is_error` | | Repeated nudges | You restating the same ask in different words | | A docs escape | A fetch of `docs.infrahub.app`, `llms.txt`, or `schema.infrahub.app` **after** the relevant skill file was already read | Fetches of `marketplace.infrahub.app` and `infrahub.opsmill.io` are not friction. Several skills fetch those as part of normal work, so flagging them would be noise. ## How it routes[​](#how-it-routes "Direct link to How it routes") Routing happens on two levels, and only skill defects reach a draft. **Level 1 decides who owns the problem.** A skill defect continues; a product defect stops here and hands off to the Issue Reporter for the right `opsmill/infrahub-*` repo, because the skills repo has no power to fix Infrahub itself; anything else stops with an explanation. **Level 2 decides the kind**, which sets both the title prefix and the target repo. The docs escape is what separates the three, and what matters is not that it happened but what happened after it: | Situation | Kind | Title | Repo | | ---------------------------------------------------------------------- | -------- | ------------ | ------------------------- | | A rule already covers the topic and the model still got it wrong | Bug | `bug:` | `opsmill/infrahub-skills` | | No rule covers it, and the escape found the answer (or never happened) | Feature | `feat:` | `opsmill/infrahub-skills` | | No rule covers it, the escape happened and still failed to answer | Docs gap | `bug(docs):` | `opsmill/infrahub` | A docs gap routes to `opsmill/infrahub` because Infrahub's own documentation lives there, not in the skills repo. It also has to clear a **settled-behavior gate**: if the workflow is still being designed or is deliberately undocumented, there is nothing to write down yet, so the skill stops and tells you which behavior is unsettled. That is the only point in the workflow that ends without a report. ## How the tracker check shapes the report[​](#how-the-tracker-check-shapes-the-report "Direct link to How the tracker check shapes the report") Before anything is drafted, `opsmill/infrahub-skills` is searched. The result decides the **shape** of the report, never whether there is one: | Tracker result | What you get | | ------------------------------------------------ | ------------------------------------------------------------- | | An issue already covers it | A comment on that issue, carrying only what this session adds | | No match, and you confirm it has happened before | A new issue labelled `recurring` | | No match, first time seeing it | A new issue labelled `unconfirmed single observation` | A single observation is weak evidence, not zero evidence. It gets labelled honestly and left for a maintainer to weigh. A thin issue takes seconds to close; an observation that was never written down cannot be recovered, and the next person to hit the same gap would start from zero. ## The method, end to end[​](#the-method-end-to-end "Direct link to The method, end to end") 1. **Gather evidence** from the current conversation 2. **Check the tracker** for an existing report, which decides comment vs. new issue 3. **Triage level 1**, skill defect or product defect 4. **Hand off** to the Issue Reporter if the product is at fault, and stop 5. **Triage level 2**, bug, feature, or docs gap 6. **Locate the artifact** by listing `skills//rules/` and grepping for the topic, so the report names a file rather than a symptom 7. **Draft and redact**, removing anything identifying your infrastructure 8. **Hand off** the payload for review and submission 9. **Report the outcome** back to you, with the URL when there is one ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **Tracker first** — a skill's users share no repo, directory, or laptop, so local session history cannot answer "has anyone seen this before" * **Comment, don't duplicate** — a second issue splits the evidence across two threads * **Never suppress a first sighting** — the report is always produced, with a confidence label rather than silence * **Cite the artifact** — the draft names a file under `skills//rules/`, or states plainly that no rule covers the topic; "the schema skill" gives a maintainer a search, not a starting point * **Redact before drafting** — error messages carry file paths, hostnames, and node kinds; issue bodies are public and cannot be retracted * **Never file directly** — this skill has no consent gate and no submission method of its own ## Common mistakes it catches[​](#common-mistakes-it-catches "Direct link to Common mistakes it catches") | Mistake | What the skill does instead | | ------------------------------------------------------- | ---------------------------------------------------------------------- | | Drafting without checking the tracker | Searches first, so the draft is a comment when an issue already exists | | Staying silent because it only happened once | Files it, labelled an unconfirmed single observation | | Filing a symptom with no proposed fix | Names the rule file to create or edit, and what it should say | | Filing an Infrahub server bug against the skills repo | Routes it to the Issue Reporter for the correct product repo | | Calling it a skill gap when the docs are what's missing | Files `bug(docs):` against `opsmill/infrahub` instead | | Pasting raw error text | Paraphrases, so hostnames and node kinds do not reach a public issue | ## What it needs[​](#what-it-needs "Direct link to What it needs") The tracker search uses the GitHub CLI: ``` gh search issues --repo opsmill/infrahub-skills --state all " " ``` If `gh` is unavailable or unauthenticated, the skill says so and asks you directly whether you have hit the problem before, rather than guessing. note This skill drafts and hands off. Every submission goes through the Issue Reporter's review gate, so nothing reaches GitHub until you have seen the exact title and body and approved them. --- # Concept Tutor The Concept Tutor teaches Infrahub concepts starting from wherever the learner is, using their own repo files and live instance as lesson material instead of generic examples. It probes what the learner already knows, explains each concept anchored to their own schema and data, sets a hands-on exercise verified solvable before it's shown, and records progress so a later session resumes instead of restarting. ## When to use[​](#when-to-use "Direct link to When to use") * Learning how a concept works: "teach me how relationships work," "explain how branches work in my instance" * New to Infrahub and wanting a guided tour rather than a task done for you * Coming from NetBox or Nautobot and wanting the Infrahub equivalent of a concept you already know * Understanding why your instance behaves a certain way, when the goal is learning rather than an operational answer * Resuming a learning session: "where were we?" ## What it produces[​](#what-it-produces "Direct link to What it produces") * `.infrahub-learning/progress.md`: one row per concept taught, tracking status (`not-seen`, `introduced`, `practiced`) and when it was last seen * `.infrahub-learning/lessons/.md`: a lesson artifact per concept, structured as Probe, Explain, Exercise, Check * `.infrahub-learning/solutions/.md`: a verified reference solution and the verification evidence, written before the exercise is shown to the learner * `.infrahub-learning/hints/.md`: the escalation log, one `## Hint N` rung per failed attempt, so a resumed session knows which rungs the learner already climbed * A closing pointer to the sibling skill that does that concept's work on real projects The curriculum is dependency-ordered and enters wherever the probe says the learner already is: | # | Concept | Graduates to | | -- | ---------------- | ----------------- | | 1 | Foundations | (none) | | 2 | Schema | Schema Manager | | 3 | Objects | Object Manager | | 4 | GraphQL | Data Analyzer | | 5 | Branches | (none) | | 6 | Repo integration | (none) | | 7 | Proposed changes | (none) | | 8 | Checks | Check Manager | | 9 | Transforms | Transform Manager | | 10 | Generators | Generator Manager | | 11 | Menus | Menu Manager | A concept outside this list isn't refused. It's taught as an off-map lesson, grounded in a docs page found through the same lookup path other skills use, and flagged to the learner as coming from that fallback rather than the curriculum. ## Example prompts[​](#example-prompts "Direct link to Example prompts") * "Teach me how relationships work" * "I'm new to Infrahub, give me a guided tour" * "What's the NetBox equivalent of a proposed change in Infrahub?" * "Explain why my instance behaves this way" (asked to understand, not to get an operational answer) * "Where were we?" ## Key rules enforced[​](#key-rules-enforced "Direct link to Key rules enforced") * **Probe before teaching**: every lesson opens with 2-3 questions that place the learner before any explanation starts * **Teach through the learner's own artifacts**: schema, objects, and a reachable live instance take priority over generic examples; a small two-node fallback example is used only when the learner has neither * **Verified exercises**: the reference solution is built and verified before the exercise is shown, so the learner is never handed an assignment that can't be solved * **Hint ladder**: a struggling learner gets a conceptual hint first, then a concrete pointer, then the full solution on a third attempt or on request, never the answer up front * **Instance writes are opt-in and branch-scoped**: reading the instance needs no consent; writing happens only after an explicit yes, targets a `learning-*` branch, is cleaned up afterward, and is never merged into the default branch * **Progress recording**: every lesson updates `.infrahub-learning/progress.md` so a later session resumes instead of restarting * **Comparisons are sourced**: a NetBox or Nautobot comparison cites that tool's official docs or is explicitly marked unverified, never asserted from memory * **Graduation**: each concept closes by naming the sibling skill that does that work on real projects ## Concept Tutor vs. Managing skills vs. Data Analyzer[​](#concept-tutor-vs-managing-skills-vs-data-analyzer "Direct link to Concept Tutor vs. Managing skills vs. Data Analyzer") | Use Concept Tutor when... | Use a Managing skill when... | Use Data Analyzer when... | | ------------------------------------------ | --------------------------------------------------------------------- | ------------------------------------------- | | The learner wants to understand a concept | The learner wants the artifact built for them | The learner wants a live operational answer | | Progress and exercises matter | Correctness and load-readiness matter | Speed and accuracy on current data matter | | Output is a lesson and a verified exercise | Output is a schema, object, check, transform, generator, or menu file | Output is an answer, not a file | tip Instance access is read-only by default. The tutor only writes to your instance for lessons on branches or proposed changes, and only after asking first. See the safety rules above. --- # Spec-Driven Development Spec-Driven Development (SDD) is a structured planning mode for complex or multi-part Infrahub builds. Instead of generating files immediately, the AI reasons through requirements with you first — capturing what needs to be built, validating the approach against Infrahub conventions, breaking the work into discrete tasks, and only generating once the plan is approved. The key benefit: structural mistakes — wrong relationship cardinality, missing `allow_upsert`, incorrect generic usage — are caught before any file is produced. ## When to use SDD vs. direct mode[​](#when-to-use-sdd-vs-direct-mode "Direct link to When to use SDD vs. direct mode") | Direct Mode | SDD | | --------------------------------------- | -------------------------------------------------------------------------- | | Adding an attribute to an existing node | Designing a new schema node with relationships | | Writing a single validation check | Building a generator chain | | Creating a menu section | Standing up a complete new domain (schema + objects + checks + generators) | | Populating a batch of objects | Refactoring relationships across multiple schema files | | Any well-scoped, single-skill task | Anything that involves design decisions or spans multiple skills | ## The SDD workflow[​](#the-sdd-workflow "Direct link to The SDD workflow") ### 1. Specify[​](#1-specify "Direct link to 1. Specify") Describe the feature or requirement in plain language. The AI captures requirements, asks clarifying questions about scope, Infrahub version, existing schema context, and dependencies. The goal is a complete picture of what needs to be built before any planning starts. ### 2. Plan[​](#2-plan "Direct link to 2. Plan") The AI produces an implementation plan. For each component to be built (schema nodes, generators, checks, transforms), it identifies which skill to use, what the inputs and outputs are, and what dependencies exist between steps. The plan is validated against Infrahub skill rules before it's presented. ### 3. Review[​](#3-review "Direct link to 3. Review") You review the plan. This is the key checkpoint — adjust the approach, correct assumptions, or request changes before any file is generated. The AI explains its reasoning for each decision so you can evaluate whether the approach is correct. ### 4. Implement[​](#4-implement "Direct link to 4. Implement") Once approved, the AI executes each task using the correct Infrahub skill. Tasks run sequentially where dependencies exist, or in parallel where they don't — the AI manages the dependency tree. ## Working with the plan[​](#working-with-the-plan "Direct link to Working with the plan") * **Interrogate** — ask why a specific decision was made, request alternatives, or ask what would change if a requirement changed * **Adjust** — describe what's wrong or what you want differently; the AI updates the plan before proceeding * **Approve** — explicitly confirm the plan is correct. The AI does not proceed without confirmation ## Sequential vs. parallel execution[​](#sequential-vs-parallel-execution "Direct link to Sequential vs. parallel execution") Simple builds execute sequentially: schema first, then objects, then checks. Complex builds with independent components can execute in parallel using sub-agents. The AI determines which tasks are independent based on the dependency tree it built during planning. ## Compatible SDD frameworks[​](#compatible-sdd-frameworks "Direct link to Compatible SDD frameworks") SDD works with any framework that supports a spec, plan, task, and implement workflow. The [infrahub-template](https://github.com/opsmill/infrahub-template) repository scaffolds an Infrahub project and documents the Spec Kit setup it expects, which is the quickest starting point. ## Infrahub routing for Spec Kit[​](#infrahub-routing-for-spec-kit "Direct link to Infrahub routing for Spec Kit") The [infrahub-speckit](https://github.com/opsmill/infrahub-speckit) extension wires the Infrahub skills into Spec Kit's own commands. It needs Spec Kit 0.8.0 or newer: ``` specify extension add infrahub-speckit --from https://github.com/opsmill/infrahub-speckit/archive/refs/heads/main.zip ``` The extension registers four hooks against the core Spec Kit skills. All four fire whether the skill was invoked by a slash command, by another skill, or by an autonomous agent, and all four no-op in a project with no `.infrahub.yml`: | Hook | When it fires | What it does | | ------------------ | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `before_specify` | Before `/speckit.specify` writes anything | Classifies the requested artifact type, verifies the Infrahub skills are installed and the instance is reachable, then selects the matching Infrahub spec template | | `before_plan` | Before `/speckit.plan` starts research | Re-invokes the skill that matches the artifact | | `before_implement` | Before `/speckit.implement` runs its tasks | Re-invokes the matching skill for each artifact type in `tasks.md` | | `after_implement` | After `/speckit.implement` finishes | Checks the finished cycle for evidence that a skill's own guidance had a gap, and offers to report it | All three `before_*` hooks halt with install guidance when the Infrahub skills are not present, and `before_specify` also halts when the instance is unreachable. The `after_implement` hook never halts: implementation is already done, so a missing skill, an error, or an ambiguous read all degrade to a no-op line rather than disrupting finished work. ### Skill-gap detection after implement[​](#skill-gap-detection-after-implement "Direct link to Skill-gap detection after implement") Infrahub skills fail quietly. A missing or unclear rule does not crash the run, it produces extra round trips and repeated nudges until the model works the answer out anyway. By the time the cycle ends, that friction is invisible. The `after_implement` hook looks for it while the session still holds the evidence. Detection is automatic; drafting and filing are not. On most cycles the whole hook is one line: ``` [infrahub-speckit] No skill-guidance friction detected this cycle. ``` When the evidence gate opens, you get an offer and nothing else: ``` [infrahub-speckit — friction offer, /speckit.implement] Skill: infrahub-managing-schemas Evidence: schema load failed on relationship cardinality, passed after correction Rule coverage: no rule file covers this topic An Infrahub skill's guidance may have a gap here. Reply "report it" to draft a skill-friction report for review. Nothing is filed without your approval. ``` Ignoring it costs nothing. Replying routes the session into the [Skill Gap Reporter](/skills/skills-reference/reporting-skill-gaps.md), which searches the tracker, decides whether the skill or Infrahub itself is at fault, and drafts a redacted report. That skill cannot file: it hands the draft to the [Issue Reporter](/skills/skills-reference/reporting-issues.md), which shows you the target repository and the full body, then asks how to submit it. You can stop at either gate, and the manual submission method sends nothing from your machine. Two probes open the gate: * **A verifier verdict**: a verifier rejected an artifact and later accepted it, red to green on the same target * **A correction delta**: you rewrote something the agent authored, in a way a rule could have prevented Nothing else does: * **A missing rule file on its own**: the rule-coverage read names the file that should have covered the topic, which is attribution for a finding rather than a trigger for one, and plenty of topics on a healthy cycle have no rule file * **Failures the skills do not own**: authentication, connectivity, a container that never started, or a product-side 5xx * **Session-shape counters**: retry counts, edit churn, repeated asks, and docs escapes rise for reasons unrelated to a skill's guidance, such as an unclear request The hook also stays quiet unless the skill guided the authoring inside that same implement run, and unless both the Skill Gap Reporter and the Issue Reporter are installed, since the accept path needs both. note Two per-project escape hatches live on the hook's entry in `.specify/extensions.yml`: `optional: true` turns the check into an opt-in offer, and `enabled: false` disables it. ## Example walkthrough: VLAN management domain[​](#example-walkthrough-vlan-management-domain "Direct link to Example walkthrough: VLAN management domain") This example uses [Spec Kit](https://github.com/github/spec-kit) to design a VLAN management domain from scratch — schema, object data, and a validation check. Any SDD framework that follows a spec, plan, task, and implement workflow produces a similar result. ### Setup[​](#setup "Direct link to Setup") Start from [infrahub-template](https://github.com/opsmill/infrahub-template), which scaffolds the repository layout the skills expect and ships `invoke` tasks for the instance: ``` uv tool run --from 'copier' copier copy https://github.com/opsmill/infrahub-template.git vlan-demo cd vlan-demo uv sync --all-packages invoke start ``` Then install the skills, Spec Kit, and the Infrahub routing extension: ``` npx skills add opsmill/infrahub-skills uv tool install specify-cli --from git+https://github.com/github/spec-kit.git specify init --here --integration claude specify extension add infrahub-speckit --from https://github.com/opsmill/infrahub-speckit/archive/refs/heads/main.zip ``` `specify init` creates the `.specify/` project configuration and installs slash commands into `.claude/commands/`. Having the instance up matters for more than the data: `before_specify` gates on `infrahubctl info`, so the cycle stops early rather than planning against an instance it cannot reach. ### Step 1: Specify[​](#step-1-specify "Direct link to Step 1: Specify") Run `/speckit.specify` and describe the feature: > I need a VLAN management domain for our data center network. We need to track VLANs with their numeric ID, name, operational status, and role. VLANs should be grouped into VLAN Groups. We also need a validation check that ensures no two VLANs in the same group share the same VLAN ID. Spec Kit produces `spec.md` capturing the requirements — two schema nodes (VLAN, VLANGroup), a grouping relationship, a uniqueness check, and the attributes for each. ### Step 2: Plan[​](#step-2-plan "Direct link to Step 2: Plan") Run `/speckit.plan`. The AI reads `spec.md` and the Infrahub skills to produce `plan.md`: > **Plan Summary** > > 1. **Schema** (Schema Manager) — Define `IpamVLAN` and `IpamVLANGroup` nodes in the `Ipam` namespace. > > > > * `IpamVLAN`: attributes `name` (Text), `vlan_id` (Number), `status` (Dropdown), `role` (Text, optional). Relationship to `IpamVLANGroup` (many-to-one). > * `IpamVLANGroup`: attribute `name` (Text). Relationship back to `IpamVLAN` (one-to-many). > * Both nodes get `human_friendly_id` and `display_label`. > * Uniqueness constraint on VLAN: `[vlan_id__value, group]`. > > 2. **Object data** (Object Manager) — Seed two VLAN groups and sample VLANs. > > 3. **Validation check** (Check Manager) — Python check querying VLANs per group, flagging duplicates. Register in `.infrahub.yml`. > > **Dependencies:** Schema must be loaded before objects. Check depends on schema being present. The plan is validated against Infrahub skills — the AI catches that `status` should be `kind: Dropdown` (not Text), that `vlan_id` is used instead of `id` (attribute names must be 3+ characters), and that the relationship between VLAN and VLANGroup needs matching identifiers on both sides. ### Step 3: Review[​](#step-3-review "Direct link to Step 3: Review") You review `plan.md` and adjust: > Looks good, but add `description` as an optional Text attribute on both nodes. And use "provisioned" instead of "provisioning" for the status choice. The AI updates the plan. ### Step 4: Tasks[​](#step-4-tasks "Direct link to Step 4: Tasks") Run `/speckit.tasks`. The AI breaks the plan into discrete steps in `tasks.md`: > * > Task 1: Create `schemas/vlan_management.yml` with IpamVLAN and IpamVLANGroup nodes > * > Task 2: Create `data/vlan_groups.yml` with two seed groups > * > Task 3: Create `data/vlans.yml` with sample VLANs assigned to groups > * > Task 4: Create `checks/check_vlan_id_uniqueness/` with Python check class and GraphQL query > * > Task 5: Register the check in `.infrahub.yml` > * > Task 6: Validate schema with `infrahubctl schema check` ### Step 5: Implement[​](#step-5-implement "Direct link to Step 5: Implement") Run `/speckit.implement`. The AI executes each task using the appropriate Infrahub skill. With the routing extension installed, the cycle closes with the friction check described in [Skill-gap detection after implement](#skill-gap-detection-after-implement). **Task 1 output** — `schemas/vlan_management.yml` (via Schema Manager): ``` version: "1.0" nodes: - name: VLAN namespace: Ipam label: "VLAN" icon: "mdi:lan-pending" human_friendly_id: - "name__value" display_label: "{{ name__value }}" uniqueness_constraints: - ["vlan_id__value", "group"] attributes: - name: name kind: Text order_weight: 1000 - name: vlan_id kind: Number order_weight: 1100 - name: status kind: Dropdown order_weight: 1200 choices: - name: active label: Active color: "#7fbf7f" - name: provisioned label: Provisioned color: "#ffff7f" - name: maintenance label: Maintenance color: "#ffd27f" - name: decommissioning label: Decommissioning color: "#bf7f7f" - name: role kind: Text optional: true order_weight: 1300 - name: description kind: Text optional: true order_weight: 1400 relationships: - name: group peer: IpamVLANGroup kind: Attribute cardinality: one optional: false identifier: "group__vlans" - name: VLANGroup namespace: Ipam label: "VLAN Group" icon: "mdi:folder-network" human_friendly_id: - "name__value" display_label: "{{ name__value }}" attributes: - name: name kind: Text order_weight: 1000 - name: description kind: Text optional: true order_weight: 1100 relationships: - name: vlans peer: IpamVLAN kind: Attribute cardinality: many identifier: "group__vlans" ``` **Task 4 output** — `checks/check_vlan_id_uniqueness/check.py` (via Check Manager): ``` from infrahub_sdk.checks import InfrahubCheck class CheckVlanIdUniqueness(InfrahubCheck): query = "check_vlan_id_uniqueness" def validate(self, data): vlans = data["IpamVLAN"]["edges"] groups = {} for vlan in vlans: node = vlan["node"] group_name = node["group"]["node"]["display_label"] vlan_id = node["vlan_id"]["value"] key = (group_name, vlan_id) if key in groups: self.log_error( message=f"Duplicate VLAN ID {vlan_id} in group {group_name}" ) groups[key] = node["display_label"] ``` **Task 5 output** — registration added to `.infrahub.yml`: ``` check_definitions: - name: check_vlan_id_uniqueness class_name: CheckVlanIdUniqueness file_path: checks/check_vlan_id_uniqueness/check.py query: check_vlan_id_uniqueness ``` ### Step 6: Report the gap it exposed[​](#step-6-report-the-gap-it-exposed "Direct link to Step 6: Report the gap it exposed") Task 6 is where the cycle hit friction. The first version of `schemas/vlan_management.yml` left `optional` off the `group` relationship, and relationships default to optional, so `infrahubctl schema check` rejected it: ``` cannot use group relationship, relationship must be mandatory ``` The AI added `optional: false`, re-ran the check, and it passed. That is a verifier going red to green on the same target inside a single implement run, which is exactly what the `after_implement` hook looks for. When the cycle closes, it prints one offer: > **\[infrahub-speckit — friction offer, /speckit.implement]** > > Skill: `infrahub-managing-schemas` Evidence: `infrahubctl schema check` rejected the VLAN node, then passed once `optional: false` was added to the constrained relationship Rule coverage: `rules/uniqueness-constraints.md` > > An Infrahub skill's guidance may have a gap here. Reply "report it" to draft a skill-friction report for review. Nothing is filed without your approval. Ignoring it ends the cycle there. Replying hands the session to the [Skill Gap Reporter](/skills/skills-reference/reporting-skill-gaps.md), which searches the tracker, classifies the finding, and drafts: > **bug: infrahub-managing-schemas: state the mandatory requirement where the constraint is written** > > **Skill**: infrahub-managing-schemas **Type**: bug **Confidence**: unconfirmed single observation **Tracker search**: `repo:opsmill/infrahub-skills uniqueness constraint mandatory relationship` returned no existing report > > **What was being attempted**: scoping a uniqueness constraint by the parent relationship of a child node. > > **Rules consulted**: `uniqueness-constraints.md`, which does state that a relationship in a constraint must be mandatory, cardinality one, and referenced bare. > > **Where it went wrong**: the constraint was written correctly but the relationship was left at its default, so the check failed on the mandatory requirement. One failed verifier run and one correction. > > **What finally worked**: `optional: false` on the constrained relationship. > > **Proposed rule change**: in `uniqueness-constraints.md`, put the three requirements next to the constraint example rather than in a later section, so the relationship and the constraint that binds it are authored together. It is a **bug** rather than a feature because a rule already claimed the ground and the model still got it wrong. Had no rule covered relationships in constraints, the same friction would have drafted as a `feat:` instead, and the offer would have read `Rule coverage: no rule file covers this topic`. That classification decides the title prefix and the target repository, which is why the coverage read runs before the draft. The draft carries the version lines the template requires, read from `infrahubctl info`, and nothing that identifies your infrastructure. The Skill Gap Reporter cannot file it: it hands the draft to the [Issue Reporter](/skills/skills-reference/reporting-issues.md), which shows you the target repository and the full body, then asks whether to submit through `gh`, a GitHub MCP server, or copy-paste. You can stop at either gate. Asking for the same thing without the extension installed works too: "report skill friction" reaches the Skill Gap Reporter directly. The hook only removes the need to notice the friction yourself. ### Result[​](#result "Direct link to Result") From a single natural-language description, the SDD workflow produced: * A schema with correct naming, Dropdown status, matching relationship identifiers, and uniqueness constraints * Seed data files with proper references * A working validation check registered in `.infrahub.yml` Each artifact follows Infrahub best practices because the AI applied the relevant skill at each step — not because the user knew the conventions upfront. --- # Infrahub Sync Infrahub Sync synchronizes infrastructure data between Infrahub and external systems — NetBox, Nautobot, IP Fabric, Slurp’it, Cisco ACI, Peering Manager, ServiceNow-style CMDBs, and any system with a REST/JSON API. Use it to migrate from a legacy system of record, run two systems side-by-side, or keep Infrahub in sync with another source of truth. Define a sync project declaratively in YAML, and the CLI generates the adapter code, calculates the diff, and applies only the deltas on each run. Infrahub Sync is open source under Apache 2.0, distributed on [PyPI](https://pypi.org/project/infrahub-sync/), and maintained on [GitHub](https://github.com/opsmill/infrahub-sync). Infrahub Sync supports the following: * **Migration from an existing system of record** — data moves from NetBox, Nautobot, or another source into Infrahub one model at a time, on the schedule the team chooses. The legacy system continues to operate during the migration. * **Recurring synchronization between systems** — a sync project runs as often as the environment requires. Each run calculates a fresh diff and applies only the deltas. * **Inventory population from network discovery** — adapters for IP Fabric and Slurp’it bring discovered network state into Infrahub as the source of truth, rather than requiring inventory to be entered by hand. * **Outbound data movement from Infrahub** — Infrahub data is published into monitoring, observability, or CMDB systems that need a current view of infrastructure. * **Translation between data models** — source fields map to destination fields through a declarative YAML configuration. Identifiers, relationships, and static values are handled in the same file. * **Diff preview before changes are applied** — `infrahub-sync diff` shows the differences between source and destination state without modifying either system. ## How it works[​](#how-it-works "Direct link to How it works") ### Concepts[​](#concepts "Direct link to Concepts") * **Sync project** — a directory containing a YAML configuration file (`config.yml`) that defines one synchronization between two systems. A project specifies the source, the destination, the sync order, and the per-model schema mapping. A team can have many sync projects, each managing a different source-destination pair. * **Adapter** — the component that connects Infrahub Sync to a specific system. Each adapter handles both communication (API calls, authentication, request handling) and translation (converting the system's data into the internal sync engine's format). Infrahub Sync ships with adapters for common systems and supports custom adapters for systems without a pre-built one. * **Schema mapping** — the part of the project configuration that defines how source fields map to destination fields. Direct field mappings, references between models, identifiers, and static values are all declared in YAML. Three CLI commands operate on a sync project: `generate`, `diff`, and `sync`. ### Define a sync project[​](#define-a-sync-project "Direct link to Define a sync project") Each sync project consists of a directory and a `config.yml` describing the sync. The configuration specifies the source adapter and destination adapter with their connection details, the order in which models should be synchronized, and how each source field maps to a destination field. Credentials reference environment variables rather than being embedded in the file. → [Create a sync project](/sync/creating-a-sync-project.md) · [Sync instance configuration](/sync/reference/config.md) ### Generate the adapter code[​](#generate-the-adapter-code "Direct link to Generate the adapter code") `infrahub-sync generate --name --directory ` reads the YAML configuration and produces the Python adapter and model code that `diff` and `sync` use. Re-run `generate` whenever the configuration or the schema mapping changes. ### Preview with `diff`[​](#preview-with-diff "Direct link to preview-with-diff") `infrahub-sync diff --name --directory ` reads both the source and destination, calculates what would change, and prints the result to the terminal. The destination is not modified. The `diff` command is read-only and is typically run before applying any sync — particularly during initial setup or when adjusting mappings. ### Execute the sync[​](#execute-the-sync "Direct link to Execute the sync") `infrahub-sync sync --name --directory ` applies the changes calculated by the diff, in the order defined by the project's `order` key — independent models first, then dependent models, then models that reference earlier ones. The sync is idempotent: if a run fails partway through, re-running calculates a fresh diff and applies whatever is still outstanding. Three `diffsync_flags` (`SKIP_UNMATCHED_DST` by default, `SKIP_UNMATCHED_SRC`, `SKIP_MODIFIED`) and per-mapping filters control what each run is allowed to change. → [Run a sync](/sync/running-a-sync.md) · [Sync CLI](/sync/reference/cli.md) ## Who it's for[​](#who-its-for "Direct link to Who it's for") ### Implementing Infrahub alongside an existing system of record[​](#implementing-infrahub-alongside-an-existing-system-of-record "Direct link to Implementing Infrahub alongside an existing system of record") Data lives in NetBox, Nautobot, IP Fabric, or another tool, and the team is adopting Infrahub. Infrahub Sync provides a path that does not require all teams to move at the same time, and that does not require writing integration code. → [Install Infrahub Sync](/sync/installation.md) · [Create a sync project](/sync/creating-a-sync-project.md) ### Operating Infrahub at steady state[​](#operating-infrahub-at-steady-state "Direct link to Operating Infrahub at steady state") Infrahub is deployed and needs to stay current with the other systems the team uses — IPAM, ITSM, monitoring, network discovery, or in-house databases. One sync project per source, run on the cadence the environment requires. → [Create a sync project](/sync/creating-a-sync-project.md) · [Run a sync](/sync/running-a-sync.md) ### Building inventory from network discovery[​](#building-inventory-from-network-discovery "Direct link to Building inventory from network discovery") Infrahub is populated from what is actually deployed in the network rather than from manually curated inventory. IP Fabric and Slurp’it adapters connect to discovery tools and bring discovered state into Infrahub. → [IP Fabric adapter](/sync/adapters/ipfabric.md) · [Slurp’it adapter](/sync/adapters/slurpit.md) ## What's included[​](#whats-included "Direct link to What's included") * **Pre-built adapter library** — adapters for Infrahub, NetBox, Nautobot, IP Fabric, Cisco ACI, LibreNMS, Observium, Peering Manager, Prometheus, and Slurp’it, plus a Generic REST API adapter for systems with HTTP/JSON APIs. Each adapter handles both communication and translation for its target system. * **Declarative YAML configuration** — a single file per sync project defines source, destination, sync order, and per-model schema mapping. Mappings support 14 filter operations (including `regex` and `is_ip_within`), per-field transforms, custom Jinja filters, and ordered cross-reference resolution. * **Sync engine** — built on the `diffsync` framework. Diffs and applies only deltas; three flags control what each run is allowed to change (creates, deletes, modifications). * **Typer-based CLI** — four commands: `list` (show available projects), `generate` (produce adapter code from the configuration), `diff` (preview changes), `sync` (apply changes). * **Custom adapter support** — for systems without a pre-built adapter, write a local custom adapter and load it from a filesystem path, a Python module path, or an installed entry point (`INFRAHUB_SYNC_ADAPTER_PATHS`). * **Custom CA certificate support** — connect to systems with self-signed or internal CA-issued TLS certificates. ### Adapter reference[​](#adapter-reference "Direct link to Adapter reference") | Adapter | Direction supported | | ---------------- | ------------------------------------------------------- | | Infrahub | source or destination | | NetBox | NetBox → Infrahub | | Nautobot | Nautobot → Infrahub | | IP Fabric | IP Fabric → Infrahub | | Cisco ACI | Cisco ACI → Infrahub | | Peering Manager | Peering Manager → Infrahub · Infrahub → Peering Manager | | Prometheus | Prometheus → Infrahub | | Slurp’it | Slurp’it → Infrahub | | LibreNMS | LibreNMS → Infrahub | | Observium | Observium → Infrahub | | Generic REST API | external system → Infrahub | ## Get started[​](#get-started "Direct link to Get started") 1. **Prerequisites** * A running [Infrahub](https://github.com/opsmill/infrahub) instance * Python 3.10–3.13 * Credentials and network access for the source and destination systems 2. **Install Infrahub Sync.** See [Install Infrahub Sync](/sync/installation.md) for the full setup steps. The short version: `pip install infrahub-sync` into a virtual environment. 3. **Choose your starting point.** * Setting up a sync project for the first time? → [Create a sync project](/sync/creating-a-sync-project.md) * Running an existing project? → [Run a sync](/sync/running-a-sync.md) ## Common questions[​](#common-questions "Direct link to Common questions") **Do I have to migrate everything at once?** No. Infrahub Sync is designed to move data one model at a time, on the team's own schedule. The legacy system keeps running throughout — there is no required cutover moment. **Does Infrahub Sync replace my scheduler?** No. Infrahub Sync runs as a CLI and is designed to plug into whatever scheduling tooling the team already uses — cron, CI jobs, Prefect, Dagster, or similar. There is no built-in scheduler by design. **What happens if a sync run fails partway through?** Sync runs are idempotent. Re-running the sync calculates a fresh diff against the current destination state and applies only what is still outstanding. Retries on failure are safe. **Can changes in the destination be overwritten by a sync?** By default, `SKIP_UNMATCHED_DST` is enabled, which preserves destination objects that have no corresponding object in the source. For destination objects that do have a source match, the sync's behavior depends on the configured `diffsync_flags`. Decide upfront which system is authoritative for each model and configure the flags accordingly. **What if my source system doesn't have a pre-built adapter?** Most systems with a REST/JSON API can use the Generic REST API adapter without modifications. For systems with non-standard APIs or custom logic requirements, build a local custom adapter. See [Local Adapters](/sync/adapters/local-adapters.md). **Can I run two sync projects at the same time?** Yes. Each sync project is independent — a separate directory, configuration, and CLI invocation. Schedule and operate each project on its own cadence. ## Additional resources[​](#additional-resources "Direct link to Additional resources") | What you want to do | Where to go | | ------------------------ | -------------------------------------------------------------------------------------------------------------------- | | Set up your environment | [Install Infrahub Sync](/sync/installation.md) | | Configure a sync project | [Create a sync project](/sync/creating-a-sync-project.md) · [Sync instance configuration](/sync/reference/config.md) | | Run a sync | [Run a sync](/sync/running-a-sync.md) | | CLI reference | [Sync CLI](/sync/reference/cli.md) | | All adapters | See the **Adapters** section in the sidebar | | Custom CA certificates | [Use custom CA certificates](/sync/custom-certificates.md) | | Build a custom adapter | [Local Adapters](/sync/adapters/local-adapters.md) | | Contribute | [Contributing guide](/sync/contributing.md) | | Source code | [github.com/opsmill/infrahub-sync](https://github.com/opsmill/infrahub-sync) | --- # Cisco ACI adapter ## What is Cisco ACI?[​](#what-is-cisco-aci "Direct link to What is Cisco ACI?") The *Cisco ACI* is a software-defined networking (SDN) solution that provides a policy-based, application-centric approach to managing and orchestrating network infrastructure. It is commonly used in data centers for scalable, policy-driven networking. ## Sync direction[​](#sync-direction "Direct link to Sync direction") Cisco ACI → Infrahub note Currently, the Cisco ACI adapter supports only **one-way synchronization** from ACI to Infrahub. Syncing data back into ACI is not yet supported. ## Configuration[​](#configuration "Direct link to Configuration") The adapter reads connection settings from the synchronization configuration and can be overridden by environment variables. Credentials should be provided via a secret manager or environment variables in production. ### Configuration parameters[​](#configuration-parameters "Direct link to Configuration parameters") ``` --- name: from-cisco-aci source: name: aci settings: url: "https://" username: "" password: "" api_endpoint: "api" # optional, default: api verify: true # boolean or string ("false","0") accepted ``` ### Environment variables[​](#environment-variables "Direct link to Environment variables") * CISCO\_APIC\_URL: overrides settings.url * CISCO\_APIC\_USERNAME: overrides settings.username * CISCO\_APIC\_PASSWORD: overrides settings.password * CISCO\_APIC\_VERIFY: overrides settings.verify; accepts true/false/0/1 (strings are normalized) Notes: * Credentials must come from environment variables or a secret manager in production. Never commit secrets. * The adapter normalizes verify to a boolean (strings like false, 0, no are treated as False). * The adapter records login timestamps in UTC to avoid timezone related issues and ensure correct token refresh behavior. ### Schema mapping examples[​](#schema-mapping-examples "Direct link to Schema mapping examples") #### Basic device mapping[​](#basic-device-mapping "Direct link to Basic device mapping") ``` - name: DcimPhysicalDevice mapping: "class/fabricNode.json" identifiers: ["name"] fields: - name: name mapping: "fabricNode.attributes.name" - name: serial mapping: "fabricNode.attributes.serial" - name: role mapping: "fabricNode.attributes.role" filters: - field: "fabricNode.attributes.fabricSt" operation: "==" value: "active" ``` #### Interface mapping with ACI Jinja filter[​](#interface-mapping-with-aci-jinja-filter "Direct link to Interface mapping with ACI Jinja filter") ``` - name: DcimPhysicalInterface mapping: "class/l1PhysIf.json" identifiers: ["device", "name"] fields: - name: name mapping: "l1PhysIf.attributes.id" - name: device mapping: "l1PhysIf.attributes.dn" reference: DcimPhysicalDevice - name: description mapping: "l1PhysIf.attributes.descr" transforms: - field: device expression: "{{ l1PhysIf.attributes.dn.split('/')[2].replace('node-', '') | aci_device_name }}" - field: status expression: "{{ 'active' if l1PhysIf.attributes.adminSt == 'up' else 'free' }}" filters: - field: "l1PhysIf.attributes.id" operation: "contains" value: "eth" ``` ## ACI-specific Jinja filters[​](#aci-specific-jinja-filters "Direct link to ACI-specific Jinja filters") The ACI adapter provides custom Jinja filters for data transformation: ### `aci_device_name` filter[​](#aci_device_name-filter "Direct link to aci_device_name-filter") The `aci_device_name` filter resolves ACI node IDs to device names automatically. This is particularly useful when mapping physical interfaces to their parent devices. **Usage:** ``` {{ node_id | aci_device_name }} ``` **Example:** * Input: `"102"` (ACI node ID) * Output: `"spine-102"` (actual device name from ACI) **Common use case in transforms:** ``` transforms: - field: device expression: "{{ l1PhysIf.attributes.dn.split('/')[2].replace('node-', '') | aci_device_name }}" ``` This transform: 1. Extracts the node ID from the ACI Distinguished Name (DN) 2. Removes the `"node-"` prefix (for example: `"node-102"` → `"102"`) 3. Uses the `aci_device_name` filter to resolve the node ID to the actual device name **How it works:** * The adapter automatically queries the ACI `fabricNode` class during initialization * Builds a mapping of node IDs to device names * The filter performs a lookup with a fallback to the original node ID if not found ## Generating the models[​](#generating-the-models "Direct link to Generating the models") Use the generate command to produce models from the schema mapping and examples: ``` uv run infrahub-sync generate --name from-cisco-aci --directory examples/ ``` ## Common issues and troubleshooting[​](#common-issues-and-troubleshooting "Direct link to Common issues and troubleshooting") ### Authentication and connectivity[​](#authentication-and-connectivity "Direct link to Authentication and connectivity") * If you see token refresh errors, ensure the APIC response includes refreshTimeoutSeconds; the adapter forces re-login when refresh data is unavailable. * For TLS verification problems, set CISCO\_APIC\_VERIFY to false in a secure environment (use with caution). ### Device reference resolution[​](#device-reference-resolution "Direct link to Device reference resolution") * **Interface-device relationship errors**: If you see "Unable to locate the node device" errors, ensure: * The `DcimPhysicalDevice` mapping runs before `DcimPhysicalInterface` in the `order` configuration * The device transform uses the `aci_device_name` filter correctly: `{{ node_id | aci_device_name }}` * The ACI fabric node query succeeds (check logs for "Built ACI device mapping" messages) ### Jinja filter issues[​](#jinja-filter-issues "Direct link to Jinja filter issues") * **`aci_device_name` filter not found**: Ensure you're using the ACI adapter and the filter is correctly spelled * **Filter returns node ID instead of device name**: Check that the fabric node query succeeded during adapter initialization * **Transform expression errors**: Verify the DN parsing logic extracts the correct node ID: ``` expression: "{{ l1PhysIf.attributes.dn.split('/')[2].replace('node-', '') | aci_device_name }}" ``` ### General debugging[​](#general-debugging "Direct link to General debugging") * Enable DEBUG logging for the adapter to see raw fetched objects and mapping decisions. Logs will not include secrets. * Check the device mapping build process in logs: look for "Built ACI device mapping with X entries" --- # Choose an adapter Infrahub Sync includes adapters for the most common infrastructure systems and a Generic REST API adapter for everything else. For systems with non-standard APIs, build a custom adapter. ## How to choose[​](#how-to-choose "Direct link to How to choose") If a pre-built adapter exists for the system you want to connect, use it. If the system has a REST/JSON API but no dedicated adapter, use the Generic REST API adapter. Only build a custom adapter when the system has a non-standard API, requires custom authentication logic, or has data access patterns the Generic REST API adapter cannot handle. ## Pre-built adapters[​](#pre-built-adapters "Direct link to Pre-built adapters") Each adapter handles two responsibilities for its target system: **communication** (talking to the system's API, authentication, request handling) and **translation** (converting the system's data structures into the `diffsync` models the sync engine works with). | Adapter | Direction supported | Best for | | ---------------- | ------------------------------------------------------- | --------------------------------------------------------------------------------- | | Infrahub | source or destination | Any sync project where Infrahub is one side of the sync | | NetBox | NetBox → Infrahub | Existing NetBox deployments migrating to or syncing with Infrahub | | Nautobot | Nautobot → Infrahub | Existing Nautobot deployments migrating to or syncing with Infrahub | | IP Fabric | IP Fabric → Infrahub | Building Infrahub inventory from IP Fabric network discovery data | | Slurp'it | Slurp'it → Infrahub | Building Infrahub inventory from Slurp'it network discovery data | | Cisco ACI | Cisco ACI → Infrahub | Syncing data from Cisco ACI fabrics into Infrahub | | Peering Manager | Peering Manager → Infrahub · Infrahub → Peering Manager | Bi-directional sync between Peering Manager and Infrahub | | PeeringDB | PeeringDB → Infrahub | Pulling autonomous system and internet exchange data from PeeringDB into Infrahub | | Prometheus | Prometheus → Infrahub | Pulling target or metric metadata from Prometheus into Infrahub | | LibreNMS | LibreNMS → Infrahub | Syncing device inventory and metadata from LibreNMS into Infrahub | | Observium | Observium → Infrahub | Syncing device inventory and metadata from Observium into Infrahub | | Device42 | Device42 → Infrahub | Syncing IT asset and dependency data from Device42 into Infrahub | | Generic REST API | external system → Infrahub | Any external system with a REST/JSON API and no dedicated adapter | For per-adapter details — connection parameters, supported models, known limitations — see the individual adapter pages in the Adapters sidebar. ## When to use the Generic REST API adapter[​](#when-to-use-the-generic-rest-api-adapter "Direct link to When to use the Generic REST API adapter") The Generic REST API adapter works with any system that exposes data over HTTP/JSON. It handles authentication and basic data access patterns out of configuration alone. Set the API endpoint, the authentication method, and the structure of the data you want to read in the same YAML format as every other adapter. Common cases where the Generic REST API adapter is the right choice: * **ServiceNow** — pull device, location, or service data from a ServiceNow CMDB. * **Infoblox** — pull IP address management data from Infoblox. * **Internal IPAM tools** — pull data from a homegrown IPAM or asset database. * **Vendor-specific inventory APIs** — pull data from a vendor's cloud-hosted inventory service. * **Spreadsheet-based data** — when a team exports a spreadsheet to a JSON file served over HTTP, the Generic REST API adapter can read it. Most source systems without a dedicated adapter are reachable directly with the Generic REST API adapter. See [Generic REST API adapter](/sync/adapters/genericrestapi.md) for configuration details. ## When to build a custom adapter[​](#when-to-build-a-custom-adapter "Direct link to When to build a custom adapter") A custom adapter is the right choice when the Generic REST API adapter can't handle the source system. Common reasons: * **Non-REST APIs.** The system uses gRPC, SOAP, GraphQL with non-standard patterns, a binary protocol, or any other non-REST interface. * **Custom authentication flows.** The system requires multi-step authentication, token refresh logic, or other auth patterns standard configuration cannot express. * **Stateful or session-based access.** The system requires maintaining a session across multiple requests in a way the Generic REST API adapter does not support. * **Complex pagination or query patterns.** The data access requires logic beyond what configuration can express — for example, conditional pagination based on response content. * **Data transformations that exceed Jinja filter capabilities.** The source data requires substantial preprocessing before mapping into `diffsync` models. A custom adapter is a Python class implementing the same interface as the pre-built adapters. Load it from a filesystem path, a Python module path, or an installed entry point. See [Local adapters](/sync/adapters/local-adapters.md) for how to build and load one. If you build a custom adapter for a system other teams might use, consider contributing it back to the Infrahub Sync repository. ## Choosing between two valid options[​](#choosing-between-two-valid-options "Direct link to Choosing between two valid options") Sometimes more than one adapter works for the same source system. For example: a Nautobot deployment is reachable through the Nautobot adapter or through the Generic REST API adapter pointed at Nautobot's REST API. Use the dedicated adapter when one exists — it understands the source system's data model and handles edge cases (nested data, authentication) that would otherwise need to be configured manually. The Generic REST API adapter is the fallback, not the default. ## Multiple adapters in one sync project[​](#multiple-adapters-in-one-sync-project "Direct link to Multiple adapters in one sync project") A sync project has one source adapter and one destination adapter. To sync data from multiple sources into Infrahub, create one sync project per source. Each project has its own configuration, sync order, and schedule. This keeps each sync independent — a failure in one project does not affect others, and each project can be scheduled and operated separately. ## Related[​](#related "Direct link to Related") * [Generic REST API adapter](/sync/adapters/genericrestapi.md) * [Local adapters](/sync/adapters/local-adapters.md) * [Create a sync project](/sync/creating-a-sync-project.md) * [Sync instance configuration](/sync/reference/config.md) --- # Device42 adapter ## What is Device42?[​](#what-is-device42 "Direct link to What is Device42?") [Device42](https://www.device42.com/) is a commercial IT infrastructure and asset management platform. It discovers and inventories physical and virtual infrastructure — servers, network devices, applications, services — and tracks the dependencies between them. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * Device42 → Infrahub info The Device42 adapter supports only **one-way synchronization** from Device42 to Infrahub. Writing data back into Device42 is not supported. Under Construction --- # GenericRestAPI adapter The GenericRestAPI adapter is a flexible, configurable adapter that can connect to any REST API following common patterns. It's designed to reduce code duplication and provide a foundation for creating adapters for new systems without having to build them from scratch. This adapter can be used in two ways: 1. As a standalone adapter for systems that have a REST API 2. As a base for building more specialized adapters (like LibreNMS, Observium, PeeringDB) ## Key features[​](#key-features "Direct link to Key features") * Flexible authentication methods (token, API key, basic auth) * Configurable endpoints and URL structures * Response format handling with customizable extraction * Transformation and filtering capabilities * Environment variable support for credentials ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * GenericRestAPI → Infrahub info Currently, the GenericRestAPI adapter supports only **one-way synchronization** from a REST API source to Infrahub. Syncing data back to the source is not yet supported and need to be done on custom adapters. Those adapters can use the generic one as base similar to what is done with Peering Manager adapter. ## Configuration[​](#configuration "Direct link to Configuration") ### Basic configuration[​](#basic-configuration "Direct link to Basic configuration") To use the GenericRestAPI adapter, specify `genericrestapi` as the name in your configuration: ``` source: name: genericrestapi settings: url: "https://api.example.com" api_endpoint: "/api/v1" auth_method: "token" token: "YOUR_API_TOKEN" # Better to use environment variables ``` ### Configuration parameters[​](#configuration-parameters "Direct link to Configuration parameters") | Parameter | Description | Default | Required | | ---------------------- | ---------------------------------------------------------------- | --------- | -------------- | | `url` | Base URL of the API | None | Yes | | `api_endpoint` | API endpoint path | `/api/v0` | No | | `auth_method` | Authentication method (token, x-auth-token, api-key, key, basic) | `token` | No | | token | API token for token-based auth | None | For token auth | | username | Username for basic auth | None | For basic auth | | password | Password for basic auth | None | For basic auth | | `verify_ssl` | Whether to verify SSL certificates | `true` | No | | timeout | Request timeout in seconds | `30` | No | | `params` | Additional query parameters to include in all requests | `{}` | No | | `response_key_pattern` | Pattern for extracting data from responses | None | No | ### Environment variables[​](#environment-variables "Direct link to Environment variables") You can specify credentials using environment variables instead of storing them directly in the configuration: | Setting | Environment Variables | Description | | -------- | --------------------- | ------------------- | | `url` | URL, ADDRESS | API base URL | | token | TOKEN | API token/key | | username | USERNAME | Basic auth username | | password | PASSWORD | Basic auth password | The adapter will check for these environment variables in the order specified. ### Custom response handling[​](#custom-response-handling "Direct link to Custom response handling") If your API has a specific response format, you can customize how the data is extracted using the `response_key_pattern` setting: ``` source: name: genericrestapi settings: # ...other settings... response_key_pattern: "data.{resource}.items" ``` This pattern will be formatted with: * `{resource}`: The endpoint being queried * `{default}`: The default key (last part of the endpoint path) ## Schema mapping[​](#schema-mapping "Direct link to Schema mapping") The schema mapping defines how data from the API is mapped to Infrahub models: ``` schema_mapping: - name: InfraDevice # Infrahub model mapping: devices # API endpoint identifiers: ["name"] # Unique identifier fields filters: # Optional filters to apply - field: hostname operation: contains value: "router" fields: - name: name # Infrahub field mapping: hostname # API field - name: type mapping: device_type - name: site # Reference to another model mapping: location reference: LocationSite ``` ### Field mapping types[​](#field-mapping-types "Direct link to Field mapping types") * **Direct mapping**: Maps a field from the API to an Infrahub field * **Static value**: Sets a constant value for an Infrahub field * **Reference**: Links to another Infrahub model ``` fields: # Direct mapping - name: name mapping: hostname # Static value - name: type static: "Network Device" # Reference to another model - name: site mapping: location_id reference: LocationSite ``` ### Filtering records[​](#filtering-records "Direct link to Filtering records") You can filter records from the API before processing: ``` filters: - field: status operation: equals value: "active" - field: hostname operation: contains value: "prod" ``` Supported operations: equals, `not_equals`, contains, `not_contains`, in, `not_in` ## Examples[​](#examples "Direct link to Examples") ### Basic REST API example[​](#basic-rest-api-example "Direct link to Basic REST API example") ``` --- name: rest-api-example source: name: genericrestapi settings: url: "http://api.example.com" api_endpoint: "/api/v1" auth_method: "token" token: "${API_TOKEN}" # Uses environment variable destination: name: infrahub settings: url: "http://localhost:8000" order: [ "InfraDevice", "IpamIPAddress", ] schema_mapping: - name: InfraDevice mapping: devices identifiers: ["name"] fields: - name: name mapping: hostname - name: serial_number mapping: serial - name: type static: "Network Device" - name: IpamIPAddress mapping: ips identifiers: ["address"] fields: - name: address mapping: ip_address - name: description mapping: description ``` ### LibreNMS example[​](#librenms-example "Direct link to LibreNMS example") LibreNMS is one of the systems that use the GenericRestAPI adapter: ``` --- name: from-librenms source: name: genericrestapi settings: url: "http://librenms.example.com" api_endpoint: "api/v0" auth_method: "x-auth-token" token: "${LIBRENMS_TOKEN}" destination: name: infrahub settings: url: "http://localhost:8000" order: [ "CoreStandardGroup", "LocationSite", "IpamIPAddress", "InfraDevice", ] schema_mapping: - name: CoreStandardGroup mapping: devicegroups fields: - name: name mapping: name - name: description mapping: desc - name: LocationSite mapping: resources/locations identifiers: ["name"] fields: - name: name mapping: location - name: description mapping: location - name: IpamIPAddress mapping: devices identifiers: ["address"] fields: - name: address mapping: ip - name: description mapping: hostname - name: InfraDevice mapping: devices identifiers: ["name"] fields: - name: name mapping: hostname - name: serial_number mapping: serial - name: type mapping: hardware - name: site mapping: location reference: LocationSite ``` ## Common issues[​](#common-issues "Direct link to Common issues") ### Authentication failures[​](#authentication-failures "Direct link to Authentication failures") If you encounter authentication issues: 1. Verify your token/credentials are correct 2. Check that you're using the correct `auth_method` for your API 3. Ensure the API endpoint path is correct ### Response parsing errors[​](#response-parsing-errors "Direct link to Response parsing errors") If the adapter fails to parse the API response: 1. Check the API documentation for the correct response format 2. Use the `response_key_pattern` setting to customize data extraction 3. Consider implementing a custom adapter extending GenericRestAPI ### Connection timeout[​](#connection-timeout "Direct link to Connection timeout") For slow APIs or large datasets: 1. Increase the timeout setting 2. Add pagination parameters if the API supports it 3. Use filters to reduce the amount of data fetched ## Building specialized adapters[​](#building-specialized-adapters "Direct link to Building specialized adapters") You can extend the GenericRestAPI adapter to create more specialized adapters: ``` from infrahub_sync.adapters.genericrestapi import GenericrestapiAdapter class MySpecializedAdapter(GenericrestapiAdapter): def __init__(self, target, adapter, config, *args, **kwargs): # Override adapter_type to change the adapter name in logs super().__init__(target, adapter, config, adapter_type="MySpecialized", *args, **kwargs) # Override methods as needed for specialized behavior def _extract_objects_from_response(self, response_data, resource_name, element): # Custom extraction logic return custom_data ``` --- # Infrahub adapter ## What is Infrahub?[​](#what-is-infrahub "Direct link to What is Infrahub?") Infrahub is an open-source infrastructure management platform that combines a version-controlled database with a schema-driven data model. It serves as the central hub for network automation, providing a single source of truth for infrastructure data. ## Requirements[​](#requirements "Direct link to Requirements") This adapter uses the [Infrahub SDK](https://pypi.org/project/infrahub-sdk), which is included as a dependency of infrahub-sync. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * Source: Infrahub can be used as a data source * Destination: Infrahub is the primary destination for most sync configurations info The Infrahub adapter supports **bidirectional synchronization**. Most commonly, Infrahub is used as the destination, receiving data from other systems like NetBox, Nautobot, or IP Fabric. ## Configuration[​](#configuration "Direct link to Configuration") The adapter reads connection settings from the synchronization configuration and can be overridden by environment variables. ### Configuration parameters[​](#configuration-parameters "Direct link to Configuration parameters") * As Destination * As Source ``` --- name: from-netbox source: name: netbox settings: url: "https://" token: "" destination: name: infrahub settings: url: "http://localhost:8000" token: "" # Optional if using env var branch: "main" # Optional, defaults to main verify_ssl: true # Optional, defaults to true source: "data-sync" # Optional, CoreAccountGroup name for lineage source owner: "network-team" # Optional, CoreAccountGroup name for lineage owner ``` ``` --- name: to-peering-manager source: name: infrahub settings: url: "http://localhost:8000" token: "" branch: "main" destination: name: peeringmanager settings: url: "https://" token: "" ``` ### Environment variables[​](#environment-variables "Direct link to Environment variables") | Variable | Description | | ------------------------------------ | --------------------------------------------------------- | | `INFRAHUB_ADDRESS` or `INFRAHUB_URL` | Infrahub server URL (overrides `settings.url`) | | `INFRAHUB_API_TOKEN` | API token for authentication (overrides `settings.token`) | ### Settings reference[​](#settings-reference "Direct link to Settings reference") | Setting | Type | Required | Default | Description | | ------------ | ------- | -------- | ------- | ---------------------------------------------- | | `url` | string | Yes\* | - | Infrahub server URL | | `token` | string | Yes\* | - | API token for authentication | | `branch` | string | No | main | Target branch for operations | | `verify_ssl` | boolean | No | true | Verify SSL certificates | | `source` | string | No | - | CoreAccountGroup name to use as lineage source | | `owner` | string | No | - | CoreAccountGroup name to use as lineage owner | \*Can be provided via environment variables instead. ## Schema mapping[​](#schema-mapping "Direct link to Schema mapping") When Infrahub is used as a destination, the schema mapping defines how source data maps to Infrahub node types. The `name` field in schema\_mapping corresponds to the Infrahub schema kind. ### Basic example[​](#basic-example "Direct link to Basic example") ``` schema_mapping: - name: InfraDevice mapping: dcim.devices identifiers: ["name"] fields: - name: name mapping: name - name: description mapping: description - name: status mapping: status.value - name: InfraInterface mapping: dcim.interfaces identifiers: ["device", "name"] fields: - name: name mapping: name - name: device mapping: device.name reference: InfraDevice ``` ### Relationship handling[​](#relationship-handling "Direct link to Relationship handling") The Infrahub adapter automatically resolves relationships based on the schema: * **One-to-one relationships**: Resolved using the unique identifier of the related node * **One-to-many relationships**: Resolved as a list of unique identifiers ``` fields: # One-to-one relationship - name: site mapping: site.name reference: LocationSite # One-to-many relationship - name: tags mapping: tags reference: BuiltinTag ``` ## Lineage tracking[​](#lineage-tracking "Direct link to Lineage tracking") When syncing data into Infrahub, the adapter tracks the source and owner of each record using Infrahub's lineage system. This enables: * Tracking which system created each record * Protecting source-managed attributes from manual edits * Assigning ownership to teams or groups Source and owner lineage is applied to newly created relationship edges and to relationship peers newly linked by an update. Rerunning a sync does not backfill lineage onto unchanged, pre-existing relationship edges. ### Default behavior[​](#default-behavior "Direct link to Default behavior") By default, the adapter looks for a `CoreAccount` with a name matching the source adapter name. This account is used as both the source and owner for all synced attributes. ### Using account groups[​](#using-account-groups "Direct link to Using account groups") You can override the default behavior by specifying `source` and/or `owner` in the destination settings. When specified, these values are used to look up a `CoreAccountGroup` by name: ``` destination: name: infrahub settings: url: "http://localhost:8000" token: "" source: "data-sync" # Uses CoreAccountGroup "data-sync" as source owner: "network-team" # Uses CoreAccountGroup "network-team" as owner ``` This allows you to: * Use different groups for source and owner * Override only one while keeping the default account for the other * Assign ownership to a team rather than an individual account | Configuration | Source | Owner | | ----------------------- | ------------------------- | ------------------------- | | Neither specified | CoreAccount (source name) | CoreAccount (source name) | | Only `source` specified | CoreAccountGroup | CoreAccount (source name) | | Only `owner` specified | CoreAccount (source name) | CoreAccountGroup | | Both specified | CoreAccountGroup | CoreAccountGroup | ## Generating the models[​](#generating-the-models "Direct link to Generating the models") Use the generate command to produce Python models from your configuration: ``` uv run infrahub-sync generate --name from-netbox --directory examples/ ``` ## Common issues and troubleshooting[​](#common-issues-and-troubleshooting "Direct link to Common issues and troubleshooting") ### Authentication errors[​](#authentication-errors "Direct link to Authentication errors") * Verify `INFRAHUB_API_TOKEN` is set correctly * Ensure the token has appropriate permissions for the target branch * Check that `INFRAHUB_ADDRESS` points to the correct server ### Schema mismatch errors[​](#schema-mismatch-errors "Direct link to Schema mismatch errors") * Ensure your Infrahub schema matches the expected node types in schema\_mapping * Run `infrahubctl schema load` to update the schema if needed * Verify field names match the Infrahub schema attribute names ### Relationship resolution failures[​](#relationship-resolution-failures "Direct link to Relationship resolution failures") * Ensure referenced objects are synced before objects that reference them (check `order` in configuration) * Verify the unique identifier used in references matches the target object's identifier * Check logs for "Unable to find ... in the Store" messages ### Branch operations[​](#branch-operations "Direct link to Branch operations") * Verify the target branch exists in Infrahub * Use `infrahubctl branch list` to see available branches * Create branches with `infrahubctl branch create ` if needed ### SSL certificate issues[​](#ssl-certificate-issues "Direct link to SSL certificate issues") * Set `verify_ssl: false` in settings for self-signed certificates (development only) * For production, ensure proper CA certificates are installed --- # IP Fabric adapter ## What is IP Fabric?[​](#what-is-ip-fabric "Direct link to What is IP Fabric?") The **IP Fabric** network infrastructure management platform provides on-demand network discovery, advanced analytics, and in-depth engineering visibility. Its lightweight discovery capabilities (via SSH or Telnet) quickly detect the current network state, including detailed data for each address and port. A network model of gathered data reconstructs the topologies for each switching and routing protocol, enabling cross-technology analysis of upstream and downstream relationships. Dependencies and dependents are calculated for each network element, allowing analysis to represent each aspect of the network in the context of productivity impact on downstream hosts and network devices. The immediate productivity impact of performance and capacity is also calculated for each user and every element. ## Requirements[​](#requirements "Direct link to Requirements") This Adapter uses [IP Fabric SDK](https://pypi.org/project/ipfabric). You will need to install it beforehand. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * IP Fabric → Infrahub info Currently, the IP Fabric adapter supports only **one-way synchronization** from IP Fabric to Infrahub. Syncing data back into IP Fabric is not yet supported. Although IP Fabric data cannot be edited, additional information such as **`Device Attributes`** and **`Site Separation`** rules can be updated from Infrahub data. ## Schema[​](#schema "Direct link to Schema") Our `infrahub` repository contains an **example schema** that serves as a starting point for syncing IP Fabric data into Infrahub. This schema follows best practices for Infrahub, **but it does not map the IP Fabric data model one-to-one** since Infrahub may have additional use cases. [Schemahttps://github.com/opsmill/infrahub/blob/stable/models/examples/ipfabric/ipfabric.yml](https://github.com/opsmill/infrahub/blob/stable/models/examples/ipfabric/ipfabric.yml) ### Installing the example schema[​](#installing-the-example-schema "Direct link to Installing the example schema") To install the example schema into Infrahub, follow these steps: ``` mkdir ipfabric-sync cd ipfabric-sync curl -o schema.yml https://raw.githubusercontent.com/opsmill/infrahub/refs/heads/stable/models/examples/ipfabric/ipfabric.yml infrahubctl schema load schema.yml ``` ## Configuration[​](#configuration "Direct link to Configuration") `infrahub-sync` allows defining what gets synchronized from a source to a destination. Included in the examples is a config.yml file that matches the example schema. [config.ymlhttps://github.com/opsmill/infrahub-sync/blob/main/examples/ipfabric\_to\_infrahub/config.yml](https://github.com/opsmill/infrahub-sync/blob/main/examples/ipfabric_to_infrahub/config.yml) To download the example `config.yml` ``` curl https://raw.githubusercontent.com/opsmill/infrahub-sync/refs/heads/main/examples/ipfabric_to_infrahub/config.yml > config.yml ``` ### Configuration parameters[​](#configuration-parameters "Direct link to Configuration parameters") The `source.name` is set to `ipfabricsync`, instructing `infrahub-sync` to use the IP Fabric adapter. The settings dictionary is passed directly to the IP Fabric Python SDK for authentication and connection details. Some of these parameters can be found in the [IP Fabric SDK Docs](https://docs.ipfabric.io/latest/integrations/python/#environment-variables). Below is a snippet from the example config.yml file: ``` --- name: from-ipfabric source: name: ipfabricsync settings: base_url: "https://" auth: "" verify_ssl: true # Default value ``` ### Schema mapping[​](#schema-mapping "Direct link to Schema mapping") The configuration file allows mapping tables from IP Fabric into Infrahub models. Below is an example showing how to: * Set the destination Infrahub model (`InfraDevice`) * Map source data from IP Fabric’s Path (`tables/inventory/devices`) * Specify field mappings between IP Fabric and Infrahub models ``` schema_mapping: - name: InfraDevice identifiers: ["hostname"] mapping: tables/inventory/devices fields: - name: hostname mapping: hostname - name: serial_number mapping: sn - name: hardware_serial_number mapping: snHw - name: fqdn mapping: fqdn - name: model mapping: model reference: TemplateDeviceType - name: location mapping: siteName reference: LocationGeneric - name: platform mapping: platform reference: InfraPlatform - name: version mapping: version reference: InfraNOSVersion ``` The table URI can be found by selecting `Table Description` in any IP Fabric table and copying everything after the version in the URL description. For more details regarding, refer to the [IP Fabric API Docs](https://docs.ipfabric.io/latest/IP_Fabric_API/?h=api+doc#technology-table-endpoints). --- # LibreNMS adapter ## What is LibreNMS?[​](#what-is-librenms "Direct link to What is LibreNMS?") LibreNMS is an open-source network monitoring system that automatically discovers, monitors, and alerts on your network devices using pre-configured setups, providing comprehensive insights for proactive network management. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * LibreNMS → Infrahub info Currently, the LibreNMS adapter supports only **one-way synchronization** from LibreNMS to Infrahub. Syncing data back into LibreNMS is not yet supported. Under Construction --- # Local Adapters ## Using local adapters[​](#using-local-adapters "Direct link to Using local adapters") The infrahub-sync tool supports loading adapters from various sources, including local filesystem paths. This feature allows you to develop and use custom adapters without having to integrate them into the main codebase. ### Adapter resolution paths[​](#adapter-resolution-paths "Direct link to Adapter resolution paths") The plugin loader can resolve adapters from: 1. **Built-ins**: Adapters that ship with infrahub-sync (`infrahub_sync.adapters.`) 2. **Dotted paths**: Python module paths (`myproj.adapters.foo:MyAdapter`) 3. **Filesystem paths**: Local files or directories (`./adapters/foo.py:MyAdapter`) 4. **Python entry points**: Installed packages that register entry points ### Configuring local adapters[​](#configuring-local-adapters "Direct link to Configuring local adapters") To use a local adapter in your configuration, specify the filesystem path in the `adapter` field: ``` source: name: mycustom adapter: ./path/to/my_adapter.py:MyCustomAdapter settings: # Your adapter settings here ``` If your adapter class name follows naming conventions (for example: `MycustomAdapter`), you can omit the class name: ``` source: name: mycustom adapter: ./path/to/my_adapter.py settings: # Your adapter settings here ``` ### Environment variables[​](#environment-variables "Direct link to Environment variables") You can also specify additional adapter search paths via the `INFRAHUB_SYNC_ADAPTER_PATHS` environment variable: ``` # Unix/Linux/macOS export INFRAHUB_SYNC_ADAPTER_PATHS="/path/to/adapters:/another/path" # Windows set INFRAHUB_SYNC_ADAPTER_PATHS="C:\path\to\adapters;D:\another\path" ``` ### Creating a custom adapter[​](#creating-a-custom-adapter "Direct link to Creating a custom adapter") A minimal custom adapter needs to extend `diffsync.Adapter` and implement the necessary methods: ``` from diffsync import Adapter from infrahub_sync import DiffSyncMixin class MyCustomAdapter(DiffSyncMixin, Adapter): def __init__(self, target, adapter, config, *args, **kwargs): super().__init__(*args, **kwargs) self.target = target self.settings = adapter.settings or {} self.config = config def model_loader(self, model_name, model): # Your implementation to load data into the model pass ``` For a more complete example, refer to the example in the repository: [Custom Adapter Examplehttps://github.com/opsmill/infrahub-sync/tree/main/examples/custom\_adapter](https://github.com/opsmill/infrahub-sync/tree/main/examples/custom_adapter) ### Adding custom Jinja filters[​](#adding-custom-jinja-filters "Direct link to Adding custom Jinja filters") Custom adapters can provide their own Jinja filters for use in transform expressions. This is particularly useful for adapter-specific data transformations. #### Implementing custom filters[​](#implementing-custom-filters "Direct link to Implementing custom filters") To add custom filters to your adapter, implement the `_add_custom_filters` class method in your DiffSync model class: ``` from typing import Any, ClassVar from diffsync import DiffSyncModel from infrahub_sync import DiffSyncModelMixin class MyCustomModel(DiffSyncModelMixin, DiffSyncModel): # Store any data needed by filters as class variables _my_mapping: ClassVar[dict[str, str]] = {} @classmethod def set_my_mapping(cls, mapping: dict[str, str]) -> None: """Set mapping data for use in filters.""" cls._my_mapping = mapping @classmethod def _add_custom_filters(cls, native_env: Any, item: dict[str, Any]) -> None: """Add custom filters to the Jinja environment.""" def my_custom_filter(value: str) -> str: """Custom filter that transforms values using stored mapping.""" return cls._my_mapping.get(str(value), value) def format_identifier(value: str) -> str: """Another custom filter for formatting identifiers.""" return f"ID-{value.upper()}" # Register filters with the Jinja environment native_env.filters["my_custom_filter"] = my_custom_filter native_env.filters["format_identifier"] = format_identifier ``` #### Setting up filter data[​](#setting-up-filter-data "Direct link to Setting up filter data") If your filters need data (like mappings, lookup values, etc.), initialize it in your adapter: ``` class MyCustomAdapter(DiffSyncMixin, Adapter): def __init__(self, target, adapter, config, *args, **kwargs): super().__init__(*args, **kwargs) # ... other initialization # Build data needed by filters my_mapping = self._build_custom_mapping() # Pass data to model class for filter use MyCustomModel.set_my_mapping(my_mapping) def _build_custom_mapping(self) -> dict[str, str]: """Build mapping data from your data source.""" # Implementation depends on your data source return {"key1": "value1", "key2": "value2"} ``` #### Using custom filters in configuration[​](#using-custom-filters-in-configuration "Direct link to Using custom filters in configuration") Once implemented, use your custom filters in transform expressions: ``` schema_mapping: - name: MyModel mapping: "api/endpoint" fields: - name: identifier mapping: "raw_id" - name: formatted_name mapping: "name" transforms: - field: identifier expression: "{{ raw_id | my_custom_filter | format_identifier }}" - field: status expression: "{{ 'active' if enabled else 'inactive' }}" ``` #### Filter implementation guidelines[​](#filter-implementation-guidelines "Direct link to Filter implementation guidelines") 1. **Keep filters focused**: Each filter should do one specific transformation 2. **Handle edge cases**: Always provide fallback values for missing data 3. **Use class variables**: Store filter data as class variables for efficient access 4. **Document your filters**: Add Python documentation strings explaining what each filter does 5. **Test thoroughly**: Ensure filters work with various input types and edge cases #### Real-world example: ACI device name filter[​](#real-world-example-aci-device-name-filter "Direct link to Real-world example: ACI device name filter") Here's how the ACI adapter implements the `aci_device_name` filter: ``` class AciModel(DiffSyncModelMixin, DiffSyncModel): _device_mapping: ClassVar[dict[str, str]] = {} @classmethod def set_device_mapping(cls, device_mapping: dict[str, str]) -> None: cls._device_mapping = device_mapping @classmethod def _add_custom_filters(cls, native_env: Any, item: dict[str, Any]) -> None: def aci_device_name(node_id: str) -> str: """Resolve ACI node IDs to device names.""" return cls._device_mapping.get(str(node_id), node_id) native_env.filters["aci_device_name"] = aci_device_name ``` Used in configuration: ``` transforms: - field: device expression: "{{ l1PhysIf.attributes.dn.split('/')[2].replace('node-', '') | aci_device_name }}" ``` ### Best practices[​](#best-practices "Direct link to Best practices") 1. **Package Structure**: Organize complex adapters as packages with `__init__.py` 2. **Testing**: Include test data and documentation with your adapter 3. **Configuration**: Use settings to make your adapter configurable 4. **Error Handling**: Implement proper error handling and logging 5. **Type Annotations**: Use type hints to make your code more maintainable 6. **Custom Filters**: Implement adapter-specific Jinja filters for complex transformations ### Local adapter example[​](#local-adapter-example "Direct link to Local adapter example") Here's an example configuration using a local adapter: ``` name: custom-example source: name: mockdb # Filesystem path to the adapter class adapter: ./examples/custom_adapter/custom_adapter_src/custom_adapter.py:MockdbAdapter settings: db_path: "./examples/custom_adapter/custom_adapter_src/mock_db.json" destination: name: infrahub settings: url: "http://localhost:8000" order: [ "InfraDevice", ] schema_mapping: # Your schema mapping here ``` --- # Nautobot adapter ## What is Nautobot?[​](#what-is-nautobot "Direct link to What is Nautobot?") Nautobot is an open-source network source of truth and automation platform, offering extended customization, plugin support, and enhanced functionality to simplify network documentation and operations. ## Requirements[​](#requirements "Direct link to Requirements") This Adapter uses [Nautobot SDK](https://pypi.org/project/pynautobot). You will need to install it beforehand. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * Nautobot → Infrahub info Currently, the Nautobot adapter supports only **one-way synchronization** from Nautobot to Infrahub. Syncing data back into Nautobot is not yet supported. ## Schema[​](#schema "Direct link to Schema") Our `infrahub` repository contains an **example schema** that serves as a starting point for syncing Nautobot data into Infrahub. This schema follows best practices for Infrahub, **but it does not map the Nautobot data model one-to-one** since Infrahub may have additional use cases. To reflect the breaking changes introduced in Nautobot v2, there is not one but two schemas. You can explore them to see the difference. * Nautobot v1 * Nautobot v2 [Schema V1https://github.com/opsmill/infrahub/blob/stable/models/examples/nautobot/nautobot-v1.yml](https://github.com/opsmill/infrahub/blob/stable/models/examples/nautobot/nautobot-v1.yml) [Schema V2https://github.com/opsmill/infrahub/blob/stable/models/examples/nautobot/nautobot-v2.yml](https://github.com/opsmill/infrahub/blob/stable/models/examples/nautobot/nautobot-v2.yml) ### Installing the example schema[​](#installing-the-example-schema "Direct link to Installing the example schema") To install the example schema into Infrahub, follow these steps: * Nautobot v1 * Nautobot v2 ``` mkdir nautobot-sync cd nautobot-sync curl -o schema.yml https://raw.githubusercontent.com/opsmill/infrahub/refs/heads/stable/models/examples/nautobot/nautobot-v1.yml infrahubctl schema load schema.yml ``` ``` mkdir nautobot-sync cd nautobot-sync curl -o schema.yml https://raw.githubusercontent.com/opsmill/infrahub/refs/heads/stable/models/examples/nautobot/nautobot-v2.yml infrahubctl schema load schema.yml ``` ## Configuration[​](#configuration "Direct link to Configuration") `infrahub-sync` allows defining what gets synchronized from a source to a destination. Included in the examples is a config.yml file that matches the example schema. * Nautobot v1 * Nautobot v2 [config.ymlhttps://github.com/opsmill/infrahub-sync/blob/main/examples/nautobot-v1\_to\_infrahub/config.yml](https://github.com/opsmill/infrahub-sync/blob/main/examples/nautobot-v1_to_infrahub/config.yml) [config.ymlhttps://github.com/opsmill/infrahub-sync/blob/main/examples/nautobot-v2\_to\_infrahub/config.yml](https://github.com/opsmill/infrahub-sync/blob/main/examples/nautobot-v2_to_infrahub/config.yml) To download the example `config.yml` * Nautobot v1 * Nautobot v2 ``` curl https://raw.githubusercontent.com/opsmill/infrahub-sync/refs/heads/main/examples/nautobot-v1_to_infrahub/config.yml > config.yml ``` ``` curl https://raw.githubusercontent.com/opsmill/infrahub-sync/refs/heads/main/examples/nautobot-v2_to_infrahub/config.yml > config.yml ``` ### Configuration parameters[​](#configuration-parameters "Direct link to Configuration parameters") To instruct `infrahub-sync` to use the Nautobot adapter, set `nautobot` in the direction 'name', you want to use for Peering Manager. Below is a snippet from the example config.yml file: * Source example ``` --- name: from-nautobot source: name: nautobot settings: url: "https://" token: "" verify_ssl: true # Default value ``` ### Schema mapping[​](#schema-mapping "Direct link to Schema mapping") The configuration file allows mapping Nautobot data to the Infrahub schema, which has been designed to loosely align with the Nautobot data model. Below is an example showing how to: * Set the destination Infrahub model (`OrganizationGeneric` and `InfraPlatform`) * Map source data from Nautobot (`dcim.manufacturers`, and `dcim.platforms`) * Specify field mappings between Nautobot and Infrahub models As you can see in this example, we can map several paths to the same Infrahub Model. ``` - name: OrganizationGeneric mapping: dcim.manufacturers identifiers: ["name"] fields: - name: name mapping: name - name: description mapping: description - name: type static: "Manufacturer" - name: InfraPlatform mapping: dcim.platforms identifiers: ["name", "manufacturer"] fields: - name: name mapping: name - name: description mapping: description - name: napalm_driver mapping: napalm_driver - name: manufacturer mapping: manufacturer reference: OrganizationGeneric ``` The models available on Nautobot can be find in the `/api/schema/swagger-ui` of your instance. [Nautobot API Docshttps://demo.nautobot.com/api/docs](https://demo.nautobot.com/api/docs) ## Related guides[​](#related-guides "Direct link to Related guides") * [Use NetBox or Nautobot with Infrahub](/sync/using-netbox-or-nautobot-with-infrahub.md) — running Nautobot and Infrahub side by side * [Migrate from NetBox or Nautobot](/sync/migrating-from-netbox-or-nautobot.md) — gradual migration into Infrahub * [Schema mapping reference](/sync/reference/schema-mapping.md) — mapping syntax with Nautobot examples * [Choose an adapter](/sync/adapters/choosing-an-adapter.md) --- # NetBox adapter ## What is NetBox?[​](#what-is-netbox "Direct link to What is NetBox?") NetBox is an open-source infrastructure resource management tool that centralizes and documents your network's devices, IP addresses, circuits, and connections using pre-configured models and automation for streamlined operations. ## Requirements[​](#requirements "Direct link to Requirements") This Adapter uses [Netbox SDK](https://pypi.org/project/pynetbox). You will need to install it beforehand. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * NetBox → Infrahub info Currently, the Netbox adapter supports only **one-way synchronization** from Netbox to Infrahub. Syncing data back into Netbox is not yet supported. ## Schema[​](#schema "Direct link to Schema") Our `infrahub` repository contains an **example schema** that serves as a starting point for syncing Netbox data into Infrahub. This schema follows best practices for Infrahub, **but it does not map the Netbox data model one-to-one** since Infrahub may have additional use cases. [Schemahttps://github.com/opsmill/infrahub/blob/stable/models/examples/netbox/netbox.yml](https://github.com/opsmill/infrahub/blob/stable/models/examples/netbox/netbox.yml) ### Installing the example schema[​](#installing-the-example-schema "Direct link to Installing the example schema") To install the example schema into Infrahub, follow these steps: ``` mkdir netbox-sync cd netbox-sync curl -o schema.yml https://raw.githubusercontent.com/opsmill/infrahub/refs/heads/stable/models/examples/netbox/netbox.yml infrahubctl schema load schema.yml ``` ## Configuration[​](#configuration "Direct link to Configuration") `infrahub-sync` allows defining what gets synchronized from a source to a destination. Included in the examples is a config.yml file that matches the example schema. [config.ymlhttps://github.com/opsmill/infrahub-sync/blob/main/examples/netbox\_to\_infrahub/config.yml](https://github.com/opsmill/infrahub-sync/blob/main/examples/netbox_to_infrahub/config.yml) To download the example `config.yml` ``` curl https://raw.githubusercontent.com/opsmill/infrahub-sync/refs/heads/main/examples/netbox_to_infrahub/config.yml > config.yml ``` ### Configuration parameters[​](#configuration-parameters "Direct link to Configuration parameters") To instruct `infrahub-sync` to use the Netbox adapter, set `netbox` in the direction 'name', you want to use for Peering Manager. Below is a snippet from the example config.yml file: * Source example ``` --- name: from-netbox source: name: netbox settings: url: "https://" token: "" verify_ssl: true # Default value ``` ### Schema mapping[​](#schema-mapping "Direct link to Schema mapping") The configuration file allows mapping Netbox data to the Infrahub schema, which has been designed to loosely align with the Netbox data model. Below is an example showing how to: * Set the destination Infrahub model (`BuiltinTag` and `LocationGeneric`) * Map source data from Netbox (`extras.tags`, `dcim.regions`, and `dcim.sites`) * Specify field mappings between Netbox and Infrahub models As you can see in this example, we can map several paths to the same Infrahub Model. ``` - name: BuiltinTag mapping: extras.tags identifiers: ["name"] fields: - name: name mapping: name - name: description mapping: description - name: LocationGeneric mapping: dcim.regions fields: - name: name mapping: slug - name: description mapping: name - name: type static: "Region" - name: tags mapping: tags reference: BuiltinTag - name: LocationGeneric mapping: dcim.sites fields: - name: name mapping: slug - name: description mapping: name - name: type static: "Site" - name: tags mapping: tags reference: BuiltinTag ``` The models available on Netbox can be find in the `/api/schema/swagger-ui` of your instance. [Netbox API Swaggerhttps://demo.netbox.dev/api/schema/swagger-ui](https://demo.netbox.dev/api/schema/swagger-ui) ## Related guides[​](#related-guides "Direct link to Related guides") * [Use NetBox or Nautobot with Infrahub](/sync/using-netbox-or-nautobot-with-infrahub.md) — running NetBox and Infrahub side by side * [Migrate from NetBox or Nautobot](/sync/migrating-from-netbox-or-nautobot.md) — gradual migration into Infrahub * [Schema mapping reference](/sync/reference/schema-mapping.md) — mapping syntax with NetBox examples * [Choose an adapter](/sync/adapters/choosing-an-adapter.md) --- # Observium adapter ## What is Observium?[​](#what-is-observium "Direct link to What is Observium?") Observium is a network monitoring platform that delivers auto-discovery, detailed performance metrics, and rich visualizations to help you effectively manage and optimize your network infrastructure. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * Observium → Infrahub info Currently, the Observium adapter supports only **one-way synchronization** from Observium to Infrahub. Syncing data back into Observium is not yet supported. Under Construction --- # Peering Manager adapter ## What is Peering Manager?[​](#what-is-peering-manager "Direct link to What is Peering Manager?") Peering Manager lets you effortlessly manage and document your network’s peering sessions and interconnections with pre-configured data models and automation for seamless BGP session management. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * Peering Manager → Infrahub * Infrahub → Peering Manager ## Schema[​](#schema "Direct link to Schema") Our `infrahub` repository contains an **example schema** that serves as a starting point for syncing Peering Manager data into Infrahub. This schema follows best practices for Infrahub, **but it does not map the Peering Manager data model one-to-one** since Infrahub may have additional use cases. This example is using the "base" demo schema and use an extension for Peering Manager. [Base Schemahttps://github.com/opsmill/infrahub/blob/stable/models/base/](https://github.com/opsmill/infrahub/blob/stable/models/base/) [Peering Manager Extensionhttps://github.com/opsmill/infrahub/blob/stable/models/examples/extension\_peering\_manager.yml](https://github.com/opsmill/infrahub/blob/stable/models/examples/extension_peering_manager.yml) ### Installing the example schema[​](#installing-the-example-schema "Direct link to Installing the example schema") To install the example schema into Infrahub, follow these steps: ``` mkdir peeringmanager-sync cd peeringmanager-sync # Fetch the list of YAML file names from the peeringmanager folder curl -s "https://api.github.com/repos/opsmill/infrahub/contents/models/base?ref=stable" | jq -r '.[].name | select(endswith(".yml"))' | while read file; do curl -O "https://raw.githubusercontent.com/opsmill/infrahub/refs/heads/stable/models/base/$file" done curl -O "https://raw.githubusercontent.com/opsmill/infrahub/refs/heads/stable/models/examples/extension_peering_manager.yml" infrahubctl schema load *.yml ``` ## Configuration[​](#configuration "Direct link to Configuration") `infrahub-sync` allows defining what gets synchronized from a source to a destination. Included in the examples there is two `config.yml` files that matches the example schema. [Source config.ymlhttps://github.com/opsmill/infrahub-sync/blob/main/examples/peering-manager\_to\_infrahub/config.yml](https://github.com/opsmill/infrahub-sync/blob/main/examples/peering-manager_to_infrahub/config.yml) [Destination config.ymlhttps://github.com/opsmill/infrahub-sync/blob/main/examples/infrahub\_to\_peering-manager/config.yml](https://github.com/opsmill/infrahub-sync/blob/main/examples/infrahub_to_peering-manager/config.yml) To download those examples `config.yml` peering-manager to infrahub ``` curl https://raw.githubusercontent.com/opsmill/infrahub-sync/refs/heads/main/examples/peering-manager_to_infrahub/config.yml > config.yml ``` infrahub to peering-manager ``` curl https://raw.githubusercontent.com/opsmill/infrahub-sync/refs/heads/main/examples/infrahub_to_peering-manager/config.yml > config.yml ``` ### Configuration parameters[​](#configuration-parameters "Direct link to Configuration parameters") To instruct `infrahub-sync` to use the Peering Manager adapter, set `peeringmanager` in the direction 'name', you want to use for Peering Manager. The settings dictionary is passed directly to the REST API client for Peering Manager for authentication and connection details. Below is a snippet from the example config.yml file: * Source example * Destination example ``` --- name: from-peering-manager source: name: peeringmanager settings: url: "https://" api_endpoint: "api" # Default value auth_method: "token" # Default value # auth_method: "token" token: "" ``` ``` --- name: to-peering-manager destination: name: peeringmanager settings: url: "https://" api_endpoint: "api" # Default value auth_method: "token" # Default value # auth_method: "token" token: "" verify_ssl: true # Default value ``` info You can pass `params` for the REST API client, used by Peering Manager adapter, in the settings. For example `limit` ### Schema mapping[​](#schema-mapping "Direct link to Schema mapping") The configuration file allows mapping tables from PeeringManager into Infrahub models. Below is an example showing how to: * Set the destination Infrahub model (`InfraBGPCommunity`) * Map source data from Peering Manager (`peering/communities`) * Specify field mappings between Peering Manager and Infrahub models ``` - name: InfraBGPCommunity mapping: peering/communities identifiers: ["name"] fields: - name: name mapping: slug - name: label mapping: name - name: description mapping: description - name: value mapping: value - name: community_type mapping: type ``` The models available on Peering Manager can be find in the /api of your instance. [Peering Manager API Swaggerhttps://demo.peering-manager.net/api/schema/swagger-ui](https://demo.peering-manager.net/api/schema/swagger-ui) ![Peering Manager API Example](/assets/images/peering-manager-api-8c7c5220752c321dbe43bd8aac08f78f.png) --- # PeeringDB adapter ## What is PeeringDB?[​](#what-is-peeringdb "Direct link to What is PeeringDB?") [PeeringDB](https://www.peeringdb.com/) is a free, public, user-maintained database of network peering information. It catalogs autonomous systems, internet exchanges, points of presence, and facilities — the data networks use to find and arrange interconnection with one another. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * PeeringDB → Infrahub info The PeeringDB adapter supports only **one-way synchronization** from PeeringDB to Infrahub. PeeringDB is a community-maintained reference database; writing data back into it is not supported. Under Construction --- # Prometheus adapter ## What is Prometheus?[​](#what-is-prometheus "Direct link to What is Prometheus?") Prometheus is an open-source monitoring and alerting toolkit that collects and stores metrics as time series data. It is widely used for monitoring infrastructure, applications, and services through its pull-based model and powerful query language (PromQL). ## Requirements[​](#requirements "Direct link to Requirements") This adapter uses the [prometheus-client](https://pypi.org/project/prometheus-client) library for parsing the Prometheus exposition format. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * Prometheus → Infrahub note Currently, the Prometheus adapter supports only **one-way synchronization** from Prometheus to Infrahub. Writing data back to Prometheus is not supported as Prometheus is a read-only metrics store. ## Operating modes[​](#operating-modes "Direct link to Operating modes") The Prometheus adapter supports two modes of operation: ### Scrape mode (default)[​](#scrape-mode-default "Direct link to Scrape mode (default)") Directly scrapes a Prometheus text exposition endpoint (for example, from `node_exporter`, application metrics endpoints). Best for collecting metrics from individual exporters. ### API mode[​](#api-mode "Direct link to API mode") Queries the Prometheus HTTP API using PromQL. Best for aggregating data from multiple targets or performing complex queries with joins. ## Configuration[​](#configuration "Direct link to Configuration") ### Scrape mode configuration[​](#scrape-mode-configuration "Direct link to Scrape mode configuration") * Scrape Mode * API Mode ``` --- name: from-node-exporter source: name: prometheus settings: mode: scrape url: "http://localhost:9100" endpoint: "/metrics" timeout: 10 # Optional, seconds verify_ssl: true # Optional destination: name: infrahub settings: url: "http://localhost:8000" ``` ``` --- name: from-prometheus-api source: name: prometheus settings: mode: api url: "http://prometheus:9090" endpoint: "/api/v1/query" promql: resources: node_info: 'node_uname_info' node_memory: 'node_memory_MemTotal_bytes' node_cpu: 'count(node_cpu_seconds_total{mode="idle"}) by (instance)' destination: name: infrahub settings: url: "http://localhost:8000" ``` ### Environment variables[​](#environment-variables "Direct link to Environment variables") | Variable | Description | | ---------------------------- | ------------------------------------------------ | | `PROM_URL` or `PROM_ADDRESS` | Prometheus server URL (overrides `settings.url`) | | `PROM_TOKEN` | Bearer token for authentication | | `PROM_USERNAME` | Username for basic authentication | | `PROM_PASSWORD` | Password for basic authentication | ### Settings reference[​](#settings-reference "Direct link to Settings reference") | Setting | Type | Required | Default | Description | | ------------------ | ------- | -------- | -------------------------------------------- | ------------------------------------------------------ | | `mode` | string | No | `scrape` | Operating mode: `scrape` or `api` | | `url` | string | Yes | - | Prometheus server or exporter URL | | `endpoint` | string | No | `/metrics` (scrape) or `/api/v1/query` (API) | Target endpoint | | `timeout` | number | No | 10 | Request timeout in seconds | | `verify_ssl` | boolean | No | true | Verify SSL certificates | | `auth_method` | string | No | `none` | Authentication: `none`, `basic`, or `bearer` | | `username` | string | No | - | Username for basic auth | | `password` | string | No | - | Password for basic auth | | `token` | string | No | - | Token for bearer auth | | `headers` | object | No | - | Additional HTTP headers | | `params` | object | No | - | Query parameters for scrape mode | | `promql.resources` | object | No | - | PromQL queries for API mode (`resource_name`: `query`) | ## Schema mapping[​](#schema-mapping "Direct link to Schema mapping") The Prometheus adapter normalizes metrics into a consistent format for schema mapping: ``` # Each metric sample contains: # - __metric__: metric name # - labels: { label_key: label_value, ... } # - value: numeric value # - timestamp: optional timestamp # - help: metric help text # - type: metric type (counter, gauge, etc.) ``` ### Basic field mapping[​](#basic-field-mapping "Direct link to Basic field mapping") Map fields using dot notation to access metric properties: ``` schema_mapping: - name: VirtualizationVirtualMachine mapping: node_uname_info # Metric name to load from identifiers: ["name"] fields: - name: name mapping: labels.nodename # Access label value - name: os_name mapping: labels.sysname - name: os_kernel mapping: labels.release ``` ### Using the lookup function[​](#using-the-lookup-function "Direct link to Using the lookup function") The `lookup()` function allows you to join data from different metrics: ``` transforms: # Syntax: lookup(metric_name, match_criteria, value_path [, default]) # Get total memory from a different metric - field: mem_total_bytes expression: "{{ lookup('node_memory_MemTotal_bytes', {}, 'value') or 0 }}" # Match by label value - field: mac_address expression: "{{ lookup('node_network_info', {'device': labels.device}, 'labels.address') or '' }}" # With default value - field: status expression: "{{ lookup('node_time_seconds', {}, 'value', 0) }}" ``` ### Lookup function parameters[​](#lookup-function-parameters "Direct link to Lookup function parameters") | Parameter | Description | | ---------------- | ---------------------------------------------------------------------- | | `metric_name` | Name of the metric to look up | | `match_criteria` | Dict of label conditions to match (empty `{}` matches first record) | | `value_path` | Dot-notation path to extract (for example: `value`, `labels.hostname`) | | `default` | Optional default value if lookup fails | ## Example: Node Exporter to Infrahub[​](#example-node-exporter-to-infrahub "Direct link to Example: Node Exporter to Infrahub") [Node Exporter Examplehttps://github.com/opsmill/infrahub-sync/tree/main/examples/prometheus\_to\_infrahub%20(node\_exporter)](https://github.com/opsmill/infrahub-sync/tree/main/examples/prometheus_to_infrahub%20\(node_exporter\)) This example syncs `node_exporter` metrics to create virtual machine records in Infrahub: ``` schema_mapping: - name: VirtualizationVirtualMachine mapping: node_uname_info identifiers: ["name"] fields: - name: name mapping: labels.nodename - name: os_name mapping: labels.sysname - name: mem_total_bytes mapping: mem_total_t - name: status mapping: status_t transforms: # Join memory info from separate metric - field: mem_total_t expression: "{{ lookup('node_memory_MemTotal_bytes', {}, 'value') or 0 }}" # Derive status from exporter availability - field: status_t expression: "{{ 'active' if ((lookup('node_time_seconds', {}, 'value') | float) > 0) else 'unknown' }}" - name: VirtualizationVMNetworkInterface mapping: node_network_mtu_bytes identifiers: ["virtual_machine", "name"] fields: - name: virtual_machine mapping: vm_name - name: name mapping: labels.device - name: mtu mapping: value transforms: # Link to parent VM - field: vm_name expression: "{{ lookup('node_uname_info', {}, 'labels.nodename') or 'unknown' }}" ``` ## Generating the models[​](#generating-the-models "Direct link to Generating the models") Use the generate command to produce Python models from your configuration: ``` uv run infrahub-sync generate --name from-node-exporter --directory examples/ ``` ## Common issues and troubleshooting[​](#common-issues-and-troubleshooting "Direct link to Common issues and troubleshooting") ### Connection errors[​](#connection-errors "Direct link to Connection errors") * Verify the exporter or Prometheus server is running and accessible * Check firewall rules allow connections to the metrics endpoint * For scrape mode, ensure the endpoint returns Prometheus text format ### Empty results[​](#empty-results "Direct link to Empty results") * In scrape mode, verify the `/metrics` endpoint returns data * In API mode, test your PromQL queries directly in Prometheus UI first * Check that `schema_mapping.mapping` matches actual metric names ### Lookup failures[​](#lookup-failures "Direct link to Lookup failures") * Ensure the metric being looked up exists in the scraped data * Verify label names match exactly (case-sensitive) * Use empty dict `{}` for `match_criteria` to get the first matching record * Provide default values for optional lookup operations ### Authentication issues[​](#authentication-issues "Direct link to Authentication issues") * For basic auth, set both `username` and `password` * For bearer auth, set `auth_method: bearer` and provide `token` * Use environment variables (`PROM_USERNAME`, `PROM_PASSWORD`, `PROM_TOKEN`) for credentials ### API mode specific[​](#api-mode-specific "Direct link to API mode specific") * Ensure `promql.resources` is a dict mapping resource names to PromQL queries * Verify queries return vector or scalar results (other types not supported) * Test queries in Prometheus UI before using in configuration --- # Slurp’it adapter ## What is Slurp’it?[​](#what-is-slurpit "Direct link to What is Slurp’it?") Slurp’it lets you easily retrieve and store any data you want from your network in an offline, structured database. Out of the box, all major vendors and templates are already pre-configured. Run the installation wizard and Slurp’it will start mining your network. You can add as many templates as you want. ## Requirements[​](#requirements "Direct link to Requirements") This Adapter uses [Slurp’it SDK](https://pypi.org/project/slurpit_sk). You will need to install it beforehand. ## Sync directions supported[​](#sync-directions-supported "Direct link to Sync directions supported") * Slurp’it → Infrahub info Currently, the Slurp’it adapter supports only **one-way synchronization** from Slurp’it to Infrahub. Syncing data back into Slurp’it is not yet supported. ## Schema[​](#schema "Direct link to Schema") Our `infrahub` repository contains an **example schema** that serves as a starting point for syncing Slurp’it data into Infrahub. This schema follows best practices for Infrahub, **but it does not map the Slurp’it data model one-to-one** since Infrahub may have additional use cases and Slurp’it can be customizable. [Schemahttps://github.com/opsmill/infrahub/blob/stable/models/examples/slurpit/slurpit.yml](https://github.com/opsmill/infrahub/blob/stable/models/examples/slurpit/slurpit.yml) ### Installing the example schema[​](#installing-the-example-schema "Direct link to Installing the example schema") To install the example schema into Infrahub, follow these steps: ``` mkdir slurpit-sync cd slurpit-sync curl -o schema.yml https://raw.githubusercontent.com/opsmill/infrahub/refs/heads/stable/models/examples/slurpit/slurpit.yml infrahubctl schema load schema.yml ``` ## Configuration[​](#configuration "Direct link to Configuration") `infrahub-sync` allows defining what gets synchronized from a source to a destination. Included in the examples is a `config.yml` file that matches the example schema. [config.ymlhttps://github.com/opsmill/infrahub-sync/blob/main/examples/slurpit\_to\_infrahub/config.yml](https://github.com/opsmill/infrahub-sync/blob/main/examples/slurpit_to_infrahub/config.yml) To download the example `config.yml` ``` curl https://raw.githubusercontent.com/opsmill/infrahub-sync/refs/heads/main/examples/slurpit_to_infrahub/config.yml > config.yml ``` ### Configuration parameters[​](#configuration-parameters "Direct link to Configuration parameters") The `source.name` is set to `slurpitsync`, instructing `infrahub-sync` to use the Slurp’it adapter. The settings dictionary is passed directly to the Slurp’it Python SDK for authentication and connection details. Some of these parameters may be found in the [Slurp’it SDK Repository](https://gitlab.com/slurpit.io/slurpit_sdk). Below is a snippet from the example config.yml file: ``` --- name: from-slurpit source: name: slurpitsync settings: url: "" api_key: "" verify_ssl: true # Default value ``` ### Schema mapping[​](#schema-mapping "Direct link to Schema mapping") The configuration file allows mapping Slurp’it data to the Infrahub schema, which has been designed to loosely align with the Slurp’it data model. Below is an example showing how to: * Set the destination Infrahub model (`OrganizationGeneric`) * Map source data from Slurp’it and run a custom function `unique_vendors` * Specify field mappings between Slurp’it and Infrahub models ``` schema_mapping: - name: OrganizationGeneric mapping: unique_vendors fields: - name: name mapping: brand - name: type static: "Vendor" ``` Schema mapping connects Slurp’it fields to Infrahub's Node attributes and relationships. Each field can either map directly to a Slurp’it field or be statically defined. This table highlights the available `mappings` the adapter has to gather and normalize data from Slurp’it | Name | Description | | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `unique_vendors` | The `unique_vendors` function retrieves a list of devices asynchronously, extracts the unique brands (vendors) from the device list, and returns a list of dictionaries, each containing a distinct brand. This ensures that only unique vendor names are included in the output. | | `unique_device_type` | The `unique_device_type` function retrieves a list of devices asynchronously, then extracts unique combinations of brand, `device_type`, and `device_os`. It returns a list of dictionaries where each dictionary contains these three distinct attributes, ensuring only unique device types are included in the output. | | `model`.`function` | This mapping directly relates to the SDK. For example once the SDK client is initiated you can get devices by using `device.get_devices` this will execute the query to gather all the devices. | | `planning_results`.`` | If the mapping starts with `planning_result` a call will be made to the Slurp’it planning API to gather the data for the data that matches the slug. For example: `planning_results.routing-table` will query the routing-table plannings for all the data it has and return it to be used in the mapping. | | `filter_networks` | The `filter_networks` function processes a list of network entries, normalizing the network and mask fields, and filters out unwanted networks based on predefined "ignore" prefixes (such as 0.0.0.0/0 and 127.0.0.0/8). It returns a list of filtered, normalized networks, excluding those with prefixes deemed unnecessary or invalid. | | `filter_interfaces` | The `filter_interfaces` function normalizes IP addresses in a list of interfaces and matches them against precomputed network prefixes from `filtered_networks`. It uses asynchronous tasks to process entries concurrently. Each entry's IP is normalized, and if it matches a known network, additional data (like VRF) is added. The function returns a list of interfaces with normalized addresses, excluding invalid or unmatched entries. | --- # Contributing This guide covers how to set up a development environment for `infrahub-sync` and contribute to the project. For the release runbook, see [RELEASING.md](https://github.com/opsmill/infrahub-sync/blob/main/RELEASING.md) at the repository root — that's maintainer-only. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Python 3.10–3.13 (3.12 recommended) * [uv](https://docs.astral.sh/uv/) for dependency management * Git ## Setting up your development environment[​](#setting-up-your-development-environment "Direct link to Setting up your development environment") ### Clone the repository[​](#clone-the-repository "Direct link to Clone the repository") ``` git clone https://github.com/opsmill/infrahub-sync.git cd infrahub-sync ``` ### Install uv[​](#install-uv "Direct link to Install uv") If you don't have uv installed, you can install it with: ``` curl -LsSf https://astral.sh/uv/install.sh | sh ``` Or see the [uv installation guide](https://docs.astral.sh/uv/getting-started/installation/) for other options. ### Install dependencies[​](#install-dependencies "Direct link to Install dependencies") ``` uv sync --group dev ``` This installs all runtime and development dependencies defined in `pyproject.toml`. ### Verify your setup[​](#verify-your-setup "Direct link to Verify your setup") ``` uv run infrahub-sync --help uv run infrahub-sync list --directory examples/ ``` ### Install the Git hooks[​](#install-the-git-hooks "Direct link to Install the Git hooks") `prek.toml` defines the commit hooks: Ruff formatting and lint for Python, `rumdl` for Markdown and MDX, and checks for whitespace, YAML, TOML, large files, and private keys. Install them with: ``` uv run --frozen --extra dev prek install --force ``` Run the same command in an existing checkout. This project used pre-commit before, and `uv sync` removes that package, so the `.git/hooks/pre-commit` file it generated stops working and blocks every commit. `--force` replaces that file. It also overwrites any other script at that path, so copy your own hook elsewhere first if you keep one there. ## Development workflow[​](#development-workflow "Direct link to Development workflow") Before committing any changes, run the following commands in order: ``` uv run invoke format # Format code with ruff uv run invoke lint # Lint code with ruff and pylint uv run mypy infrahub_sync/ --ignore-missing-imports ``` ### Validate the CLI[​](#validate-the-cli "Direct link to Validate the CLI") After making changes, verify the CLI still works: ``` uv run infrahub-sync --help uv run infrahub-sync list --directory examples/ uv run infrahub-sync generate --name from-netbox --directory examples/ ``` ### Running tests[​](#running-tests "Direct link to Running tests") ``` uv run pytest -q ``` ## Code standards[​](#code-standards "Direct link to Code standards") ### Python style[​](#python-style "Direct link to Python style") * Python 3.10–3.13 compatible * Type hints on new or changed code * Ruff-formatted and lint-clean * Mypy-checked (do not increase existing error count) * Public functions and classes require documentation strings * Raise specific exceptions; avoid broad `except Exception:` ### Line length[​](#line-length "Direct link to Line length") * Maximum line length: 120 characters (configured in `pyproject.toml`) ## Documentation[​](#documentation "Direct link to Documentation") If you make user-facing changes (CLI flags, configuration options, new adapters), update the documentation. ### Generate command-line documentation[​](#generate-command-line-documentation "Direct link to Generate command-line documentation") ``` uv run invoke docs.generate ``` ### Build documentation site[​](#build-documentation-site "Direct link to Build documentation site") First-time setup (requires Node.js): ``` cd docs && npm install ``` Build the site: ``` uv run invoke docs.docusaurus ``` ### Lint markdown files[​](#lint-markdown-files "Direct link to Lint markdown files") ``` npx markdownlint-cli "docs/docs/**/*.{md,mdx}" npx markdownlint-cli --fix "docs/docs/**/*.{md,mdx}" ``` ## Changelog entries[​](#changelog-entries "Direct link to Changelog entries") Release notes are written by contributors rather than generated from pull request titles, so every pull request into `main` must add a news fragment under `changelog/`. CI fails the pull request if it does not. Create one with towncrier, naming it after the issue or pull request number: ``` uv run --extra dev towncrier create -c "Short description of what changed." 123.fixed.md ``` The file must be a direct child of `changelog/` named `..md`, where the type is one of `security`, `removed`, `deprecated`, `added`, `changed`, `fixed`, or `housekeeping`. Use `+` as the identifier when the change has no issue number, for example `+short-slug.housekeeping.md`. Nested paths and unknown types are ignored by towncrier, so the check rejects them rather than let your entry disappear at release time. If a change genuinely needs no entry — a dependency bump or a typo fix — a maintainer can label the pull request `ci/skip-changelog`. Do not edit `CHANGELOG.md` or the version in `pyproject.toml` by hand. Both are generated when a release is prepared; see [RELEASING.md](https://github.com/opsmill/infrahub-sync/blob/main/RELEASING.md). ## Adding a new adapter[​](#adding-a-new-adapter "Direct link to Adding a new adapter") 1. Create `infrahub_sync/adapters/.py` following existing adapter patterns 2. Add connection configuration schema and an example under `examples/` 3. Provide `list` and `diff` pathways before enabling `sync` 4. Document required environment variables and expected error cases 5. Create a documentation page in `docs/docs/adapters/` 6. Add the adapter to the sidebar in `docs/sidebars.ts` ## Invoke tasks[​](#invoke-tasks "Direct link to Invoke tasks") View all available tasks: ``` uv run invoke --list ``` Common tasks: | Task | Description | | -------------------- | ----------------------------- | | `linter.format-ruff` | Format Python code with ruff | | `linter.lint-ruff` | Lint Python code with ruff | | `linter.lint-pylint` | Lint Python code with pylint | | `linter.lint-yaml` | Lint YAML files with yamllint | | `docs.generate` | Generate CLI documentation | | `docs.docusaurus` | Build documentation website | | `format` | Alias for ruff format | | `lint` | Run all linters | --- # Create a sync project A sync project is a directory containing a `config.yml` file that defines one synchronization between two systems. Configure four parts: source and destination, sync order, schema mapping, and sync behavior. A Nautobot → Infrahub example runs through each. ## Define the source and destination[​](#define-the-source-and-destination "Direct link to Define the source and destination") The `source` and `destination` keys identify which adapter to use on each side of the sync and how to connect to each system. Credentials reference environment variables rather than being embedded in the file. ``` --- name: example-sync-task source: name: nautobot settings: url: "https://nautobot.example.com" token: "NAUTOBOT_TOKEN" # This can also be loaded from environment variables destination: name: infrahub settings: url: "https://infrahub.example.com" token: "INFRAHUB_API_TOKEN" # This can also be loaded from environment variables ``` For the full list of adapters and their connection parameters, see [Choose an adapter](/sync/adapters/choosing-an-adapter.md). ## Set the sync order[​](#set-the-sync-order "Direct link to Set the sync order") The `order` key specifies the sequence in which objects should be synchronized. Order matters because some objects depend on others — a device cannot be created until its location, role, and platform already exist in the destination. ``` order: - "InfraDevice" - "InfraInterface" ``` ## Map the schema fields[​](#map-the-schema-fields "Direct link to Map the schema fields") The `schema_mapping` section defines how data is translated from the source's schema into the destination's schema. info * The `name` key in the destination model corresponds to the Infrahub attribute. * The `mapping` key corresponds to the key in the source payload to use. * If `reference` is used, it links to a model that has been synchronized prior to this model. ``` schema_mapping: - name: InfraDevice mapping: "dcim.devices" identifiers: ["name"] fields: - name: "name" mapping: "name" - name: "device_type" mapping: "device_type.display_name" - name: "manufacturer" mapping: "device_type.manufacturer.name" - name: InfraInterface mapping: "dcim.interfaces" identifiers: ["device", "name"] fields: - name: "name" mapping: "name" - name: "interface_type" static: "10gbe" - name: "description" mapping: "description" - name: "device" reference: "InfraDevice" ``` In this example, `device_type` and `manufacturer` are attributes of `InfraDevice`. For destination objects that have relationships to other models, the related models must be synchronized first — see how `InfraInterface` references `InfraDevice` via the `reference` key. For the full mapping syntax — direct mappings, nested attributes, static values, references, identifiers, filters, transforms, and worked examples — see [Schema mapping reference](/sync/reference/schema-mapping.md). ## Tune sync behavior[​](#tune-sync-behavior "Direct link to Tune sync behavior") The `diffsync_flags` key controls how the synchronization handles three scenarios: unmatched objects in the destination, unmatched objects in the source, and modified objects. ``` # Optional: control sync behavior with diffsync flags diffsync_flags: - "SKIP_UNMATCHED_DST" # Skip objects in destination that don't exist in source ``` Understanding diffsync flags Available flags: | Flag | Description | | -------------------- | ------------------------------------------------------------------------------------ | | `SKIP_UNMATCHED_DST` | Skip objects in the destination that don't exist in the source (prevents deletion) | | `SKIP_UNMATCHED_SRC` | Skip objects in the source that don't exist in the destination (prevents creation) | | `SKIP_MODIFIED` | Skip objects that exist in both systems but have different values (prevents updates) | If no flags are specified, `SKIP_UNMATCHED_DST` is used by default — destination objects that don't exist in the source are preserved rather than deleted. For more on customizing sync configuration and troubleshooting, see [Sync instance configuration](/sync/reference/config.md). When the project is configured, run the sync — see [Run a sync](/sync/running-a-sync.md). --- # Use custom CA certificates For sync sources or destinations that use TLS certificates signed by an internal CA — common in enterprise networks — install the CA in the system trust store, then point Python's HTTP client at that store. ## Add the certificate to the system trust store[​](#add-the-certificate-to-the-system-trust-store "Direct link to Add the certificate to the system trust store") Create a subdirectory for the custom CA and copy the root certificate into it: ``` sudo mkdir /usr/local/share/ca-certificates/custom-ca sudo cp customRCA.crt /usr/local/share/ca-certificates/custom-ca/ ``` Then load the certificates into the trusted root store: ``` sudo update-ca-certificates ``` The exact paths above are for Debian/Ubuntu systems. On other distributions the trust store lives elsewhere (`/etc/pki/ca-trust/source/anchors/` on RHEL/Fedora, for example) — adapt accordingly. ## Point Python's HTTP client at the trust store[​](#point-pythons-http-client-at-the-trust-store "Direct link to Point Python's HTTP client at the trust store") Infrahub Sync uses `requests` under the hood, which reads the `REQUESTS_CA_BUNDLE` environment variable to find a custom certificate bundle: ``` export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt ``` Set this in the same shell environment where `infrahub-sync` runs — either exported in the user's shell profile, set in the scheduler that runs the sync, or passed in the container environment if running containerized. --- # Install Infrahub Sync Infrahub Sync is available on [PyPI](https://pypi.org/project/infrahub-sync/) and can be installed using the pip package installer. It is recommended to install the Sync into a virtual environment. ``` python3 -m venv .venv source .venv/bin/activate pip install infrahub-sync ``` --- # Migrate from NetBox or Nautobot Migrate data from NetBox or Nautobot into Infrahub one model at a time. The existing system keeps running throughout, and you decide when — or whether — to retire it. Before you start, install Infrahub Sync if you haven't already — see [Install Infrahub Sync](/sync/installation.md). ## The gradual migration approach[​](#the-gradual-migration-approach "Direct link to The gradual migration approach") A one-shot cutover from NetBox or Nautobot to Infrahub carries real risk: the new system has to be completely ready, all dependent automation has to switch at the same time, and rollback is hard if something surfaces afterward. The gradual approach removes that risk by syncing data into Infrahub model by model, verifying each step, and migrating dependent workflows on your own schedule. The phases: 1. **Prepare the Infrahub schema.** Define the schema in Infrahub that will hold the migrated data. 2. **Build the sync project.** Create a YAML configuration that maps NetBox or Nautobot models and fields to the corresponding Infrahub models and fields. 3. **Seed Infrahub with initial data.** Run the sync to populate Infrahub with the current state of the source system. 4. **Run in side-by-side mode.** Keep both systems active. Sync runs on a recurring schedule to keep Infrahub current with source-system changes. 5. **Migrate workflows.** Update automation, scripts, and dashboards to read from Infrahub instead of the source system. 6. **Retire the legacy system (optional).** Once all dependent workflows read from Infrahub and the legacy system is no longer the source of truth for any data, decommission it — or keep it running for the workflows where it remains the better tool. For long-running side-by-side operation (where the legacy system is not being retired), see [Use NetBox or Nautobot with Infrahub](/sync/using-netbox-or-nautobot-with-infrahub.md). The phases below focus on migration specifically. ## Phase 1 — Prepare the Infrahub schema[​](#phase-1--prepare-the-infrahub-schema "Direct link to Phase 1 — Prepare the Infrahub schema") Infrahub uses a custom schema that you define. Before migrating data, decide on the schema Infrahub will use for the data coming over from NetBox or Nautobot. The Infrahub schema does not have to mirror the source system one-to-one. Migration is an opportunity to model the data the way you need going forward, not the way the previous tool happened to model it. For a concrete starting point, see the example sync projects at `examples/netbox_to_infrahub/` and `examples/nautobot_to_infrahub/` in the [Infrahub Sync repository](https://github.com/opsmill/infrahub-sync). Each references an Infrahub demo schema you can use as a reference when defining your own. Replace its model and field names with the equivalents in your Infrahub schema. ## Phase 2 — Build the sync project[​](#phase-2--build-the-sync-project "Direct link to Phase 2 — Build the sync project") A sync project is a directory containing a `config.yml` that defines the source (NetBox or Nautobot), the destination (Infrahub), and the per-model schema mapping. ### Start from the example[​](#start-from-the-example "Direct link to Start from the example") The Infrahub Sync repository includes working examples at `examples/netbox_to_infrahub/` and `examples/nautobot_to_infrahub/`. Copy the directory matching your source system as the starting point. The examples use the Infrahub demo schema — update model and field names to match the schema you chose in Phase 1. ### Define sync order[​](#define-sync-order "Direct link to Define sync order") Models must be synced in dependency order: anything referenced by another model must be synced first. A typical order: 1. Independent objects: tags, locations, manufacturers, device roles, platforms 2. Rack and site data (depends on locations) 3. Devices (depends on locations, roles, manufacturers, platforms) 4. Interfaces (depends on devices) 5. IP addresses and prefixes (depends on interfaces, VRFs) 6. VLANs and VLAN groups The exact order depends on your schema. See [Sync instance configuration](/sync/reference/config.md) for the syntax of the `order` key. ### Map fields between the source system and Infrahub[​](#map-fields-between-the-source-system-and-infrahub "Direct link to Map fields between the source system and Infrahub") For each model, define how source fields map to Infrahub fields. The schema mapping section of the YAML handles direct field mappings, identifier mappings, references to other models, and static values. For detail on each mapping pattern, see [Schema mapping reference](/sync/reference/schema-mapping.md). ## Phase 3 — Seed Infrahub with initial data[​](#phase-3--seed-infrahub-with-initial-data "Direct link to Phase 3 — Seed Infrahub with initial data") Run `infrahub-sync diff` to verify what the sync will do before applying changes: ``` infrahub-sync diff --name --directory ``` The diff prints the proposed changes to the terminal without modifying Infrahub. Review it model by model: * Are the right models being created in Infrahub? * Are field values being mapped correctly? * Are relationships between models being resolved correctly? * Are there any unexpected creates, updates, or deletes? When the diff looks correct, run the sync: ``` infrahub-sync sync --name --directory ``` The sync applies changes in the order defined by the project's `order` key. Independent models are created first, then dependent models, then models that reference earlier ones. ### Migrate one model at a time[​](#migrate-one-model-at-a-time "Direct link to Migrate one model at a time") For the first sync, enable only the most independent models (tags, locations). Verify the data in Infrahub, then add the next layer of models (racks, devices), run again, and so on. This keeps the diff output reviewable and isolates any mapping issues to the model that caused them. ## Phase 4 — Run in side-by-side mode[​](#phase-4--run-in-side-by-side-mode "Direct link to Phase 4 — Run in side-by-side mode") After the initial seed, schedule the sync to run on a recurring cadence so Infrahub stays current with changes in the source system. The cadence depends on how often the source data changes — hourly, daily, or on-demand are all reasonable. Infrahub Sync has no built-in scheduler. Use cron, CI, Prefect, or another workflow tool — see [Schedule sync runs](/sync/orchestration.md). ### Default behavior preserves Infrahub-only data[​](#default-behavior-preserves-infrahub-only-data "Direct link to Default behavior preserves Infrahub-only data") By default, the `SKIP_UNMATCHED_DST` flag is enabled — Infrahub objects without a corresponding object in the source system are left alone. This is the safe default. Add Infrahub-only data (new schemas, design objects, intent data) without worrying that a sync run will delete it. To change this behavior — for example, removing Infrahub objects when they're deleted in the source system — adjust the `diffsync_flags` in the sync configuration. See [Sync instance configuration](/sync/reference/config.md). ### Decide on conflict policy early[​](#decide-on-conflict-policy-early "Direct link to Decide on conflict policy early") During side-by-side operation, the same model may be edited in both tools. Decide upfront which tool is authoritative for each model and document the decision in the project configuration. If both tools allow edits to the same model, conflicts will eventually occur — and Infrahub Sync's default behavior is for the source system to win. ## Phase 5 — Migrate workflows[​](#phase-5--migrate-workflows "Direct link to Phase 5 — Migrate workflows") As Infrahub becomes a reliable mirror of the source system, update dependent automation to read from Infrahub instead. This is the longest phase of a migration and typically happens workflow by workflow: * Identify each script, pipeline, or dashboard that reads from the source system. * Update it to read from Infrahub (using the Python SDK, infrahubctl, or GraphQL). * Verify the workflow behaves correctly with Infrahub as the source. * Once you're confident, retire the source-reading version of the workflow. For each migrated workflow, you can also stop writing to the source system for the affected data — at which point that data type is fully migrated. ## Phase 6 — Retire the legacy system (optional)[​](#phase-6--retire-the-legacy-system-optional "Direct link to Phase 6 — Retire the legacy system (optional)") Once all dependent workflows read from Infrahub and the legacy system is no longer the source of truth for any data: * **Decommission the legacy system entirely** if it's no longer needed. * **Keep the legacy system running for specific workflows** (rack elevation, cable tracing, or any other capability you prefer there). Reconfigure the sync to keep the legacy system in sync with Infrahub for those workflows, or stop syncing entirely if the data does not need to flow. Many teams reach a stable side-by-side state and stay there indefinitely. See [Use NetBox or Nautobot with Infrahub](/sync/using-netbox-or-nautobot-with-infrahub.md). ## Differences between migrating from NetBox versus Nautobot[​](#differences-between-migrating-from-netbox-versus-nautobot "Direct link to Differences between migrating from NetBox versus Nautobot") The same phases apply for both source systems. The mechanical differences: * Use the [NetBox adapter](/sync/adapters/netbox.md) for NetBox sources and the [Nautobot adapter](/sync/adapters/nautobot.md) for Nautobot sources. * Start from `examples/netbox_to_infrahub/` for NetBox, `examples/nautobot_to_infrahub/` for Nautobot. * Field names and nested structures differ between the two systems — for example, Nautobot status fields are objects while NetBox status fields are strings. The schema mapping work is source-specific, but the overall approach is identical. ## Common issues[​](#common-issues "Direct link to Common issues") * **Sync runs slowly on large datasets.** Initial syncs of large source-system installations can take a while. Sync performance is an active area of development. Recurring sync runs are typically fast — only deltas are applied. * **A model is being created in Infrahub that should not be.** Check the sync project's `order` key and per-model filters — the model may be enabled when it shouldn't be, or a reference from another model may be pulling it in. * **Field values are not mapping correctly.** Check the schema mapping section of the YAML. Common causes: a source field has a different name than expected, or a relationship reference is not resolving because the referenced model was not synced first. * **Custom fields in the source system are not appearing in Infrahub.** NetBox and Nautobot custom fields require explicit mapping in the YAML — they are not synced automatically. ## Related[​](#related "Direct link to Related") * [Use NetBox or Nautobot with Infrahub](/sync/using-netbox-or-nautobot-with-infrahub.md) * [Create a sync project](/sync/creating-a-sync-project.md) * [Run a sync](/sync/running-a-sync.md) * [Schema mapping reference](/sync/reference/schema-mapping.md) * [Schedule sync runs](/sync/orchestration.md) * [NetBox adapter](/sync/adapters/netbox.md) * [Nautobot adapter](/sync/adapters/nautobot.md) --- # Schedule sync runs Infrahub Sync runs as a single CLI command. Schedule it with whatever tooling already runs scheduled jobs in your environment — cron, CI, Prefect, Dagster, or a homegrown runner. ## Common orchestration options[​](#common-orchestration-options "Direct link to Common orchestration options") Any tool that can run a CLI command on a schedule and capture its output works. Pick whichever fits your existing operational model. ### Cron[​](#cron "Direct link to Cron") The most direct option. A cron entry runs `infrahub-sync sync` on a defined schedule. Output and exit codes are captured by your normal cron logging. **When this works well:** * You already operate a cron host for scheduled jobs. * The sync is straightforward — one project, one schedule, no dependencies on other jobs. * Failure handling can be reactive (logs are reviewed when something looks off). **Trade-offs:** No native retry on failure, no visibility beyond logs, no built-in alerting. For mission-critical syncs, wrap the cron entry in a script that handles retries and alerts. ### CI jobs (GitHub Actions, GitLab CI, etc.)[​](#ci-jobs-github-actions-gitlab-ci-etc "Direct link to CI jobs (GitHub Actions, GitLab CI, etc.)") Define the sync as a scheduled workflow in your CI system. CI provides run history, log retention, secret management, and notification on failure. **When this works well:** * You treat infrastructure data movement as part of your software delivery pipeline. * You want run history and structured failure notifications from tooling you already operate. * Sync project configurations live in a git repository alongside other infrastructure code. **Trade-offs:** CI systems are designed for short-lived jobs. For very large or long-running syncs, watch out for job timeout limits. ### Prefect[​](#prefect "Direct link to Prefect") Prefect is a Python-based workflow orchestrator. Wrap a sync run in a Prefect flow to get retry policies, dependency management, observability through the Prefect UI, and the ability to compose syncs with other Python tasks. **When this works well:** * You already use Prefect for other infrastructure or data workflows. * Syncs need to be composed with other tasks — for example, run a sync, then trigger downstream automation. * You want centralized observability across many sync projects. **Trade-offs:** Prefect itself has to be operated. If you don't already run it, this adds operational surface area. ### Dagster, Airflow, and other workflow engines[​](#dagster-airflow-and-other-workflow-engines "Direct link to Dagster, Airflow, and other workflow engines") Any workflow engine that can run a CLI command works the same way as Prefect. Pick based on what you already use and what other workloads share the orchestration platform. ### Event-driven execution[​](#event-driven-execution "Direct link to Event-driven execution") Some sync use cases are better triggered by an event than by a schedule. For example: "when a device is created in Infrahub, sync its details from the inventory system." For event-driven patterns, Infrahub's trigger event system can call `infrahub-sync` directly instead of running on a fixed cadence. Event-driven sync is the exception, not the rule. Most data movement is well-served by a recurring schedule. Use event-driven patterns when the data volume is small, the freshness requirement is high, or the source system is itself event-driven. ## Choosing a cadence[​](#choosing-a-cadence "Direct link to Choosing a cadence") Match the cadence to the rate at which source data changes and your tolerance for staleness: * **Every few minutes** — for high-change-rate sources that need near-real-time freshness. Verify the sync run completes faster than the cadence to avoid overlap. * **Hourly** — a common default for active infrastructure data. * **Daily** — appropriate for slower-moving data or for syncs where the next run can absorb a missed one. * **On demand** — for migrations, audits, or one-off seeding rather than steady-state operation. Each sync run calculates a fresh diff and applies only deltas, so running more often does not multiply work. The constraint is the time the sync takes, not the work it does. ## Observability and failure handling[​](#observability-and-failure-handling "Direct link to Observability and failure handling") Whatever scheduling tool you use, plan for these operational concerns: * **Logging.** Infrahub Sync emits structured logs via `structlog`. Pipe the output to your log aggregator (Splunk, Datadog, Loki, ELK, etc.). * **Failure detection.** The CLI returns a non-zero exit code on failure. Wire the scheduling tool to catch this and alert the appropriate channel. * **Idempotency.** Sync runs are idempotent. If a run fails partway through, re-running it calculates a fresh diff against the current destination state and applies what is still outstanding. Retries on failure are safe. * **Run isolation.** Avoid overlapping runs of the same sync project. Set the cadence longer than the run time, or have the scheduler skip a run if the previous one is still active. * **Sync project versioning.** Store sync project directories in version control. Tag or release configuration changes the same way you handle infrastructure code. ## Running Infrahub Sync in a container[​](#running-infrahub-sync-in-a-container "Direct link to Running Infrahub Sync in a container") To run Infrahub Sync from any orchestrator while keeping a single Python environment under your control, package it as a container. Build the image with `infrahub-sync` installed and the sync project directory included, set the sync command as the entrypoint, and pass credentials via environment variables. Infrahub Sync does not ship a reference container image — examples in the [OpsMill GitHub repository](https://github.com/opsmill/infrahub-sync) show common patterns to adapt. ## What's not in Infrahub Sync (and what to use instead)[​](#whats-not-in-infrahub-sync-and-what-to-use-instead "Direct link to What's not in Infrahub Sync (and what to use instead)") * **Built-in scheduler.** Use one of the options above. * **Sync history dashboard.** The scheduling tool typically provides this — CI run history, Prefect UI, or aggregated logs. * **Built-in retry policies.** Configure retries in the scheduling tool. Runs are idempotent, so retrying is safe. * **Alerting on failure.** Configure alerts in the scheduling tool or log aggregator. ## Related[​](#related "Direct link to Related") * [Run a sync](/sync/running-a-sync.md) * [Create a sync project](/sync/creating-a-sync-project.md) * [CLI reference](/sync/reference/cli.md) * [Migrate from NetBox or Nautobot](/sync/migrating-from-netbox-or-nautobot.md) --- # Cache layout `infrahub-sync diff` and `infrahub-sync apply` persist run state under: ``` .infrahub-sync-cache// ├── .lock # per-pipeline filelock (held during runs) ├── last-successful-rowcounts.json # baseline for the rowcount guardrail └── / ├── A/ # source snapshot │ ├── BuiltinTag.parquet │ └── ... ├── B/ # destination snapshot │ └── ... ├── plan.parquet # the diff plan ├── errors.parquet # only when errors > 0 ├── cursors.json # {A: {Resource: cursor}, B: {Resource: cursor}} ├── schema-sub-hash.txt # invalidates the cache when shape changes └── run.json # status, mode, summary, finished_at ``` Override the root with `INFRAHUB_SYNC_CACHE_DIR=/path/to/shared/cache`. ## plan.parquet[​](#planparquet "Direct link to plan.parquet") One row per change. The columns are: | Column | Description | | ---------------- | ---------------------------------------------------------------------------------------------------------------------- | | `action` | `create`, `update`, or `delete`. Empty for no-op elements (which are skipped during serialization). | | `resource` | Kind name as declared in `schema_mapping[].name`. | | `source_id` | DiffSync `unique_id` of the source-side element. | | `dest_id` | Reserved for the destination's primary key once adapters return it. Empty today. | | `attribute` | Reserved for per-attribute granularity. Empty today (rows are per-element). | | `old_value` | JSON-encoded mapping of `{attr: prior_value}` from `element.get_attrs_diffs()["-"]`. Populated on `update` actions. | | `new_value` | JSON-encoded mapping of `{attr: new_value}` from `element.get_attrs_diffs()["+"]`. Populated on `create` and `update`. | | `owner` | Reserved for sync-identity-based skip logic. Empty today. | | `skip_reason` | Empty unless the engine deliberately skipped a row. | | `conflict_class` | Empty unless the engine flagged a write conflict. | Query with DuckDB without any import step: ``` duckdb -c "SELECT action, resource, source_id, new_value FROM read_parquet('.infrahub-sync-cache/from-netbox//plan.parquet') WHERE action <> 'create' LIMIT 20" ``` ## Commands[​](#commands "Direct link to Commands") * `infrahub-sync diff --name X` — writes side A, side B, and `plan.parquet`. * `infrahub-sync sync --name X` — runs diff then sync; writes the same cache artifacts as `diff` plus updates `last-successful-rowcounts.json` on success. * `infrahub-sync apply --name X --run-id ` — dispatches the cached plan against the destination without re-extracting the source. Refuses if the destination's schema sub-hash has drifted. * `--allow-rowcount-drop` (on `sync`) bypasses the rowcount guardrail when the operator knows the source has legitimately shrunk. * `--continue-on-error` (on `sync`) skips peer relationships missing identifier values rather than aborting; the engine logs each skip so you can review what was dropped. * `--no-concurrent-load` (on `diff` and `sync`) falls back to loading source then destination sequentially. The default (concurrent) is safe with all built-in adapters and roughly halves load wall-clock time on real APIs. --- # `infrahub-sync` Infrahub-sync: synchronize data between infrastructure sources and destinations. **Usage**: ``` $ infrahub-sync [OPTIONS] COMMAND [ARGS]... ``` **Options**: * `--verbosity [quiet|default|verbose]`: Log verbosity level \[default: default] * `-v, --verbose`: Shorthand for --verbosity verbose * `-q, --quiet`: Shorthand for --verbosity quiet * `--install-completion`: Install completion for the current shell. * `--show-completion`: Show completion for the current shell, to copy it or customize the installation. * `--help`: Show this message and exit. **Commands**: * `list`: List all available SYNC projects. * `diff`: Calculate and print the differences... * `sync`: Synchronize the data between source and... * `apply`: Apply a previously cached plan against the... * `generate`: Generate all the Python files for a given... ## `infrahub-sync list`[​](#infrahub-sync-list "Direct link to infrahub-sync-list") List all available SYNC projects. **Usage**: ``` $ infrahub-sync list [OPTIONS] ``` **Options**: * `--directory TEXT`: Base directory to search for sync configurations * `--help`: Show this message and exit. ## `infrahub-sync diff`[​](#infrahub-sync-diff "Direct link to infrahub-sync-diff") Calculate and print the differences between the source and the destination systems for a given project. **Usage**: ``` $ infrahub-sync diff [OPTIONS] ``` **Options**: * `--name TEXT`: Name of the sync to use * `--config-file TEXT`: File path to the sync configuration YAML file * `--directory TEXT`: Base directory to search for sync configurations * `--branch TEXT`: Branch to use for the diff. * `--show-progress / --no-show-progress`: Show a progress bar (default: auto-detect terminal) * `--adapter-path TEXT`: Paths to look for adapters. Can be specified multiple times. * `--run-id TEXT`: Re-use a specific cache run id. * `--concurrent-load / --no-concurrent-load`: Load source and destination concurrently. Disable when a custom adapter isn't thread-safe. \[default: concurrent-load] * `--full-extract / --no-full-extract`: Re-extract every resource from scratch (default). Pass --no-full-extract to enable the cursor-driven incremental path on warm runs — see docs/reference/incremental-extraction. \[default: full-extract] * `--help`: Show this message and exit. ## `infrahub-sync sync`[​](#infrahub-sync-sync "Direct link to infrahub-sync-sync") Synchronize the data between source and the destination systems for a given project or configuration file. **Usage**: ``` $ infrahub-sync sync [OPTIONS] ``` **Options**: * `--name TEXT`: Name of the sync to use * `--config-file TEXT`: File path to the sync configuration YAML file * `--directory TEXT`: Base directory to search for sync configurations * `--branch TEXT`: Branch to use for the sync. * `--diff / --no-diff`: Print the differences between the source and the destination before syncing \[default: diff] * `--show-progress / --no-show-progress`: Show a progress bar (default: auto-detect terminal) * `--adapter-path TEXT`: Paths to look for adapters. Can be specified multiple times. * `--parallel / --no-parallel`: Sync tier-by-tier using the auto-computed dep graph. Requires order: to be omitted from config.yml. \[default: parallel] * `--allow-rowcount-drop / --no-allow-rowcount-drop`: Skip the rowcount drop guardrail. Use only when you know the source intentionally shrank. \[default: no-allow-rowcount-drop] * `--continue-on-error / --no-continue-on-error`: Log and skip peer relationships whose identifier values are missing instead of aborting. Useful when source data is partial; review the warnings before relying on the result. \[default: no-continue-on-error] * `--concurrent-load / --no-concurrent-load`: Load source and destination concurrently. Disable when a custom adapter isn't thread-safe. \[default: concurrent-load] * `--full-extract / --no-full-extract`: Re-extract every resource from scratch (default). Pass --no-full-extract to enable the cursor-driven incremental path on warm runs — see docs/reference/incremental-extraction. \[default: full-extract] * `--help`: Show this message and exit. ## `infrahub-sync apply`[​](#infrahub-sync-apply "Direct link to infrahub-sync-apply") Apply a previously cached plan against the destination — no source extraction. **Usage**: ``` $ infrahub-sync apply [OPTIONS] ``` **Options**: * `--name TEXT`: Name of the sync to use * `--config-file TEXT`: File path to the sync configuration YAML file * `--directory TEXT`: Base directory to search for sync configurations * `--run-id TEXT`: Cache run id produced by a previous `diff`. \[required] * `--branch TEXT`: Branch to use for the apply. * `--help`: Show this message and exit. ## `infrahub-sync generate`[​](#infrahub-sync-generate "Direct link to infrahub-sync-generate") Generate all the Python files for a given sync based on the configuration. **Usage**: ``` $ infrahub-sync generate [OPTIONS] ``` **Options**: * `--name TEXT`: Name of the sync to use * `--config-file TEXT`: File path to the sync configuration YAML file * `--directory TEXT`: Base directory to search for sync configurations * `--branch TEXT`: Branch to use for the sync. * `--adapter-path TEXT`: Paths to look for adapters. Can be specified multiple times. * `--help`: Show this message and exit. --- # Sync instance configuration The configuration file allows you to define the resources needs for the sync. The file should be formatted as a Yaml file, have the filename `config.yml`. During the generation phase, the Sync adapters, and models will be generated in the same folder. The following settings can be defined: ## Configuration fields[​](#configuration-fields "Direct link to Configuration fields") Describes the overall synchronization configuration. | Property | Type | Description | Mandatory | | ---------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | --------- | | `name` | string | Unique identifier for the sync instance. | Yes | | `store` | `SyncStore` | Configuration for the optional storage mechanism. | No | | `source` | `SyncAdapter` | Configuration for the source adapter. | Yes | | `destination` | `SyncAdapter` | Configuration for the destination adapter. | Yes | | `order` | List of strings | Order in which objects should be synchronized. Optional — when omitted, infrahub-sync auto-computes tiers from schema\_mapping. | No | | `schema_mapping` | List of `SchemaMappingModel` | Defines how data is mapped from source to destination. | Yes | | `diffsync_flags` | List of `DiffSyncFlags` | Instruct Infrahub Sync how to handle some specific situation without changing the data | No | ### Auto-tiered execution[​](#auto-tiered-execution "Direct link to Auto-tiered execution") `order:` is now optional. When it is omitted, infrahub-sync derives a write-order graph from the `reference:` entries in each `schema_mapping` field and groups kinds into **tiers**: * Tier 0: kinds with no outgoing references. * Tier N: kinds whose references all live in tiers `0..N-1`. The flattened tier order replaces the manual `order:` list. Tiers and any optional edges dropped to break cycles are logged at `INFO` level when `diff` or `sync` runs. `infrahub-sync sync` runs with `--parallel` on by default: the engine narrows the destination's `top_level` to one tier at a time so no tier starts before the previous tier's writes have completed. Pass `--no-parallel` to disable the tier boundary and fall back to the legacy single-pass code path. If you must override the computed order (because it doesn't match an adapter quirk), keep the `order:` list — it always wins, and `--parallel` will warn and fall back to serial when an explicit order is set. ### Sync store[​](#sync-store "Direct link to Sync store") Optional configuration for a storage mechanism used for stateful synchronization. | Property | Type | Description | Mandatory | | -------- | ---------- | ---------------------------------------------------- | --------- | | type | string | Type of the store (for example `redis`). | Yes | | settings | Dictionary | Connection details and other settings for the store. | No | ### Source and Destination[​](#source-and-destination "Direct link to Source and Destination") Configuration for source and destination adapters. | Property | Type | Description | Mandatory | | -------- | ---------- | -------------------------------------------------------------------------------------------------------------------- | --------- | | name | string | Identifier for the adapter. | Yes | | settings | Dictionary | Adapter-specific settings like `url` and `token`. If not provided, values will be loaded from environment variables. | No | ### Schema Mapping[​](#schema-mapping "Direct link to Schema Mapping") Defines the mappings from source to destination schemas. The tables below are the structural field reference. For the mapping syntax in depth — direct mappings, references, identifiers, filters, transforms, and worked NetBox and Nautobot examples — see [Schema mapping reference](/sync/reference/schema-mapping.md). #### Mapping models[​](#mapping-models "Direct link to Mapping models") | Property | Type | Description | Mandatory | | ----------- | ------------------------------ | ------------------------------------------------------- | --------- | | name | string | Infrahub model name to be mapped. | Yes | | mapping | string | The source's schema mapping (API) path. | Yes | | identifiers | List of strings | Fields used to uniquely identify an object. | Yes | | fields | List of SchemaMappingField | Describes individual field mappings or transformations. | Yes | | filters | List of SchemaMappingFilter | Filters you want to apply during the sync. | No | | transforms | List of SchemaMappingTransform | Transforms you want to apply during the sync. | No | ##### Mapping field[​](#mapping-field "Direct link to Mapping field") | Property | Type | Description | Mandatory | | --------- | ------ | ------------------------------------------------------------------------------------------- | --------- | | name | string | Name of the field. | Yes | | mapping | string | How the field is mapped from source to destination. | No | | static | Any | A static value to assign to the field, used if `mapping` is not provided. | No | | reference | string | Reference to another object in the configuration, used if direct mapping is not applicable. | No | ##### Mapping filters[​](#mapping-filters "Direct link to Mapping filters") | Property | Type | Description | Mandatory | | --------- | ------ | ------------------------------------------------------------------------------------- | ---------------------------------------------------------- | | field | string | Name of the source field to filter on. | Yes | | operation | string | Which filters will be applied on the field. Available operations are described below. | No | | value | string | The expected value linked to the operation. | Yes except for both is\_empty and is\_not\_empty operation | For the full list of filter operations (`==`, `!=`, `>`, `<`, `>=`, `<=`, `in`, `not in`, `contains`, `not contains`, `is_empty`, `is_not_empty`, `regex`, `is_ip_within`) with examples, see [Available filter operations](/sync/reference/schema-mapping.md#available-filter-operations). ##### Mapping transforms[​](#mapping-transforms "Direct link to Mapping transforms") | Property | Type | Description | Mandatory | | ---------- | ------ | ------------------------------------- | --------- | | field | string | Name of the target field. | Yes | | expression | string | A Jinja filter compatible expression. | Yes | --- # Incremental Extraction `infrahub-sync` can skip re-extracting unchanged data on warm runs by asking each backend "what changed since the last successful run?". ## Default behavior[​](#default-behavior "Direct link to Default behavior") `infrahub-sync` defaults to `--full-extract`: every run re-extracts every resource from scratch. The cursor-driven warm path is **opt-in** because timestamp filters miss deletes and because a fresh extract is the safer posture for a tool that writes to a downstream system. The cache machinery still runs under `--full-extract` — snapshots and cursor sidecars are written under the run dir so that the opt-in warm path is immediately usable when you switch to `--no-full-extract`. See [Cache layout](/sync/reference/cache-layout.md) for the on-disk shape. ## Enabling the incremental warm path[​](#enabling-the-incremental-warm-path "Direct link to Enabling the incremental warm path") ``` uv run infrahub-sync sync --name from-netbox --directory examples/ --no-full-extract ``` When `--no-full-extract` is set, the engine takes the cursor path **if all** of these hold: 1. A prior run exists under `.infrahub-sync-cache//` with `run.json` status `applied` (or `dry-run`). 2. `schema-sub-hash.txt` from that run matches the current schema mapping + destination schema. Any mapping change forces a full extract. 3. The adapter declares a non-NONE cursor tier for the resource (`cursor_tier_for()` — see adapter docs). 4. The run counter has not hit the configured cadence (default: every 10 runs, configurable via `incremental.full_resync_every` in `config.yml`). If any condition fails the engine falls back to the full extract path for that side / resource. ## When to keep `--full-extract`[​](#when-to-keep---full-extract "Direct link to when-to-keep---full-extract") * Investigating a discrepancy and you suspect cached state. * A backend has had data deleted and you want the delete reflected immediately (timestamp filters do not catch deletes — the cadence knob handles this routinely). ## Supported backends[​](#supported-backends "Direct link to Supported backends") | Adapter | Tier | Notes | | -------------------- | --------- | ---------------------------------- | | NetBox source | TIMESTAMP | `last_updated__gte` | | Nautobot source | TIMESTAMP | `last_updated__gte` | | Infrahub destination | TIMESTAMP | `node_metadata__updated_at__after` | | Others | NONE | Always full extract today | ## Soft deletes[​](#soft-deletes "Direct link to Soft deletes") Timestamp-based incremental misses DELETEs (the deleted row has no `last_updated` to match). The engine forces a full extract every N runs (default 10) to reconcile deletes. Set `incremental.full_resync_every: 1` to disable incremental entirely. A future optimization will add an ID-only sweep (`adapter.list_existing_ids`) so deletes are caught on every warm run — the contract is in place but not yet wired into the engine. --- # Schema mapping reference The schema mapping section of a sync project's `config.yml` defines how data translates between the source system and the destination system. Worked examples for NetBox → Infrahub and Nautobot → Infrahub appear at the end. For the full configuration schema (not just the mapping section), see [Sync instance configuration](/sync/reference/config.md). ## Anatomy of a schema mapping[​](#anatomy-of-a-schema-mapping "Direct link to Anatomy of a schema mapping") Each entry in the `schema_mapping` section maps one model from the source system to one model in the destination system. A mapping entry contains: * **Name** — the destination model name. * **Mapping** — the source model name in the source system. * **Identifiers** — the field(s) that uniquely identify an object in this model. * **Fields** — the per-field mappings between source and destination. A minimal example: ``` schema_mapping: - name: Device mapping: dcim.device identifiers: ["name"] fields: - name: name mapping: name - name: serial mapping: serial - name: status mapping: status.value ``` This says: for each `dcim.device` in the source system, create or update a `Device` object in the destination using the `name` field as the unique identifier, and map the `name`, `serial`, and `status.value` fields directly. ## Field mappings[​](#field-mappings "Direct link to Field mappings") Each entry in the `fields` list defines one field mapping. The most common patterns are direct mapping, nested attribute mapping, static values, and references to other models. ### Direct field mapping[​](#direct-field-mapping "Direct link to Direct field mapping") The source field name maps directly to the destination field name: ``` - name: name mapping: name ``` This is the most common case. The destination field `name` gets the value of the source field `name`. ### Renaming fields[​](#renaming-fields "Direct link to Renaming fields") When the destination field has a different name than the source field, list them separately: ``` - name: device_role mapping: role.slug ``` The destination field `device_role` gets the value of the source's `role.slug`. ### Nested attribute mapping[​](#nested-attribute-mapping "Direct link to Nested attribute mapping") For source data with nested structure, use dotted notation to reach into the nested attribute: ``` - name: status mapping: status.value - name: site mapping: site.slug - name: primary_ip mapping: primary_ip4.address ``` ### Static values[​](#static-values "Direct link to Static values") When a destination field should always be set to the same value regardless of source data, use `static`: ``` - name: source_system static: netbox ``` This is useful for fields that record provenance, type discriminators, or any field where the value is determined by the sync configuration rather than the source data. ### References to other models[​](#references-to-other-models "Direct link to References to other models") When a destination field is a relationship to another model that is also being synced, use `reference` to point to that model: ``` - name: device mapping: device.name reference: Device ``` This says: the value of the source's `device.name` field should resolve to the `Device` object with that name in the destination. The referenced model must be synced before this one — see the `order` key in the project configuration. ## Identifiers[​](#identifiers "Direct link to Identifiers") The `identifiers` list defines which field(s) uniquely identify an object of this model. Identifiers are used to determine whether an object already exists in the destination and should be updated, or whether a new object should be created. ### Single-field identifiers[​](#single-field-identifiers "Direct link to Single-field identifiers") For most models, a single field — typically `name` — is enough: ``` identifiers: ["name"] ``` ### Composite identifiers[​](#composite-identifiers "Direct link to Composite identifiers") Some models need multiple fields to be uniquely identified. For example, an IP address is unique by address plus VRF: ``` identifiers: ["address", "vrf"] ``` All listed fields must be present and unique together. Each identifier field must also be mapped in the `fields` list. ## Filters[​](#filters "Direct link to Filters") Filters control which objects from the source get synced. Apply them at the model level to include or exclude specific objects. ``` - name: Device mapping: dcim.device identifiers: ["name"] filters: - field: status.value operation: "==" value: active fields: - name: name mapping: name ``` This syncs only devices with `status.value == "active"`. Other devices are skipped. ### Available filter operations[​](#available-filter-operations "Direct link to Available filter operations") Infrahub Sync supports the following 14 filter operations: | Operation | Description | | -------------- | -------------------------------------------------------------- | | `==` | Equal to the value | | `!=` | Not equal to the value | | `>` | Greater than (numeric) | | `<` | Less than (numeric) | | `>=` | Greater than or equal (numeric) | | `<=` | Less than or equal (numeric) | | `in` | Value is in a list or string | | `not in` | Value is not in a list or string | | `contains` | Field value contains the given value | | `not contains` | Field value does not contain the given value | | `is_empty` | Field is `None` or empty (no `value` argument needed) | | `is_not_empty` | Field is not `None` and not empty (no `value` argument needed) | | `regex` | Field matches the regular expression in `value` | | `is_ip_within` | Field (an IP address) is within the IP range in `value` | Multiple filters on the same model are combined with AND — an object must match all filters to be included. ## Transforms and custom Jinja filters[​](#transforms-and-custom-jinja-filters "Direct link to Transforms and custom Jinja filters") For cases where a field value needs to be transformed during the mapping — uppercase a string, parse a date, compute a derived value — apply a `transforms` entry to the mapping. The transform takes a `field` (the target field name) and an `expression` (a Jinja-compatible expression evaluated against the source object). ``` - name: Device mapping: dcim.device identifiers: ["name"] fields: - name: name mapping: name transforms: - field: name expression: "{{ name | upper }}" ``` Adapters can also register custom Jinja filters via a `_add_custom_filters` class method on the adapter model. The ACI adapter, for example, includes an `aci_device_name` filter for resolving ACI node IDs to device names. See [Local Adapters](/sync/adapters/local-adapters.md) for the implementation pattern, and [Sync instance configuration](/sync/reference/config.md) for the full `transforms` syntax. ## Validating a mapping[​](#validating-a-mapping "Direct link to Validating a mapping") Validate the mapping with the diff command before applying any changes: ``` infrahub-sync diff --name --directory ``` The diff prints every proposed create, update, and delete. Review it model by model: * Are the field values being mapped correctly? * Are references resolving to the right objects in the destination? * Are filters excluding the right objects? * Are there unexpected creates or deletes? Adjust the mapping and re-run the diff until it matches expectations. Only then run the sync. *** Two common source systems — NetBox and Nautobot — illustrate how the syntax above plays out: ## Common patterns for NetBox → Infrahub[​](#common-patterns-for-netbox--infrahub "Direct link to Common patterns for NetBox → Infrahub") The patterns below cover the most common NetBox → Infrahub mappings. Adapt the destination field names to match your Infrahub schema. ### Sites and locations[​](#sites-and-locations "Direct link to Sites and locations") ``` - name: Location mapping: dcim.site identifiers: ["name"] fields: - name: name mapping: name - name: slug mapping: slug - name: description mapping: description - name: status mapping: status.value ``` ### Devices with role and platform references[​](#devices-with-role-and-platform-references "Direct link to Devices with role and platform references") ``` - name: Device mapping: dcim.device identifiers: ["name"] fields: - name: name mapping: name - name: serial mapping: serial - name: location mapping: site.slug reference: Location - name: role mapping: role.slug reference: DeviceRole - name: platform mapping: platform.slug reference: Platform ``` ### Interfaces with device reference[​](#interfaces-with-device-reference "Direct link to Interfaces with device reference") ``` - name: Interface mapping: dcim.interface identifiers: ["name", "device"] fields: - name: name mapping: name - name: device mapping: device.name reference: Device - name: enabled mapping: enabled - name: type mapping: type.value ``` ### IP addresses with VRF reference[​](#ip-addresses-with-vrf-reference "Direct link to IP addresses with VRF reference") ``` - name: IPAddress mapping: ipam.ip-address identifiers: ["address", "vrf"] fields: - name: address mapping: address - name: vrf mapping: vrf.name reference: VRF - name: status mapping: status.value - name: description mapping: description ``` ### Filtering by tenant or tag[​](#filtering-by-tenant-or-tag "Direct link to Filtering by tenant or tag") ``` - name: Device mapping: dcim.device identifiers: ["name"] filters: - field: tenant.slug operation: "==" value: production fields: - name: name mapping: name ``` ## Common patterns for Nautobot → Infrahub[​](#common-patterns-for-nautobot--infrahub "Direct link to Common patterns for Nautobot → Infrahub") Nautobot's API and model structure are similar to NetBox but with some differences in field names and nested structure. The mapping patterns above largely apply; the main differences: * Nautobot uses `display` instead of `name` in some places. * Nautobot status fields are objects rather than strings — use `status.name` instead of `status.value`. * Custom fields and computed fields appear in different sections of the API response. Start from the `examples/nautobot_to_infrahub/` directory in the Infrahub Sync repository and adapt from there. ## Related[​](#related "Direct link to Related") * [Sync instance configuration](/sync/reference/config.md) * [Create a sync project](/sync/creating-a-sync-project.md) * [NetBox adapter](/sync/adapters/netbox.md) * [Nautobot adapter](/sync/adapters/nautobot.md) --- ## [🗃Infrahub Sync](/sync/release-notes/infrahub-sync.md) [4 items](/sync/release-notes/infrahub-sync.md) --- ## [📄️Release 2.0.1](/sync/release-notes/infrahub-sync/release-2_0_1.md) [Release Number](/sync/release-notes/infrahub-sync/release-2_0_1.md) --- # Release 1.5.6 | Release Number | 1.5.6 | | -------------- | -------------------------------------------------------------------- | | Release Date | February 18th, 2026 | | Tag | [1.5.6](https://github.com/opsmill/infrahub-sync/releases/tag/1.5.6) | ## Fixed[​](#fixed "Direct link to Fixed") * Fixed a race in the Slurp’it adapter by processing interface records sequentially instead of concurrently. Slurp’it sync runs now produce deterministic interface output. ([#115](https://github.com/opsmill/infrahub-sync/pull/115)) --- # Release 1.6.0 | Release Number | 1.6.0 | | -------------- | -------------------------------------------------------------------- | | Release Date | March 26th, 2026 | | Tag | [1.6.0](https://github.com/opsmill/infrahub-sync/releases/tag/1.6.0) | Infrahub Sync 1.6.0 is available. This release updates CLI, engine, and adapter messages to use Python logging instead of direct terminal printing. It also adds verbosity and progress controls for scheduled runs and interactive troubleshooting. ## Main changes[​](#main-changes "Direct link to Main changes") ### Automation-friendly logging[​](#automation-friendly-logging "Direct link to Automation-friendly logging") The package CLI, sync engine, and built-in adapters now emit operational messages through Python logging instead of writing directly to the terminal. A run produces one consistent log stream rather than a mix of printed lines and application logs. The default CLI format remains human-readable. When you embed Infrahub Sync in Python automation, CI, cron, or workflow schedulers, attach your own handlers and formatters to route logs alongside the rest of your pipeline. ### Verbosity and progress controls[​](#verbosity-and-progress-controls "Direct link to Verbosity and progress controls") The CLI now includes global verbosity controls: ``` infrahub-sync --verbosity quiet ... infrahub-sync --verbosity verbose ... infrahub-sync -q ... infrahub-sync -v ... ``` Commands that show progress now support `--show-progress / --no-show-progress`. Progress bars are auto-detected for interactive terminals by default. Use `--no-show-progress` in automation that needs stable, machine-readable output. ## Upgrade notes[​](#upgrade-notes "Direct link to Upgrade notes") * No configuration changes are required. * Review any scripts that parse human-readable output from `list`, `diff`, `sync`, or `generate`. Operational messages now flow through Python logging. * Use `--quiet`, `--verbose`, or `--verbosity` to tune log volume for automation and troubleshooting. * Use `--no-show-progress` in non-interactive environments that require stable log output. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * Added global `--verbosity`, `--verbose`/`-v`, and `--quiet`/`-q` controls. ([#117](https://github.com/opsmill/infrahub-sync/pull/117)) * Added `--show-progress / --no-show-progress` to `diff` and `sync`, with terminal auto-detection by default. ([#117](https://github.com/opsmill/infrahub-sync/pull/117)) ### Changed[​](#changed "Direct link to Changed") * Replaced direct `print()` and Rich console output with Python logging in the package CLI, sync engine, and built-in adapters. ([#117](https://github.com/opsmill/infrahub-sync/pull/117)) * Updated adapter documentation command examples from Poetry to uv. ([#117](https://github.com/opsmill/infrahub-sync/pull/117)) * Updated development guidance for the uv-based workflow and Python 3.13 support. ([#117](https://github.com/opsmill/infrahub-sync/pull/117)) --- # Release 2.0.0 | Release Number | 2.0.0 | | -------------- | -------------------------------------------------------------------- | | Release Date | June 8th, 2026 | | Tag | [2.0.0](https://github.com/opsmill/infrahub-sync/releases/tag/2.0.0) | Infrahub Sync 2.0.0 is available. This release focuses on recurring synchronization at scale. Run large syncs on a schedule, automate them safely, and inspect exactly what changed on every run. The focus of 2.0 is operational reliability: faster incremental syncs, automatic dependency ordering, built-in safety guardrails, and on-disk run artifacts. Together these make it practical to run synchronization as a recurring part of infrastructure operations. ## Release highlights[​](#release-highlights "Direct link to Release highlights") * **Schedule recurring syncs without re-reading entire datasets.** Recurring syncs used to re-read the full source and destination on every run. After the first run, `--no-full-extract` lets supported adapters extract only changed records, so large sync jobs stay fast enough to run on a schedule. See [Incremental syncs for large datasets](#incremental-syncs-for-large-datasets). * **Add and evolve models without maintaining a write order.** Sync ordering used to be maintained manually in `config.yml` and updated on every schema change. It is now derived from your schema mapping automatically. Evolve data models, onboard new systems, and extend integrations without updating sync configuration on every change. See [Write ordering derived from schema mapping](#write-ordering-derived-from-schema-mapping). * **Run synchronization safely in unattended environments.** Source system outages, permission issues, or incomplete datasets no longer risk deleting valid destination data. Built-in guardrails detect unexpected source drops and stop the run, or skip the affected records, before writing to the destination. Run scheduled syncs in CI/CD pipelines and cron jobs without risking accidental data loss from source anomalies. See [Row count guardrails for unattended runs](#row-count-guardrails-for-unattended-runs). * **Inspect, audit, and troubleshoot every sync run.** Every `diff` and `sync` used to be discarded after execution. Both are now saved to disk. Review exactly what changed, investigate unexpected results, and build repeatable operational workflows around sync. See [Per-run artifacts for diff and sync](#per-run-artifacts-for-diff-and-sync). What to expect after upgrading Your existing sync projects keep working without changes, and most run faster, because loading and writing now happen in parallel by default. Three situations need action: * If you use a custom adapter that is not thread-safe, pass `--no-concurrent-load`. * If a sync depends on a specific write order, set an explicit `order` list in `config.yml`. * `diff` and `sync` now write a cache under `.infrahub-sync-cache/`. In scheduled environments, plan to clean it up so per-run artifacts do not accumulate. The [upgrade notes](#upgrade-notes) list every default that changed. ## Main changes[​](#main-changes "Direct link to Main changes") ### Incremental syncs for large datasets[​](#incremental-syncs-for-large-datasets "Direct link to Incremental syncs for large datasets") Large synchronization jobs previously required reading both source and destination systems in full on every run, so execution time grew with the dataset. Infrahub Sync now loads both systems concurrently and, after the first run, can extract only changed records on supported adapters. Run large syncs on a schedule and keep systems in sync as datasets change. The approved benchmark for [#127](https://github.com/opsmill/infrahub-sync/pull/127) used the `nautobot-v2` demo dataset with interfaces omitted and about 14 object kinds: | Scenario | Cold | Warm | | ------------------------------------------- | ------ | ------ | | Baseline, serial, no concurrent load | 463.8s | 154.3s | | `--parallel` with `--concurrent-load` | 437.5s | 132.3s | | `--no-full-extract`, cursor-driven warm run | 434.5s | 6.3s | * Pass `--no-concurrent-load` for custom adapters that are not thread-safe. Concurrent loading is on by default and does not change the sync result. * Enable incremental extraction with `--no-full-extract` to read only changed records after the first run. The default `--full-extract` re-reads every resource on each run. On supported adapters it uses timestamp cursors: NetBox and Nautobot use `last_updated__gte`, and Infrahub uses `node_metadata__updated_at__after`. * Infrahub Sync falls back to a full extract when the schema mapping changes, when no prior successful run exists, when an adapter has no cursor for a resource, or when the full-resync cadence is reached. * Timestamp cursors do not detect deletes. Infrahub Sync forces a full extract every 10 runs by default to reconcile them. See the [incremental extraction reference](/sync/reference/incremental-extraction.md) for details. ### Write ordering derived from schema mapping[​](#write-ordering-derived-from-schema-mapping "Direct link to Write ordering derived from schema mapping") Add and evolve models without updating sync configuration. Synchronization writes objects in dependency order, and that order used to be maintained manually in `config.yml` and updated on every schema change. Infrahub Sync now derives it from the `reference` relationships in your schema mapping. * Infrahub Sync builds the dependency graph from the `reference` relationships in your `schema_mapping`. It groups object kinds into tiers; objects within a tier have no cross-dependencies. * When `order` is omitted, Infrahub Sync writes each tier in full before starting the next, so no object is created before what it references. `--parallel` is on by default and writes objects within a tier concurrently. * Set an explicit `order` list when you need a specific sequence; Infrahub Sync logs a warning and uses it. ### Row count guardrails for unattended runs[​](#row-count-guardrails-for-unattended-runs "Direct link to Row count guardrails for unattended runs") Schedule syncs in CI jobs or cron without manually reviewing each run for source anomalies. An outage, a partial restore, or a permissions change can make a source return far fewer records than it holds. Previously, a sync would apply that drop as a deletion. Infrahub Sync now checks record counts against a baseline before writing and stops the run, or skips the affected records, when the drop exceeds the configured threshold. * Pass `--allow-rowcount-drop` only when the source intentionally shrank. By default, the rowcount check stops a run when a resource drops by more than 50 percent from its last successful baseline. * Pass `--continue-on-error` to log and skip a record that links to an object it cannot find, such as a peer relationship with a missing identifier, instead of aborting the run. Use it for partial source data, then review the warnings before relying on the result. ### Per-run artifacts for diff and sync[​](#per-run-artifacts-for-diff-and-sync "Direct link to Per-run artifacts for diff and sync") Review, audit, and troubleshoot any run after it completes. `diff` used to print to the terminal and keep nothing. Every `diff` and `sync` now saves its snapshots and change plan to disk, so you can inspect what changed and investigate an unexpected result. * Find each run's source snapshot, destination snapshot, `plan.parquet`, run metadata, cursors, and schema sub-hash under `.infrahub-sync-cache///`. The sync-level rowcount baseline is stored at `.infrahub-sync-cache//last-successful-rowcounts.json`. See the [cache layout reference](/sync/reference/cache-layout.md) for the file structure and plan columns. * Query the plan with tools such as DuckDB before the next run, or to investigate an unexpected diff: ``` uv run infrahub-sync diff --name from-netbox --directory examples/ duckdb -c "SELECT action, resource, source_id FROM read_parquet('.infrahub-sync-cache/from-netbox//plan.parquet') LIMIT 20" ``` * Use `apply` (preview) to replay a cached plan without re-extracting the source. In 2.0.0 it requires destination adapter support for cached-row application, so use `sync` for built-in adapter workflows until that support is available. `apply` refuses to run when the current schema mapping and destination schema shape no longer match the cached schema sub-hash. ### Type checking uses ty in invoke lint[​](#type-checking-uses-ty-in-invoke-lint "Direct link to Type checking uses ty in invoke lint") Run `invoke lint` with stricter type checking and no override blocks. The project used mypy with overrides that hid known type errors. Migrating to ty removed those overrides and fixed several latent runtime bugs in the process. * [ty](https://github.com/astral-sh/ty) runs alongside ruff, pylint, and yamllint in `invoke lint`. * The migration also fixed latent runtime failures in CLI narrowing, adapter peer resolution, resource-name handling, and Slurp'it conversion return types. ## Upgrade notes[​](#upgrade-notes "Direct link to Upgrade notes") * `order` is now optional when `schema_mapping` contains enough `reference` information to derive the dependency graph. Keep `order` when you need a configured sequence. * `--parallel`, `--concurrent-load`, and `--full-extract` are enabled by default. Use `--no-concurrent-load` for custom adapters that are not thread-safe. * Incremental extraction is opt-in with `--no-full-extract`. * `.infrahub-sync-cache/` is now part of normal operation and is ignored by Git. * Review automation that parses command output. The new cache workflow adds run identifiers and cache paths to successful `diff` and `sync` runs. * Review retention and cleanup for `.infrahub-sync-cache/` in scheduled environments, because recurring runs now persist snapshots and plans. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Added[​](#added "Direct link to Added") * Added automatic sync ordering from `schema_mapping` reference relationships, making `order` optional for configurations with enough dependency information. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Added tier-by-tier sync execution with `--parallel` enabled by default when `order` is omitted. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Added per-run cache artifacts under `.infrahub-sync-cache///`, including source and destination snapshots, run metadata, cursors, and schema-sub-hash data. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Added sync-level rowcount baselines under `.infrahub-sync-cache//last-successful-rowcounts.json`. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Added Parquet diff plans for `diff` and `sync`, so run output can be inspected with external tools. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Added the `infrahub-sync apply` command (preview) as the foundation for replaying cached plans with destination adapters that support cached-row application. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Added cursor-based incremental extraction for NetBox, Nautobot, and Infrahub, enabled with `--no-full-extract`. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Added rowcount guardrails that stop `sync` when a resource count drops by more than 50 percent from the previous successful baseline. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Added `--continue-on-error` to skip peer relationships with missing identifier values while logging warnings. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) ### Changed[​](#changed "Direct link to Changed") * Enabled concurrent source and destination loading by default for `diff` and `sync`, with `--no-concurrent-load` available for custom adapters that are not thread-safe. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Enabled `--parallel` and `--full-extract` by default for `sync`. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Updated example configurations to omit `order` by default and show how to opt back into a manual sequence. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Replaced mypy with ty in the development and CI linting workflow. ([#126](https://github.com/opsmill/infrahub-sync/pull/126)) ### Fixed[​](#fixed "Direct link to Fixed") * Improved peer identifier handling so missing peer identifier values produce clearer errors, or warnings when `--continue-on-error` is used. ([#127](https://github.com/opsmill/infrahub-sync/pull/127)) * Fixed several type-checker-discovered runtime failure paths in CLI narrowing, resource-name handling, Infrahub peer resolution, and Slurp'it conversion return values. ([#126](https://github.com/opsmill/infrahub-sync/pull/126)) --- # Release 2.0.1 | Release Number | 2.0.1 | | -------------- | -------------------------------------------------------------------- | | Release Date | September 11th, 2026 | | Tag | [2.0.1](https://github.com/opsmill/infrahub-sync/releases/tag/2.0.1) | Infrahub Sync 2.0.1 is a maintenance release for 2.0.0. Install the package on its own and every module it imports at runtime is present. Synchronize NetBox interfaces in tagged L2 modes, update relationships on an existing Infrahub object, or generate models from a schema read over the API, and each completes correctly. ## Bug fixes[​](#bug-fixes "Direct link to Bug fixes") ### Synchronize tagged and tagged-all interfaces from NetBox[​](#synchronize-tagged-and-tagged-all-interfaces-from-netbox "Direct link to Synchronize tagged and tagged-all interfaces from NetBox") Run a NetBox synchronization that includes interfaces in `tagged` or `tagged-all` 802.1Q mode. These previously failed while being translated to the destination model. * NetBox `q-in-q` mode is refused explicitly, because the example destination schema cannot represent it. * A malformed non-null mode fails at the transform boundary with the offending value named, instead of producing an invalid interface. ([#176](https://github.com/opsmill/infrahub-sync/pull/176)) ### Keep lineage on relationships changed by an update[​](#keep-lineage-on-relationships-changed-by-an-update "Direct link to Keep lineage on relationships changed by an update") Attribute source and owner on relationship edges that a destination update adds or changes. Lineage was applied when an object was first created, but edges written by a later update to an existing object carried none, so attribution depended on which run happened to write the edge. ([#143](https://github.com/opsmill/infrahub-sync/pull/143)) ### Generate models from schemas read over the API[​](#generate-models-from-schemas-read-over-the-api "Direct link to Generate models from schemas read over the API") Run `infrahub-sync generate` against a recent infrahub-sdk and get the same annotations as before. Optional attributes and many-relationships are annotated and defaulted correctly again, because `get_kind()` is duck-typed rather than checked with `isinstance` against write-side schema classes. * String attribute defaults are emitted as Python literals, so a default containing a quote, a newline, or a backslash stays valid Python and keeps its exact schema value. ([#188](https://github.com/opsmill/infrahub-sync/pull/188), [#187](https://github.com/opsmill/infrahub-sync/issues/187)) ### Read non-string attributes without converting them[​](#read-non-string-attributes-without-converting-them "Direct link to Read non-string attributes without converting them") Read attributes from Infrahub and keep their type. Every non-string attribute value was converted to a string, so an integer or a boolean reached the destination as text. Only IP-related types are converted to strings now. ([#130](https://github.com/opsmill/infrahub-sync/pull/130)) ## Installation and dependencies[​](#installation-and-dependencies "Direct link to Installation and dependencies") Install infrahub-sync into a clean environment and every package the product imports is declared by the package itself, rather than arriving through the `dev` extra or through another dependency. * `requests` and `urllib3` are runtime dependencies. `packaging`, `pydantic`, and `typing-extensions` are declared directly instead of relying on other packages to bring them. * infrahub-sync no longer installs the `infrahub-sdk[all]` extra. It declares `jinja2`, `pyyaml`, and `typer` directly, which are the only packages it used from that extra, so a plain install pulls fewer unrelated packages. The supported SDK range stays `>=1.17,<2` and infrahub-sdk 1.23.2 is the tested version, so infrahub-sync can be paired with the SDK release an Infrahub server needs. * The `netutils` and `pyarrow` lower bounds match the oldest versions that work on Python 3.13. The locked and tested diffsync is 2.2.3; the previously locked 2.2.2 was yanked upstream. ([#189](https://github.com/opsmill/infrahub-sync/pull/189)) ## Upgrade notes[​](#upgrade-notes "Direct link to Upgrade notes") * If your environment relied on a package that arrived transitively through `infrahub-sdk[all]`, declare it in your own project. Only `jinja2`, `pyyaml`, and `typer` are still pulled in through infrahub-sync. * Regenerate your sync models with `infrahub-sync generate` if you generated them against a recent infrahub-sdk and saw optional or list annotations missing. ## Full changelog[​](#full-changelog "Direct link to Full changelog") ### Changed[​](#changed "Direct link to Changed") * infrahub-sync no longer installs the `infrahub-sdk[all]` extra, and declares `jinja2`, `pyyaml`, and `typer` directly. The supported range stays `>=1.17,<2`, with infrahub-sdk 1.23.2 as the tested version. ([#189](https://github.com/opsmill/infrahub-sync/pull/189)) ### Fixed[​](#fixed "Direct link to Fixed") * Added `requests` and `urllib3` as runtime dependencies, declared `packaging`, `pydantic`, and `typing-extensions` directly, raised the `netutils` and `pyarrow` floors to versions that work on Python 3.13, and locked diffsync 2.2.3 after 2.2.2 was yanked upstream. ([#189](https://github.com/opsmill/infrahub-sync/pull/189)) * Applied source and owner lineage to relationship edges added or changed during an Infrahub destination update. ([#143](https://github.com/opsmill/infrahub-sync/pull/143)) * Fixed NetBox interface synchronization for `tagged` and `tagged-all` L2 modes, refused `q-in-q` explicitly, and made malformed non-null modes fail contextually at the transform boundary. ([#176](https://github.com/opsmill/infrahub-sync/pull/176)) * Fixed generated DiffSync model annotations for schemas read from the Infrahub API, and emitted string attribute defaults as Python literals. ([#188](https://github.com/opsmill/infrahub-sync/pull/188)) * Limited automatic string conversion to IP-related types, so other non-string attribute values keep their type. ([#130](https://github.com/opsmill/infrahub-sync/pull/130)) * Restored the `from-netbox` name of the shipped NetBox example sync project, which the tutorial work in [#140](https://github.com/opsmill/infrahub-sync/pull/140) had renamed. ([#147](https://github.com/opsmill/infrahub-sync/pull/147)) ### Housekeeping[​](#housekeeping "Direct link to Housekeeping") * Gave development tools declared without a lower bound one, so a lowest-version resolution can no longer select releases that predate Python 3. The `typer-cli` floor matters most: older releases pull `typer-slim`, which shadows the real `typer` package and breaks the CLI. ([#189](https://github.com/opsmill/infrahub-sync/pull/189)) --- # Run a sync Learn how to use Infrahub Sync's commands to calculate differences, synchronize data, and apply previously cached plans against your destination. ![Infrahub-Sync process](/assets/images/infrahub_sync_process.excalidraw-77809c6d9cf8697772543001bc5ad1a8.svg) info Before generating the necessary Python code for your sync adapters and models and synchronizing, you need to create a configuration. To create a new configuration, please refer to [Create a sync project](/sync/creating-a-sync-project.md). ## Listing available sync projects[​](#listing-available-sync-projects "Direct link to Listing available sync projects") ``` infrahub-sync list --directory ``` Prints every sync project found under the given directory along with its source, destination, and on-disk location. Useful as a quick sanity check. ## Calculating differences[​](#calculating-differences "Direct link to Calculating differences") The `diff` command compares the source and destination without writing anything to the destination. It also writes a Parquet **plan** to the local cache so you can review the change set and replay it later with `apply`. ### Command[​](#command "Direct link to Command") ``` infrahub-sync diff --name --directory ``` ### Parameters[​](#parameters "Direct link to Parameters") * `--name` — name of the sync project to diff. * `--directory` — directory holding your sync configuration. * `--branch` — Infrahub branch to diff against (default `main`). * `--show-progress / --no-show-progress` — toggle the per-resource progress bar. * `--run-id` — re-use a specific cache run id; useful when you want to overwrite a previous run's plan in place. * `--concurrent-load / --no-concurrent-load` — load source and destination concurrently (default on). Disable if a custom adapter isn't thread-safe; see [Concurrent loads](#concurrent-loads) below. * `--full-extract / --no-full-extract` — default on; re-extract everything every run. Pass `--no-full-extract` to enable the cursor-driven incremental warm path. See [Incremental extraction](/sync/reference/incremental-extraction.md). Each invocation logs a `Cached run at ` line on success. Note that id — you can hand it to `apply` to dispatch the plan without re-extracting the source. ## Synchronizing data[​](#synchronizing-data "Direct link to Synchronizing data") The `sync` command runs `diff` and immediately applies the changes to the destination. ### Command[​](#command-1 "Direct link to Command") ``` infrahub-sync sync --name --directory ``` ### Parameters[​](#parameters-1 "Direct link to Parameters") * `--name` — name of the sync project to run. * `--directory` — directory holding your sync configuration. * `--branch` — Infrahub branch to sync against (default `main`). * `--diff / --no-diff` — print the diff before syncing (default on). * `--show-progress / --no-show-progress` — progress bar during sync. * `--parallel / --no-parallel` — run tier-by-tier using the auto-computed dep graph (default on). Requires `order:` to be omitted from `config.yml` (see [Auto-tiered execution](/sync/reference/config.md#auto-tiered-execution)). Falls back to serial when `order:` is set; a warning is logged so the no-op is visible. * `--allow-rowcount-drop / --no-allow-rowcount-drop` — bypass the rowcount guardrail. Use only when you know the source intentionally shrank — otherwise sync refuses to proceed when any resource's row count has dropped by more than 50% since the last successful run. * `--continue-on-error / --no-continue-on-error` — log and skip peer relationships whose identifier values are missing, instead of aborting the run. Useful when source data is partial; review the warnings before relying on the result. * `--concurrent-load / --no-concurrent-load` — load source and destination concurrently (default on). See [Concurrent loads](#concurrent-loads) below. * `--full-extract / --no-full-extract` — default on. Pass `--no-full-extract` for the cursor-driven incremental warm path; see [Incremental extraction](/sync/reference/incremental-extraction.md). Example: ``` infrahub-sync sync --name my_project --directory configs --diff --show-progress ``` ### Concurrent loads[​](#concurrent-loads "Direct link to Concurrent loads") Source and destination loads run on a 2-thread pool by default. They hit independent services, write to independent in-memory stores, and write to disjoint cache subdirectories (`A/` vs `B/`), so the two loads are safe to run in parallel — and roughly halve the wall-clock time spent in the load phase on real APIs. Disable with `--no-concurrent-load` if a custom adapter you've plugged in isn't thread-safe (most aren't an issue — the built-in NetBox, Nautobot, and Infrahub adapters are all fine). ### Tier-by-tier execution[​](#tier-by-tier-execution "Direct link to Tier-by-tier execution") When `--parallel` is set and `order:` is omitted, Infrahub Sync derives a write-order graph from the `reference:` entries in your `schema_mapping` and groups kinds into tiers. The engine narrows the destination's working set to one tier at a time, so no tier starts until every kind in the previous tier has finished writing. See [Auto-tiered execution](/sync/reference/config.md#auto-tiered-execution) for the full rationale. ### Rowcount guardrail[​](#rowcount-guardrail "Direct link to Rowcount guardrail") After a successful sync, Infrahub Sync writes a per-resource baseline to `.infrahub-sync-cache//last-successful-rowcounts.json`. The next run reads it; if any resource has shrunk by more than 50% the sync refuses to proceed unless you pass `--allow-rowcount-drop`. The threshold catches accidents like a partially-restored source or a credential that lost permissions, where syncing would otherwise wipe legitimate data on the destination. ## Reviewing and applying a cached plan[​](#reviewing-and-applying-a-cached-plan "Direct link to Reviewing and applying a cached plan") The cache pattern lets you split a run into two steps: produce a plan (`diff`), then apply it (`apply`). This is useful when you want a human approval gate, when the destination is briefly unreachable, or when you want to re-apply the same plan without re-fetching the source. ``` # 1. Dry-run — extracts source + destination, writes plan.parquet infrahub-sync diff --name from-netbox --directory examples/ # Look at the logged line: # INFO | infrahub_sync.cli | Cached run 20260518T1430-abc12345 at .infrahub-sync-cache/from-netbox/20260518T1430-abc12345 # # Inspect the diff or query the parquet directly with DuckDB: # duckdb -c "SELECT * FROM read_parquet('.infrahub-sync-cache/from-netbox/20260518T1430-abc12345/plan.parquet')" # 2. Apply the cached plan — no source extraction infrahub-sync apply --name from-netbox --run-id 20260518T1430-abc12345 --directory examples/ ``` `apply` refuses to proceed if the destination's schema shape has drifted since the plan was built — the cached `schema-sub-hash.txt` must match the freshly-computed hash. When it doesn't, re-run `diff` to rebuild the plan. For the full on-disk layout (per-resource Parquet snapshots, sidecar JSON files, the per-pipeline filelock), see the [Cache layout reference](/sync/reference/cache-layout.md). ## Generating sync adapters and models[​](#generating-sync-adapters-and-models "Direct link to Generating sync adapters and models") `infrahub-sync generate` reads your configuration file and emits Python code for the sync adapters and models used at runtime. ``` infrahub-sync generate --name --directory ``` You typically only run this once per configuration (and after editing `config.yml`). --- # Sync NetBox to Infrahub This tutorial is for people getting started with Infrahub: it starts from a blank page, with no existing Infrahub instance or schema required. We will use Infrahub Sync to copy data from the [public NetBox demo](https://demo.netbox.dev) into Infrahub, installing and configuring everything we need along the way. This tutorial covers the fundamental work needed to get started; more advanced topics will be covered in subsequent guides. Already running Infrahub? If you already have an Infrahub instance running, you can skip straight to the [NetBox adapter documentation](/sync/adapters/netbox.md). By the end of this tutorial, you will know how to: * run Infrahub in a Docker container * install Infrahub Sync * load a schema into Infrahub * create a NetBox → Infrahub sync project * synchronize NetBox objects into Infrahub * review the import in a branch and open a proposed change **Time**: \~30 minutes **What you will build**: A running Infrahub instance with a production-grade schema covering the same core domains as NetBox (locations, devices, interfaces, IPAM, and organizations), populated with data synchronized from the public NetBox demo into a branch, ready to review as a proposed change. **Prerequisites**: * [Docker](https://docs.docker.com/get-docker/) installed and running (Docker Desktop or OrbStack) * [uv](https://docs.astral.sh/uv/getting-started/installation/) (Python package manager) * [Python 3.11+](https://www.python.org/downloads/) Public demo data The NetBox demo instance is public and resets regularly. Object counts, names, and sample data may differ from the examples in this tutorial. Because anyone can edit it, it can also contain malformed or unexpected data that breaks the sync — see [Troubleshooting](#troubleshooting) if you run into errors. ## Create a project[​](#create-a-project "Direct link to Create a project") [Copier](https://copier.readthedocs.io/) is a project scaffolding tool. Use it to create a new Infrahub project from the official template, which includes the standard file structure, task definitions, and a `schemas/` folder: 1. Run the following command to create a new project directory: ``` uv tool run --from 'copier' copier copy https://github.com/opsmill/infrahub-template infrahub-automation ``` When prompted, enter a project name (for example, `infrahub-automation`), then press **Enter** to accept the default for every remaining prompt (they all default to **No**). 2. Navigate to the project directory: ``` cd infrahub-automation ``` 3. Open the project in your IDE. If you have Visual Studio Code installed, you can run: ``` code . ``` Verification Run `ls` in the project directory. You should see files including `pyproject.toml`, `tasks.py`, and a `schemas/` folder. ## Start Infrahub[​](#start-infrahub "Direct link to Start Infrahub") The project template includes [Invoke](https://www.pyinvoke.org/) tasks that wrap Docker Compose commands. 1. Start all Infrahub services with a single command: ``` uv run invoke start ``` The first run takes a few minutes while Docker downloads the container images. 2. Open your browser and go to . 3. Log in from the bottom-left corner using the default credentials: * Username: `admin` * Password: `infrahub` Verification You should see the Infrahub web interface with a navigation menu on the left side. ## Install infrahub-sync[​](#install-infrahub-sync "Direct link to Install infrahub-sync") 1. Install the Python dependencies with uv: ``` uv add infrahub-sync pynetbox ``` This installs the `infrahub-sync` command. `pynetbox` is required by the NetBox adapter. 2. Verify infrahub-sync command is available: ``` uv run infrahub-sync --help ``` For more about supported adapters and their Python requirements, see the [NetBox adapter documentation](/sync/adapters/netbox.md). ## Load a schema into Infrahub[​](#load-a-schema-into-infrahub "Direct link to Load a schema into Infrahub") Infrahub stores data according to its schema. Before we can import any data, we need Infrahub to know the kinds of objects that the sync will create. We provide a production-grade Infrahub schema covering DCIM and IPAM features similar to NetBox's. It is not a one-to-one port of NetBox's own data model, and that's intentional. info This is a production-grade Infrahub schema, not a copy of NetBox's data model. Infrahub gives you a flexible graph model, so a real migration can preserve the parts of NetBox that matter to you while adapting the model to your own workflows. The schemas are published on the [Infrahub Marketplace](https://marketplace.infrahub.app/collections/infrahub/traditional-infrastructure-sot) as the `infrahub/traditional-infrastructure-sot` collection. Download it with `infrahubctl`, which the project template already installs: ``` uv run infrahubctl marketplace get infrahub/traditional-infrastructure-sot --collection ``` The command writes the collection's 16 schema files into `schemas/traditional-infrastructure-sot/`. It reads from the marketplace only, so your Infrahub instance does not need to be reachable yet. Verification Run `ls schemas/traditional-infrastructure-sot`. You should see files including `dcim.yml`, `location.yml`, `ipam.yml`, and `organization.yml`. Export the local Infrahub address and API token. Infrahub Sync will need them later to authenticate against your instance: ``` export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_API_TOKEN="06438eb2-8019-4776-878c-0941b1f1d1ec" ``` This is the default admin token that the project's Docker Compose stack seeds automatically, and it matches the default already configured in `infrahubctl.toml`. Do not reuse it for an internet-facing or shared Infrahub instance. Then load the schema into Infrahub using the project's `load-schema` task, which walks the `schemas/` folder recursively and picks up everything you just downloaded: ``` uv run invoke load-schema ``` Refresh [the local Infrahub web interface](http://localhost:8000). You should see additional schema objects in the left navigation. You can also open the schema view in the UI to explore the kinds and relationships that were loaded. ## Create a NetBox API token[​](#create-a-netbox-api-token "Direct link to Create a NetBox API token") Create a token in the public NetBox demo instance so Infrahub Sync can read data from NetBox. 1. Open the [public NetBox demo](https://demo.netbox.dev). 2. Log in with username `admin` and password `admin`. 3. Open your user profile. 4. Create an API token. 5. Copy the complete generated token value. Export the token locally: ``` export NETBOX_URL="https://demo.netbox.dev" export NETBOX_TOKEN="nbt_..." ``` Export the complete `nbt_...` token value. Do not include an authorization prefix such as `Bearer`, and do not copy only the short **Key** field. ## Create the sync project[​](#create-the-sync-project "Direct link to Create the sync project") A sync project is a directory containing a `config.yml` file. The configuration names the source and destination adapters, maps source fields to destination fields, and includes references that let Infrahub Sync compute write order. From your project directory (`infrahub-automation`), create a sync project directory and download the example NetBox to Infrahub configuration. ``` mkdir -p sync-projects/netbox-demo curl -L \ https://raw.githubusercontent.com/opsmill/infrahub-sync/refs/heads/main/examples/netbox_to_infrahub/config.yml \ -o sync-projects/netbox-demo/config.yml ``` The downloaded configuration is named `from-netbox`. It maps selected NetBox objects onto the Infrahub schema. Open `sync-projects/netbox-demo/config.yml` in an editor and scan the top-level keys: * `name` identifies the sync project. * `source` configures the NetBox adapter. * `destination` configures the Infrahub adapter. * `schema_mapping` defines how NetBox API resources become Infrahub objects. The configuration includes default endpoint values, but the environment variables exported above take precedence for tokens and URLs. The example configuration comments also mention `apply`, `--run-id`, and `--parallel`. This tutorial uses the beginner path: preview with `diff`, then write with `sync --diff`. See [Run a sync](/sync/running-a-sync.md) for cached plans and parallel execution. For a fuller explanation of this file, see [Create a sync project](/sync/creating-a-sync-project.md) and the [schema mapping reference](/sync/reference/schema-mapping.md). ## Generate the sync code[​](#generate-the-sync-code "Direct link to Generate the sync code") Generate the Python models and adapter code for this sync project. ``` uv run infrahub-sync generate --name from-netbox --directory sync-projects ``` The `generate` command reads the sync configuration and the destination schema, then writes the Python code used at runtime. Run `generate` again any time you edit `config.yml`. For more about the command flow, see [Run a sync](/sync/running-a-sync.md). ## Create a branch[​](#create-a-branch "Direct link to Create a branch") Infrahub tracks changes through branches, so you can review a batch of imported data as a proposed change before it lands on `main`. Create a branch to hold this import: ``` uv run infrahubctl branch create netbox-import ``` The rest of this tutorial runs `diff` and `sync` against this branch with `--branch netbox-import`, so the import can be reviewed before merging. For more about branches and proposed changes, see [Infrahub version control](https://docs.infrahub.app/topics/version-control/). ## Preview the changes[​](#preview-the-changes "Direct link to Preview the changes") Run a dry-run diff against the `netbox-import` branch before writing any data to Infrahub. ``` uv run infrahub-sync diff --name from-netbox --directory sync-projects --branch netbox-import ``` The `diff` command loads data from NetBox and Infrahub, compares both sides, and prints the planned changes. It also writes a cached plan under `.infrahub-sync-cache/from-netbox//plan.parquet`. Review the output before continuing. On a first run against an empty branch, most planned changes should be creates. Exact counts depend on the current public NetBox demo data. ## Sync the data[​](#sync-the-data "Direct link to Sync the data") After reviewing the diff, run the sync against the same branch. ``` uv run infrahub-sync sync --name from-netbox --directory sync-projects --branch netbox-import --diff ``` The `--diff` option prints the diff before applying the changes. The first sync can take a few minutes because it writes the imported objects and relationships into the `netbox-import` branch — `main` is untouched until you merge the resulting proposed change. Because this configuration omits `order:`, Infrahub Sync derives the write order from the mapping references. ## Verify the imported data[​](#verify-the-imported-data "Direct link to Verify the imported data") The imported data lives on the `netbox-import` branch, not on `main`. Open [the local Infrahub web interface](http://localhost:8000): 1. Use the branch selector in the top-left corner to switch from `main` to `netbox-import`. 2. Browse the left navigation for imported objects from the NetBox demo. Depending on the current demo data and the example mapping, you may see objects such as: * tags and organizations (manufacturers, providers, RIRs) * sites and racks * devices and interfaces * VRFs, VLANs, and VLAN groups * prefixes, IP addresses, and aggregates * circuits This is a minimal mapping Some records and relationships may be missing — that doesn't mean the sync failed. 3. Once you're happy with the data, open a proposed change from `netbox-import` toward `main` so the import can be reviewed before it's merged. If you switch back to `main`, none of this data is there yet — that's expected, since it's still isolated in the branch. ## What happened[​](#what-happened "Direct link to What happened") You used Infrahub Sync to move data from NetBox into Infrahub in a controlled sequence: 1. Infrahub provided the destination graph and schema. 2. NetBox provided the source data. 3. A branch isolated the import from `main` so it could be reviewed first. 4. `config.yml` described the adapters, field mappings, references, filters, and Transformations. 5. `generate` converted the configuration into runnable sync code. 6. `diff` compared the source and destination without writing changes. 7. `sync` applied the reviewed changes to the branch. The same pattern applies to larger migrations: start with a clear schema, map a small set of objects, condition source data where needed, review the diff inside a branch, and then synchronize before opening a proposed change toward `main`. ## Stop the local Infrahub instance[​](#stop-the-local-infrahub-instance "Direct link to Stop the local Infrahub instance") When you are finished, stop the Docker Compose stack. ``` uv run invoke stop ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### `infrahub-sync` fails with `Both url and token must be specified`[​](#infrahub-sync-fails-with-both-url-and-token-must-be-specified "Direct link to infrahub-sync-fails-with-both-url-and-token-must-be-specified") ``` ERROR | infrahub_sync.cli | Failed to initialize the Sync Instance: Error initializing InfrahubAdapter: Both url and token must be specified! ``` The NetBox and/or Infrahub adapter can't find credentials. `generate`, `diff`, `sync`, and `apply` all read them from the environment, not from `config.yml`, so make sure every variable is exported in the shell you're running the command from: ``` export NETBOX_URL="https://demo.netbox.dev" export NETBOX_TOKEN="" export INFRAHUB_ADDRESS="http://localhost:8000" export INFRAHUB_API_TOKEN="06438eb2-8019-4776-878c-0941b1f1d1ec" ``` This error names whichever adapter (NetBox or Infrahub) is missing its variables — the same message is raised for either one. ### `infrahub-sync` fails with `Object ... already present`[​](#infrahub-sync-fails-with-object--already-present "Direct link to infrahub-sync-fails-with-object--already-present") ``` ValueError: An error occurred while loading Netbox: ('Object 172.16.0.2/24__Alpha already present', IpamIPAddress "172.16.0.2/24__Alpha") ``` Two source objects map to the same identifier — here, two IP addresses sharing an address and VRF. Delete one of the duplicates in NetBox (**IPAM > IP Addresses**, search the address from the message), then re-run the command. ## Next steps[​](#next-steps "Direct link to Next steps") Now that you have completed a first sync, you have covered some of the basic objects in NetBox. This is only the first step. You will likely want to bring in the parts that are unique to your own NetBox instance (for example, roles or custom fields). Other guides will soon be available to cover: * How to sync locations/regions * How to deal with VLAN/Prefix/Device roles * How to cover custom attributes / relationships * Migrate configuration context --- # Use NetBox or Nautobot with Infrahub If you're adopting Infrahub but already have an established system of record in NetBox or Nautobot — with accumulated data, custom fields, and automation built around it — run Infrahub side-by-side with the existing tool and use Infrahub Sync to keep both systems holding the same data. This is the most common adoption pattern, whether you plan to eventually consolidate on Infrahub or keep both systems running long-term. ## The side-by-side pattern[​](#the-side-by-side-pattern "Direct link to The side-by-side pattern") Both tools continue to operate. The existing system (NetBox or Nautobot) remains the system of record for some data; Infrahub becomes the system of record for the rest. A sync project moves data between them on a defined cadence, so each system holds the data it owns plus a current view of what the other system owns. The side-by-side pattern fits when: * One team is ready to move to Infrahub, but other teams still depend on the existing system. * Existing automation pipelines, dashboards, or scripts read from NetBox or Nautobot and need time to migrate. * The existing system has features with no direct Infrahub equivalent (rack elevation, native cable tracing), and you want to keep using it for those workflows. * You want to evaluate Infrahub on production data. This pattern is also the typical starting point for a gradual migration. See [Migrate from NetBox or Nautobot](/sync/migrating-from-netbox-or-nautobot.md) for the migration-specific walkthrough. ## Choosing which tool owns which data[​](#choosing-which-tool-owns-which-data "Direct link to Choosing which tool owns which data") Decide which system is authoritative for each type of data before configuring a sync project. Sync behavior on each model is controlled by which side is the source and which is the destination, plus the `diffsync_flags` that govern what each run is allowed to change. Common patterns: * **The existing system is authoritative for everything during initial evaluation.** Sync runs NetBox → Infrahub or Nautobot → Infrahub. Infrahub mirrors the existing system; edits in Infrahub are not propagated back. This is the safest starting configuration. * **Infrahub is authoritative for newly modeled data; the existing system stays authoritative for existing data.** The sync project covers only the models that overlap. New data types (intent, design objects, custom schemas) live in Infrahub and are not synced. * **Each system owns specific models.** For example, the existing system owns rack and cable data; Infrahub owns device intent, IP allocations, and configuration data. Sync moves only the models each system needs to consume. Document the ownership decision in the sync project's configuration comments — anyone reading the YAML later needs to understand the boundary. ## Mapping data between the existing system and Infrahub[​](#mapping-data-between-the-existing-system-and-infrahub "Direct link to Mapping data between the existing system and Infrahub") NetBox and Nautobot each have a fixed schema. Infrahub has a custom schema that you define. There is no universal mapping — the YAML configuration depends on the specific Infrahub schema in use. For a typical mapping: * Start from the example configuration in the [Infrahub Sync repository](https://github.com/opsmill/infrahub-sync) under `examples/netbox_to_infrahub/` or `examples/nautobot_to_infrahub/`. The example references the Infrahub demo schema; replace its model and field names with the equivalents in your Infrahub schema. * For each source model you want to sync (sites, racks, devices, interfaces, IP addresses, VLANs, etc.), find the equivalent Infrahub model and map fields directly. * For field name differences (NetBox's `device_role` versus an Infrahub `role` attribute, or Nautobot's `status.name` versus NetBox's `status.value`), the schema mapping section of the YAML handles the translation. * For relationship differences, use `references` in the YAML to resolve them. See [Sync instance configuration](/sync/reference/config.md) for the full syntax. For a step-by-step guide on building the schema mapping itself, see [Schema mapping reference](/sync/reference/schema-mapping.md). ## Schema differences to be aware of[​](#schema-differences-to-be-aware-of "Direct link to Schema differences to be aware of") Some NetBox and Nautobot features have no direct Infrahub equivalent today. For each: keep using the existing system for that workflow, or model the capability in Infrahub with available workarounds. ### Rack elevation views[​](#rack-elevation-views "Direct link to Rack elevation views") NetBox and Nautobot both include a built-in rack elevation visualization. Infrahub does not have a native rack elevation view, and rack capacity calculation (free U positions, weight, power draw) is not automatic. If rack elevation is important to your workflow, the most direct path is to keep managing racks in the existing system and exclude rack data from the sync. Alternatively, model rack data in Infrahub with a custom schema and handle capacity calculation through a generator or GraphQL queries. ### Cable tracing[​](#cable-tracing "Direct link to Cable tracing") NetBox and Nautobot both include a cable tracing feature that follows connections through patch panels and intermediate devices. Infrahub does not have a single-click cable trace view, but the same data is queryable via Infrahub's graph traversal capabilities. For visualization, build a custom artifact or query with GraphQL from external tools. ### Other differences[​](#other-differences "Direct link to Other differences") For any feature where the existing system has a direct capability and Infrahub does not, document the decision in the sync project configuration and update internal runbooks. Keeping a NetBox or Nautobot workflow alongside Infrahub is a valid long-term choice. ## Operational considerations[​](#operational-considerations "Direct link to Operational considerations") * **Run direction matters for conflict handling.** When syncing NetBox → Infrahub or Nautobot → Infrahub, edits made in Infrahub to synced models will be overwritten on the next sync run unless flags are configured otherwise. Make sure everyone on the team understands which side is the source of truth for each model. * **Use `diff` regularly.** Before applying any sync — and especially during initial setup — run `infrahub-sync diff` to verify what would change. The diff is read-only. * **Schedule syncs on a cadence that matches data change rates.** If the source data changes hourly, sync hourly. If it changes daily, sync daily. Infrahub Sync has no built-in scheduler — use cron, CI, or a workflow tool. See [Schedule sync runs](/sync/orchestration.md). * **Set explicit ownership boundaries.** If both tools allow edits to the same model, conflicts will eventually occur. Decide upfront which tool is authoritative; avoid bi-directional syncing of the same model unless conflict resolution is well-defined. ## When to move toward full migration[​](#when-to-move-toward-full-migration "Direct link to When to move toward full migration") The side-by-side pattern is fine as a long-term steady state, but most teams eventually consolidate on Infrahub once it has proven itself and the gaps that kept the existing system in place have been addressed. Signals that you're ready to migrate fully: * All workflows that depended on NetBox-only or Nautobot-only features have been replaced or moved into Infrahub. * Existing automation pipelines have been updated to read from Infrahub. * Data ownership in Infrahub has been stable for a meaningful period — no surprises in what's there. At that point, see [Migrate from NetBox or Nautobot](/sync/migrating-from-netbox-or-nautobot.md) for the full migration walkthrough. ## Related[​](#related "Direct link to Related") * [Migrate from NetBox or Nautobot](/sync/migrating-from-netbox-or-nautobot.md) * [Create a sync project](/sync/creating-a-sync-project.md) * [Schema mapping reference](/sync/reference/schema-mapping.md) * [NetBox adapter](/sync/adapters/netbox.md) * [Nautobot adapter](/sync/adapters/nautobot.md) --- # VSCode Extension Welcome to the comprehensive documentation for the Infrahub VSCode Extension. This extension transforms Visual Studio Code into a powerful development environment for working with [Infrahub](https://docs.infrahub.app), the open-source infrastructure automation platform. ## What is the Infrahub VSCode extension?[​](#what-is-the-infrahub-vscode-extension "Direct link to What is the Infrahub VSCode extension?") The Infrahub VSCode Extension provides intelligent tooling that connects directly to your Infrahub servers, enabling you to develop infrastructure schemas, execute GraphQL queries, run Jinja2 and Python transforms, and manage branch-based workflows without leaving your IDE. It brings the power of Infrahub's graph database and version control directly into your development workflow. ## Documentation structure[​](#documentation-structure "Direct link to Documentation structure") This documentation is organized following the [Diataxis framework](https://diataxis.fr/) to help you find exactly what you need: ### 🎓 Tutorials[​](#-tutorials "Direct link to 🎓 Tutorials") **Learning-oriented** - Start here if you're new to the extension * [Getting Started with Infrahub VSCode Extension](/vscode/tutorials/getting-started.md) - Complete walkthrough from installation to first query ### 📋 How-to guides[​](#-how-to-guides "Direct link to 📋 How-to guides") **Task-oriented** - Practical steps for specific goals * [How to Configure Multiple Infrahub Servers](/vscode/guides/configure-multiple-servers.md) - Set up dev, staging, and production environments * [How to Execute GraphQL Queries](/vscode/guides/execute-graphql-queries.md) - Run queries with variables and branch selection * [How to Run Transforms and Artifacts](/vscode/guides/running-transforms.md) - Execute Jinja2 and Python transforms with automatic command selection * [How to Manage Branches](/vscode/guides/manage-branches.md) - Create, delete, and work with branches * [How to use Infrahub Snippets](/vscode/guides/snippets.md) - Insert and customize Infrahub YAML and automation snippets ### 💡 Topics[​](#-topics "Direct link to 💡 Topics") **Understanding-oriented** - Explanations and background * [Understanding the Extension Architecture](/vscode/topics/extension-architecture.md) - Deep dive into how the extension works * [Schema Validation and YAML Intelligence](/vscode/topics/schema-validation.md) - How intelligent editing and validation work ### 📚 Reference[​](#-reference "Direct link to 📚 Reference") **Information-oriented** - Technical descriptions and specifications * [Extension Commands and Settings Reference](/vscode/reference/commands-settings.md) - Complete list of commands, settings, and configuration options ## Quick start[​](#quick-start "Direct link to Quick start") If you want to get up and running quickly: 1. **Install the extension** from the VSCode Marketplace 2. **Configure a server** in your VSCode settings 3. **Create a schema** file in your project 4. **Execute a query** to test your connection For detailed instructions, see the [Getting Started tutorial](/vscode/tutorials/getting-started.md). ## Key features at a glance[​](#key-features-at-a-glance "Direct link to Key features at a glance") ### 🔗 Multi-server management[​](#-multi-server-management "Direct link to 🔗 Multi-server management") Connect to multiple Infrahub instances simultaneously, perfect for working across development, staging, and production environments. ### 🌳 Visual tree views[​](#-visual-tree-views "Direct link to 🌳 Visual tree views") Navigate your infrastructure configuration with intuitive tree views for servers, branches, and YAML structures. ### 📝 Intelligent Yaml editing[​](#-intelligent-yaml-editing "Direct link to 📝 Intelligent Yaml editing") Get real-time validation, auto-completion, and go-to-definition support for your Infrahub schemas. ### ✂️ Snippets for automation and YAML objects[​](#️-snippets-for-automation-and-yaml-objects "Direct link to ✂️ Snippets for automation and YAML objects") Quickly scaffold Infrahub transforms, scripts, generators, checks, and YAML objects using built-in VSCode snippets. ### 🚀 GraphQL integration[​](#-graphql-integration "Direct link to 🚀 GraphQL integration") Execute queries directly from VSCode with variable support and formatted results display. ### 🔀 Branch management[​](#-branch-management "Direct link to 🔀 Branch management") Create and manage branches for version-controlled infrastructure changes. ### 📊 Real-time status[​](#-real-time-status "Direct link to 📊 Real-time status") Monitor server connections with live status updates in the VSCode status bar. ## Use cases[​](#use-cases "Direct link to Use cases") The extension is designed for several key workflows: * **Schema Development**: Design and validate infrastructure models with immediate feedback * **Query Testing**: Develop and test GraphQL queries against live data * **Branch Workflows**: Manage infrastructure changes across multiple branches * **Multi-Environment Development**: Work seamlessly across different Infrahub environments * **Infrastructure Automation**: Build automation workflows with validated schemas ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before using the extension, ensure you have: * Visual Studio Code version 1.99.0 or higher * Access to at least one Infrahub server (local or remote) * Basic familiarity with YAML and GraphQL (helpful but not required) ## Getting help[​](#getting-help "Direct link to Getting help") ### Within the extension[​](#within-the-extension "Direct link to Within the extension") * **Problems Panel** (`Ctrl+Shift+M`): View validation errors and warnings * **Output Panel**: Check extension logs for debugging * **Command Palette** (`Ctrl+Shift+P`): Search for Infrahub commands ### External resources[​](#external-resources "Direct link to External resources") * **Infrahub Documentation**: [docs.infrahub.app](https://docs.infrahub.app) * **Extension Repository**: [GitHub.com/opsmill/infrahub-vscode](https://github.com/opsmill/infrahub-vscode) * **Issue Tracker**: [GitHub Issues](https://github.com/opsmill/infrahub-vscode/issues) * **Community Support**: [Infrahub Discord](https://discord.gg/infrahub) ## Common tasks[​](#common-tasks "Direct link to Common tasks") ### First time setup[​](#first-time-setup "Direct link to First time setup") Start with the [Getting Started tutorial](/vscode/tutorials/getting-started.md) for a complete walkthrough. ### Working with multiple environments[​](#working-with-multiple-environments "Direct link to Working with multiple environments") See [How to Configure Multiple Servers](/vscode/guides/configure-multiple-servers.md) for setting up dev, staging, and production. ### Testing queries[​](#testing-queries "Direct link to Testing queries") Learn query execution in [How to Execute GraphQL Queries](/vscode/guides/execute-graphql-queries.md). ### Understanding how it works[​](#understanding-how-it-works "Direct link to Understanding how it works") Explore [Understanding the Extension Architecture](/vscode/topics/extension-architecture.md) for technical details. ## Contributing[​](#contributing "Direct link to Contributing") We welcome contributions to both the extension and its documentation! Visit our [GitHub repository](https://github.com/opsmill/infrahub-vscode) to: * Report bugs or request features * Submit pull requests * Improve documentation ## Version information[​](#version-information "Direct link to Version information") * **Current Extension Version**: 0.0.2 * **Minimum VSCode Version**: 1.99.0 * **Recommended Infrahub Version**: 0.15.0+ ## License[​](#license "Direct link to License") The Infrahub VSCode Extension is part of the Infrahub project. See the [LICENSE](https://github.com/opsmill/infrahub-vscode/blob/main/LICENSE) file for details. *** *Built with ❤️ by the [OpsMill](https://opsmill.com) team and the Infrahub community.* --- # How to Configure Multiple Infrahub Servers If you need to work with multiple Infrahub environments (development, staging, production), this guide shows you how to configure and manage multiple server connections in the VSCode extension. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Infrahub VSCode extension installed * Access to multiple Infrahub servers * API tokens for each server (if authentication is required) ## Step 1: Open extension settings[​](#step-1-open-extension-settings "Direct link to Step 1: Open extension settings") Open your VSCode settings configuration: 1. Press `Ctrl+Shift+P` (or `Cmd+Shift+P` on macOS) 2. Type "Preferences: Open Settings (JSON)" 3. Press Enter ## Step 2: Add multiple server configurations[​](#step-2-add-multiple-server-configurations "Direct link to Step 2: Add multiple server configurations") Add your server configurations to the settings file: ``` { "infrahub-vscode.servers": [ { "name": "Development", "address": "http://localhost:8000", "api_token": "${env:INFRAHUB_DEV_TOKEN}" }, { "name": "Staging", "address": "https://staging.infrahub.example.com", "api_token": "${env:INFRAHUB_STAGING_TOKEN}" }, { "name": "Production", "address": "https://infrahub.example.com", "api_token": "${env:INFRAHUB_PROD_TOKEN}" } ] } ``` tip If you're using a development server with a self-signed certificate, you can disable certificate verification by adding `"tls_insecure": true` to the server configuration. **This should only be used in development environments.** ## Step 3: Set up environment variables[​](#step-3-set-up-environment-variables "Direct link to Step 3: Set up environment variables") For security, use environment variables for API tokens: ### On macOS and Linux[​](#on-macos-and-linux "Direct link to On macOS and Linux") Add to your shell configuration file (`~/.bashrc`, `~/.zshrc`, etc.): ``` export INFRAHUB_DEV_TOKEN="your-dev-token" export INFRAHUB_STAGING_TOKEN="your-staging-token" export INFRAHUB_PROD_TOKEN="your-prod-token" ``` Then reload your shell or restart VSCode. ### On windows[​](#on-windows "Direct link to On windows") Using PowerShell: ``` [System.Environment]::SetEnvironmentVariable('INFRAHUB_DEV_TOKEN', 'your-dev-token', 'User') [System.Environment]::SetEnvironmentVariable('INFRAHUB_STAGING_TOKEN', 'your-staging-token', 'User') [System.Environment]::SetEnvironmentVariable('INFRAHUB_PROD_TOKEN', 'your-prod-token', 'User') ``` Restart VSCode after setting environment variables. ## Step 4: Verify all connections[​](#step-4-verify-all-connections "Direct link to Step 4: Verify all connections") 1. Open the Infrahub Servers tree view in the Activity Bar 2. You should see all three servers listed 3. Expand each server to verify it can retrieve branches 4. Check the status bar - it shows the first server's connection status ## Step 5: Switch between servers for queries[​](#step-5-switch-between-servers-for-queries "Direct link to Step 5: Switch between servers for queries") When executing GraphQL queries: 1. Select the play icon next to the query 2. You'll be prompted to select a server from your configured list 3. Choose the appropriate environment 4. Select the branch to query against ## Advanced configuration[​](#advanced-configuration "Direct link to Advanced configuration") ### Using different schemas per environment[​](#using-different-schemas-per-environment "Direct link to Using different schemas per environment") Configure different schema directories for each environment: ``` { "infrahub-vscode.servers": [ { "name": "Development", "address": "http://localhost:8000", "api_token": "${env:INFRAHUB_DEV_TOKEN}" }, { "name": "Production", "address": "https://infrahub.example.com", "api_token": "${env:INFRAHUB_PROD_TOKEN}" } ] } ``` ### Server groups with naming conventions[​](#server-groups-with-naming-conventions "Direct link to Server groups with naming conventions") Use clear naming conventions to organize servers: ``` { "infrahub-vscode.servers": [ { "name": "[LOCAL] Development", "address": "http://localhost:8000" }, { "name": "[AWS] US-East-1 Staging", "address": "https://staging-us-east-1.infrahub.example.com", "api_token": "${env:INFRAHUB_AWS_STAGING_TOKEN}" }, { "name": "[AWS] US-East-1 Production", "address": "https://us-east-1.infrahub.example.com", "api_token": "${env:INFRAHUB_AWS_PROD_TOKEN}" }, { "name": "[Azure] Europe-West Production", "address": "https://eu-west.infrahub.example.com", "api_token": "${env:INFRAHUB_AZURE_PROD_TOKEN}" } ] } ``` ### Working with self-signed certificates[​](#working-with-self-signed-certificates "Direct link to Working with self-signed certificates") For development or testing environments using self-signed SSL certificates, you can disable certificate verification: ``` { "infrahub-vscode.servers": [ { "name": "Development (Self-Signed)", "address": "https://dev.infrahub.local", "api_token": "${env:INFRAHUB_DEV_TOKEN}", "tls_insecure": true } ] } ``` warning The `tls_insecure` option disables TLS certificate verification, which makes connections vulnerable to man-in-the-middle attacks. **Never use this option for production servers.** Only use it in controlled development environments with self-signed certificates. ## Validation[​](#validation "Direct link to Validation") To verify your configuration is working correctly: 1. **Check Status Bar**: The first server's status appears in the status bar 2. **Tree View**: All servers should appear in the Infrahub Servers tree view 3. **Branch Listing**: Expanding each server should show its branches 4. **Query Execution**: Test a query against each server ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Server not appearing in tree view[​](#server-not-appearing-in-tree-view "Direct link to Server not appearing in tree view") * Save your settings.json file * Reload VSCode window (`Ctrl+R` or `Cmd+R`) * Check for JSON syntax errors in settings ### Authentication failures[​](#authentication-failures "Direct link to Authentication failures") * Verify environment variables are set correctly: ``` echo $INFRAHUB_DEV_TOKEN # macOS/Linux echo %INFRAHUB_DEV_TOKEN% # Windows CMD ``` * Ensure tokens have necessary permissions * Check token expiration dates ### Connection timeouts[​](#connection-timeouts "Direct link to Connection timeouts") * Verify server URLs are accessible from your network * Check for proxy/firewall restrictions * Ensure VPN connection if required ### Certificate verification errors[​](#certificate-verification-errors "Direct link to Certificate verification errors") If you encounter certificate errors with development servers: * For self-signed certificates, add `"tls_insecure": true` to the server configuration * For production servers with certificate issues, fix the certificate rather than disabling verification * Check that your system's CA certificate store is up to date ## Related resources[​](#related-resources "Direct link to Related resources") * [How to Execute GraphQL Queries](/vscode/guides/execute-graphql-queries.md) * [How to Manage Branches](/vscode/guides/manage-branches.md) * [Security Configuration and Best Practices](/vscode/topics/security-configuration.md) * [Extension Commands Reference](/vscode/reference/commands-settings.md) --- # How to Execute GraphQL Queries If you want to test GraphQL queries against your Infrahub instance without leaving VSCode, this guide shows you how to write, organize, and execute queries using the extension. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Infrahub VSCode extension installed and configured * At least one Infrahub server configured * A workspace with `.infrahub.yml` file ## Step 1: Create query files[​](#step-1-create-query-files "Direct link to Step 1: Create query files") Create a directory for your GraphQL queries: ``` mkdir queries ``` Create a GraphQL query file `queries/get_devices.gql`: ``` query GetDevices($status: String = "active") { NetworkDevice(status__value: $status) { edges { node { id hostname { value } model { value } status { value } } } } } ``` ## Step 2: Register queries in .infrahub.yml[​](#step-2-register-queries-in-infrahubyml "Direct link to Step 2: Register queries in .infrahub.yml") Create or update your `.infrahub.yml` file to reference your queries: ``` --- queries: - name: get_active_devices file: queries/get_devices.gql - name: get_interfaces file: queries/get_interfaces.gql - name: topology_report file: queries/topology.gql ``` ## Step 3: Execute a query[​](#step-3-execute-a-query "Direct link to Step 3: Execute a query") 1. Open the Infrahub YAML tree view in the Activity Bar 2. Expand your `.infrahub.yml` file 3. Navigate to the queries section 4. Right-click on "get\_active\_devices" 5. Select **Execute GraphQL Query** 6. Choose your server when prompted 7. Select the branch (usually "main") 8. View results in the new panel that opens ## Step 4: Work with query variables[​](#step-4-work-with-query-variables "Direct link to Step 4: Work with query variables") For queries with variables, create `queries/get_devices_by_location.gql`: ``` query GetDevicesByLocation( $location: String!, $limit: Int = 10, $offset: Int = 0 ) { NetworkDevice( location__name__value: $location, limit: $limit, offset: $offset ) { edges { node { hostname { value } location { node { name { value } } } } } } } ``` When executing this query: 1. You'll be prompted for required variables (`$location`) 2. Optional variables (`$limit`, `$offset`) can be left empty to use defaults 3. Enter values in the input prompts that appear ## Step 5: Query different branches[​](#step-5-query-different-branches "Direct link to Step 5: Query different branches") To execute queries against specific branches: 1. Right-click on your query in the tree view 2. Select **Execute GraphQL Query** 3. Choose your server 4. You'll see a list of available branches 5. Select the branch you want to query 6. The query executes against that branch's data ## Step 6: Complex query examples[​](#step-6-complex-query-examples "Direct link to Step 6: Complex query examples") ### Query with nested relationships[​](#query-with-nested-relationships "Direct link to Query with nested relationships") ``` query GetDeviceDetails($device_id: String!) { NetworkDevice(id: $device_id) { edges { node { hostname { value } interfaces { edges { node { name { value } ip_addresses { edges { node { address { value } prefix { node { prefix { value } } } } } } } } } } } } } ``` ### Mutation example[​](#mutation-example "Direct link to Mutation example") ``` mutation CreateDevice( $hostname: String!, $model: String!, $location_id: String! ) { NetworkDeviceCreate( data: { hostname: { value: $hostname } model: { value: $model } location: { id: $location_id } } ) { ok object { id hostname { value } } } } ``` ## Step 7: Organize queries by category[​](#step-7-organize-queries-by-category "Direct link to Step 7: Organize queries by category") Structure your `.infrahub.yml` for better organization: ``` --- queries: # Inventory Queries - name: inventory_all_devices file: queries/inventory/all_devices.gql - name: inventory_by_location file: queries/inventory/by_location.gql # Network Topology - name: topology_layer2 file: queries/topology/layer2.gql - name: topology_layer3 file: queries/topology/layer3.gql # Reporting - name: report_capacity file: queries/reports/capacity.gql - name: report_compliance file: queries/reports/compliance.gql ``` ## Validation[​](#validation "Direct link to Validation") To verify your queries are working: 1. **Syntax Check**: The extension validates GraphQL syntax 2. **Test Execution**: Run each query with sample variables 3. **Branch Testing**: Execute against different branches 4. **Result Verification**: Check the output panel for expected data ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Using fragments[​](#using-fragments "Direct link to Using fragments") Create reusable fragments in `queries/fragments.gql`: ``` fragment DeviceBasicInfo on NetworkDevice { id hostname { value } model { value } status { value } } query GetDevicesWithFragment { NetworkDevice { edges { node { ...DeviceBasicInfo location { node { name { value } } } } } } } ``` ### Batch operations[​](#batch-operations "Direct link to Batch operations") For multiple related queries, create a batch file: ``` query BatchDeviceQuery( $location: String!, $status: String = "active" ) { devices: NetworkDevice( location__name__value: $location, status__value: $status ) { count edges { node { hostname { value } } } } interfaces: NetworkInterface( device__location__name__value: $location ) { count } locations: LocationSite( name__value: $location ) { edges { node { name { value } } } } } ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Query execution fails[​](#query-execution-fails "Direct link to Query execution fails") * Verify the GraphQL syntax is correct * Check that all required variables are provided * Ensure the schema types exist in your Infrahub instance * Verify you have permission to access the queried data ### Variables not prompting[​](#variables-not-prompting "Direct link to Variables not prompting") * Check variable definitions in your query * Required variables use `!` (for example, `$location: String!`) * Optional variables have defaults (for example, `$limit: Int = 10`) ### No results returned[​](#no-results-returned "Direct link to No results returned") * Verify data exists on the selected branch * Check filters in your query * Ensure proper permissions for the data ## Related resources[​](#related-resources "Direct link to Related resources") * [How to Manage Branches](/vscode/guides/manage-branches.md) * [Understanding Schema Validation](/vscode/topics/schema-validation.md) * [Extension Commands Reference](/vscode/reference/commands-settings.md) --- # How to Manage Branches If you need to work with Infrahub's branch-based version control system, this guide shows you how to create, delete, and manage branches using the VSCode extension. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Infrahub VSCode extension installed * At least one configured Infrahub server * Appropriate permissions to create/delete branches on your server ## Step 1: View existing branches[​](#step-1-view-existing-branches "Direct link to Step 1: View existing branches") To see all branches on a server: 1. Open the Infrahub icon in the Activity Bar 2. Expand the "Infrahub Servers" tree view 3. Click on your server to expand it 4. You'll see all branches listed with: * Branch name * "(default)" marker for the main branch * Branch metadata when available ## Step 2: Create a new branch[​](#step-2-create-a-new-branch "Direct link to Step 2: Create a new branch") ### From the tree view[​](#from-the-tree-view "Direct link to From the tree view") 1. Right-click on your server in the Infrahub Servers view 2. Select **New Branch** from the context menu 3. Enter the branch name (for example, "feature-network-update") 4. Optionally add a description 5. Press Enter to create the branch ### Naming conventions[​](#naming-conventions "Direct link to Naming conventions") Follow these branch naming patterns for clarity: * `feature-[description]` - New features * `fix-[issue-number]` - Bug fixes * `update-[component]` - Updates to existing components * `test-[scenario]` - Testing branches * `dev-[username]` - Personal development branches Example: `feature-datacenter-schema` ## Step 3: Delete a branch[​](#step-3-delete-a-branch "Direct link to Step 3: Delete a branch") To remove a branch you no longer need: 1. Expand your server in the tree view 2. Right-click on the branch to delete 3. Select **Delete Branch** 4. Confirm the deletion when prompted > **Warning**: Branch deletion is permanent. Ensure you've merged or saved any important changes before deleting. ## Step 4: Work with branch-specific data[​](#step-4-work-with-branch-specific-data "Direct link to Step 4: Work with branch-specific data") ### Execute queries on specific branches[​](#execute-queries-on-specific-branches "Direct link to Execute queries on specific branches") When running GraphQL queries: 1. Right-click on a query in the Infrahub YAML view 2. Select **Execute GraphQL Query** 3. Choose your server 4. Select the target branch from the list 5. The query runs against that branch's data ### Compare data across branches[​](#compare-data-across-branches "Direct link to Compare data across branches") To compare data between branches, execute the same query on different branches: ``` query BranchComparison { NetworkDevice { count edges { node { hostname { value } _updated_at } } } } ``` Run this query on: 1. Main branch - baseline data 2. Feature branch - modified data 3. Compare the results to see differences ## Step 5: Branch workflow examples[​](#step-5-branch-workflow-examples "Direct link to Step 5: Branch workflow examples") ### Feature development workflow[​](#feature-development-workflow "Direct link to Feature development workflow") 1. **Create Feature Branch** ``` Name: feature-add-firewall-schema Description: Adding firewall device schema and relationships ``` 2. **Develop and Test** * Modify schemas * Test queries against the branch * Validate changes 3. **Merge Process** * Review changes * Merge via Infrahub UI or API * Delete feature branch after merge ### Hotfix workflow[​](#hotfix-workflow "Direct link to Hotfix workflow") 1. **Create Hotfix Branch** ``` Name: fix-device-status-issue Description: Urgent fix for device status validation ``` 2. **Apply Fix** * Make necessary corrections * Test on hotfix branch 3. **Deploy** * Merge to main * Verify fix * Clean up branch ## Step 6: Advanced branch management[​](#step-6-advanced-branch-management "Direct link to Step 6: Advanced branch management") ### Working with long-lived branches[​](#working-with-long-lived-branches "Direct link to Working with long-lived branches") For branches that persist over time: ``` # Document branch purposes in your project # branches.yml branches: - name: develop purpose: Integration branch for ongoing development merge_target: main - name: staging purpose: Pre-production testing merge_target: main - name: feature-q4-updates purpose: Q4 feature development merge_target: develop ``` ### Branch metadata tracking[​](#branch-metadata-tracking "Direct link to Branch metadata tracking") Track branch information in your queries: ``` query BranchInfo { Branch { edges { node { name description created_at branched_from is_default has_schema_changes } } } } ``` ## Validation[​](#validation "Direct link to Validation") To ensure branch operations are working: 1. **Creation Verification**: New branch appears in tree view immediately 2. **Query Testing**: Execute a query against the new branch 3. **Deletion Confirmation**: Deleted branch disappears from tree view 4. **Refresh Check**: Tree view updates every 10 seconds automatically ## Best practices[​](#best-practices "Direct link to Best practices") ### Branch lifecycle[​](#branch-lifecycle "Direct link to Branch lifecycle") 1. **Create** with descriptive names 2. **Develop** in isolation 3. **Test** thoroughly on the branch 4. **Review** changes before merging 5. **Merge** when ready 6. **Delete** after successful merge ### Naming strategy[​](#naming-strategy "Direct link to Naming strategy") Use consistent prefixes: * `feature-` for new functionality * `fix-` for bug fixes * `update-` for updates * `test-` for experiments * `release-` for release preparation ### Documentation[​](#documentation "Direct link to Documentation") Document active branches: ``` ## Active Branches ### feature-network-redesign - **Created**: 2024-01-15 - **Owner**: Network Team - **Purpose**: Redesigning network schema for multi-vendor support - **Target Merge**: 2024-02-01 ### fix-validation-error - **Created**: 2024-01-20 - **Owner**: DevOps - **Purpose**: Fix schema validation for IP addresses - **Target Merge**: ASAP ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Branch creation fails[​](#branch-creation-fails "Direct link to Branch creation fails") * Check server permissions * Verify branch name doesn't already exist * Ensure valid characters in branch name (alphanumeric, hyphens, underscores) ### Branch not appearing[​](#branch-not-appearing "Direct link to Branch not appearing") * Wait 10 seconds for automatic refresh * Manually reload VSCode window if needed * Check server connectivity ### Cannot delete branch[​](#cannot-delete-branch "Direct link to Cannot delete branch") * Verify you have deletion permissions * Ensure branch isn't protected * Check if branch is the default branch (cannot be deleted) ## Related resources[​](#related-resources "Direct link to Related resources") * [How to Execute GraphQL Queries](/vscode/guides/execute-graphql-queries.md) * [Understanding the Extension Architecture](/vscode/topics/extension-architecture.md) * [Extension Commands Reference](/vscode/reference/commands-settings.md) --- # How to Run Transforms and Artifacts This guide shows you how to execute Infrahub transforms (both Jinja2 and Python) directly from VSCode. The extension automatically detects transform types and uses the appropriate `infrahubctl` command for execution. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Infrahub VSCode extension installed and configured * At least one Infrahub server configured * A workspace with `.infrahub.yml` file containing transforms and artifact definitions * `infrahubctl` CLI tool installed and available in your system PATH ## Understanding Transform Types[​](#understanding-transform-types "Direct link to Understanding Transform Types") The extension supports two types of transforms: * **Jinja2 Transforms**: Template-based transforms using Jinja2 syntax (executed with `infrahubctl render`) * **Python Transforms**: Code-based transforms using Python classes (executed with `infrahubctl transform`) ## Step 1: Configure transforms in .infrahub.yml[​](#step-1-configure-transforms-in-infrahubyml "Direct link to Step 1: Configure transforms in .infrahub.yml") ### Jinja2 Transforms[​](#jinja2-transforms "Direct link to Jinja2 Transforms") Define Jinja2 transforms in your `.infrahub.yml`: ``` jinja2_transforms: - name: topology_clab description: Template to generate a containerlab topology query: topology_simulator template_path: templates/clab_topology.j2 ``` ### Python Transforms[​](#python-transforms "Direct link to Python Transforms") Define Python transforms in your `.infrahub.yml`: ``` python_transforms: - name: leaf class_name: Leaf file_path: transforms/leaf.py - name: spine class_name: Spine file_path: transforms/spine.py - name: edge class_name: Edge file_path: transforms/edge.py ``` ### Artifact Definitions[​](#artifact-definitions "Direct link to Artifact Definitions") Define artifact definitions that reference your transforms: ``` artifact_definitions: - name: leaf_config artifact_name: leaf content_type: text/plain targets: leafs transformation: leaf # References python_transforms parameters: device: name__value - name: Containerlab Topology artifact_name: containerlab-topology content_type: text/plain targets: topologies_clab transformation: topology_clab # References jinja2_transforms parameters: name: name__value ``` ## Step 2: Execute transforms from VSCode[​](#step-2-execute-transforms-from-vscode "Direct link to Step 2: Execute transforms from VSCode") ### Method 1: From Artifact Definitions[​](#method-1-from-artifact-definitions "Direct link to Method 1: From Artifact Definitions") 1. Open the **Infrahub YAML** tree view in VSCode 2. Expand your `.infrahub.yml` file 3. Navigate to **artifact\_definitions** 4. You'll see each artifact with its transform type displayed in parentheses: * `leaf_config (python)` * `Containerlab Topology (jinja)` 5. Click the play icon next to the desired artifact ### Method 2: From Transform Definitions[​](#method-2-from-transform-definitions "Direct link to Method 2: From Transform Definitions") 1. In the **Infrahub YAML** tree view, navigate to: * **jinja2\_transforms** for Jinja2 templates, or * **python\_transforms** for Python transforms 2. Select the play icon to run the transform directly ### Transform Execution Process[​](#transform-execution-process "Direct link to Transform Execution Process") When you run a transform, the extension will: 1. **Auto-detect transform type**: The extension automatically determines whether to use `infrahubctl render` (Jinja2) or `infrahubctl transform` (Python) 2. **Prompt for branch selection**: Choose which Infrahub branch to execute against 3. **Collect transform variables**: Enter any required variables in `key=value` format: ``` site=nyc device=router01 environment=production ``` 4. **Execute the appropriate command**: * For Jinja2: `infrahubctl render topology_clab site=nyc --branch main` * For Python: `infrahubctl transform leaf device=router01 --branch main` ## Step 3: Working with transform variables[​](#step-3-working-with-transform-variables "Direct link to Step 3: Working with transform variables") ### Adding Variables[​](#adding-variables "Direct link to Adding Variables") When prompted for variables: 1. Enter each variable in `key=value` format 2. Press Enter to add another variable 3. Leave empty and press Enter to finish ### Variable Examples[​](#variable-examples "Direct link to Variable Examples") ``` # Network configuration site=atl01 rack=A12 vlan=100 # Device specifics device=spine01 role=spine asn=65001 # Environment settings environment=production region=us-east ``` ### Variable Validation[​](#variable-validation "Direct link to Variable Validation") The extension validates variable format: * ✅ `device=router01` (valid) * ✅ `site=nyc` (valid) * ❌ `device=` (invalid - empty value) * ❌ `=router01` (invalid - empty key) * ❌ `devicerouter01` (invalid - missing =) ## Step 4: Understanding command execution[​](#step-4-understanding-command-execution "Direct link to Step 4: Understanding command execution") ### Automatic Command Selection[​](#automatic-command-selection "Direct link to Automatic Command Selection") The extension intelligently chooses the correct `infrahubctl` command: | Transform Type | Command Used | Example | | -------------- | ----------------------- | --------------------------------------------------------- | | Jinja2 | `infrahubctl render` | `infrahubctl render topology_clab --branch main` | | Python | `infrahubctl transform` | `infrahubctl transform leaf device=spine01 --branch main` | ### Transform Type Detection[​](#transform-type-detection "Direct link to Transform Type Detection") The extension determines transform types by: 1. **For artifact definitions**: Looking up the `transformation` field in both `jinja2_transforms` and `python_transforms` sections 2. **For direct transforms**: Using the section they're defined in (`jinja2_transforms` vs `python_transforms`) ### Terminal Integration[​](#terminal-integration "Direct link to Terminal Integration") Commands execute in the VSCode integrated terminal, allowing you to: * See real-time output * Monitor progress * Debug any errors * Access command history ## Step 5: Example workflow[​](#step-5-example-workflow "Direct link to Step 5: Example workflow") Here's a complete example of setting up and running transforms: ### 1. Create directory structure[​](#1-create-directory-structure "Direct link to 1. Create directory structure") ``` mkdir -p transforms templates ``` ### 2. Define transforms in .infrahub.yml[​](#2-define-transforms-in-infrahubyml "Direct link to 2. Define transforms in .infrahub.yml") ``` --- jinja2_transforms: - name: device_config description: Generate device configuration query: device_query template_path: templates/device.j2 python_transforms: - name: topology_builder class_name: TopologyBuilder file_path: transforms/topology.py artifact_definitions: - name: router_config artifact_name: router-config content_type: text/plain targets: routers transformation: device_config # Jinja2 transform parameters: device: name__value - name: network_topology artifact_name: topology content_type: application/json targets: networks transformation: topology_builder # Python transform parameters: network: name__value ``` ### 3. Execute from VSCode[​](#3-execute-from-vscode "Direct link to 3. Execute from VSCode") 1. Navigate to **artifact\_definitions** → **router\_config (jinja)** 2) Click play icon 3) Select branch: `main` 4) Add variables: `device=router01`, `site=nyc` 5) Command executes: `infrahubctl render device_config device=router01 site=nyc --branch main` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Common Issues[​](#common-issues "Direct link to Common Issues") **"Transform type not determined"** * Verify the `transformation` field matches a name in `jinja2_transforms` or `python_transforms` * Check YAML syntax and indentation **"infrahubctl command not found"** * Ensure `infrahubctl` is installed: `pip install infrahubctl` * Verify it's in your system PATH * Restart VSCode after installation **"No transform selected"** * Ensure your artifact definition has a valid `transformation` field * Verify the referenced transform exists in your configuration **Transform execution fails** * Check the terminal output for specific error messages * Verify branch exists and is accessible * Ensure required variables are provided * Check transform syntax (Jinja2 templates or Python code) ### Best Practices[​](#best-practices "Direct link to Best Practices") 1. **Organize transforms logically**: Group related transforms in clearly named sections 2. **Use descriptive names**: Transform names should clearly indicate their purpose 3. **Document variables**: Add comments in your `.infrahub.yml` describing expected variables 4. **Test incrementally**: Start with simple transforms and add complexity gradually 5. **Version control**: Keep transform files and `.infrahub.yml` in version control ## Next Steps[​](#next-steps "Direct link to Next Steps") * **[Managing Branches](/vscode/guides/manage-branches.md)**: Learn how to work with different Infrahub branches * **[Configure Multiple Servers](/vscode/guides/configure-multiple-servers.md)**: Set up development, staging, and production environments * **[Extension Commands Reference](/vscode/reference/commands-settings.md)**: Complete list of available commands ## Further Resources[​](#further-resources "Direct link to Further Resources") * [Infrahub Transforms Documentation](https://docs.infrahub.app/topics/transformation) * [infrahubctl CLI Reference](https://docs.infrahub.app/reference/infrahub-cli) * [Jinja2 Template Documentation](https://jinja.palletsprojects.com/) --- # How to use Infrahub snippets in VSCode This guide shows you how to quickly insert and customize Infrahub YAML objects and automation scripts using built-in snippets in Visual Studio Code. By following these steps, you’ll save time and reduce errors when authoring Infrahub resources. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Infrahub VSCode extension installed * Workspace containing YAML or Python files * Basic familiarity with editing files in VSCode ## Steps[​](#steps "Direct link to Steps") ### 1. Insert an Infrahub object snippet[​](#1-insert-an-infrahub-object-snippet "Direct link to 1. Insert an Infrahub object snippet") 1. Open any `.yaml` or `.yml` file in your project. 2. Type `infrahubobject` and select the snippet from the suggestion list. 3. The following template will be inserted: ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: ${1:kind} data: - name: "${2:name}" ``` 4. Replace the placeholder values (`kind`, `name`) with your desired values. ### 2. Insert a Infrahub Python Snippet[​](#2-insert-a-infrahub-python-snippet "Direct link to 2. Insert a Infrahub Python Snippet") 1. Open a `.py` file in your workspace. 2. Type one of the following snippet prefixes and select it from the suggestion list: * `infrahubtransform` for a transform * `infrahubscript` for a script * `infrahubgenerator` for a generator * `infrahubcheck` for a check 3. Fill in the placeholders as needed to scaffold your automation script. ## Related resources[​](#related-resources "Direct link to Related resources") * [How to Configure Multiple Servers](/vscode/guides/configure-multiple-servers.md) * [How to Execute GraphQL Queries](/vscode/guides/execute-graphql-queries.md) --- # How to Visualize Your Schema If you want to explore and understand the structure of your Infrahub schema, this guide shows you how to use the Schema Visualizer to view nodes, relationships, and attributes in an interactive graph. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Infrahub VSCode extension installed and configured * At least one Infrahub server configured and online * Network access to your Infrahub instance ## Step 1: Open the Schema Visualizer[​](#step-1-open-the-schema-visualizer "Direct link to Step 1: Open the Schema Visualizer") 1. Open the **Infrahub Servers** panel in the Activity Bar 2. Locate your connected server (indicated by a green status dot) 3. Click the graph icon next to the server name ![Infrahub Server Tree with Graph Icon](/assets/images/infrahub-server-tree-graph-e9daa17f9434791d44bef34236681290.png) Alternatively, right-click on the server and select **Visualize Schema** from the context menu. ## Step 2: Select a branch[​](#step-2-select-a-branch "Direct link to Step 2: Select a branch") When prompted, select the branch whose schema you want to visualize. The extension fetches the schema directly from the server for the selected branch. ## Step 3: Explore the schema graph[​](#step-3-explore-the-schema-graph "Direct link to Step 3: Explore the schema graph") The visualizer displays your schema as an interactive graph with nodes representing schema types and edges representing relationships. ![Schema Visualizer Main View](/assets/images/infrahub-graph-b80d49e589dc544429944cd11d0d1f37.png) The **Schema Overview** panel in the top-left shows statistics: * **Visible**: Number of schemas currently displayed * **Total**: Total schemas in the branch * **Nodes**: Regular schema nodes * **Profiles**: Profile configurations * **Templates**: Template definitions * **Generics**: Generic/reusable types ### Navigate the graph[​](#navigate-the-graph "Direct link to Navigate the graph") * **Pan**: Click and drag on the background * **Zoom**: Use mouse scroll or the zoom controls in the bottom toolbar * **Move nodes**: Click and drag individual schema nodes * **Select node**: Click on a node to view its details ## Step 4: Filter schemas[​](#step-4-filter-schemas "Direct link to Step 4: Filter schemas") Click the filter icon in the bottom toolbar to open the Filter Schemas panel. ![Filter Panel](/assets/images/infrahub-filter-panel-6e10589ec24e6830d665ce493024b8f0.png) The filter panel provides: * **Search**: Type to find schemas by name, label, or kind * **Namespace toggles**: Show or hide entire namespaces * **Individual toggles**: Enable or disable specific nodes By default, the visualizer hides schemas from the Core and Builtin namespaces to reduce visual clutter. Enable them in the filter panel if needed. ## Step 5: View node details[​](#step-5-view-node-details "Direct link to Step 5: View node details") Click on any schema node to open the details panel on the right side. ![Node Details Panel](/assets/images/infrahub-graph-node-detail-a7e773143f278158cb543563f2b51294.png) The details panel shows: ### Properties[​](#properties "Direct link to Properties") * **Namespace**: The schema's namespace * **Name**: The schema name * **Kind**: Full identifier (Namespace + Name) * **Description**: What this schema represents * **Inherit from**: Parent schemas this type inherits from ### Attributes[​](#attributes "Direct link to Attributes") Lists all attributes with their: * Name and type (Text, Number, Boolean, Dropdown) * Flags: optional, unique, read-only * Inheritance status (inherited attributes shown with "inherited" label) ### Relationships[​](#relationships "Direct link to Relationships") Lists all relationships with: * Relationship name * Cardinality: `one` or `many` * Target schema type * Inheritance status ## Step 6: Use the toolbar controls[​](#step-6-use-the-toolbar-controls "Direct link to Step 6: Use the toolbar controls") The bottom toolbar provides visualization controls: ![Bottom Toolbar](/assets/images/infrahub-bottom-menu-bar-8f6aeee413fcf312c53367525a712645.png) From left to right: | Control | Function | | ------------------ | ------------------------------------------------------ | | **-** | Zoom out | | **Fit** | Fit all nodes in view | | **+** | Zoom in | | **Smooth/Stepped** | Toggle edge style between curved and right-angle lines | | **→** | Auto-layout horizontally (left to right) | | **↓** | Auto-layout vertically (top to bottom) | | **Filter** | Toggle the filter panel | | **Reset** | Reset view to default state | | **Export** | Download as PNG or SVG | ## Step 7: Export the visualization[​](#step-7-export-the-visualization "Direct link to Step 7: Export the visualization") To save your schema visualization: 1. Arrange the graph as desired 2. Click the export button (download icon) in the toolbar 3. Choose PNG or SVG format 4. The image downloads with your current view Exported images are useful for documentation, architecture reviews, and sharing schema designs with team members. ## Validation[​](#validation "Direct link to Validation") To verify the visualizer is working correctly: 1. **Connection check**: Ensure your server shows a green status indicator 2. **Branch selection**: Verify you can see and select branches 3. **Schema loading**: The overview panel should show non-zero counts 4. **Interaction**: Click nodes to confirm the details panel opens ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Visualize Schema option is disabled[​](#visualize-schema-option-is-disabled "Direct link to Visualize Schema option is disabled") * The server must be online (green status indicator) * Check your network connection to the Infrahub instance * Verify server credentials in extension settings ### Schema shows zero nodes[​](#schema-shows-zero-nodes "Direct link to Schema shows zero nodes") * The selected branch may have no custom schemas defined * Enable Core and Builtin namespaces in the filter panel * Check if the branch exists and has schema data ### Graph is too cluttered[​](#graph-is-too-cluttered "Direct link to Graph is too cluttered") * Use the filter panel to hide unnecessary namespaces * Use auto-layout to organize nodes automatically * Zoom out to see the full structure ### Export produces blank image[​](#export-produces-blank-image "Direct link to Export produces blank image") * Ensure nodes are visible in the current view * Try using "Fit" before exporting * Check that your browser allows downloads ## Related resources[​](#related-resources "Direct link to Related resources") * [How to Manage Branches](/vscode/guides/manage-branches.md) * [Understanding Schema Validation](/vscode/topics/schema-validation.md) * [Extension Commands Reference](/vscode/reference/commands-settings.md) --- # Extension Commands and Settings Reference This reference document provides comprehensive information about all available commands, settings, keyboard shortcuts, and configuration options in the Infrahub VSCode extension. ## Commands[​](#commands "Direct link to Commands") ### Available Commands[​](#available-commands "Direct link to Available Commands") The extension registers the following commands that can be executed via the Command Palette (`Ctrl+Shift+P` / `Cmd+Shift+P`): | Command ID | Title | Description | Context | | ------------------------------ | --------------------- | ---------------------------------------------------------------------- | ---------------------------- | | `infrahub.editInfrahubYaml` | Edit file | Opens the selected YAML file at a specific location | Tree view item | | `infrahub.editGqlQuery` | Edit GraphQL Query | Opens the GraphQL query file for editing | Query tree item | | `infrahub.executeGraphQLQuery` | Execute GraphQL Query | Runs a GraphQL query against selected server/branch | Query tree item | | `infrahub.runTransform` | Run Transform | Executes a Jinja2 or Python transform with automatic command selection | Transform/Artifact tree item | | `infrahub.newBranch` | New Branch | Creates a new branch on the selected server | Server tree item | | `infrahub.deleteBranch` | Delete Branch | Deletes the selected branch | Branch tree item | | `infrahub.visualizeSchema` | Visualize Schema | Opens an interactive graph visualization of the server's schema | Server tree item (online) | ### Command Execution[​](#command-execution "Direct link to Command Execution") #### From Command Palette[​](#from-command-palette "Direct link to From Command Palette") 1. Press `Ctrl+Shift+P` (Windows/Linux) or `Cmd+Shift+P` (macOS) 2. Type "Infrahub" to filter commands 3. Select the desired command #### From Context Menus[​](#from-context-menus "Direct link to From Context Menus") Right-click on tree view items to access contextual commands: * **Server Items**: New Branch, Visualize Schema * **Branch Items**: Delete Branch * **Query Items**: Execute Query, Edit Query * **Transform/Artifact Items**: Run Transform (automatically detects Jinja2 vs Python) * **YAML Items**: Edit File #### Programmatic Execution[​](#programmatic-execution "Direct link to Programmatic Execution") Commands can be executed programmatically: ``` // Execute a command from another extension vscode.commands.executeCommand('infrahub.executeGraphQLQuery', queryItem); ``` ## Extension Settings[​](#extension-settings "Direct link to Extension Settings") ### Configuration Properties[​](#configuration-properties "Direct link to Configuration Properties") All settings are prefixed with `infrahub-vscode.`: | Setting | Type | Default | Description | | ----------------- | ------ | ----------- | ----------------------------------------- | | `servers` | array | `[]` | List of Infrahub server configurations | | `schemaDirectory` | string | `"schemas"` | Path to directory containing schema files | ### Server Configuration Schema[​](#server-configuration-schema "Direct link to Server Configuration Schema") Each server in the `servers` array follows this structure: ``` interface ServerConfig { name: string; // Display name for the server address: string; // Server URL (http/https) api_token?: string; // Optional API token for authentication tls_insecure?: boolean; // Optional: Disable TLS certificate verification (default: false) } ``` #### TLS configuration details[​](#tls-configuration-details "Direct link to TLS configuration details") The `tls_insecure` property controls TLS certificate verification behavior: * **Default value**: `false` (secure, certificates are verified) * **When `true`**: Disables certificate verification for development environments * **Scope**: Affects all HTTPS connections when any server has this enabled * **Security impact**: Makes connections vulnerable to man-in-the-middle attacks **Use cases for `tls_insecure: true`**: * Development servers with self-signed certificates * Internal testing environments with custom CA certificates * Docker containers with self-signed certificates **Never use in production** as it compromises security. ### Example Configuration[​](#example-configuration "Direct link to Example Configuration") ``` { "infrahub-vscode.servers": [ { "name": "Local Development", "address": "http://localhost:8000" }, { "name": "Production", "address": "https://infrahub.example.com", "api_token": "inf_1234567890abcdef" }, { "name": "Development (Self-Signed)", "address": "https://dev.infrahub.local", "api_token": "inf_dev_token", "tls_insecure": true } ], "infrahub-vscode.schemaDirectory": "infrastructure/schemas" } ``` warning Setting `tls_insecure: true` disables certificate verification and is **not recommended for production environments**. Only use this option in development/testing environments with self-signed certificates. ### Environment Variable Substitution[​](#environment-variable-substitution "Direct link to Environment Variable Substitution") Settings support environment variable substitution using `${env:VARIABLE_NAME}`: ``` { "infrahub-vscode.servers": [ { "name": "Production", "address": "${env:INFRAHUB_SERVER_URL}", "api_token": "${env:INFRAHUB_API_TOKEN}" } ] } ``` ## Tree View Components[​](#tree-view-components "Direct link to Tree View Components") ### Infrahub Servers Tree View[​](#infrahub-servers-tree-view "Direct link to Infrahub Servers Tree View") **View ID**: `InfrahubServerTreeView` **Location**: Activity Bar → Infrahub Container #### Tree Structure[​](#tree-structure "Direct link to Tree Structure") ``` 📁 Infrahub Servers └─ 🖥️ [Server Name] ├─ 🌿 main (default) ├─ 🌿 feature-branch-1 └─ 🌿 feature-branch-2 ``` #### Item Types[​](#item-types "Direct link to Item Types") | Item Type | View Item ID | Available Actions | | ---------------- | ------------------------ | ---------------------------- | | Server (online) | `infrahubServer:online` | New Branch, Visualize Schema | | Server (offline) | `infrahubServer:offline` | New Branch | | Branch (default) | `infrahubBranch-default` | Delete Branch | | Branch (regular) | `infrahubBranch` | Delete Branch | ### Infrahub YAML Tree View[​](#infrahub-yaml-tree-view "Direct link to Infrahub YAML Tree View") **View ID**: `infrahubYamlTreeView` **Location**: Activity Bar → Infrahub Container #### Tree Structure[​](#tree-structure-1 "Direct link to Tree Structure") ``` 📁 Infrahub YAML └─ 📄 .infrahub.yml ├─ 📁 schemas │ ├─ 📄 network.yml │ └─ 📄 location.yml ├─ 📁 queries │ ├─ 📊 get_devices │ └─ 📊 topology_report └─ 📁 checks └─ 🔍 validate_hostnames ``` #### Item Types[​](#item-types-1 "Direct link to Item Types") | Item Type | View Item ID Pattern | Available Actions | | --------- | -------------------- | ------------------------- | | YAML File | `file` | Edit File | | Query | `queries/*` | Execute Query, Edit Query | | Schema | `schemas/*` | Edit File | | Check | `checks/*` | Edit File | ## Status Bar[​](#status-bar "Direct link to Status Bar") ### Status Bar Item[​](#status-bar-item "Direct link to Status Bar Item") **Alignment**: Left **Priority**: 100 #### Status States[​](#status-states "Direct link to Status States") | State | Display Text | Background Color | | ----------- | --------------------------------- | -------------------- | | Connected | `Infrahub: v[version] ([server])` | Default | | No Server | `Infrahub: No server set` | No folder background | | Unreachable | `Infrahub: Server unreachable` | Error background | #### Update Interval[​](#update-interval "Direct link to Update Interval") * Status updates every 10 seconds * Displays first configured server's status ## Activation Events[​](#activation-events "Direct link to Activation Events") The extension activates when: ``` "activationEvents": [ "workspaceContains:.infrahub.yml", "workspaceContains:.infrahub.yaml" ] ``` ## File Associations[​](#file-associations "Direct link to File Associations") ### YAML Validation[​](#yaml-validation "Direct link to YAML Validation") Files matching these patterns receive schema validation: ``` "yamlValidation": [ { "fileMatch": [ "models/**/*.yml", "models/**/*.yaml", "schemas/**/*.yml", "schemas/**/*.yaml" ], "url": "https://schema.infrahub.app/infrahub/schema/latest.json" } ] ``` ## Language Features[​](#language-features "Direct link to Language Features") ### Definition Provider[​](#definition-provider "Direct link to Definition Provider") **Language**: YAML **Feature**: Go-to-definition (`F12` or `Ctrl+Click`) Navigates to: * Schema definitions * Referenced nodes * Relationship targets ### Document Symbol Provider[​](#document-symbol-provider "Direct link to Document Symbol Provider") **Language**: YAML **Feature**: Document outline Provides symbols for: * Nodes * Attributes * Relationships * Queries * Checks ## Context Menu Integration[​](#context-menu-integration "Direct link to Context Menu Integration") ### View Item Context Menus[​](#view-item-context-menus "Direct link to View Item Context Menus") Commands appear in context menus based on `when` clauses: ``` "menus": { "view/item/context": [ { "command": "infrahub.newBranch", "when": "view == InfrahubServerTreeView && viewItem == infrahubServer" }, { "command": "infrahub.deleteBranch", "when": "view == InfrahubServerTreeView && viewItem =~ /^infrahubBranch/" } ] } ``` ## Extension Dependencies[​](#extension-dependencies "Direct link to Extension Dependencies") ### Required Extensions[​](#required-extensions "Direct link to Required Extensions") ``` "extensionDependencies": [ "redhat.vscode-yaml" // YAML language support ] ``` The Red Hat YAML extension must be installed for schema validation to work. ## API Token Management[​](#api-token-management "Direct link to API Token Management") ### Token Format[​](#token-format "Direct link to Token Format") Infrahub API tokens typically follow this format: ``` inf_[32-character-alphanumeric-string] ``` ### Token Security Best Practices[​](#token-security-best-practices "Direct link to Token Security Best Practices") 1. **Never commit tokens**: Use environment variables 2. **Rotate regularly**: Change tokens periodically 3. **Minimal permissions**: Use read-only tokens when possible 4. **Separate environments**: Different tokens per environment ### Token Configuration Methods[​](#token-configuration-methods "Direct link to Token Configuration Methods") #### Method 1: Direct in settings (not recommended)[​](#method-1-direct-in-settings-not-recommended "Direct link to Method 1: Direct in settings (not recommended)") ``` { "infrahub-vscode.servers": [ { "api_token": "inf_direct_token_not_secure" } ] } ``` #### Method 2: Environment variables (recommended)[​](#method-2-environment-variables-recommended "Direct link to Method 2: Environment variables (recommended)") ``` { "infrahub-vscode.servers": [ { "api_token": "${env:INFRAHUB_TOKEN}" } ] } ``` #### Method 3: VSCode secrets (future)[​](#method-3-vscode-secrets-future "Direct link to Method 3: VSCode secrets (future)") Planned support for VSCode's secret storage API. ## Troubleshooting Reference[​](#troubleshooting-reference "Direct link to Troubleshooting Reference") ### Common Issues and Solutions[​](#common-issues-and-solutions "Direct link to Common Issues and Solutions") | Issue | Possible Cause | Solution | | ----------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------- | | Extension not activating | No `.infrahub.yml` file | Create `.infrahub.yml` in workspace root | | Server unreachable | Invalid URL or network issue | Check server URL and network connection | | Commands not appearing | Wrong context | Ensure correct tree view item is selected | | Validation not working | Missing YAML extension | Install Red Hat YAML extension | | Token not working | Incorrect format or permissions | Verify token format and permissions | | TLS certificate errors | Self-signed or invalid certificates | Add `"tls_insecure": true` for development servers | | CERT\_HAS\_EXPIRED | Expired SSL certificate | Renew certificate or use `tls_insecure` for dev | | SELF\_SIGNED\_CERT\_IN\_CHAIN | Self-signed certificate | Use `"tls_insecure": true` for development | ### TLS error messages[​](#tls-error-messages "Direct link to TLS error messages") The extension provides specific error messages for common TLS issues: * **"TLS Certificate expired - check tls\_insecure setting"**: The server's certificate has expired * **"Self-signed certificate - check tls\_insecure setting"**: The server uses a self-signed certificate * **"TLS Verification failed - check tls\_insecure setting"**: General certificate verification failure These messages appear in the server tree view when connection attempts fail due to certificate issues. ### Debug Output[​](#debug-output "Direct link to Debug Output") Enable debug logging: 1. Open VSCode Developer Tools: `Help → Toggle Developer Tools` 2. Check Console tab for extension logs 3. Look for messages starting with "Infrahub Extension" ## Keyboard Shortcuts[​](#keyboard-shortcuts "Direct link to Keyboard Shortcuts") The extension doesn't define default keyboard shortcuts, but you can add custom ones: ### Adding Custom Shortcuts[​](#adding-custom-shortcuts "Direct link to Adding Custom Shortcuts") 1. Open Keyboard Shortcuts: `Ctrl+K Ctrl+S` (Windows/Linux) or `Cmd+K Cmd+S` (macOS) 2. Search for "Infrahub" 3. Click the `+` icon to add a keybinding ### Suggested Shortcuts[​](#suggested-shortcuts "Direct link to Suggested Shortcuts") ``` { "key": "ctrl+alt+q", "command": "infrahub.executeGraphQLQuery", "when": "view == infrahubYamlTreeView" }, { "key": "ctrl+alt+b", "command": "infrahub.newBranch", "when": "view == InfrahubServerTreeView" } ``` ## Performance Settings[​](#performance-settings "Direct link to Performance Settings") ### Refresh Intervals[​](#refresh-intervals "Direct link to Refresh Intervals") | Component | Interval | Configurable | | ---------------- | ---------- | -------------- | | Server Tree View | 10 seconds | No (hardcoded) | | Status Bar | 10 seconds | No (hardcoded) | ### Resource Limits[​](#resource-limits "Direct link to Resource Limits") * Maximum servers: No limit (performance may degrade with >10) * Query result size: Limited by VSCode webview memory * Tree view items: No hard limit ## Version Compatibility[​](#version-compatibility "Direct link to Version Compatibility") ### VSCode Version[​](#vscode-version "Direct link to VSCode Version") * **Minimum**: 1.99.0 * **Recommended**: Latest stable version ### Infrahub Server Version[​](#infrahub-server-version "Direct link to Infrahub Server Version") * **Minimum**: 0.14.0 (basic functionality) * **Recommended**: 0.15.0+ (all features) ### Node.js Runtime[​](#nodejs-runtime "Direct link to Node.js Runtime") * **Target**: ES2022 * **Module System**: Node16 ## Extension Metadata[​](#extension-metadata "Direct link to Extension Metadata") ### Publisher Information[​](#publisher-information "Direct link to Publisher Information") * **Publisher ID**: OpsMill * **Publisher Name**: OpsMill * **Homepage**: ### Gallery Banner[​](#gallery-banner "Direct link to Gallery Banner") ``` "galleryBanner": { "color": "#2183F7", "theme": "dark" } ``` ## Further Information[​](#further-information "Direct link to Further Information") * [Getting Started Tutorial](/vscode/tutorials/getting-started.md) * [How to Visualize Your Schema](/vscode/guides/visualize-schema.md) * [Understanding the Extension Architecture](/vscode/topics/extension-architecture.md) * [Security Configuration and Best Practices](/vscode/topics/security-configuration.md) * [GitHub Repository](https://github.com/opsmill/infrahub-vscode) --- # Understanding the Extension Architecture This document explains the architecture of the Infrahub VSCode extension, its design decisions, and how the various components interact to provide a seamless development experience for infrastructure automation. ## Overview[​](#overview "Direct link to Overview") The Infrahub VSCode extension follows a modular architecture pattern that separates concerns into distinct components. This design enables maintainability, testability, and extensibility while providing real-time connectivity to Infrahub servers. ## Core architecture components[​](#core-architecture-components "Direct link to Core architecture components") ### Extension activation[​](#extension-activation "Direct link to Extension activation") The extension activates when VSCode detects specific conditions in your workspace: ``` "activationEvents": [ "workspaceContains:.infrahub.yml", "workspaceContains:.infrahub.yaml" ] ``` This lazy loading approach ensures the extension only consumes resources when you're working with Infrahub projects. The activation triggers when VSCode finds an `.infrahub.yml` or `.infrahub.yaml` file in your workspace, signaling that this is an Infrahub-enabled project. ### Component hierarchy[​](#component-hierarchy "Direct link to Component hierarchy") The extension architecture consists of several interconnected layers: ``` ┌─────────────────────────────────────┐ │ VSCode Extension API │ ├─────────────────────────────────────┤ │ Extension Entry Point │ │ (extension.ts) │ ├─────────────────────────────────────┤ │ Tree View Providers │ Language │ │ │ Providers │ ├─────────────────────┼────────────────┤ │ Commands │ Utilities │ ├─────────────────────┴────────────────┤ │ Infrahub SDK Client │ ├─────────────────────────────────────┤ │ Infrahub Server (Remote) │ └─────────────────────────────────────┘ ``` ## Key architectural patterns[​](#key-architectural-patterns "Direct link to Key architectural patterns") ### Provider pattern[​](#provider-pattern "Direct link to Provider pattern") The extension uses VSCode's provider pattern extensively. Providers are classes that implement specific interfaces to extend VSCode's functionality: #### Tree data providers[​](#tree-data-providers "Direct link to Tree data providers") Tree data providers manage the hierarchical views in the sidebar: * **InfrahubServerTreeViewProvider**: Manages server connections and branch listings * **InfrahubYamlTreeViewProvider**: Parses and displays `.infrahub.yml` structure These providers implement the `TreeDataProvider` interface, which requires: * `getTreeItem()`: Returns the UI representation of an item * `getChildren()`: Returns child elements for expandable items * `onDidChangeTreeData`: Event emitter for refreshing the tree #### Language providers[​](#language-providers "Direct link to Language providers") Language providers enhance the editing experience: * **YamlDefinitionProvider**: Enables go-to-definition for schema references * **YamlDocumentSymbolProvider**: Creates document outline/symbols ### Event-driven updates[​](#event-driven-updates "Direct link to Event-driven updates") The extension uses an event-driven architecture for real-time updates: ``` // Automatic refresh every 10 seconds setInterval(() => InfrahubServerTreeView.refresh(), 10000); // Configuration change listener vscode.workspace.onDidChangeConfiguration(e => { if (e.affectsConfiguration('infrahub-vscode.servers')) { this.refresh(); } }); ``` This approach ensures: * Status bar updates reflect current server state * Tree views stay synchronized with server changes * Configuration changes take effect immediately ### Client management[​](#client-management "Direct link to Client management") The extension maintains a client pool for server connections: ``` private clients: Map = new Map(); ``` Each server configuration creates a dedicated client instance, allowing: * Concurrent connections to multiple servers * Isolated authentication per server * Efficient connection reuse ## Design decisions and rationale[​](#design-decisions-and-rationale "Direct link to Design decisions and rationale") ### Why tree views?[​](#why-tree-views "Direct link to Why tree views?") Tree views were chosen as the primary UI component because: 1. **Familiar Paradigm**: Developers are accustomed to file explorers and tree structures 2. **Information Hierarchy**: Natural representation of servers → branches → schemas 3. **Contextual Actions**: Right-click menus provide discoverable functionality 4. **Space Efficiency**: Collapsible nodes manage complex information efficiently ### Status bar integration[​](#status-bar-integration "Direct link to Status bar integration") The status bar provides ambient awareness of connection state: * **Always Visible**: Users see server status without opening panels * **Non-Intrusive**: Doesn't interrupt workflow * **Quick Feedback**: Color coding indicates connection health instantly ### Configuration strategy[​](#configuration-strategy "Direct link to Configuration strategy") The extension uses VSCode's built-in settings system rather than custom configuration files: **Advantages**: * Integrated with VSCode's settings UI * Supports workspace and user-level settings * Environment variable substitution for security * Settings sync across devices **Trade-offs**: * Limited to JSON structure * No runtime configuration changes * Requires VSCode restart for some changes ### SDK integration[​](#sdk-integration "Direct link to SDK integration") The extension uses the official `infrahub-sdk` package: ``` import { InfrahubClient, InfrahubClientOptions } from 'infrahub-sdk'; ``` **Benefits**: * Consistent API with other Infrahub tools * Maintained by the Infrahub team * Type safety with TypeScript * Automatic API version compatibility ## Mental models[​](#mental-models "Direct link to Mental models") ### Connection lifecycle[​](#connection-lifecycle "Direct link to Connection lifecycle") Understanding how connections are established and maintained: 1. **Initialization**: Server configurations are loaded from settings 2. **Client Creation**: Each server gets an InfrahubClient instance 3. **Health Checking**: Status bar polls server health every 10 seconds 4. **Query Execution**: Clients are reused for GraphQL operations 5. **Cleanup**: Connections persist until VSCode closes ### Data flow[​](#data-flow "Direct link to Data flow") How data moves through the extension: ``` User Action → Command Handler → Client Method → Server API ↓ ↓ UI Update ← Result Processing ← API Response ``` Example: Executing a GraphQL query: 1. User selects play icon next to query in tree view 2. Command handler prompts for variables 3. Client sends query to selected server/branch 4. Results are formatted and displayed in webview 5. Tree view updates if data changed ### State management[​](#state-management "Direct link to State management") The extension maintains minimal state: * **Server Configurations**: Read from settings, cached in memory * **Client Instances**: Stored in Map, reused across operations * **Tree View State**: Managed by VSCode, persists across sessions * **No Persistent Storage**: Extension doesn't write to disk ## Integration points[​](#integration-points "Direct link to Integration points") ### VSCode extension API[​](#vscode-extension-api "Direct link to VSCode extension API") The extension integrates with multiple VSCode APIs: * **Commands API**: Registers executable commands * **Tree View API**: Creates custom sidebar views * **Language API**: Provides IntelliSense features * **Webview API**: Displays query results * **Status Bar API**: Shows server connection status ### Infrahub server[​](#infrahub-server "Direct link to Infrahub server") Communication with Infrahub servers occurs through: * **GraphQL Endpoint**: Query and mutation execution * **REST API**: Server version and health checks ### File system integration[​](#file-system-integration "Direct link to File system integration") The extension interacts with the workspace: * **Schema Files**: Validates YAML in configured directories * **Query Files**: Reads `.gql` files for execution * **Configuration**: Parses `.infrahub.yml` for project structure ## Performance considerations[​](#performance-considerations "Direct link to Performance considerations") ### Lazy loading[​](#lazy-loading "Direct link to Lazy loading") Components load only when needed: * Tree views populate on expansion * Clients connect on first use * Validation runs on file save ### Caching strategy[​](#caching-strategy "Direct link to Caching strategy") The extension implements strategic caching: * Server configurations cached until settings change * Branch lists refresh every 10 seconds * Client connections persist across operations ### Resource management[​](#resource-management "Direct link to Resource management") Efficient resource usage through: * Single client instance per server * Debounced validation on typing * Incremental tree view updates ## Extension boundaries[​](#extension-boundaries "Direct link to Extension boundaries") ### What the extension does[​](#what-the-extension-does "Direct link to What the extension does") * **Client-Side Operations**: All processing happens locally * **Read Operations**: Primarily queries data * **UI Enhancement**: Improves developer experience * **Validation**: Schema and syntax checking ### What the extension doesn't do[​](#what-the-extension-doesnt-do "Direct link to What the extension doesn't do") * **Server Management**: Cannot start/stop Infrahub servers * **Data Persistence**: Doesn't store data locally * **Background Sync**: No automatic data synchronization * **Conflict Resolution**: Merge conflicts handled server-side ## Further reading[​](#further-reading "Direct link to Further reading") * [Schema Validation and YAML Intelligence](/vscode/topics/schema-validation.md) * [Extension Commands Reference](/vscode/reference/commands-settings.md) * [VSCode Extension API Documentation](https://code.visualstudio.com/api) --- # Schema Validation and YAML Intelligence This document explores how the Infrahub VSCode extension provides intelligent YAML editing capabilities, real-time schema validation, and navigation features that enhance the developer experience when working with infrastructure schemas. ## Overview[​](#overview "Direct link to Overview") Schema validation and YAML intelligence are fundamental features that transform VSCode into a powerful Infrahub development environment. These capabilities help catch errors early, provide contextual assistance, and accelerate schema development through intelligent code completion and navigation. ## The Role of Schemas in Infrahub[​](#the-role-of-schemas-in-infrahub "Direct link to The Role of Schemas in Infrahub") ### Infrastructure as Schema[​](#infrastructure-as-schema "Direct link to Infrastructure as Schema") In Infrahub, schemas define the structure of your infrastructure data: ``` version: '1.0' nodes: - name: Device namespace: Network attributes: - name: hostname kind: Text unique: true ``` This schema-first approach means: * **Type Safety**: Data must conform to defined structures * **Consistency**: All infrastructure follows the same patterns * **Validation**: Invalid data is rejected before it causes issues * **Documentation**: Schemas serve as living documentation ### Why Validation Matters[​](#why-validation-matters "Direct link to Why Validation Matters") Schema validation prevents several categories of errors: 1. **Syntax Errors**: Malformed YAML that won't parse 2. **Type Mismatches**: Wrong data types for attributes 3. **Constraint Violations**: Breaking uniqueness or required fields 4. **Relationship Errors**: Invalid references between objects 5. **Version Conflicts**: Incompatible schema versions ## How Schema Validation Works[​](#how-schema-validation-works "Direct link to How Schema Validation Works") ### Validation Pipeline[​](#validation-pipeline "Direct link to Validation Pipeline") The extension implements a multi-stage validation pipeline: ``` File Change → YAML Parse → Schema Load → Validation → Diagnostics ↓ ↓ ↓ ↓ ↓ Detected Structure Reference Rules Display Checking Resolution Applied Errors ``` ### Real-Time Validation[​](#real-time-validation "Direct link to Real-Time Validation") Validation occurs at multiple points: 1. **On Type**: Incremental validation as you type (debounced) 2. **On Save**: Full validation when file is saved 3. **On Open**: Initial validation when file is opened 4. **On Focus**: Re-validation when switching between files ### Validation Scope[​](#validation-scope "Direct link to Validation Scope") The extension validates files based on location: ``` # Automatically validated directories models/ # Infrastructure models schemas/ # Schema definitions ``` These directories are configured through: ``` { "infrahub-vscode.schemaDirectory": "schemas" } ``` ## YAML Intelligence Features[​](#yaml-intelligence-features "Direct link to YAML Intelligence Features") ### Syntax Highlighting[​](#syntax-highlighting "Direct link to Syntax Highlighting") The extension provides enhanced syntax highlighting for Infrahub-specific constructs: * **Keywords**: `version`, `nodes`, `attributes`, `relationships` * **Types**: `Text`, `Number`, `Boolean`, `IPHost`, `IPNetwork` * **Modifiers**: `unique`, `optional`, `default_value` ### Auto-Completion[​](#auto-completion "Direct link to Auto-Completion") Context-aware suggestions appear as you type: ``` nodes: - name: Router attributes: - name: hostname kind: | # Cursor here triggers type suggestions # Text, Number, Boolean, IPHost, etc. ``` ### Go-to-Definition[​](#go-to-definition "Direct link to Go-to-Definition") Navigate between related schemas with `F12` or `Ctrl+Click`: ``` relationships: - name: interfaces peer: NetworkInterface # Ctrl+Click jumps to NetworkInterface definition kind: Component ``` ### Document Symbols[​](#document-symbols "Direct link to Document Symbols") The outline view shows schema structure: ``` 📄 network.yml └─ nodes └─ Device ├─ attributes │ ├─ hostname │ └─ model └─ relationships └─ interfaces ``` ## Schema Validation Rules[​](#schema-validation-rules "Direct link to Schema Validation Rules") ### Structural Validation[​](#structural-validation "Direct link to Structural Validation") The extension enforces schema structure: ``` # Valid structure version: '1.0' nodes: - name: Device namespace: Network # Invalid - missing version nodes: # ❌ Error: Schema must include version - name: Device ``` ### Attribute Validation[​](#attribute-validation "Direct link to Attribute Validation") Attributes must follow specific rules: ``` attributes: - name: hostname kind: Text # ✅ Valid kind unique: true # ✅ Valid modifier - name: port kind: String # ❌ Error: Invalid kind 'String', use 'Text' - name: ip_address kind: IPHost mandatory: true # ❌ Error: Use 'optional: false' instead ``` ### Relationship Validation[​](#relationship-validation "Direct link to Relationship Validation") Relationships are validated for consistency: ``` relationships: - name: location peer: LocationSite # Must reference existing node kind: Parent # Must be valid relationship kind cardinality: one # Must be valid cardinality - name: interfaces peer: NonExistent # ❌ Error: Node 'NonExistent' not found kind: Invalid # ❌ Error: Invalid relationship kind ``` ## Understanding Validation Messages[​](#understanding-validation-messages "Direct link to Understanding Validation Messages") ### Error Categories[​](#error-categories "Direct link to Error Categories") Validation produces different message types: 1. **Errors** (Red): Must be fixed before schema works * Missing required fields * Invalid types * Syntax errors 2. **Warnings** (Yellow): Should be addressed but won't break functionality * Deprecated patterns * Performance concerns * Best practice violations 3. **Information** (Blue): Helpful suggestions * Optimization opportunities * Alternative approaches * Documentation hints ### Message Structure[​](#message-structure "Direct link to Message Structure") Each validation message contains: ``` [Severity] [Location] Message Example: Error at line 15: Attribute kind 'String' is not valid ``` ### Common Validation Errors[​](#common-validation-errors "Direct link to Common Validation Errors") #### Missing Required Fields[​](#missing-required-fields "Direct link to Missing Required Fields") ``` nodes: - namespace: Network # ❌ Error: Missing required field 'name' ``` **Solution**: Add the missing field: ``` nodes: - name: Device namespace: Network ``` #### Invalid Attribute Kind[​](#invalid-attribute-kind "Direct link to Invalid Attribute Kind") ``` attributes: - name: count kind: Integer # ❌ Error: Use 'Number' instead of 'Integer' ``` **Solution**: Use correct type: ``` attributes: - name: count kind: Number ``` #### Circular Dependencies[​](#circular-dependencies "Direct link to Circular Dependencies") ``` nodes: - name: A relationships: - peer: B kind: Parent - name: B relationships: - peer: A kind: Parent # ❌ Error: Circular dependency detected ``` **Solution**: Redesign relationship hierarchy ## Advanced Validation Concepts[​](#advanced-validation-concepts "Direct link to Advanced Validation Concepts") ### Schema Inheritance[​](#schema-inheritance "Direct link to Schema Inheritance") Understanding how schemas inherit from base definitions: ``` nodes: - name: Device namespace: Network inherit_from: - BuiltinDevice # Inherits attributes and relationships attributes: - name: custom_field # Adds to inherited attributes kind: Text ``` The validator checks: * Base schema exists * No attribute conflicts * Relationship compatibility ### Cross-File Validation[​](#cross-file-validation "Direct link to Cross-File Validation") The extension validates references across files: ``` # File: schemas/network.yml nodes: - name: Device namespace: Network # File: schemas/interface.yml nodes: - name: Interface namespace: Network relationships: - name: device peer: NetworkDevice # Validated across files ``` ### Version Compatibility[​](#version-compatibility "Direct link to Version Compatibility") Schema versions affect validation rules: ``` version: '1.0' # Original schema format version: '1.1' # Supports additional features generics: # New in 1.1 - name: GenericDevice ``` ## Performance Considerations[​](#performance-considerations "Direct link to Performance Considerations") ### Incremental Validation[​](#incremental-validation "Direct link to Incremental Validation") The extension uses incremental validation for performance: 1. **Parse Caching**: YAML AST cached between edits 2. **Partial Updates**: Only changed sections re-validated 3. **Debouncing**: Validation delayed during rapid typing 4. **Background Processing**: Validation doesn't block UI ### Large File Handling[​](#large-file-handling "Direct link to Large File Handling") For large schema files: * **Lazy Loading**: Schemas loaded on-demand * **Chunked Processing**: Large files processed in segments * **Priority Validation**: Visible content validated first * **Throttling**: Rate-limited validation for huge files ## Customizing Validation[​](#customizing-validation "Direct link to Customizing Validation") ### Validation Settings[​](#validation-settings "Direct link to Validation Settings") Configure validation behavior: ``` { "yaml.schemas": { "https://schema.infrahub.app/infrahub/schema/latest.json": [ "schemas/**/*.yml", "models/**/*.yaml" ] }, "yaml.validate": true, "yaml.customTags": [ "!include", "!secret" ] } ``` ### Disabling Validation[​](#disabling-validation "Direct link to Disabling Validation") To temporarily disable validation: 1. **Per File**: Add comment at top ``` # yaml-language-server: $schema=none ``` 2. **Per Workspace**: Modify settings ``` { "yaml.validate": false } ``` ## Integration with Other Tools[​](#integration-with-other-tools "Direct link to Integration with Other Tools") ### YAML Language Server[​](#yaml-language-server "Direct link to YAML Language Server") The extension leverages the Red Hat YAML extension: * **Schema Association**: Links files to Infrahub schemas * **JSON Schema**: Uses JSON Schema for validation * **Custom Tags**: Supports Infrahub-specific tags ### Linting Integration[​](#linting-integration "Direct link to Linting Integration") Works alongside linting tools: * **yamllint**: Style and formatting checks * **Prettier**: Code formatting * **Vale**: Documentation linting ## Troubleshooting Validation Issues[​](#troubleshooting-validation-issues "Direct link to Troubleshooting Validation Issues") ### Validation Not Working[​](#validation-not-working "Direct link to Validation Not Working") If validation isn't functioning: 1. **Check File Location**: Ensure file is in configured directory 2. **Verify Extension**: Confirm YAML extension is installed 3. **Schema Association**: Check schema URL is correct 4. **Reload Window**: Try reloading VSCode ### False Positives[​](#false-positives "Direct link to False Positives") If seeing incorrect errors: 1. **Update Extension**: Ensure latest version 2. **Check Schema Version**: Verify schema compatibility 3. **Clear Cache**: Reload VSCode window 4. **Report Issue**: File bug report if persistent ### Performance Issues[​](#performance-issues "Direct link to Performance Issues") If validation is slow: 1. **File Size**: Consider splitting large files 2. **Disable Real-time**: Turn off on-type validation 3. **Increase Debounce**: Adjust validation delay 4. **Check Extensions**: Disable conflicting extensions ## Further Reading[​](#further-reading "Direct link to Further Reading") * [Understanding the Extension Architecture](/vscode/topics/extension-architecture.md) * [How to Execute GraphQL Queries](/vscode/guides/execute-graphql-queries.md) * [Extension Commands Reference](/vscode/reference/commands-settings.md) --- # Security Configuration and Best Practices This document covers security considerations, TLS configuration, API token management, and best practices when using the Infrahub VSCode extension across different environments. ## Overview[​](#overview "Direct link to Overview") The Infrahub VSCode extension handles connections to Infrahub servers which may contain sensitive infrastructure data. Proper security configuration ensures your connections are encrypted and authenticated appropriately for your environment. ## TLS certificate configuration[​](#tls-certificate-configuration "Direct link to TLS certificate configuration") ### Understanding TLS in the extension[​](#understanding-tls-in-the-extension "Direct link to Understanding TLS in the extension") The extension connects to Infrahub servers over HTTPS using TLS encryption. By default, the extension verifies server certificates to ensure secure connections. However, development environments often use self-signed certificates that fail standard verification. ### Production environment (default secure settings)[​](#production-environment-default-secure-settings "Direct link to Production environment (default secure settings)") For production environments, always use properly signed certificates: ``` { "infrahub-vscode.servers": [ { "name": "Production", "address": "https://infrahub.company.com", "api_token": "${env:INFRAHUB_PROD_TOKEN}" // tls_insecure defaults to false - secure mode } ] } ``` **Security Benefits:** * Prevents man-in-the-middle attacks * Ensures server identity verification * Maintains encryption integrity * Complies with enterprise security policies ### Development environment (insecure mode)[​](#development-environment-insecure-mode "Direct link to Development environment (insecure mode)") For development environments with self-signed certificates: ``` { "infrahub-vscode.servers": [ { "name": "Development", "address": "https://dev.infrahub.local", "api_token": "${env:INFRAHUB_DEV_TOKEN}", "tls_insecure": true } ] } ``` **When to Use `tls_insecure: true`:** * Local development servers with self-signed certificates * Docker containers with self-generated certificates * Internal testing environments with custom CA certificates * Development environments where proper certificates aren't feasible **Security Implications:** * Disables certificate verification * Vulnerable to man-in-the-middle attacks * Should never be used in production * Affects all HTTPS connections in the VSCode process ### Mixed environment configuration[​](#mixed-environment-configuration "Direct link to Mixed environment configuration") You can configure different TLS settings for different environments: ``` { "infrahub-vscode.servers": [ { "name": "Production", "address": "https://infrahub.company.com", "api_token": "${env:INFRAHUB_PROD_TOKEN}" // Secure by default }, { "name": "Staging", "address": "https://staging.infrahub.company.com", "api_token": "${env:INFRAHUB_STAGING_TOKEN}" // Uses proper certificates }, { "name": "Development", "address": "https://dev.infrahub.local:8000", "api_token": "${env:INFRAHUB_DEV_TOKEN}", "tls_insecure": true // Allows self-signed certificates } ] } ``` ### How TLS configuration works internally[​](#how-tls-configuration-works-internally "Direct link to How TLS configuration works internally") When any server has `tls_insecure: true`: 1. **Environment Variable**: Sets `NODE_TLS_REJECT_UNAUTHORIZED = '0'` 2. **Global Effect**: Affects all HTTPS connections in the VSCode process 3. **Restoration**: Original setting restored when extension deactivates 4. **Dynamic Updates**: Changes apply immediately when configuration updates This approach was chosen to ensure compatibility with the Infrahub SDK and provide reliable certificate bypassing when needed. ## API token security[​](#api-token-security "Direct link to API token security") ### Token storage best practices[​](#token-storage-best-practices "Direct link to Token storage best practices") #### ❌ Don't store tokens directly[​](#-dont-store-tokens-directly "Direct link to ❌ Don't store tokens directly") ``` // NEVER do this - tokens visible in settings { "infrahub-vscode.servers": [ { "api_token": "inf_1234567890abcdef" // Visible in settings! } ] } ``` #### ✅ Use Environment Variables[​](#-use-environment-variables "Direct link to ✅ Use Environment Variables") ``` // Secure approach - tokens in environment variables { "infrahub-vscode.servers": [ { "api_token": "${env:INFRAHUB_API_TOKEN}" // References environment } ] } ``` ### Setting up environment variables[​](#setting-up-environment-variables "Direct link to Setting up environment variables") #### macOS and Linux[​](#macos-and-linux "Direct link to macOS and Linux") Add to your shell configuration file (`~/.zshrc`, `~/.bashrc`, etc.): ``` # Development environment export INFRAHUB_DEV_TOKEN="inf_dev_1234567890abcdef" # Staging environment export INFRAHUB_STAGING_TOKEN="inf_staging_1234567890abcdef" # Production environment export INFRAHUB_PROD_TOKEN="inf_prod_1234567890abcdef" ``` #### Windows[​](#windows "Direct link to Windows") Using PowerShell: ``` # Set user-level environment variables [System.Environment]::SetEnvironmentVariable('INFRAHUB_DEV_TOKEN', 'inf_dev_1234567890abcdef', 'User') [System.Environment]::SetEnvironmentVariable('INFRAHUB_STAGING_TOKEN', 'inf_staging_1234567890abcdef', 'User') [System.Environment]::SetEnvironmentVariable('INFRAHUB_PROD_TOKEN', 'inf_prod_1234567890abcdef', 'User') ``` ### Token management best practices[​](#token-management-best-practices "Direct link to Token management best practices") #### Token rotation[​](#token-rotation "Direct link to Token rotation") Regularly rotate API tokens: ``` # Example rotation script #!/bin/bash # Generate new token via Infrahub API or web interface NEW_TOKEN="inf_new_token_here" # Update environment variable export INFRAHUB_PROD_TOKEN="$NEW_TOKEN" # Update your shell configuration echo "export INFRAHUB_PROD_TOKEN=\"$NEW_TOKEN\"" >> ~/.zshrc ``` #### Minimal permissions[​](#minimal-permissions "Direct link to Minimal permissions") Create tokens with minimal required permissions: * **Read-only tokens** for query execution * **Limited scope tokens** for specific resources * **Separate tokens** for each environment * **Short-lived tokens** for temporary access #### Token security checklist[​](#token-security-checklist "Direct link to Token security checklist") * Tokens stored in environment variables, not settings files * Different tokens for each environment (dev, staging, prod) * Tokens have minimal required permissions * Regular token rotation schedule established * Tokens excluded from version control (`.env` files in `.gitignore`) * Access to tokens limited to authorized personnel ## Network security considerations[​](#network-security-considerations "Direct link to Network security considerations") ### Firewall and proxy configuration[​](#firewall-and-proxy-configuration "Direct link to Firewall and proxy configuration") #### Corporate proxy settings[​](#corporate-proxy-settings "Direct link to Corporate proxy settings") If behind a corporate proxy, VSCode inherits system proxy settings. For manual configuration: ``` { "http.proxy": "http://proxy.company.com:8080", "http.proxyStrictSSL": true, "http.proxyAuthorization": null } ``` #### Network access requirements[​](#network-access-requirements "Direct link to Network access requirements") The extension requires outbound HTTPS access to: * Infrahub server endpoints (configured addresses) * Schema validation endpoints (`https://schema.infrahub.app/`) ### VPN and network isolation[​](#vpn-and-network-isolation "Direct link to VPN and network isolation") #### VPN-protected servers[​](#vpn-protected-servers "Direct link to VPN-protected servers") For servers behind VPN: ``` { "infrahub-vscode.servers": [ { "name": "Internal Production", "address": "https://internal.infrahub.corp", "api_token": "${env:INFRAHUB_INTERNAL_TOKEN}" // Ensure VPN is connected before use } ] } ``` #### Network segmentation[​](#network-segmentation "Direct link to Network segmentation") Consider network segmentation best practices: * Development servers on isolated networks * Production servers with restricted access * API tokens with network-based restrictions ## Compliance and audit considerations[​](#compliance-and-audit-considerations "Direct link to Compliance and audit considerations") ### Logging and monitoring[​](#logging-and-monitoring "Direct link to Logging and monitoring") The extension logs connection attempts and errors. Monitor logs for: * Failed authentication attempts * Certificate verification bypasses * Unusual connection patterns * Token usage patterns ### Compliance requirements[​](#compliance-requirements "Direct link to Compliance requirements") For regulated environments: #### SOC 2 / ISO 27001[​](#soc-2--iso-27001 "Direct link to SOC 2 / ISO 27001") * Document TLS configuration decisions * Maintain token rotation procedures * Log security-relevant events * Regular security configuration reviews #### GDPR / data privacy[​](#gdpr--data-privacy "Direct link to GDPR / data privacy") * Understand what data is transmitted to servers * Ensure proper encryption in transit * Document data processing activities * Implement data retention policies ### Security scanning integration[​](#security-scanning-integration "Direct link to Security scanning integration") #### VS Code security extensions[​](#vs-code-security-extensions "Direct link to VS Code security extensions") Consider installing complementary security extensions: ``` { "recommendations": [ "ms-vscode.vscode-json", "redhat.vscode-yaml", "ms-python.python" ] } ``` #### Static analysis[​](#static-analysis "Direct link to Static analysis") Regularly scan configuration files for: * Hardcoded tokens or secrets * Insecure TLS configurations * Overly permissive settings ## Troubleshooting security issues[​](#troubleshooting-security-issues "Direct link to Troubleshooting security issues") ### Common TLS errors and solutions[​](#common-tls-errors-and-solutions "Direct link to Common TLS errors and solutions") #### Certificate expired[​](#certificate-expired "Direct link to Certificate expired") **Error**: `CERT_HAS_EXPIRED` **Solutions**: 1. Renew the server certificate (production) 2. Add `"tls_insecure": true` (development only) #### Self-signed certificate[​](#self-signed-certificate "Direct link to Self-signed certificate") **Error**: `SELF_SIGNED_CERT_IN_CHAIN` **Solutions**: 1. Install proper CA-signed certificate (production) 2. Add `"tls_insecure": true` (development) 3. Add certificate to system trust store #### Certificate verification failed[​](#certificate-verification-failed "Direct link to Certificate verification failed") **Error**: `UNABLE_TO_VERIFY_LEAF_SIGNATURE` **Solutions**: 1. Check certificate chain configuration 2. Update system CA certificate store 3. Use `"tls_insecure": true` for development ### Authentication issues[​](#authentication-issues "Direct link to Authentication issues") #### Token not working[​](#token-not-working "Direct link to Token not working") **Symptoms**: `401 Unauthorized` errors **Check**: 1. Token format (should start with `inf_`) 2. Token permissions and scope 3. Token expiration date 4. Environment variable substitution #### Environment variable not found[​](#environment-variable-not-found "Direct link to Environment variable not found") **Error**: Token resolves to literal `${env:VARIABLE_NAME}` **Solutions**: 1. Verify environment variable is set 2. Restart VSCode after setting variables 3. Check variable name spelling ### Network connectivity issues[​](#network-connectivity-issues "Direct link to Network connectivity issues") #### Proxy interference[​](#proxy-interference "Direct link to Proxy interference") **Symptoms**: Connection timeouts or SSL errors **Solutions**: 1. Configure VSCode proxy settings 2. Add Infrahub server to proxy bypass list 3. Verify proxy supports HTTPS CONNECT #### Firewall blocking[​](#firewall-blocking "Direct link to Firewall blocking") **Symptoms**: Connection refused or timeouts **Solutions**: 1. Verify outbound HTTPS (443) access 2. Add Infrahub server to firewall allowlist 3. Check for deep packet inspection interference ## Security best practices summary[​](#security-best-practices-summary "Direct link to Security best practices summary") ### Development environment[​](#development-environment "Direct link to Development environment") * ✅ Use `tls_insecure: true` for self-signed certificates * ✅ Use separate development API tokens * ✅ Limit development token permissions * ✅ Regular token rotation * ❌ Never use production tokens in development ### Staging environment[​](#staging-environment "Direct link to Staging environment") * ✅ Use proper certificates when possible * ✅ Separate staging API tokens * ✅ Mirror production security settings * ✅ Test certificate configurations * ❌ Don't use `tls_insecure` unless necessary ### Production environment[​](#production-environment "Direct link to Production environment") * ✅ Always use proper CA-signed certificates * ✅ Never use `tls_insecure: true` * ✅ Implement token rotation procedures * ✅ Monitor and log access * ✅ Regular security reviews * ❌ Never compromise on certificate verification ### Token management[​](#token-management "Direct link to Token management") * ✅ Store in environment variables * ✅ Use minimal required permissions * ✅ Regular rotation schedule * ✅ Audit token usage * ❌ Never commit tokens to version control * ❌ Don't share tokens between environments ## Related documentation[​](#related-documentation "Direct link to Related documentation") * [Configure Multiple Servers Guide](/vscode/guides/configure-multiple-servers.md) * [Commands and Settings Reference](/vscode/reference/commands-settings.md) * [Getting Started Tutorial](/vscode/tutorials/getting-started.md) --- # Getting Started with Infrahub VSCode Extension This tutorial will walk you through installing the Infrahub VSCode extension, connecting to your first Infrahub server, and exploring the core features. By the end of this tutorial, you'll be ready to start developing infrastructure schemas and executing GraphQL queries directly from VSCode. ## What you'll learn[​](#what-youll-learn "Direct link to What you'll learn") * How to install the Infrahub extension * Setting up your first server connection * Navigating the extension interface * Creating your first schema * Executing a GraphQL query ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this tutorial, ensure you have: * Visual Studio Code version 1.99.0 or higher installed * Access to an Infrahub server (either local or remote) * Basic familiarity with VSCode extensions * (Optional) An API token for your Infrahub server ## Step 1: Install the extension[​](#step-1-install-the-extension "Direct link to Step 1: Install the extension") ### Option A: from VSCode marketplace[​](#option-a-from-vscode-marketplace "Direct link to Option A: from VSCode marketplace") 1. Open Visual Studio Code 2. Click on the Extensions icon in the Activity Bar (`Ctrl+Shift+X` on Windows/Linux, `Cmd+Shift+X` on macOS) 3. Search for "Infrahub" 4. Find the extension published by "opsmill" 5. Click the **Install** button ### Option B: VSCode based editors[​](#option-b-vscode-based-editors "Direct link to Option B: VSCode based editors") 1. We publish the extension to [Open VSX](https://open-vsx.org/extension/opsmill/infrahub) which is often used by VSCode Forks like Cursor. 2. In your editor of choice search Infrahub. 3. Install 4. If this is not an option download the VSIX file and import manually. ### Option C: from a VSIX file[​](#option-c-from-a-vsix-file "Direct link to Option C: from a VSIX file") If you have a `.vsix` file: 1. Open VSCode 2. Press `Ctrl+Shift+P` (or `Cmd+Shift+P` on macOS) to open the Command Palette 3. Type "Extensions: Install from VSIX" 4. Navigate to your `.vsix` file and select it ## Step 2: Configure your first server[​](#step-2-configure-your-first-server "Direct link to Step 2: Configure your first server") Once installed, you'll need to connect to an Infrahub server. 1. Open VSCode Settings: * Press `Ctrl+,` (Windows/Linux) or `Cmd+,` (macOS) * Or go to File → Preferences → Settings 2. Search for "Infrahub" in the settings search bar 3. Find the **Infrahub-vscode: Servers** setting 4. Click **Edit in settings.json** 5. Add your server configuration: ``` { "infrahub-vscode.servers": [ { "name": "Local Development", "address": "http://localhost:8000" } ] } ``` If your server requires authentication, add an API token: ``` { "infrahub-vscode.servers": [ { "name": "Local Development", "address": "http://localhost:8000", "api_token": "your-api-token-here" } ] } ``` > **Tip:** You can use environment variables for sensitive tokens: > > ``` > "api_token": "${env:INFRAHUB_API_TOKEN}" > ``` ### TLS configuration for HTTPS servers[​](#tls-configuration-for-https-servers "Direct link to TLS configuration for HTTPS servers") If you're connecting to an HTTPS server with a self-signed certificate (common in development environments), you may need to disable certificate verification: ``` { "infrahub-vscode.servers": [ { "name": "Development (Self-Signed)", "address": "https://dev.infrahub.local:8000", "api_token": "${env:INFRAHUB_DEV_TOKEN}", "tls_insecure": true } ] } ``` warning The `tls_insecure` option disables TLS certificate verification and should **only be used in development environments** with self-signed certificates. Never use this option for production servers as it makes connections vulnerable to security attacks. ## Step 3: Verify your connection[​](#step-3-verify-your-connection "Direct link to Step 3: Verify your connection") After configuring your server: 1. Look at the VSCode status bar (bottom of the window) 2. You should see "Infrahub: v\[version] (Local Development)" 3. If you see "Infrahub: Server unreachable", check your server address and network connection The extension checks the connection every 10 seconds and updates the status automatically. ## Step 4: Explore the extension interface[​](#step-4-explore-the-extension-interface "Direct link to Step 4: Explore the extension interface") ### Activity bar[​](#activity-bar "Direct link to Activity bar") Click on the Infrahub icon in the Activity Bar (left side of VSCode). You'll see two tree views: 1. **Infrahub Servers**: Shows your configured servers and their branches 2. **Infrahub YAML**: Displays the structure of `.infrahub.yml` files in your workspace ### Server tree view[​](#server-tree-view "Direct link to Server tree view") 1. Expand your server in the Infrahub Servers view 2. You'll see a list of branches available on that server 3. The default branch is marked with "(default)" ### Context menu actions[​](#context-menu-actions "Direct link to Context menu actions") Right-click on elements in the tree views to see available actions: * Servers: Create new branches * Branches: Delete branches * YAML items: Edit files, execute queries ## Step 5: Create your first schema[​](#step-5-create-your-first-schema "Direct link to Step 5: Create your first schema") Let's create a network device schema. 1. Create a new directory called `schemas` in your project root: ``` mkdir schemas ``` 2. Create a new file `schemas/device.yml`: 3. Add the following schema definition: ``` --- version: '1.0' nodes: - name: Device namespace: Network description: Basic network device attributes: - name: hostname kind: Text unique: true description: Device hostname - name: model kind: Text optional: true description: Device model number - name: status kind: Text default_value: "active" description: Operational status ``` 4. Save the file. The extension will automatically validate your schema. 5. If there are any validation errors, they'll appear in the Problems panel (`Ctrl+Shift+M`) ## Step 6: Execute your first GraphQL query[​](#step-6-execute-your-first-graphql-query "Direct link to Step 6: Execute your first GraphQL query") Now let's query the devices in your Infrahub instance. 1. Create a `.infrahub.yml` file in your project root: ``` --- queries: - name: list_devices file: queries/list_devices.gql ``` 2. Create the query file `queries/list_devices.gql`: ``` query ListDevices { NetworkDevice { edges { node { hostname { value } model { value } status { value } } } } } ``` 3. In the Infrahub YAML tree view, expand your `.infrahub.yml` file 4. Find the "list\_devices" query 5. Right-click and select **Execute GraphQL Query** 6. When prompted: * Select your server * Choose a branch (usually "main") 7. The query results will appear in a new panel ## Step 7: Working with branches[​](#step-7-working-with-branches "Direct link to Step 7: Working with branches") Let's create a new branch for development work. 1. In the Infrahub Servers tree view, right-click on your server 2. Select **New Branch** 3. Enter a branch name (for example, "feature-network-schema") 4. Optionally add a description 5. The new branch will appear under your server in the tree You can now execute queries against this branch by selecting it when prompted during query execution. ## Next steps[​](#next-steps "Direct link to Next steps") Congratulations! You've successfully: * Installed and configured the Infrahub VSCode extension * Connected to an Infrahub server * Created a schema with validation * Executed a GraphQL query * Created a development branch ### What to explore next[​](#what-to-explore-next "Direct link to What to explore next") * **[How to Run Transforms and Artifacts](/vscode/guides/running-transforms.md)**: Execute Jinja2 and Python transforms directly from VSCode * **[How to Configure Multiple Servers](/vscode/guides/configure-multiple-servers.md)**: Work with development, staging, and production environments * **[Understanding Schema Validation](/vscode/topics/schema-validation.md)**: Deep dive into how the extension validates your schemas * **[Security Configuration and Best Practices](/vscode/topics/security-configuration.md)**: Comprehensive guide to TLS settings, API tokens, and security practices * **[Extension Commands Reference](/vscode/reference/commands-settings.md)**: Complete list of available commands and settings ### Troubleshooting tips[​](#troubleshooting-tips "Direct link to Troubleshooting tips") If you encounter issues: * **Server connection fails**: Verify the server URL and check if the server is running * **Schema validation errors**: Check the Problems panel for specific error messages * **GraphQL queries fail**: Ensure the schema is loaded in Infrahub and you're querying the correct branch * **TLS certificate errors**: If connecting to a server with self-signed certificates, add `"tls_insecure": true` to your server configuration * **"Server unreachable" status**: Check network connectivity, firewall settings, and ensure the Infrahub server is accessible ## Further resources[​](#further-resources "Direct link to Further resources") * [Infrahub Documentation](https://docs.infrahub.app) * [Extension GitHub Repository](https://github.com/opsmill/infrahub-vscode) * [Infrahub Community Discord](https://discord.gg/infrahub) --- # Configure storage The storage API provides direct access to Infrahub's object storage layer for uploading and retrieving content by identifier. ## Uploading a file to the object-storage[​](#uploading-a-file-to-the-object-storage "Direct link to Uploading a file to the object-storage") To upload a file to the object-storage we can use the `/api/storage/upload/file` REST API endpoint. Create a file on your system. You can save the file anywhere you want, with any filename you want. But in this guide we will be using `/tmp/test.json` ``` { "key": "value", "list": [ "item1", "item2", "item3" ] } ``` We will be using the `curl` command to upload the file. note You might have to change the X-INFRAHUB-KEY header value to the API token of your user, if you are not using the default. ``` curl -X POST --header "X-INFRAHUB-KEY: 06438eb2-8019-4776-878c-0941b1f1d1ec" http://localhost:8000/api/storage/upload/file -F file=@/tmp/test.json ``` The response will be a JSON object representing the object in the object-storage. Take note of the identifier. ``` {"identifier":"17d198df-3102-00e0-3349-c51ae276e246","checksum":"f4516d51f2c894fc47df1782c6661ebc"} ``` ## Retrieving the object from the object-storage[​](#retrieving-the-object-from-the-object-storage "Direct link to Retrieving the object from the object-storage") To retrieve objects from the object-storage we can use the `/api/storage/object/` REST API endpoint. The identifier in the URL should be the identifier that you gathered from the previous step. note You might have to change the X-INFRAHUB-KEY header value to the API token of your user, if you are not using the default. ``` curl --header "X-INFRAHUB-KEY: 06438eb2-8019-4776-878c-0941b1f1d1ec" http://localhost:8000/api/storage/object/17d198df-3102-00e0-3349-c51ae276e246 { "key": "value", "list": [ "item1", "item2", "item3" ] } ``` ## Storing objects in the object-storage[​](#storing-objects-in-the-object-storage "Direct link to Storing objects in the object-storage") You can upload any string based object into the object-storage, as an alternative to uploading a file. In this step we will be storing the string "interface Ethernet1\n description: Connected to Ethernet2" into the object-storage. note You might have to change the X-INFRAHUB-KEY header value to the API token of your user, if you are not using the default. ``` curl -X POST --header "Content-Type: application/json" --header "X-INFRAHUB-KEY: 06438eb2-8019-4776-878c-0941b1f1d1ec" http://localhost:8000/api/storage/upload/content --data '{"content": "interface Ethernet1\n description: Connected to Ethernet2"}' ``` ``` {"identifier":"17d19962-4420-4972-334d-c51beb03e477","checksum":"843cde3df258ab3027618712952499cd"} ``` Retrieving the object works as described in the previous step. ``` curl --header "X-INFRAHUB-KEY: 06438eb2-8019-4776-878c-0941b1f1d1ec" http://localhost:8000/api/storage/object/17d19962-4420-4972-334d-c51beb03e477 ``` --- # About storage Infrahub uses an object storage layer to persist binary and text content outside of the graph database. This layer stores the raw bytes of [file objects](/schema/file-object.md) and rendered [artifacts](/artifacts/overview.md). Separating file content from graph data allows Infrahub to keep the graph database focused on relationships, metadata, and version control while delegating bulk storage to a system optimized for that purpose. ## What uses object storage[​](#what-uses-object-storage "Direct link to What uses object storage") Object storage serves two use cases in Infrahub: * **File objects**: User-uploaded files attached to nodes in the graph. These are managed through the file object system and access is controlled by Infrahub's [permission system](/deploy-manage/user-management/permissions-roles/overview.md). See [file objects](/schema/file-object.md) for details. * **Artifacts**: System-generated outputs from Transformations. These are created automatically by artifact definitions and stored in the same storage layer. See [artifacts](/artifacts/overview.md) for details. Both file objects and artifacts store a `storage_id` in the graph database that references the content in object storage. The difference is in how the content is created — user uploads for file objects, automated Transformations for artifacts — and how access is governed. ## Storage backends[​](#storage-backends "Direct link to Storage backends") Infrahub supports two storage backends. The choice of backend is transparent to the rest of the system — all API endpoints, SDK methods, and internal operations work identically regardless of which backend is configured. info Use **local filesystem** storage for development and testing. For production and multi-node deployments, use **S3-compatible storage** to avoid shared filesystem requirements. ### Local filesystem[​](#local-filesystem "Direct link to Local filesystem") The default backend stores files on the local filesystem. This is suitable for development, testing, and single-node deployments. ``` INFRAHUB_STORAGE_DRIVER: "local" INFRAHUB_STORAGE_LOCAL_PATH: "/opt/infrahub/storage" ``` All Infrahub API servers and task workers must have access to the configured directory. In multi-node deployments with local storage, this typically means a shared network filesystem. ### S3-compatible storage[​](#s3-compatible-storage "Direct link to S3-compatible storage") For production and multi-node deployments, Infrahub supports Amazon S3 and any S3-compatible service (MinIO, Ceph, and others). ``` INFRAHUB_STORAGE_DRIVER: "s3" AWS_ACCESS_KEY_ID: "my_access_key" AWS_SECRET_ACCESS_KEY: "secret_access_key" INFRAHUB_STORAGE_BUCKET_NAME: "my-infrahub-bucket" INFRAHUB_STORAGE_ENDPOINT_URL: "s3.eu-central-1.amazonaws.com" ``` Additional S3 options include SSL configuration, ACL defaults, query string authentication, and custom domains. See the [configuration reference](/reference/configuration.md) for the full list. ## How it works[​](#how-it-works "Direct link to How it works") ### Concepts and definitions[​](#concepts-and-definitions "Direct link to Concepts and definitions") **Object storage** is a key-value store where each entry is identified by a UUID. When content is uploaded, Infrahub generates a UUID using a time-sortable format (UUIDT), stores the content under that key, and returns the identifier to the caller. All subsequent operations — retrieval, deletion — reference this identifier. A **storage driver** is the backend that handles the actual persistence of content. Infrahub abstracts the driver behind a unified interface, so the rest of the system interacts with object storage the same way regardless of whether files land on a local filesystem or in an S3 bucket. A **storage identifier** (or `storage_id`) is the UUID that links a graph node to its stored content. File object nodes and artifact nodes both carry a `storage_id` attribute that points to the corresponding entry in object storage. ### Architecture[​](#architecture "Direct link to Architecture") The object storage layer sits between the Infrahub API and the configured storage backend. It exposes four operations: * **Store**: Write content under a given identifier * **Retrieve**: Read content as a decoded string by identifier * **Retrieve binary**: Read content as raw bytes by identifier * **Delete**: Remove content by identifier These operations are driver-agnostic. The `InfrahubObjectStorage` class loads the configured driver at startup and delegates all calls to it. This design means that adding a new storage backend only requires implementing the driver interface — no changes to the API or core logic. ### Immutability[​](#immutability "Direct link to Immutability") Content in object storage is immutable. Once a file is stored, it is never modified in place. When a file is updated, Infrahub stores the new version under a new UUID. The previous version remains in storage, which is what enables time travel and branch isolation for file objects and artifacts. This approach avoids the complexity of in-place updates and means the storage layer does not need to understand branches, merge conflicts, or version history. All of that logic lives in the graph database, where Infrahub already has mature support for it. ### Relationship to the graph database[​](#relationship-to-the-graph-database "Direct link to Relationship to the graph database") The graph database and object storage serve complementary roles: ``` Graph database (branch-aware, time-aware): Stores metadata, relationships, and storage_id references Handles branching, merging, time travel, permissions Object storage (branch-agnostic): Stores raw file content by UUID Simple key-value store, no version control logic ``` When you query a file object on a specific branch or at a specific point in time, the graph database resolves which `storage_id` was active in that context. The storage layer then retrieves the corresponding content. This separation keeps both systems focused on their core responsibility. ## Operational considerations[​](#operational-considerations "Direct link to Operational considerations") ### File size limits[​](#file-size-limits "Direct link to File size limits") The maximum upload size defaults to 50 MB and can be adjusted through the `INFRAHUB_STORAGE_MAX_FILE_SIZE` environment variable (value in MB). For deployments behind a reverse proxy, the proxy must also be configured to allow matching request body sizes (for example, `client_max_body_size` in NGINX or request size middleware in Traefik). [How to configure Infrahub../deploy-manage/install-configure/configure-infrahub](/deploy-manage/install-configure/configure-infrahub.md) ### Storage growth[​](#storage-growth "Direct link to Storage growth") Because object storage is immutable, content accumulates over time. Every file update creates a new entry while the previous version remains in storage to support time travel and branch history. There is currently no automatic garbage collection for orphaned content. In environments with frequent file updates, storage usage should be monitored. ### Shared access in multi-node deployments[​](#shared-access-in-multi-node-deployments "Direct link to Shared access in multi-node deployments") When using local filesystem storage across multiple API servers or task workers, all nodes must have access to the same storage directory. This typically requires a shared network filesystem (NFS or similar). S3-compatible storage avoids this constraint entirely since all nodes access the same bucket over the network. [Production deployment guide../deploy-manage/install-configure/production-deployment/overview](/deploy-manage/install-configure/production-deployment/overview.md) ### Backups[​](#backups "Direct link to Backups") Object storage should be included in your backup strategy alongside the graph database. The two are tightly coupled: the graph database holds `storage_id` references that point to content in object storage. Restoring one without the other results in broken references (dangling `storage_id` values pointing to missing files) or orphaned files (content in storage with no corresponding graph node). Data loss risk Unlike artifacts, which can be regenerated from their definitions, user-uploaded file objects may only exist in object storage. If storage is lost without a backup, those files are unrecoverable. [How to backup object storage../deploy-manage/maintain-upgrade/database-backup/backup-and-restore#step-3-backup-the-object-storage](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md#step-3-backup-the-object-storage) ### Backend migration[​](#backend-migration "Direct link to Backend migration") Backend migration Switching storage backends (for example, from `local` to `s3`) does not migrate existing content. Files stored under the previous backend become inaccessible. Artifacts can be regenerated, but user-uploaded file objects must be re-uploaded. ## Connection to other concepts[​](#connection-to-other-concepts "Direct link to Connection to other concepts") * **[File objects](/schema/file-object.md)**: The primary way users interact with stored files. File objects are graph nodes that combine metadata with a reference to content in object storage. * **[Artifacts](/artifacts/overview.md)**: System-generated content stored in the same storage layer, produced by Transformation pipelines. * **[Permissions and roles](/deploy-manage/user-management/permissions-roles/overview.md)**: Access to file content is enforced at the API level based on permissions on the corresponding graph node. ## Further reading[​](#further-reading "Direct link to Further reading") * [Configure storage](/artifact-file-storage/configure.md): Practical steps for uploading and retrieving content * [File objects](/schema/file-object.md): Attaching files to nodes with full version control * [Configuration reference](/reference/configuration.md): All storage-related environment variables and settings --- # Composing artifact content This guide shows you how to build a composite [artifact](/artifacts/overview.md) whose [Transformation](/transformations/overview.md) pulls in content from other artifacts or [file objects](/schema/file-object.md). This enables modular configuration pipelines where each artifact generates one section of a configuration, and a composite artifact assembles the final result. For example, you can split a network device's startup configuration into separate NTP, syslog, and interface artifacts — each maintained independently with its own Transformation and query logic — then compose them into a single full configuration artifact. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this guide, ensure you have: * A running Infrahub instance with task workers * At least one source artifact already generating content * An [external repository](/git-integration/connect-repository.md) connected to Infrahub * Familiarity with [creating a Jinja2 Transformation](/learn/tutorials/transformations/build-a-jinja2-transformation.md) or [creating a Python Transformation](/learn/tutorials/transformations/build-a-python-transformation.md) * Familiarity with [generating artifacts](/artifacts/use.md) ## 1. Understand the composition model[​](#1-understand-the-composition-model "Direct link to 1. Understand the composition model") During artifact generation, your Jinja2 templates have access to Infrahub filters that can fetch content from the [object storage](/artifact-file-storage/overview.md). To build a composite artifact, your Transformation's GraphQL query retrieves the `storage_id` of another artifact or file object, and your template uses a filter to fetch and include that content inline. The filters always return content as a raw string. If you need to work with structured data, you can chain the `from_json` or `from_yaml` parsing filters. note The `artifact_content`, `file_object_content`, `file_object_content_by_id`, and `file_object_content_by_hfid` filters are only available during artifact generation on the task worker. They are not available in [computed attributes](/computed-attributes/overview.md) or when rendering locally with `infrahubctl render` without a connected server. The `from_json` and `from_yaml` filters are available in all contexts. ## 2. Query for artifact storage IDs[​](#2-query-for-artifact-storage-ids "Direct link to 2. Query for artifact storage IDs") Your composite Transformation's GraphQL query needs to return the `storage_id` of any artifacts whose content you want to include. You can use the `artifacts` relationship on any `CoreArtifactTarget` node, or query `CoreArtifact` directly. ``` query CompositeDeviceConfig($device_name: String!) { InfraDevice(name__value: $device_name) { edges { node { name { value } artifacts { edges { node { name { value } storage_id { value } } } } } } } } ``` This query returns all artifacts for the device, each with its `storage_id`. In your template, you can then select the relevant artifact by name and fetch its content. ## 3. Compose content in a Jinja2 Transformation[​](#3-compose-content-in-a-jinja2-transformation "Direct link to 3. Compose content in a Jinja2 Transformation") ### Include raw artifact content[​](#include-raw-artifact-content "Direct link to Include raw artifact content") You can use the `artifact_content` filter to fetch an artifact's content by its `storage_id`: ``` {% set device = data.InfraDevice.edges[0].node %} hostname {{ device.name.value }} ! {% for artifact in device.artifacts.edges %} {% if artifact.node.name.value == "ntp_config" %} {{ artifact.node.storage_id.value | artifact_content }} {% endif %} {% endfor %} ``` ### Parse structured content[​](#parse-structured-content "Direct link to Parse structured content") You can chain `artifact_content` with `from_json` or `from_yaml` to work with structured data: ``` {% set device = data.InfraDevice.edges[0].node %} {% for artifact in device.artifacts.edges %} {% if artifact.node.name.value == "interface_data" %} {% set interfaces = artifact.node.storage_id.value | artifact_content | from_json %} {% for intf in interfaces %} interface {{ intf.name }} ip address {{ intf.address }} {{ intf.mask }} no shutdown ! {% endfor %} {% endif %} {% endfor %} ``` The same approach works for YAML content: ``` {% set syslog = artifact.node.storage_id.value | artifact_content | from_yaml %} {% for server in syslog.servers %} logging host {{ server.address }} port {{ server.port }} {% endfor %} ``` ### Include file object content[​](#include-file-object-content "Direct link to Include file object content") In addition to artifacts, you can also include content from [file objects](/schema/file-object.md). There are several filters available depending on how you identify the file: | Filter | Input | Use when | | ----------------------------- | ------------------ | ------------------------------------------------------------- | | `artifact_content` | `storage_id` | Fetching artifact content | | `file_object_content` | `storage_id` | You have the file's `storage_id` from a query | | `file_object_content_by_id` | node UUID | You have the file object's node ID | | `file_object_content_by_hfid` | HFID list + `kind` | You want to reference a file by its human-friendly identifier | Example using `file_object_content`: ``` {{ file_storage_id | file_object_content }} ``` Example using `file_object_content_by_hfid`: ``` {{ ["my-banner-file"] | file_object_content_by_hfid(kind="CoreFileObject") }} ``` info The `file_object_content_by_hfid` filter requires the `kind` parameter. Omitting it produces an error. ## 4. Compose content in a Python Transformation[​](#4-compose-content-in-a-python-transformation "Direct link to 4. Compose content in a Python Transformation") If you are using a Python Transformation instead of Jinja2, you can retrieve artifact content using `self.client.object_store.get()`: ``` from infrahub_sdk.transforms import InfrahubTransform class CompositeDeviceConfig(InfrahubTransform): query = "composite_device_query" async def transform(self, data): import json device = data["InfraDevice"]["edges"][0]["node"] hostname = device["name"]["value"] sections = [] for artifact in device["artifacts"]["edges"]: storage_id = artifact["node"]["storage_id"]["value"] if storage_id: content = await self.client.object_store.get(identifier=storage_id) sections.append(content) return f"hostname {hostname}\n!\n" + "\n!\n".join(sections) ``` If you need to work with structured content, parse the string after retrieval: ``` import json storage_id = artifact["node"]["storage_id"]["value"] raw = await self.client.object_store.get(identifier=storage_id) parsed = json.loads(raw) ``` You can also retrieve file object content in a Python Transformation. The object store provides methods to fetch files by storage ID, node ID, or human-friendly identifier: ``` from infrahub_sdk.transforms import InfrahubTransform class DeviceConfigWithBanner(InfrahubTransform): query = "device_with_banner_query" async def transform(self, data): device = data["InfraDevice"]["edges"][0]["node"] hostname = device["name"]["value"] # Fetch a file object by its storage_id banner_storage_id = device["banner_file"]["node"]["storage_id"]["value"] banner = await self.client.object_store.get_file_by_storage_id( storage_id=banner_storage_id ) # Alternatively, fetch by node ID or human-friendly identifier # banner = await self.client.object_store.get_file_by_id(node_id="...") # banner = await self.client.object_store.get_file_by_hfid( # kind="CoreFileObject", hfid=["my-banner-file"] # ) return f"hostname {hostname}\nbanner motd {banner}\n" ``` ## 5. Register the composite Transformation[​](#5-register-the-composite-transformation "Direct link to 5. Register the composite Transformation") Once you have created your query and template files, define the query, Transformation, and artifact definition in your [`.infrahub.yml`](/git-integration/infrahub-yml.md) file: .infrahub.yml ``` # yaml-language-server: $schema=https://schema.infrahub.app/python-sdk/repository-config/latest.json --- queries: - name: composite_device_query file_path: "queries/composite_device.gql" jinja2_transforms: - name: composite_device_config query: composite_device_query template_path: "templates/composite_device.j2" artifact_definitions: - name: "Full device configuration" artifact_name: "full_device_config" parameters: device_name: "name__value" content_type: "text/plain" targets: "device_group" transformation: "composite_device_config" ``` ## 6. Handle errors[​](#6-handle-errors "Direct link to 6. Handle errors") You should guard against missing `storage_id` values in your templates. A source artifact that has not yet been generated will have a null `storage_id`: ``` {% set storage_id = artifact.node.storage_id.value %} {% if storage_id %} {{ storage_id | artifact_content }} {% else %} ! Section not yet available {% endif %} ``` The composition filters raise errors in the following situations: | Situation | Error | Resolution | | ------------------------------------------------ | --------------------------------- | ---------------------------------------------- | | `storage_id` is `None` or empty | Filter error: identifier required | Ensure source artifacts have been generated | | Insufficient permissions | Filter error: permission denied | Verify worker authentication configuration | | Content is not valid JSON | `from_json` filter error | Verify the source artifact produces valid JSON | | Content is not valid YAML | `from_yaml` filter error | Verify the source artifact produces valid YAML | | `kind` omitted for `file_object_content_by_hfid` | Filter error: kind is required | Pass the `kind` parameter | ## Known limitations[​](#known-limitations "Direct link to Known limitations") warning 1. **No dependency ordering**: Artifacts may generate in parallel. If artifact B includes content from artifact A, artifact A must already have been generated. There is no built-in mechanism to enforce generation order. Re-running artifact generation resolves this once all source artifacts exist. 2. **Worker context only**: The `artifact_content` and `file_object_content` family of filters require an active Infrahub client connection. They are not available during local `infrahubctl render` without a server connection. 3. **Text content only**: Only text-based content can be composed. Binary files (images, PDF documents) are not supported by these filters. 4. **Circular references**: If artifact A includes content from artifact B and artifact B includes content from artifact A, the result is undefined. Design composition pipelines as a directed acyclic graph. 5. **Not refreshed after a merge changes the source**: If artifact B inlines content from artifact A and a merge changes A, B keeps the stale inlined content. Post-merge regeneration selects the artifacts the merge affected, and nothing records that one artifact's content is embedded in another, so B is not selected. Regenerate B explicitly, or set `INFRAHUB_SELECTIVE_EXECUTION_AFTER_MERGE` to `false` to regenerate every artifact after each merge. ## Next steps[​](#next-steps "Direct link to Next steps") * [Write a Jinja2 Transformation](/learn/tutorials/transformations/build-a-jinja2-transformation.md) * [Write a Python Transformation](/learn/tutorials/transformations/build-a-python-transformation.md) * [Use artifacts](/artifacts/use.md) * [Object storage](/artifact-file-storage/overview.md) * [File objects](/schema/file-object.md) * [SDK Templating Reference](/python-sdk/reference/templating.md) --- # Artifacts An artifact is the result of a [Transformation](/transformations/overview.md) for a specific context and/or object. The following MIME types or formats are supported and will be rendered properly in Infrahub's web interface: * application/json * application/yaml * application/xml * application/hcl * text/plain * text/markdown * text/csv * image/svg+xml In Infrahub's web interface, rendering follows the format: JSON, YAML, XML, HCL, and plain text display as syntax-highlighted code; Markdown renders as formatted text, including diagrams from fenced mermaid code blocks; CSV renders as a table; and SVG renders as an image. Examples * For a network device, you can use an artifact to track the configuration generated from a Jinja template (Transformation). * For a Security Device, an artifact can be the list of rules in JSON in the format of your choice generated by a Python Transformation. * An artifact can also represent the configuration of a DNS server or the configuration of a specific Virtual IP on a load balancer. While it's always possible to generate [Transformations](/transformations/overview.md) on demand via the API, having an artifact provides additional benefits: * **Caching**: Generated artifacts are stored in the internal [object storage](/artifact-file-storage/overview.md). For resource-intensive Transformations, it will significantly reduce the load of the system if an artifact can be served from the cache instead of regenerating each time. * **Traceability**: Past values of an artifact remain available. In a future release, it will be possible to compare the value of an artifact over time. * **Peer Review**: artifacts are automatically part of the [Proposed Change](/proposed-changes/overview.md) review process. * **Database**: artifact nodes are stored in the database and other nodes can optionally have a relationship with them, which makes it possible to perform certain artifact related queries. While the content of an artifact can change, its identifier will remain the same over time. ## High level design[​](#high-level-design "Direct link to High level design") Artifacts are defined by grouping a [Transformation](/transformations/overview.md) with a [group](/groups/overview.md) of targets in an *artifact definition*. An **artifact definition** centralizes all the information required to generate an artifact. * [Group](/groups/overview.md) of targets - Specifies which objects will generate artifacts * Transformation - Defines how to process the data * Format of the output - MIME type for the generated content * Information to extract from each target that must be passed to the Transformation Groups provide flexible targeting that decouples artifact generation from specific object lists. You can add or remove objects from groups without modifying artifact definitions. See [Groups](/groups/overview.md) for details on creating target groups. From an **artifact definition** artifact nodes are created, for each target which is part of the group. The result of the Transformation is stored in the [object storage](/artifact-file-storage/overview.md). The generation of the artifacts is performed by the Task worker(s). ![Artifact architecture diagram](/assets/images/architecture.excalidraw-abfccb75f8819c4043a9288d6d2f8512.svg) ## CoreArtifactTarget[​](#coreartifacttarget "Direct link to CoreArtifactTarget") A node for which you want to generate an artifact must inherit from the `CoreArtifactTarget` generic. As a result of this, an artifacts tab will show up in the node's detailed view in the UI, which allows you to access all of the artifacts that have been generated for this node. ![artifact tab](/assets/images/node_detail_view_artifact_tab-751067199c4a6e96530153ce75c1c82c.png) ``` nodes: - name: "Device" namespace: "Infra" inherit_from: ["CoreArtifactTarget"] ``` Deletion behavior When a `CoreArtifactTarget` node is deleted, all artifacts associated with it are automatically deleted as well. ## Composing content across artifacts[​](#composing-content-across-artifacts "Direct link to Composing content across artifacts") A Transformation can reference and include the rendered content of other artifacts or file objects using built-in Jinja2 filters or the Python SDK object store API. This enables modular configuration pipelines where each artifact generates one section, and a composite artifact assembles the final result. For step-by-step instructions, see [Composing artifact content](/artifacts/content-composition.md). ## When artifacts regenerate[​](#when-artifacts-regenerate "Direct link to When artifacts regenerate") An artifact is the cached output of a Transformation, so Infrahub regenerates it when its inputs change. Four kinds of change trigger regeneration or cleanup: * **The target's data changes** — a node read by the artifact's GraphQL query is modified, so that target's artifact is regenerated. * **A new target joins the group** — an artifact is generated for the new member; existing artifacts are left untouched. * **A target leaves the group** — the former member's artifact is deleted on the next generation pass over the definition (for example after the change merges), so no stale artifact is left behind. * **The definition's code or configuration changes** — when a proposed change commits to a linked repository, Infrahub regenerates an artifact only if the change touches that definition's GraphQL query, its Transformation's [dependency closure](/transformations/overview.md#dependency-tracking-and-regeneration), or the artifact definition itself. An unrelated commit — a README edit, or a helper no Transformation uses — regenerates nothing, once the Transformation is precise enough to say so: a Python Transformation that has not declared [`watch`](/git-integration/infrahub-yml.md#declaring-extra-dependencies-with-watch) regenerates on every commit, because Infrahub cannot rule out a dependency it has no way to see. Data a Generator creates during a proposed change is not a reliable trigger for an artifact in that same pipeline: the two run concurrently, so the artifact can render before the data exists. To review artifacts built from Generator output, run the Generator on the branch before the pipeline starts. See [Ordering between Generators and artifacts](/proposed-changes/overview.md#ordering-between-generators-and-artifacts). Every regeneration decision during a proposed change is recorded in the pipeline's task log, naming the file, query, or field that triggered it. See [Understanding artifact regeneration](/proposed-changes/overview.md#understanding-artifact-regeneration). ## How artifacts relate to other features[​](#how-artifacts-relate-to-other-features "Direct link to How artifacts relate to other features") * **[Transformations](/transformations/overview.md)** define the logic that produces an artifact's content; an artifact is the cached output of a Transformation for a specific target. * **Generators** create or update graph data; some Generators consume artifacts as part of their input pipeline. * **[Object storage](/artifact-file-storage/overview.md)** is where artifact content lives on disk. See the storage topic for backend configuration and sizing. * **[Proposed Changes](/proposed-changes/overview.md)** automatically include artifact diffs in their review surface, so changes to generated artifacts are visible alongside data changes. --- # Use artifacts Generate configuration files and other artifacts by combining Infrahub data with templates. This guide shows you how to create artifacts that automatically update when your infrastructure data changes. For conceptual information about artifacts and their architecture, see [Artifacts](/artifacts/overview.md). > Assumes a working Infrahub instance, an existing Transformation ([Jinja2](/learn/tutorials/transformations/build-a-jinja2-transformation.md) or [Python](/learn/tutorials/transformations/build-a-python-transformation.md)), permission to create and modify schemas, and a Git repository connected to Infrahub. ## Enable artifact generation on your schema[​](#enable-artifact-generation-on-your-schema "Direct link to Enable artifact generation on your schema") To generate artifacts for specific nodes, modify your schema to inherit from `CoreArtifactTarget`. ``` --- version: "1.0" nodes: - name: Device namespace: Network display_label: "{{ name__value }}" inherit_from: - CoreArtifactTarget attributes: - name: name kind: Text label: Name optional: false unique: true - name: description kind: Text label: Description optional: true ``` Load the modified schema into Infrahub: ``` infrahubctl schema load /tmp/schema.yml ``` ## Create a target group[​](#create-a-target-group "Direct link to Create a target group") Create a Standard Group to define which objects will generate artifacts. 1. Navigate to the Groups section in the web interface 2. Create a new Standard Group named `DeviceGroup` 3. Add your target devices (`switch1`, `switch2`, `switch3`) as members For detailed group creation steps, see [Create a group](/groups/create.md). ## Define the artifact generation[​](#define-the-artifact-generation "Direct link to Define the artifact generation") Add an artifact definition to your repository's `.infrahub.yml` file: ``` artifact_definitions: - name: "Device configuration file" artifact_name: "device_configuration" parameters: name: "name__value" content_type: "text/plain" targets: "DeviceGroup" transformation: "device_config_transform" ``` This configuration specifies: * **name**: Machine identifier (no spaces) * **artifact\_name**: Human-readable label * **parameters**: Values passed to the Transformation query * **content\_type**: MIME type of the generated artifact * **targets**: Group containing target objects * **transformation**: Name of the Jinja2 or Python Transformation For complete `.infrahub.yml` syntax, see [infrahub.yml configuration](/git-integration/infrahub-yml.md). ## Deploy the artifact definition[​](#deploy-the-artifact-definition "Direct link to Deploy the artifact definition") Commit and push your changes to activate the artifact generation: ``` git add . git commit -m "add device_configuration artifact definition" git push origin main ``` The task workers will detect the repository change and create the artifact definition in the database. ## Verify artifact generation[​](#verify-artifact-generation "Direct link to Verify artifact generation") Check that your artifacts are being generated correctly. ### Through the web interface[​](#through-the-web-interface "Direct link to Through the web interface") 1. Navigate to **Object Management** → **Artifacts** 2. Locate your generated artifacts in the list ![Artifact view](/assets/images/artifact_view-777fb5878c3215e4ec8c75e7035564e5.png) 3. Click on an artifact to view its content ![Artifact detail](/assets/images/artifact_detail-c200bf9e9f6ab6441f26557b43449922.png) ### Through object details[​](#through-object-details "Direct link to Through object details") 1. Navigate to a specific device (for example, `switch1`) 2) Select the **Artifacts** tab 3. View all artifacts generated for this object ## Access generated artifacts[​](#access-generated-artifacts "Direct link to Access generated artifacts") Download or retrieve your artifacts using these methods: ### Web interface download[​](#web-interface-download "Direct link to Web interface download") Click the download button on any artifact detail page. ### REST API access[​](#rest-api-access "Direct link to REST API access") Download artifacts using the storage object endpoint (authentication required): ``` curl -H "X-INFRAHUB-KEY: " \ http:///api/storage/object/ ``` Copy the artifact ID from the artifact menu: ![Artifact menu](/assets/images/artifact_menu-39d8eebb0ed771074c03a119ddfaf5d1.png) ### Programmatic access[​](#programmatic-access "Direct link to Programmatic access") Query artifacts through the GraphQL API for automation workflows. ## Validation[​](#validation "Direct link to Validation") Confirm your artifact generation is working: * ✓ artifact definitions appear in the web interface * ✓ artifacts are generated for all group members * ✓ Generated content matches your template expectations * ✓ artifacts update when source data changes ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") If artifacts aren't generating: 1. Check task worker logs for processing errors 2. Verify the Transformation name matches exactly 3. Ensure group members inherit from `CoreArtifactTarget` 4. Confirm the Git repository sync is working ## Related[​](#related "Direct link to Related") * [Artifacts](/artifacts/overview.md) — concept and architecture * [Composing artifact content](/artifacts/content-composition.md) — assemble a composite artifact from other artifacts or file objects * [Write a Python Transformation](/learn/tutorials/transformations/build-a-python-transformation.md) — for non-template logic * [Transformations](/transformations/overview.md) — reshape graph data into another format * [Connect a repository](/git-integration/connect-repository.md) — automate artifact deployment --- ## [🗃Generators](/generators/overview.md) [4 items](/generators/overview.md) --- ## [🗃Immutable History](/immutable-history/overview.md) [1 item](/immutable-history/overview.md) --- # Branch-agnostic data A branch-agnostic attribute or relationship on a **branch-aware** object stores its value once and shares that single value with every branch, instead of keeping a copy per branch. Deleting the object that holds such a field on one branch therefore does not release the value — the object may still be readable on another branch, and its branch-agnostic fields must remain readable with it. For how to configure a schema element as branch-agnostic, see [Branch awareness](/schema/branch-awareness.md). ## When the value is released[​](#when-the-value-is-released "Direct link to When the value is released") To release a branch-agnostic attribute or relationship for good, the field has to be unreachable from **every** branch. There are two ways for a branch to lose its reach: * **The object is gone.** No branch holds a live object for the field any more. * **The schema no longer declares the field.** Removing the attribute or relationship from the schema releases the value without deleting any object. A single branch that still reaches the field is enough to keep the value reserved, whichever of the two is in play. Infrahub keeps the value reserved for as long as **any** branch can still reach the object, and releases it as soon as none can. ## What releases the value[​](#what-releases-the-value "Direct link to What releases the value") Infrahub re-checks every branch at each operation that could have removed the field's last reader. Any of them can turn out to be the one that releases it: * **Deleting the object** on the last branch that still held it live. * **Merging the branch the deletion was made on**, which carries that deletion to the target branch. Merging a branch that still *reads* the object releases nothing: the branch survives its own merge and goes on reading the object exactly as before. * **Rebasing a branch past the deletion**, which replays it on a base where the object is already gone, so the branch stops reading it. * **Deleting a branch** that was still reading the object. * **Removing the attribute or relationship from the schema**, once no branch declares the field any more. None of these releases the value unconditionally. Each one re-evaluates every branch, and only the operation that leaves no reader at all releases the value. ## Which branches still count[​](#which-branches-still-count "Direct link to Which branches still count") A branch still counts if the object is live on it under that branch's own view of history — including a branch created *before* the deletion, because at its point of divergence the object was still there. Such a branch goes on reading the object and its branch-agnostic values exactly as before, and nothing about the deletion is visible on it. A branch created *after* the deletion stops counting only if the deletion had already reached the branch it forked from — then the object was gone at its point of divergence. A branch that forks from somewhere the deletion has not reached, such as the default branch while the deletion still exists only on a feature branch, reads the object as live and counts like any other reader. ## What counts as gone[​](#what-counts-as-gone "Direct link to What counts as gone") * An **attribute** is released when no branch holds its object. * A **relationship** is released when no branch holds **both** of its peers. Only one peer has to go — a relationship left with a single peer is already unreachable as a relationship, so it is released even while the remaining peer is live. On the default branch with no other branch open, there is nothing to defer to: deleting the object, or removing the attribute or relationship from the schema, releases the value as part of the same operation. The same applies to an object that only ever existed on one branch: deleting it there releases the value immediately, because no other branch could read it. note Values held by a resource pool follow the same rule. An allocated [resource pool](/resource-manager/overview.md) value stays allocated while any branch can still read the object it was allocated to, and returns to the pool once the value is released. ## Related[​](#related "Direct link to Related") * [Branch awareness](/schema/branch-awareness.md) — configuring `branch: agnostic` on a schema element * [Branches](/branches/overview.md) — branch lifecycle and concepts * [Delete a branch](/branches/delete.md) — branch deletion as a release trigger * [Schema migration](/schema/migration.md) — removing an attribute or relationship from the schema * [Resource Manager](/resource-manager/overview.md) — pools that allocate values to attributes and relationships --- # Create a branch Start a new branch from the default branch (typically `main` or `master`). This creates a snapshot of the current state that can be modified independently. Creating a new branch is almost instantaneous because Infrahub uses [copy-on-write semantics](/branches/overview.md#copy-on-write-semantics) — no data is duplicated, only a pointer to the base timestamp and the delta of subsequent changes. ## When you create a branch[​](#when-you-create-a-branch "Direct link to When you create a branch") When creating a branch, you specify: * A **name** that clearly identifies the branch's purpose, for example `feature-network-redesign` or `fix-datacenter-connectivity` * An optional **description** to help team members understand its scope * Whether the branch should **synchronize with Git repositories** (`sync_with_git: true`) — see [Branch isolation modes](/branches/overview.md#branch-isolation-modes) for the differences between full branches and data-only branches ## Branch creation methods[​](#branch-creation-methods "Direct link to Branch creation methods") Branch creation can be initiated through the Infrahub user interface, GraphQL API, the Python SDK, or the `infrahubctl` CLI. The branch is created immediately and you can begin making changes once it exists. ## Related[​](#related "Direct link to Related") * [Branches](/branches/overview.md) — concepts, hierarchy, isolation modes * [Merge a branch](/branches/merge.md) — when work is complete * [Rebase a branch](/branches/rebase.md) — incorporate upstream changes * [Selective branch synchronization](/git-integration/branch-synchronization.md) — control which Git branches sync to Infrahub --- # Delete a branch After a branch has served its purpose — whether it's been merged, abandoned, or superseded — it can be deleted. Deleting a branch that has not been merged will permanently discard all changes accumulated on it. ## Ways to delete a branch[​](#ways-to-delete-a-branch "Direct link to Ways to delete a branch") Branches can be deleted in two ways: * **Manually** via the branch detail view or any Infrahub interface (CLI, GraphQL). * **Automatically after merge** when `delete_branch_after_merge` is enabled in the [configuration](/reference/configuration.md). ## Automatic branch cleanup after merge[​](#automatic-branch-cleanup-after-merge "Direct link to Automatic branch cleanup after merge") Infrahub can automatically delete a branch after it is successfully merged, keeping your branch list tidy without manual cleanup. The branch is deleted immediately after a successful merge, whether initiated through a direct branch merge or a Proposed Change. This behavior is opt-in: set `delete_branch_after_merge = true` (or `INFRAHUB_DELETE_BRANCH_AFTER_MERGE=true`) to enable it. To also remove the linked Git branch, set `delete_git_branch_after_merge = true` (`INFRAHUB_GIT_DELETE_GIT_BRANCH_AFTER_MERGE=true`). This has no effect unless Infrahub branch deletion is also enabled. ## What happens after deletion[​](#what-happens-after-deletion "Direct link to What happens after deletion") Once deleted, a branch no longer exists and cannot be interacted with. If the branch was linked to a Git repository, the deletion is propagated to Git. If the remote deletion fails, a log entry is available under the repository view in Infrahub. Deleting a branch can also release branch-agnostic values that the branch was the last to read — including [resource pool](/resource-manager/overview.md) allocations, which return to the pool. See [Branch-agnostic data](/branches/branch-agnostic-data.md). ## Related[​](#related "Direct link to Related") * [Branches](/branches/overview.md) — branch lifecycle and concepts * [Merge a branch](/branches/merge.md) — common predecessor to deletion * [Branch-agnostic data](/branches/branch-agnostic-data.md) — when deleting a branch releases a branch-agnostic value * [Configuration](/reference/configuration.md) — `delete_branch_after_merge` and `delete_git_branch_after_merge` settings --- # Merge a branch Once changes are complete, they merge back into the parent branch through a controlled process — almost always via a [Proposed Change](/proposed-changes/overview.md). This ensures that all modifications are validated and conflicts are resolved before integration. The merge process creates a permanent record in the commit history that documents what changed and why. ## What happens at merge[​](#what-happens-at-merge "Direct link to What happens at merge") Infrahub implements a **"latest-value wins"** merge strategy that optimizes for clarity and efficiency. During a merge, only the most recent value for each changed attribute transfers to the destination branch. The complete change history within the branch doesn't carry over — instead, the final state becomes a single change at the merge point. This approach provides several benefits: * **Maintains immutability** of the main branch by creating a clean, atomic change * **Simplifies conflict resolution** by focusing only on the final state rather than intermediate changes * **Provides clear merge points** in history that are straightforward to trace and understand * **Reduces storage overhead** by not duplicating the entire change history of the branch This merge strategy aligns with Infrahub's focus on the current state of infrastructure rather than the historical evolution of individual files, making it particularly well-suited for infrastructure management workflows. ## During the merge[​](#during-the-merge "Direct link to During the merge") While a merge is running, Infrahub temporarily blocks writes to keep the data consistent: * **The source branch** (the branch being merged) rejects all modifications. It transitions to the frozen state once the merge completes. * **The default branch** (the merge target) rejects modifications until the merge finishes, then becomes writable again. * **All other branches** remain fully writable. * Creating a new branch, or deleting a branch that is not involved in the merge, is always allowed. * Starting another merge or rebase is rejected while a merge is in progress. A blocked write fails with the message `A merge is currently in progress; writes are temporarily blocked. Please retry shortly.` The GraphQL error carries the structured code `MERGE_IN_PROGRESS` (HTTP status 423) in its `extensions`, along with the branch being written and the branch being merged: ``` { "extensions": { "code": "MERGE_IN_PROGRESS", "http_status": 423, "data": { "branch_name": "main", "merging_branch": "my-feature-branch" } } } ``` info The block on the default branch is transient. Clients and automation should match on the `MERGE_IN_PROGRESS` error code and retry after a short delay rather than treating the failure as permanent. See the [error catalogue](/reference/error-catalogue.md) for the full error contract. ## After the merge[​](#after-the-merge "Direct link to After the merge") After a successful merge, the source branch is automatically **frozen**: * GraphQL mutations (create, upsert, update, delete) are blocked on the merged branch * The UI disables editing controls with a visual indicator * Opening a new Proposed Change against the merged branch is prevented warning Once merged, a branch enters a frozen state and no further mutations are allowed on it. The merged source branch can be deleted manually or automatically — see [Delete a branch](/branches/delete.md) for the deletion options, including the `delete_branch_after_merge` configuration that removes branches automatically right after a successful merge. ## When a merge fails[​](#when-a-merge-fails "Direct link to When a merge fails") A merge is not a single write and it can be too large to be wrapped in a transaction. While the write protection described above is in place, the merge copies the data into the default branch and, when the source branch changed the schema, applies the schema migrations that follow from it. Only once all of that has succeeded does the merge reach its point of no return, where the source branch becomes frozen and the write protection lifts. All of that work runs on one worker, and that worker can disappear part-way through — the process crashes or is killed, its container is restarted, or the database becomes unreachable while the merge is mid-flight. Infrahub is designed so that this leaves a *recoverable* state rather than a silently half-merged default branch. The write protection is not lifted on failure: it stays in place, so nothing writes on top of a partial merge, and the branch keeps its merge status until an administrator recovers it. The trade-off is deliberate — the default branch rejects writes for longer in exchange for never presenting a partially merged graph as if the merge had succeeded. ### How a failed merge is detected[​](#how-a-failed-merge-is-detected "Direct link to How a failed merge is detected") A merge cannot report its own death, so Infrahub infers it. A background check runs once a minute and looks for a branch still in the merging state whose merge worker is no longer among the live workers. When it finds one, and the merge has been running longer than the grace period, it records the branch as having failed and escalates the write protection from the transient in-progress block to a recovery-required block. note A failed merge is therefore not flagged the instant the worker dies. Expect a few minutes to pass before the rejection changes from `MERGE_IN_PROGRESS` to `MERGE_RECOVERY_REQUIRED`. A merge whose worker is still alive is never flagged, however long it runs. ### The error clients see[​](#the-error-clients-see "Direct link to The error clients see") Once the merge is flagged as failed, writes to the default branch and to the merge source branch are rejected with the structured code `MERGE_RECOVERY_REQUIRED` (HTTP status 423): ``` { "extensions": { "code": "MERGE_RECOVERY_REQUIRED", "http_status": 423, "data": { "branch_name": "main", "merging_branch": "my-feature-branch" } } } ``` The accompanying message names the remedy directly: > A previous merge failed and left the default branch protected. Writes stay blocked until an administrator runs `infrahub recover merge`. Please contact an administrator. This is the important distinction from `MERGE_IN_PROGRESS`, which carries the same HTTP status: `MERGE_IN_PROGRESS` is transient and clears on its own, so retrying is the correct response. `MERGE_RECOVERY_REQUIRED` is durable. It does not clear with time and no amount of retrying will lift it — it requires an administrator to act. Automation should treat the two codes differently rather than retrying both. warning Client-side retry logic that matches only on HTTP 423 will retry a failed merge forever. Branch on the `code` in `extensions`, not on the status alone. ### Recovering the failed merge[​](#recovering-the-failed-merge "Direct link to Recovering the failed merge") An administrator recovers the branch with the [`infrahub recover merge`](/reference/infrahub-cli/infrahub-recover.md) CLI command, run against the Infrahub server: ``` infrahub recover merge ``` The command finds the failed merge on its own; naming a branch explicitly restricts it to that branch. It first previews what it found — the branch, when the merge started, and any associated Proposed Change — and asks for confirmation before changing anything. `--yes` skips the prompt for unattended use. Recovery reverses the partial merge rather than completing it. It rolls back every default-branch write the merge made, returns the branch and any associated Proposed Change to the open state, and lifts the write protection last, so an interruption part-way through leaves the branch protected rather than exposed. It is idempotent: running it again after a partial recovery re-detects the branch and finishes the job, and running it when there is nothing to recover reports that and makes no changes. Once recovery reports success, the default branch is writable again and the branch is back in the state it was in before the merge started. The merge can then be retried. By default, recovery acts only on a merge whose worker is confirmed dead. A merge that is stuck without an identifiable worker is ambiguous — it can look the same as a healthy merge whose bookkeeping was lost — so recovering it requires `--force`. ## Related[​](#related "Direct link to Related") * [Branches](/branches/overview.md) — branch lifecycle and concepts * [Proposed Changes](/proposed-changes/overview.md) — the recommended path for merging * [Resolve conflicts](/branches/resolve-conflicts.md) — handle conflicts before or during merge * [`infrahub recover`](/reference/infrahub-cli/infrahub-recover.md) — CLI reference for recovering a failed merge * [Error catalogue](/reference/error-catalogue.md) — the full `MERGE_IN_PROGRESS` and `MERGE_RECOVERY_REQUIRED` contracts * [Rebase a branch](/branches/rebase.md) — keep your branch up-to-date before merging * [Delete a branch](/branches/delete.md) — what happens after merging --- # Branches Infrahub's branching model allows teams to work on features, fixes, or experiments in isolated environments. Each branch represents a separate line of development, enabling parallel work streams without interference. Unlike traditional version control systems, Infrahub's approach combines the familiar Git branching concepts with the unique data management capabilities of its graph database foundation. Key aspects of branching in Infrahub: * **Isolated realities**: Each branch provides a complete, isolated environment where changes can be made without affecting other branches or the production environment. * **Powerful diffing**: Compare changes between branches to understand exactly what has been modified, added, or removed in your infrastructure data. * **Controlled integration**: Changes flow between branches through a structured merge process that includes validation and conflict resolution. * **Data integrity**: The branching system preserves data integrity through validation at key transition points. ## Use cases[​](#use-cases "Direct link to Use cases") Branching in Infrahub supports various infrastructure management workflows: * **Large scale changes**: Create a branch for a large feature or refactor, allowing for extensive changes without impacting the production pipeline. This is particularly valuable when redesigning network architectures or migrating to a new vendor. * **Collaborative review**: Create a branch for a feature, allowing team members to review and discuss changes before merging. This enables quality control and knowledge sharing within the team. * **Experimentation**: Use branches to test new ideas or approaches without affecting the production environment. This provides a safe sandbox to validate new configurations or architectures before committing to them. * **Transaction support**: Group related changes into a single branch, ensuring atomic updates and easier rollbacks. This maintains consistency when implementing interdependent infrastructure changes. Change Management Workflow Blog Post Want to see how branches can be used in a change management workflow? Read our blog post on [Infrahub's Change Management Workflow Is Built for Infrastructure Data](https://opsmill.com/blog/infrastructure-change-management-workflow/). ## Core concepts[​](#core-concepts "Direct link to Core concepts") Branches in Infrahub serve as isolated workspaces where changes can be prepared, validated, and reviewed before merging into production. Each branch maintains its own independent view of the data while sharing the underlying immutable history. Unlike traditional Git repositories that track file changes, Infrahub branches track changes to graph data representing your infrastructure. This approach provides greater flexibility when working with complex, interconnected infrastructure components while maintaining the familiar branching workflow. ### Branch hierarchy[​](#branch-hierarchy "Direct link to Branch hierarchy") Infrahub implements a hierarchical branch model where: * Branches are created from a parent branch (currently always `main`) * Changes flow back to the parent through controlled merge operations * Each branch tracks its creation point (`branched_from`) for accurate diffing Currently, Infrahub supports a single level of hierarchy—all branches must be created from and merged back to the default branch. This constraint simplifies conflict resolution and ensures clear change lineage. ### Branch isolation modes[​](#branch-isolation-modes "Direct link to Branch isolation modes") When creating a branch, you choose how it interacts with external Git repositories, allowing fine-tuned control over the scope of your changes: * **Full branches** (`sync_with_git: true`): Extend to Git repositories, allowing infrastructure-as-code changes. These branches are synchronized with corresponding Git branches, enabling coordinated changes across both Infrahub data and infrastructure code. * **Data-only branches** (`sync_with_git: false`): Isolate changes to Infrahub data without affecting Git repositories. These branches are ideal for data updates, corrections, or enhancements that don't require changes to infrastructure code. note Branch synchronization only applies to read-write repositories. Any branch created in a read-write repository will be considered a full branch and will be propagated to the Infrahub database. This flexibility allows different workflows—from quick data fixes to comprehensive infrastructure changes involving both data and code. Teams can choose the appropriate branch type based on the nature and scope of the changes they need to make. ### Branch-aware vs branch-agnostic[​](#branch-aware-vs-branch-agnostic "Direct link to Branch-aware vs branch-agnostic") Unlike Git where all content is branch-specific, Infrahub introduces a nuanced approach where some elements of the schema can bypass the branching system and be global to all branches. This design decision allows for more efficient management of system-level configurations while still maintaining isolation for user data. * **Branch-aware**: Records that are local to branches and follow the standard branching workflow. Changes to branch-aware records in one branch do not affect other branches until explicitly merged. Most user-created infrastructure data falls into this category. * **Branch-agnostic**: Records that exist globally regardless of branch. These typically include system data, global configurations, and reference information that should remain consistent across all branches. Changes to these records are immediately visible in all branches. * **Branch-local**: Records that exist only within a specific branch and are not meant to be merged to other branches. These include temporary data, experimental configurations, or branch-specific metadata that supports the workflow but isn't part of the final solution. note You can define in the schema the branch behavior for a given node or attribute of a node. Because a branch-agnostic value is stored once and shared by every branch, deleting the object that holds it on a single branch does not remove the value. It is released only once no branch can still read that object, which may take a later delete, merge, rebase or branch deletion — see [Branch-agnostic data](/branches/branch-agnostic-data.md). ### The default branch[​](#the-default-branch "Direct link to The default branch") The default branch (typically named `main`) in Infrahub has unique characteristics that distinguish it from other branches. It serves as the authoritative source of truth for your infrastructure and enforces stricter rules to maintain data integrity. #### Integrity guarantees[​](#integrity-guarantees "Direct link to Integrity guarantees") The default branch enforces comprehensive integrity constraints to ensure production data quality: * **Schema validation**: All data must conform to the current schema definitions, ensuring consistency and preventing invalid structures. * **Relationship integrity**: All references between objects must be valid, preventing dangling references or broken relationships in the infrastructure graph. * **Uniqueness constraints**: Duplicate values in unique fields are prevented, maintaining the integrity of identifiers and other unique attributes. * **Required fields**: All mandatory attributes must have values, ensuring complete records that can function properly in production environments. These protections ensure production data remains consistent, valid, and ready for deployment to real infrastructure environments. #### Performance trade-offs[​](#performance-trade-offs "Direct link to Performance trade-offs") The integrity guarantees of the main branch come with performance implications that users should be aware of: * **Sequential writes**: Certain operations serialize to prevent conflicts, ensuring data consistency at the cost of some concurrency. * **Validation overhead**: Every change undergoes comprehensive validation, adding processing time but ensuring data quality. * **Lock contention**: Concurrent modifications may experience delays as the system prevents potentially conflicting changes. Feature branches strategically relax some of these constraints to optimize for development speed and flexibility. This balanced approach maintains data integrity where it matters most (in production) while enabling rapid iteration in development branches. Full validation occurs during the merge process via Proposed Changes, catching any issues before they reach the default branch. ## Best practices[​](#best-practices "Direct link to Best practices") To get the most out of Infrahub's branching capabilities, consider these recommended practices: * **Use descriptive branch names**: Name branches in a way that clearly identifies their purpose, such as `feature-network-redesign` or `fix-datacenter-connectivity`. * **Keep branches short-lived**: Complete work and merge branches promptly to minimize drift from the main branch and reduce conflict potential. Enable `delete_branch_after_merge` in the [configuration](/reference/configuration.md) to automate cleanup of merged branches. * **Rebase before creating a Proposed Change**: Update your branch with the latest changes from the default branch to identify and resolve conflicts early. See [Rebase a branch](/branches/rebase.md). * **Use data-only branches for quick fixes**: When changes don't require infrastructure code updates, data-only branches provide a streamlined workflow. * **Leverage branch permissions**: Restrict who can create branches and merge to the default branch to maintain quality control in production environments. * **Document branch purpose**: Add clear descriptions to branches to help team members understand their purpose and scope. ## Branch lifecycle[​](#branch-lifecycle "Direct link to Branch lifecycle") Understanding the branch lifecycle helps teams manage the flow of changes through Infrahub and integrate them effectively into production environments: ![Branch lifecycle diagram](/assets/images/branch_merge.excalidraw-dd3d64f5312a3911e1d16d709937b90f.svg) 1. **Branch creation**: Establishes an isolated workspace at a specific point in time, capturing the state of the parent branch as a starting point. See [Create a branch](/branches/create.md). 2. **Development**: Changes accumulate in the branch without affecting the main branch, allowing for focused work and experimentation. 3. **Validation**: Proposed Changes run automated checks and tests to verify the integrity and functionality of modifications before integration. 4. **Conflict resolution**: Any conflicts with the main branch are identified and resolved, ensuring smooth integration of changes. See [Resolve conflicts](/branches/resolve-conflicts.md). 5. **Merge**: Approved changes integrate into the main branch through a controlled process that preserves the integrity of the production environment. While the merge runs, writes to the source branch and to the default branch are temporarily blocked to keep the data consistent — see [Merge a branch](/branches/merge.md). After the merge completes, the branch enters a **frozen state**: all further mutations (create, update, upsert, delete) are blocked on that branch, the UI disables editing controls with a visual indicator, and no new Proposed Change can be opened against it. 6. **Cleanup**: After a successful merge, the branch can be deleted manually or automatically when `delete_branch_after_merge` is enabled in the [configuration](/reference/configuration.md). If the branch is synchronized with a Git repository, enabling `delete_git_branch_after_merge` also removes the corresponding remote branch. warning Once merged, a branch enters a frozen state and no further mutations are allowed on it. Creating a new Proposed Change against a merged branch is also prevented. Once deleted, a branch enters a terminal state and can no longer be updated. Creating a Proposed Change against it, rebasing it, and triggering validation on it are no longer possible. This structured approach ensures that changes are thoroughly validated before reaching production while giving teams the flexibility to work independently. The pages below cover each branch operation in detail. ## Implementation details[​](#implementation-details "Direct link to Implementation details") ### Copy-on-write semantics[​](#copy-on-write-semantics "Direct link to Copy-on-write semantics") Branches don't duplicate the entire database. Instead, they maintain: * A pointer to their base (`branched_from` timestamp) * Only the delta of changes made within the branch * References to unchanged data in the parent branch This approach means creating a branch has minimal overhead, and storage grows only with actual changes. It also means that creating a new branch is almost instantaneous, as it doesn't involve copying large amounts of data. ## Related topics[​](#related-topics "Direct link to Related topics") * [Branching Blog Post](https://opsmill.com/blog/infrastructure-change-management-workflow/) * [Git Integration](/git-integration/overview.md) * [Proposed Changes](/proposed-changes/overview.md) * [Immutable History](/immutable-history/overview.md) * [Schema](/schema/overview.md) * [Branch awareness](/schema/branch-awareness.md) --- # Rebase a branch Rebasing is a powerful operation that updates a branch with the latest changes from its parent branch. This is essential for maintaining branch health and ensuring smooth integration of changes. note Rebasing acts on the Infrahub branch in Infrahub's versioned graph database. It is distinct from `git rebase`: it does not rebase a connected Git repository, rewrite Git history, or push to a Git remote. For how Infrahub synchronizes with connected Git repositories, see [Git integration](/git-integration/overview.md#read-only-vs-core). ## Why rebase[​](#why-rebase "Direct link to Why rebase") Rebasing serves several important purposes: * **Resolving conflicts before merge**: Identify and address conflicts in the development branch before creating a Proposed Change, reducing complexity during the review process. * **Incorporating upstream changes**: Ensure your branch includes the latest updates from the main branch, preventing drift and maintaining compatibility with recent changes. * **Maintaining a clean merge history**: Produce a cleaner, more linear history that's easier to understand and navigate. ![Branch rebase diagram](/assets/images/branch_rebase_no_conflict.excalidraw-0f871797f9e65cdf87d88a7a6f17c542.svg) ## How rebase works[​](#how-rebase-works "Direct link to How rebase works") During the rebase process, Infrahub performs the following steps: 1. The branch's base point (`branched_from`) updates to the current time, establishing a new foundation for the branch. 2. Changes in the branch are analyzed and replayed on top of the new base, incorporating the latest state from the parent branch. 3. Conflicts that arise during this process require manual resolution, allowing developers to make informed decisions about how to integrate conflicting changes. See [Resolve conflicts](/branches/resolve-conflicts.md). 4. The branch history resets, preserving only the final state. This simplification makes the branch easier to understand and work with. ## How to rebase[​](#how-to-rebase "Direct link to How to rebase") Rebasing can be initiated through the Infrahub user interface or API. ## Related[​](#related "Direct link to Related") * [Branches](/branches/overview.md) — branch lifecycle and concepts * [Resolve conflicts](/branches/resolve-conflicts.md) — when rebase surfaces conflicting changes * [Merge a branch](/branches/merge.md) — once rebase is complete and the branch is up-to-date --- # Resolve conflicts Conflicts occur when the same data changes in both a feature branch and the main branch. Infrahub's conflict management system is designed to identify these conflicts with precision and provide tools for effective resolution. ## Types of conflicts Infrahub detects[​](#types-of-conflicts-infrahub-detects "Direct link to Types of conflicts Infrahub detects") The system automatically detects several types of conflicts: * **Attribute conflicts**: The same field of an object has been modified in both branches with different values. For example, the hostname of a device is changed to `router-1` in one branch and `router-primary` in another. For `List` and `JSON`-array attributes marked [`ordered: false`](/reference/schema/attribute.md#ordered) in the schema, a difference in element order alone doesn't raise a conflict. * **Relationship conflicts**: Conflicting changes to object relationships occur when the connections between objects are modified differently in separate branches. For instance, a device might be assigned to datacenter A in one branch and datacenter B in another. * **Schema conflicts**: Incompatible schema modifications between branches can cause structural conflicts. This might happen when a field is removed in one branch but modified in another. * **Uniqueness conflicts**: Changes that would violate uniqueness constraints upon merge are flagged to prevent data integrity issues. This occurs when both branches create different objects with the same unique identifier. ![Branch conflict](/assets/images/branch_conflict.excalidraw-740cda09db4ca20c9b2fe6b13509822f.svg) ## How to resolve conflicts[​](#how-to-resolve-conflicts "Direct link to How to resolve conflicts") The [Proposed Change](/proposed-changes/overview.md) feature in Infrahub provides comprehensive tools for managing these conflicts with intuitive side-by-side diffs that clearly highlight differences between branches. For step-by-step resolution mechanics during review, see [Resolve a proposed-change conflict](/proposed-changes/resolve-conflict.md). Conflicts can also surface during a [rebase](/branches/rebase.md), in which case they require manual resolution before the rebase completes. ## Related[​](#related "Direct link to Related") * [Branches](/branches/overview.md) — branch lifecycle and concepts * [Rebase a branch](/branches/rebase.md) — surfaces conflicts early before merge * [Resolve a proposed-change conflict](/proposed-changes/resolve-conflict.md) — review-time conflict resolution --- ## [📄️Computed attributes](/computed-attributes/overview.md) [What a computed attribute is, when to use it, and how Infrahub recalculates it as your data changes.](/computed-attributes/overview.md) --- ## [📄️SSO Reference](/reference/sso.md) [This reference document describes the available SSO protocols, configuration options, and parameters in Infrahub.](/reference/sso.md) --- ## [📄️Infrahub Configuration](/reference/configuration.md) [Configuration options for Infrahub](/reference/configuration.md) --- ## [📄️Field visibility](/schema/field-visibility.md) [Infrahub schemas can declare how attributes and relationships should appear in the frontend, independently of their presence in the data model. These declarations are purely UI hints — they do not affect data access via the API, GraphQL, or forms.](/schema/field-visibility.md) --- ## [📄️Hardware requirements](/deploy-manage/install-configure/hardware-requirements.md) [This page outlines the hardware requirements for running Infrahub, including minimum and recommended specifications, enterprise sizing, cloud provider machine types, task manager database storage, and a utility for benchmarking your system's performance.](/deploy-manage/install-configure/hardware-requirements.md) --- ## [🗃Database backup](/deploy-manage/maintain-upgrade/database-backup/overview.md) [2 items](/deploy-manage/maintain-upgrade/database-backup/overview.md) --- ## [📄️Permissions](/reference/permissions.md) [Reference documentation for Infrahub's global and object permissions](/reference/permissions.md) --- ## [📄️Tasks](/deploy-manage/run-observe/tasks.md) [The Tasks system in Infrahub is designed to manage and control various backend operations with robust error reporting, improved supervision, and enhanced logging capabilities.](/deploy-manage/run-observe/tasks.md) --- ## [📄️Build your schema with AI](/schema/build-your-schema.md) [Describe a schema change in plain language and have your assistant produce the schema YAML for it, validated before it reaches you.](/schema/build-your-schema.md) --- ## [📄️Authentication](/deploy-manage/user-management/authentication.md) [Infrahub provides flexible authentication options to fit various organizational needs.](/deploy-manage/user-management/authentication.md) --- # How to implement a change approval workflow This guide walks you through implementing a change approval workflow in Infrahub. By the end, you'll have a structured process ensuring all infrastructure changes are properly reviewed and approved before being merged into production, enhancing your governance and safety practices. Enterprise Edition Some features in this guide require the Enterprise Edition of Infrahub. If you are using the Community Edition, the enforcement mechanisms of the change approval workflow will not be available, though you can still implement a process-based approach. Change Management Workflow Blog Post Want to see how branches can be used in a change management workflow? Read our blog post on [Infrahub’s Change Management Workflow Is Built for Infrastructure Data](https://opsmill.com/blog/infrastructure-change-management-workflow/). ## What you'll build[​](#what-youll-build "Direct link to What you'll build") By the end of this guide, you'll have a complete governance system for infrastructure changes that includes: * A protected main branch where direct modifications are prevented * A clear separation between requesters who propose changes and reviewers who approve them * A mandatory review process requiring explicit approval before changes can be merged * Automatic approval revocation if changes are modified after being approved ## Steps overview[​](#steps-overview "Direct link to Steps overview") 1. Set up roles and permissions for change approvers 2. Enable the change policy configuration in your Infrahub instance 3. Set up permissions for Git repositories 4. Create a proposed change ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting this guide, make sure you have: * An Infrahub instance with administrator access * Basic understanding of Infrahub's [roles and permissions system](/deploy-manage/user-management/permissions-roles/overview.md) * Familiarity with [branching](/immutable-history/overview.md) and [proposed changes](/proposed-changes/overview.md) concepts * Administrative access to the Git repository used by Infrahub * User management permissions in your Infrahub instance ## Creating users and groups[​](#creating-users-and-groups "Direct link to Creating users and groups") warning This step can be skipped if you already have appropriate users and groups configured in your Infrahub instance via SSO for instance. Adjust the following steps to match your existing user structure. We'll create a structure representing a common organizational setup with separate teams having different responsibilities: 1. Navigate to the **Admin** → **Users & Permissions** 2. Select the **Groups** tab and create the following groups: * Group name: `Support team` (responsible for day-to-day support operations) * Group name: `Network team` (responsible for network infrastructure changes) 3. Select the **Accounts** tab and create the following users: * Username: `Alice` → Assign to group: `Network team` * Username: `Charlie` → Assign to group: `Network team` * Username: `Bob` → Assign to group: `Support team` * Username: `John` → Assign to group: `Support team` Validation You should now see both groups listed in the **Groups** section and all four users in the **Users** section, with proper group assignments visible in each user's details. ## Step 1: Configure roles and permissions for change approvers[​](#step-1-configure-roles-and-permissions-for-change-approvers "Direct link to Step 1: Configure roles and permissions for change approvers") Now, we'll create and assign the necessary roles and permissions to implement our approval workflow: tip Adjust these roles and permissions to match your organization's specific needs. For a complete list of available global and object permissions, see the [Permissions documentation](/reference/permissions.md). 1. Navigate to the **Admin** → **Users & Permissions** 2. Select the **Object Permissions** tab: * Create the following permission * Namespace `Core` * Name `Proposed Change` * Action `Update` * Decision `Allow in all branches` * Create the following permission * Namespace `Core` * Name `Proposed Change` * Action `Create` * Decision `Allow in all branches` * Create the following permission * Namespace `Core` * Name `Change Comment` * Action `*` * Decision `Allow in all branches` * Create the following permission * Namespace `Core` * Name `Change Thread` * Action `*` * Decision `Allow in all branches` * Create the following permission * Namespace `Core` * Name `Thread Comment` * Action `*` * Decision `Allow in all branches` 3. Select the **Roles** tab and create the following roles: * Role name: `Restricted Access` (this role allows edits only to branches other than default) * Add group `Support team` and `Network team` * Add permission: `global:edit_default_branch:allow_all` * Add permission: `object:*:*:view:allow_all` * Add permission: `object:*:*:any:allow_other` * Add permission: `object:Core:ChangeComment:any:allow_all` * Add permission: `object:Core:ChangeThread:any:allow_all` * Add permission: `object:Core:ThreadComment:any:allow_all` * Add permission: `object:Core:ProposedChange:create:allow_all` * Role name: `Network Change Reviewer` (this role allows full review, approve and merge changes) * Add group `Network team` * Add permission: `global:review_proposed_change:allow_all` * Add permission: `object:Core:ProposedChange:update:allow_all` * Add permission: `global:merge_proposed_change:allow_all` Validation With these permissions in place: * No one can edit directly in the default branch * All changes must be made in branches and submitted as proposed changes * Only members of the Network team can review and merge changes * The Network team is autonomous and can manage the entire approval workflow ## Step 2: Enforce change policy Enterprise Edition[​](#step-2-enforce-change-policy- "Direct link to step-2-enforce-change-policy-") Now we'll configure Infrahub's change policy to enforce our approval workflow in order to: 1. Require at least one approval before a proposed change can be merged 2. Automatically revoke approvals if changes are made after approval important The configuration process will require a restart of the Infrahub server and depends on your deployment method. For detailed instructions on how to apply configuration changes to your Infrahub instance, see [How to configure Infrahub](/deploy-manage/install-configure/configure-infrahub.md). Essentially you will need to set the following variables to the desired values: ``` INFRAHUB_POLICY_REQUIRED_PROPOSED_CHANGE_APPROVALS= INFRAHUB_POLICY_REVOKE_PROPOSED_CHANGE_APPROVALS= ``` So in our case, we will set: ``` INFRAHUB_POLICY_REQUIRED_PROPOSED_CHANGE_APPROVALS=1 INFRAHUB_POLICY_REVOKE_PROPOSED_CHANGE_APPROVALS=true ``` [How to configure Infrahub../deploy-manage/install-configure/configure-infrahub](../deploy-manage/install-configure/configure-infrahub) Validation After setting up those environment variables, proposed changes should now follow the defined approval logic. ## Step 3: Secure your Git repositories[​](#step-3-secure-your-git-repositories "Direct link to Step 3: Secure your Git repositories") To complete our change approval workflow, we need to ensure that only the Infrahub service account can push changes directly to the main branch of your Git repository. This prevents users from bypassing the approval process by making direct commits. info * This step only applies if you are using Infrahub's Git integration with a read/write repository. * Please refer to your Git provider's documentation for specific instructions on configuring branch protection rules. * Adjust the settings to match your organization's requirements. **Using GitHub** 1. Go to your repository's **Settings** tab 2. Select **Branches** in the left sidebar 3. Click "Add classic protection rule" or edit existing rule for **main** 4. Configure the following settings: * Branch name pattern: `main` * Check "Require pull request reviews before merging" * Tick "Require approvals" and set "Required number of approvals before merging" to `1` * Tick "Dismiss stale pull request approvals when new commits are pushed" * In "Allow specified actors to bypass required pull requests" add your Infrahub service account to the allowed list 5. Click **Create** or **Save changes** Validation After setting up protection rules, attempt to push directly to the main branch as a regular user - you should receive an error message indicating the branch is protected. ## Step 4: Test the approval workflow[​](#step-4-test-the-approval-workflow "Direct link to Step 4: Test the approval workflow") Let's verify our workflow by creating a test change and taking it through the entire approval process: ### As a requester from Support team[​](#as-a-requester-from-support-team "Direct link to As a requester from Support team") 1. Log in as `Bob` 2. Attempt to edit an object in the main branch * You should see that direct edits are not allowed 3. Create a new branch: * Name the branch: `new-tag` * Click **Create** 4. Create a new `Tag` object: * Navigate to **Other** > **Tags** * Click **Add tag** * Fill in the following attributes: * Name: `test-tag` * Description: `This is a test tag` * Click **Save** 5. Create a proposed change: * Click **Create proposed change** in the branch view * Add a title and description for your change * Submit the proposed change * Note that you cannot merge the proposed change (because approval is required) * Note that you cannot approve the proposed change (because you can't approve your own changes) 6. Log out and log in as `John` 7. Navigate to the **Proposed changes** section * You should see Bob's proposed change listed * Note that you cannot approve the proposed change (because you don't have the permission to do so) ### As a reviewer from Network team[​](#as-a-reviewer-from-network-team "Direct link to As a reviewer from Network team") 1. Log out and log in as `Alice` 2. Navigate to the **Proposed changes** section * You should see Bob's proposed change listed 3. Review the proposed change: * Click on the proposed change to open it * Review the changes made * Add comments if necessary * Click **Approve** to approve the changes 4. Test the approval revocation: * Log back in as `Bob` * Make an additional change to the `test-tag` object * Observe that Alice's approval is automatically revoked 5. Complete the workflow: * Log back in as `Alice` * Review the updated changes * Approve the proposed change again * Click **Merge** to merge the changes into the main branch Validation After completing these steps: * The test-tag should now appear in the main branch * The change history should show the proper approval flow * The proposed change should be marked as merged You now have a fully functional change approval workflow in place! ## Advanced use cases[​](#advanced-use-cases "Direct link to Advanced use cases") ### Emergency bypass ("break glass") procedure[​](#emergency-bypass-break-glass-procedure "Direct link to Emergency bypass (\"break glass\") procedure") In emergency situations, you may need to temporarily bypass the change approval workflow to address critical issues quickly. Emergency use only This procedure should only be used in genuine emergency situations, as it bypasses important governance controls and could lead to unintended consequences. All emergency changes should be documented and reviewed after the fact. To implement an emergency bypass: 1. Use the local "admin" account or a designated emergency account with `Super Administrator` privileges 2. Log in with this account 3. Make the necessary changes directly or through an expedited proposed change success Users with `Super Administrator` permission can: * Approve their own proposed changes * Bypass the minimum approval requirements * Make direct edits to the main branch ## Related resources[​](#related-resources "Direct link to Related resources") * [Understanding proposed changes](/proposed-changes/overview.md) * [Managing users and permissions](/deploy-manage/user-management/permissions-roles/overview.md) * [Working with branches](/immutable-history/overview.md) * [Configuring Git repositories](/git-integration/connect-repository.md) * [Change Management Workflow Blog Post](https://opsmill.com/blog/infrastructure-change-management-workflow/) --- # Checks & Validation Checks are user-defined logic, stored in an [external repository linked to Infrahub](/git-integration/overview.md), that run as part of a [proposed change](/proposed-changes/overview.md). They let users perform any kind of data validation logic during a proposed change. If a check does not complete successfully, the proposed change cannot be merged. Some examples: * Name validation against a naming convention for all infrastructure components in the database * Validate that we always have a redundant WAN circuit in operational state for every site * Validate that all internet-facing interfaces have an inbound access-list associated ## High level design[​](#high-level-design "Direct link to High level design") A check is composed of two main components: * A GraphQL query that defines the input data * Check logic in the Python language that validates the data ![High level design](/assets/images/checks.excalidraw-1ca180370098e8f0521c0f362b63e627.svg) ## Targeted checks[​](#targeted-checks "Direct link to Targeted checks") Targeted checks are exactly the same as a check; the main difference is that they target a specific [group](/groups/overview.md) of nodes in Infrahub. The check will then only be executed against the nodes in that group. This approach leverages groups to enable scalable validation — instead of running checks against all objects of a type, you can focus validation on logically related objects. Groups provide flexible targeting that can be updated independently of the check definitions. An example: you want to validate that all devices for which we generate an OpenConfig artifact have the NETCONF service enabled. By targeting the group used for artifact generation, the check automatically stays synchronized with the artifact scope. See [Groups](/groups/overview.md) for details on creating target groups for your checks. ## Learn by doing[​](#learn-by-doing "Direct link to Learn by doing") Walk through [Build a check](/learn/tutorials/build-a-check.md) in Academy to set up the GraphQL query, implement the check logic, configure `.infrahub.yml`, and validate the check against a proposed change end-to-end. ## Related[​](#related "Direct link to Related") * [Proposed Changes](/proposed-changes/overview.md) — where checks run * [Repository Management](/git-integration/overview.md) — where check code lives * [Groups](/groups/overview.md) — used to target checks at specific subsets of objects --- # Computed attributes A computed attribute is a read-only `Text` or `URL` value calculated from data already modeled in Infrahub. Use a computed attribute where that value should stay synchronized with the attributes and relationships it is built from, rather than being entered and maintained separately. Define the calculation in the schema as a Jinja2 template, or reference a Python Transformation where the calculation needs cardinality-many relationships, nested relationships, or logic a template cannot express. ## When to use a computed attribute[​](#when-to-use-a-computed-attribute "Direct link to When to use a computed attribute") Computed attributes fit values that can be calculated from data you already model: * **Produce a name or identifier in the pattern another system requires.** Where a tool or a process reads a hostname, a circuit ID, an interface description, or a URL and expects a particular pattern, compute that string from the attributes and relationships holding the data it encodes — a device name from its site, role, and index, for example. * **Include data from a related object in a description.** An interface description can name the peer device, the circuit, and the role at the other end of the link. Connect the interface to a different peer, and Infrahub recalculates the description from the new relationship. * **Combine several modeled values into one string.** A circuit description can list its endpoints and their locations; a URL can combine an environment with an object identifier. * **Compute more than one string from the same data.** A node can carry several computed attributes — one name for network operations, another for asset or finance reporting — each calculated from the same attributes and relationships. Modeling site, role, platform, index, and status as separate attributes and relationships keeps each value available to query on its own, and lets you compute whatever combined string a person or a downstream system needs to read. ## Choose the right derived value[​](#choose-the-right-derived-value "Direct link to Choose the right derived value") Infrahub provides several ways to represent an object using data from its attributes and relationships. Choose the mechanism based on how the value will be used. | You need | Use | | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | | A readable representation of an object in Infrahub lists, selectors, breadcrumbs, and diffs | [`display_label`](/schema/display_label.md) | | A stable, unique way to identify or reference an object across systems | [`human_friendly_id`](/schema/nodes-and-attributes.md#human-friendly-identifier-hfid) | | A Text or URL attribute calculated from modeled data that automation, integrations, or users need to query as an attribute | Computed attribute | Use a computed attribute when the generated value needs to exist as part of the object and be available to queries, automation, or integrations. `display_label` also uses a Jinja2 template, but it defines a single label for the whole node — not an attribute you can query — and only supports one level of direct, cardinality-one relationships. Use a computed attribute instead of `display_label` when you need to query the derived value, filter on it, or feed it to automation or integrations. ## Choose between Jinja2 and Python[​](#choosing-between-jinja2-and-python "Direct link to Choose between Jinja2 and Python") Use Jinja2 when the value can be calculated from attributes on the same object and direct cardinality-one relationships. Use a Python Transformation when the calculation needs more complex logic, cardinality-many relationships, or nested relationships. | | Jinja2 | Python Transformation | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | | **Use when** | The value can be calculated from attributes and direct cardinality-one relationships | The calculation needs more complex logic, cardinality-many relationships, or nested relationships | | **Relationship support** | Direct relationships only; cardinality `many` is not supported | No equivalent relationship restriction | | **Mandatory attribute support** | Supported | Not supported; the attribute must be optional | | **Execution** | A change on the object itself is recalculated as part of the same update; a change on a related object is processed asynchronously. Recalculation is deferred if the template reads a not-yet-allocated pool-sourced attribute | Runs asynchronously on workers | Limitations Computed attributes have some inherent restrictions due to system constraints and performance considerations. Keep these in mind when designing your schema: * Only `URL` and `Text` attribute kinds are supported for computed attributes. ## When Infrahub recalculates a value[​](#when-infrahub-recalculates-a-value "Direct link to When Infrahub recalculates a value") Infrahub recalculates a computed attribute when data used by its calculation changes. The inputs Infrahub tracks, and when the updated value becomes available, depend on whether the attribute uses Jinja2 or a Python Transformation. * **Jinja2** — Infrahub tracks the attributes and direct relationships referenced by the template. When one of those inputs changes, Infrahub recalculates the computed value. Changes to inputs on the same node are recalculated as part of the same update; changes to data on a related object are processed asynchronously. If the template references a pool-sourced attribute on the same node that has not been allocated yet, Infrahub defers that recalculation until the pool assigns a value. A template that references a path the schema does not have, or the attribute itself, is rejected when the schema loads. * **Python** — Infrahub uses the Transformation's GraphQL query to determine which data changes affect the computed value, tracking only the specific fields the query reads on each node kind, not every field. Infrahub recomputes the value directly whenever the object that owns it is created or updated. A change to a related object recomputes only the owners whose Transformation already read that object — not every object of that kind. When one of those tracked changes happens, Infrahub schedules the Transformation on a worker and updates the computed attribute once the task completes. When you change a Python Transformation, Infrahub recomputes the attributes that use it. ## Jinja2 computed attributes[​](#jinja-computed-attribute "Direct link to Jinja2 computed attributes") Define a Jinja2 computed attribute directly on the attribute in your node schema. Use Jinja2 when the calculation is concise and depends only on attributes on the same object or direct cardinality-one relationships. ### Restrictions[​](#restrictions "Direct link to Restrictions") warning * Jinja2 computed attributes cannot reference relationships with cardinality `many`. * Only direct relationships are available; a relationship of a relationship cannot be referenced. * A computed attribute can be inherited from only one generic, because Infrahub cannot determine the evaluation order if multiple generics define the same computed attribute. * Infrahub evaluates computed attributes inside the API server, in a restricted execution context that only allows fully trusted filters — filters that make network calls or run regular expressions, for example, are blocked. See [Execution contexts in the SDK Templating Reference](/python-sdk/reference/templating.md#execution-contexts) for the full list of allowed filters, and `security.restrict_untrusted_jinja2_filters` in the [configuration reference](/reference/configuration.md#security) to change this default. ### Add a Jinja2 computed attribute[​](#add-a-jinja2-computed-attribute "Direct link to Add a Jinja2 computed attribute") Everything happens in the schema definition. See [Schema Creation](/learn/tutorials/build-your-first-schema.md) for how to build and load a schema. This example calculates a `description` for a `NetworkDevice` from the device's role and the name of its related site. ``` --- version: "1.0" nodes: - name: Site namespace: Location attributes: - name: name kind: Text unique: true relationships: - name: devices cardinality: many peer: NetworkDevice kind: Component - name: Device namespace: Network attributes: - name: hostname kind: Text unique: true - name: role kind: Dropdown choices: - name: core - name: edge - name: spine - name: leaf - name: description kind: Text computed_attribute: kind: Jinja2 jinja2_template: "{{ role__value|title }} device located on {{ site__name__value|lower }}" read_only: true # You must set the attribute to read-only as the value will be handled by the system optional: false # As it's a Jinja2 kind of attribute you can make it mandatory relationships: - name: site peer: LocationSite optional: false cardinality: one kind: Attribute ``` note In your template, you can utilize most of the **filters** provided by **Jinja2** and **Netutils**! For more information, please consult the [SDK Templating Reference](/python-sdk/reference/templating.md). After loading the schema, Infrahub calculates the `description` attribute from the values referenced by the template. ### Test Jinja2 computed attributes locally[​](#test-jinja2-computed-attributes-locally "Direct link to Test Jinja2 computed attributes locally") Test Jinja2 calculation logic locally with Pytest and the `Jinja2Template` class from the Infrahub Python SDK before loading the updated schema. This validates formatting, filters, and edge cases independently of the schema-loading workflow. ``` import pytest from infrahub_sdk.template import Jinja2Template INTF_INDEX_JINJA2 = "{{ \"%03d\"| format(name__value | split_interface | last | int) }}" @pytest.mark.parametrize( "intf_name,expected", [ ("Ethernet12", "012"), ("Ethernet4", "004"), ] ) async def test_intf_index(intf_name, expected): tpl = Jinja2Template(template=INTF_INDEX_JINJA2) rendered = await tpl.render(variables={"name__value": intf_name}) assert rendered == expected ``` Add test cases for the input formats and boundary conditions your template needs to support. ## Python computed attributes[​](#python-computed-attribute "Direct link to Python computed attributes") A Python computed attribute uses a [Python Transformation](/transformations/overview.md) to calculate the value. Reference the Transformation from the schema, then define the GraphQL query and Python logic in a linked repository. Use Python when the calculation needs cardinality-many relationships, nested relationships, or logic that cannot be expressed cleanly in Jinja2. Python computed attributes run asynchronously on Infrahub workers, so the updated value may not appear immediately after a dependent value changes. ### Restrictions[​](#restrictions-1 "Direct link to Restrictions") warning * Python computed attributes cannot be mandatory. * Python computed attributes run as asynchronous Transformations. Consider the amount of data queried and the work performed by the Transformation when designing the calculation. ### Add a Python computed attribute[​](#add-a-python-computed-attribute "Direct link to Add a Python computed attribute") Creating a Python computed attribute has five parts: 1. Add the computed attribute to the schema and reference the Transformation. 2. Create the GraphQL query that provides the Transformation inputs. 3. Create the Python Transformation that returns the computed string. 4. Register the query and Transformation in `.infrahub.yml`. 5. Commit the Transformation files to a Git repository connected to Infrahub. This example calculates a `description` for an `InfraCircuit` from its endpoints and their locations. #### Add the attribute to the schema[​](#add-the-attribute-to-the-schema "Direct link to Add the attribute to the schema") Here's the schema we'll use as a starting point: ``` --- version: "1.0" nodes: - name: Site namespace: Location display_label: "{{ name__value }}" attributes: - name: name kind: Text unique: true relationships: - name: circuit_endpoints cardinality: many optional: true peer: InfraCircuitEndpoint kind: Component - name: Circuit namespace: Infra display_label: "{{ circuit_id__value }}" attributes: - name: circuit_id kind: Text unique: true - name: computed_description kind: Text computed_attribute: kind: TransformPython transform: computed_circuit_description # Transform's name read_only: true # You must set the attribute to read-only as the value will be handled by the system. optional: true # As it's a Python kind of attribute it must be optional relationships: - name: endpoints peer: InfraCircuitEndpoint optional: true cardinality: many kind: Component - name: CircuitEndpoint namespace: Infra label: Circuit endpoint display_label: "{{ name__value }}" attributes: - name: name kind: Text unique: true relationships: - name: circuit peer: InfraCircuit cardinality: one kind: Attribute - name: location peer: LocationSite cardinality: one kind: Attribute ``` Load this schema into Infrahub. note You can specify a Transformation that doesn't yet exist in Infrahub, and the attribute remains empty. Once the Transformation is loaded, Infrahub automatically catches up and computes the value of this attribute. #### Create the Python Transformation[​](#create-the-python-transformation "Direct link to Create the Python Transformation") Please refer to the [Python Transformation guide](/learn/tutorials/transformations/build-a-python-transformation.md) for further details. 1. Create a GraphQL query that returns the data required by the Transformation. warning GraphQL query used to compute attribute's value requires `ID` as parameter. ``` query MyQuery($id: ID!) { MyNode(ids: [$id]) { # ... } } ``` computed\_circuit\_description.gql ``` query CircuitDescriptionQuery($id: ID!) { InfraCircuit(ids: [$id]) { edges { node { circuit_id { value } endpoints { edges { node { name { value } location { node { name { value } } } } } } } } } } ``` 2. Create the Python Transformation and return the computed value as a string. computed\_circuit\_description.py ``` from infrahub_sdk.transforms import InfrahubTransform class ComputedCircuitDescription(InfrahubTransform): query = "computed_circuit_description" url = "computed_circuit_description" async def transform(self, data): circuit_dict: dict = data["InfraCircuit"]["edges"][0]["node"] detailed_endpoints: list[str] = [] for endpoint in circuit_dict["endpoints"]["edges"]: detailed_endpoints.append( f'{endpoint["node"]["location"]["node"]["name"]["value"]}::{endpoint["node"]["name"]["value"]}' ) return f' <- {circuit_dict["circuit_id"]["value"]} -> '.join(detailed_endpoints) ``` note Make sure that the script returns a string! 3. Register the query and Transformation in `.infrahub.yml`. .infrahub.yml ``` --- python_transforms: - name: computed_circuit_description class_name: ComputedCircuitDescription file_path: computed_circuit_description.py queries: - name: computed_circuit_description file_path: computed_circuit_description.gql ``` Test the Transformation with `infrahubctl` before loading it into Infrahub. note To create a meaningful test, you may want to create: two sites, one circuit, and two endpoints connecting the circuit to the two sites. ❯ infrahubctl transform computed\_circuit\_description id=180b4900-7ea4-f4b3-3b5f-c519831a4b93 ``` "site1::endpoint-a <- circuit1 -> site2::endpoint-b" ``` 4. Commit the Transformation files to a Git repository and connect the repository to Infrahub. See [adding a repository to Infrahub](/git-integration/connect-repository.md). success The circuit's description is now automatically computed (this process may take a few seconds). Any changes to related attributes (i.e., endpoint name) will trigger a reevaluation and update the value accordingly. ## Related[​](#related "Direct link to Related") * [Display labels](/schema/display_label.md) — the readable representation of an object across the web interface and GraphQL * [Human-friendly identifier](/schema/nodes-and-attributes.md#human-friendly-identifier-hfid) — referencing an object uniquely across systems * [Transformations](/transformations/overview.md) — the Python Transformations a computed attribute can call * [Create and load schema](/schema/create-and-load.md) — loading a schema that defines a computed attribute --- # How to configure Infrahub This guide explains how to configure your Infrahub instance by setting environment variables that control various aspects of the system, including timeouts, security settings, and integration parameters. ## What you'll accomplish[​](#what-youll-accomplish "Direct link to What you'll accomplish") By following this guide, you'll learn how to: * Identify available configuration options * Set environment variables in various deployment methods * Apply configuration changes to a running Infrahub instance * Verify that your changes have been applied correctly ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting, ensure you have: * A running Infrahub instance * Command-line access to the server running Infrahub * Basic understanding of environment variables * For production deployments, plan a maintenance window as services will be restarted ## Step 1: Identify the configuration option[​](#step-1-identify-the-configuration-option "Direct link to Step 1: Identify the configuration option") Determine which configuration setting you want to change. 1. Refer to the [configuration reference](/reference/configuration.md) to find available environment variables 2. Note the exact variable name and its expected value type info For this guide, we'll use `INFRAHUB_HTTP_TIMEOUT` as an example. It affects how Infrahub interacts with external HTTP servers such as webhooks and OAuth2 providers. The default value is 10 seconds. ## Step 2: Set the variable[​](#step-2-set-the-variable "Direct link to Step 2: Set the variable") Configure the variable before starting or restarting your Infrahub containers. * Docker Compose * Kubernetes note Using a `.env` file keeps your configuration organized and allows you to track changes with version control. warning Avoid hardcoding information directly in the `docker-compose.yml` file. Part of the upgrade process consists of replacing this file, which could lead to loss of your configuration. Avoid mixing multiple configuration methods as it can lead to confusion about which settings are applied. ### Option A: use a .env file (recommended)[​](#option-a-use-a-env-file-recommended "Direct link to Option A: use a .env file (recommended)") Create or edit a `.env` file in the same directory as your `docker-compose.yml` file: .env ``` INFRAHUB_HTTP_TIMEOUT=20 ``` This method is preferred for persistent configuration as Docker Compose automatically loads variables from the `.env` file. ### Option B: export in your shell[​](#option-b-export-in-your-shell "Direct link to Option B: export in your shell") Export the environment variable in your shell session: ``` export INFRAHUB_HTTP_TIMEOUT=20 ``` This method works for temporary changes or testing, but the variable persists only for the current shell session. ### Option C: use a .toml file (not recommended)[​](#option-c-use-a-toml-file-not-recommended "Direct link to Option C: use a .toml file (not recommended)") Create or edit an `infrahub.toml` file with your configuration: infrahub.toml ``` [settings] http_timeout = 20 ``` Then reference the TOML file when starting Infrahub. The application will automatically load configuration from the TOML file. note TOML configuration files use lowercase with underscores for setting names (for example, `http_timeout`) rather than the uppercase environment variable format (for example, `INFRAHUB_HTTP_TIMEOUT`). ### Use Helm values[​](#use-helm-values "Direct link to Use Helm values") Update your `values.yaml` or provide values during upgrade: values.yaml ``` infrahubServer: infrahubServer: env: INFRAHUB_HTTP_TIMEOUT: "20" infrahubTaskWorker: infrahubTaskWorker: env: INFRAHUB_HTTP_TIMEOUT: "20" ``` Or set directly via command line: ``` helm upgrade infrahub opsmill/infrahub \ --set infrahubServer.infrahubServer.env.INFRAHUB_HTTP_TIMEOUT=20 --set infrahubTaskWorker.infrahubTaskWorker.env.INFRAHUB_HTTP_TIMEOUT=20 \ -n ``` ## Step 3: Apply the configuration changes[​](#step-3-apply-the-configuration-changes "Direct link to Step 3: Apply the configuration changes") Restart your Infrahub containers to apply the new configuration. * Docker Compose * Kubernetes note Depending on your setup, you might need to adjust the commands below to add options. Recreate and restart your Docker containers to apply the changes: ``` docker compose up -d --force-recreate ``` The `--force-recreate` flag ensures that containers are recreated even if their configuration and images haven't changed, which is necessary for applying new environment variables. warning This command restarts all services defined in your `docker-compose.yml` file. There will be a brief service interruption during the restart. ### For Helm deployments[​](#for-helm-deployments "Direct link to For Helm deployments") If you're using Helm, upgrade your release with the new values: ``` helm upgrade infrahub opsmill/infrahub \ -f values.yaml \ -n ``` Or with inline values: ``` helm upgrade infrahub opsmill/infrahub \ --set infrahubServer.infrahubServer.env.INFRAHUB_HTTP_TIMEOUT=20 --set infrahubTaskWorker.infrahubTaskWorker.env.INFRAHUB_HTTP_TIMEOUT=20 \ -n ``` warning Rolling out changes will trigger a pod restart. Depending on your deployment configuration (replicas, rolling update strategy), there may be a brief service interruption. ## Step 4: Verify the configuration[​](#step-4-verify-the-configuration "Direct link to Step 4: Verify the configuration") Confirm that your configuration change has been applied successfully. * Docker Compose * Kubernetes note Depending on your setup, you might need to adjust the commands below to match your container names if you are using a different project name for instance. Check that the environment variable is set correctly inside the container: ``` docker exec infrahub-server-1 env | grep INFRAHUB_HTTP_TIMEOUT ``` You should see output similar to: ``` INFRAHUB_HTTP_TIMEOUT=20 ``` success If you see the expected value in the output, your configuration change has been applied successfully. note Replace `` and `infrahub-server-*` with your actual namespace and pod name. First, get the name of your Infrahub pod: ``` kubectl get pods -n -l app=infrahub-server ``` Then check the environment variable inside the pod: ``` kubectl exec -n -- env | grep INFRAHUB_HTTP_TIMEOUT ``` For example: ``` kubectl exec -n infrahub infrahub-server-7d8f9b5c6d-xyz12 -- env | grep INFRAHUB_HTTP_TIMEOUT ``` You should see output similar to: ``` INFRAHUB_HTTP_TIMEOUT=20 ``` Alternatively, you can describe the pod to see all environment variables: ``` kubectl describe pod -n ``` success If you see the expected value in the output, your configuration change has been applied successfully. ## Related resources[​](#related-resources "Direct link to Related resources") * [Configuration reference](/reference/configuration.md) - Complete list of available environment variables * [Tune performance](/deploy-manage/install-configure/performance-tuning.md) - Which settings to revisit as an instance grows, and what each one costs * [Production deployment](/deploy-manage/install-configure/production-deployment/overview.md) - Best practices for deploying Infrahub * [Installing Infrahub](/deploy-manage/install-configure/install/overview.md) - Installation methods for Infrahub --- # Hardware requirements This page outlines the hardware requirements for running Infrahub, including minimum and recommended specifications, enterprise sizing, cloud provider machine types, task manager database storage, and a utility for benchmarking your system's performance. If you only want to try Infrahub, follow the [Quick Start](/overview/quickstart.md). ## General hardware requirements[​](#general-hardware-requirements "Direct link to General hardware requirements") The system on which you want to run Infrahub should meet the following requirements: | Level | CPU Cores | RAM | Storage / database (Neo4j) | | ----------- | --------- | ---- | -------------------------- | | Minimum | 6 | 12GB | SSD and/or >= 5000 IOPS | | Recommended | 8 | 16GB | SSD and/or >= 5000 IOPS | For cloud deployments, use at least the following machine types: | Cloud provider | Machine type | | --------------------------- | ------------------ | | Google Cloud Platform | c4d-standard-4 | | Amazon Web Services | m8a.xlarge | | Microsoft Azure | Standard\_D4as\_v7 | | Oracle Cloud Infrastructure | VM.Standard.E6 | | Alibaba Cloud | ecs.g9a.xlarge | ## Local development and evaluation[​](#local-development-and-evaluation "Direct link to Local development and evaluation") To evaluate Infrahub or build a proof-of-value on a single machine, use the community docker-compose deployment. The requirements below apply to that setup, not to production. warning These are starting points, not a measured specification. Real resource use depends on your dataset size and pipeline concurrency. Validate against your own data with a load test — schema load, data import, and a proposed-change merge — before you treat any figure as a hard requirement. | Resource | Minimum (light evaluation) | Recommended | Notes | | ------------- | -------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------- | | RAM | 8 GB | 16 GB | Recommended matches the general requirements above. | | CPU | 6 cores | 8 cores | A proposed-change or merge pipeline uses several cores concurrently, so provision headroom. | | Disk | 20 GB | 40 GB | SSD. Container images require 1.3 GB; data volumes grow with your data and graph history. | | Storage class | SSD | SSD | Neo4j and PostgreSQL are latency-sensitive, so the storage class matters more than a specific IOPS target. | As your dataset grows, large merges and imports slow down on Community Edition — the signal that a single machine is no longer enough. For production scale and performance, see [Community vs enterprise](/overview/community-vs-enterprise.md). ## Task manager database (PostgreSQL) storage[​](#task-manager-database-postgresql-storage "Direct link to Task manager database (PostgreSQL) storage") The storage figures in the tables above size the Neo4j graph database. The task manager (Prefect) keeps its own [PostgreSQL database](/overview/architecture.md#task-manager), which you size separately. PostgreSQL storage grows with **activity, not with the amount of data stored in Infrahub**. It is driven by how busy the instance is — the number of open proposed changes, the volume of changes per day, task execution history, and events — not by how many objects (IP addresses, devices, services) you manage. An instance with a small dataset but heavy automation can use more PostgreSQL storage than a large but mostly static one. Use the following as starting points: | Instance type | PostgreSQL storage | | --------------------------- | ------------------ | | Local or one-off | 8 GB | | Long-running or development | 20 GB | | Production | 100 GB | Infrahub periodically prunes old activity logs, so storage usage tends to plateau rather than grow without bound. Monitor disk utilization over time and resize the volume as activity increases. For what the task manager database contains and how to back it up, see the [database backup overview](/deploy-manage/maintain-upgrade/database-backup/overview.md). For the signals to watch in production, see the [FAQ](/faq.md). ## Enterprise sizing Enterprise Edition[​](#enterprise-sizing- "Direct link to enterprise-sizing-") For enterprise deployments, use these guidelines: | Enterprise Product | CPU Cores | RAM | Storage / database (Neo4j) | | ------------------ | --------- | ---- | -------------------------- | | Small | 8 | 16GB | SSD and/or >= 5000 IOPS | | Medium Data | 16 | 32GB | SSD and/or >= 5000 IOPS | | Medium Action | 16 | 32GB | SSD and/or >= 5000 IOPS | | Large Data | 32 | 64GB | SSD and/or >= 5000 IOPS | | Large Action | 32 | 64GB | SSD and/or >= 5000 IOPS | For detailed information about enterprise products and their features, see the [OpsMill pricing page](https://opsmill.com/pricing/). ## Performance benchmark utility[​](#performance-benchmark-utility "Direct link to Performance benchmark utility") To evaluate your system's performance, use our benchmarking utility. It tests CPU, memory, and disk IOPS. note Even with the right amount of CPU cores and memory, other factors such as CPU speed can impact Infrahub performance. Run the tool using Docker: ❯ docker run --pull always --rm registry.opsmill.io/opsmill/bench ``` docker run --pull always --rm registry.opsmill.io/opsmill/bench ``` Sample output: ``` latest: Pulling from opsmill/bench 09f376ebb190: Already exists 6008598ecd4d: Already exists b70c7f4d3f19: Pull complete 84e80b988953: Pull complete Digest: sha256:6f0031a0f61823cca7d319033087f80524eadffa87b1981b2e82d841ada12895 Status: Downloaded newer image for registry.opsmill.io/opsmill/bench:latest Running Disk IOPS benchmark... hold on Running CPU/Memory benchmark... hold on Benchmark results: Memory: 15611 MB - Required: 8000 MB : OK CPU Perf: 2337 - Required: 1500 : OK Disk Read IOPS: 47742 - Required: 5000 : OK Disk Write IOPS: 15885 - Required: 5000 : OK ``` --- # Install Infrahub Community Infrahub Community is deployed as a container-based architecture and can be installed using several methods. Before you install, review the [hardware requirements](/deploy-manage/install-configure/hardware-requirements.md), including the local development and evaluation sizing. * Docker compose via curl * Local development (git clone) * Kubernetes with Helm ## Using curl and Docker Compose[​](#using-curl-and-docker-compose "Direct link to Using curl and Docker Compose") To quickly spin up the latest Infrahub locally, retrieve the Docker Compose file from [infrahub.opsmill.io](https://infrahub.opsmill.io). You can also specify a specific version or the `develop` branch in the URL: * * ### Prerequisites[​](#prerequisites "Direct link to Prerequisites") * [Docker](https://docs.docker.com/engine/install/) (version 24.x minimum) * [Docker Compose](https://docs.docker.com/compose/install/) ### Start an Infrahub environment[​](#start-an-infrahub-environment "Direct link to Start an Infrahub environment") * macOS * Linux ``` curl https://infrahub.opsmill.io > docker-compose.yml docker compose -p infrahub up -d ``` ``` curl https://infrahub.opsmill.io > docker-compose.yml sudo docker compose -p infrahub up -d ``` After running the command, you should see Docker downloading the necessary images and starting the containers. success Verify that Infrahub is running by accessing the web interface or checking container status: ``` docker ps | grep infrahub ``` ### Stop and remove an Infrahub environment[​](#stop-and-remove-an-infrahub-environment "Direct link to Stop and remove an Infrahub environment") * macOS * Linux ``` curl https://infrahub.opsmill.io > docker-compose.yml docker compose -p infrahub down -v ``` ``` curl https://infrahub.opsmill.io > docker-compose.yml sudo docker compose -p infrahub down -v ``` ## Cloning the repository[​](#cloning-the-repository "Direct link to Cloning the repository") The recommended method for running Infrahub for development uses the Docker Compose files included with the project combined with helper commands defined in `invoke`. info This method is suitable for local development and demo environments. It is not recommended for production deployments. ### Prerequisites[​](#prerequisites-1 "Direct link to Prerequisites") * [uv](https://docs.astral.sh/uv/) * [Docker](https://docs.docker.com/engine/install/) (version 24.x minimum) ### Step 1: clone the repository[​](#step-1-clone-the-repository "Direct link to Step 1: clone the repository") Create the base directory for the Infrahub installation. For this guide, we'll use `/opt/infrahub`. ``` cd /opt/ ``` warning Usage of the `/opt/infrahub` directory is merely a suggestion. You can use any directory on your system, especially for development or demo purposes. ``` mkdir -p ~/source/ cd ~/source/ ``` Clone Infrahub repository into the current directory: ``` git clone --recursive https://github.com/opsmill/infrahub.git ``` success The `git clone` command should generate output similar to the following: ``` Cloning into '.'... remote: Enumerating objects: 1312, done. remote: Counting objects: 100% (1312/1312), done. remote: Compressing objects: 100% (1150/1150), done. remote: Total 1312 (delta 187), reused 691 (delta 104), pack-reused 0 Receiving objects: 100% (1312/1312), 33.37 MiB | 14.46 MiB/s, done. Resolving deltas: 100% (187/187), done. ``` ### Step 2: install dependencies[​](#step-2-install-dependencies "Direct link to Step 2: install dependencies") Navigate to the cloned Infrahub directory: ``` cd infrahub ``` Install the Python dependencies by running: ``` uv sync --all-groups ``` success You should see uv installing the required dependencies. When complete, you'll be returned to the command prompt without errors. ### Step 3: start Infrahub[​](#step-3-start-infrahub "Direct link to Step 3: start Infrahub") Start and initialize Infrahub: ``` uv run invoke demo.start ``` [Explore additional Invoke commands for development../../../development-resources/local-demo-environment](/development-resources/local-demo-environment.md) ## Using Helm and Kubernetes[​](#using-helm-and-kubernetes "Direct link to Using Helm and Kubernetes") It's possible to deploy Infrahub on Kubernetes using Helm charts. This method is suitable for production deployments and provides a more resilient architecture. [Infrahub Helm Charthttps://github.com/opsmill/infrahub-helm/tree/stable/charts/infrahub](https://github.com/opsmill/infrahub-helm/tree/stable/charts/infrahub)[ArtifactHubhttps://artifacthub.io/packages/helm/infrahub/infrahub](https://artifacthub.io/packages/helm/infrahub/infrahub) ### Prerequisites[​](#prerequisites-2 "Direct link to Prerequisites") * A Kubernetes cluster * [Helm](https://helm.sh/docs/intro/install/) installed on your system Storage persistence for testing and lab environments By default, the Helm chart disables storage persistence on all components (Neo4j, RabbitMQ, Redis) to enable quicker deployments using `emptyDir` storage and reduce requirements for test installations. However, if pods get rescheduled on the Kubernetes cluster for any reason, all data will be lost. This can happen unexpectedly due to node maintenance, resource pressure, or cluster updates. For long-running tests or lab environments, you should either: * Enable storage persistence for the Neo4j database at minimum (but also on the other components if possible) * Use Docker Compose instead, which provides more predictable behavior for non-production environments ### Production deployment requirements[​](#production-deployment-requirements "Direct link to Production deployment requirements") The following are required for production deployments using Helm: * Data persistence for the database must be enabled * Multiple replicas of the Infrahub API Server and Infrahub Task workers should be deployed: you can use the `affinity` variable to define the affinity policy for the pods * S3 storage should be configured for the Infrahub API Server, it is required if you have multiple replicas warning We do not recommend using the included dependencies (Neo4j, RabbitMQ, Redis) for production. They are present to ease deployment on non-production environments. ### Step 1: Fill in the values file[​](#step-1-fill-in-the-values-file "Direct link to Step 1: Fill in the values file") Create a `values.yml` file with the following configuration: ``` infrahubServer: replicas: 3 affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: service operator: In values: - infrahub-server topologyKey: topology.kubernetes.io/zone persistence: enabled: false ingress: enabled: true infrahubServer: env: INFRAHUB_ALLOW_ANONYMOUS_ACCESS: "true" INFRAHUB_CACHE_PORT: 6379 INFRAHUB_DB_TYPE: neo4j INFRAHUB_LOG_LEVEL: INFO INFRAHUB_PRODUCTION: "true" INFRAHUB_INITIAL_ADMIN_TOKEN: 06438eb2-8019-4776-878c-0941b1f1d1ec INFRAHUB_SECURITY_SECRET_KEY: 327f747f-efac-42be-9e73-999f08f86b92 INFRAHUB_STORAGE_DRIVER: s3 AWS_ACCESS_KEY_ID: xxxx AWS_SECRET_ACCESS_KEY: xxxx AWS_S3_BUCKET_NAME: infrahub-data AWS_S3_ENDPOINT_URL: https://s3 infrahubTaskWorker: replicas: 3 affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: service operator: In values: - infrahub-task-worker topologyKey: topology.kubernetes.io/zone neo4j: services: admin: enabled: true volumes: data: mode: dynamic dynamic: storageClassName: premium-rwo requests: storage: 100Gi ``` warning Be sure to replace the placeholder values with your actual values. See the [configuration reference](/reference/configuration.md) for details on all available environment variables. ### Step 2: install the chart[​](#step-2-install-the-chart "Direct link to Step 2: install the chart") Install using a local chart: ``` helm install infrahub -f values.yml path/to/infrahub/chart ``` Or install using the OpsMill registry: ``` helm install infrahub -f values.yml oci://registry.opsmill.io/opsmill/chart/infrahub ``` success Verify the installation by checking that all pods are running: ``` kubectl get pods -l app=infrahub ``` ### Observability (optional)[​](#observability-optional "Direct link to Observability (optional)") To deploy the observability stack (Grafana, Prometheus, Loki, Tempo, Alloy) on Kubernetes, enable it on this release or install the `infrahub-observability` Helm chart alongside it. See [Observability stack](/deploy-manage/install-configure/install/observability-stack.md). ## Load a schema[​](#load-a-schema "Direct link to Load a schema") Infrahub ships with a minimal built-in schema. After installation, load a domain schema to model your infrastructure. Use the Marketplace for pre-built schemas The [Infrahub Marketplace](https://marketplace.infrahub.app) hosts ready-to-use schemas for common domains such as DCIM, IPAM, locations, and routing. Fetch a schema with `infrahubctl marketplace get infrahub/dcim`, then load it with `infrahubctl schema load schemas/dcim.yml`. See [Infrahub Marketplace](/schema/marketplace.md) for the complete workflow. ## Related resources[​](#related-resources "Direct link to Related resources") * [Infrahub Marketplace](/schema/marketplace.md) — Fetch and load pre-built schemas * [Database backup and restore](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md) — Learn how to backup and restore your Infrahub database * [Production deployment](/deploy-manage/install-configure/production-deployment/overview.md) — Understand resilient deployment architectures --- # Install Infrahub Enterprise Enterprise Edition Infrahub Enterprise is based on the Community version, with several enhancements for: * Enterprise features * High availability * Better performance * Security hardening (Docker image, etc.) Infrahub Enterprise can be deployed using the same methods as Infrahub Community. * Docker compose via curl * Kubernetes with Helm * Bare metal ## Using curl and Docker Compose[​](#using-curl-and-docker-compose "Direct link to Using curl and Docker Compose") To quickly spin up the latest Infrahub Enterprise locally, retrieve the Docker Compose file from [infrahub.opsmill.io/enterprise](https://infrahub.opsmill.io/enterprise). You can also specify a specific version in the URL: * You can also specify a sizing preset in the URL. This will automatically configure replica count for each component according to your sizing plan: * (requires 16 GB of RAM) * (requires 32 GB of RAM) * (requires 64 GB of RAM) | Size | Total required memory | API workers | Task workers | Task manager API workers | Task manager background workers | DB heap size | DB page cache size | | ----------- | --------------------- | ----------- | ------------ | ------------------------ | ------------------------------- | ------------ | ------------------ | | small | 16 GB | 4 | 2 | 1 | 1 | 8G | 1G | | medium | 32 GB | 4 | 4 | 2 | 2 | 16G | 4G | | medium-data | 32 GB | 4 | 2 | 1 | 1 | 16G | 4G | | large | 64 GB | 4 | 8 | 4 | 2 | 31G | 16G | | large-data | 64 GB | 4 | 2 | 1 | 1 | 31G | 16G | ### Prerequisites[​](#prerequisites "Direct link to Prerequisites") * [Docker](https://docs.docker.com/engine/install/) (version 24.x minimum) * [Docker Compose](https://docs.docker.com/compose/install/) ### Start an Infrahub Enterprise environment[​](#start-an-infrahub-enterprise-environment "Direct link to Start an Infrahub Enterprise environment") * macOS * Linux ``` curl https://infrahub.opsmill.io/enterprise > docker-compose.yml docker compose -p infrahub up -d ``` ``` curl https://infrahub.opsmill.io/enterprise > docker-compose.yml sudo docker compose -p infrahub up -d ``` ### Stop and remove an Infrahub Enterprise environment[​](#stop-and-remove-an-infrahub-enterprise-environment "Direct link to Stop and remove an Infrahub Enterprise environment") * macOS * Linux ``` curl https://infrahub.opsmill.io/enterprise > docker-compose.yml docker compose -p infrahub down -v ``` ``` curl https://infrahub.opsmill.io/enterprise > docker-compose.yml sudo docker compose -p infrahub down -v ``` ### Enable observability[​](#enable-observability "Direct link to Enable observability") Add the `?observability=true` query parameter when fetching the compose file to also deploy the observability stack (Grafana, Prometheus, Loki, Tempo, Alloy): ``` curl "https://infrahub.opsmill.io/enterprise?observability=true" > docker-compose.yml ``` It combines with other parameters such as `size`. See [Observability stack](/deploy-manage/install-configure/install/observability-stack.md) for Grafana access, request tracing, and configuration. ## Using Helm and Kubernetes[​](#using-helm-and-kubernetes "Direct link to Using Helm and Kubernetes") The Enterprise Helm chart is based on the original Infrahub chart and uses it as a Helm dependency. Most configuration related to Infrahub goes inside the `infrahub` top-level key. [Infrahub Enterprise Helm Charthttps://github.com/opsmill/infrahub-helm/tree/stable/charts/infrahub-enterprise](https://github.com/opsmill/infrahub-helm/tree/stable/charts/infrahub-enterprise)[ArtifactHubhttps://artifacthub.io/packages/helm/infrahub-enterprise/infrahub-enterprise](https://artifacthub.io/packages/helm/infrahub-enterprise/infrahub-enterprise) ### Production deployment requirements[​](#production-deployment-requirements "Direct link to Production deployment requirements") The following are required for production deployments using Helm: * Data persistence for the database must be enabled * Multiple replicas of the Infrahub API Server and Infrahub Task workers should be deployed: you can use the `affinity` variable to define the affinity policy for the pods * S3 storage should be configured for the Infrahub API Server, it is required if you have multiple replicas warning We do not recommend using the included dependencies (Neo4j, RabbitMQ, Redis) for production. They are present to ease deployment on non-production environments. ### Prerequisites[​](#prerequisites-1 "Direct link to Prerequisites") * A Kubernetes cluster * [Helm](https://helm.sh/docs/intro/install/) installed on your system Storage persistence for testing and lab environments By default, the Helm chart disables storage persistence on all components (Neo4j, RabbitMQ, Redis) to enable quicker deployments using `emptyDir` storage and reduce requirements for test installations. However, if pods get rescheduled on the Kubernetes cluster for any reason, all data will be lost. This can happen unexpectedly due to node maintenance, resource pressure, or cluster updates. For long-running tests or lab environments, you should either: * Enable storage persistence for the Neo4j database at minimum (but also on the other components if possible) * Use Docker Compose instead, which provides more predictable behavior for non-production environments ### Step 1: Fill in the values file[​](#step-1-fill-in-the-values-file "Direct link to Step 1: Fill in the values file") Create a `values.yml` file with the following configuration: ``` infrahub: infrahubServer: replicas: 3 affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: service operator: In values: - infrahub-server topologyKey: topology.kubernetes.io/zone persistence: enabled: false ingress: enabled: true infrahubServer: env: INFRAHUB_ALLOW_ANONYMOUS_ACCESS: "true" INFRAHUB_CACHE_PORT: 6379 INFRAHUB_DB_TYPE: neo4j INFRAHUB_LOG_LEVEL: INFO INFRAHUB_PRODUCTION: "true" INFRAHUB_INITIAL_ADMIN_TOKEN: 06438eb2-8019-4776-878c-0941b1f1d1ec INFRAHUB_SECURITY_SECRET_KEY: 327f747f-efac-42be-9e73-999f08f86b92 INFRAHUB_STORAGE_DRIVER: s3 AWS_ACCESS_KEY_ID: xxxx AWS_SECRET_ACCESS_KEY: xxxx AWS_S3_BUCKET_NAME: infrahub-data AWS_S3_ENDPOINT_URL: https://s3 infrahubTaskWorker: replicas: 3 affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: service operator: In values: - infrahub-task-worker topologyKey: topology.kubernetes.io/zone neo4j: services: admin: enabled: true volumes: data: mode: dynamic dynamic: storageClassName: premium-rwo requests: storage: 100Gi ``` warning Be sure to replace the placeholder values with your actual values. See the [configuration reference](/reference/configuration.md) for details on all available environment variables. Starting with version 4.0.0 of the Helm chart, you can use flavored Helm charts that come pre-configured with sizing presets. These charts automatically configure replica counts and resource allocations for each component according to your sizing plan: | Flavor | Chart version example | Required memory | | ----------- | --------------------- | --------------- | | small | 4.0.0-small | 16 GB | | medium | 4.0.0-medium | 32 GB | | medium-data | 4.0.0-medium-data | 32 GB | | large | 4.0.0-large | 64 GB | | large-data | 4.0.0-large-data | 64 GB | You can review the configuration values for each sizing preset here: [Small size config presethttps://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.small.yaml](https://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.small.yaml)[Medium size config presethttps://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.medium.yaml](https://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.medium.yaml)[Medium (data) size config presethttps://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.medium-data.yaml](https://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.medium-data.yaml)[Large size config presethttps://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.large.yaml](https://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.large.yaml)[Large (data) size config presethttps://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.large-data.yaml](https://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-enterprise/values.large-data.yaml) Redis host configuration for flavored charts When using a flavored chart (sizing preset), you must also configure the Redis host for the Prefect background services. Add the following to your `values.yml`: ``` infrahub: prefect-server: backgroundServices: messaging: redis: host: "infrahub-cache-master" ``` The `host` value should match your Redis service name. If you deployed Redis using the bundled dependency, this defaults to `infrahub-cache-master`. This configuration is only required when using sizing presets — the base chart (for example, `4.0.0`) does not require it. #### Migrating from chart versions before 4.0.0[​](#migrating-from-chart-versions-before-400 "Direct link to Migrating from chart versions before 4.0.0") If you are upgrading from a Helm chart version earlier than 4.0.0, update your `values.yml` file: 1. **Remove old configuration preset values:** Delete any environment variables and settings that were previously copied from sizing presets (for example, `INFRAHUB_CACHE_ADDRESS`, `PREFECT_REDIS_MESSAGING_HOST`, replica counts, resource limits). The flavored charts now handle these automatically. 2. **Add the Redis host configuration:** Add the following to your `values.yml` file: ``` infrahub: prefect-server: backgroundServices: messaging: redis: host: "infrahub-cache-master" # Use the value previously set in INFRAHUB_CACHE_ADDRESS or PREFECT_REDIS_MESSAGING_HOST ``` ### Step 2: install the chart[​](#step-2-install-the-chart "Direct link to Step 2: install the chart") Install using a flavored chart from the OpsMill registry (recommended for production): ``` helm install infrahub -f values.yml oci://registry.opsmill.io/opsmill/chart/infrahub-enterprise --version 4.0.0-medium ``` Or install the base chart and provide your own sizing configuration: ``` helm install infrahub -f values.yml oci://registry.opsmill.io/opsmill/chart/infrahub-enterprise --version 4.0.0 ``` Install using a local chart: ``` helm install infrahub -f values.yml path/to/infrahub-enterprise/chart ``` success Verify the installation by checking that all pods are running: ``` kubectl get pods -l app=infrahub ``` ### Observability (optional)[​](#observability-optional "Direct link to Observability (optional)") To deploy the observability stack (Grafana, Prometheus, Loki, Tempo, Alloy) on Kubernetes, enable it on this release or install the `infrahub-observability` Helm chart alongside it. See [Observability stack](/deploy-manage/install-configure/install/observability-stack.md). ## Bare metal deployment[​](#bare-metal-deployment "Direct link to Bare metal deployment") Infrahub Enterprise supports bare metal installations for customers who require direct hardware deployment without containerization. To learn more about bare metal deployment options and requirements, reach out to the OpsMill team: * **Discord:** [discord.gg/opsmill](https://discord.gg/opsmill) * **Email:** * **Schedule a meeting:** [cal.com/team/opsmill/meet](https://cal.com/team/opsmill/meet) ## Load a schema[​](#load-a-schema "Direct link to Load a schema") Infrahub ships with a minimal built-in schema. After installation, load a domain schema to model your infrastructure. Use the Marketplace for pre-built schemas The [Infrahub Marketplace](https://marketplace.infrahub.app) hosts ready-to-use schemas for common domains such as DCIM, IPAM, locations, and routing. Fetch a schema with `infrahubctl marketplace get infrahub/dcim`, then load it with `infrahubctl schema load schemas/dcim.yml`. See [Infrahub Marketplace](/schema/marketplace.md) for the complete workflow. ## Related resources[​](#related-resources "Direct link to Related resources") * [Infrahub Marketplace](/schema/marketplace.md) — Fetch and load pre-built schemas * [Database backup and restore](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md) — Learn how to backup and restore your Infrahub database * [Production deployment](/deploy-manage/install-configure/production-deployment/overview.md) — Understand resilient deployment architectures --- # Observability stack Infrahub ships an observability stack that runs alongside Infrahub: * **Grafana Alloy** — collects logs and metrics * **Loki** — log storage * **Prometheus** — metric storage * **Tempo** — distributed tracing * **Grafana** — dashboards and visualization * **Prefect exporter** — task-manager metrics You can deploy it with **Docker Compose** (Infrahub Enterprise) or on **Kubernetes** with the `infrahub-observability` Helm chart. Grafana comes provisioned with pre-built dashboards and data sources for Infrahub, Neo4j, RabbitMQ, and Prefect. [infrahub-observability Helm charthttps://github.com/opsmill/infrahub-helm/tree/stable/charts/infrahub-observability](https://github.com/opsmill/infrahub-helm/tree/stable/charts/infrahub-observability) ## Docker Compose[​](#docker-compose "Direct link to Docker Compose") The Infrahub Enterprise Docker Compose deployment can include the observability stack. Add the `?observability=true` query parameter when you fetch the compose file: ``` curl "https://infrahub.opsmill.io/enterprise?observability=true" > docker-compose.yml docker compose -p infrahub up -d ``` It combines with other parameters such as a sizing preset: ``` curl "https://infrahub.opsmill.io/enterprise?size=small&observability=true" > docker-compose.yml ``` Once running, Grafana is available at with the default credentials `admin` / `admin`. Enable request tracing Infrahub can export OpenTelemetry traces so you can follow a single request as it moves across the API server, task workers, and the database. Traces are sent to the bundled Tempo instance and surfaced in Grafana under the Tempo data source. To enable it, add the following to a `.env` file alongside the compose file: ``` INFRAHUB_TRACE_ENABLE=true INFRAHUB_TRACE_EXPORTER_TYPE=otlp INFRAHUB_TRACE_EXPORTER_PROTOCOL=grpc INFRAHUB_TRACE_EXPORTER_ENDPOINT=http://infrahub-tempo:4317 INFRAHUB_TRACE_INSECURE=true ``` `INFRAHUB_TRACE_INSECURE=true` sends spans over a plaintext connection, which is what the bundled Tempo instance expects. Set it to `false` when your collector terminates TLS. ### Exporting to a collector that uses TLS[​](#exporting-to-a-collector-that-uses-tls "Direct link to Exporting to a collector that uses TLS") When the collector presents a certificate issued by a private or internal certificate authority, the default trust store cannot verify it and every export fails with `StatusCode.UNAVAILABLE` (gRPC) or a certificate verification error (HTTP). Point Infrahub at the issuing CA instead: ``` INFRAHUB_TRACE_EXPORTER_ENDPOINT=otel-collector.example.com:4317 INFRAHUB_TRACE_TLS_CA_BUNDLE=/etc/ssl/certs/internal-ca.pem ``` The bundle is a PEM file that must be readable by the Infrahub server and task workers, so mount it into both containers. With the `grpc` protocol, setting it also switches the connection to TLS, so `INFRAHUB_TRACE_INSECURE` can be left at its default. With `http/protobuf` the endpoint URL scheme is what selects TLS, so the endpoint must start with `https://`. Infrahub checks the bundle when configuration loads and refuses to start if it cannot be loaded, rather than starting and silently dropping every span. To skip certificate verification entirely in a test environment, set `INFRAHUB_TRACE_TLS_INSECURE=true`. This requires `INFRAHUB_TRACE_EXPORTER_PROTOCOL=http/protobuf` and an `https://` endpoint, because the gRPC client cannot skip verification; use a CA bundle for gRPC collectors. Never enable it in production. For the full Infrahub Enterprise Compose install, see the [Enterprise install guide](/deploy-manage/install-configure/install/enterprise.md). ## Kubernetes with Helm[​](#kubernetes-with-helm "Direct link to Kubernetes with Helm") On Kubernetes the stack is the `infrahub-observability` Helm chart, which wraps the upstream Grafana and Prometheus community charts. Deploy it bundled with your Infrahub release or as a standalone release. ### Prerequisites[​](#prerequisites "Direct link to Prerequisites") * A Kubernetes cluster (version 1.24 or later) * [Helm](https://helm.sh/docs/intro/install/) (version 3 or later) installed on your system * A persistent volume provisioner in the cluster. Loki, Prometheus, Tempo, and Grafana enable persistence by default The stack installs and runs on its own. To collect Infrahub's own metrics and Prefect task data, install it in the same namespace as an Infrahub or Infrahub Enterprise release (or set `global.infrahubReleaseName` when the release name differs). Without a reachable Infrahub, the stack still starts, but the Prefect exporter stays unhealthy and the Infrahub scrape targets report no data. ### Deploy[​](#deploy "Direct link to Deploy") Deploy the stack either bundled with your Infrahub release or as its own release. #### Bundled with Infrahub[​](#bundled-with-infrahub "Direct link to Bundled with Infrahub") The `infrahub` and `infrahub-enterprise` charts package the observability stack as a subchart, gated by `infrahub-observability.enabled` (default `false`). Enable it in your Infrahub values to deploy the stack as part of the same release and namespace — no separate install. For the `infrahub` (Community) chart: ``` # infrahub values infrahub-observability: enabled: true ``` For the `infrahub-enterprise` chart, nest the key under `infrahub`: ``` # infrahub-enterprise values infrahub: infrahub-observability: enabled: true ``` Apply the change with `helm upgrade` on your Infrahub release. Enabling the bundled stack also turns tracing on automatically: Infrahub emits traces to the bundled Tempo at `-tempo:4317` (where `` is your Infrahub release name) with no further configuration. #### Standalone release[​](#standalone-release "Direct link to Standalone release") To manage the stack on its own release lifecycle — for example, to add it next to an existing Infrahub release without changing that release — install the chart directly. The `--create-namespace` flag creates the target namespace if it does not already exist; drop it when installing into the namespace where Infrahub already runs: ``` helm install obs oci://registry.opsmill.io/opsmill/chart/infrahub-observability --version 0.1.0 -n infrahub --create-namespace ``` The release name (`obs` above) prefixes the service names referenced throughout this guide. A standalone release does not wire Infrahub's tracing for you. To send traces to its Tempo, set `global.tracing` on the Infrahub release: ``` # infrahub values global: tracing: enabled: true endpoint: "obs-tempo:4317" protocol: grpc insecure: true ``` ### Services and access[​](#services-and-access "Direct link to Services and access") Every component is exposed through a `ClusterIP` service, reachable only from inside the cluster: | Service | Purpose | | --------------------------------------------------- | ----------------------------------------------- | | `-grafana` | Grafana UI | | `-loki` | Log storage (Alloy pushes logs here) | | `-prometheus-server` | Metric storage (Alloy remote-writes here) | | `-tempo` | Trace storage (OTLP receiver on port 4317) | | `-prometheus-node-exporter` | Host metrics | | `-infrahub-observability-prefect-exporter` | Prefect metrics (scraped by Alloy on port 8000) | `` is your Infrahub release name when bundled, or the observability release name (`obs` above) when standalone. Run `kubectl get svc -n infrahub` to list the exact service names for your release. The log, metric, and trace services only need to be reachable from inside the cluster, so keep them as `ClusterIP`. Grafana is the only component intended for people to browse. To open Grafana, forward its service to your machine (replace `obs` with your Infrahub release name if you bundled the stack): ``` kubectl port-forward svc/obs-grafana 3000:80 -n infrahub ``` Then browse to and sign in with the default credentials `admin` / `admin`. To reach Grafana without port-forwarding, set `grafana.service.type` to `LoadBalancer` or `NodePort`, or enable an ingress with `grafana.ingress.enabled`. warning Change the default Grafana credentials before exposing it outside the cluster. Set `grafana.admin.existingSecret` to a secret you manage rather than relying on the default `admin` / `admin`. ### Configure the stack[​](#configure-the-stack "Direct link to Configure the stack") The component values below set the size, retention, and exposure of each part of the stack: ``` # Turn off components you don't need tempo: enabled: false # disable tracing # Persistence and retention loki: singleBinary: persistence: size: 20Gi prometheus: server: retention: 7d persistentVolume: size: 50Gi # Expose Grafana through an ingress and use a managed admin secret grafana: ingress: enabled: true admin: existingSecret: grafana-admin ``` Where you place these values depends on how you deploy: * **Standalone** — set them at the top level of the observability `values.yml`. * **Bundled** — nest the same blocks under `infrahub-observability:` (or `infrahub.infrahub-observability:` for Enterprise) in your Infrahub values: ``` # infrahub values (bundled) infrahub-observability: tempo: enabled: false grafana: ingress: enabled: true ``` global values are shared, not nested `global.*` values are Helm global values, shared across Infrahub and the bundled subchart, so they always stay at the **top level** of your values — never nest them under `infrahub-observability:`. The same applies to the Infrahub chart's `global.tracing`. When bundled, the stack resolves the Infrahub release name and namespace from the release itself, so `global.infrahubReleaseName` and `global.infrahubNamespace` are only needed for a standalone release that points at a separately named or located Infrahub. | Option | Default | Description | | -------------------------------------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | `.enabled` | `true` | Toggle any of `alloy`, `loki`, `tempo`, `grafana`, `prometheus`, `prometheus-node-exporter`, `prefectExporter` | | component persistence `size` | Loki `10Gi`, Tempo `10Gi`, Prometheus `20Gi`, Grafana `5Gi` | Persistent volume size per component | | `loki.loki.limits_config.retention_period` | `24h` | Log retention | | `tempo.tempo.retention`, `prometheus.server.retention` | `96h` | Trace and metric retention | | `grafana.service.type`, `grafana.ingress.enabled` | `ClusterIP`, `false` | How Grafana is exposed | | `grafana.adminPassword`, `grafana.admin.existingSecret` | `admin` | Grafana credentials | | `alloy.cadvisor.enabled` | `true` | Scrape per-container metrics from the kubelet cAdvisor endpoint. Disable where cluster policy forbids `nodes/proxy` access | | `tempo.tempo.metricsGenerator.enabled` | `false` | Generate request metrics from spans. Requires `tempo.tempo.metricsGenerator.remoteWriteUrl` | | `prefectExporter.enabled` | `true` | Deploy the Prefect metrics exporter | | `global.infrahubReleaseName`, `global.infrahubNamespace` | `infrahub`, release namespace | Standalone only (top-level, never nested): point the release at a separately named or located Infrahub. Auto-resolved when bundled | For the complete list of values, see the chart's `values.yaml`. [infrahub-observability values.yamlhttps://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-observability/values.yaml](https://github.com/opsmill/infrahub-helm/blob/stable/charts/infrahub-observability/values.yaml) ### Verify the deployment[​](#verify-the-deployment "Direct link to Verify the deployment") Check that the stack's pods are running: ``` kubectl get pods -n infrahub ``` You should see an Alloy pod on each node (deployed as a DaemonSet), single-instance Loki, Tempo, Prometheus, and Grafana pods, a node exporter on each node, and the Prefect exporter. The Prefect exporter only becomes healthy once it can reach the Infrahub task manager, so deploy it alongside an Infrahub release. Once Grafana is reachable, open it and confirm that the Infrahub dashboards render and that the Prometheus and Loki data sources connect successfully. To upgrade the stack later, see [Upgrade the observability stack](/deploy-manage/maintain-upgrade/upgrade/observability-stack.md). --- # Installation Infrahub Community and Enterprise are deployed as container-based architectures. The installation methods below are for non-resilient deployments suitable for development, testing, and single-node production environments. For high-availability production deployments, see [High availability deployment examples](/deploy-manage/install-configure/production-deployment/high-availability.md). ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Verify your system meets the [hardware requirements](/deploy-manage/install-configure/hardware-requirements.md) before installing Infrahub. * Each installation method has additional prerequisites listed in its own section. info Allocating more CPU cores to the Neo4j database only improves performance on Infrahub Enterprise, which uses parallel query execution. ## Installation guides[​](#installation-guides "Direct link to Installation guides") Choose the edition that matches your license: * [Install Infrahub Community](/deploy-manage/install-configure/install/community.md) — Docker Compose (curl), git clone, or Kubernetes with Helm. * [Install Infrahub Enterprise](/deploy-manage/install-configure/install/enterprise.md) — Docker Compose (curl), Kubernetes with Helm, or bare metal. ## Related resources[​](#related-resources "Direct link to Related resources") * [Database backup and restore](/deploy-manage/maintain-upgrade/database-backup/overview.md) * [High availability architecture](/overview/architecture.md#scalability-and-high-availability) * [Local demo environment](/development-resources/local-demo-environment.md) * [Hardware requirements](/deploy-manage/install-configure/hardware-requirements.md) --- # Tune performance Infrahub's defaults target a general-purpose deployment. As an instance grows — more objects in the database, more open branches, more automation — some of those defaults stop matching the workload. The settings below are the ones worth revisiting, each with the symptom it addresses and what you give up in exchange. Start with the [hardware requirements](/deploy-manage/install-configure/hardware-requirements.md): no setting compensates for an undersized machine or slow storage. Then change one setting at a time and measure the result against your own data. | Symptom | Setting | Default | | ------------------------------------------------------------- | --------------------------------------------------------------------------------------- | --------------- | | Merges queue a burst of tasks while several branches are open | `INFRAHUB_DIFF_UPDATE_AFTER_MERGE` | `true` | | Queries over a large dataset make many database round trips | `INFRAHUB_DB_QUERY_SIZE_LIMIT` | `5000` | | The database is saturated under concurrent load | `INFRAHUB_DB_MAX_CONCURRENT_QUERIES` | `0` (unlimited) | | Graph traversal queries are truncated or time out | `INFRAHUB_DB_PATH_TRAVERSAL_QUERY_TIMEOUT`, `INFRAHUB_DB_REACHABLE_NODES_QUERY_TIMEOUT` | `30`, `75` | | Hierarchy queries traverse further than your model needs | `INFRAHUB_DB_MAX_DEPTH_SEARCH_HIERARCHY` | `5` | | The task list is slow on an instance with a long task history | `INFRAHUB_WORKFLOW_FLOW_RUN_COUNT_CACHE_THRESHOLD` | `100000` | note For how to set an environment variable and apply it, see [Configure Infrahub](/deploy-manage/install-configure/configure-infrahub.md). Set these on both the Infrahub server and the task workers — both read the same configuration. ## Diff updates after a merge[​](#diff-updates-after-a-merge "Direct link to Diff updates after a merge") Merging a branch changes the branch you merged into, which makes the stored diff of every other open branch out of date. Infrahub queues a diff update for each of them. With a few branches open this is not noticeable. With dozens of long-lived branches, a single merge queues dozens of diff recalculations, and they compete with generators, artifacts, and checks for task workers. Turn the automatic update off when you routinely keep many branches open at the same time, or when merges into the default branch are frequent: .env ``` INFRAHUB_DIFF_UPDATE_AFTER_MERGE=false ``` **What you trade.** Stored diffs are no longer refreshed after a merge, so a branch diff you open later reflects the state before that merge until it is recalculated. Refresh it from the diff view, or through the `DiffUpdate` GraphQL mutation. A proposed change recalculates its own diff every time its pipeline runs, so proposed changes under active review are unaffected. ## Query size on large datasets[​](#query-size-on-large-datasets "Direct link to Query size on large datasets") Queries that can return many records fetch them in pages of `INFRAHUB_DB_QUERY_SIZE_LIMIT` records, repeating with an increasing offset until a page comes back short. Over a large dataset, one logical query becomes many round trips to the database. Diff calculation and diff storage derive their own batch sizes from the same value. Raise it when the database holds a large number of objects and the server and workers have memory to spare: .env ``` INFRAHUB_DB_QUERY_SIZE_LIMIT=10000 ``` **What you trade.** Each page is held in the process that issued the query, so peak memory per query grows with the value. Raise it in steps, and watch memory on both `infrahub-server` and the task workers before going further. ## Concurrent queries against the database[​](#concurrent-queries-against-the-database "Direct link to Concurrent queries against the database") By default Infrahub issues as many queries as the workload produces. `INFRAHUB_DB_MAX_CONCURRENT_QUERIES` caps them: before running a query, Infrahub compares the number of in-use connections in the driver pool against the limit, and while it is above, waits `INFRAHUB_DB_MAX_CONCURRENT_QUERIES_DELAY` seconds and checks again. Use it when the database is the bottleneck rather than the application. That is most often the case on Neo4j Community, which runs queries on a single core: [more cores do not increase throughput](/deploy-manage/install-configure/install/overview.md), and neither do more queries in parallel. .env ``` INFRAHUB_DB_MAX_CONCURRENT_QUERIES=50 INFRAHUB_DB_MAX_CONCURRENT_QUERIES_DELAY=0.01 ``` The `infrahub_db_last_connection_pool_usage` Prometheus metric reports the in-use connection count this limit is compared against. Use it to pick a starting value. **What you trade.** Queries above the limit wait inside the application, so request latency rises under load. A limit set too low starves the pipeline: workers wait instead of running tasks. ## Graph traversal timeouts[​](#graph-traversal-timeouts "Direct link to Graph traversal timeouts") Path traversal runs many small queries, one depth at a time. When a single query exceeds `INFRAHUB_DB_PATH_TRAVERSAL_QUERY_TIMEOUT`, the search stops and returns the paths found so far with `truncated_at_depth` set, rather than failing the request. `INFRAHUB_DB_REACHABLE_NODES_QUERY_TIMEOUT` bounds reachable-nodes queries, which abort with an error once it is exceeded. .env ``` INFRAHUB_DB_PATH_TRAVERSAL_QUERY_TIMEOUT=30 INFRAHUB_DB_REACHABLE_NODES_QUERY_TIMEOUT=75 ``` Lower them to stop exploratory traversals from occupying the database on a large graph. Raise them when legitimate queries are truncated more often than they complete. **What you trade.** A lower timeout truncates more searches, so broad queries return partial results more often. Before changing the server-side budget, narrow the search itself with the per-query limits — `max_depth`, `max_paths`, `max_results`, and the kind and namespace filters documented in the [graph traversal reference](/reference/graph-traversal.md). ## Hierarchy search depth[​](#hierarchy-search-depth "Direct link to Hierarchy search depth") `INFRAHUB_DB_MAX_DEPTH_SEARCH_HIERARCHY` bounds how far hierarchy queries traverse: the number of levels to search, applied as a maximum relationship path length of twice that value. The default is 5 and the maximum is 20. .env ``` INFRAHUB_DB_MAX_DEPTH_SEARCH_HIERARCHY=5 ``` **What you trade.** Every additional level widens the search space for hierarchy queries and for the diff engine, which uses the same bound. Raise it only when your hierarchy is deeper than the default. ## Task list on a long task history[​](#task-list-on-a-long-task-history "Direct link to Task list on a long task history") Infrahub counts task runs to paginate the task list. Counts at or above `INFRAHUB_WORKFLOW_FLOW_RUN_COUNT_CACHE_THRESHOLD` are cached for one minute; counts below it are recomputed on every request. On a busy instance the count is expensive to compute, and with the default threshold of `100000` most counts fall below it and are never cached. Set it to `0` to cache every count: .env ``` INFRAHUB_WORKFLOW_FLOW_RUN_COUNT_CACHE_THRESHOLD=0 ``` **What you trade.** A cached count can be up to a minute old, so the total shown alongside a task list lags behind tasks that started or finished within that window. ## Related resources[​](#related-resources "Direct link to Related resources") * [Configure Infrahub](/deploy-manage/install-configure/configure-infrahub.md) — how to set and apply an environment variable * [Configuration reference](/reference/configuration.md) — every available setting, with types and defaults * [Hardware requirements](/deploy-manage/install-configure/hardware-requirements.md) — sizing, cloud machine types, and the benchmark utility * [What drives Infrahub performance at scale](/faq.md#what-drives-infrahub-performance-at-scale-and-what-can-i-tune) — query patterns and pipeline bottlenecks beyond configuration * [Community vs Enterprise](/overview/community-vs-enterprise.md) — where the scaling limits of each edition sit --- # High availability This guide covers how to deploy Infrahub in a high availability configuration that eliminates single points of failure. ## How HA works in Infrahub[​](#how-ha-works-in-infrahub "Direct link to How HA works in Infrahub") A highly available Infrahub deployment runs redundant instances of every stateful and stateless component, so the system continues to operate even if individual components or availability zones fail: * **Neo4j cluster** — 3+ nodes for graph database redundancy with automatic failover * **Redis Sentinel** — automatic detection of master failures and promotion of replicas * **RabbitMQ cluster** — message bus redundancy with mirrored queues * **PostgreSQL replication** — task manager database with primary-replica setup * **Multiple API and worker instances** — stateless services distributed across availability zones * **Redundant load balancers** — fronting the API layer for traffic distribution For the full architecture diagram and deeper explanation, see [Scalability and High Availability](/overview/architecture.md#scalability-and-high-availability). The following sections show how to deploy this architecture using Terraform or Helm. ## Deployment examples[​](#deployment-examples "Direct link to Deployment examples") The following examples demonstrate how to deploy Infrahub in a highly available configuration using Kubernetes. These deployments provide resilience, scalability, and are suitable for production environments. warning These examples are for reference purposes and may require customization for your specific environment. Ensure you understand the requirements and dependencies before deploying to production. ### Using Terraform[​](#using-terraform "Direct link to Using Terraform") Example HA deployment using Terraform on a 3-node Kubernetes cluster deploy.tf ``` terraform { required_providers { kubectl = { source = "alekc/kubectl" version = "2.1.3" } } } provider "helm" { kubernetes = { config_path = "~/.kube/config" } } provider "kubernetes" { config_path = "~/.kube/config" } provider "kubectl" { config_path = "~/.kube/config" } locals { target_namespace = "infrahub" infrahub_version = "1.7.6" } ### Infrahub resource "helm_release" "infrahub_ha" { depends_on = [helm_release.cache_ha, helm_release.messagequeue_ha, helm_release.database_ha, helm_release.objectstore_ha, kubectl_manifest.taskmanagerdb_ha] name = "infrahub" chart = "oci://registry.opsmill.io/opsmill/chart/infrahub-enterprise" version = "4.2.4-small" create_namespace = true namespace = local.target_namespace values = [ < INFRAHUB_INITIAL_ADMIN_PASSWORD= INFRAHUB_INITIAL_ADMIN_TOKEN= # Generate with: uuidgen INFRAHUB_INITIAL_AGENT_TOKEN= # Generate with: uuidgen # Database & message broker security INFRAHUB_BROKER_PASSWORD= INFRAHUB_DB_PASSWORD= # TLS & Certificates INFRAHUB_DB_TLS_ENABLED=true INFRAHUB_BROKER_TLS_ENABLED=true INFRAHUB_CACHE_TLS_ENABLED=true # Only when your infrastructure uses a private CA: mount the bundle and uncomment, see "Trust a private CA" below # INFRAHUB_TLS_CA_BUNDLE=/opt/infrahub/tls/ca-bundle.pem ``` warning Never use default passwords or tokens in production. Generate strong unique values for each environment. You can generate UUIDs using the `uuidgen` command or an online UUID generator. If your infrastructure uses a private certificate authority, mount its bundle into the containers, then set `INFRAHUB_TLS_CA_BUNDLE` to the mounted path. A component can override the bundle with its own `*_TLS_CA_FILE` or `*_TLS_CA_BUNDLE` setting. See [Trust a private CA](/deploy-manage/install-configure/production-deployment/private-ca.md) for the mount and verification steps. [Complete configuration reference/reference/configuration](/reference/configuration.md) ## Step 2: Install Infrahub[​](#step-2-install-infrahub "Direct link to Step 2: Install Infrahub") Install Infrahub using your chosen deployment technology, applying your hardened configuration. info For high availability deployments on Kubernetes, use the HA manifest which includes proper replication and resource requests/limits. [Detailed installation instructions/deploy-manage/install-configure/install/overview](/deploy-manage/install-configure/install/overview.md) success Navigate to `https://your-server-address` in your browser. You should see the Infrahub login page. ## Step 3: Configure SSO (recommended)[​](#step-3-configure-sso-recommended "Direct link to Step 3: Configure SSO (recommended)") Connect Infrahub to your organization's identity provider to enhance security and simplify user management. [Detailed SSO configuration guide/deploy-manage/user-management/sso/overview](/deploy-manage/user-management/sso/overview.md) ## Step 4: Set up database backups[​](#step-4-set-up-database-backups "Direct link to Step 4: Set up database backups") Implement regular database backups to prevent data loss in case of hardware failure or other issues. [Complete backup and restore guide/deploy-manage/maintain-upgrade/database-backup/overview](/deploy-manage/maintain-upgrade/database-backup/overview.md) success Test your backup and restore process periodically to ensure it works as expected. ## Least-privilege service credentials[​](#least-privilege-service-credentials "Direct link to Least-privilege service credentials") note This section is informational and optional. Infrahub works with administrative credentials for its backing services, but it does not require them. The privileges below let you run Infrahub with restricted users if your security policy calls for it. Infrahub does not manage users, roles, or server configuration. It primarily uses application-level privileges; only optional first-start Neo4j database bootstrap requires `CREATE DATABASE`. ### Neo4j[​](#neo4j "Direct link to Neo4j") Infrahub reads and writes graph data, declares new labels, relationship types and property names, and manages its own indexes and constraints. The matching privileges are: * `ACCESS` to the database, plus read (`MATCH`) and `WRITE` on the graph * `NAME MANAGEMENT` to declare new labels, relationship types, and property names * `INDEX MANAGEMENT` and `CONSTRAINT MANAGEMENT` for the indexes and constraints it maintains info Restricting privileges per user requires Neo4j Enterprise (role-based access control). Neo4j Community supports only the built-in `neo4j` user. A dedicated role and user, created by an administrator against the `system` database: ``` CREATE ROLE infrahub IF NOT EXISTS; GRANT ACCESS ON DATABASE neo4j TO infrahub; GRANT MATCH {*} ON GRAPH neo4j TO infrahub; GRANT WRITE ON GRAPH neo4j TO infrahub; GRANT NAME MANAGEMENT ON DATABASE neo4j TO infrahub; GRANT INDEX MANAGEMENT ON DATABASE neo4j TO infrahub; GRANT CONSTRAINT MANAGEMENT ON DATABASE neo4j TO infrahub; CREATE USER infrahub IF NOT EXISTS SET PLAINTEXT PASSWORD '' CHANGE NOT REQUIRED; GRANT ROLE infrahub TO infrahub; ``` Replace `neo4j` with the database name you set in `INFRAHUB_DB_DATABASE` if you use a dedicated database. Infrahub creates that database on first start if it does not exist, which requires the `CREATE DATABASE` privilege, so pre-create it as an administrator to keep the application user restricted. If you collect telemetry, also `GRANT SHOW SERVERS ON DBMS` and `GRANT EXECUTE PROCEDURE dbms.queryJmx ON DBMS`, otherwise telemetry logs a harmless permission error. ### Redis[​](#redis "Direct link to Redis") Infrahub uses Redis for caching and distributed locks, and in distributed deployments the task manager also uses it for workflow coordination. The user needs read and write access to keys, the connection commands used to authenticate, the scripting commands used by the locks, and transactions (`MULTI`/`EXEC`) used by the task manager. It does not need administrative or destructive commands. With a Redis ACL file: users.acl ``` user default off user infrahub on > ~* resetchannels +@read +@write +@connection +@scripting +@transaction -@dangerous ``` note `+@transaction` is only exercised when the task manager is backed by Redis (distributed or high-availability deployments). It is harmless to grant otherwise. ### RabbitMQ[​](#rabbitmq "Direct link to RabbitMQ") Infrahub talks to RabbitMQ over AMQP only: it declares its own exchanges and queues and publishes and consumes messages. It does not use the management API, so the user needs `configure`, `write` and `read` on the virtual host but no administrator or management tags. ``` rabbitmqctl add_user infrahub '' rabbitmqctl set_permissions -p / infrahub ".*" ".*" ".*" # Do not set any user tags: the application does not need management access. ``` ### PostgreSQL[​](#postgresql "Direct link to PostgreSQL") The task manager stores its state in PostgreSQL and runs its own schema migrations, so its role must be able to create and change objects in its own database, but it does not need to be a cluster superuser. Create a non-superuser role that owns its database and schema: ``` CREATE ROLE prefect WITH LOGIN PASSWORD ''; ALTER DATABASE prefect OWNER TO prefect; ALTER SCHEMA public OWNER TO prefect; GRANT ALL ON SCHEMA public TO prefect; ``` Then point the task manager's database connection string at this role instead of a superuser. ### Point Infrahub at the restricted users[​](#point-infrahub-at-the-restricted-users "Direct link to Point Infrahub at the restricted users") .env ``` INFRAHUB_DB_USERNAME=infrahub INFRAHUB_DB_PASSWORD= INFRAHUB_CACHE_USERNAME=infrahub INFRAHUB_CACHE_PASSWORD= INFRAHUB_BROKER_USERNAME=infrahub INFRAHUB_BROKER_PASSWORD= ``` ## Operations and maintenance[​](#operations-and-maintenance "Direct link to Operations and maintenance") ### Upgrading Infrahub[​](#upgrading-infrahub "Direct link to Upgrading Infrahub") To upgrade to a new version of Infrahub: 1. Review the release notes for breaking changes 2. Create a full backup of your database 3. Update the container images danger Always create a backup before upgrading to ensure you can restore if needed. [Detailed upgrade procedures/deploy-manage/maintain-upgrade/upgrade/overview](/deploy-manage/maintain-upgrade/upgrade/overview.md) ## Support options[​](#support-options "Direct link to Support options") ### Community support[​](#community-support "Direct link to Community support") * GitHub Issues: [github.com/opsmill/infrahub](https://github.com/opsmill/infrahub) * Discord Community: [discord.gg/infrahub](https://discord.gg/infrahub) * Documentation: [docs.infrahub.app](https://docs.infrahub.app) ### Enterprise support Enterprise Edition[​](#enterprise-support- "Direct link to enterprise-support-") * 24/7 support with SLA guarantees * Dedicated support engineer * Professional services for deployment * Training and certification programs Contact for enterprise support. ## Related resources[​](#related-resources "Direct link to Related resources") * [Architecture overview](/overview/architecture.md) * [Configuration reference](/reference/configuration.md) * [Tune performance](/deploy-manage/install-configure/performance-tuning.md) * [High availability](/deploy-manage/install-configure/production-deployment/high-availability.md) --- # Trust a private CA This guide explains how to make Infrahub trust certificates issued by your organization's private certificate authority (CA). Once configured, every outbound TLS connection Infrahub opens, such as cloning a repository from an internal Git server, calling a webhook, reaching an SSO provider, or connecting to the database, the message broker and S3 object storage, trusts your CA without rebuilding the Infrahub Docker image. ## What you'll accomplish[​](#what-youll-accomplish "Direct link to What you'll accomplish") By following this guide, you'll learn how to: * Prepare a CA bundle in PEM format * Mount the bundle into the Infrahub containers with Docker Compose, Docker or Helm * Point every component at the bundle with a single setting * Override the bundle for one component when it needs a different CA * Verify that git and the HTTP client trust your CA ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before starting, ensure you have: * A running Infrahub instance deployed with Docker Compose, Docker or Helm * Your CA certificate chain in PEM format: the root certificate and every intermediate certificate * Command-line access to the host or cluster running Infrahub * For production deployments, a maintenance window, as the containers restart ## How Infrahub resolves the CA bundle[​](#how-infrahub-resolves-the-ca-bundle "Direct link to How Infrahub resolves the CA bundle") Infrahub picks the trust store of each outbound connection in this order: 1. The component's `*_TLS_INSECURE` setting, when enabled: certificate validation is skipped and CA settings are ignored for that component. 2. The component's own CA setting, such as `INFRAHUB_DB_TLS_CA_FILE` or `INFRAHUB_HTTP_TLS_CA_BUNDLE`. 3. The global `INFRAHUB_TLS_CA_BUNDLE` setting. 4. The system trust store of the container. Every CA setting accepts either the path of a PEM file or the PEM text itself. Infrahub writes PEM text to a file named after its content under the system temporary directory (`TMPDIR`, `/tmp` by default) at startup, so components that only read a file, such as git, use it too. Literal `\n` sequences in place of line breaks are accepted, for tooling that cannot pass multi-line values. A CA bundle replaces the system trust store for the components it applies to, it does not extend it. If a component must keep reaching services with publicly issued certificates, such as `github.com`, append the public root certificates to your bundle as shown in the first step. | Component | Connections | Component setting | | -------------- | ---------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- | | Git | Clone, fetch and push over HTTPS | `INFRAHUB_GIT_TLS_CA_FILE` | | HTTP client | Webhooks, SSO providers, telemetry, task manager, the Infrahub API reached by the task workers and the git credential helper | `INFRAHUB_HTTP_TLS_CA_BUNDLE` | | Database | Neo4j | `INFRAHUB_DB_TLS_CA_FILE` | | Message broker | RabbitMQ or NATS | `INFRAHUB_BROKER_TLS_CA_FILE` | | Cache | Redis or NATS | `INFRAHUB_CACHE_TLS_CA_FILE` | | Object storage | S3-compatible endpoint | `INFRAHUB_STORAGE_TLS_CA_FILE` | | Trace exporter | OTLP collector, when the exporter connection is encrypted | `INFRAHUB_TRACE_TLS_CA_BUNDLE` | | Log forwarding | Syslog destinations over TLS | `tls_ca_bundle` of the destination | | LDAP | Directory servers | `INFRAHUB_LDAP_TLS_CA_BUNDLE` | ## Step 1: Prepare the CA bundle[​](#step-1-prepare-the-ca-bundle "Direct link to Step 1: Prepare the CA bundle") Concatenate your root and intermediate certificates into a single PEM file: ``` cat corporate-root-ca.pem corporate-intermediate-ca.pem > ca-bundle.pem ``` If Infrahub also connects to services with publicly issued certificates, append the public root certificates. On a Debian or Ubuntu host: ``` cat /etc/ssl/certs/ca-certificates.crt >> ca-bundle.pem ``` Check that the file parses as a certificate bundle: ``` openssl crl2pkcs7 -nocrl -certfile ca-bundle.pem | openssl pkcs7 -print_certs -noout ``` ## Step 2: Mount the bundle into the containers[​](#step-2-mount-the-bundle-into-the-containers "Direct link to Step 2: Mount the bundle into the containers") Skip this step if you pass the PEM text through the setting itself, as shown in the next step. Otherwise mount the bundle read-only into both the `infrahub-server` and `task-worker` containers. Every Infrahub process validates the configured path at startup, so a container that reads the setting without the file mounted refuses to start. * Docker Compose * Docker * Helm Create a `docker-compose.override.yml` file next to your `docker-compose.yml`. Docker Compose merges it automatically and keeps the shipped `docker-compose.yml` untouched for upgrades. See the [Docker Compose documentation](https://docs.docker.com/compose/how-tos/multiple-compose-files/merge/) for details on merging files. docker-compose.override.yml ``` --- services: infrahub-server: volumes: - ./ca-bundle.pem:/opt/infrahub/tls/ca-bundle.pem:ro task-worker: volumes: - ./ca-bundle.pem:/opt/infrahub/tls/ca-bundle.pem:ro ``` Add a bind mount to every `docker run` command that starts an Infrahub server or task worker container: ``` docker run \ -v "$(pwd)/ca-bundle.pem:/opt/infrahub/tls/ca-bundle.pem:ro" \ -e INFRAHUB_TLS_CA_BUNDLE=/opt/infrahub/tls/ca-bundle.pem \ registry.opsmill.io/opsmill/infrahub:latest ``` Store the bundle in a ConfigMap and mount it through the `extraVolumes` and `extraVolumeMounts` values of the server and the task worker: ``` kubectl create configmap infrahub-ca-bundle --from-file=ca-bundle.pem ``` values.yaml ``` infrahubServer: infrahubServer: extraVolumes: - name: ca-bundle configMap: name: infrahub-ca-bundle extraVolumeMounts: - name: ca-bundle mountPath: /opt/infrahub/tls readOnly: true infrahubTaskWorker: infrahubTaskWorker: extraVolumes: - name: ca-bundle configMap: name: infrahub-ca-bundle extraVolumeMounts: - name: ca-bundle mountPath: /opt/infrahub/tls readOnly: true ``` ## Step 3: Configure the CA bundle[​](#step-3-configure-the-ca-bundle "Direct link to Step 3: Configure the CA bundle") Point Infrahub at the mounted file. The mount location is not detected automatically: the setting is the single source of truth for where the bundle is. * Docker Compose * Docker * Helm Add the setting to the `.env` file next to your `docker-compose.yml`: .env ``` INFRAHUB_TLS_CA_BUNDLE=/opt/infrahub/tls/ca-bundle.pem ``` Pass the setting with `-e INFRAHUB_TLS_CA_BUNDLE=/opt/infrahub/tls/ca-bundle.pem`, as shown in the mount step. Add the variable to the environment of both the server and the task worker: values.yaml ``` infrahubServer: infrahubServer: env: INFRAHUB_TLS_CA_BUNDLE: /opt/infrahub/tls/ca-bundle.pem infrahubTaskWorker: infrahubTaskWorker: env: INFRAHUB_TLS_CA_BUNDLE: /opt/infrahub/tls/ca-bundle.pem ``` To avoid the mount, set the variable to the PEM text instead of a path, for example from a Kubernetes Secret exposed as an environment variable: .env ``` INFRAHUB_TLS_CA_BUNDLE="-----BEGIN CERTIFICATE----- MIIB... -----END CERTIFICATE-----" ``` To give one component a different CA, set its own setting alongside the global one. The component setting wins for that component, every other component keeps using the global bundle: .env ``` INFRAHUB_TLS_CA_BUNDLE=/opt/infrahub/tls/ca-bundle.pem INFRAHUB_DB_TLS_CA_FILE=/opt/infrahub/tls/neo4j-ca.pem ``` To scope the bundle to git only, leave `INFRAHUB_TLS_CA_BUNDLE` unset and set `INFRAHUB_GIT_TLS_CA_FILE` instead. warning When a CA setting and the matching `*_TLS_INSECURE` setting are both set, the insecure setting wins and the bundle is ignored, so verification can be switched off temporarily without dropping the bundle. The trace exporter and LDAP are the exception and refuse to start with both set. Infrahub also refuses to start when you set `INFRAHUB_STORAGE_TLS_CA_FILE` on a plaintext S3 endpoint (`INFRAHUB_STORAGE_USE_SSL=false`), and the global bundle skips components whose connection is plaintext. ## Step 4: Restart and verify[​](#step-4-restart-and-verify "Direct link to Step 4: Restart and verify") Restart the containers so they pick up the new mount and setting: ``` docker compose up -d ``` Confirm that the containers started. A wrong path fails fast with a message such as `tls.ca_bundle must be the path to an existing file`: ``` docker compose ps docker compose logs infrahub-server task-worker | grep -i "ca_bundle" ``` Confirm that the task worker configured git with the bundle. Infrahub writes its git settings to the file named by `INFRAHUB_GIT_GLOBAL_CONFIG_FILE` (default `/opt/infrahub/.gitconfig`) and points git at it through `GIT_CONFIG_GLOBAL` in its own process only, so read that file explicitly: a `git config --global` call from an exec shell does not see the variable and reads a different file. ``` docker compose exec task-worker git config --file /opt/infrahub/.gitconfig --get http.sslCAInfo ``` The command prints the path of the bundle, or the path of the file Infrahub wrote under the temporary directory when the setting holds PEM text. Then add or refresh a repository hosted on your internal Git server, following [How to connect external Git repositories](/git-integration/connect-repository.md). The repository reaches the `Online` operational status instead of failing with `SSL verification failed`. To verify the HTTP client, trigger a webhook targeting an internal endpoint or log in through your SSO provider. A remaining `Unable to validate TLS certificate` error means the bundle is missing an intermediate certificate for that endpoint. Validation Infrahub now trusts certificates issued by your private CA for every outbound connection, and the stock Docker image is used as-is. ## Related resources[​](#related-resources "Direct link to Related resources") * [Configuration reference](/reference/configuration.md) - Every `*_TLS_*` setting and its default * [How to connect external Git repositories](/git-integration/connect-repository.md) - Connecting repositories, including the git-specific TLS settings * [Production deployment](/deploy-manage/install-configure/production-deployment/overview.md) - Hardened configuration checklist * [How to configure Infrahub](/deploy-manage/install-configure/configure-infrahub.md) - Setting environment variables per deployment method [Complete configuration reference/reference/configuration](/reference/configuration.md) --- # Backup and restore This guide shows you how to create comprehensive backups of your Infrahub deployment and restore them when needed. You'll learn to backup the Neo4j graph database, object storage, and task management data to ensure complete data recovery capabilities. For Neo4j cluster deployments, see [Cluster backup and restore](/deploy-manage/maintain-upgrade/database-backup/cluster-backup-and-restore.md). ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Running Infrahub deployment (Docker Compose or Kubernetes) * Administrative access to the Neo4j database * Access to the object storage location (S3 or local filesystem) * Sufficient storage space for backup files * For cluster deployments: Understanding of your cluster topology ## Create a full backup[​](#create-a-full-backup "Direct link to Create a full backup") ### Step 1: Install the backup tool[​](#step-1-install-the-backup-tool "Direct link to Step 1: Install the backup tool") * infrahub-backup CLI (Recommended) * Kubernetes Helm * Docker Compose * Remote Database Install the infrahub-backup CLI tool: ``` curl https://infrahub.opsmill.io/ops/$(uname -s)/$(uname -m)/infrahub-backup -o infrahub-backup chmod +x infrahub-backup ``` For Kubernetes deployments using Helm, see the dedicated backup guide: [Kubernetes Backup Guide](https://docs.infrahub.app/backup/guides/kubernetes-backup) If you prefer manual control, proceed to backup each component individually as described in the following steps. Alternatively, you can use the [legacy tool](#using-the-python-based-backup-utility) to backup a remote Neo4j database. ### Step 2: Backup the databases[​](#step-2-backup-the-databases "Direct link to Step 2: Backup the databases") * infrahub-backup CLI * Kubernetes Helm * Docker Compose * Remote Database Create a backup of your running Infrahub instance: ``` ./infrahub-backup create ``` The tool automatically: * Checks for running tasks before starting (use `--force` to skip) * Creates a timestamped backup archive (for example, `infrahub_backup_20250129_153045.tar.gz`) * Backs up Neo4j database with metadata (configurable with `--neo4jmetadata`) * Backs up Prefect/PostgreSQL task management database * Calculates SHA256 checksums for integrity verification note We plan to add object storage backup in a future release. Handle object storage backups separately for now. For Kubernetes deployments using Helm, see the dedicated backup guide: [Kubernetes Backup Guide](https://docs.infrahub.app/backup/guides/kubernetes-backup) Connect to your Neo4j container and create a backup: ``` # Connect as neo4j user to avoid permission issues docker exec -it -u neo4j infrahub-database-1 bash # Create backup directory and run backup mkdir -p backups neo4j-admin database backup --to-path=backups/ # Verify backup creation ls backups/ # Output: neo4j-2025-03-24T19-57-18.backup ``` Backup the Prefect PostgreSQL database containing task logs and execution history: ``` # Export Prefect database (using default credentials) docker compose exec -T task-manager-db \ pg_dump -Fc -U postgres -d prefect > prefect.dump ``` For remote database backups using the Python utility: ``` # Clone the repository or use Docker image python -m utilities.db_backup neo4j backup \ --database-url=172.28.64.1 \ /infrahub_backups # If network access issues occur, use host network python -m utilities.db_backup neo4j backup \ --host-network \ --database-url=172.28.64.1 \ /infrahub_backups ``` ### Step 3: Backup the object storage[​](#step-3-backup-the-object-storage "Direct link to Step 3: Backup the object storage") The [object storage](/artifact-file-storage/overview.md) layer holds all file content (file objects and artifacts) outside of the graph database. The graph database references this content through `storage_id` values, so both must be backed up together to maintain consistency. * S3 Storage * Local Filesystem If using S3 for object storage, use AWS CLI or your preferred S3 backup tool: ``` # Sync S3 bucket to local backup directory aws s3 sync s3://your-infrahub-bucket /backup/object_store/ ``` For local filesystem storage, copy the object storage directory: ``` # Copy object storage directory to backup location docker compose cp infrahub-server:/opt/infrahub/storage/. /backup/object_store/ ``` ## Restore from backup[​](#restore-from-backup "Direct link to Restore from backup") ### Step 1: Prepare the environment[​](#step-1-prepare-the-environment "Direct link to Step 1: Prepare the environment") Ensure Infrahub services are running before starting the restore process. You can start from a scratch/blank deployment. * infrahub-backup CLI * Kubernetes Helm * Manual Process Restore from a backup archive: ``` ./infrahub-backup restore infrahub_backup_20250129_153045.tar.gz ``` The tool automatically: * Validates backup integrity using checksums * Wipes cache and message queue data * Stops application containers * Restores PostgreSQL database first * Restores Neo4j database with metadata * Restarts all services in correct order For Kubernetes deployments using Helm, see the dedicated restore guide: [Kubernetes Restore Guide](https://docs.infrahub.app/backup/guides/kubernetes-restore) If restoring manually, follow the steps below for each component. ### Step 2: Restore the databases[​](#step-2-restore-the-databases "Direct link to Step 2: Restore the databases") * infrahub-backup CLI * Kubernetes Helm * Docker Compose * Remote Database This is automatically handled by infrahub-backup. For Kubernetes deployments using Helm, see the dedicated restore guide: [Kubernetes Restore Guide](https://docs.infrahub.app/backup/guides/kubernetes-restore) ``` # Stop app services docker compose stop task-worker infrahub-server task-manager # Copy backup directory to container docker cp database-backup infrahub-database-1:/tmp/backup # Connect to container as neo4j user docker exec -it -u neo4j infrahub-database-1 bash # Drop existing database cypher-shell -d system -u neo4j DROP DATABASE neo4j; exit; # Clean residual data rm -rf /data/databases/neo4j rm -rf /data/transactions/neo4j # Restore from backup neo4j-admin database restore \ --from-path=/tmp/backup neo4j \ --overwrite-destination=true # Recreate database cypher-shell -d system -u neo4j CREATE DATABASE neo4j; SHOW DATABASES; ``` Restore the task manager PostgreSQL database ``` # Restore Prefect database docker compose exec -T task-manager-db \ pg_restore -d postgres -U postgres --clean --create prefect.dump # Restart task manager to apply changes docker compose restart task-manager ``` ``` # Restore using Python utility python -m utilities.db_backup neo4j restore \ /infrahub_backups \ --database-cypher-port=7687 ``` ### Step 3: Restore the object storage[​](#step-3-restore-the-object-storage "Direct link to Step 3: Restore the object storage") * S3 Storage * Local Filesystem ``` # Restore S3 bucket from backup aws s3 sync /backup/object_store/ s3://your-infrahub-bucket ``` ``` # Restore object storage directory into the container docker compose cp /backup/object_store/. infrahub-server:/opt/infrahub/storage/ ``` ### Step 4: Restart Infrahub services[​](#step-4-restart-infrahub-services "Direct link to Step 4: Restart Infrahub services") * infrahub-backup CLI * Kubernetes Helm * Docker Compose This is automatically handled by infrahub-backup. For Kubernetes deployments using Helm, see the dedicated restore guide: [Kubernetes Restore Guide](https://docs.infrahub.app/backup/guides/kubernetes-restore) Restart services in the correct order to ensure proper initialization: ``` # Restart API servers first docker compose restart infrahub-server # Then restart task workers docker compose restart task-worker ``` ## Validation[​](#validation "Direct link to Validation") Verify your restoration was successful: 1. **Check database status:** ``` docker compose exec -T database cypher-shell -u neo4j \ -c "SHOW DATABASES;" ``` The Neo4j database should show as "online". 2. **Verify Infrahub API:** ``` curl http://localhost:8000/api/schema/summary ``` You should receive a valid schema response. 3. **Check task manager:** ``` docker compose logs task-manager --tail 50 ``` Logs should show normal operation without errors. 4. **Test artifact retrieval:** Access the Infrahub UI and verify that stored artifacts (Transformations, queries) are accessible. ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Using the Python-based backup utility[​](#using-the-python-based-backup-utility "Direct link to Using the Python-based backup utility") Legacy Tool The Python-based utility (`utilities/db_backup`) is still available in the main Infrahub repository but is being replaced by infrahub-backup. Use it only if infrahub-backup doesn't meet your specific requirements. ### Use non-default ports[​](#use-non-default-ports "Direct link to Use non-default ports") If your deployment uses custom ports, specify them during backup and restore operations: ``` # Backup with custom backup port python -m utilities.db_backup neo4j backup \ --database-backup-port=12345 \ /infrahub_backups # Restore with custom Cypher port python -m utilities.db_backup neo4j restore \ /infrahub_backups \ --database-cypher-port=9876 ``` ### Run backup tool via Docker[​](#run-backup-tool-via-docker "Direct link to Run backup tool via Docker") If you don't have the repository cloned locally, run the backup tool directly from the Infrahub Docker image: ``` docker run --rm \ -v /var/run/docker.sock:/var/run/docker.sock \ registry.opsmill.io/opsmill/infrahub \ python -m utilities.db_backup ``` ## Related resources[​](#related-resources "Direct link to Related resources") * [Database backup overview](/deploy-manage/maintain-upgrade/database-backup/overview.md) - Architecture and backup strategy concepts * [Cluster backup and restore](/deploy-manage/maintain-upgrade/database-backup/cluster-backup-and-restore.md) - Neo4j cluster-specific backup and restore * [infrahub-backup CLI reference](https://docs.infrahub.app/backup/) - Command-line reference for the infrahub-backup tool --- # Cluster backup and restore Enterprise Edition If you're running Infrahub with a Neo4j cluster, follow these steps to backup from one node and restore to another while maintaining cluster integrity. For standalone deployments, see [Backup and restore](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md). ## Prerequisites for cluster operations[​](#prerequisites-for-cluster-operations "Direct link to Prerequisites for cluster operations") * Neo4j cluster with at least 3 nodes * Administrative access to all cluster nodes * Understanding of your cluster topology (leader and follower nodes) caution Always run backup and restore commands as the `neo4j` user inside containers to avoid permission issues with data files. Example cluster topology | Node | Role | | ---------------- | -------- | | `database` | Leader | | `database-core2` | Follower | | `database-core3` | Follower | ## Backup and restore within a cluster[​](#backup-and-restore-within-a-cluster "Direct link to Backup and restore within a cluster") ### Step 1: Create backup from a follower node[​](#step-1-create-backup-from-a-follower-node "Direct link to Step 1: Create backup from a follower node") ``` docker exec -it -u neo4j infrahub-database-core2-1 bash mkdir -p backups neo4j-admin database backup --to-path=backups/ neo4j ls backups # Output should include: # neo4j-2025-03-24T19-57-18.backup ``` ### Step 2: Transfer backup to target node[​](#step-2-transfer-backup-to-target-node "Direct link to Step 2: Transfer backup to target node") ``` # Copy from source container to local docker cp infrahub-database-core2-1:/var/lib/neo4j/backups/neo4j-2025-03-24T19-57-18.backup . # Copy from local to target container docker cp neo4j-2025-03-24T19-57-18.backup \ infrahub-database-core3-1:/var/lib/neo4j/ ``` ### Step 3: Drop database cluster-wide[​](#step-3-drop-database-cluster-wide "Direct link to Step 3: Drop database cluster-wide") Connect to any cluster node: ``` cypher-shell -d system -u neo4j DROP DATABASE neo4j; SHOW SERVERS; ``` ![drop database](/assets/images/database_backup_restore_step3-fd0b160866f076367e4335c79818d326.png) ### Step 4: Clean target node data[​](#step-4-clean-target-node-data "Direct link to Step 4: Clean target node data") Connect to the target container: ``` docker exec -it -u neo4j infrahub-database-core3-1 bash ``` Remove any existing data to avoid corruption: ``` rm -rf /data/databases/neo4j rm -rf /data/transactions/neo4j ``` Then restart the container to ensure a clean state: ``` docker restart infrahub-database-core3-1 ``` ### Step 5: Restore backup on target node[​](#step-5-restore-backup-on-target-node "Direct link to Step 5: Restore backup on target node") Reconnect to the container: ``` docker exec -it -u neo4j infrahub-database-core3-1 bash ``` Run the restore command: ``` neo4j-admin database restore \ --from-path=/var/lib/neo4j/neo4j-2025-03-24T19-57-18.backup neo4j ``` ![Restore database](/assets/images/database_backup_restore_step3-fd0b160866f076367e4335c79818d326.png) ### Step 6: Identify seed instance id[​](#step-6-identify-seed-instance-id "Direct link to Step 6: Identify seed instance id") Connect via Cypher shell (on the system database): ``` cypher-shell -d system -u neo4j ``` Run: ``` SHOW SERVERS; ``` Note the `serverId` for your target node (example: `d05fce79-e63e-485a-9ce7-1abbf9d18fce`). ![Seed database](/assets/images/database_backup_restore_step5-573126f8e6dd775aa426f4da34f26423.png) ### Step 7: Recreate database from seed[​](#step-7-recreate-database-from-seed "Direct link to Step 7: Recreate database from seed") Run the following Cypher command: ``` CREATE DATABASE neo4j TOPOLOGY 3 PRIMARIES OPTIONS { existingData: 'use', existingDataSeedInstance: 'd05fce79-e63e-485a-9ce7-1abbf9d18fce' }; ``` ![Choose seed database](/assets/images/database_backup_restore_step7-242198bb2467a113c6fe4ac882b6dc54.png) ### Step 8: Verify cluster sync[​](#step-8-verify-cluster-sync "Direct link to Step 8: Verify cluster sync") Check that the database is coming online: ``` SHOW DATABASES; ``` ![Online database](/assets/images/database_backup_restore_step8_1-0eaf5a4c5bf50a76df2621f93276d9be.png) Then validate cluster sync status: ``` SHOW SERVERS; ``` ![Status sync servers](/assets/images/database_backup_restore_step8_2-68e6cc517142992da051489050e35f6c.png) All nodes should eventually show the Neo4j database as online. Troubleshooting * If nodes show as **dirty** or **offline**, check logs and verify `/data/databases/neo4j/neostore` exists * The `CREATE DATABASE ... OPTIONS { existingData: 'use' }` command is required to register restored data with the cluster ## Restore cluster backup to standalone instance[​](#restore-cluster-backup-to-standalone-instance "Direct link to Restore cluster backup to standalone instance") If you need to analyze data from a production cluster in an isolated environment, follow these steps to restore a cluster backup to a standalone Neo4j instance. ### Step 1: Create cluster backup[​](#step-1-create-cluster-backup "Direct link to Step 1: Create cluster backup") Create a backup from any cluster node: ``` neo4j-admin database backup --to-path=backups/ neo4j # Resulting file: neo4j-2025-03-24T19-57-18.backup ``` ### Step 2: Transfer backup to standalone instance[​](#step-2-transfer-backup-to-standalone-instance "Direct link to Step 2: Transfer backup to standalone instance") ``` docker cp neo4j-2025-03-24T19-57-18.backup \ infrahub-database-1:/var/lib/neo4j/ ``` ### Step 3: Prepare standalone instance[​](#step-3-prepare-standalone-instance "Direct link to Step 3: Prepare standalone instance") Connect to the container: ``` docker exec -it -u neo4j infrahub-database-1 bash ``` Clean any existing Neo4j database (optional but recommended): ``` rm -rf /data/databases/neo4j rm -rf /data/transactions/neo4j ``` Drop the Neo4j Database ``` cypher-shell -d system -u neo4j DROP DATABASE neo4j; SHOW SERVERS; ``` ![Choose seed database](/assets/images/database_backup_restore_step7-242198bb2467a113c6fe4ac882b6dc54.png) ### Step 4: Restore the backup[​](#step-4-restore-the-backup "Direct link to Step 4: Restore the backup") Restore the backup file: ``` neo4j-admin database restore \ --from-path=/var/lib/neo4j/neo4j-2025-03-24T19-57-18.backup neo4j ``` ### Step 5: Create the database[​](#step-5-create-the-database "Direct link to Step 5: Create the database") Run the following Cypher command: ``` CREATE DATABASE neo4j ``` ### Step 6: Verify the status[​](#step-6-verify-the-status "Direct link to Step 6: Verify the status") Check that the database is coming online: ``` SHOW DATABASES; ``` ![Choose seed database](/assets/images/database_backup_restore_standalone_step6_1-cc2b480d48725f35af37e7cf131d47bd.png) Then validate database status: ``` SHOW SERVERS; ``` ![Choose seed database](/assets/images/database_backup_restore_standalone_step6_2-e7fa4d42dcf78119ab614415fd3d813b.png) info This process restores only data, not cluster roles, replication, or configuration settings. ## Related resources[​](#related-resources "Direct link to Related resources") * [Database backup overview](/deploy-manage/maintain-upgrade/database-backup/overview.md) - Architecture and backup strategy concepts * [Backup and restore](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md) - Step-by-step instructions for standalone deployments --- # Understanding database backup and restore This topic explains how Infrahub's database backup and restore system works, the architectural decisions behind it, and the various approaches available for protecting your data. Understanding these concepts helps you make informed decisions about your backup strategy and troubleshoot issues when they arise. ## Overview[​](#overview "Direct link to Overview") Infrahub's backup system is designed around three core principles: 1. **Completeness**: Capturing all data necessary for full system recovery 2. **Consistency**: Ensuring data integrity across distributed components 3. **Flexibility**: Supporting various deployment scenarios and recovery needs The backup process involves more than just the Neo4j graph database—it encompasses the entire data ecosystem that Infrahub relies on to function correctly. ## Backup architecture[​](#backup-architecture "Direct link to Backup architecture") ### Components requiring backup[​](#components-requiring-backup "Direct link to Components requiring backup") Infrahub's data is distributed across multiple systems, each serving a specific purpose: **Neo4j graph database** The core of Infrahub's data model, storing all infrastructure relationships, schemas, and configuration data. This uses Neo4j's transactional graph database engine, which maintains ACID compliance and supports point-in-time recovery through transaction logs. **artifact storage** Transformations, queries, and other generated artifacts are stored separately from the graph database. This can be either an S3-compatible object store or local filesystem, depending on your deployment configuration. The separation allows for efficient storage of large files without impacting database performance. **Task management database** Prefect's PostgreSQL database contains task execution history, logs, and workflow state. While not critical for data recovery, this information is valuable for auditing and troubleshooting past operations. The infrahubops tool backs this up using `pg_dump` with the custom format (`-Fc`) for efficient compression and restoration. ### Why multiple components?[​](#why-multiple-components "Direct link to Why multiple components?") This distributed architecture might seem complex, but it serves important purposes: * **Performance optimization**: artifacts don't belong in a graph database * **Scalability**: Each component can be scaled independently * **Flexibility**: Different storage backends for different data types * **Cost efficiency**: Use appropriate storage tiers for different data ## Backup strategies[​](#backup-strategies "Direct link to Backup strategies") ### Full vs incremental backups[​](#full-vs-incremental-backups "Direct link to Full vs incremental backups") Neo4j supports both full and incremental backups, each with distinct characteristics: **Full backups** create a complete copy of the database at a specific point in time. They're self-contained and straightforward to restore but require more storage space and time to complete. **Incremental backups** only capture changes since the last backup. Neo4j tracks transaction IDs to ensure no data loss even during active database use. The backup tool automatically determines whether to perform a full or incremental backup based on existing backup files in the target directory. ### Online vs offline backups[​](#online-vs-offline-backups "Direct link to Online vs offline backups") **Online backups** (the default approach) allow the database to remain operational during the backup process. Neo4j uses a checkpoint mechanism to ensure consistency: 1. A checkpoint is triggered to flush pending transactions 2. Data files are copied while tracking new transactions 3. Transaction logs ensure no data loss between checkpoint and completion **Offline backups** require stopping the database but guarantee a perfectly consistent snapshot. These are typically only necessary for major migrations or when changing database versions. ## The backup process explained[​](#the-backup-process-explained "Direct link to The backup process explained") ### How Neo4j ensures consistency[​](#how-neo4j-ensures-consistency "Direct link to How Neo4j ensures consistency") Neo4j's backup mechanism uses several techniques to maintain consistency: 1. **Transaction logs**: Every change is written to a transaction log before being applied to the data files 2. **Checkpoints**: Periodic checkpoints ensure transaction logs are applied to data files 3. **Backup coordination**: The backup process coordinates with the checkpoint mechanism to capture a consistent view When you initiate a backup with infrahubops: ``` Check Running Tasks → Create Temp Directory → Backup Neo4j → Backup PostgreSQL → Calculate Checksums → Create Tarball → Cleanup ``` The tool ensures no tasks are running before starting (unless `--force` is used), preventing potential data inconsistencies. ### Helper container approach[​](#helper-container-approach "Direct link to Helper container approach") The infrahubops tool uses Docker Compose commands to execute operations directly on running containers, providing several benefits: * **No additional containers**: Unlike the old utility, infrahubops doesn't create helper containers * **Direct execution**: Uses `docker compose exec` to run commands in existing containers * **Simplified networking**: No need to manage separate Docker networks * **Project awareness**: Automatically detects and targets the correct Docker Compose project * **Integrity verification**: Calculates and validates SHA256 checksums for all backed-up files ## Restore considerations[​](#restore-considerations "Direct link to Restore considerations") ### Data consistency during restore[​](#data-consistency-during-restore "Direct link to Data consistency during restore") Restoring a database is more disruptive than backing it up because it requires: 1. **Stopping the target database** to prevent concurrent modifications 2. **Clearing existing data** to avoid conflicts 3. **Restoring data files** from the backup 4. **Replaying transaction logs** to reach the backup point 5. **Recreating metadata** including users, roles, and permissions ### The system database challenge[​](#the-system-database-challenge "Direct link to The system database challenge") Neo4j maintains a special `system` database containing: * User accounts and authentication data * Role definitions and permissions * Database metadata and configuration While the backup tool captures the system database, restoring it requires special consideration: * In standalone deployments, the system database is typically not restored to preserve existing configurations * In cluster deployments, system database restoration requires cluster-wide coordination * For disaster recovery scenarios, system database restoration may be necessary but requires additional steps ### Cluster restore complexity[​](#cluster-restore-complexity "Direct link to Cluster restore complexity") Restoring a Neo4j cluster involves additional challenges: **Cluster topology preservation** The backup contains data but not cluster roles (leader/follower relationships). After restoration, you must: 1. Identify a seed instance with the restored data 2. Use `CREATE DATABASE ... OPTIONS { existingData: 'use' }` to register the data 3. Allow the cluster to replicate data to other nodes **Consistency across nodes** All nodes must be synchronized to prevent split-brain scenarios. The restoration process typically involves: * Dropping the database cluster-wide * Restoring to a single node * Recreating the database from that seed node * Monitoring replication to ensure all nodes synchronize ## Alternative approaches[​](#alternative-approaches "Direct link to Alternative approaches") ### Direct filesystem copies[​](#direct-filesystem-copies "Direct link to Direct filesystem copies") While possible, directly copying Neo4j data files has significant limitations: * Requires complete database shutdown * No transaction log coordination * Risk of incomplete or corrupted copies * No automatic metadata handling ### Logical exports[​](#logical-exports "Direct link to Logical exports") Using Cypher queries to export and import data: * **Advantages**: Human-readable, version-independent, selective export * **Disadvantages**: Slower, no transaction consistency, requires custom scripting ### Continuous replication[​](#continuous-replication "Direct link to Continuous replication") Setting up read replicas for backup purposes: * **Advantages**: Near-zero RPO (Recovery Point Objective), instant failover capability * **Disadvantages**: Requires additional infrastructure, ongoing synchronization overhead ## Implementation details[​](#implementation-details "Direct link to Implementation details") ### Backup file structure[​](#backup-file-structure "Direct link to Backup file structure") The infrahubops tool creates a tarball with the following structure: ``` infrahub_backup_YYYYMMDD_HHMMSS.tar.gz ├── backup/ │ ├── backup_information.json # Metadata and checksums │ ├── database/ # Neo4j backup files │ │ └── neo4j-*.backup │ └── prefect.dump # PostgreSQL dump ``` ### Task safety mechanism[​](#task-safety-mechanism "Direct link to Task safety mechanism") Before creating a backup, the tool checks for running tasks using an embedded Python script that queries the Infrahub API. This prevents backing up data in an inconsistent state. The `--force` flag bypasses this check but should be used with caution. ### Checksum validation[​](#checksum-validation "Direct link to Checksum validation") Every file in the backup is protected by SHA256 checksums stored in `backup_information.json`. During restoration, these checksums are verified to ensure data integrity. Any mismatch will abort the restoration process. ## Further reading[​](#further-reading "Direct link to Further reading") * [Backup and restore](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md) - Step-by-step instructions for standalone deployments * [Cluster backup and restore](/deploy-manage/maintain-upgrade/database-backup/cluster-backup-and-restore.md) - Step-by-step instructions for Neo4j cluster deployments * [infrahub-ops-cli source code](https://github.com/opsmill/infrahub-ops-cli) - Implementation details and latest features * [infrahub-backup CLI reference](https://docs.infrahub.app/backup/) - Command-line reference for the infrahub-backup tool --- # Upgrade Community The process to migrate your instance of Infrahub to the latest version will vary depending on your deployment of Infrahub. However, at a high-level, it will involve getting the latest version and performing any needed Database Migrations and Schema updates. For installation instructions, please refer to our [Installation guide](/deploy-manage/install-configure/install/overview.md). Follow the instructions below for your current release version and deployment method. Upgrade path support Upgrades are supported only from the previous minor version (N-1), for example from Infrahub 1.3 to 1.4. If you need to upgrade from N-2 or older, upgrade sequentially one minor version at a time (for example: 1.2 -> 1.3 -> 1.4). info Even though a "smooth" migration is anticipated, we nonetheless strongly suggest creating a backup beforehand. For detailed information, see our [Backup guide](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md). info In Infrahub 1.2 and later, the upgrade process has been streamlined with a unified upgrade command. For earlier versions, please refer to the [release notes](/release-notes/infrahub.md) for specific upgrade instructions. ## Upgrading to Infrahub 1.2 and after[​](#upgrading-to-infrahub-12-and-after "Direct link to Upgrading to Infrahub 1.2 and after") * Docker * Helm * Dev/Demo ``` # Stop the current instance docker compose down # Retrieve the latest docker-compose version curl https://infrahub.opsmill.io > docker-compose.yml # Run the unified upgrade command docker compose run infrahub-server infrahub upgrade # Restart the instance docker compose up -d ``` note Old Docker images of Infrahub may remain on your machine and consume disk space after an upgrade. To free up space and keep disk usage under control, consider removing unused images. You can do this with: `docker image prune -a` ``` # Upgrade the Helm release helm upgrade # Run the unified upgrade command inside the server pod kubectl exec infrahub-infrahub-server-xxxxxx -- infrahub upgrade # Restart deployments to apply changes kubectl rollout restart deployment/prefect-server kubectl rollout restart deployment/infrahub-infrahub-server ``` ``` # Ensure you pull the latest opsmill/infrahub repo git pull # Stop the current instance invoke demo.stop # Rebuild and run migrations invoke demo.pull invoke demo.upgrade # Restart the instance invoke demo.start ``` ## Verify the upgrade[​](#verify-the-upgrade "Direct link to Verify the upgrade") After the `infrahub upgrade` command exits with `Upgrade complete SUCCESS`, run `infrahub db showmigrations` against the same database to confirm every migration shows `[X]` and that the database version matches the target version reported in the header. If any migrations are still pending, re-run `infrahub upgrade`. If you skipped `--rebase-branches` and the upgrade reported open branches that need rebase, they are left in the `needs-rebase` state. You can rebase them manually later, or re-run with `infrahub upgrade --rebase-branches` (optionally combined with `--interactive` to confirm each branch). For details on what each step of the upgrade does, the output you should expect, and what `--verbose` actually controls, see the [upgrade overview](/deploy-manage/maintain-upgrade/upgrade/overview.md#what-the-upgrade-does). --- # Upgrade Enterprise Enterprise Edition The process to migrate your instance of Infrahub Enterprise to the latest version will vary depending on your deployment of Infrahub Enterprise. However, at a high-level, it will involve getting the latest version and performing any needed Database Migrations and Schema updates. For installation instructions, please refer to our [Enterprise installation guide](/deploy-manage/install-configure/install/enterprise.md). Follow the instructions below for your current release version and deployment method. info Even though a "smooth" migration is anticipated, we nonetheless strongly suggest creating a backup beforehand. For detailed information, see our [Backup guide](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md). info In Infrahub 1.2 and later, the upgrade process has been streamlined with a unified upgrade command. For earlier versions, please refer to the [release notes](/release-notes/infrahub.md) for specific upgrade instructions. ## Upgrading to Infrahub Enterprise 1.2 and after[​](#upgrading-to-infrahub-enterprise-12-and-after "Direct link to Upgrading to Infrahub Enterprise 1.2 and after") * Docker * Helm ``` # Stop the current instance docker compose down # Retrieve the latest docker-compose version curl https://infrahub.opsmill.io/enterprise > docker-compose.yml # Run the unified upgrade command docker compose run infrahub-server infrahub upgrade # Restart the instance docker compose up -d ``` note Old Docker images of Infrahub may remain on your machine and consume disk space after an upgrade. To free up space and keep disk usage under control, consider removing unused images. You can do this with: `docker image prune -a` ``` # Upgrade the Helm release helm upgrade # Run the unified upgrade command inside the server pod kubectl exec infrahub-infrahub-server-xxxxxx -- infrahub upgrade # Restart deployments to apply changes kubectl rollout restart deployment/prefect-server kubectl rollout restart deployment/infrahub-infrahub-server ``` --- # Upgrade the observability stack ## Docker Compose[​](#docker-compose "Direct link to Docker Compose") When a new compose file changes the embedded observability configuration, services backed by Docker Compose `configs` do not pick up those changes on a regular `up -d`. Use `--force-recreate` on the affected services to reload the embedded configuration content: ``` docker compose -p infrahub up -d --force-recreate infrahub-alloy infrahub-loki infrahub-tempo infrahub-grafana ``` ## Kubernetes with Helm[​](#kubernetes-with-helm "Direct link to Kubernetes with Helm") How you upgrade the stack depends on how it was deployed. If it is **bundled** with your Infrahub release (`infrahub-observability.enabled`), the observability subchart version is pinned by the Infrahub chart, so it upgrades together with Infrahub — upgrade the Infrahub release as usual. If it is a **standalone** release, upgrade it to a new chart version directly: ``` helm upgrade obs oci://registry.opsmill.io/opsmill/chart/infrahub-observability --version -n infrahub ``` Pass your `values.yml` with `-f` if you customized the deployment. Persistent volumes for Loki, Prometheus, Tempo, and Grafana are retained across upgrades, so dashboards and historical data survive the upgrade. For how to deploy and configure the stack, see [Observability stack](/deploy-manage/install-configure/install/observability-stack.md). --- # Upgrade Upgrading Infrahub involves pulling the latest container images, running database and schema migrations, and restarting services. The specific steps vary by edition (Community vs Enterprise) and deployment method (Docker Compose vs Helm). ## Before you upgrade[​](#before-you-upgrade "Direct link to Before you upgrade") **Upgrade path**: Upgrades are supported only from the previous minor version (N-1), for example from Infrahub 1.3 to 1.4. If you need to upgrade from N-2 or older, upgrade sequentially one minor version at a time. **Backup first**: Even though a smooth migration is anticipated, we strongly recommend creating a backup before upgrading. See [Backup and restore](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md). **Release notes**: Review the [release notes](/release-notes/infrahub.md) for any version-specific upgrade instructions. In Infrahub 1.2 and later, the upgrade process was streamlined with a unified upgrade command. For earlier versions, refer to the release notes for specific instructions. ## What the upgrade does[​](#what-the-upgrade-does "Direct link to What the upgrade does") `infrahub upgrade` runs as a structured six-step pipeline against the database the configured Infrahub instance points at: | Step | What runs | | ---- | ------------------------------------------------------------- | | 1/6 | Database migrations (same code path as `infrahub db migrate`) | | 2/6 | Internal schema initialization | | 3/6 | Core schema update | | 4/6 | Internal objects — menu and permissions | | 5/6 | Task manager — workflow blocks, worker pools, deployments | | 6/6 | Branch rebase (only writes when `--rebase-branches` is set) | The command logs a header for each step, per-migration timing inside Step 1/6, and a final `Upgrade complete SUCCESS` line. Use `--check` to see what the upgrade would do without writing anything. ### What you will see during an upgrade[​](#what-you-will-see-during-an-upgrade "Direct link to What you will see during an upgrade") Each step prints a header, and migrations inside Step 1/6 print per-migration progress. Long, data-rewriting migrations — recomputing HFID and display-label values, normalizing MAC addresses, unifying IP pool resource identifiers — also print their own progress messages so you can confirm which data they are touching. Example excerpt: ``` Step 1/6: Database migrations Database version: 67 Target version: 73 Pending: 6 migrations 068 cleanup_branch_schema_parameters (ArbitraryMigration) 069 set_comment_thread_created_by_on_node (GraphMigration) 070 normalize_mac_address_values_to_colon (MigrationRequiringRebase) 071 recompute_hfid_for_ip_attributes (MigrationRequiringRebase) 072 index_hfid_values (ArbitraryMigration) 073 unify_ip_pool_resource_identifier (ArbitraryMigration) Applying migrations: Cleaning up spurious branch schema attribute parameters from Migration056 Checking 2 open user branches Applying 068_cleanup_branch_schema_parameters... OK (0.01s) Applying 069_set_comment_thread_created_by_on_node... OK (0.00s) Normalizing MacAddress values for InfraDevice.mac_address Applying 070_normalize_mac_address_values_to_colon... OK (0.02s) Recomputing HFID/display_label for InternalIPPrefixAvailable Applying 071_recompute_hfid_for_ip_attributes... OK (0.01s) Normalized 14 HFID value(s) to all-string format Applying 072_index_hfid_values... OK (0.01s) Rewrote CoreIPPrefixPool.resources.identifier → ippool__resource. Applying 073_unify_ip_pool_resource_identifier... OK (0.04s) Applied 6 migrations successfully in 1.44s Step 2/6: Internal schema Internal schema initialized Step 3/6: Core schema Core Schema updated successfully Step 4/6: Internal objects Menu has been updated Permissions up to date, nothing to update Step 5/6: Task manager Task manager configured Step 6/6: Branch rebase Found 2 branches that need to be rebased Rebased branch 'clean-branch' (ID: ...) SUCCESS Rebased branch 'conflict-branch' (ID: ...) SUCCESS Upgrade complete SUCCESS ``` Lines such as `Cleaning up …`, `Recomputing HFID/display_label …`, `Normalized N HFID value(s) …`, and `Rewrote … → …` are migration progress messages, not errors. They tell you which data the current migration is about to touch and are expected output. ### What `--verbose` does[​](#what---verbose-does "Direct link to what---verbose-does") `--verbose` does **not** control per-migration progress (that is always shown). It only lifts the suppression on the `infrahub` and `prefect` Python loggers so their schema-loader warnings, validator-determiner messages, and workflow-client logs become visible during Step 1/6 and Step 6/6. Use it when a migration is failing and you want the full internal log trail; in normal operation you do not need it. ## Database migration commands[​](#database-migration-commands "Direct link to Database migration commands") note You do not need any of the `infrahub db` commands for day-to-day upgrades — `infrahub upgrade` takes care of everything. However, `infrahub db showmigrations` is a useful verification step after an upgrade, while the remaining commands are intended for advanced debugging: previewing what would run, or applying the migration step in isolation when investigating a failure. `infrahub db migrate` runs the same code path as **Step 1/6 of `infrahub upgrade`**, with identical formatting. Use it only when you specifically want the migration step in isolation — for example, to reproduce a migration failure outside the full upgrade pipeline. ### Listing migrations[​](#listing-migrations "Direct link to Listing migrations") `infrahub db showmigrations` lists every known migration and whether it has been applied to the connected database: ``` $ infrahub db showmigrations Migration list (database version: 71, target version: 73): [X] 001 add_version_to_graph GraphMigration [X] 002 attribute_is_default GraphMigration ... [X] 071 recompute_hfid_for_ip_attributes MigrationRequiringRebase [ ] 072 index_hfid_values ArbitraryMigration [ ] 073 unify_ip_pool_resource_identifier ArbitraryMigration ``` `[X]` marks migrations that have already been applied to this database. `[ ]` marks migrations that are pending. The header reports the current graph version stored on the Root node and the target version that the running Infrahub binary expects. Use this command before and after `infrahub db migrate` (or `infrahub upgrade`) to confirm which migrations were applied. #### Migration types[​](#migration-types "Direct link to Migration types") The third column of `showmigrations` reports the migration's class. The four types behave differently: * **GraphMigration** — applies one or more Cypher queries to the graph. * **InternalSchemaMigration** — updates the internal schema definition. * **ArbitraryMigration** — runs arbitrary Python logic. May print its own progress messages while it works. * **MigrationRequiringRebase** — rewrites data that any open branches were built on top of. After this kind of migration, every open branch needs to be rebased before it can be used again — see `infrahub upgrade --rebase-branches`. ### Inspecting a single migration[​](#inspecting-a-single-migration "Direct link to Inspecting a single migration") `infrahub db showmigration N` reports details about a specific migration — its type, the minimum graph version it requires, whether it has been applied, and a human-readable description of what data it touches: ``` $ infrahub db showmigration 70 Migration 070: 070_normalize_mac_address_values_to_colon Type: MigrationRequiringRebase Minimum version: 69 Status: Pending Description: Rewrite stored MacAddress values to colon-separated uppercase form and recompute any HFID or display_label that depends on them. ``` Use this command when `showmigrations` flags a migration as pending and you want to understand what it will change before applying it. ### Previewing without writing[​](#previewing-without-writing "Direct link to Previewing without writing") `infrahub db migrate --plan` lists the pending migrations and reports what would be applied, without touching the database: ``` $ infrahub db migrate --plan Database version: 71 Target version: 73 Pending: 2 migrations 072 index_hfid_values (ArbitraryMigration) 073 unify_ip_pool_resource_identifier (ArbitraryMigration) Migration plan (2 pending): 1. index_hfid_values ArbitraryMigration 2. unify_ip_pool_resource_identifier ArbitraryMigration No migrations were applied. Run 'infrahub db migrate' to apply. ``` `--plan` does not write to the database. Re-running `infrahub db showmigrations` immediately afterwards must show identical state. ## Upgrade guides[​](#upgrade-guides "Direct link to Upgrade guides") Select the guide for your edition and deployment: * [Upgrade Community](/deploy-manage/maintain-upgrade/upgrade/community.md) — Docker Compose, Helm, and Dev/Demo deployments * [Upgrade Enterprise](/deploy-manage/maintain-upgrade/upgrade/enterprise.md) — Docker Compose and Helm deployments * [Upgrade the observability stack](/deploy-manage/maintain-upgrade/upgrade/observability-stack.md) — Force-recreate observability services after compose file changes ## Known issues[​](#known-issues "Direct link to Known issues") ### Migration failing because of transaction memory limit reached in Neo4j[​](#migration-failing-because-of-transaction-memory-limit-reached-in-neo4j "Direct link to Migration failing because of transaction memory limit reached in Neo4j") For large database/schema migrations, you may encounter a `Transaction memory limit reached` error in Neo4j. info To work around this, you can disable the transaction memory limit by setting the `dbms.memory.transaction.total.max` to `0` in your Neo4j configuration. We are working on a more robust solution to handle large migrations without hitting this limit in future releases. --- # Activity log Changes (events) in Infrahub are documented in the **Activity log**. It helps you see which objects were impacted, when a change was made, and who made it. It can be used to troubleshoot unforeseen changes, audit previous operations, and comprehend the order of updates across various branches. To export these events to an external SIEM or centralized logging system, see [log forwarding](/deploy-manage/run-observe/log-forwarding/overview.md). Permissions Because figuring out who may view which changes requires complicated query requirements, we have currently set aside the permission framework for this functionality. The activity log, in general, compiles and arranges events from several branches and objects into a single timeline: * **Global view**: A consolidated list of all branch-wide activities (events). * **Object-level view**: A timeline that is particular to a single object and only displays events that are pertinent to that object. * **Filtering**: You can narrow down your search using a variety of filters (by branch, event type, account, principal node, linked node, date range). * **Nested / child events**: Cascade actions are observed when specific top-level events trigger additional child events. ## Accessing the activity log[​](#accessing-the-activity-log "Direct link to Accessing the activity log") The activity log can be accessed in any of two ways: globally or at the object level. 1. **Global activity log**: * Menu location: In the main navigation, go to **Activity** → **Activity log**. * URL: `https:///activities` ![Global Activity log page](/assets/images/activity_log_global-c921297d857c7c80a446778b0e00f176.png) Time and date The activity log's date and time are determined by the local time in your browser. Child Events Events containing child events have an extra (blue) icon at the end of the line. 2. **Object-level activity log**: * Node / object detail pages: Within a node's detail view (for example an IP address or device), you'll see a right-hand "Activity logs" panel or a separate tab. ![Device detail page with Activity logs panel](/assets/images/activity_log_device-28ba8f590e6cebb91c7836eb83024424.png) ## Filters and search[​](#filters-and-search "Direct link to Filters and search") There are several ways to hone your view in the global activity log: * **Branch**: Select the branch you wish to view the events in, such as `main`. * **Event type**: Filter by categories such as `Node Created`, `Branch Deleted`. You can find more information in the [Infrahub Events](/reference/infrahub-events/overview.md) topic. * **Account / user**: Only display events that have been initiated by a certain account. * **Primary / related node**: Highlight activities associated with a specific node. * **Has children**: Whether a certain occurrence led to other smaller ones. * **Date range**: Set a start and end time and date for the timeline. ![Activity logs filters - primary](/assets/images/activity_log_global_filters_primary-b081a1645dfbf07b005a1a15d8de4c13.png) ![Activity logs filters - children](/assets/images/activity_log_global_filters_children-317290318c3ae0a518b903b2b775b8d4.png) ## Querying the activity log over the API[​](#querying-the-activity-log-over-the-api "Direct link to Querying the activity log over the API") Use the `InfrahubEvent` GraphQL query to read the activity log from a script: collect every operation in an incident window, or line those operations up against the data as it stood at the time. Events in a time range ``` query IncidentEvents { InfrahubEvent( since: "2026-03-09T00:00:00Z" until: "2026-03-10T00:00:00Z" branches: ["main"] ) { count edges { node { id event branch occurred_at account_id primary_node { id kind } } } } } ``` `since` defaults to 180 days back and `until` defaults to the current time. A query that omits `since` therefore returns only the last 180 days rather than the full history, and nothing in the response says so. Set `since` explicitly when you need events older than 180 days. The filters available in the web interface map onto these parameters, and the query accepts several that the interface does not expose: | Parameter | Selects | | ------------------- | ------------------------------------------------------- | | `since`, `until` | Events in a time range | | `branches` | Events on specific branches | | `account__ids` | Events initiated by specific accounts | | `event_type` | Events matching a type, such as `infrahub.node.updated` | | `event_type_filter` | Filters specific to one event type | | `primary_node__ids` | Events whose primary node is one of these | | `related_node__ids` | Events whose related nodes include one of these | | `parent__ids` | Events listing one of these as a parent | | `has_children` | Events that did or did not trigger child events | | `level` | Root events (`0`) or a given depth of child event | | `ids` | Specific events by ID | | `order` | Sort order, descending by default | | `limit`, `offset` | Page through the results | See [Infrahub Events](/reference/infrahub-events/overview.md) for the event types and the fields each one carries. ## Viewing event details[​](#viewing-event-details "Direct link to Viewing event details") If you choose **View more** from the global or object-level list, a separate detail page or popover will show up. Additional details are shown in this view, such as: * **Event ID**: A unique UUID referencing the event. * **Event type**: The type of the event, for example, `infrahub.node.updated`. You can find more information in the [Infrahub Events](/reference/infrahub-events/overview.md) topic. * **Occurred at**: The exact moment the incident took place * **Account**: The account that carried out the activity. * **Primary node**: The primary object that the event affects. * **Related nodes**: Additional affected objects. * **Changes**: Any before and after adjustments to updated attributes, if any ![Activity details page with children](/assets/images/activity_log_global_details_children-406d8ed65247deed14dc35ed67fc04f4.png) ![Activity logs details popover](/assets/images/activity_log_device_popover-2b36b6d169805caf37fda27eaf4e2977.png) --- # Configure log forwarding Enterprise Edition This guide walks through configuring Infrahub to forward audit events and application logs to external SIEM or syslog systems. For conceptual background on how log forwarding works, see the [log forwarding overview](/deploy-manage/run-observe/log-forwarding/overview.md). Enterprise Edition Log forwarding requires the Enterprise Edition of Infrahub. The Community Edition includes the configuration surface but does not transmit messages. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Infrahub Enterprise Edition running * A syslog-compatible receiver (rsyslog, syslog-ng, Splunk, Datadog Agent, Logstash, or similar) * Network connectivity from Infrahub to the receiver on the chosen port * For TLS: a CA certificate or bundle trusted by the syslog server ## Configuring a single destination[​](#configuring-a-single-destination "Direct link to Configuring a single destination") * Via the configuration file * Via environment variables Add a `[[log_forwarding.destinations]]` section to your Infrahub TOML configuration file: ``` [[log_forwarding.destinations]] name = "siem_primary" type = "syslog" host = "syslog.example.com" port = 514 ``` This minimal configuration sends audit events over UDP using RFC 5424 format to port 514. The defaults are: | Setting | Default | | -------- | --------------------------------- | | Protocol | UDP | | Format | RFC 5424 | | Port | 514 (or 6514 when TLS is enabled) | Environment variables are useful for Docker Compose and Kubernetes deployments where configuration files may not be practical. Define destination names as a comma-separated list, then set per-destination fields using the pattern `INFRAHUB_LOG_FORWARDING_DESTINATION_{NAME}_{FIELD}`: ``` export INFRAHUB_LOG_FORWARDING_DESTINATION_NAMES="siem_primary" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_PRIMARY_HOST="syslog.example.com" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_PRIMARY_PORT="514" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_PRIMARY_PROTOCOL="udp" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_PRIMARY_FORMAT="rfc5424" ``` Alternatively, pass the full destination configuration as JSON: ``` export INFRAHUB_LOG_FORWARDING_DESTINATIONS='[{"name":"siem_primary","type":"syslog","host":"syslog.example.com","port":514}]' ``` note Destination names must match `[a-z0-9_]+` (lowercase letters, digits, and underscores only). warning `INFRAHUB_LOG_FORWARDING_DESTINATION_NAMES` and `INFRAHUB_LOG_FORWARDING_DESTINATIONS` should not both be set unless they define the exact same destinations. Mixing the two approaches is not recommended — pick one method and use it consistently. note When `INFRAHUB_LOG_FORWARDING_HOSTNAME` is not set, Infrahub uses `socket.getfqdn()` to determine the hostname in syslog headers. In Docker environments, this resolves to the container ID rather than a meaningful hostname. Set `hostname` explicitly in your configuration or via the `INFRAHUB_LOG_FORWARDING_HOSTNAME` environment variable when running in containers. Restart Infrahub to apply the configuration. For the complete list of destination parameters, see the [configuration reference](/reference/configuration.md#log-forwarding). ## Enabling TLS encryption[​](#enabling-tls-encryption "Direct link to Enabling TLS encryption") TLS requires the TCP protocol. Configuring TLS with UDP raises a validation error. ``` [[log_forwarding.destinations]] name = "siem_secure" type = "syslog" host = "syslog.example.com" protocol = "tcp" tls_enabled = true tls_ca_bundle = "/etc/ssl/certs/ca-certificates.crt" ``` When TLS is enabled and no explicit port is set, the port defaults to 6514. The `tls_ca_bundle` field accepts either a file path or an inline PEM string. Environment variable equivalent: ``` export INFRAHUB_LOG_FORWARDING_DESTINATION_NAMES="siem_secure" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_SECURE_HOST="syslog.example.com" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_SECURE_PROTOCOL="tcp" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_SECURE_TLS_ENABLED="true" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_SECURE_TLS_CA_BUNDLE="/etc/ssl/certs/ca-certificates.crt" ``` ## Configuring multiple destinations[​](#configuring-multiple-destinations "Direct link to Configuring multiple destinations") Use multiple `[[log_forwarding.destinations]]` blocks to send events to more than one receiver. Each destination operates independently with its own queue and connection. ``` [log_forwarding] hostname = "infrahub-prod-01" [[log_forwarding.destinations]] name = "siem_primary" type = "syslog" host = "syslog-primary.example.com" protocol = "tcp" tls_enabled = true tls_ca_bundle = "/etc/ssl/certs/ca-certificates.crt" [[log_forwarding.destinations]] name = "siem_backup" type = "syslog" host = "syslog-backup.example.com" port = 514 protocol = "udp" format = "rfc3164" ``` The optional `hostname` field at the top level overrides the syslog header hostname for all destinations. If not set, it defaults to the system FQDN. With environment variables, use comma-separated destination names: ``` export INFRAHUB_LOG_FORWARDING_DESTINATION_NAMES="siem_primary,siem_backup" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_PRIMARY_HOST="syslog-primary.example.com" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_PRIMARY_PROTOCOL="tcp" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_PRIMARY_TLS_ENABLED="true" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_BACKUP_HOST="syslog-backup.example.com" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_BACKUP_PORT="514" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_BACKUP_PROTOCOL="udp" export INFRAHUB_LOG_FORWARDING_DESTINATION_SIEM_BACKUP_FORMAT="rfc3164" ``` note Destination names must be unique. Duplicate names are rejected during configuration validation. ## Enabling application log forwarding[​](#enabling-application-log-forwarding "Direct link to Enabling application log forwarding") By default, only audit events are forwarded. Enabling application log forwarding adds permission denied events (`infrahub.permission.denied`) to the forwarded stream. These are the only events currently sent through this channel. They require explicit opt-in per destination: ``` [[log_forwarding.destinations]] name = "siem_all_logs" type = "syslog" host = "syslog.example.com" protocol = "tcp" forward_application_logs = true ``` Permission denied events use syslog facility `LOG_LOCAL0` (16), while audit events use `LOG_AUTH` (4). This distinction helps receivers separate and route the two categories independently. warning Permission denied events are only forwarded when `forward_application_logs` is enabled. If you need these events for security monitoring, ensure this setting is active on the relevant destinations. ## Verifying logs are arriving[​](#verifying-logs-are-arriving "Direct link to Verifying logs are arriving") After configuring log forwarding and restarting Infrahub: 1. **Check Infrahub startup logs** for messages confirming that log forwarding destinations have been initialized. 2. **Trigger an event** by performing an action that generates an audit event, such as logging in, creating a node, or updating an object. 3. **Check your syslog receiver** for incoming messages. An RFC 5424 audit event looks like: ``` <38>1 2025-01-15T10:30:00.000000Z infrahub.example.com infrahub worker-01 - - {"event":"infrahub.node.created","meta":{"branch":"main","account_id":"01234567-abcd-0000-0000-000000000001"},"kind":"InfraDevice","node_id":"01234567-abcd-0000-0000-000000000002","action":"created"} ``` The same event in RFC 3164 format: ``` <38>Jan 15 10:30:00 infrahub.example.com infrahub[worker-01]: {"event":"infrahub.node.created","meta":{"branch":"main","account_id":"01234567-abcd-0000-0000-000000000001"},"kind":"InfraDevice","node_id":"01234567-abcd-0000-0000-000000000002","action":"created"} ``` The PRI value `38` comes from `facility * 8 + severity`: LOG\_AUTH (4) \* 8 + INFORMATIONAL (6) = 38. ## Tuning and troubleshooting[​](#tuning-and-troubleshooting "Direct link to Tuning and troubleshooting") ### Queue sizing[​](#queue-sizing "Direct link to Queue sizing") The `queue_size` parameter (default 10,000) controls how many messages can be buffered per destination. If the queue fills because the receiver is slow or unreachable, new messages are dropped. Increase this value for high-volume environments. ### Reconnection behavior[​](#reconnection-behavior "Direct link to Reconnection behavior") For TCP connections, `max_reconnect_interval` (default 60 seconds) caps the exponential backoff between reconnection attempts. Lower this value if faster recovery is needed. ### Graceful shutdown[​](#graceful-shutdown "Direct link to Graceful shutdown") `shutdown_drain_timeout` (default 10 seconds) controls how long Infrahub waits for queued messages to be transmitted during shutdown. Increase this value if you need to ensure all buffered messages are delivered before the process exits. ### TCP framing[​](#tcp-framing "Direct link to TCP framing") The `tcp_framing` option controls how TCP messages are delimited: * `newline` (default): messages are separated by newline characters * `octet-counting`: messages are length-prefixed per RFC 6587; required by some rsyslog and syslog-ng configurations ### Common issues[​](#common-issues "Direct link to Common issues") | Symptom | Cause | Resolution | | --------------------------- | ---------------------------------------- | -------------------------------------------------------------------------------- | | Validation error on startup | TLS enabled with UDP protocol | Change protocol to `tcp` — TLS is only supported with TCP | | Validation error on startup | Duplicate destination names | Ensure each destination has a unique `name` | | No messages received | Firewall blocking syslog port | Verify network connectivity from Infrahub to the receiver on the configured port | | No messages received | Community Edition | Log forwarding requires Enterprise Edition | | Truncated messages | Receiver does not support octet-counting | Switch `tcp_framing` to `newline`, or configure the receiver for octet-counting | ## Related resources[​](#related-resources "Direct link to Related resources") * [Log forwarding overview](/deploy-manage/run-observe/log-forwarding/overview.md) * [Configuration reference](/reference/configuration.md#log-forwarding) * [Infrahub events reference](/reference/infrahub-events/overview.md) * [Activity log](/deploy-manage/run-observe/activity-log.md) * [Webhooks](/webhooks/overview.md) --- # Log forwarding Enterprise Edition Log forwarding enables Infrahub to stream audit events and application logs to external systems via the syslog protocol. This supports compliance, security monitoring, and operational visibility by integrating with SIEM platforms and centralized log management. Enterprise Edition Log forwarding is available exclusively in the Enterprise Edition. The Community Edition includes the configuration model but uses a no-op implementation that does not transmit messages. See [Community vs Enterprise](/overview/community-vs-enterprise.md) for details. To configure log forwarding destinations, see [Configure log forwarding](/deploy-manage/run-observe/log-forwarding/configure-log-forwarding.md). ## Why log forwarding matters[​](#why-log-forwarding-matters "Direct link to Why log forwarding matters") Many organizations face regulatory and operational requirements that demand centralized, tamper-evident logging across all infrastructure tools: * **Compliance**: Standards such as SOC 2, PCI-DSS, and ISO 27001 require historical tracing of user actions across all systems in a central location * **Security monitoring**: Real-time detection of suspicious activity such as permission denied events, unexpected schema changes, or mass deletions * **Operational visibility**: Correlate infrastructure data changes in Infrahub with network incidents tracked in external monitoring systems * **Production deployment approval**: Security teams often require centralized log export before approving tools for production use Infrahub provides two complementary mechanisms for event visibility: * The [activity log](/deploy-manage/run-observe/activity-log.md) provides in-app visibility for browsing and filtering events through the web interface * Log forwarding exports those same events to purpose-built external tools for long-term retention, alerting, and cross-system correlation Log forwarding is distinct from [webhooks](/webhooks/overview.md). Webhooks are HTTP callbacks designed for real-time integrations with custom payloads. Log forwarding uses the syslog protocol for compliance and SIEM use cases where standardized log ingestion is required. ## Architecture overview[​](#architecture-overview "Direct link to Architecture overview") Log forwarding uses a queue-based, non-blocking architecture to ensure event processing is never delayed by slow or unreachable syslog receivers. The data flow is: 1. An action in Infrahub generates an event (or a permission denied exception) 2. The log forwarding service converts it to a syslog message and enqueues it 3. Each configured destination has its own independent async queue (bounded by `queue_size`, default 10,000) 4. A per-destination consumer task reads from the queue and transmits over the configured protocol 5. TCP connections use exponential backoff reconnection (capped at `max_reconnect_interval`) 6. On graceful shutdown, queues are drained up to `shutdown_drain_timeout` seconds before connections close In the Community Edition, a no-op stub replaces the enterprise implementation. All methods return immediately with zero overhead. note The syslog header hostname defaults to `socket.getfqdn()`. In Docker environments, this resolves to the container ID. Set `hostname` explicitly in the log forwarding configuration when running in containers. ## Message categories[​](#message-categories "Direct link to Message categories") Log forwarding handles two categories of messages, each mapped to a different syslog facility: * **Audit events** (`AUDIT_EVENT`): Generated from Infrahub events representing user and system actions. Uses syslog facility `LOG_AUTH` (4). Always forwarded when a destination is configured. * **Application logs** (`APPLICATION_LOG`): Currently used exclusively for forwarding permission denied exceptions. Uses syslog facility `LOG_LOCAL0` (16). Opt-in per destination via `forward_application_logs`. The facility distinction allows receivers to separate and route audit events independently from application-level messages. ### Forwarded audit event types[​](#forwarded-audit-event-types "Direct link to Forwarded audit event types") The following event types are forwarded as audit events: * **Account**: `infrahub.account.logged_in`, `infrahub.account.logged_out` * **Node**: `infrahub.node.created`, `infrahub.node.updated`, `infrahub.node.deleted` * **Branch**: `infrahub.branch.created`, `infrahub.branch.deleted`, `infrahub.branch.merged`, `infrahub.branch.rebased` * **Proposed change**: `infrahub.proposed_change.approved`, `infrahub.proposed_change.rejected`, `infrahub.proposed_change.merged`, and others * **Schema**: `infrahub.schema.updated` For the complete list of event types and their payload fields, see the [Infrahub events reference](/reference/infrahub-events/overview.md). ### Permission denied events[​](#permission-denied-events "Direct link to Permission denied events") Permission denied events (`infrahub.permission.denied`) are categorized as `APPLICATION_LOG`, not `AUDIT_EVENT`. They are forwarded through the `forward_exception()` path and require `forward_application_logs = true` on the destination to be received. These events carry security context useful for detecting unauthorized access attempts: * `account_id`: ID of the account that made the request * `auth_type`: Authentication method used * `ip_address`: Source IP address of the request * `request_path`: API endpoint or path targeted * `operation`: GraphQL operation name (if applicable) * `query_type`: Whether it was a query or mutation (if applicable) * `graphql_operations`: List of GraphQL operations requested (if applicable) warning If `forward_application_logs` is not enabled on a destination, permission denied events are silently dropped for that destination. ## Syslog format standards[​](#syslog-format-standards "Direct link to Syslog format standards") Infrahub supports two syslog format standards. The format is configured per destination. ### RFC 5424 (default)[​](#rfc-5424-default "Direct link to RFC 5424 (default)") RFC 5424 is the modern syslog standard with structured headers and ISO 8601 timestamps. This is the recommended format for new deployments. Message structure: ``` VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID STRUCTURED-DATA MSG ``` Example: ``` <38>1 2025-01-15T10:30:00.000000Z infrahub.example.com infrahub worker-01 - - {"event":"infrahub.node.created","meta":{"branch":"main","account_id":"01234567-abcd-0000-0000-000000000001"},"kind":"InfraDevice","node_id":"01234567-abcd-0000-0000-000000000002","action":"created"} ``` ### RFC 3164 (legacy)[​](#rfc-3164-legacy "Direct link to RFC 3164 (legacy)") RFC 3164 is the older BSD syslog format with shorter timestamps and a simpler header. Use this when the receiving system does not support RFC 5424. Message structure: ``` TIMESTAMP HOSTNAME TAG[PID]: MSG ``` Example: ``` <38>Jan 15 10:30:00 infrahub.example.com infrahub[worker-01]: {"event":"infrahub.node.created","meta":{"branch":"main","account_id":"01234567-abcd-0000-0000-000000000001"},"kind":"InfraDevice","node_id":"01234567-abcd-0000-0000-000000000002","action":"created"} ``` ### PRI value calculation[​](#pri-value-calculation "Direct link to PRI value calculation") The PRI (priority) value encodes both facility and severity: `PRI = facility * 8 + severity`. For audit events using LOG\_AUTH (facility 4) at INFORMATIONAL severity (6): `4 * 8 + 6 = 38`. The severity codes follow RFC 5424: | Code | Severity | Description | | ---- | ------------- | -------------------------------- | | 0 | Emergency | System is unusable | | 1 | Alert | Action must be taken immediately | | 2 | Critical | Critical conditions | | 3 | Error | Error conditions | | 4 | Warning | Warning conditions | | 5 | Notice | Normal but significant condition | | 6 | Informational | Informational messages | | 7 | Debug | Debug-level messages | ### Message payload[​](#message-payload "Direct link to Message payload") The MSG field always contains a JSON representation of the event. For audit events, this includes the full event payload with metadata (branch, account, timestamps, affected objects). For application log messages such as permission denials, this includes the exception details and request context. ## Transport protocols[​](#transport-protocols "Direct link to Transport protocols") ### UDP (default)[​](#udp-default "Direct link to UDP (default)") UDP is connectionless and fire-and-forget. It offers lower overhead but provides no delivery guarantee. Messages may be lost if the receiver is unavailable or the network drops packets. * Default port: 514 * No connection state or reconnection logic * Suitable for high-volume, loss-tolerant scenarios ### TCP[​](#tcp "Direct link to TCP") TCP provides connection-oriented, reliable delivery with acknowledgment. The sender maintains a persistent connection to the receiver and automatically reconnects with exponential backoff if the connection drops. * Default port: 514 (or 6514 with TLS) * Recommended for compliance use cases where message loss is unacceptable * Two framing options: * `newline` (default): messages are delimited by a newline character * `octet-counting`: messages are length-prefixed per RFC 6587; required by some receivers (notably certain rsyslog configurations) ### TLS[​](#tls "Direct link to TLS") TLS adds encryption on top of TCP. It is only valid with TCP — configuring TLS with UDP raises a validation error. * Default port changes to 6514 when TLS is enabled without an explicit port * `tls_ca_bundle` accepts a file path to a CA certificate bundle or an inline PEM string * The server certificate is validated against the provided CA bundle ## Security considerations[​](#security-considerations "Direct link to Security considerations") * **Use TLS** for any deployment transmitting logs over untrusted networks. Syslog payloads contain sensitive data including account IDs, IP addresses, node details, and permission denial reasons. * **Queue overflow**: When a destination queue fills (default 10,000 messages), new messages are dropped rather than blocking event processing. Size the queue appropriately for your event volume and receiver throughput. * **Receiver authentication**: The syslog protocol itself does not authenticate the sender. Use network-level controls (firewall rules, VPN, network segmentation) in addition to TLS. * **Certificate management**: The `tls_ca_bundle` should point to a properly managed certificate store that is rotated according to your organization's security policies. ## SIEM platform integration[​](#siem-platform-integration "Direct link to SIEM platform integration") Log forwarding works with any syslog-compatible receiver. Brief guidance for common platforms: * **Splunk**: Configure a TCP or UDP syslog input on a Heavy Forwarder or Universal Forwarder. Set the sourcetype to `syslog` or a custom type. RFC 5424 enables structured field extraction. * **Datadog**: Use the Datadog Agent's syslog input or configure a syslog-to-Datadog pipeline. Point Infrahub at the agent's syslog listening port. * **Elastic (ELK)**: Configure Logstash with a `syslog` input plugin or use the Filebeat syslog input for a lighter footprint. RFC 5424 works well with the Logstash syslog codec. * **IBM QRadar / ArcSight**: Use standard syslog ingestion. RFC 5424 is recommended. Configure a log source pointing to Infrahub's hostname. In all cases, configure the receiver to parse JSON from the MSG field to extract structured event data for indexing and alerting. ## Relationship to other features[​](#relationship-to-other-features "Direct link to Relationship to other features") * **Activity log**: The in-app UI for browsing and filtering events. Log forwarding exports the same events to external systems. See [Activity log](/deploy-manage/run-observe/activity-log.md). * **Webhooks**: HTTP callbacks for real-time integrations with custom payloads. Log forwarding uses syslog for compliance and SIEM use cases. See [Webhooks](/webhooks/overview.md). * **Event framework**: Log forwarding consumes the same events that power the activity log, webhooks, and event rules. See [Events](/events/event-system.md) and the [Infrahub events reference](/reference/infrahub-events/overview.md). * **Configuration reference**: All log forwarding parameters are documented in the [configuration reference](/reference/configuration.md#log-forwarding). --- # Tasks The **Tasks system** in Infrahub is designed to manage and control various backend operations with robust error reporting, improved supervision, and enhanced logging capabilities. In Infrahub 1.0, we integrated Prefect as the task manager to orchestrate workflows. The task system is comprised of: * A **Task manager** (based on Prefect) that orchestrates tasks. * One or more **Task workers** (written in Python) that execute operations In Infrahub 1.1, key backend tasks began to be migrated to this framework, and subsequent releases have further refined its functionality. See the [Architecture](/overview/architecture.md) subject for further information about the Task manager and Task worker as well as the overall system design. Key operations managed by the task system include: * Jinja template rendering. * Performing checks and Transform functions. * Carrying out pull, merge, and diff Git operations. More logging and oversight are made possible by these enhancements, which increase background task dependability and transparency. The tasks indicator at the top right of the Infrahub window is a noticeable feature for users. It provides a quick access to examine both in-progress and finished tasks and changes color while a job is active. ![Tasks list](/assets/images/tasks_list-fac908959729979a5d3094379ed5c6e9.png) --- # Local telemetry storage Infrahub stores a daily telemetry snapshot locally in the Neo4j database, regardless of whether remote telemetry reporting is enabled. This ensures all deployments - including air-gapped and opted-out environments - retain usage data for support, auditing, and license compliance. ## How it works[​](#how-it-works "Direct link to How it works") The daily telemetry workflow: 1. Gathers anonymous usage data (schema counts, feature usage, database stats) 2. Stores a `TelemetrySnapshot` in the local Neo4j database 3. If remote telemetry is enabled, sends the data to the remote endpoint 4. Records the remote send status (`sent`, `skipped`, or `failed`) on each snapshot Each snapshot includes a SHA-256 checksum for data integrity verification. ## Viewing stored snapshots[​](#viewing-stored-snapshots "Direct link to Viewing stored snapshots") List stored telemetry snapshots: ``` infrahubctl telemetry list ``` Filter by date range: ``` infrahubctl telemetry list --start-date 2025-01-01 --end-date 2026-01-01 ``` Limit the number of results: ``` infrahubctl telemetry list --limit 10 ``` ## Exporting telemetry data[​](#exporting-telemetry-data "Direct link to Exporting telemetry data") Export all snapshots to a JSON file: ``` infrahubctl telemetry export --output my-telemetry.json ``` Export snapshots from a specific date range: ``` infrahubctl telemetry export \ --start-date 2025-11-01 \ --end-date 2026-02-16 \ --output last-90-days.json ``` The export file contains a JSON array of snapshot objects with full telemetry payloads. ## REST API[​](#rest-api "Direct link to REST API") Retrieve telemetry snapshots programmatically: ``` GET /api/telemetry/snapshots ``` ### Query parameters[​](#query-parameters "Direct link to Query parameters") | Parameter | Type | Default | Description | | ------------ | ----------------- | ------- | ------------------------------------------------ | | `start_date` | string (ISO 8601) | None | Include snapshots created on or after this date | | `end_date` | string (ISO 8601) | None | Include snapshots created on or before this date | | `limit` | integer | 1000 | Maximum number of snapshots to return | | `offset` | integer | 0 | Number of snapshots to skip | ### Example request[​](#example-request "Direct link to Example request") ``` curl -H "Authorization: Bearer $TOKEN" \ "https://infrahub.example.com/api/telemetry/snapshots?start_date=2025-01-01&limit=50" ``` ## Permissions[​](#permissions "Direct link to Permissions") Access to telemetry data requires the `READ_TELEMETRY` global permission. Users with `SUPER_ADMIN` permission have access by default. To grant access to other users, assign the `READ_TELEMETRY` permission to their role via the Infrahub UI or API. ## Backup inclusion[​](#backup-inclusion "Direct link to Backup inclusion") Telemetry snapshots are stored as standard Neo4j nodes and are automatically included in database backups performed with `neo4j-admin database backup`. No additional configuration is required. Restoring a backup with `neo4j-admin database restore` restores all telemetry snapshots. ## Storage estimates[​](#storage-estimates "Direct link to Storage estimates") Each daily snapshot is approximately 3-5 KB. Five years of daily snapshots require less than 50 MB of storage including Neo4j overhead. --- # Authentication Infrahub provides flexible authentication options to fit various organizational needs. You can authenticate users through a local user store maintained within Infrahub or by integrating with external identity providers through single sign-on protocols. This topic explains the authentication mechanisms supported by Infrahub and how they integrate with your existing identity management systems. ## User authentication methods[​](#user-authentication-methods "Direct link to User authentication methods") ### Local user store[​](#local-user-store "Direct link to Local user store") The local user store is Infrahub's built-in user management system. It maintains user accounts directly within Infrahub's database and provides complete control over user authentication without requiring external systems. Key characteristics of the local user store: * User accounts are created and managed directly within Infrahub * Passwords are securely stored using modern cryptographic standards * Ideal for standalone deployments or testing environments ### Single sign-on (SSO)[​](#single-sign-on-sso "Direct link to Single sign-on (SSO)") Single sign-on allows users to authenticate once with an external identity provider and gain access to Infrahub without needing to enter separate credentials. Infrahub integrates seamlessly with popular identity providers such as Microsoft Entra ID (formerly Azure AD), Okta, Google Workspace, and others through industry-standard authentication protocols. Key characteristics of SSO in Infrahub: * Supports both OAuth 2.0 and OpenID Connect (OIDC) protocols * Multiple identity providers can be configured simultaneously * Users authenticated via SSO are automatically created in the local user store * Optional automatic group assignment based on identity provider attributes * Reduces credential management overhead and improves security [Configuring Single Sign-Onsso/overview](/deploy-manage/user-management/sso/overview.md) Recommendation Local user store is suitable for testing and development environments. For production use, integrating with an external identity provider via SSO is recommended for better security and user management. ### LDAP Enterprise Edition[​](#ldap- "Direct link to ldap-") With LDAP authentication, users sign in with the credentials from an existing directory — Active Directory, OpenLDAP, or another RFC 4510-compliant LDAP server. It suits organizations that manage identities in a directory and do not run an OIDC or OAuth2 identity provider. Key characteristics of LDAP authentication in Infrahub: * Users authenticate against the directory through a service-account lookup followed by a credential bind * Accounts are auto-provisioned on first sign-in, the same as SSO * Directory group memberships map to local groups, with support for nested groups * LDAPS and STARTTLS encrypt the connection, with configurable certificate validation * Available in the Enterprise Edition [LDAP authenticationldap/overview](/deploy-manage/user-management/ldap/overview.md) ## OAuth 2.0 vs OpenID Connect[​](#oauth-20-vs-openid-connect "Direct link to OAuth 2.0 vs OpenID Connect") While both protocols are used in authentication flows, they serve different primary purposes: ### OAuth 2.0[​](#oauth-20 "Direct link to OAuth 2.0") OAuth 2.0 is primarily an authorization framework that enables applications to gain limited access to resources on behalf of users. * **Primary purpose**: Authorization (controlling what resources a user can access) * **Use case**: Granting applications limited access to a user's data * **Common scenario**: Third-party applications requesting permissions from services like GitHub, Google, or Microsoft * **Limitation**: Does not define standards for user identity verification ### OpenID Connect (OIDC)[​](#openid-connect-oidc "Direct link to OpenID Connect (OIDC)") OIDC extends OAuth 2.0 by adding a standardized authentication layer. * **Primary purpose**: Authentication (verifying who a user is) + Authorization * **Use case**: Identity verification along with access control * **Common scenario**: Enterprise single sign-on implementations * **Key feature**: Provides an ID Token (JWT) containing identity claims about the authenticated user Recommendation When configuring an external identity provider, OIDC is generally recommended as it provides standardized identity information. [Configuring Single Sign-Onsso/overview](/deploy-manage/user-management/sso/overview.md) ## User permission management[​](#user-permission-management "Direct link to User permission management") Users are allocated permissions through a hierarchical system of groups and roles. * **Users** are members of **Groups** * **Groups** are associated with **Roles** * **Roles** are allocated specific **Permissions** Authentication & Authorization Authentication works in conjunction with Infrahub's authorization system, which controls what actions authenticated users can perform. While authentication verifies who you are, authorization determines what you can do within the system. Authentication is the first step - it creates users and assigns them to groups. Authorization then attaches permissions and roles to those groups. [Roles and permissions./permissions-roles/overview](/deploy-manage/user-management/permissions-roles/overview.md) ## Anonymous access[​](#anonymous-access "Direct link to Anonymous access") By default, Infrahub allows anonymous access in read-only mode. This can be disabled through either: * The configuration parameter: `main.allow_anonymous_access` * The environment variable: `INFRAHUB_ALLOW_ANONYMOUS_ACCESS` warning Disabling anonymous access is recommended for production environments to ensure that only authenticated users can access the system. ## Authentication flow[​](#authentication-flow "Direct link to Authentication flow") When a user attempts to access Infrahub, the following authentication flow occurs: 1. The user is presented with available authentication options (local login, LDAP, and configured SSO providers) info Multiple SSO providers can be configured simultaneously. Even with SSO providers enabled, local login remains available for fallback scenarios (such as when an identity provider is unavailable). 2. Upon successful authentication: * For local users: Credentials are verified against the local user store * For LDAP users: Credentials are verified against the directory through a service-account lookup and a credential bind * For SSO users: The user is redirected to the identity provider, authenticated there, and returned to Infrahub 3. A user session is created after successful authentication 4. For first-time SSO users, a corresponding local user record is automatically created 5. Group memberships are applied: * On a user's first login, the default group (if configured) is assigned, if the authentication provider does not provide any group membership information * On every login, group memberships derived from the provider's claims are added. This is additive — Infrahub never removes a user from a group automatically, even if a claim is dropped 6. The user is granted access according to their assigned permissions ## Application authentication[​](#application-authentication "Direct link to Application authentication") Infrahub supports two authentication methods for applications and integrations: * **JWT Tokens**: Short-lived tokens generated on demand from the API * **API Tokens**: Long-lived tokens generated ahead of time for persistent access ### Authentication method compatibility[​](#authentication-method-compatibility "Direct link to Authentication method compatibility") The table below shows which authentication methods are supported by different Infrahub interfaces: | Interface | JWT Token | API Token | | ------------------ | --------- | --------- | | API / GraphQL | Yes | Yes | | Frontend | Yes | No | | Python SDK | Yes | Yes | | infrahubctl | Yes | Yes | | GraphQL Playground | No | Yes | [Managing API Tokens Guide./managing-api-tokens](/deploy-manage/user-management/managing-api-tokens.md) ## Related concepts[​](#related-concepts "Direct link to Related concepts") * [Roles and permissions](/deploy-manage/user-management/permissions-roles/overview.md) --- # Advanced LDAP configuration Enterprise Edition These options extend a working LDAP setup with directory group resolution, nested-group handling, server failover, and stricter TLS. Configure the connection and user lookup first — see [Configure LDAP authentication](/deploy-manage/user-management/ldap/configure-ldap.md). ## Resolve directory groups[​](#resolve-directory-groups "Direct link to Resolve directory groups") With group resolution enabled, Infrahub looks up the groups a user belongs to and grants the permissions of the local groups whose names match. Group names are matched against **existing** local groups: create the matching `CoreAccountGroup`s first, or auto-create them (see [Auto-create groups](#auto-create-groups-from-ldap)). * environment-variables * infrahub-toml ``` export INFRAHUB_LDAP_GROUP_ENABLED=true export INFRAHUB_LDAP_GROUP_BASE_DN="OU=Groups,DC=corp,DC=example,DC=com" # Group-name attribute, matched against local group names export INFRAHUB_LDAP_GROUP_NAME_ATTRIBUTE="cn" ``` ``` [ldap] group_enabled = true group_base_dn = "OU=Groups,DC=corp,DC=example,DC=com" # Group-name attribute, matched against local group names group_name_attribute = "cn" ``` The group lookup filter defaults to `(member={user_dn})`. The `{user_dn}` placeholder is substituted with the user's distinguished name at sign-in time and escaped to prevent filter injection. Override it with `group_filter` for directories that record membership differently. info When group resolution is turned off, users sign in successfully but receive no permissions until they are assigned to local groups manually. ## Nested groups[​](#nested-groups "Direct link to Nested groups") Directories often nest groups inside other groups. Choose how Infrahub resolves transitive memberships with `group_strategy`: | Strategy | How it resolves nesting | Use when | | --------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------ | | `bfs` (default) | Walks group membership level by level | Any LDAP-compatible directory | | `ad_in_chain` | Uses Active Directory's transitive-membership search to retrieve all nested groups in one query | Active Directory, where it is the fastest option | * environment-variables * infrahub-toml ``` export INFRAHUB_LDAP_GROUP_STRATEGY="ad_in_chain" # bfs only: how many nesting levels to traverse (minimum 10) export INFRAHUB_LDAP_GROUP_BFS_MAX_DEPTH=16 ``` ``` [ldap] group_strategy = "ad_in_chain" # bfs only: how many nesting levels to traverse (minimum 10) group_bfs_max_depth = 16 ``` `group_bfs_max_depth` applies only to the `bfs` strategy. Cycles in the group structure are detected automatically. Raise it above the default of 16 only when groups nest deeper than that — common in large Active Directory forests with deeply nested organizational units. The `ad_in_chain` strategy has no depth limit but is available only against Active Directory. ## Auto-create groups from LDAP[​](#auto-create-groups-from-ldap "Direct link to Auto-create groups from LDAP") By default, only directory groups that already exist as local groups take effect. Infrahub can instead create local groups on demand from the directory group names a user belongs to, so administrators do not have to mirror every directory group manually. The feature is opt-in and scoped by a regular-expression filter. The same mechanism handles SSO claims and LDAP group names. For configuration — the filter syntax, the per-login cap, the `origin` provenance attribute, and the audit events — see [Auto-create groups from claims](/deploy-manage/user-management/sso/advanced-sso.md#auto-create-groups-from-claims). ## Multiple servers and failover[​](#multiple-servers-and-failover "Direct link to Multiple servers and failover") List several server URIs to tolerate an unreachable directory. Infrahub tries them in declaration order, falling through to the next when one does not respond within `per_server_timeout`. * environment-variables * infrahub-toml ``` export INFRAHUB_LDAP_SERVERS="ldaps://dc1.corp.example.com:636,ldaps://dc2.corp.example.com:636" # Seconds to wait for a server before trying the next export INFRAHUB_LDAP_PER_SERVER_TIMEOUT=10 ``` ``` [ldap] servers = ["ldaps://dc1.corp.example.com:636", "ldaps://dc2.corp.example.com:636"] # Seconds to wait for a server before trying the next per_server_timeout = 10 ``` ## Tune for large directories[​](#tune-for-large-directories "Direct link to Tune for large directories") In directories with many entries, keep each sign-in lookup fast and predictable: * **Scope `user_search_base` (and `group_base_dn`) to the narrowest subtree** that contains the relevant entries, so each search examines fewer objects. * **Index the sign-in attribute** (`attribute_username`, such as `sAMAccountName` or `uid`) on the directory side. An attribute that is not indexed forces a full scan on every sign-in. * **Account for `per_server_timeout` during failover.** Each unreachable server is given the full timeout before Infrahub moves to the next, so a sign-in attempt can wait up to `per_server_timeout` × the number of unreachable servers. Keep the timeout tight enough that failover stays within an acceptable sign-in latency. ## Harden the TLS connection[​](#harden-the-tls-connection "Direct link to Harden the TLS connection") For production, verify the directory's certificate against a trusted authority and set a minimum protocol version. * environment-variables * infrahub-toml ``` export INFRAHUB_LDAP_TLS_ENABLED=true # Path to a PEM bundle, or the PEM contents directly; checked at startup export INFRAHUB_LDAP_TLS_CA_BUNDLE="/etc/infrahub/ldap-ca.pem" export INFRAHUB_LDAP_TLS_MINIMUM_VERSION="TLSv1.2" ``` ``` [ldap] tls_enabled = true # Path to a PEM bundle, or the PEM contents directly; checked at startup tls_ca_bundle = "/etc/infrahub/ldap-ca.pem" tls_minimum_version = "TLSv1.2" ``` warning `tls_insecure` skips certificate validation entirely. Restrict it to test and development environments; never enable it in production. ### Choose LDAPS or STARTTLS[​](#choose-ldaps-or-starttls "Direct link to Choose LDAPS or STARTTLS") Both modes give you an encrypted connection; pick one based on the port your directory exposes: * **LDAPS (implicit TLS)** — connect to `ldaps://` URIs on port 636. TLS is established before any LDAP traffic. * **STARTTLS** — connect to plain `ldap://` URIs on port 389 and set `tls_starttls = true` to upgrade the connection in place. Infrahub rejects contradictory combinations at startup: `tls_starttls` cannot be paired with an `ldaps://` URI, and `tls_insecure` cannot be combined with a `tls_ca_bundle`. ## Related resources[​](#related-resources "Direct link to Related resources") * [LDAP authentication overview](/deploy-manage/user-management/ldap/overview.md) * [Configure LDAP authentication](/deploy-manage/user-management/ldap/configure-ldap.md) * [LDAP reference](/reference/ldap.md) * [Permissions and roles](/deploy-manage/user-management/permissions-roles/overview.md) --- # Configure LDAP authentication Enterprise Edition This guide explains how to connect Infrahub to your LDAP directory so users can sign in with their directory credentials. When complete, users will see an LDAP sign-in button on the login page and authenticate against your directory through a service-account lookup. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before configuring LDAP, you need: * An Infrahub deployment running the Enterprise Edition. The Community Edition rejects LDAP sign-ins with an enterprise-required error. * A directory service account (distinguished name and password) with **read-only** access to search the user subtree — and the group subtree, if you enable group resolution. It never needs write permissions; grant it the least privilege required. * The connection details for your directory: server URIs, the user search base, and the attribute that holds the sign-in name. * Administrative access to your Infrahub configuration. ## Steps overview[​](#steps-overview "Direct link to Steps overview") 1. Collect directory information 2. Configure the connection 3. Configure the service account and user lookup 4. Map user attributes 5. Enable LDAP and restart 6. Validate the configuration ## Step 1: Collect directory information[​](#step-1-collect-directory-information "Direct link to Step 1: Collect directory information") Gather the following from your directory administrator: | Information | Example | Notes | | ------------------------ | ------------------------------------------------------ | ---------------------------------------------------------------------------------------- | | Server URIs | `ldaps://dc1.corp.example.com:636` | One or more; the `ldap` or `ldaps` scheme is required | | Service account DN | `CN=infrahub-svc,OU=Service,DC=corp,DC=example,DC=com` | Read-only access is sufficient; used to look up users before verifying their credentials | | Service account password | — | Store as a secret | | User search base | `OU=Users,DC=corp,DC=example,DC=com` | Subtree where user entries live | | Sign-in name attribute | `sAMAccountName` (AD), `uid` (OpenLDAP) | Defaults to `sAMAccountName` | ## Step 2: Configure the connection[​](#step-2-configure-the-connection "Direct link to Step 2: Configure the connection") List your directory servers and choose how to encrypt the connection. Use `ldaps://` URIs (port 636) for implicit TLS, or keep plain `ldap://` URIs (port 389) and set `tls_starttls` to upgrade the connection with STARTTLS. Pick one — combining `tls_starttls` with an `ldaps://` URI is rejected at startup. For production certificate verification, see [Harden the TLS connection](/deploy-manage/user-management/ldap/advanced-ldap.md#harden-the-tls-connection). * environment-variables * infrahub-toml ``` # Tried in declaration order; list the primary first export INFRAHUB_LDAP_SERVERS="ldaps://dc1.corp.example.com:636,ldaps://dc2.corp.example.com:636" # Encrypt the connection export INFRAHUB_LDAP_TLS_ENABLED=true ``` ``` [ldap] # Tried in declaration order; list the primary first servers = ["ldaps://dc1.corp.example.com:636", "ldaps://dc2.corp.example.com:636"] # Encrypt the connection tls_enabled = true ``` warning Do not enable `tls_insecure` outside test and development environments — it skips certificate validation and exposes the connection to interception. For private certificate authorities, supply a CA bundle instead (see [Advanced LDAP configuration](/deploy-manage/user-management/ldap/advanced-ldap.md#harden-the-tls-connection)). ## Step 3: Configure the service account and user lookup[​](#step-3-configure-the-service-account-and-user-lookup "Direct link to Step 3: Configure the service account and user lookup") Infrahub binds with the service account, searches the user subtree for the sign-in name, then re-binds as the located user to verify the password. * environment-variables * infrahub-toml ``` export INFRAHUB_LDAP_SERVICE_ACCOUNT_DN="CN=infrahub-svc,OU=Service,DC=corp,DC=example,DC=com" export INFRAHUB_LDAP_SERVICE_ACCOUNT_PASSWORD="service-account-password" export INFRAHUB_LDAP_USER_SEARCH_BASE="OU=Users,DC=corp,DC=example,DC=com" ``` ``` [ldap] service_account_dn = "CN=infrahub-svc,OU=Service,DC=corp,DC=example,DC=com" service_account_password = "service-account-password" user_search_base = "OU=Users,DC=corp,DC=example,DC=com" ``` info By default the user search filter is generated from the sign-in name attribute, so it stays aligned when you change `attribute_username`. To override it, set `user_search_filter`; the `{username}` placeholder is substituted at sign-in time and escaped to prevent filter injection. ## Step 4: Map user attributes[​](#step-4-map-user-attributes "Direct link to Step 4: Map user attributes") Map directory attributes to the Infrahub account fields. The defaults match Active Directory; adjust them for other directories. * environment-variables * infrahub-toml ``` # sAMAccountName on AD, uid on OpenLDAP export INFRAHUB_LDAP_ATTRIBUTE_USERNAME="sAMAccountName" export INFRAHUB_LDAP_ATTRIBUTE_DISPLAY_NAME="displayName" # AD's disabled-account flag; leave empty on directories without an equivalent attribute to skip the check export INFRAHUB_LDAP_ATTRIBUTE_DISABLED="userAccountControl" ``` ``` [ldap] # sAMAccountName on AD, uid on OpenLDAP attribute_username = "sAMAccountName" attribute_display_name = "displayName" # AD's disabled-account flag; leave empty on directories without an equivalent attribute to skip the check attribute_disabled = "userAccountControl" ``` ## Step 5: enable LDAP and restart[​](#step-5-enable-ldap-and-restart "Direct link to Step 5: enable LDAP and restart") Set `enabled` to turn LDAP sign-in on, then restart the Infrahub server to apply the configuration. * environment-variables * infrahub-toml ``` export INFRAHUB_LDAP_ENABLED=true ``` ``` [ldap] enabled = true ``` [Configure Infrahub../../install-configure/configure-infrahub](../../install-configure/configure-infrahub) Configuration is validated at startup With `enabled = true`, Infrahub refuses to start until `servers`, `service_account_dn`, `service_account_password`, and `user_search_base` are set — plus `group_base_dn` when group resolution is on. A missing or contradictory setting fails fast at startup with a message naming the problem, rather than surfacing later at sign-in. ## Step 6: Validate the configuration[​](#step-6-validate-the-configuration "Direct link to Step 6: Validate the configuration") 1. Navigate to your Infrahub login page. 2. Look for the LDAP sign-in button next to the local login form. Its text comes from `display_label` (default `Sign in with LDAP`). 3. Sign in with a set of directory credentials. 4. Confirm a local account was created for the user under `Admin` > `Users and Permissions` > `Accounts`. success When correctly configured, directory users sign in with their existing credentials and an Infrahub account is provisioned on their first sign-in. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") Every LDAP sign-in attempt is recorded in the Infrahub server logs. Review them for the error returned by the directory. | Symptom | Possible cause | Solution | | -------------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | No LDAP button on the login page | `enabled` is false, or the deployment is running the Community Edition | Set `INFRAHUB_LDAP_ENABLED=true` and confirm the deployment is running the Enterprise Edition | | Sign-in returns `403 ENTERPRISE_REQUIRED` | The deployment is running the Community Edition | Run the Enterprise Edition (see [Community vs enterprise](/overview/community-vs-enterprise.md)) | | Sign-in fails with `401` | Wrong credentials, or the user is not found by the search base and filter | Verify the service account, `user_search_base`, and `attribute_username` | | Sign-in returns `409 LDAP_ACCOUNT_COLLISION` | The directory sign-in name matches an existing local-only account | Reconcile or rename the local account before the user signs in via LDAP | | Sign-in returns `502` | Every configured server was unreachable, or the TLS handshake failed | Check the server URIs and that the servers are reachable, and the TLS settings and CA bundle | ### Check the directory independently[​](#check-the-directory-independently "Direct link to Check the directory independently") When a sign-in fails, reproduce the service-account lookup directly against the directory with `ldapsearch`. This separates a directory-side problem (credentials, search base, filter) from an Infrahub one: ``` ldapsearch -H ldaps://dc1.corp.example.com:636 \ -D "CN=infrahub-svc,OU=Service,DC=corp,DC=example,DC=com" -W \ -b "OU=Users,DC=corp,DC=example,DC=com" \ "(sAMAccountName=jdoe)" ``` A successful bind that returns the expected user entry confirms the service-account credentials, `user_search_base`, and sign-in attribute are correct — which narrows the problem to Infrahub's configuration. Add `-ZZ` to test STARTTLS on a plain `ldap://` connection. ## Related resources[​](#related-resources "Direct link to Related resources") * [LDAP authentication overview](/deploy-manage/user-management/ldap/overview.md) * [Advanced LDAP configuration](/deploy-manage/user-management/ldap/advanced-ldap.md) * [LDAP reference](/reference/ldap.md) * [Authentication](/deploy-manage/user-management/authentication.md) --- # LDAP authentication Enterprise Edition With LDAP authentication, users sign in to Infrahub with the credentials from your existing directory — Active Directory, OpenLDAP, or any RFC 4510-compliant LDAP server. Use it when your organization manages identities in a central directory and does not run an OIDC or OAuth2 identity provider. Enterprise Edition LDAP authentication is available exclusively in the Enterprise Edition. The Community Edition includes the configuration model but rejects LDAP sign-ins with an enterprise-required error. See [Community vs enterprise](/overview/community-vs-enterprise.md) for details. ## When to use LDAP[​](#when-to-use-ldap "Direct link to When to use LDAP") Use LDAP authentication when your directory is the system of record for identity and you want Infrahub access to follow it directly: * Infrahub access tracks the Active Directory or LDAP groups you already maintain — group membership in the directory maps to permissions in Infrahub. * There is no separate set of Infrahub credentials to provision, rotate, or revoke. Joiner, mover, and leaver changes flow through your existing directory processes. * Users sign in with the directory credentials they already have, with no parallel local accounts to keep in step. Infrahub also supports single sign-on through OIDC and OAuth2. OIDC remains the recommended path when an identity provider is available, because it standardizes identity information and supports modern security features such as PKCE. LDAP is the direct-to-directory option for organizations that run a directory but no OIDC or OAuth2 identity provider. ## How LDAP fits with other authentication methods[​](#how-ldap-fits-with-other-authentication-methods "Direct link to How LDAP fits with other authentication methods") LDAP runs alongside local accounts and SSO — all configured methods stay active at the same time. When LDAP is enabled, the login page shows an LDAP sign-in button next to the local login form and any SSO buttons. A successful LDAP sign-in produces the same session as a local or SSO login, so downstream behavior — tokens, permissions, and the activity log — is identical regardless of how the user authenticated. | Method | Choose when | Credentials live in | | ------------------- | ------------------------------------------------------------------------------------------------------------------------ | ---------------------- | | Local accounts | No central directory, small teams, or break-glass access kept for when the directory or identity provider is unreachable | Infrahub | | LDAP | A directory (Active Directory, OpenLDAP) is your source of truth for identity and you do not run an identity provider | Your directory | | SSO (OIDC / OAuth2) | You run an identity provider and want federation and features such as MFA and PKCE | Your identity provider | ## How authentication works[​](#how-authentication-works "Direct link to How authentication works") On each LDAP sign-in, Infrahub: 1. Binds to the directory with a configured service account and searches the user subtree for the entry matching the sign-in name. 2. Re-binds as the located user with the supplied password to verify the credentials. 3. Creates a local account on first sign-in (auto-provisioning), mapping directory attributes to the account's name and display label. 4. Resolves the user's directory groups and maps them to local groups by name, when group resolution is enabled. 5. Issues a session. Multiple directory servers can be configured for high availability, and connections can be encrypted with LDAPS or STARTTLS. ## Account lifecycle[​](#account-lifecycle "Direct link to Account lifecycle") Infrahub resolves identity and group membership on every sign-in rather than syncing the directory in the background. Plan for a few consequences: * **Provisioning is lazy.** A local account is created the first time a directory user signs in, not when they are added to the directory. * **Permission changes apply at the next sign-in.** Because groups resolve on each sign-in, a directory group change takes effect the next time the user authenticates — not immediately. Removing a user from a directory group does not revoke an already-active Infrahub session. * **Removal blocks future sign-ins but does not delete the account.** When a user is removed or disabled in the directory, their next sign-in fails — the service-account lookup no longer finds them, or the disabled-account check rejects the bind. Infrahub does not delete the local account or end active sessions on its own. To revoke access ahead of the directory change, disable or remove the local account under `Admin` > `Users and Permissions` > `Accounts`. ## Supported directories[​](#supported-directories "Direct link to Supported directories") The implementation works against any RFC 4510-compliant LDAP server. The attribute defaults are tuned for Active Directory (`sAMAccountName` for the sign-in name, `userAccountControl` for disabled-account detection), and every attribute is configurable for directories such as OpenLDAP that use different conventions (`uid`, `posixGroup`). ## LDAP guides[​](#ldap-guides "Direct link to LDAP guides") * [Configure LDAP authentication](/deploy-manage/user-management/ldap/configure-ldap.md) — connect Infrahub to your directory and validate sign-in * [Advanced LDAP configuration](/deploy-manage/user-management/ldap/advanced-ldap.md) — group resolution, nested groups, failover, and TLS hardening ## Related resources[​](#related-resources "Direct link to Related resources") * [Authentication](/deploy-manage/user-management/authentication.md) * [Single sign-on (SSO)](/deploy-manage/user-management/sso/overview.md) * [LDAP reference](/reference/ldap.md) * [Permissions and roles](/deploy-manage/user-management/permissions-roles/overview.md) --- # Managing API tokens API tokens can be used as an authentication mechanism for Infrahub's REST- and GraphQL API, the Python SDK and infrahubctl. * Via the Web Interface * Via the GraphQL Interface 1. Login to Infrahub's web interface as an administrator. 2. Click on the user in the left side menu. 3. Navigate to the **Account settings**. 4. In the user Profile page, click on **Tokens** tab. ![Profile Tokens](/assets/images/profile_tokens-b6ffd0794847c56959af9a2b1a51c4af.png) 5. Click **+ Add Account Token**. ![Creating an API Token](/assets/images/profile_tokens_create-7407c11fb07c99f72b504000421e3a63.png) 6. Fill in the details and click **Save**. 7. Copy the generated token and store in a safe location. ![Copy API Token](/assets/images/profile_tokens_copy-99004d3c7729a274ddd9eb04df6390bc.png) 8. Deleting a token can be achieved by selecting the trash icon on the token table item. ## Creating a new API Token[​](#creating-a-new-api-token "Direct link to Creating a new API Token") In the GraphQL sandbox, execute the following mutation, replace the name of the token in the mutation with a value that is appropriate for your use case: ``` mutation { InfrahubAccountTokenCreate(data: {name: "token name"}) { object { token { value } } } } ``` The result of the query will show you the value of the token that was generated for the token. Store the token in a secure location, as there will be no way to retrieve the token from Infrahub at a later stage. ## Listing existing API Tokens for a user[​](#listing-existing-api-tokens-for-a-user "Direct link to Listing existing API Tokens for a user") In the GraphQL sandbox, execute the following query: ``` query { InfrahubAccountToken { edges { node { name expiration id } } } } ``` ## Deleting an API token for a user[​](#deleting-an-api-token-for-a-user "Direct link to Deleting an API token for a user") In the GraphQL sandbox, execute the following mutation, replace the id of the token in the mutation with the id of the token that you want to delete: ``` mutation { InfrahubAccountTokenDelete(data: {id: "17d8cde3-d36b-a0a3-370e-c51707234f19"}) { ok } } ``` ## Using API tokens[​](#using-api-tokens "Direct link to Using API tokens") info While using the API, the authentication token must be provided in the header: ``` X-INFRAHUB-KEY: 06438eb2-8019-4776-878c-0941b1f1d1ec ``` --- # Managing preferences Infrahub stores per-user preferences that control aspects of the web interface and how it displays data. Preferences follow your account to any session or device, and — with the `manage_global_preferences` permission — an administrator can set organisation-wide defaults for users who haven't set their own. Today that covers **date format** and **timezone**, controlling how Infrahub renders dates and times throughout the interface. Both use the same per-user and organisation-wide model, designed so Infrahub can add more personal settings later without changing how preferences resolve. ## Set your preferences[​](#set-your-preferences "Direct link to Set your preferences") * Via the web interface * Via the GraphQL interface 1. Open **Account settings** from the user menu in the left sidebar. 2. On the **Profile** tab, find the **Preferences** card. 3. Select a **Date format** and a **Timezone**. 4. Click **Save**. A field you leave on **Automatic (inherited)** takes the organisation default, or your browser default when no organisation default is set. The information icon next to each field shows where the current value comes from. To clear an override and go back to the inherited value, re-select the value that is currently applied. Write your own preferences with `InfrahubSetPreferences` using `scope: USER`. The mutation only ever writes your own row — there is no account argument. ``` mutation { InfrahubSetPreferences(scope: USER, date_format: EU_DATETIME, timezone: "Europe/Paris") { ok date_format timezone } } ``` An omitted argument leaves that field unchanged. An explicit `null` clears the field, so it falls back to the organisation default or your browser default. Read your resolved values — the ones the interface renders with — with `InfrahubEffectivePreferences`. Each field returns its `value` and the `source` it resolved from (`USER`, `GLOBAL`, or `DEFAULT`), plus `inherited`, the value that field would resolve to if you set nothing of your own. ``` query { InfrahubEffectivePreferences { date_format { value source inherited } timezone { value source inherited } } } ``` `inherited` reports the organisation default, or `null` when none is set and the client applies its own default. When you have no override of your own, it repeats `value`, since that value is already the one you inherit. It is readable without the `manage_global_preferences` permission, so an interface can show the organisation default that a personal override is shadowing. ## Set organisation defaults[​](#set-organisation-defaults "Direct link to Set organisation defaults") Organisation defaults apply to every user who has not set their own value. Setting them requires the `manage_global_preferences` permission (super administrators have it implicitly). See the [permissions reference](/reference/permissions.md) for the full list of permissions. * Via the web interface * Via the GraphQL interface 1. Open **Global preferences** from the user menu in the left sidebar. The menu item is visible only to users with the `manage_global_preferences` permission. 2. Select a **Date format** and a **Timezone**. 3. Click **Save**. A field left on **Automatic (browser default)** sets no organisation default for that field, so users without their own value fall back to their browser default. Write organisation defaults with `InfrahubSetPreferences` using `scope: GLOBAL`. The mutation enforces the `manage_global_preferences` permission before it writes. ``` mutation { InfrahubSetPreferences(scope: GLOBAL, date_format: ISO_DATETIME, timezone: "UTC") { ok date_format timezone } } ``` Read the raw organisation defaults with `InfrahubGlobalPreferences`. This query also requires the `manage_global_preferences` permission. ``` query { InfrahubGlobalPreferences { date_format timezone } } ``` ## Preference precedence[​](#preference-precedence "Direct link to Preference precedence") Infrahub resolves each preference field on its own, in this order: 1. **Your value** — the value you set for yourself. 2. **The organisation default** — the value an administrator set for everyone. 3. **The browser default** — your browser's locale for date formatting, and your browser's timezone. Resolution is per field, so you can take the date format from your own value and the timezone from the organisation default at the same time. A field you never set falls through to the next layer; it is not stored until you set it. ## Available preferences[​](#available-preferences "Direct link to Available preferences") ### Date format[​](#date-format "Direct link to Date format") The date format is a semantic key, not a rendering pattern. Each preset renders both a date and a time. The web interface renders the following presets: | Key | Format | Example | | ------------------------ | -------------------------- | --------------------------- | | `ISO_DATETIME` (default) | `yyyy-MM-dd HH:mm` | `2026-07-01 14:30` | | `ISO_DATETIME_SECONDS` | `yyyy-MM-dd HH:mm:ss` | `2026-07-01 14:30:00` | | `ISO_8601` | `yyyy-MM-dd'T'HH:mm:ssXXX` | `2026-07-01T14:30:00+02:00` | | `EU_DATETIME` | `dd/MM/yyyy HH:mm` | `01/07/2026 14:30` | | `US_12H` | `MM/dd/yyyy hh:mm a` | `07/01/2026 02:30 PM` | When you select a format, the interface shows a live example next to the field. ### Timezone[​](#timezone "Direct link to Timezone") The timezone is an IANA name, such as `Europe/Paris` or `UTC`. The picker lists the timezones your browser supports. A field left unset uses your browser's own timezone. ## Best practices[​](#best-practices "Direct link to Best practices") * **Set an organisation default for the timezone when your teams work in one region.** Everyone then reads timestamps in the same zone until they choose their own. * **Leave the date format on the default (`ISO_DATETIME`) unless your organisation has a house style.** The ISO presets sort and compare the same way in every locale. * **Preferences affect display only.** They change how the web interface renders timestamps; they never change the stored data. A timezone preference renders each timestamp in that zone without altering the underlying value. --- # Manage accounts and permissions In Infrahub, managing access and control starts with creating accounts, assigning them to groups, and managing their roles and permissions. This guide outlines how to create new accounts, accounts groups, and assign roles and permissions. For more information on roles and permissions, see the [Permissions and roles](/deploy-manage/user-management/permissions-roles/overview.md) overview. ## Creating a new account[​](#creating-a-new-account "Direct link to Creating a new account") * Via the Web Interface * Via the GraphQL Interface ### Via the Web Interface[​](#via-the-web-interface "Direct link to Via the Web Interface") 1. Log in to the Infrahub UI as an administrator. 2. Go to **Admin > Role Management** in the left side menu. 3. In the **Accounts** tab, click on **Create Account**. 4. Fill in the account's details (name, email, and password). 5. Optionally, assign the account to a group. 6. Click **Create** to create the account. ![New Account](/assets/images/permissions_account-8e8426b2e619ea4d75eeebd2e495363b.png) ### Via the GraphQL Interface[​](#via-the-graphql-interface "Direct link to Via the GraphQL Interface") In the GraphQL sandbox, execute the following mutation to create a new account, replacing the appropriate values as needed: ``` mutation AddAccount { CoreAccountCreate( data: { name: {value: ""}, password: {value: ""} # Optional - Assign the account to an existing group member_of_groups: [{hfid: "Infrahub Users"}] } ) { ok object { hfid } } } ``` ## Creating a new account group[​](#creating-a-new-account-group "Direct link to Creating a new account group") * Via the Web Interface * Via the GraphQL Interface ### Via the Web Interface[​](#via-the-web-interface-1 "Direct link to Via the Web Interface") 1. Log in to the Infrahub UI as an administrator. 2. Go to **Admin > Role Management** in the left side menu. 3. In the **Groups** tab, click on **Create Account Group**. 4. Enter a name for the group. 5. Optionally, assign roles to the group. 6. Click **Create** to create the group. ![New Group](/assets/images/permissions_group-0217fdf7a62e4f9a4bacd61ce5148056.png) ### Via the GraphQL Interface[​](#via-the-graphql-interface-1 "Direct link to Via the GraphQL Interface") In the GraphQL sandbox, execute the following mutation to create a new group: ``` mutation AddGroup { CoreAccountGroupCreate( data: { name: {value: ""}, # Optional - Assign existing roles roles: [{hfid: "General Access"}] } ) { ok object { hfid } } } ``` ## Creating and assigning roles[​](#creating-and-assigning-roles "Direct link to Creating and assigning roles") * Via the Web Interface * Via the GraphQL Interface ### Via the Web Interface[​](#via-the-web-interface-2 "Direct link to Via the Web Interface") 1. Log in to the Infrahub UI as an administrator. 2. Go to **Admin > Role Management** in the left side menu. 3. In the **Roles** tab, click on **Create Account Role**. 4. Provide a name for the role. 5. Select the permissions you wish to assign to the role. 6. Optionally, assign the role to an existing group. 7. Click **Create** to create the role. ![New Role](/assets/images/permissions_role-a5d527b5c9182c417f093a8ec96d9f45.png) ### Via the GraphQL Interface[​](#via-the-graphql-interface-2 "Direct link to Via the GraphQL Interface") In the GraphQL sandbox, execute the following mutation to create a new role: ``` mutation AddRole { CoreAccountRoleCreate( data: { name: {value: "test role"}, # Optional - Assign the role to an existing group groups: [{hfid: "Infrahub Users"}] } ) { ok object { hfid } } } ``` ## Managing permissions[​](#managing-permissions "Direct link to Managing permissions") Permissions can be managed through roles assigned to users or groups. Infrahub supports **Global** and **Object-specific** permissions, allowing fine-grained control over what users can do within the system. For a complete list of available global and object permissions, see the [Roles and Permissions documentation](/reference/permissions.md). ### Creating and global permissions[​](#creating-and-global-permissions "Direct link to Creating and global permissions") * Via the Web Interface * Via the GraphQL Interface ### Via the Web Interface[​](#via-the-web-interface-3 "Direct link to Via the Web Interface") 1. Log in to the Infrahub UI as an administrator. 2. Go to **Admin > Role Management** in the left side menu. 3. In the **Global Permissions** tab, click on **Create Global Permission**. 4. Select the action you which to use. 5. Select the decision for this action. 6. Optionally, assign the permission to an existing role. 7. Click **Create** to create the permission. ### Via the GraphQL Interface[​](#via-the-graphql-interface-3 "Direct link to Via the GraphQL Interface") In the GraphQL sandbox, execute the following mutation to create a new global permission: ``` mutation AddGlobalPermissions { CoreGlobalPermissionCreate( data: { action: {value: "manage_accounts"}, # 6 is the enum value for "allow" decision: {value: 6} } ) { ok object { identifier { value } } } } ``` ### Creating and objects permissions[​](#creating-and-objects-permissions "Direct link to Creating and objects permissions") * Via the Web Interface * Via the GraphQL Interface ### Via the Web Interface[​](#via-the-web-interface-4 "Direct link to Via the Web Interface") 1. Log in to the Infrahub UI as an administrator. 2. Go to **Admin > Role Management** in the left side menu. 3. In the **Objects Permissions** tab, click on **Create Object Permission**.. 4. Provide the namespace and name of the object(s) you want to interact with. 5. Select the action and decision you wish to use for this permission. 6. Optionally, assign the permission to an existing role. 7. Click **Create** to create the permission. ### Via the GraphQL Interface[​](#via-the-graphql-interface-4 "Direct link to Via the GraphQL Interface") In the GraphQL sandbox, execute the following mutation to create a new global permission: ``` mutation AddObjectPermissions { CoreObjectPermissionCreate( data: { namespace: {value: "Builtin"}, name: {value: "Tag"}, action: {value: "view"}, # 4 is the enum value for "allow_other" decision: {value: 4 } } ) { ok object { identifier { value } } } } ``` --- # Permissions and roles Roles and permissions are essential for controlling user access and behavior in Infrahub. Within the platform, they provide precise control over what users can see, modify, or manage. Permissions fall into two categories: **global** and **object-specific**, while roles act as convenient bundles of permissions. To simplify management, account groups let you manage permissions for multiple users at once. * **GlobalPermissions**: Provide users with system-wide rights to perform specific actions. [See full list of available global permissions](/reference/permissions.md#global-permissions). * **ObjectPermissions**: Are tied to individual objects within Infrahub and control what actions users can take on those objects. [See full list of available object permissions](/reference/permissions.md#object-permissions). * **AccountRoles**: Function as groups of permissions you can assign to accounts. * **AccountGroups**: Allow you to manage permissions for multiple users all at once. ## User permission management[​](#user-permission-management "Direct link to User permission management") Users are allocated permissions through a hierarchical system of groups and roles: * **Users** are members of **Groups** * **Groups** are associated with **Roles** * **Roles** are allocated specific **Permissions** Authentication & Authorization Authentication works in conjunction with Infrahub's authorization system, which controls what actions authenticated users can perform. While authentication verifies who you are, authorization determines what you can do within the system. Authentication is the first step - it creates users and assigns them to groups. Authorization then attaches permissions and roles to those groups. [Authentication/deploy-manage/user-management/authentication](/deploy-manage/user-management/authentication.md) ## Types of permissions[​](#types-of-permissions "Direct link to Types of permissions") ### Global permissions[​](#global-permissions "Direct link to Global permissions") With a global permission, a user can act on the entire system, not just on particular objects. For example, a person with the permission to manage accounts can do so across the entire platform. If the required permission is not granted, the action is blocked. Global permission example Take the `global:manage_accounts:allow_all` permission: * **Action**: `manage_accounts` * **Decision**: `allow_all` This gives the user the ability to manage all user accounts throughout the system. [Permissions and roles reference../../../reference/permissions](/reference/permissions.md) ### Object permissions[​](#object-permissions "Direct link to Object permissions") Object permissions specify actions that apply to a certain kind of object. Actions like create, update, remove, and view are supported. Depending on the kind of object or branch, object permissions may be granted or refused. Key features: * Supports wildcards (`*`) to apply permissions across multiple object types * Can define different permissions per branch type (default or non-default branches) * Grants or denies actions based on the assigned permission Object permission example Here are some examples of object permissions and their descriptions: | Identifier | Object Type | Action | Decision | Description | | --------------------------------- | --------------- | -------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `object:*:*:create:allow_other` | `*` (all types) | `create` | `allow_other` | Allows creating any object, but only on non-default branches. | | `object:*:*:view:allow_all` | `*` (all types) | `view` | `allow_all` | Allows viewing any object, anywhere, across both default and non-default branches. | | `object:Builtin:Tag:update:deny` | `BuiltinTag` | `update` | `deny` | Denies the ability to update any object of type BuiltinTag, across all branches. | | `object:*:Generic:view:allow_all` | `*Generic` | `view` | `allow_all` | Allows viewing all objects that contain 'Generic' in their type (example: LocationGeneric, DeviceGeneric) in all namespaces, across all branches. | [Permissions and roles reference../../../reference/permissions](/reference/permissions.md) ## Future developments[​](#future-developments "Direct link to Future developments") The authorization structure for Infrahub is constantly evolving. Here are some upcoming features: * **Attribute-based permissions**: Grant permissions at the attribute level within objects * **Metadata-based permissions**: Use metadata to specify access controls * **Group-based permissions**: Deepen the integration of group memberships for permission assignments These new features will make Infrahub's permission system even more powerful and flexible in the future. ## Related concepts[​](#related-concepts "Direct link to Related concepts") * [Authentication](/deploy-manage/user-management/authentication.md) * [Permissions and roles reference](/reference/permissions.md) --- # Advanced SSO configuration ## Multiple identity providers[​](#multiple-identity-providers "Direct link to Multiple identity providers") To configure multiple identity providers: * Configure each provider using the same steps as above, but with the appropriate slot name (for example `provider1`, `provider2`) * Make sure to set the `INFRAHUB_SECURITY_OIDC_PROVIDERS` or `INFRAHUB_SECURITY_OAUTH2_PROVIDERS` variable to include all configured providers - environment-variables - infrahub-toml ``` # Configuration for first provider export INFRAHUB_OIDC_PROVIDER1_* # Configuration for second provider export INFRAHUB_OIDC_PROVIDER2_* # Then enable the providers export INFRAHUB_SECURITY_OIDC_PROVIDERS='["provider1", "provider2"]' ``` ``` [security.oidc_provider_settings.provider1] # Configuration for first provider [security.oidc_provider_settings.provider2] # Configuration for second provider [security] # Then enable the providers oidc_providers = ["provider1", "provider2"] ``` ## Group mapping[​](#group-mapping "Direct link to Group mapping") When a user signs in through SSO, Infrahub maps the group claims your identity provider sends to membership in Infrahub account groups. You have three ways to turn a claim into membership: * **[Map to existing groups](#map-to-existing-groups)** — create the groups yourself and match them by name. * **[Auto-create groups](#auto-create-groups-from-claims)** — let Infrahub create groups from the claims on first login. * **[Set a default group](#set-a-default-group)** — assign a fallback group when no claim maps to one. Membership is additive Group memberships derived from provider claims are evaluated on **every** login and are **additive**: each login adds the user to the groups their claims map to. Infrahub never removes a user from a group based on claims — if a group is dropped from the provider, that removal is **not** synced automatically. If an administrator removes a user from a group that still appears in provider claims, the user is added back on the next login. Revoking group membership permanently requires updating provider claims and removing membership in Infrahub. All three rely on your identity provider sending group information in the first place. Configure that first, then pick an approach. [How Infrahub resolves membership](#how-infrahub-resolves-membership) explains how the approaches interact on each login. The [default group](#set-a-default-group) is the one exception: it is applied only on a user's first login, not on every login. ### Send group claims from your identity provider[​](#send-group-claims-from-your-identity-provider "Direct link to Send group claims from your identity provider") Configure your identity provider application to include group information in the authentication tokens sent to Infrahub. info Refer to your provider's documentation for instructions on "group claims" or "configuring OAuth2/OIDC group mappings". warning For OIDC, the `id_token` is verified (signature, audience and issuer) by default before its claims are used. Setting `INFRAHUB_OIDC__ID_TOKEN_VERIFY_SIGNATURE=false` disables this and makes Infrahub trust any token presented to the callback; use it only as a temporary workaround for a misconfigured provider. Every SSO authentication attempt is logged in the Infrahub server logs, including the groups received from your identity provider: ``` SSO user authenticated [infrahub] app=infrahub.api body={'user_name': 'Otto the otter', 'groups': ['Admin Otter']} ``` ### Map to existing groups[​](#map-to-existing-groups "Direct link to Map to existing groups") Create groups in Infrahub whose names match the groups your identity provider sends. On login, Infrahub adds the user to every existing account group whose name matches a received claim. danger Some providers send group IDs instead of display names. Create groups in Infrahub with the exact same IDs your provider sends, and use the label field to store human-friendly names. 1. Navigate to `Admin` > `Users and Permissions` > `Groups` 2. Click `+ Create Account Group` 3. Enter the exact name of the group as sent by your identity provider 4. Optionally, add a description and assign permissions 5. Click `Save` 6. Repeat for each group you want to map success To confirm group mapping is working, log in through SSO and check your user Profile in Infrahub. You should see the groups assigned based on your identity provider's data. ### Auto-create groups from claims[​](#auto-create-groups-from-claims "Direct link to Auto-create groups from claims") Infrahub can create local groups on demand from the claims your identity provider sends, so you don't have to mirror every provider group by hand. The feature is **opt-in**: it activates only when a regular-expression filter is configured. #### How it works[​](#how-it-works "Direct link to How it works") Applied on first login only The default group is assigned only on a user's **first** login, when their account is created. It is not re-applied on subsequent logins, so once an administrator removes a user from the default group — or moves them to a different group — that change sticks and the default group is not added back. On every SSO login, each provider-supplied claim is matched against the configured filter. The first matching pattern derives an **effective name** — either from a `(?P...)` named capture group or, when the pattern has no named capture, from the full matched claim. Infrahub then finds-or-creates a `CoreAccountGroup` with that name and adds the logging-in account as a member. A claim that does not match the filter is ignored. #### Configure the filter[​](#configure-the-filter "Direct link to Configure the filter") * environment-variables * infrahub-toml ``` # Capture everything after `LDAP/group/` as the local group name export INFRAHUB_SECURITY_AUTO_CREATE_GROUPS_FILTER='^LDAP/group/(?P.+)$' # Cap how many new groups one login may create (existing-group reuse is uncapped) export INFRAHUB_SECURITY_AUTO_CREATE_GROUPS_MAX_PER_LOGIN=50 ``` ``` [security] auto_create_groups_filter = "^LDAP/group/(?P.+)$" auto_create_groups_max_per_login = 50 ``` With the example above, a claim `LDAP/group/network-engineering` produces a local group named `network-engineering`. warning Anchor your regex (`^...$`) and scope it tightly. A permissive pattern can create unintended groups on any login. #### Use multiple filters[​](#use-multiple-filters "Direct link to Use multiple filters") The filter also accepts an ordered list of patterns. Each claim is tested against the patterns in declared order, and the first one that matches wins — later patterns are not tried for that claim. Each pattern can use its own `(?P...)` capture, or none. * environment-variables * infrahub-toml ``` # Provide a list as a JSON array string export INFRAHUB_SECURITY_AUTO_CREATE_GROUPS_FILTER='["^LDAP/group/(?P.+)$", "^okta-(?P.+)$"]' ``` ``` [security] auto_create_groups_filter = ["^LDAP/group/(?P.+)$", "^okta-(?P.+)$"] ``` Through an environment variable, the list must be a JSON array string — a comma-separated value is treated as one pattern, not split. Every pattern is compiled at startup; an invalid regex stops startup with an error that names its position in the list. #### Per-login cap[​](#per-login-cap "Direct link to Per-login cap") `auto_create_groups_max_per_login` bounds how many **new** groups a single login can create. Reuse of already-existing groups is uncapped. When the cap is reached, further claims that would require a fresh group are dropped and the login still completes. The login then emits a single `GroupAutoCreateCappedEvent` carrying the cap value and the dropped claims. #### Provenance: the `origin` attribute[​](#provenance-the-origin-attribute "Direct link to provenance-the-origin-attribute") Every auto-created `CoreAccountGroup` row stores the configured **provider name** (for example, `"AzureAD-corp"`) on its `origin` attribute. The value is written verbatim at creation time and is never overwritten on subsequent logins, so the audit trail of which IdP first provisioned a group is preserved. `origin` is read-only and hidden from the default UI; toggle the **Show extra attributes** view to inspect it, or query it via the GraphQL API. #### Audit events[​](#audit-events "Direct link to Audit events") Three event types are emitted on the standard event bus — useful for compliance pipelines and operational dashboards: * `GroupAutoCreatedEvent` — a new group was created from a claim * `GroupAutoCreateRejectedEvent` — a claim matched but produced an invalid effective name (empty or whitespace-only) * `GroupAutoCreateCappedEvent` — the per-login cap was reached and surplus claims were dropped See the [group events reference](/reference/infrahub-events/group.md) for full payload shapes. ### Set a default group[​](#set-a-default-group "Direct link to Set a default group") When your identity provider sends no group information, assign a default group to SSO users. The default group applies to SSO logins only; accounts that sign in with a local username and password are unaffected and keep the groups they already belong to. warning You must create this default group in Infrahub before configuring it here. * environment-variables * infrahub-toml ``` # Set the default group for SSO users export INFRAHUB_SECURITY_SSO_USER_DEFAULT_GROUP='default-group' ``` ``` [security] sso_user_default_group = "default-group" ``` ### How Infrahub resolves membership[​](#how-infrahub-resolves-membership "Direct link to How Infrahub resolves membership") How a claim becomes membership depends on whether auto-creation is enabled: * **Auto-creation enabled** (a filter is configured): Infrahub evaluates each claim against the filter and grants membership to the matching groups, creating them or reusing existing ones. If no claim grants membership, Infrahub assigns the default group, when one is configured. * **Auto-creation disabled** (no filter): Infrahub adds the user to existing account groups whose names exactly match the received claims. When the provider sends no group claims, Infrahub assigns the default group, when one is configured. warning Enabling auto-creation changes how unmatched claims are handled. With a filter configured, only claims that match it grant membership — a claim that doesn't match no longer falls through to an existing group of the same name. If you map claims manually and then enable auto-creation, widen the filter to cover those claims too; auto-creation reuses an existing group with the same name instead of creating a duplicate. success Now that group mapping is configured, manage user permissions in Infrahub by [assigning permissions and roles](/deploy-manage/user-management/permissions-roles/manage-accounts-and-permissions.md) to these groups. ## Related resources[​](#related-resources "Direct link to Related resources") * [SSO overview](/deploy-manage/user-management/sso/overview.md) * [Configure SSO](/deploy-manage/user-management/sso/configure-sso.md) * [Authentication](/deploy-manage/user-management/authentication.md) * [SSO reference](/reference/sso.md) * [Permissions and roles](/deploy-manage/user-management/permissions-roles/overview.md) --- # Configure SSO (single sign-on) This guide walks you through configuring single sign-on (SSO) in Infrahub using OpenID Connect (OIDC) or OAuth2 authentication protocols. When complete, users will be able to authenticate with Infrahub through your existing organizational identity provider, eliminating the need for separate Infrahub credentials. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before configuring SSO, you need: * Administrative access to your identity provider (like Entra ID, Okta, Google Workspace, etc.) * Administrative access to your Infrahub configuration ## Steps overview[​](#steps-overview "Direct link to Steps overview") 1. Choose your protocol and collect necessary information 2. Create the application in your identity provider 3. Configure Infrahub to connect to your identity provider 4. Validate the SSO integration ## Step 1: Prepare and collect information[​](#step-1-prepare-and-collect-information "Direct link to Step 1: Prepare and collect information") ### Choose an authentication protocol[​](#choose-an-authentication-protocol "Direct link to Choose an authentication protocol") Select a protocol based on your identity provider's support. Most modern providers support both OIDC and OAuth2. info OIDC provides standardized user information and is recommended for new implementations. See [Authentication](/deploy-manage/user-management/authentication.md) for details on the differences between OIDC and OAuth2. * oidc * oauth2 OpenID Connect (OIDC) is recommended when available as it provides standardized user Profile information. OAuth2 is a widely supported authorization protocol that may require additional configuration for user Profile mapping. ### Select a provider configuration slot[​](#select-a-provider-configuration-slot "Direct link to Select a provider configuration slot") Infrahub provides six configuration slots for identity providers: * **OIDC slots**: PROVIDER1, PROVIDER2, GOOGLE * **OAuth2 slots**: PROVIDER1, PROVIDER2, GOOGLE warning This guide uses the first provider slot (`PROVIDER1`). For multiple providers, follow the same steps with the appropriate slot name. Google Workspace integration requires the `GOOGLE` slot. ### Determine your redirect URI[​](#determine-your-redirect-uri "Direct link to Determine your redirect URI") When registering Infrahub in your identity provider, you'll need a redirect URI - the URL where users are sent after successful authentication. Use this format: ``` https:///auth///callback ``` For a production Infrahub instance at `infrahub.example.com`: * oidc * oauth2 ``` https://infrahub.example.com/auth/oidc/provider1/callback ``` ``` https://infrahub.example.com/auth/oauth2/provider1/callback ``` Reverse proxy setup Use the `INFRAHUB_PUBLIC_URL` environment variable to specify the externally accessible URL for your Infrahub instance. This configuration is critical when Infrahub is deployed behind a reverse proxy, load balancer, or in containerized environments where the internal and external URLs differ. When configuring SSO ensure this URL matches the redirect URI or callback URL configured in your SSO provider. For a local development instance: * oidc * oauth2 ``` http://localhost:8000/auth/oidc/provider1/callback ``` ``` http://localhost:8000/auth/oauth2/provider1/callback ``` ## Step 2: Configure the identity provider[​](#step-2-configure-the-identity-provider "Direct link to Step 2: Configure the identity provider") ### Create the application in your identity provider[​](#create-the-application-in-your-identity-provider "Direct link to Create the application in your identity provider") Create an application in your identity provider that Infrahub will use for authentication. info While this guide tries to cover common identity providers, if your specific provider isn't listed, please refer to your provider's documentation for instructions on "creating an application registration" or "configuring OAuth2/OIDC integration." **Using Entra ID (formerly Azure AD)** #### Create an application in Entra ID[​](#create-an-application-in-entra-id "Direct link to Create an application in Entra ID") 1. Sign in to the [Microsoft Entra admin center](https://entra.microsoft.com) 2. Navigate to **App registrations** 3. Click **+ New registration** 4. Enter a name for your application (for example "Infrahub") 5. Select the appropriate account type (usually "Accounts in this organizational directory only" for internal use) 6. Under **Redirect URI**: * Select **Web** for the platform * Enter your redirect URI determined earlier 7. Click **Register** #### Create a client secret[​](#create-a-client-secret "Direct link to Create a client secret") 1. In your new app registration, navigate to **Certificates & secrets** 2. Click **+ New client secret** 3. Add a description and select an expiration period 4. Click **Add** danger Copy the **Value** immediately - you won't be able to see it again after navigating away from this page! #### Collect the required information[​](#collect-the-required-information "Direct link to Collect the required information") Make note of the following information for Infrahub configuration: info On the registration overview page, click the **Endpoints** tab to find the necessary URLs. * oidc * oauth2 | Field | Location/Value | | ------------- | -------------------------------------------------------------------------- | | Client ID | The "Application (client) ID" shown on the app overview page | | Client Secret | The value of the client secret you created (only visible at creation time) | | Discovery URL | Called "OpenID Connect metadata document" under the **Endpoints** tab | | Display Label | For example: `Microsoft Entra ID` | | Icon | For example: `mdi:microsoft` | | Field | Location/Value | | ----------------- | --------------------------------------------------------------------------- | | Client ID | The "Application (client) ID" shown on the app overview page | | Client Secret | The value of the client secret you created (only visible at creation time) | | Authorization URL | The "OAuth 2.0 authorization endpoint (v2)" URL under the **Endpoints** tab | | Token URL | The "OAuth 2.0 token endpoint (v2)" URL under the **Endpoints** tab | | Userinfo URL | The value is static: `https://graph.microsoft.com/oidc/userinfo` | | Display Label | For example: `Microsoft Entra ID` | | Icon | For example: `mdi:microsoft` | success At the end of this step, you should have gathered all the necessary information to configure SSO in Infrahub. * oidc * oauth2 | Client ID | Client Secret | Discovery URL | | --------- | ------------- | ------------- | | Client ID | Client Secret | Authorization URL | Token URL | Userinfo URL | | --------- | ------------- | ----------------- | --------- | ------------ | ## Step 3: configure Infrahub[​](#step-3-configure-infrahub "Direct link to Step 3: configure Infrahub") Now configure Infrahub to connect to your identity provider using either environment variables or the `infrahub.toml` configuration file. info The configuration process will require a restart of the Infrahub server and depends on your deployment method. For detailed instructions on how to apply configuration changes to your Infrahub instance, see [Configure Infrahub](/deploy-manage/install-configure/configure-infrahub.md). For detailed configuration options, see the [configuration reference](/reference/configuration.md). For multiple identity providers, refer to the [Advanced Configuration Section](/deploy-manage/user-management/sso/advanced-sso.md#multiple-identity-providers). * oidc * oauth2 - environment-variables - infrahub-toml ``` # Replace values with informations collected during the previous step export INFRAHUB_OIDC_PROVIDER1_CLIENT_ID="client-id" export INFRAHUB_OIDC_PROVIDER1_CLIENT_SECRET="client-secret" export INFRAHUB_OIDC_PROVIDER1_DISCOVERY_URL="discovery-url" # Optional: Set display label and icon for the login screen export INFRAHUB_OIDC_PROVIDER1_DISPLAY_LABEL="display-label" export INFRAHUB_OIDC_PROVIDER1_ICON="mdi:key" # Then enable the provider export INFRAHUB_SECURITY_OIDC_PROVIDERS='["provider1"]' ``` ``` [security.oidc_provider_settings.provider1] # Replace values with informations collected during the previous step client_id = "client-id" client_secret = "client-secret" discovery_url = "discovery-url" # Optional: Set display label and icon for the login screen display_label = "display-label" icon = "mdi:key" [security] # Then enable the provider oidc_providers = ["provider1"] ``` * environment-variables * infrahub-toml ``` # Replace values with informations collected during the previous step export INFRAHUB_OAUTH2_PROVIDER1_CLIENT_ID="client-id" export INFRAHUB_OAUTH2_PROVIDER1_CLIENT_SECRET="client-secret" export INFRAHUB_OAUTH2_PROVIDER1_AUTHORIZATION_URL="authorization-url" export INFRAHUB_OAUTH2_PROVIDER1_TOKEN_URL="token-url" export INFRAHUB_OAUTH2_PROVIDER1_USERINFO_URL="userinfo-url" # Optional: Set display label and icon for the login screen export INFRAHUB_OAUTH2_PROVIDER1_DISPLAY_LABEL="display-label" export INFRAHUB_OAUTH2_PROVIDER1_ICON="mdi:key" # Then enable the provider export INFRAHUB_SECURITY_OAUTH2_PROVIDERS='["provider1"]' ``` ``` [security.oauth2_provider_settings.provider1] # Replace values with informations collected during the previous step client_id = "client-id" client_secret = "client-secret" authorization_url = "authorization-url" token_url = "token-url" userinfo_url = "userinfo-url" # Optional: Set display label and icon for the login screen display_label = "display-label" icon = "mdi:key" [security] # Then enable the provider oauth2_providers = ["provider1"] ``` [Configure Infrahub../../install-configure/configure-infrahub](../../install-configure/configure-infrahub) success After restarting, Infrahub will authenticate users through your configured identity provider. If Infrahub fails to restart, check the [Troubleshooting section](#troubleshooting). ## Step 4: Validate the SSO configuration[​](#step-4-validate-the-sso-configuration "Direct link to Step 4: Validate the SSO configuration") To verify your SSO configuration: 1. Navigate to your Infrahub login page 2. Look for a button labeled with your configured `display_label` next to the standard login form 3. Click the SSO button 4. You should be redirected to your identity provider's login screen 5. After authenticating, you should be redirected back to Infrahub and logged in success When correctly configured, you'll successfully log in through your identity provider. If authentication fails, check the [Troubleshooting section](#troubleshooting). ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") info Every SSO authentication attempt is logged in the Infrahub server logs. Review these logs for detailed error messages that can help diagnose issues. | Issue | Possible Cause | Solution | | ----------------------------- | --------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | "Redirect URI mismatch" error | The redirect URI in your identity provider doesn't match the one Infrahub expects | Verify the redirect URI follows the exact format: `https:///auth///callback` | | "Invalid client" error | Client ID or secret is incorrect | Double-check your client ID and secret values for typos or extra spaces | ## Related resources[​](#related-resources "Direct link to Related resources") * [SSO overview](/deploy-manage/user-management/sso/overview.md) * [Advanced SSO configuration](/deploy-manage/user-management/sso/advanced-sso.md) * [Authentication](/deploy-manage/user-management/authentication.md) * [SSO reference](/reference/sso.md) --- # Single sign-on (SSO) Single sign-on (SSO) allows users to authenticate once with an external identity provider and gain access to Infrahub without needing separate credentials. Infrahub integrates with popular identity providers — Microsoft Entra ID, Okta, Google Workspace, and others — through industry-standard protocols. Infrahub supports two authentication protocols: * **OpenID Connect (OIDC)**: Recommended for new implementations. Provides standardized user Profile information on top of OAuth 2.0. Most modern identity providers support OIDC. * **OAuth 2.0**: Widely supported authorization protocol. May require additional configuration for user Profile mapping. Infrahub provides six configuration slots for identity providers — three OIDC slots (PROVIDER1, PROVIDER2, GOOGLE) and three OAuth2 slots (PROVIDER1, PROVIDER2, GOOGLE). This allows multiple identity providers to be configured simultaneously. When SSO is enabled, users are redirected to the identity provider's login screen. After successful authentication, they are returned to Infrahub and a corresponding local user record is created automatically if one does not already exist. For background on authentication concepts and the differences between OIDC and OAuth2, see [Authentication](/deploy-manage/user-management/authentication.md). ## SSO guides[​](#sso-guides "Direct link to SSO guides") * [Configure SSO](/deploy-manage/user-management/sso/configure-sso.md) — Set up a new SSO integration step by step * [Advanced SSO configuration](/deploy-manage/user-management/sso/advanced-sso.md) — Multiple identity providers and group mapping --- ## [🗃Install & configure](/category/install--configure.md) [6 items](/category/install--configure.md) --- ## [📄️How to work with Git when developing Infrahub](/development/git-best-practices.md) [This guide explains Git workflows and best practices when contributing to Infrahub, covering submodule management, branching strategies, and pull request workflows.](/development/git-best-practices.md) --- ## [📄️Developer Guide](/development-resources/developer-guide.md) [Infrahub support various form of extensibility that rely on users providing their own code that then will be executed by Infrahub.](/development-resources/developer-guide.md) --- # Developer guide Infrahub support various form of extensibility that rely on users providing their own code that then will be executed by Infrahub. This guide provides best practices to help you develop robust, efficient, and maintainable extensions. The following features in Infrahub are based on user provided code : * [**Transformations** (Python and Jinja2)](/transformations/overview.md): Used to generate [artifacts](/artifacts/overview.md) and define [Computed Attributes](/computed-attributes/overview.md). * [**Generators**](/generators/overview.md): Automate complex tasks by generating outputs dynamically. * [**Checks**](/checks/overview.md): Validate your infrastructure with custom rules. ## Development lifecycle[​](#development-lifecycle "Direct link to Development lifecycle") ### Initializing an Infrahub repository[​](#initializing-an-infrahub-repository "Direct link to Initializing an Infrahub repository") Before you start developing extensions or custom code for Infrahub, you need to initialize a repository. This sets up the directory structure and configuration files required for Infrahub to manage your infrastructure data. To initialize a new Infrahub repository, run the following command (replace `` with your desired project name): ``` uv tool run --from 'copier' copier copy https://github.com/opsmill/infrahub-template ``` This uses [copier](https://copier.readthedocs.io/) to create a new Infrahub repository from the [Infrahub Template Repository](https://github.com/opsmill/infrahub-template). The command will guide you through interactive prompts to customize your repository (such as including example Generators, Transformations, and scripts). No separate installation is needed as `uv tool run` handles everything automatically. Once complete, your new repository will be ready for you to add schemas, Transformations, and Generators. Recommended file structure for manually creating an Infrahub repository If you prefer to create your Infrahub repository manually, use the following folder structure as a reference: ``` ├── .gitignore ├── .infrahub.yml ├── .vscode │ └── extensions.json ├── generators ├── lib │ ├── __init__.py │ └── example.py ├── menus ├── objects │ └── example.yml ├── pyproject.toml ├── queries ├── README.md ├── schemas │ └── example.yml ├── scripts │ └── example_script.py ├── tests │ ├── __init__.py │ └── integration │ ├── conftest.py │ └── test_infrahub.py ├── transformations │ └── templates └── uv.lock ``` ### Development[​](#development "Direct link to Development") #### Execute your code locally[​](#execute-your-code-locally "Direct link to Execute your code locally") During development, you can use the `infrahubctl` CLI to execute your code locally. This provides an environment similar to Infrahub's execution environment, enabling quick testing without deploying your code. Key points: * Your code can be executed locally without needing to be checked into Git or pushed to the server. * GraphQL queries, if required, will be discovered and executed locally. * The branch to use in Infrahub can be auto-detected from Git or specified explicitly in the CLI. Refer to the command documentation for details: | Command | Description | | ---------------------------------------------------------------- | --------------------------------------- | | [`infrahubctl transform`](/infrahubctl/infrahubctl-transform.md) | Execute Python Transformations locally. | | [`infrahubctl render`](/infrahubctl/infrahubctl-render.md) | Render Jinja2 Transformations locally. | | [`infrahubctl generator`](/infrahubctl/infrahubctl-generator.md) | Test custom generators. | | [`infrahubctl check`](/infrahubctl/infrahubctl-check.md) | Run local checks | #### Typing & protocols (Python)[​](#typing--protocols-python "Direct link to Typing & protocols (Python)") Infrahub supports Python Typing and Protocols to improve the developer experience by providing static analysis and detecting issues early. What are Protocols? Protocols are a way to define structural subtyping in Python, allowing you to specify expected behavior without requiring inheritance. Infrahub leverages this to enforce typing for flexible schemas. ##### Generating protocols[​](#generating-protocols "Direct link to Generating protocols") You can generate Protocols for your schema with the `infrahubctl protocols` command. The schema can be provided locally or pulled from a running Infrahub instance. ``` infrahubctl protocols --schema my-schema.yaml --output protocols.py ``` > If you don't have a Python module, save the protocols in a protocols.py file in the same directory as your script. ##### Using protocols in code[​](#using-protocols-in-code "Direct link to Using protocols in code") The Python SDK natively supports Protocol in place of `kind` for most functions that interact with objects (`all`, `filters`, `get`, `create`) ``` # With Protocols from infrahub_sdk.protocols import BuiltinTag tags = await client.all(kind=BuiltinTag) for tag in tags: print(tag.name.value) # Type-safe access to the 'name' attribute ``` Same code without Protocols ``` # Without Protocols tags = await client.all(kind="BuiltinTag") for tag in tags: print(tag.name.value) # May raise type errors as attributes are not checked ``` Protocol for Infrahub Schema Protocols for all objects natively provided by Infrahub are available within the SDK in `infrahub_sdk.protocols` ##### Learn more about Python protocols[​](#learn-more-about-python-protocols "Direct link to Learn more about Python protocols") * [Python Protocols: Leveraging Structural Subtyping - Real Python](https://realpython.com/python-protocol/) * [What are "Protocols" In Python? - YouTube](https://www.youtube.com/watch?v=2jN11lyKvfA) #### Git integration[​](#git-integration "Direct link to Git integration") Infrahub integrates seamlessly with Git, allowing you to use your current branch name as the default branch. To enable this feature, set the environment variable: ``` export INFRAHUB_DEFAULT_BRANCH_FROM_GIT=true ``` ### Testing[​](#testing "Direct link to Testing") Testing is crucial for maintaining high-quality code. Infrahub provides a testing framework based on Pytest to simplify unit and integration tests. Refer to the [Testing Framework](/testing-framework/overview.md) for details on validating Transformations, Checks, and other resources inside Infrahub's CI pipeline. For integration tests that run against a full Infrahub instance in Docker, see [Integration testing with Testcontainers](/development-resources/testcontainers.md). Under Construction This section is still under development. Contact the team via Discord for additional resources. ## Logging (Python)[​](#logging-python "Direct link to Logging (Python)") You can use Python's standard logging library to gain insights into the execution of your code. Infrahub captures logs from the `infrahub.tasks` logger by default. The logger `infrahub.tasks` is configure by default and it's possible to configure additional loggers using the settings (see below) ``` import logging log = logging.getLogger("infrahub.tasks") log.info("This log will be captured within the task") ``` ### Configuring additional loggers[​](#configuring-additional-loggers "Direct link to Configuring additional loggers") To capture logs from additional loggers, configure them using the environment variable `INFRAHUB_WORKFLOW_EXTRA_LOGGERS`. The level of the extra loggers can be controlled with the environment variable `INFRAHUB_WORKFLOW_EXTRA_LOG_LEVEL`. ``` export INFRAHUB_WORKFLOW_EXTRA_LOGGERS = '["mylogger", "otherlogger"]' export INFRAHUB_WORKFLOW_EXTRA_LOG_LEVEL = "DEBUG" ``` --- # Using GraphQL fragments GraphQL fragments are reusable field selections that can be shared across multiple stored queries. Instead of duplicating the same nested field selections in every query, you define them once as a fragment and reference them using the standard GraphQL spread syntax (`...fragmentName`). This is particularly useful when multiple [Transformations](/transformations/overview.md) or [Generators](/generators/overview.md) need to query the same fields from a model. Update the fragment once, and all queries that reference it pick up the change on their next execution. ## How fragments work in Infrahub[​](#how-fragments-work-in-infrahub "Direct link to How fragments work in Infrahub") Fragment files are `.gql` files containing one or more `fragment ... on Type { }` definitions. They are declared in the [`.infrahub.yml`](/git-integration/infrahub-yml.md) configuration file under `graphql_fragments`: ``` graphql_fragments: - name: interface_fields file_path: "fragments/interface_fields.gql" ``` When Infrahub imports a query from a Git repository, it checks for fragment spread references (`...fragmentName`). If any are found, the corresponding fragment definitions are resolved from the declared fragment files and included inline in the query before it is stored in the database. ### Fragment resolution behavior[​](#fragment-resolution-behavior "Direct link to Fragment resolution behavior") * **Transitive resolution**: If fragment A references fragment B, both are included automatically * **Duplicate detection**: A fragment name defined in multiple files raises an error * **Circular dependency detection**: Fragments that reference each other in a cycle are rejected * **Directory support**: The `file_path` in `.infrahub.yml` can point to a directory, loading all `.gql` files within it ### Using fragments in queries[​](#using-fragments-in-queries "Direct link to Using fragments in queries") Reference a fragment in any query file declared in the same repository: ``` query DeviceInterfaces($device_name: String!) { InfraDevice(name__value: $device_name) { edges { node { interfaces { edges { node { ...interfaceFields } } } } } } } ``` ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * A running Infrahub instance * An [external Git repository](/git-integration/connect-repository.md) connected to Infrahub (or a local repository for testing with `infrahubctl`) * Familiarity with [GraphQL queries](/development-resources/graphql/overview.md) in Infrahub ## 1. Define a fragment file[​](#1-define-a-fragment-file "Direct link to 1. Define a fragment file") Create a `.gql` file containing one or more fragment definitions. Each fragment targets a specific GraphQL type and selects the fields you want to reuse. Create a file named `fragments/interface_fields.gql`: ``` fragment interfaceFields on InfraInterfaceL3 { name { value } description { value } speed { value } ip_addresses { edges { node { address { value } } } } } ``` info The type after `on` must match a valid model in your Infrahub schema. In this example, `InfraInterfaceL3` refers to the model kind as it appears in GraphQL queries. ## 2. Declare the fragment in `.infrahub.yml`[​](#2-declare-the-fragment-in-infrahubyml "Direct link to 2-declare-the-fragment-in-infrahubyml") Add a `graphql_fragments` section to your `.infrahub.yml` file. Each entry requires a `name` and a `file_path` relative to the repository root. ``` graphql_fragments: - name: interface_fields file_path: "fragments/interface_fields.gql" queries: - name: device_interfaces_query file_path: "queries/device_interfaces.gql" ``` The `file_path` field accepts either a single `.gql` file or a directory. When pointing to a directory, Infrahub loads all `.gql` files within it. ## 3. Reference fragments in a query[​](#3-reference-fragments-in-a-query "Direct link to 3. Reference fragments in a query") Use the standard GraphQL spread syntax (`...fragmentName`) to reference a fragment in any query file declared in the same repository. Create a file named `queries/device_interfaces.gql`: ``` query DeviceInterfaces($device_name: String!) { InfraDevice(name__value: $device_name) { edges { node { name { value } interfaces { edges { node { ...interfaceFields } } } } } } } ``` When Infrahub imports this query from the repository, it detects the `...interfaceFields` spread, resolves it against the declared fragment files, and stores the complete query with the fragment definition included inline. ## 4. Sync the repository[​](#4-sync-the-repository "Direct link to 4. Sync the repository") After committing the fragment file, the `.infrahub.yml` update, and the query file, sync the repository with Infrahub. Fragments are processed during the same repository sync that imports GraphQL queries. If you are testing locally with `infrahubctl`, the fragment resolution happens automatically when rendering Transformations: ``` infrahubctl render device_config_transform device_name=router01 ``` ## 5. Verify the result[​](#5-verify-the-result "Direct link to 5. Verify the result") Open the GraphQL query in the Infrahub web interface or query it via the API. The stored query contains the fragment definition included inline alongside the operation, so it executes without any additional resolution at runtime. ``` curl -s "https:///api/query/device_interfaces_query?device_name=router01" | python -m json.tool ``` ## Advanced usage[​](#advanced-usage "Direct link to Advanced usage") ### Multiple fragments per file[​](#multiple-fragments-per-file "Direct link to Multiple fragments per file") A single `.gql` file can contain multiple fragment definitions: ``` fragment interfaceFields on InfraInterfaceL3 { name { value } speed { value } } fragment addressFields on InfraIPAddress { address { value } prefix_length { value } } ``` Both fragments become available to any query in the repository. ### Directory-based declarations[​](#directory-based-declarations "Direct link to Directory-based declarations") Point `file_path` to a directory to load all `.gql` files within it: ``` graphql_fragments: - name: all_fragments file_path: "fragments/" ``` Infrahub loads all `.gql` files in the `fragments/` directory in alphabetical order. ### Transitive fragments[​](#transitive-fragments "Direct link to Transitive fragments") Fragments can reference other fragments. Infrahub resolves dependencies transitively and detects circular references. ``` fragment deviceDetails on InfraDevice { name { value } interfaces { edges { node { ...interfaceFields } } } } ``` When a query uses `...deviceDetails`, Infrahub automatically includes both `deviceDetails` and `interfaceFields` in the resolved query. ## Error handling[​](#error-handling "Direct link to Error handling") Infrahub validates fragments during repository sync and raises specific errors: | Error | Cause | | --------------------------- | ------------------------------------------------------------------------------------ | | `FragmentFileNotFoundError` | The `file_path` in `.infrahub.yml` does not exist in the repository | | `FragmentNotFoundError` | A query references a fragment name that is not defined in any declared fragment file | | `DuplicateFragmentError` | The same fragment name is defined in multiple files | | `CircularFragmentError` | Two or more fragments reference each other, creating a dependency cycle | ## Related resources[​](#related-resources "Direct link to Related resources") * [GraphQL in Infrahub](/development-resources/graphql/overview.md) — query format, stored queries, and endpoint reference * [`.infrahub.yml` configuration](/git-integration/infrahub-yml.md) — repository manifest structure and resource types * [Creating a Jinja Transformation](/learn/tutorials/transformations/build-a-jinja2-transformation.md) — using queries with Jinja2 templates * [Creating a Python Transformation](/learn/tutorials/transformations/build-a-python-transformation.md) — using queries with Python code --- # Working with groups [Groups](/groups/overview.md) are first-class objects in Infrahub that can be queried and manipulated through GraphQL. Groups provide powerful ways to organize and operate on collections of infrastructure objects. ## Querying groups and their members[​](#querying-groups-and-their-members "Direct link to Querying groups and their members") Query a specific group and its members: ``` query { CoreStandardGroup(name__value: "ProductionRouters") { edges { node { name { value } members { edges { node { display_label __typename } } } } } } } ``` ## Finding groups for an object[​](#finding-groups-for-an-object "Direct link to Finding groups for an object") Every object automatically gains relationships to find its group memberships: ``` query { InfraDevice(name__value: "router01") { edges { node { name { value } member_of_groups { edges { node { name { value } } } } } } } } ``` ## Query groups for bulk operations[​](#query-groups-for-bulk-operations "Direct link to Query groups for bulk operations") Groups enable efficient bulk queries across related objects: ``` query { CoreStandardGroup(name__value: "EdgeDevices") { edges { node { members { edges { node { ... on InfraDevice { interfaces { edges { node { name { value } ip_addresses { edges { node { address { value } } } } } } } } } } } } } } } ``` This pattern enables powerful operations where you can process all objects in a group with a single query, making groups essential for scalable infrastructure management. See [organizing objects with groups](/groups/overview.md) for creating and managing groups, and [understanding groups](/groups/overview.md) for architectural concepts. --- # GraphQL The GraphQL interface is the main interface to interact with Infrahub. The GraphQL schema is automatically generated based on the core models and the user-defined schema models. The endpoint to interact with the main branch is accessible at `https:///graphql`. To interact with a branch the URL must include the name of the branch, such as `https:///graphql/`. If you need to extract the current GraphQL schema in your environment you can issue an HTTP get request to: * `https:///schema.graphql` * `https:///schema.graphql?branch=some-other-branch` ## Introduction to GraphQL videos[​](#introduction-to-graphql-videos "Direct link to Introduction to GraphQL videos") This short demo shows how to use the GraphQL query interface to explore and read data from Infrahub. It walks through how to open the built-in GraphQL interface and run your first queries. This video demonstrates how to use filters and relationships in GraphQL to find specific information, such as IP addresses for a particular device, using the Infrahub query interface. --- # Queries & mutations In GraphQL, a query is used to fetch data and mutations are used to create/update or delete data. In Infrahub, a GraphQL query and 4 mutations will be generated for each model you define in the schema. The name of the query or mutation is based on the namespace and name of the model. For example, for the model `CoreRepository` the following query and mutations have been generated: * `Query` : **CoreRepository** to fetch `CoreRepository` nodes from Infrahub * `Mutation` : **CoreRepositoryCreate** to create a `CoreRepository` node * `Mutation` : **CoreRepositoryUpdate** to update an existing `CoreRepository` node * `Mutation` : **CoreRepositoryUpsert** to create or update a `CoreRepository` node * `Mutation` : **CoreRepositoryDelete** to delete a `CoreRepository` node ## Query format[​](#query-format "Direct link to Query format") The top level query for each model will always return a list of objects and the query will have the following format `CoreRepository` > `edges` > `node` > `display_label` ``` query { CoreRepository { # PaginatedCoreRepository object count edges { # EdgedCoreRepository object node { # CoreRepository object id hfid display_label __typename } } } } ``` info All list of objects will be nested under `edges` & `node` to make it possible to control the pagination and access the attribute `count`. ### `ID`, `hfid` and `display_label`[​](#id-hfid-and-display_label "Direct link to id-hfid-and-display_label") For all nodes, the attribute `id`, `hfid` and `display_label` are automatically available. The value used to generate the `display_label` can be defined for each model in the schema. If no value has been provided a generic display label with the kind and the ID of the Node will be generated. The value used to generate the `hfid` can be defined for each model in the schema. If no value has been provided and the `model` has a single uniqueness constraint defined, then the `hfid` will be automatically generated from the uniqueness constraint. At the object level, there are mainly 3 types of resources that can be accessed, each with a different format: * `Attribute` * `Relationship` of `Cardinality One` * `Relationship` of `Cardinality Many` ### Attribute[​](#attribute "Direct link to Attribute") Each attribute is its own object in GraphQL to expose the value and all the metadata. In the query below, to access the attribute **name** of the object the query must be `CoreRepository` > `edges` > `node` > `name` > `value`. At the same level all the metadata of the attribute are also available, for example: `is_protected`, `source` & `owner` Example query to access the value and the properties of the attribute 'name' ``` query { CoreRepository { count edges { node { name { # TextAttribute object value is_protected source { id display_label } } } } } } ``` ### Relationship of `Cardinality One`[​](#relationship-of-cardinality-one "Direct link to relationship-of-cardinality-one") A relationship to another model with a cardinality of `One` will be represented with a `NestedEdged` object composed of a `node` and a `properties` objects. The `node` gives access to the remote `node` (the peer of the relationship) while `properties` gives access to the properties of the relationship itself. Example query to access the peer and the properties of the relationship 'account', with a cardinality of one. ``` query { CoreRepository { count edges { node { account { properties { is_protected source { id display_label } } node { display_label hfid id } } } } } } ``` ### Relationship of `Cardinality Many`[​](#relationship-of-cardinality-many "Direct link to relationship-of-cardinality-many") A relationship with a cardinality of `Many` will be represented with a `NestedPaginated` object composed. It was the same format as the top level `PaginatedObject` with `count` and `edges` but the child element will expose both `node` and `properties`. The `node` gives access to the remote `node` (the peer of the relationship) while `properties` gives access to the properties of the relationship itself. Example query to access the relationship 'tags', with a cardinality of Many. ``` query { CoreRepository { count edges { node { tags { # NestedPaginatedBuiltinTag object count edges { # NestedEdgedBuiltinTag object properties { is_protected source { id } } node { display_label hfid id } } } } } } } ``` ## Ordering results[​](#ordering-results "Direct link to Ordering results") Every list query and every many-cardinality relationship field accepts an `order` argument to control the sort order of the objects returned. When `order` is omitted, the results use the schema's [default ordering](/schema/default-ordering.md). The `order` argument has the following fields: * `by: [OrderByItem!]` - an ordered list of sort entries. Each entry is an object with two fields: * `field: String!` - what to sort on: an attribute path (`name__value`), a relationship-attribute path (`author__name__value`), or object metadata (`node_metadata__created_at` / `node_metadata__updated_at`). The field carries **no** direction suffix. * `direction: OrderDirection` - `ASC` or `DESC`. Defaults to `ASC` when omitted. Entries apply in order, so the first is the primary sort, the second the tiebreaker, and so on. * `disable: Boolean` - set to `true` to return results without any ordering. * `node_metadata` *(deprecated)* - the legacy field-per-direction form, for example `{ created_at: DESC }`. Use `by` with the `node_metadata__created_at` / `node_metadata__updated_at` fields instead. Order tags newest-first ``` query { BuiltinTag(order: { by: [{ field: "node_metadata__created_at", direction: DESC }] }) { edges { node { name { value } } } } } ``` Sort on more than one field - descending name, then ascending serial number as a tiebreaker. The `direction` on the second entry is omitted, so it defaults to `ASC`: Order by multiple fields ``` query { InfraDevice(order: { by: [{ field: "name__value", direction: DESC }, { field: "serial_number__value" }] }) { edges { node { name { value } } } } } ``` A few rules govern how `order` interacts with the schema default: * When `order` is provided, it **fully replaces** the schema's `order_by` for that query. The two are not combined, and the schema default contributes no fallback tiebreaker. * `by` cannot be combined with the deprecated `node_metadata` field in the same `order` argument. ## Mutations format[​](#mutations-format "Direct link to Mutations format") The format of the mutation to `Create`, `Update` and `Upsert` an object has some similarities with the query format. The format will be slightly different for: * An `Attribute` * A relationship of `Cardinality One` * A relationship of `Cardinality Many` ### Create, update and upsert[​](#create-update-and-upsert "Direct link to Create, update and upsert") To `Create`, `Update` or `Upsert` an object, the mutations will have the following properties. * The input for the mutation must be provided inside `data`. * All mutations will return `ok` and `object` to access some information after the mutation has been executed. * `Update` mutations require you to provide an `id` or `hfid` to identify the object you want to update. * `Upsert` mutations do not require you to provide the `id` or the `hfid`, but enough information needs to be provided for the back-end to uniquely identify the node. Typically this means that all the attribute or relationship values need to be provided that make up the `hfid` or `uniqueness_constraints` of the node. ``` mutation { CoreRepositoryCreate( data: { name: { value: "myrepop" }, # Attribute location: { value: "myrepop" }, # Attribute account: { hfid: ["my_account"] }, # Relationship One tags: [ { hfid: ["my_tag"] } ]} # Relationship Many ) { ok object { id hfid } } } ``` ### Delete[​](#delete "Direct link to Delete") For a `Delete` mutation, we have to provide the `id` or the `hfid` of the node as part of the `data` argument. ``` mutation { CoreRepositoryDelete(data: {hfid: ["myrepo"]}) { ok } } ``` ## Branch management[​](#branch-management "Direct link to Branch management") In addition to the queries and the mutations automatically generated based on the schema, there are some queries and mutations to interact with the branches. * **Query**: `Branch`, Query a list of all branches * **Mutation**: `BranchCreate`, Create a new branch * **Mutation**: `BranchUpdate`, Update the description of a branch * **Mutation**: `BranchDelete`, Delete an existing branch * **Mutation**: `BranchRebase`, Rebase an existing branch with the main branch * **Mutation**: `BranchMerge`, Merge a branch into main * **Mutation**: `BranchValidate`, Validate if a branch has some conflicts --- # Single-target queries When writing GraphQL queries for [transformations](/transformations/overview.md), [generators](/generators/overview.md), [artifacts](/artifacts/overview.md), and [computed attributes](/computed-attributes/overview.md), it's critical to use a **single-target query pattern** to ensure proper tracking by the system. ## What is a single-target query?[​](#what-is-a-single-target-query "Direct link to What is a single-target query?") A single-target query is a GraphQL query that targets a **unique node** using a unique attribute or ID. This pattern enables Infrahub to identify exactly which objects are affected by a change, allowing it to selectively trigger the necessary action instead of everything. ## Why is this important?[​](#why-is-this-important "Direct link to Why is this important?") Without single-target queries, Infrahub cannot determine which specific actions need to be triggered when data changes. This can lead to excessive processing that significantly impacts performance. **Real-world impact:** In one production scenario, a proposed change pipeline generated 600 artifact regenerations when only 5 actually required execution. Properly using single-target queries resolved this issue. ## Requirements for a valid single-target query[​](#requirements-for-a-valid-single-target-query "Direct link to Requirements for a valid single-target query") For a query to be recognized as single-target, it must meet **all** of these criteria: 1. **Filter on a unique identifier**: Use either `id` or a unique attribute like `name__value` 2. **Use a required variable**: The filter must use a required variable, for example, `$name: String!`. A literal value is also valid but limits the query to a single fixed object 3. **Use exact match filters**: Use singular filters, for example, `name__value: $name`, **not** list filters, for example, `name__values: $name` ## Valid single-target query examples[​](#valid-single-target-query-examples "Direct link to Valid single-target query examples") **Using a unique attribute with required variable:** ``` query DeviceConfig($device_name: String!) { InfraDevice(name__value: $device_name) { edges { node { id name { value } interfaces { edges { node { name { value } } } } } } } } ``` **Using ID with required variable:** ``` query DeviceById($device_id: String!) { InfraDevice(ids: [$device_id]) { edges { node { id name { value } } } } } ``` ## Invalid query examples (will cause excessive artifact generation)[​](#invalid-query-examples-will-cause-excessive-artifact-generation "Direct link to Invalid query examples (will cause excessive artifact generation)") **Missing filter (queries all objects):** ``` query AllDevices { InfraDevice { edges { node { id name { value } } } } } ``` **Using optional variable:** ``` query DeviceConfig($device_name: String) { # NOT required (no !) InfraDevice(name__value: $device_name) { edges { node { id } } } } ``` **Using list filter instead of exact match:** ``` query DeviceConfig($device_name: String!) { InfraDevice(name__values: $device_name) { # name__values instead of name__value edges { node { id } } } } ``` **Filtering on non-unique attribute:** ``` query DevicesByRole($role: String!) { InfraDevice(role__value: $role) { # role is not unique edges { node { id } } } } ``` **Part of the query is not unique:** ``` query DeviceConfig($device_name: String!) { InfraDevice(name__value: $device_name) { edges { node { id name { value } } } } BuiltinTag { # This part is not unique edges { node { name { value } } } } } ``` ## Ensuring your query is single-target[​](#ensuring-your-query-is-single-target "Direct link to Ensuring your query is single-target") There is currently no automated way to verify that a query is single-target. The best way to verify is to review the query and ensure it meets all the criteria outlined above. When loaded into the system, Infrahub will analyze the query and determine if it is single-target or not. If it is not single-target, Infrahub will log a warning. It's planned to add more integrated checks in the future to streamline the development process and ensure that queries are correctly structured directly during development. ## When single-target queries are required[​](#when-single-target-queries-are-required "Direct link to When single-target queries are required") Single-target queries are **required** for: * **Python transformations** - See [Creating a Python transformation](/learn/tutorials/transformations/build-a-python-transformation.md) * **Jinja2 transformations** - See [Creating a Jinja transformation](/learn/tutorials/transformations/build-a-jinja2-transformation.md) * **Generators** - See [Generators](/generators/overview.md) * **Artifact definitions** - See [Artifacts](/artifacts/overview.md) * **Computed attributes** - See [Computed attributes](/computed-attributes/overview.md) ## When single-target queries are NOT required[​](#when-single-target-queries-are-not-required "Direct link to When single-target queries are NOT required") You do **not** need single-target queries for: * **Ad-hoc queries** via the GraphQL interface or API * **Reporting queries** that intentionally fetch multiple objects * **Dashboard queries** for UI components * **Bulk data exports** In these cases, you can freely query multiple objects without unique filters. --- # Stored queries Infrahub can store GraphQL queries in the database to simplify execution and associate them with other internal objects such as [Transformations](/transformations/overview.md). ## Stored GraphQL queries in the database[​](#stored-graphql-queries-in-the-database "Direct link to Stored GraphQL queries in the database") The `GraphQLQuery` model has been designed to store a GraphQL query in order to simplify its execution and to associate it with other internal objects like `Transformation`. A `GraphQLQuery` object can be created via the web interface, the API or it can be imported from a Git repository. Every time a `GraphQLQuery` is created or updated, the content of the query will be analyzed to: * Ensure the query is valid and compatible with the schema. * Extract some information about the query itself (see below). ### Information extracted from the query[​](#information-extracted-from-the-query "Direct link to Information extracted from the query") * Type of operations present in the Query \[Query, Mutation, Subscription] * Variables accepted by the query * Depth, number of nested levels in the query * Height, total number of fields requested in the query * List of Infrahub models referenced in the query ### Import from a Git repository[​](#import-from-a-git-repository "Direct link to Import from a Git repository") GraphQL queries could be defined in file(s) with a `.gql` extension in a remote repository. Then queries must also be explicitly identified in the `.infrahub.yml` file under `queries`. More details on the `.infrahub.yml` file format can be found in [.infrahub.yml topic](/git-integration/infrahub-yml.md). ### Executing stored GraphQL queries[​](#executing-stored-graphql-queries "Direct link to Executing stored GraphQL queries") Stored GraphQL queries can be executed by using the `/api/query/{query_id}` REST API endpoint. The `{query_id}` can be the name or the id of the `GraphQLQuery` node in the database. More information can be found in the [Swagger documentation](http://localhost:8000/api/docs). --- # Local demo environment A local environment based on Docker Compose is available for demo and testing. It's designed to be controlled by `invoke` using a list of predefined commands. | Command | Description | | ------------------------ | ------------------------------------------------------------------ | | `demo.pull` | Pull external containers from registry. | | `demo.start` | Start a local instance of Infrahub within Docker Compose. | | `demo.stop` | Stop the running instance of Infrahub. | | `demo.restart` | Restart Infrahub API Server and Task worker within Docker Compose. | | `demo.destroy` | Destroy all containers and volumes. | | `demo.migrate` | Apply the latest database migrations. | | `demo.cli-git` | Launch a bash shell inside the running Infrahub container. | | `demo.cli-server` | Launch a bash shell inside the running Infrahub container. | | `demo.status` | Display the status of all containers. | | `demo.load-infra-schema` | Load the `infrastructure_base` schema into Infrahub. | | `demo.load-infra-data` | Generate some data representing a small network with 6 devices. | | `demo.load-infra-menu` | Generate some data representing a small network with 6 devices. | ## Topology[​](#topology "Direct link to Topology") | Container Name | Image | Description | | ------------------- | ---------------------------- | ----------------------------------------------------------------------------------------- | | **database** | `neo4j:2026.05.0-enterprise` | Graph Database | | **message-queue** | `rabbitmq:4.2.1-management` | Message bus based on RabbitMQ | | **cache** | `redis:8.4.0` | Cache based on Redis, mainly used for distributed lock | | **infrahub-server** | Dockerfile | Instance of the API server, running GraphQL | | **task-worker** | Dockerfile | Instance of the Task worker, executing specific tasks such as managing the Git Repository | | **task-manager** | Dockerfile | Task orchestrator | | **task-manger-db** | `postgres:18-alpine` | Task orchestrator state database | [Check the architecture diagram to have more information about each component../overview/architecture](/overview/architecture.md) ## Getting started[​](#getting-started "Direct link to Getting started") ### Prerequisites[​](#prerequisites "Direct link to Prerequisites") The system that you want to use to run the demo environment has to meet the [hardware requirements](/deploy-manage/install-configure/hardware-requirements.md). In order to run the demo environment, the following applications must be installed on the systems: * [pyinvoke](https://www.pyinvoke.org/) * Docker & Docker Compose info On a Laptop, both Docker & Docker Compose can be installed by installing [Docker Desktop](https://www.docker.com/products/docker-desktop/). ### First utilization[​](#first-utilization "Direct link to First utilization") Initialize the database and start the application ``` invoke demo.start ``` ### Load some data[​](#load-some-data "Direct link to Load some data") Once you have an environment up and running you can load your own schema or you can explore the one provided with the project using the following commands. ``` invoke demo.load-infra-schema invoke demo.load-infra-data ``` ### Control the local environment[​](#control-the-local-environment "Direct link to Control the local environment") * `invoke demo.start` : Start all the containers in detached mode. * `invoke demo.stop` : Stop All the containers * `invoke demo.destroy` : Destroy all containers and volumes. info `invoke demo.debug` can be used as an alternative to `invoke demo.start`, the main difference is that it will stay *attached* to the containers and all the logs will be displayed in real time in the CLI. ## Advanced settings[​](#advanced-settings "Direct link to Advanced settings") ### Support for `sudo`[​](#support-for-sudo "Direct link to support-for-sudo") On a Linux system, the system will try to automatically detect if `sudo` is required to run the Docker command or not. It's possible to control this setting with the environment variable: `INVOKE_SUDO` ``` export INVOKE_SUDO=1 to force sudo export INVOKE_SUDO=0 to disable it completely ``` ### Support for `pty`[​](#support-for-pty "Direct link to support-for-pty") On Linux and MacOS, all commands will be executed with PTY enabled by default. It's possible to control this setting with the environment variable: `INVOKE_PTY` ``` export INVOKE_PTY=1 to force pty export INVOKE_PTY=0 to disable it completely ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") It's recommended to check if all containers are still running using `invoke demo.status`. The 5 containers should be running and be present. * If one is not running, you can try to restart it with `invoke demo.start`. * If the container is still not coming up, you can watch the logs with `docker logs ` (the container name will include the name of the project and a number, i.e., `infrahub-task-worker-1` ). If some containers are still not coming up, it's recommended to start from a fresh install with `invoke demo.destroy`. --- # Image signing and SBOM A software bill of materials (SBOM) is a complete inventory of the components, libraries, and dependencies contained in a piece of software. SBOMs are a core building block for supply-chain security: they let you audit what ships inside an image, match it against vulnerability databases, and satisfy compliance requirements such as those described in the US Executive Order on cybersecurity. OpsMill signs every released Infrahub container image and publishes a signed SBOM alongside it. The sections below cover how both are produced, how to verify that an image was built by an OpsMill release workflow, and how to retrieve and verify the SBOM for the image you deploy. ## How OpsMill generates SBOMs[​](#how-opsmill-generates-sboms "Direct link to How OpsMill generates SBOMs") SBOMs are produced automatically by the release pipeline whenever an Infrahub image is published. You do not need to generate them yourself — they are attached to the published image and are ready to retrieve. The pipeline performs the following steps for each release: 1. **Scan the image**: [Syft](https://github.com/anchore/syft) scans the published multi-architecture image and catalogs every package it contains. 2. **Emit two formats**: Syft writes the inventory in both [SPDX](https://spdx.dev/) (`infrahub-sbom.spdx.json`) and [CycloneDX](https://cyclonedx.org/) (`infrahub-sbom.cdx.json`) JSON. Both are industry-standard formats; most security tooling accepts at least one of them. 3. **Sign as attestations**: [Cosign](https://github.com/sigstore/cosign) attaches each SBOM to the image digest as a signed attestation. Signing is keyless — it uses [Sigstore](https://www.sigstore.dev/) and the release workflow's OpenID Connect identity rather than a long-lived private key, so the attestation is cryptographically tied to the build that produced the image. 4. **Sign the image**: Cosign signs the image digest itself with the same keyless identity. You can verify the image on its own, without retrieving its SBOM. Because the SBOMs are attested against the image digest, they are attached to the image: pulling the image and retrieving its SBOM always returns the inventory for exactly the bits you are running. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") To verify an image and retrieve its SBOM you need: * [Cosign](https://docs.sigstore.dev/system_config/installation/) **v3 or later** installed locally. Cosign v2 is not compatible — it emits a different attestation envelope and the verification commands below will not work with it. * [`jq`](https://jqlang.github.io/jq/) to extract the SBOM document from the attestation payload. * Access to the registry that hosts the image you deploy, for example `registry.opsmill.io/opsmill/infrahub`. Set the image reference and its signing identity as variables so the commands below can be reused. The community and enterprise images are built from different repositories, so each has its own signing identity. For the community image: ``` export IMAGE="registry.opsmill.io/opsmill/infrahub:" export IDENTITY="^https://github\.com/opsmill/infrahub/\.github/workflows/ci-docker-image\.yml@refs/tags/infrahub-v" ``` For the enterprise image: ``` export IMAGE="registry.opsmill.io/opsmill/infrahub-enterprise:" export IDENTITY="^https://github\.com/opsmill/infrahub-private/\.github/workflows/ci-docker-image\.yml@refs/tags/infrahub-enterprise-v" ``` Replace `` with the version you deploy, and verify that reference rather than `latest`. `latest` points to a different image after each release, so verifying it does not confirm the image you are running. The released versions are listed on the [releases page](https://github.com/opsmill/infrahub/releases). Both the signature and the attestations are attached to the image digest, so what you verify matches the version you run. These patterns match release images, which are signed at the release tag that built them. Every other image is signed at the ref its build ran from, and that ref depends on how the build was triggered. A rolling tag can carry either identity, because more than one workflow writes it: `stable` is published both by a release and by the nightly build, so which identity it carries depends on which ran last. That is a further reason to verify the version you deploy rather than a rolling tag. To verify an image that is not a release, run the command with the release pattern first. Verification fails and reports the identity it found, which is the value to pin. Replace the whole ref portion of the pattern, `@refs/tags/infrahub-v` for the community image or `@refs/tags/infrahub-enterprise-v` for the enterprise one, with the ref from that output. ## Verifying the image signature[​](#verifying-the-image-signature "Direct link to Verifying the image signature") `cosign verify` confirms that the image was built and signed by an OpsMill release workflow. Run it before you retrieve the SBOM. ``` cosign verify \ --certificate-identity-regexp "$IDENTITY" \ --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \ "$IMAGE" ``` Cosign lists the checks it performed: ``` Verification for registry.opsmill.io/opsmill/infrahub: -- The following checks were performed on each of these signatures: - The cosign claims were validated - Existence of the claims in the transparency log was verified offline - The code-signing certificate was verified using trusted certificate authority certificates ``` It then prints the signed payloads as JSON. With cosign v3 and no additional flags, those payloads are the SBOM attestations, listed by the types `https://spdx.dev/Document` and `https://cyclonedx.org/bom`. The command confirms the signature, the signing identity and the transparency-log entry, which is what the checks above report. It does not return the SBOM documents themselves. Use `cosign verify-attestation` below to retrieve those. If the identity does not match, verification fails with `no matching CertificateIdentity found`, and the error reports both the pattern it expected and the identity it found. Each pattern names one repository and one tag prefix, so the community pattern rejects an enterprise image and the enterprise pattern rejects a community one. ## Verifying and retrieving the SBOM[​](#verifying-and-retrieving-the-sbom "Direct link to Verifying and retrieving the SBOM") Use `cosign verify-attestation` to confirm the signature is valid **and** return the SBOM in a single step. Because signing is keyless, you must specify the signing identity and the OIDC issuer. The `jq` selector `(.payload // .dsseEnvelope.payload)` below is format-agnostic: it reads the `payload` field from the legacy cosign output and falls back to `dsseEnvelope.payload` for the newer DSSE bundle format that cosign v3 emits. ### SPDX[​](#spdx "Direct link to SPDX") ``` cosign verify-attestation \ --type spdxjson \ --certificate-identity-regexp "$IDENTITY" \ --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \ "$IMAGE" ``` If verification succeeds, cosign prints the signed attestations to standard output. Extract the SPDX document itself with `jq`: ``` cosign verify-attestation \ --type spdxjson \ --certificate-identity-regexp "$IDENTITY" \ --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \ "$IMAGE" \ | jq -r '(.payload // .dsseEnvelope.payload)' | base64 -d | jq '.predicate' > infrahub-sbom.spdx.json ``` ### CycloneDX[​](#cyclonedx "Direct link to CycloneDX") The CycloneDX SBOM is retrieved the same way, using the `cyclonedx` attestation type: ``` cosign verify-attestation \ --type cyclonedx \ --certificate-identity-regexp "$IDENTITY" \ --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \ "$IMAGE" \ | jq -r '(.payload // .dsseEnvelope.payload)' | base64 -d | jq '.predicate' > infrahub-sbom.cdx.json ``` note The `--certificate-identity-regexp` value must match the workflow identity that signed the image. The `$IDENTITY` patterns in [Prerequisites](#prerequisites) pin verification to the release workflow and tag prefix of a single image. A broader pattern such as `^https://github.com/opsmill/.*` also passes, but it accepts any workflow in the `opsmill` GitHub organization and is a weaker trust policy. The `subject.name` field inside an attestation payload is the registry backend hostname, `9r2s1098.c1.gra9.container-registry.ovh.net`, rather than `registry.opsmill.io`. Both names serve the same registry. The attestation is bound to the image digest, which is what cosign checks, so the hostname in that field does not affect verification. ### Download without verifying[​](#download-without-verifying "Direct link to Download without verifying") To pull the attestation without checking the signature — for example, in an environment that cannot reach the Sigstore transparency log — use `cosign download attestation` and filter by predicate type: ``` # SPDX cosign download attestation "$IMAGE" \ | jq -r 'select((.payload // .dsseEnvelope.payload) | @base64d | fromjson | .predicateType == "https://spdx.dev/Document") | (.payload // .dsseEnvelope.payload)' \ | base64 -d | jq '.predicate' > infrahub-sbom.spdx.json # CycloneDX cosign download attestation "$IMAGE" \ | jq -r 'select((.payload // .dsseEnvelope.payload) | @base64d | fromjson | .predicateType == "https://cyclonedx.org/bom") | (.payload // .dsseEnvelope.payload)' \ | base64 -d | jq '.predicate' > infrahub-sbom.cdx.json ``` warning `cosign download attestation` does not verify the signature. Prefer `cosign verify-attestation` whenever your environment can reach Sigstore so you confirm the SBOM was produced by a trusted OpsMill build. ## Inspecting the SBOM[​](#inspecting-the-sbom "Direct link to Inspecting the SBOM") Once retrieved, the SBOM is a standard JSON document you can feed into your existing tooling. For example, scan it for known vulnerabilities with [Grype](https://github.com/anchore/grype): ``` grype sbom:infrahub-sbom.spdx.json ``` Or list the cataloged packages directly with `jq`: ``` jq -r '.packages[].name' infrahub-sbom.spdx.json | sort -u ``` ## Reference[​](#reference "Direct link to Reference") | Artifact | Format | Filename | Standard | | -------------- | -------------- | ------------------------- | --------------------------------------- | | SPDX SBOM | SPDX JSON | `infrahub-sbom.spdx.json` | [spdx.dev](https://spdx.dev/) | | CycloneDX SBOM | CycloneDX JSON | `infrahub-sbom.cdx.json` | [cyclonedx.org](https://cyclonedx.org/) | Both SBOMs are attached to every published image digest as Cosign attestations and cover the same image; choose whichever format your security tooling consumes. --- # Integration testing with Testcontainers The `infrahub-testcontainers` package starts a complete Infrahub stack in Docker during your test session. Use it to write integration tests that validate schemas, SDK usage, or GraphQL queries against a live Infrahub instance. Testcontainers vs. the built-in testing framework Infrahub ships with a [built-in testing framework](/testing-framework/overview.md) for validating Transformations, Checks, and other resources defined in a connected Git repository. `infrahub-testcontainers` serves a different purpose: it provisions a full Infrahub instance on demand so that external code — custom integrations, schemas under development, or SDK scripts — can be tested without maintaining a persistent environment. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Docker Engine installed and running * Python 3.10 or later * At least 4 GB of available RAM (Neo4j requires significant memory to start) * Sufficient disk space for the Docker images (approximately 2 GB on first pull) ## Installation[​](#installation "Direct link to Installation") Install both `infrahub-testcontainers` and the Python SDK. The SDK provides the `TestInfrahubDockerClient` base class used in all examples below. ``` pip install infrahub-testcontainers infrahub-sdk ``` Or with `uv`: ``` uv add infrahub-testcontainers infrahub-sdk ``` ## Quick start[​](#quick-start "Direct link to Quick start") `TestInfrahubDockerClient` (from `infrahub_sdk.testing.docker`) extends `TestInfrahubDocker` with pre-configured `client` (async) and `client_sync` (sync) fixtures that point at the running stack. The example below covers the three steps that appear in almost every project: load a schema, populate initial data, and connect a Git repository. ``` from pathlib import Path from typing import Any import pytest from infrahub_sdk import InfrahubClient, InfrahubClientSync from infrahub_sdk.protocols import CoreGenericRepository from infrahub_sdk.testing.docker import TestInfrahubDockerClient from infrahub_sdk.testing.repository import GitRepo from infrahub_sdk.yaml import SchemaFile class TestMyProject(TestInfrahubDockerClient): def test_schema_load( self, client_sync: InfrahubClientSync, schema_dir: Path, default_branch: str, ) -> None: schema_files = SchemaFile.load_from_disk(paths=[schema_dir]) schemas = [f.content for f in schema_files if f.content] client_sync.schema.load(schemas=schemas) client_sync.schema.wait_until_converged(branch=default_branch) async def test_data_load(self, client: InfrahubClient, default_branch: str) -> None: tag = await client.create(kind="BuiltinTag", name="production") await tag.save() tags = await client.all(kind="BuiltinTag") assert len(tags) == 1 async def test_load_repository( self, client: InfrahubClient, root_dir: Path, remote_repos_dir: Path, ) -> None: repo = GitRepo( name="my-project", src_directory=root_dir, dst_directory=remote_repos_dir, ) await repo.add_to_infrahub(client=client) in_sync = await repo.wait_for_sync_to_complete(client=client, interval=10, retries=30) assert in_sync repos = await client.all(kind=CoreGenericRepository) assert repos ``` Run the tests with: ``` pytest tests/ ``` The `infrahub_app` fixture starts the full stack — Infrahub server, Neo4j, RabbitMQ, Redis, Prefect task manager, and task workers — before any test in the class runs. After the last test completes, the stack is stopped and containers are removed. If startup fails, the container logs are printed automatically. ## Conftest setup[​](#conftest-setup "Direct link to Conftest setup") The `root_dir`, `schema_dir`, and `data_dir` fixtures used in the examples above are not provided by the library — define them in `conftest.py` for your project: ``` # tests/conftest.py from pathlib import Path from typing import Any import pytest from infrahub_sdk.yaml import SchemaFile @pytest.fixture(scope="session") def root_dir() -> Path: return Path(__file__).parent.parent @pytest.fixture(scope="session") def schema_dir(root_dir: Path) -> Path: return root_dir / "schemas" @pytest.fixture(scope="session") def data_dir(root_dir: Path) -> Path: return root_dir / "data" ``` ## Available fixtures[​](#available-fixtures "Direct link to Available fixtures") `TestInfrahubDockerClient` provides all fixtures from `TestInfrahubDocker`, plus two SDK client fixtures: | Fixture | Type | Description | | ------------------- | -------------------- | --------------------------------------------------------------- | | `client` | `InfrahubClient` | Async SDK client pointed at the running stack | | `client_sync` | `InfrahubClientSync` | Sync SDK client pointed at the running stack | | `infrahub_app` | `dict[str, int]` | Starts the stack; yields a mapping of service name to host port | | `infrahub_port` | `int` | Host port for the Infrahub API server | | `task_manager_port` | `int` | Host port for the Prefect task manager | | `tmp_directory` | `Path` | Temporary directory shared across the test class | | `remote_repos_dir` | `Path` | Subdirectory used for local Git repositories | | `default_branch` | `str` | Returns `"main"` — override in your class to change it | `client` and `client_sync` authenticate with username `admin` and password `infrahub`. Using TestInfrahubDocker directly `TestInfrahubDockerClient` is the recommended starting point for most projects. Use `TestInfrahubDocker` directly only when you need a different authentication mechanism or want to instantiate the SDK client yourself. ## Configuration[​](#configuration "Direct link to Configuration") The following environment variables control the test stack: | Variable | Default | Description | | ------------------------------------- | -------------------------------------- | --------------------------------------------------------------------- | | `INFRAHUB_TESTING_IMAGE_VER` | Package version | Docker image tag to pull and run | | `INFRAHUB_TESTING_DOCKER_IMAGE` | `registry.opsmill.io/opsmill/infrahub` | Docker image to use | | `INFRAHUB_TESTING_ENTERPRISE` | — | Set to any value to switch to the enterprise image | | `INFRAHUB_TESTING_DOCKER_PULL` | `true` | Set to `false` to skip image pulls (useful with locally built images) | | `INFRAHUB_TESTING_SCHEMA_STRICT_MODE` | `true` | Reject schemas that contain unknown or invalid fields | | `INFRAHUB_TESTING_WEB_CONCURRENCY` | `4` | Number of Gunicorn worker processes for the API server | | `INFRAHUB_TESTING_API_SERVER_COUNT` | `2` | Number of API server replicas | | `INFRAHUB_TESTING_TASK_WORKER_COUNT` | `2` | Number of task worker replicas | The full set of variables and their defaults is defined in `infrahub_testcontainers.container.PROJECT_ENV_VARIABLES`. ## Deployment types[​](#deployment-types "Direct link to Deployment types") By default, the stack starts a single-node Infrahub instance. To test against a Neo4j cluster topology, pass `--deployment-type cluster` to pytest: ``` pytest tests/ --deployment-type cluster ``` Cluster mode requires Neo4j Enterprise edition. Set `INFRAHUB_TESTING_ENTERPRISE=1` before running. ## Pausing a test to inspect the running instance[​](#pausing-a-test-to-inspect-the-running-instance "Direct link to Pausing a test to inspect the running instance") When debugging a failing test, it can help to pause execution mid-run and inspect the live Infrahub instance directly — via the UI, the SDK, or `infrahubctl`. Add a `breakpoint()` call at the point in your test where you want to pause: ``` async def test_load_repository(self, client: InfrahubClient, ...) -> None: repo = GitRepo(...) await repo.add_to_infrahub(client=client) breakpoint() # execution pauses here; containers stay up in_sync = await repo.wait_for_sync_to_complete(client=client) assert in_sync ``` Run pytest with `-s` to keep stdin attached (required for the interactive prompt): ``` pytest tests/ -s ``` When the breakpoint is hit, pytest drops into a Python debugger (pdb). While paused, the Infrahub containers are still running. You can find the port from the `infrahub_port` fixture value printed in the test output, or look it up in the pdb session: ``` (Pdb) print(infrahub_port) 52341 ``` Open `http://localhost:` in a browser to access the Infrahub UI, or point `infrahubctl` at it: ``` INFRAHUB_ADDRESS=http://localhost:52341 infrahubctl schema list ``` Type `c` (continue) in the pdb prompt to resume the test and let the containers shut down normally. Keeping containers up after a test failure If the test fails before reaching a breakpoint, the `infrahub_app` fixture still tears down the containers. To keep them running after a failure for post-mortem inspection, override the fixture in your test class: ``` @pytest.fixture(scope="class") def infrahub_app(self, request, infrahub_compose): infrahub_compose.start() yield infrahub_compose.get_services_port() # containers are NOT stopped — remember to clean up manually ``` Remove this override once you're done debugging. ## Running in CI[​](#running-in-ci "Direct link to Running in CI") `TestInfrahubDockerClient` works in any CI environment that can run Docker. The example below shows a GitHub Actions job: ``` jobs: integration: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: "3.12" - run: pip install infrahub-testcontainers infrahub-sdk pytest pytest-asyncio - run: pytest tests/ env: INFRAHUB_TESTING_IMAGE_VER: "1.9.0" ``` Resource-constrained runners On runners with limited RAM (the free GitHub-hosted runners have 7 GB), set `INFRAHUB_TESTING_WEB_CONCURRENCY=2` and `INFRAHUB_TESTING_API_SERVER_COUNT=1` to reduce memory pressure. ## Real-world examples[​](#real-world-examples "Direct link to Real-world examples") The following public repositories show complete integration test setups: | Repository | What it demonstrates | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | [infrahub-demo-dc — test\_workflow.py](https://github.com/opsmill/infrahub-demo-dc/blob/main/tests/integration/test_workflow.py) | End-to-end DC workflow: schema load, bootstrap data, repository import, branch creation, Generator run, proposed change, and merge | | [infrahub-demo-service-catalog — test\_create\_service.py](https://github.com/opsmill/infrahub-demo-service-catalog/blob/main/tests/integration/test_create_service.py) | Schema and data load using `ObjectFile`, repository import, and a Streamlit portal test | | [devnet-live-2025 — test\_end2end.py](https://github.com/opsmill/devnet-live-2025/blob/main/tests/integration/test_end2end.py) | Schema load from YAML files, data population, and repository sync | --- # Backend guide In order start developing on Infrahub backend, it is recommended to have a decent knowledge about topics such as Docker, Python and generally UNIX systems. Tools such as Docker and Python virtual environment help us in isolating the development work without interfering with the system itself. In this guide, we will use: * [Python](https://www.python.org/) to be able to run the code * [Invoke](https://www.pyinvoke.org/) to run some commands bundled with Infrahub * [uv](https://docs.astral.sh/uv/) to manage our Python virtual environment * [Docker](https://www.docker.com/) and its Compose extension to run dependencies such as the database, cache and queueing system To fetch Infrahub's code, we will use Git and we will use the `develop` branch (default). ``` git clone --recursive git@github.com:opsmill/infrahub.git cd infrahub ``` ## Basic settings[​](#basic-settings "Direct link to Basic settings") Most of Infrahub and tools around it rely on some settings. These settings are in general set as environment variables, dealing with many of these can be hard to maintain and manage. We can use a tool such as [direnv](https://direnv.net/) to help. It allows to define environment variables (or pretty much anything bash can make sense of) in a file that will be interpreted when entering a given directory. Here is an example of a `.envrc` file providing development friendly setting values: ``` export INFRAHUB_PRODUCTION=false export INFRAHUB_SECURITY_SECRET_KEY=super-secret export INFRAHUB_USERNAME=admin export INFRAHUB_PASSWORD=infrahub export INFRAHUB_TIMEOUT=20 export INFRAHUB_METRICS_PORT=8001 export INFRAHUB_DB_TYPE=neo4j # Accepts Neo4j or Memgraph export INFRAHUB_INITIAL_ADMIN_TOKEN="${ADMIN_TOKEN}" # Random string which can be generated using: openssl rand -hex 16 export INFRAHUB_STORAGE_LOCAL_PATH="${HOME}/Development/infrahub-storage" export INFRAHUB_API_CORS_ALLOW_ORIGINS='["http://localhost:8080"]' # Allow frontend/backend communications without CORS issues ``` The exported environment variables are very important and must be set before moving to another step. Without these, you will likely face some errors or issues later. ## Required services[​](#required-services "Direct link to Required services") Infrahub uses several external services to work: * A Neo4j database * A Redis in-memory store * A RabbitMQ message broker To run all these services, we will use Docker, but for local development some ports will need to be bound to local ones. To do so, a very basic Docker Compose override file is provided in the `development` directory, but it has a `tmp` extension which makes Compose ignore it by default. We will copy this file to a new one without the `tmp` extension. In a development environment, having only the services in Docker and Infrahub running on local Python is convenient to take advantage of the server auto-reload feature when making changes. ``` cp development/docker-compose.dev-override.yml.tmp development/docker-compose.dev-override.yml ``` Now we need to make sure we have a compatible version of Python that Infrahub can run on top of, uv to create virtual environment and Invoke to run commands. Invoke can be installed in many ways, but we recommend to use the `pipx` way to get it available user wide while without messing with the system Python. Assuming we have these utilities ready, we can run the following commands to build a proper Python environment: ``` cd infrahub # or the directory of your choice uv sync --all-groups ``` Some tests require some services to work. By default, they are automatically started by pytest before tests run. It is also possible to disable the automatic startup of services and to rely on existing services using an environment variable: ``` export INFRAHUB_USE_TEST_CONTAINERS=false ``` To build the required services locally, run the following command: ``` uv run invoke dev.build ``` The required services now need to be started using dedicated commands: ``` uv run invoke dev.destroy dev.deps ``` This will actually pass two commands, one to destroy any remains of a previous run and one to start services. So this will effectively bring up clean services without leftovers. We can see which services are running by using: ``` uv run invoke dev.status ``` This should yield a Docker output like the following: docs ``` NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS infrahub-cache-1 redis:7.2 "docker-entrypoint.s…" cache 2 hours ago Up 2 hours (healthy) 0.0.0.0:6379->6379/tcp infrahub-database-1 memgraph/memgraph:2.13.0 "/usr/lib/memgraph/m…" database 2 hours ago Up 2 hours (healthy) 0.0.0.0:7444->7444/tcp, 0.0.0.0:7474->7474/tcp, 0.0.0.0:7687->7687/tcp infrahub-message-queue-1 rabbitmq:3.12-management "docker-entrypoint.s…" message-queue 2 hours ago Up 2 hours (healthy) 4369/tcp, 5671/tcp, 0.0.0.0:5672->5672/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:15672->15672/tcp ``` When following a guide, like the [installation guide](/deploy-manage/install-configure/install/overview.md), the command `demo.start` is mentioned. It is slightly different from the `dev.deps` that is mentioned here. The `demo.start` will bring up a demo environment as a whole including services and Infrahub while the `dev.deps` will only start the services as seen in the code block above. ## Running Infrahub test suite[​](#running-infrahub-test-suite "Direct link to Running Infrahub test suite") With the required services working and properly setup Python virtual environment we can now run the Infrahub test suite to make sure the code works as intended. ``` INFRAHUB_LOG_LEVEL=CRITICAL uv run pytest -v backend/tests/unit ``` The environment variable at the beginning of the command is useful to have a much more cleaner output when running tests. ## Running Infrahub server[​](#running-infrahub-server "Direct link to Running Infrahub server") We can run the Infrahub server with the built-in command: ``` infrahub server start --debug ``` The `debug` flag allows the server to be reloaded when a change is detected in the source code. Note that this will only make the backend service usable, the frontend will not be available. Only Swagger documentation should be available at `http://localhost:8000/api/docs`. GraphQL sandbox is available through the frontend. For running the frontend, please refer to its [dedicated documentation section](/development/frontend.md). ## Loading a new schema via CLI[​](#loading-a-new-schema-via-cli "Direct link to Loading a new schema via CLI") For testing code changes, you may want to load a new schema from a YAML file. This can be performed in the development environment using: ``` uv run infrahubctl schema load ${PATH_TO_SCHEMA_FILE} ``` ## Code format[​](#code-format "Direct link to Code format") Formatting code in the backend relies on [Ruff](https://docs.astral.sh/ruff/) and [yamllint](https://yamllint.readthedocs.io/en/stable/). To ensure all files are as close as possible to the expected format, it is recommended to run the `format` command: ``` uv run invoke format ``` ## Python SDK[​](#python-sdk "Direct link to Python SDK") The Infrahub backend and workers rely on the `infrahub-sdk` package to function, and occasionally, updates or additions to the repository are necessary. This guide will help you ensure that the SDK is correctly initialized, updated, and maintained. ### Cloning the repository with submodules[​](#cloning-the-repository-with-submodules "Direct link to Cloning the repository with submodules") If you find that the `python_sdk` folder is empty, it may be because the repository was not cloned with the `--recursive` flag. This can be fixed using the following command: ``` git submodule update --init ``` To prevent this issue in the future, always clone the repository with submodules using: ``` git clone --recursive git@github.com:opsmill/infrahub.git ``` To get the latest commits for both `Infrahub` and the `infrahub-sdk` use `invoke pull`. --- # Changelog guide Infrahub utilizes a tool called [`towncrier`](https://towncrier.readthedocs.io/) for Changelog management and generation. The Changelog is maintained in a `CHANGELOG.md` file in the main directory of each package in this repository. Multiple Changelogs can be merged into a single Release Note for the overall project. Infrahub follows the best practices from [Keep a Changelog](https://keepachangelog.com/), and our categories of change are: * **Added** for new features * **Changed** for changes in existing functionality * **Deprecated** for soon-to-be removed features * **Removed** for now removed features * **Fixed** for any bug fixes * **Security** in case of vulnerabilities * **Housekeeping** for internal maintenance and tooling (build, CI, dev scripts) — an Infrahub addition beyond the Keep a Changelog set ## Creating changelog entries[​](#creating-changelog-entries "Direct link to Creating changelog entries") What this means in practice for contributing to Infrahub is: 1. Any PR to the `develop` (or `stable`) branch that closes an issue should contain at least one Markdown formatted "Newsfragment" file in the related `changelog` directory * `changelog/` for Infrahub changes * `python-sdk/changelog` for Infrahub SDK changes 2. This file should be named with the format `..md`. For example: * A Newsfragment file named `1234.fixed.md` represents a bug fix PR closing GitHub Issue #1234 * This allows `towncrier` to populate the correct section of the Changelog with the relevant information * *Note*: If a PR doesn't close (or at least reference) a specific issue, you can utilize plain-text for the prefix by appending `+` to the filename. * For example: `+this_is_not_a_github_issue.fixed.md` * *Note*: Use the GitHub issue number, or `+` if there is no GitHub issue — never an internal ticket ID. `towncrier`'s `issue_format` turns the fragment name directly into a GitHub issue URL, so an internal ID ships as a broken link in the changelog. 3. Available Newsfragment suffixes are: * `added` * `changed` * `deprecated` * `removed` * `fixed` * `security` * `housekeeping` * *Note*: dependency-version bumps are recorded as `changed` (see the `upgrade-backend-dependencies` command), not `housekeeping`. Reserve `housekeeping` for tooling, build, and CI maintenance. 4. Upon release of a new version, maintainers (and eventually CI) will execute `towncrier build` in each package directory which will do the following: 1. Consolidate all Newsfragment files in each `changelog` directories into a respective file each named `CHANGELOG.md`, and automatically prepend the changelog into the relevant release section 2. `git rm` all individual Newsfragment files --- # Documentation guide Welcome to the Infrahub documentation guide. This document aims to answer any questions that may come up when creating or updating documentation. ## Base prerequisites[​](#base-prerequisites "Direct link to Base prerequisites") | Prerequisite Tool(s) | Post Installation Steps | | ----------------------------------- | ----------------------- | | [Python](https://www.python.org/) | | | [Invoke](https://www.pyinvoke.org/) | | | [uv](https://docs.astral.sh/uv/) | `uv sync --all-groups` | ## Frontend prerequisites[​](#frontend-prerequisites "Direct link to Frontend prerequisites") | Prerequisite Tool(s) | Post Installation Steps | | --------------------------------------------------------------- | ----------------------- | | [Node.js (and npm)](https://nodejs.org/en) | `docs.install` | | [Vale](https://vale.sh/) | | | [Markdownlint-cli2](https://github.com/DavidAnson/markdownlint) | | Supported Node versions: `24`. ## Working with the docs site locally[​](#working-with-the-docs-site-locally "Direct link to Working with the docs site locally") The recommended way to run and build the docs locally is with Infrahub's suite of `invoke`-driven tasks. Use the Invoke tasks to build the documentation and generate source code derived documentation. ``` invoke docs.build docs.generate ``` Once the documentation has been built, make sure to validate the documentation. ``` invoke docs.validate ``` After the documentation has been validated, it can be served locally on port **3000**. ``` invoke docs.serve ``` [Explore the local documentationhttp://localhost:3000/](http://localhost:3000/) To see more documentation Invoke tasks use the help functionality. ``` invoke -l docs ``` ``` Available 'docs' tasks: .build Build documentation website. .format This will run all formatter. .format-markdownlint Run markdownlint-cli2 to format all .md/mdx files. .generate Generate all documentation output from code. .generate-bus-events Generate documentation for the Bus events. .generate-infrahub-cli Generate documentation for the infrahub cli. .generate-infrahubctl Generate documentation for the infrahubctl cli. .generate-python-sdk Generate documentation for the Python SDK. .generate-repository Generate documentation for the repository configuration file. .generate-schema Generate documentation for the schema. .install Install documentation dependencies. .lint This will run all linter. .markdownlint .serve Run documentation server in development mode. .vale Run vale to validate the documentation. .validate Validate that the generated documentation is committed to Git. ``` ## Linting before submitting[​](#linting-before-submitting "Direct link to Linting before submitting") Run the documentation linters locally before opening a pull request that touches files under `docs/docs/`. CI runs the same checks and will reject the PR otherwise. ``` invoke docs.lint ``` This task runs both `markdownlint-cli2` and `vale`. The markdownlint binary is provided by `docs/node_modules` after `cd docs && npm install`; Vale is a separate native binary and the task warns and skips it when it is not available locally. CI runs both regardless. ## Writing documentation with AI[​](#writing-documentation-with-ai "Direct link to Writing documentation with AI") We are increasingly using AI to assist developers in writing technical documentation. To ensure consistency, we maintain a set of instructions that guide the AI when generating documentation for Infrahub. You can find these instructions in the `.github/instructions/documentation.instructions.md` file. warning This process is experimental. We are evaluating available options and will refine the process as we learn more. ### Example use cases and prompts[​](#example-use-cases-and-prompts "Direct link to Example use cases and prompts") Below are some examples of how to use AI to write documentation. These are not exhaustive, but should give you a good starting point. * **Writing new documentation**: `Write a new documentation guide about [feature]. You can find the existing topic in [topic file]. The feature works like this: [description]. Make sure to include examples for [graphql, cli, etc.].` * **Updating existing documentation**: `We recently changed [this feature] to include [change]. Identify the documentation that needs to be updated and update it accordingly.` * **Assessing documentation quality**: `Review the documentation in [folder], evaluate it against our guidelines, and draft a report prioritizing which pages need updates.` ### General guidelines[​](#general-guidelines "Direct link to General guidelines") * Always use the `.github/instructions/documentation.instructions.md` file as your reference when writing documentation. * We have not yet committed to a specific LLM, so you may use the one you prefer. * Treat AI-generated documentation as a starting point. Always review and refine the content to ensure it meets our standards and accurately reflects the intended message. * Provide clear, specific prompts to the AI. The more context and detail you give, the better the output will be. ### Using an IDE[​](#using-an-ide "Direct link to Using an IDE") success Using an agent like GitHub Copilot, Cursor, or windsurf in your IDE improves the documentation writing experience. These tools can access files directly and provide better context. Example steps using GitHub Copilot in VSCode: * Open a chat with GitHub Copilot. * Add the `.github/instructions/documentation.instructions.md` file to the chat context. * Prompt the AI to perform tasks on documentation files. * Depending on the task, switch between `Ask`, `Edit`, and `Agent` modes. ### Using a web interface[​](#using-a-web-interface "Direct link to Using a web interface") If you prefer a web interface, more manual work is required. Example steps using OpenAI's ChatGPT: * Create a new project dedicated to writing documentation for Infrahub. * Copy the instructions from `.github/instructions/documentation.instructions.md` into the project. * Copy the content of reference files (`docs/docs/development/docs.mdx`, `.vale/styles/`, `.markdownlint.yaml`) into the project. * Prompt the AI to perform documentation tasks. * Copy the generated content into the appropriate `.mdx` file in the `docs/docs` folder. warning Update the instructions and reference files in your project whenever they change in the repository. ## Linting and automation[​](#linting-and-automation "Direct link to Linting and automation") Infrahub uses [Vale](https://vale.sh) to check grammar, style, and word usage. You can find Vale's configuration in `.vale.ini`, and the Infrahub styles located in `.vale/styles/Infrahub`. [Markdownlint](https://github.com/DavidAnson/markdownlint) is used to encourage consistent markdown files, and Infrahub's configuration is located at `.markdownlint.yaml`. Most Vale warnings match up with the [style guide](/development/style-guide.md) explanations. Other warnings often fall into the `Infrahub.spelling` rule. These are caused by misspellings, product names, names of people, or otherwise unknown technical terms. See the [procedures for updating rules](#spelling-errors) below for details on adding terms to the approved list. ### Install VS Code linting extensions[​](#install-vs-code-linting-extensions "Direct link to Install VS Code linting extensions") It is preferred to install extensions into VSCode so that you can see the visual errors and fix them as you write documentation. * [Vale](https://marketplace.visualstudio.com/items?itemName=chrischinchilla.vale-vscode) * [markdownlint](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint) ### Disabling Vale and markdownlint[​](#disabling-vale-and-markdownlint "Direct link to Disabling Vale and markdownlint") You can disable Vale and markdownlint in-line with the following markdown comments: ``` Ignored Specialized Phrase ignored by vale ## Ignored markdown line ``` This is useful in situations where specific style choices or markdown quirks force the use of an otherwise conflicting rule. In general, it is better to update existing configurations or create new rules rather than disable scanning of individual files. ### Creating new Vale rules[​](#creating-new-vale-rules "Direct link to Creating new Vale rules") For questions regarding how to add to or update an existing rule, see the [Vale styles documentation](https://vale.sh/docs/topics/styles/). A wealth of examples are also available in [GitLab's vale configuration](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc/.vale/gitlab). #### Spelling errors[​](#spelling-errors "Direct link to Spelling errors") If Vale warns of a spelling mistake and the word is valid, you can fix it by updating the `spelling-exceptions.txt` file in the `.vale/styles/` directory. When adding a new term, update and alphabetize the list to make future scanning easier. #### Common replacement words[​](#common-replacement-words "Direct link to Common replacement words") Add common shorthand words and phrases that have better alternatives to the `swap.yml` rule. For example, `repo` becomes `repository`. Add special case capitalization words to the `branded-terms-case-swap.yml` rule. For example, `hooli` becomes `Hooli`. ## Writing markdown[​](#writing-markdown "Direct link to Writing markdown") Pages are written in MDX, which is a enhanced version of markdown or generated by the app source. In addition, Docusaurus has its own [markdown-inspired components](https://docusaurus.io/docs/markdown-features). You'll often find reference links, panels, and snippets used throughout the Infrahub docs. ### Markdown tips[​](#markdown-tips "Direct link to Markdown tips") #### Ensure proper newlines[​](#ensure-proper-newlines "Direct link to Ensure proper newlines") Use two full returns between paragraphs (one empty line). This ensures a new paragraph is created. #### Notification blocks[​](#notification-blocks "Direct link to Notification blocks") When writing documentation, it's essential to guide the reader's attention to specific types of information. Notification blocks are a powerful tool to achieve this, allowing you to highlight information based on its nature and importance. Here are the types of notification blocks and how to use them: * **Info:** Use info blocks for additional, helpful information that isn't required to complete the task but offers more context or useful tips. ``` :::info ``` **Example** This feature is available in version 2.1 and later. * **Success:** Use success blocks to highlight expected outcomes and "status checks" to ensure the reader is on track with the guide. These blocks can reinforce the reader's progress and provide positive feedback. ``` :::success ``` **Example** If you've followed the steps correctly, your installation should now be complete. * **Warning:** Warning blocks should be used to highlight common errors or mistakes that may occur during the process. They serve as preventive measures to help the reader avoid potential pitfalls. ``` :::warning ``` **Example** Ensure you've backed up your files before proceeding with this step to prevent data loss. * **Danger:** Use danger blocks to highlight irreversible or breaking actions. These notifications are critical for steps that could significantly affect the system or data if mishandled. ``` :::danger ``` **Example** This action will permanently delete your data and cannot be undone. Incorporating these blocks into your documentation makes it more interactive and user-friendly, guiding the reader through different stages of their learning or implementation process with visual cues that emphasize the significance of each piece of information. ## Documentation sync to `infrahub-docs`[​](#documentation-sync-to-infrahub-docs "Direct link to documentation-sync-to-infrahub-docs") Infrahub documentation is synced and ultimately published via the [`infrahub-docs`](https://github.com/opsmill/infrahub-docs) repository. This is done via the GitHub action `.github/workflows/sync-docs.yml`. Please note the following important points: * Today, spelling (the `.vale` directory) is authoritative in the `infrahub` repository, not the `infrahub-docs` repository. If you need to add a spelling exception in another repository (i.e. `infrahub-demo-dc`), you have to add the exception to the `infrahub` repository. * All documentation URLs need to be relative: * Do this: `[some page](../path/to/file.mdx)` * Not this: `[some page](/absolute_path/to/file)` ## Organizing new pages[​](#organizing-new-pages "Direct link to Organizing new pages") We organize all documentation into **four** categories: tutorials, guides, topics, and reference. This is heavily influenced by the [Diátaxis framework](https://diataxis.fr/). The goal is to maintain a more organized, understandable set of docs. Here are questions to ask when deciding where to place a new document: * Are you walking the user through a scenario? Select **Tutorials**. * Are you providing steps to complete a specific task? Select **Guides**. * Are you providing background information, explanation, or abstract concepts? Select **Topics**. * Are you providing APIs, command references, or concise reference information? Select **Reference**. If you're unsure where something goes, diátaxis offers a [map](https://diataxis.fr/map/) and [compass](https://diataxis.fr/compass/) to help. When creating a new page in the documentation, in addition to creating the `.mdx` file containing the documentation itself, you must also add the page to the relevant section of the `sidebars.ts` file. ### Tutorials[​](#tutorials "Direct link to Tutorials") Tutorials are an opportunity to guide users through a repeatable process. The purpose is to **provide basic competence** in Infrahub or a feature-set. They should: * Introduce the user to the end goal. * Be repeatable by any user. * Describe practical steps, rather than abstract concepts. * Provide immediate results. The "Getting started" tutorial is a good example, as it walks the user through a scripted scenario in a demo environment. For a deeper dive into tutorials, refer to the [diátaxis tutorials page](https://diataxis.fr/tutorials/). > Tutorials are complex learning endeavors. Before deciding if a tutorial is necessary, consider how you might update an existing tutorial or if a guide would be a better option. ### Guides[​](#guides "Direct link to Guides") Guides may seem like tutorials, but they are a shorter set of universal instructions that can apply to any user's task. The purpose is to **teach how to perform a specific task**. **Naming guideline:** Describe the task that the guide describes, preferably in 2-5 words. For example: * Installing Infrahub * Creating new devices * How to invite collaborators For a deeper dive into guides, refer to the [diátaxis guides page](https://diataxis.fr/how-to-guides/). ### Topics[​](#topics "Direct link to Topics") Sometimes called *explanations*, topics offer additional context and rationale into the workings of Infrahub. They should answer the question: "how does X work?" **Naming guideline:** Write the topic name, but not a sentence. For example: * Artifact * User management and authentication Begin by giving a one to two sentence description of the topic, then dive in deeper as needed. For a deeper dive into topics, refer to the [diátaxis explanations page](https://diataxis.fr/explanation/). ### Reference[​](#reference "Direct link to Reference") Reference docs serve a single purpose. To provide quick, clear information when a user needs it. The intention is not that users *read* the reference, but instead they *consult* it as needed when working with Infrahub. **Naming guidelines:** Mirror the code-level naming guidelines where possible. This makes it easier to connect docs to code quickly. For a deeper dive into reference docs, refer to the [diátaxis reference page](https://diataxis.fr/reference/). ## Application screenshots[​](#application-screenshots "Direct link to Application screenshots") To ensure that Infrahub's screenshots remain up to date and to check that our guides work properly, we use the Python end-to-end (e2e) test suite. You'll find the e2e tests specifically designed for tutorials and guides located in `tests/e2e/tutorial`. ### Updating all screenshots manually[​](#updating-all-screenshots-manually "Direct link to Updating all screenshots manually") #### 1. Run the tutorial e2e tests with screenshots enabled[​](#1-run-the-tutorial-e2e-tests-with-screenshots-enabled "Direct link to 1. Run the tutorial e2e tests with screenshots enabled") The test suite provisions its own Infrahub environment with fresh demo data on every run (using `infrahub-testcontainers`), so no manual environment preparation is needed. From the repository root, execute: ``` UPDATE_DOCS_SCREENSHOTS=1 uv run pytest -c tests/e2e/pytest.ini tests/e2e/tutorial ``` info By default, tests are run in headless mode. To run them in a visible browser, append `--headed` to the command. #### 2. Check the results[​](#2-check-the-results "Direct link to 2. Check the results") The screenshots will be saved in `docs/docs/media`. You can then use them in our documentation: ``` ![optional caption](../../media/my-screenshot-name.png) ``` ### Add a screenshot[​](#add-a-screenshot "Direct link to Add a screenshot") #### 1. Locate on which test the screenshot should be added[​](#1-locate-on-which-test-the-screenshot-should-be-added "Direct link to 1. Locate on which test the screenshot should be added") If the test does not exist yet, you must create it first. Refer to `tests/e2e/README.md` for how the suite is structured and how to write tests. #### 2. Add the screenshot[​](#2-add-the-screenshot "Direct link to 2. Add the screenshot") To add a new screenshot in the documentation, use the `save_screenshot_for_docs` helper from `tests/e2e/helpers.py` within the test: ``` await save_screenshot_for_docs(page, "my-screenshot-name") ``` You can also organize your screenshots using folders, by specifying the folder name like this: ``` await save_screenshot_for_docs(page, "my-folder/my-screenshot-name") ``` ## Documentation release checklist[​](#documentation-release-checklist "Direct link to Documentation release checklist") Before publishing new changes to documentation, complete the following tasks: * Generate output files for automated pages with `invoke docs.generate`. * Confirm build of `infrahubctl` pages. * Confirm build of `infrahub-cli` pages. * Confirm build of schema pages. * [ ] [Update application screenshots](#updating-all-screenshots-manually). * If there is a new app version, create a new release notes document in `docs/release-notes`. * Run [linters](#linting-and-automation) and fix valid errors on all source files. * Perform test build of docs, `invoke docs.build`. --- info More details coming soon # Visual Studio Code ## Extensions[​](#extensions "Direct link to Extensions") * [Excalidraw](https://marketplace.visualstudio.com/items?itemName=pomdtr.excalidraw-editor) * [Jinja](https://marketplace.visualstudio.com/items?itemName=wholroyd.jinja) * [Pylance](https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance) * [Trailing Space](https://marketplace.visualstudio.com/items?itemName=shardulm94.trailing-spaces) * [Even Better Toml](https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml) * [GraphQL](https://marketplace.visualstudio.com/items?itemName=GraphQL.vscode-graphql) * [Ruff](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) ## Sample `.vscode/settings.json`[​](#sample-vscodesettingsjson "Direct link to sample-vscodesettingsjson") ``` { "[python]": { "editor.defaultFormatter": "charliermarsh.ruff", "editor.formatOnSave": true, "editor.codeActionsOnSave": { "source.organizeImports": "explicit" } }, "python.analysis.typeCheckingMode": "basic", "python.testing.pytestArgs": [ "backend" ], "python.testing.unittestEnabled": false, "python.testing.pytestEnabled": true, "ruff.enable": true, } ``` --- # Frontend guide Welcome to the Infrahub frontend guide! This guide details the technologies and steps required to contribute effectively to the Infrahub frontend. ## Infrahub stack[​](#infrahub-stack "Direct link to Infrahub stack") Infrahub frontend is developed with: * **UI Framework**: [React](https://react.dev/) * **Typing**: [TypeScript](https://www.typescriptlang.org/) * **Component Library**: [Headless UI](https://headlessui.com/) * **CSS Framework**: [Tailwind CSS](https://tailwindcss.com/) * **GraphQL Client**: [Apollo Client](https://www.apollographql.com/docs/react/) * **State Management**: [Jotai](https://jotai.org/) For testing, we rely on: * **Unit testing**: [Vitest](https://vitest.dev/) * **Integration testing**: [Vitest browser react](https://github.com/vitest-dev/vitest-browser-react) * **end-to-end testing**: [Playwright](https://playwright.dev/) ## Accessing the frontend code[​](#accessing-the-frontend-code "Direct link to Accessing the frontend code") To access Infrahub's codebase, use Git and switch to the `develop` branch to access the latest changes. All frontend code resides in `/frontend`. ``` git clone --recursive git@github.com:opsmill/infrahub.git cd infrahub/frontend ``` Before contributing, we recommended starting with [Getting set up with frontend](/development/frontend/getting-set-up.md). --- # Getting set up with frontend Before we start Make sure [Infrahub Backend](/development/backend.md) is up and running. If not, in your terminal execute: ``` invoke demo.destroy demo.start demo.load-infra-schema demo.load-infra-data ``` Infrahub is built with React and uses [pnpm](https://pnpm.io/) as its package manager. Make sure you're running Node.js 24+ and pnpm 10+, to verify, run: ``` node --version pnpm --version ``` ## 1. Install dependencies[​](#1-install-dependencies "Direct link to 1. Install dependencies") ``` cd frontend/app pnpm install ``` ## 2. Start a local server[​](#2-start-a-local-server "Direct link to 2. Start a local server") ``` pnpm start ``` You can access your local server at . If you are not familiar with Infrahub, follow our [Overview Guide](/overview.md). ## 3. Run all tests[​](#3-run-all-tests "Direct link to 3. Run all tests") ### Unit & integration tests[​](#unit--integration-tests "Direct link to Unit & integration tests") ``` pnpm test # same with coverage pnpm test:coverage ``` ### E2E tests[​](#e2e-tests "Direct link to E2E tests") E2E tests live in `tests/e2e/` and run from the repository root (not from `frontend/app`), against a locally built Infrahub image (`uv run invoke dev.build`): ``` INFRAHUB_TESTING_IMAGE_VER=local INFRAHUB_TESTING_DOCKER_PULL=false \ uv run pytest -c tests/e2e/pytest.ini tests/e2e ``` All tests should succeed. For more information on testing, read [Running & Writing Tests](/development/frontend/testing-guidelines.md). --- # Running & writing tests for frontend Before we start If you have never run Infrahub tests before, we highly suggest following the [frontend guide](/development/frontend/getting-set-up.md). We're expecting to see proper tests for each feature/bugfix you make. If you're not sure how to write these tests, this page is made to help you get started. Infrahub frontend has 3 types of testing: * [end-to-end (e2e) tests](#e2e-tests) * [unit & integration tests (documentation WIP)](#unit--integration-tests) ## E2E tests[​](#e2e-tests "Direct link to E2E tests") End-to-end tests live in `tests/e2e/` at the repository root. The suite is written in Python with [pytest-playwright](https://playwright.dev/python/) and boots a complete Infrahub stack with `infrahub-testcontainers`. You need a running Docker daemon and a locally built Infrahub image (`uv run invoke dev.build`). Run from the repository root (not from `frontend/app`): ``` INFRAHUB_TESTING_IMAGE_VER=local INFRAHUB_TESTING_DOCKER_PULL=false \ uv run pytest -c tests/e2e/pytest.ini tests/e2e ``` To run against an already-running Infrahub instead of booting a stack, set `INFRAHUB_ADDRESS`. See `tests/e2e/README.md` for the suite architecture, the data fixtures, and the debugging workflow. ## Unit & integration tests[​](#unit--integration-tests "Direct link to Unit & integration tests") ``` pnpm test # same with coverage pnpm test:coverage ``` --- # How to work with Git when developing Infrahub This guide explains Git workflows and best practices when contributing to Infrahub, covering submodule management, branching strategies, and pull request workflows. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before you begin, ensure you have: * Git 2.20+ installed with submodule support * SSH keys configured for GitHub access * Basic understanding of Git branching and merging concepts * Development environment set up per the [backend guide](/development/backend.md) ## Working with submodules[​](#working-with-submodules "Direct link to Working with submodules") Infrahub includes the Python SDK as a Git submodule. This section covers essential submodule operations. ### Clone the repository with submodules[​](#clone-the-repository-with-submodules "Direct link to Clone the repository with submodules") When cloning Infrahub for the first time, always include the `--recursive` flag to initialize submodules: * SSH (recommended) * HTTPS ``` git clone --recursive git@github.com:opsmill/infrahub.git cd infrahub ``` ``` git clone --recursive https://github.com/opsmill/infrahub.git cd infrahub ``` This automatically initializes and updates the `python_sdk` submodule to the correct commit. ### Pull latest changes on an existing branch[​](#pull-latest-changes-on-an-existing-branch "Direct link to Pull latest changes on an existing branch") When updating your local repository, ensure submodules stay synchronized: ```` # Update the main repository and all submodules git pull --recurse-submodules # Alternative: pull main repo first, then sync submodules to recorded commits git pull git submodule update --init --recursive **Important**: Always use `--recurse-submodules` when pulling to avoid submodule synchronization issues. ### Update the submodule to a specific commit When you need to update the submodule to track a different version of the Python SDK: ```bash # Navigate to the submodule directory cd python_sdk # Fetch the latest tags and commits git fetch --tags origin # Check out the desired version (tag or commit) git checkout v1.10.0 # Return to the main repository root cd .. # Stage the submodule update git add python_sdk # Commit the submodule pointer update git commit -m "update python_sdk to v1.10.0" ```` ### Manage merge conflicts in submodules[​](#manage-merge-conflicts-in-submodules "Direct link to Manage merge conflicts in submodules") Submodule conflicts appear as conflicting commit pointers. Here's how to resolve them: 1. **Identify the conflict**: ``` git status # Shows: both modified: python_sdk ``` 2. **Check the conflicting commits**: ``` git diff python_sdk ``` 3. **Choose the resolution strategy**: **Option A: Use the version from your branch** ``` cd python_sdk git checkout cd .. git add python_sdk ``` **Option B: Use the version from the target branch** ``` cd python_sdk git checkout cd .. git add python_sdk ``` **Option C: Update to the latest version** ``` cd python_sdk git fetch origin git checkout origin/main # or desired branch cd .. git add python_sdk ``` 4. **Complete the merge**: ``` git commit -m "resolve submodule conflict in python_sdk" ``` ## Infrahub branching and release model[​](#infrahub-branching-and-release-model "Direct link to Infrahub branching and release model") Infrahub follows a structured branching model designed for stable releases and continuous development. ### Branch structure[​](#branch-structure "Direct link to Branch structure") **Main branches**: * `stable`: Production-ready code, protected branch, only updated via PRs * `develop`: Integration branch for new features, default branch for development **Release branches**: * `release-x.y`: Cut from `develop` during release prep, merged into `stable` and then back into `develop`; an automation bot keeps them in sync by continuously PR-ing `stable → release-x.y` and `release-x.y → develop` (or `stable → develop` when no release branch exists) **Feature branches**: * `feature/description`: New features or enhancements * `fix/description`: Bug fixes * `docs/description`: Documentation updates ### Infrahub release model[​](#infrahub-release-model "Direct link to Infrahub release model") Infrahub follows semantic versioning (MAJOR.MINOR.PATCH) with these release types: **Major releases** (for example, 1.0.0 → 2.0.0): * Breaking API changes * Significant architectural updates * Migration guides provided **Minor releases** (for example, 1.3.0 → 1.4.0): * New features and improvements * Backward-compatible changes * Regular monthly cadence **Patch releases** (for example, 1.3.6 → 1.3.7): * Bug fixes and security updates * No new features * Released as needed ### Release workflow[​](#release-workflow "Direct link to Release workflow") 1. **Development happens on `develop`** 2. **Release preparation**: * Create release branch from `develop` * Update the changelog (the version is derived from the release tag, not edited in `pyproject.toml`) * Final testing and bug fixes 3. **Release**: * Merge release branch to `stable` * Create the annotated `infrahub-v` Git tag — this tag is the sole source of the release version * Merge `stable` back to `develop` ### Versioning[​](#versioning "Direct link to Versioning") `infrahub-server` and `infrahub-testcontainers` derive their version from the `infrahub-v*` git tags at build time (via hatch-vcs). There is no `version` field in either `pyproject.toml`, so releases never require a version-bump commit. Keep these in mind when developing locally: * **Fetch tags on a fresh clone**: run `git fetch --tags` (or clone with full history). Without a reachable `infrahub-v*` tag a build still succeeds, but it reports a development fallback version rather than the real one. * **Editable installs cache the version**: the version reported by `infrahub` is fixed when you run `uv sync`. After checking out a different commit or fetching new tags, re-run `uv sync` to refresh it. * **Maintenance-branch hygiene**: never merge a newer main-line `infrahub-v*` tag into an older `release-x.y` branch — the resolver would then resolve the wrong version line. Cherry-pick fixes onto the release branch instead of merging `stable`/`develop` into it. ## Best practices for pull requests[​](#best-practices-for-pull-requests "Direct link to Best practices for pull requests") Follow these practices to ensure smooth code review and integration. ### Before creating a pull request[​](#before-creating-a-pull-request "Direct link to Before creating a pull request") 1. **Start from the correct base branch**: ``` # For new features git checkout develop git pull origin develop git checkout -b feature/your-feature-name # For urgent fixes git checkout stable git pull origin stable git checkout -b fix/urgent-bug-fix ``` 2. **Keep commits focused and atomic**: * One logical change per commit * Write clear, descriptive commit messages * Use conventional commit format: `type(scope): description` 3. **Update submodules if needed**: ``` # Ensure submodules are current git submodule update --recursive --remote # If submodule updates are needed, commit them git add python_sdk git commit -m "chore(deps): update python_sdk submodule" ``` 4. **Test your changes**: ``` # Install dependencies uv sync --all-groups # Run the full test suite uv run invoke test # Run linting and formatting uv run invoke lint uv run invoke format # Test with submodules uv run invoke backend.test ``` ### Creating the pull request[​](#creating-the-pull-request "Direct link to Creating the pull request") 1. **Push to your feature branch**: ``` git push -u origin feature/your-feature-name ``` 2. **Create PR with detailed description**: * Clear title summarizing the change * Detailed description explaining the what and why * Reference related issues using `#issue-number` * Include testing instructions * Add screenshots for UI changes 3. **Use proper PR labels**: * `type/feature`: New functionality * `type/bug`: Bug fixes * `type/docs`: Documentation changes * `breaking`: Breaking changes * `needs-review`: Ready for review ### Pull request review process[​](#pull-request-review-process "Direct link to Pull request review process") 1. **Automated checks must pass**: * All CI/CD pipelines * Code quality checks * Test coverage requirements * Submodule consistency checks 2. **Code review**: * At least one approving review required * Address all feedback and comments * Update PR description if scope changes 3. **Final merge**: * Rebase on target branch if requested * Squash commits if needed * Use merge commit for feature branches ## Documentation branching workflow[​](#documentation-branching-workflow "Direct link to Documentation branching workflow") Use the branch that matches when the change must appear on [docs.infrahub.app](https://docs.infrahub.app): ### Minor fixes to published docs[​](#minor-fixes-to-published-docs "Direct link to Minor fixes to published docs") * Branch directly from `stable` (for example, `docs/stable-doc-workflow`) so the fix deploys as soon as the PR merges * If you already authored the fix on `develop`, rebase or cherry-pick it onto a `stable`-based branch before opening the PR * Target `stable` in the pull request ### Docs for work on a release branch[​](#docs-for-work-on-a-release-branch "Direct link to Docs for work on a release branch") * When a feature (or its documentation) lives on an active `release-x.y` branch, create your docs branch from that same release branch (for example, `release-1.5`) * Target the release branch in your PR so the docs ship with the release and flow back into `develop` via the automated sync * Rebase on the release branch regularly to pick up stabilization commits ### Future or feature documentation[​](#future-or-feature-documentation "Direct link to Future or feature documentation") * Branch from `develop` for docs tied to unreleased features * This keeps upcoming content out of `stable` until the feature is available ## Bringing latest changes from stable into develop[​](#bringing-latest-changes-from-stable-into-develop "Direct link to Bringing latest changes from stable into develop") Periodically sync `develop` with `stable` to incorporate fixes and releases. ### Regular sync process[​](#regular-sync-process "Direct link to Regular sync process") 1. **Switch to develop and update**: ``` git checkout develop git pull origin develop ``` 2. **Merge stable into develop**: ``` git merge origin/stable ``` 3. **Handle any conflicts**: * Resolve merge conflicts manually * Pay special attention to submodule conflicts * Test the merge thoroughly 4. **Push the updated develop**: ``` git push origin develop ``` ### After a release[​](#after-a-release "Direct link to After a release") When a new version is released to `stable`: 1. **Update your local branches**: ``` git fetch origin --tags git checkout stable git pull origin stable git checkout develop git pull origin develop ``` 2. **Merge stable to develop**: ``` git merge stable ``` 3. **Update submodules if needed**: ``` git submodule update --recursive --remote ``` 4. **Rebase your feature branches**: ``` git checkout feature/your-feature git rebase develop ``` ## Troubleshooting common issues[​](#troubleshooting-common-issues "Direct link to Troubleshooting common issues") ### Submodule shows as modified after pull[​](#submodule-shows-as-modified-after-pull "Direct link to Submodule shows as modified after pull") **Problem**: `git status` shows the submodule as modified even after pulling. **Solution**: ``` # Reset submodule to the commit specified in the main repo git submodule update --recursive # Alternative: force submodule to match main repo cd python_sdk git reset --hard HEAD cd .. ``` ### Submodule update fails with authentication error[​](#submodule-update-fails-with-authentication-error "Direct link to Submodule update fails with authentication error") **Problem**: Cannot fetch submodule due to SSH/authentication issues. **Solution**: ``` # Configure Git to use SSH for GitHub git config url."git@github.com:".insteadOf "https://github.com/" # Or update submodule URLs to SSH git submodule set-url python_sdk git@github.com:opsmill/infrahub-sdk-python.git ``` ### Merge conflicts with submodule pointers[​](#merge-conflicts-with-submodule-pointers "Direct link to Merge conflicts with submodule pointers") **Problem**: Git shows conflicts in submodule files during merge. **Solution**: 1. Don't edit the submodule files directly in the main repository 2. Navigate to the submodule directory to resolve conflicts 3. Check out the appropriate commit in the submodule 4. Return to main repository and stage the submodule update ### Branch diverged after submodule update[​](#branch-diverged-after-submodule-update "Direct link to Branch diverged after submodule update") **Problem**: Local branch has diverged after updating submodules. **Solution**: ``` # If you need to keep your changes git stash git pull --rebase origin develop git stash pop # If the submodule update should be ignored git checkout HEAD -- python_sdk git submodule update --recursive ``` ## Advanced workflows[​](#advanced-workflows "Direct link to Advanced workflows") ### Working with multiple feature branches[​](#working-with-multiple-feature-branches "Direct link to Working with multiple feature branches") When working on multiple features simultaneously: ``` # Create and switch between feature branches git checkout -b feature/feature-a develop git checkout -b feature/feature-b develop # Keep branches updated with develop git checkout feature/feature-a git rebase develop # Handle submodule updates per branch git submodule update --recursive ``` ### Custom submodule workflows[​](#custom-submodule-workflows "Direct link to Custom submodule workflows") For advanced submodule management: ``` # Track a specific submodule branch git submodule set-branch --branch main python_sdk git submodule update --recursive --remote # Temporarily work on submodule locally cd python_sdk git checkout -b temp-changes # Make changes... git add . && git commit -m "temporary changes" cd .. git add python_sdk git commit -m "temp: use local submodule changes" ``` ## Further reading[​](#further-reading "Direct link to Further reading") * [Understanding Git repositories in Infrahub](/git-integration/overview.md) - Repository integration concepts * [Backend development guide](/development/backend.md) - Setting up the development environment * [How to connect external Git repositories](/git-integration/connect-repository.md) - External repository setup * [Proposed changes](/proposed-changes/overview.md) - Understanding Infrahub's change management --- # Documentation style guide This guide defines writing style, capitalization, and grammar rules for Infrahub documentation. As a general rule, prefer consistency and simplicity when possible. For anything not answered below, reference the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/welcome/). General tips: * Avoid words like *easy*, *just*, or *simple* to describe how to do something or how "easy" a task is. - If a sentence looks too long, it probably is. Try and simplify it or break it into multiple sentences. - Avoid jargon unless you are sure the reader knows the term. - Don't hesitate to link between pages and concepts. - Avoid repeating information when possible, and instead link out to topic or reference pages. ## Infrahub terminology[​](#infrahub-terminology "Direct link to Infrahub terminology") > **Capitalize nouns that represent first-class, named Infrahub capabilities with defined behavior and APIs.** **Do not capitalize generic industry concepts, even when Infrahub produces or uses them.** ### Generators[​](#generators "Direct link to Generators") **Always capitalize** when referring to the Infrahub feature. Generator is not a widely established industry term. In Infrahub, a Generator is a named system concept with specific semantics (idempotent, service-model-driven, graph-aware). | Usage | Correct | Incorrect | | ----------------- | -------------------------------------------------------------- | ---------------------------------- | | Feature reference | "Infrahub **Generators** convert service models into objects." | `"Infrahub generators convert..."` | | Plural | "Configure your **Generators** in the repository." | `"Configure your generators..."` | ### Transformations[​](#transformations "Direct link to Transformations") **Always capitalize** when referring to the Infrahub feature. Use "transform" only as a verb, never as a noun. While "transformation" is a common word, Infrahub Transformations have a specific execution model, defined inputs (GraphQL queries), and defined outputs (artifacts). | Usage | Correct | Incorrect | | ----------------- | ----------------------------------------------------------- | ------------------------------ | | Feature reference | "**Transformations** convert graph data into artifacts." | `"transformations convert..."` | | As a verb | "Use this to **transform** data into vendor formats." | `"Use this Transform to..."` | | Noun form | "Create a **Transformation** for configuration generation." | `"Create a transform for..."` | **Never use "transform" or "transforms" as a noun.** Always use "Transformation" or "Transformations". ### Artifacts[​](#artifacts "Direct link to Artifacts") **Do NOT capitalize** unless it starts a sentence or is at the beginning of a bullet point where other items are also capitalized. Artifact is a broadly accepted industry term. Infrahub artifacts are not conceptually novel in the same way Generators or Transformations are. | Usage | Correct | Incorrect | | --------------------- | ----------------------------------------------- | ---------------------------------------------------------------------------- | | Mid-sentence | "The **artifact** is stored in object storage." | `"The Artifact is stored..."` | | Start of sentence | "**Artifacts** are generated automatically." | `"artifacts are generated..."` | | In a capitalized list | "- **Artifacts**: Generated outputs..." | `"- **artifacts**: Generated outputs..."` (if other items start capitalized) | ### Profiles[​](#profiles "Direct link to Profiles") **Always capitalize** when referring to the Infrahub feature. | Usage | Correct | Incorrect | | ----------------- | --------------------------------------- | ------------------------------ | | Feature reference | "Infrahub **Profiles** allow you to..." | `"Infrahub profiles allow..."` | | Plural | "Create **Profiles** for your devices." | `"Create profiles for..."` | ### Resource Manager[​](#resource-manager "Direct link to Resource Manager") **Always capitalize** and **always use singular form**. | Usage | Correct | Incorrect | | ----------------- | ------------------------------------------- | ------------------------------ | | Feature reference | "Use **Resource Manager** to allocate IPs." | "Use Resource Managers..." | | Plural context | "Configure **Resource Manager** instances." | "Configure Resource Managers." | ### Quick reference table[​](#quick-reference-table "Direct link to Quick reference table") | Term | Capitalize? | Notes | | ----------------- | ----------- | --------------------------------------------------------- | | Generator(s) | Yes | Infrahub-specific primitive | | Transformation(s) | Yes | First-class capability (noun form) | | transform | No | Verb only | | artifact(s) | No | Generic industry term (capitalize at sentence/list start) | | Profile(s) | Yes | Infrahub-specific feature | | Resource Manager | Yes | Always singular, system-level capability | ## Language[​](#language "Direct link to Language") We use American English for most standard text. Unique technical terms are [included below](#product-and-technology-names), or in the [Microsoft A-Z word list](https://learn.microsoft.com/en-us/style-guide/welcome/). ## Trailing commas[​](#trailing-commas "Direct link to Trailing commas") Use a trailing comma when listing multiple items. This is commonly known as the Oxford comma or serial comma. **❌ Don't do this:** There are devices, organizations, and users. **✅ Do this:** There are devices, organizations, and users. ## Headings and titles[​](#headings-and-titles "Direct link to Headings and titles") Headings and titles should capitalize the first word only and end with no punctuation. The exception being any proper noun. **❌ Don't do this**: Getting Started! **✅ Do this**: Getting started Every page should have a top-level heading. Additional heading tiers can only exist if a higher tier has been used. **❌ Don't do this**: ``` # Page title ### Smaller heading ``` ## Avoid over-capitalization[​](#avoid-over-capitalization "Direct link to Avoid over-capitalization") It is tempting to want to capitalize all feature names. Unless the term is a named marketing feature, avoid capitalization. **❌ Don't do this**: Git Repository, API Server, User Management **✅ Do this**: Git repository, API server, user management ## Lists[​](#lists "Direct link to Lists") Capitalize the first letter of each list item. If an item is a complete sentence, give it a period at the end. If it's not, it is okay to omit punctuation. The [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/scannable-content/lists) has a good explanation of how to handle list punctuation. When listing items and descriptions, prefer the use of a colon (:) instead of a dash (-). ``` - Not - this - Or - this - Do: this - And: this ``` ### Bullet point lists[​](#bullet-point-lists "Direct link to Bullet point lists") When a word appears at the start of a bullet point in a list where other items begin with capitalized words, capitalize it for consistency. **Correct:** ``` - **Caching**: Generated artifacts are stored... - **Traceability**: Past values remain available... - **Peer Review**: Artifacts are automatically part of... - **Database**: Artifact nodes are stored... ``` **Incorrect:** ``` - **Caching**: Generated artifacts are stored... - **Traceability**: Past values remain available... - **Peer Review**: artifacts are automatically part of... - **Database**: artifact nodes are stored... ``` ### Link text in lists[​](#link-text-in-lists "Direct link to Link text in lists") When a word appears as link text at the start of a bullet point in a list where other items begin with capitalized words, capitalize it. **Correct:** ``` - [Version control](version-control): Understand how branches work... - [Schema](schema): Learn about data models... - [Artifacts](artifact): Explore generated outputs... - [Generators](generator): Learn about code generation... ``` **Incorrect:** ``` - [Version control](version-control): Understand how branches work... - [Schema](schema): Learn about data models... - [artifacts](artifact): Explore generated outputs... - [generators](generator): Learn about code generation... ``` ## Colons[​](#colons "Direct link to Colons") Avoid extra spaces before a colon. ``` Feature : Explanation of feature Feature: Explanation of feature ``` ## Code blocks[​](#code-blocks "Direct link to Code blocks") When creating a code block or snippet with three backticks, make sure to include a language designation. ```` ```shell this is a shell script ``` ```` ## Marking code items[​](#marking-code-items "Direct link to Marking code items") Sometimes you need to mention a `function` or `ModelName`. To do so, use the inline code backticks in markdown. ## Excalidraw diagrams[​](#excalidraw-diagrams "Direct link to Excalidraw diagrams") You can attach `.excalidraw` diagrams to visually explain concepts, workflows, or processes within the documentation. warning Always save Excalidraw files with the "Embed Scene" option checked to maintain the ability to edit in VS Code. ## Use i.e. for examples[​](#use-ie-for-examples "Direct link to Use i.e. for examples") Prefer `i.e.` over `e.g.` or `ex.`. In a sentence, `i.e.` is surrounded by commas. For example: *Select the current branch, i.e., 'main'.* It's also acceptable and clearer to use "for example" or "such as". ## Product and technology names[​](#product-and-technology-names "Direct link to Product and technology names") Always use correct capitalization for these terms: * Ansible * Docker, DockerHub * Git * GitHub, GitLab, GitPod * Grafana * GraphQL * InfluxDB * Infrahub * Jinja2 * K3s, K8s, Kubernetes * Linux * MacOS * MySQL * Neo4j * NGINX * Node.js * OpenAPI, OpenConfig * OpsMill * PostgreSQL * Prometheus * Python * RabbitMQ * Terraform * Ubuntu * VS Code Additional terms can be found in `.vale/styles/Infrahub/branded-terms-case-swap.yml`. --- # Event actions An event action is a node you create in Infrahub that defines an outcome: add a device to a group, run a Generator definition. Actions on their own do nothing — they fire only when an [event rule](/events/event-rules.md) matches and points at them. Actions inherit from the `CoreAction` generic. Two action kinds exist because the most common automated outcomes fall into two categories: modifying group membership, or running a Generator against the impacted object. | Action kind | Effect | Example | | --------------------- | ------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | `CoreGroupAction` | Adds or removes members from a group | Add a device to `arista_devices` when a node trigger rule fires | | `CoreGeneratorAction` | Runs a Generator definition against the impacted object | Run `create_circuit_endpoints` when a group trigger rule fires on `provisioning_circuits` | [Webhooks](/webhooks/overview.md) are a related concept — they also fire in response to events — but they live outside the rule/action loop because their setup is more involved. Use a Generator action when the outcome stays inside Infrahub; use a webhook when you need to send a payload to an external HTTP endpoint. ## Configure a group action[​](#configure-a-group-action "Direct link to Configure a group action") A group action adds or removes nodes from a group when triggered. 1. Navigate to the **Actions** page. 2. Click **Create** and select **Group Action**. 3. Configure the action: * Enter a name (example: `add-to-group-arista_devices`) * Select the associated kind of group (example: `Standard Group Core`) * Choose the target group (example: `arista_devices`) * Click **Save**. ![Group Action creation form](/assets/images/grp_actions-form-creation-d757bb6ea6a491f959820830cf590107.png) After saving, the details page shows the action's configuration: ![Group Action details](/assets/images/grp_actions-details-f92e37708f8e697f9ae9009dbba4a014.png) The action appears in the actions list and can be referenced by any trigger rule. ## Configure a Generator action[​](#configure-a-generator-action "Direct link to Configure a Generator action") A Generator action runs a Generator definition when triggered. The Generator receives the impacted object as its target. 1. Navigate to the **Actions** page. 2. Click **Create** and select **Generator Action**. 3. Configure the action: * Enter a name (example: `create_circuit_endpoints`) * Select the Generator (example: `create_circuit_endpoints`) * Click **Save**. ![Generator Action creation form](/assets/images/generator-action-form-creation-34c7932d1b6ceea3bee8de79fb6cac67.png) The action appears in the actions list and can be referenced by any trigger rule. ## Related[​](#related "Direct link to Related") * [Event rules](/events/event-rules.md) — the conditions that fire actions * [Event system](/events/event-system.md) — what events are and where they come from * [Webhooks](/webhooks/overview.md) — a related event-driven outcome that lives outside the rule/action loop --- # Event rules An event rule is a node you create in Infrahub that names a set of conditions and ties one or more actions to fire when those conditions match. Rules turn raw events into useful automation: instead of reacting to every `infrahub.node.updated` event, you describe the precise update you care about — "an `InfraDevice` whose status changed to `active`" — and Infrahub runs your action only when that pattern shows up. Rules inherit from the `CoreTriggerRule` generic. Two rule kinds exist because matchable events come from two sources: mutations on specific nodes, and changes to group membership. | Rule kind | Use it to match | Example | | ---------------------- | --------------------------- | --------------------------------------------------------------------------------- | | `CoreNodeTriggerRule` | mutations on specific nodes | An attribute change on `InfraDevice`, a relationship update on `InfraInterface` | | `CoreGroupTriggerRule` | changes to group membership | A member added to `provisioning_circuits`, a member removed from `arista_devices` | ## Match filters[​](#match-filters "Direct link to Match filters") By default, a rule fires for any event of the configured type. To narrow it down, add match filters. All conditions must be satisfied — match filters use logical AND, not OR. **Attribute matches** filter on the new value, the previous value, or both: * New value — match the attribute's value after the change * Previous value — match the attribute's value before the change * Both — match a transition (for example, previous = `inactive` AND new = `active`) **Relationship matches** filter on the modification type and the peer: * Modification type — added, removed, or modified * Peer — the node type or specific node instance on the other side of the relationship > Example: to trigger only when a device's status transitions from `inactive` to `active`, create an attribute match with previous value = `inactive` and new value = `active`. ## Configure a node trigger rule[​](#configure-a-node-trigger-rule "Direct link to Configure a node trigger rule") A node trigger rule fires when a node is created, updated, or deleted. 1. Navigate to the **Trigger Rules** page. 2. Click **Create** and select **Node Trigger**. 3. Configure the rule: * Enter a name (example: `new-arista-devices`) * Select the node kind (example: `Device Infra`) * Choose the mutation action (example: `created`) * Select the action kind (example: `Group Action Core`) * Choose the action that the rule will fire (example: `add-to-group-arista_devices`) * Click **Save**. ![Node Trigger form creation](/assets/images/node-trigger-form-creation-9901e133baf23d4da80289b5bfae1932.png) 4. Add match conditions to refine when the rule fires: * Open the **Matches** tab. * Click **Add Match**. * Select **Node Trigger Relationship**. * Choose the relationship name (example: `Platform`). * Select the peer (example: `Arista EOS`). * Click **Save**. ![Node Trigger matches form](/assets/images/node-trigger-matches-form-creation-1cad5213281dc45f534fd7b40aa1398a.png) After saving, the rule appears in the trigger rules list as inactive. Activate it when you're ready to start matching events. Best practice Keep the rule inactive until you've configured all match conditions, then activate it. An active rule starts evaluating events immediately. ## Configure a group trigger rule[​](#configure-a-group-trigger-rule "Direct link to Configure a group trigger rule") A group trigger rule fires when a group's membership changes (member added or removed). 1. Navigate to the **Trigger Rules** page. 2. Click **Create** and select **Group Trigger**. 3. Configure the rule: * Enter a name (example: `added-to-provisioning-circuits-group`) * Select the kind (example: `Standard Group Core`) * Choose the standard group (example: `provisioning_circuits`) * Select the action kind (example: `Generator Action Core`). A group trigger rule can only fire a Generator action * Choose the Generator action that the rule will fire (example: `create_circuit_endpoints`) * Click **Save**. ![Group Trigger form creation](/assets/images/group-trigger-form-creation-d575c2551fc910995a5c9dd89afce99c.png) After saving, the rule appears in the trigger rules list as inactive. Activate it when you're ready to start matching events. To change group membership from an event, use a node trigger rule with a [group action](/events/event-actions.md#configure-a-group-action). ## Where trigger rules are stored and evaluated[​](#where-trigger-rules-are-stored-and-evaluated "Direct link to Where trigger rules are stored and evaluated") Infrahub reads trigger rules from the default branch only. A rule created on another branch does not fire, and you see no error when you create it. Infrahub skips the rule while it reconciles the trigger rules, and the task worker logs a warning, `Skipping trigger rule with an unresolved relationship`, naming the rule and the relationship it could not resolve. Create rules on the default branch, together with the groups and Generator definitions they reference, and use `branch_scope` to control which branches they fire on: * `default_branch` (the default): the rule fires only for events on the default branch. * `other_branches`: the rule fires only for events on other branches. Use this when the outcome should be reviewed in a proposed change before it reaches the default branch. * `all_branches`: the rule fires everywhere. ## Known limitations[​](#known-limitations "Direct link to Known limitations") * [10463](https://github.com/opsmill/infrahub/issues/10463) attaching a group action to a group trigger rule stops Infrahub from applying any trigger rule change until that rule is deleted. Existing rules keep firing, so the failure can go unnoticed. ## Related[​](#related "Direct link to Related") * [Event actions](/events/event-actions.md) — the outcomes that rules fire * [Event system](/events/event-system.md) — what events are and where they come from --- # Event system Infrahub emits a structured event every time a significant mutation happens in the system: a node is created, updated, or deleted; a branch is created, merged, rebased, or deleted; a group gains or loses a member; and many more. Events are the foundation of every event-driven feature — automation, webhook delivery, the activity log, and external SIEM forwarding all consume them. ## Example event types[​](#example-event-types "Direct link to Example event types") A few common ones: * `infrahub.node.created` * `infrahub.node.updated` * `infrahub.node.deleted` * `infrahub.group.member_added` * `infrahub.group.member_removed` * `infrahub.branch.created` * `infrahub.branch.deleted` For the exhaustive list, see [Infrahub events](/reference/infrahub-events/overview.md). ## What events feed[​](#what-events-feed "Direct link to What events feed") Each event is processed by several consumers: * **Automation** — events drive [event rules](/events/event-rules.md) and [event actions](/events/event-actions.md) that you configure to react to specific conditions. * **[Activity log](/deploy-manage/run-observe/activity-log.md)** — every event lands in the chronological timeline of system changes. * **[Webhooks](/webhooks/overview.md)** — events can fire HTTP callbacks to external systems. * **[Log forwarding](/deploy-manage/run-observe/log-forwarding/overview.md)** — Enterprise deployments can stream events to external SIEM systems. --- # Events Infrahub emits events on every significant mutation in the system — node creates, branch operations, group membership changes, and more. The event system lets you react to those events automatically: when a device is updated, when a circuit joins a provisioning group, when a branch merges. Common uses include keeping groups in sync as data changes, triggering Generators on attribute updates, and notifying external systems through webhooks. Build automations using events, rules, and actions: * **[Events](/events/event-system.md)** — the underlying framework. Infrahub generates a structured event for every key operation (`infrahub.node.created`, `infrahub.group.member_added`, etc.). Events feed multiple consumers: the [activity log](/deploy-manage/run-observe/activity-log.md), the rule/action automation below, [webhooks](/webhooks/overview.md), and [log forwarding](/deploy-manage/run-observe/log-forwarding/overview.md). * **[Event rules](/events/event-rules.md)** — the conditions you match against events. You create a rule that names which events it cares about (a node kind, a group, a particular attribute change) and ties one or more actions to fire when the conditions match. * **[Event actions](/events/event-actions.md)** — the outcomes that fire when a rule matches. You create an action of one of two kinds today: a group action (add or remove members from a group) or a Generator action (run a Generator definition against the impacted object). The flow always has the same shape: Infrahub emits an event, evaluates it against your rules, and fires the actions of any rule that matches. ``` mutation happens ↓ Infrahub emits an event (Event system) ↓ Infrahub evaluates your rules (Event rules) ↓ Matching rules fire actions (Event actions) ``` [Webhooks](/webhooks/overview.md) consume events too, but live outside this rule/action loop because their setup is more involved. Use webhooks when you need to send a payload to an external HTTP endpoint; use event rules + actions when the outcome stays inside Infrahub. ## Example workflows[​](#example-workflows "Direct link to Example workflows") The cleanest way to think about an automation is to start from the outcome and work backward to the event. ### Add Arista devices to a group automatically[​](#add-arista-devices-to-a-group-automatically "Direct link to Add Arista devices to a group automatically") **Goal**: every time a device with platform Arista EOS is created, it should land in the `arista_devices` group. * **Action** — Create a group action that adds nodes to `arista_devices`. * **Rule** — Create a node trigger rule on `InfraDevice` that fires on the `created` mutation when the platform relationship points at Arista EOS. Tie it to the action above. * **Event** — `infrahub.node.created`. Infrahub emits this automatically; no setup. When a new Arista device is created, the event fires, the rule matches the platform condition, and the action adds the device to the group. For the click-by-click setup, see [Event actions](/events/event-actions.md) and [Event rules](/events/event-rules.md). ### Auto-create circuit endpoints when a circuit enters provisioning[​](#auto-create-circuit-endpoints-when-a-circuit-enters-provisioning "Direct link to Auto-create circuit endpoints when a circuit enters provisioning") **Goal**: when a circuit joins the `provisioning_circuits` group, run a [Generator](/generators/overview.md) that creates its endpoints. * **Action** — Create a Generator action that runs the `create_circuit_endpoints` Generator. * **Rule** — Create a group trigger rule on `provisioning_circuits` that fires on member-added events. Tie it to the Generator action above. * **Event** — `infrahub.group.member_added`. Infrahub emits this automatically. When a circuit is added to the group, the event fires, the rule matches, and the Generator runs against the new member. For the click-by-click setup, see [Event rules](/events/event-rules.md) and [Event actions](/events/event-actions.md). --- # Frequently asked questions Here are the key details about Infrahub: ## About Infrahub[​](#about-infrahub "Direct link to About Infrahub") ### What is Infrahub?[​](#what-is-infrahub "Direct link to What is Infrahub?") Infrahub is an open-source platform that enables infrastructure as code by providing a unified system to model, store, version, validate, and generate infrastructure data and configurations. It acts as a centralized source of truth for all infrastructure resources across an organization. ### What are the main features and capabilities?[​](#what-are-the-main-features-and-capabilities "Direct link to What are the main features and capabilities?") * User-Defined Schema: Build and update data models to support business requirements for infrastructure resources. * Unified Storage: Version control for data and files * Versioning: Track current state and future intended state of the infrastructure with built-in peer review and a CI pipeline for validation. * Immutable Database: Go back in time to view the state of infrastructure at any point. * Artifact Generation: Transform infrastructure data into any format (text configurations, XML, JSON, etc.) using Generators and templates. * Data Lineage: View the source and owner for each piece of infrastructure data. * Multiple interfaces for each team and need: * Web UI: View and manage all infrastructure data, configurations, and CI/CD pipelines through a user interface. * Query Engine: Powerful API and support for GraphQL and REST to query and retrieve infrastructure data. * Git Integration: File storage for code and templates * Integrations: Integrations with tools like NetBox, Nautobot, Ansible, and Nornir. Ability to build custom integrations. * Python SDK: Programmatically interact with Infrahub using a Python SDK. For more information, see our [Infrahub overview](/overview.md). ### I'm just getting started with infrastructure automation. Do I need Infrahub?[​](#im-just-getting-started-with-infrastructure-automation-do-i-need-infrahub "Direct link to I'm just getting started with infrastructure automation. Do I need Infrahub?") If you're new to infrastructure automation, Infrahub is a great place to start. It lets you store the intended state of your infrastructure, this is a crucial foundation for any automation you build later. From this intended state, you can generate configurations directly within Infrahub, and also use the data for resource planning, service deployment, invoicing, and more. Check out the [blog section on our website](https://opsmill.com/blog/) to learn more about our vision and dive deeper into infrastructure automation. ### How can I see Infrahub in action?[​](#how-can-i-see-infrahub-in-action "Direct link to How can I see Infrahub in action?") You have many options to quickly try out Infrahub, please refer to the [Explore Infrahub](/overview/explore.md) page for more details. Here are some quick ways to get started: * [Infrahub Sandbox](https://sandbox.infrahub.app/) - Instantly login to the UI of a demo environment of Infrahub with sample data pre-loaded. * [Getting Started Lab](https://opsmill.instruqt.com/pages/labs) - Online lab to see all Infrahub fundamentals in action. ### What are the deployment options for Infrahub?[​](#what-are-the-deployment-options-for-infrahub "Direct link to What are the deployment options for Infrahub?") This question is about *how* Infrahub is packaged and run. For *where* it can run (on-prem, air-gapped, cloud), see [Where can I deploy Infrahub?](#where-can-i-deploy-infrahub) below. Infrahub supports the following packaging options: * **Docker Compose** — A virtual machine running Docker, with Infrahub and its dependencies deployed via Docker Compose. Suited to evaluation, development, and smaller production environments. * **Kubernetes** — A high-availability setup using the official Helm chart to deploy Infrahub. This is the recommended topology for production deployments that require HA, because it provides the cleanest upgrade path and native handling of component failover. * **Bare metal** — Infrahub Enterprise supports direct installation on physical servers without containerization, for environments where containers are not an option. note Infrahub depends on several supporting services that must be installed either within the same environment or externally. See the [What is Infrahub](/overview.md) page for more details. Each packaging option comes with trade-offs. It's important to also consider your existing tools and processes when choosing the best fit. ### How can I install Infrahub?[​](#how-can-i-install-infrahub "Direct link to How can I install Infrahub?") Check out the [Installing Infrahub](/deploy-manage/install-configure/install/overview.md) guide for detailed setup instructions. ### Where can I deploy Infrahub?[​](#where-can-i-deploy-infrahub "Direct link to Where can I deploy Infrahub?") This question is about *where* Infrahub can run. For *how* it is packaged (Docker Compose, Kubernetes, bare metal), see [What are the deployment options for Infrahub?](#what-are-the-deployment-options-for-infrahub) above. Infrahub is platform-agnostic and runs in any environment that can host its supported packaging: * **On-premises** — Deploy in your own data center on virtual machines or Kubernetes clusters. This keeps all infrastructure data inside your network perimeter and under your existing operational controls. * **Air-gapped environments** — Infrahub can be deployed in fully air-gapped networks with no inbound or outbound internet connectivity. Container images, Helm charts, and dependencies can be mirrored to an internal registry, and telemetry can be disabled (see [Does Infrahub send telemetry?](#does-infrahub-send-telemetry-and-how-can-i-disable-it)). This pattern is common in regulated, classified, or otherwise isolated networks. * **Customer-managed cloud** — Runs on any customer-managed instance in a public cloud, including AWS, Azure, and GCP. You operate it inside your own cloud account using your existing networking, IAM, and observability stack — there is no dependency on an OpsMill-managed control plane. * **Hybrid** — Multiple independent instances can be deployed across these environments, for example one on-premises and one in cloud, each acting as the source of truth for its own domain. See [Can I run multiple Infrahub instances?](#can-i-run-multiple-infrahub-instances) for details on this topology. A managed cloud SaaS version of Infrahub is in development. [Subscribe to our mailing list](https://opsmill.com) to stay updated. ### I deployed Infrahub, now what?[​](#i-deployed-infrahub-now-what "Direct link to I deployed Infrahub, now what?") Once Infrahub is deployed, you can start using it to manage your infrastructure resources and begin your automation journey. Visit the [Next Steps](/overview/next-steps.md) section for practical examples. ### What is the status of the project? Can I deploy Infrahub in production?[​](#what-is-the-status-of-the-project-can-i-deploy-infrahub-in-production "Direct link to What is the status of the project? Can I deploy Infrahub in production?") Infrahub is production-ready and has been deployed in a number of organizations as a central component of their automation workflows. If you are planning to deploy Infrahub in a critical environment we recommend reaching out to our customer success team via [Discord](https://discord.gg/opsmill), or [Online Meeting](https://cal.com/team/opsmill/meet). For teams preparing a production rollout, the [production deployment guide](/deploy-manage/install-configure/production-deployment/overview.md) collects the hardening recommendations we apply in customer environments. ### How much data can Infrahub handle right now?[​](#how-much-data-can-infrahub-handle-right-now "Direct link to How much data can Infrahub handle right now?") Capacity depends heavily on the schema you define, so we publish sizing tiers rather than one number. The [Community vs Enterprise](/overview/community-vs-enterprise.md) page documents recommended resource allocations across deployment profiles, from moderate environments up to tens of thousands of device-class objects. Production deployments today manage datasets well into the hundreds of thousands of managed objects on a single instance. If you want to evaluate Infrahub against your own schema and dataset size, reach out on [Discord](https://discord.gg/opsmill) or at , and we happy to help. ### What does a high-availability Infrahub deployment look like?[​](#what-does-a-high-availability-infrahub-deployment-look-like "Direct link to What does a high-availability Infrahub deployment look like?") Infrahub Enterprise runs every component redundantly, so an individual component failure does not take the platform down. A highly available deployment uses a Neo4j cluster with automatic failover, Redis Sentinel for cache failover, a RabbitMQ cluster for the message bus, PostgreSQL replication for the task manager database, several API server and worker replicas, and redundant load balancers in front of the API layer. Kubernetes is the recommended topology for it, and the [Community vs Enterprise](/overview/community-vs-enterprise.md) page describes the HA tiers. A highly available deployment is still one logical instance: one schema, one resource allocation authority for IPs, prefixes, and identifiers, and one target for monitoring and upgrades. See [High availability](/deploy-manage/install-configure/production-deployment/high-availability.md) for the architecture and for Terraform and Helm deployment examples. ### Can I run multiple Infrahub instances?[​](#can-i-run-multiple-infrahub-instances "Direct link to Can I run multiple Infrahub instances?") Yes. Infrahub supports deploying multiple independent instances, each as a fully autonomous deployment with its own database, schema, and resource allocation authority. Instances do not share state or synchronise data between them, making them suitable for: * Organisations with distinct governance domains or regional operational boundaries, where each domain is authoritative for its own infrastructure data. * Regional deployments requiring data residency within specific geographies or regulatory jurisdictions. * Failure isolation: an outage or maintenance in one region has no impact on others. * Canary upgrade patterns, rolling out new versions region by region before global rollout. Each instance functions as a complete, independent source of truth for its domain. Use this model when your organisation already manages separate operational boundaries. ### Can a single Infrahub instance span multiple regions?[​](#can-a-single-infrahub-instance-span-multiple-regions "Direct link to Can a single Infrahub instance span multiple regions?") No. A single Infrahub instance is designed as one clustered deployment in one region. To operate Infrahub in multiple regions, you deploy independent instances per region. See [Can I run multiple Infrahub instances?](#can-i-run-multiple-infrahub-instances) above. ### How do I back up and restore Infrahub?[​](#how-do-i-back-up-and-restore-infrahub "Direct link to How do I back up and restore Infrahub?") Automate database backups alongside your files and artifacts, and store everything in S3-compatible cloud storage. A backup is a single archive containing the graph database and the task manager database. The backup tool writes a SHA256 checksum for every file and verifies them before a restore writes anything. Files and artifacts are stored in object storage rather than in the graph database, which references them by `storage_id`. Capture both at the same point, so a restore finds the content the graph expects. Compressed backups for large production deployments typically fall in the single-digit to tens of gigabytes range. See the [backup and restore guide](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md) for the procedure, or [cluster backup and restore](/deploy-manage/maintain-upgrade/database-backup/cluster-backup-and-restore.md) for a Neo4j cluster. The [database backup topic](/deploy-manage/maintain-upgrade/database-backup/overview.md) covers how backup and restore work, and the [Infrahub Backup documentation](https://docs.infrahub.app/backup/) has the commands, the S3 settings, scheduling, and retention. ### What are realistic recovery targets for Infrahub?[​](#what-are-realistic-recovery-targets-for-infrahub "Direct link to What are realistic recovery targets for Infrahub?") With hourly backups, the recovery point objective (RPO) is one hour. The recovery time objective (RTO) for restoring a large instance is also about one hour, mainly driven by restoring the task manager database. In deployment patterns where data-plane agents enforce their last-known desired state rather than querying Infrahub directly, the effective data-plane RTO during an Infrahub outage is zero — running infrastructure keeps operating, and only the build pipeline pauses until Infrahub is available again. ### How do I upgrade Infrahub?[​](#how-do-i-upgrade-infrahub "Direct link to How do I upgrade Infrahub?") Upgrading is a routine operation, not a major project. The schema is stored in Infrahub rather than in application code, so a new release does not require redesigning it or rebuilding the integrations that read it. `infrahub upgrade` applies the database migrations and updates the core schema, the menu, the permissions, and the task manager configuration for the new version. Run `infrahub upgrade --check` first to report the pending migrations and the branches that need a rebase, without writing anything. Upgrades are supported from the previous minor version, so anything further behind moves one version at a time. A minor release can also include breaking changes that need work first, such as renaming a schema element or updating a GraphQL query. Check the release notes for each version in between. On Kubernetes, the Helm chart can run the upgrade as a pre-upgrade hook job, which you enable with `upgrade.enabled`. On Docker Compose, the sequence is stop the instance, fetch the new compose file, run the upgrade, and start again. Neither path is zero downtime, so plan a maintenance window. [Bare metal deployments](/deploy-manage/install-configure/install/enterprise.md) are supported, and we provide the upgrade procedure on request. We recommend a full database backup before every upgrade and validation in a staging environment that mirrors production topology. The [upgrade guide](/deploy-manage/maintain-upgrade/upgrade/overview.md) has the detailed steps for each edition and deployment method. ### What monitoring and observability does Infrahub provide?[​](#what-monitoring-and-observability-does-infrahub-provide "Direct link to What monitoring and observability does Infrahub provide?") Infrahub exposes a metrics endpoint by default, and the Infrahub Exporter provides Prometheus-format scraping and OTLP push to monitoring systems such as Prometheus and OpenTelemetry collectors. Workflow orchestration runs on Prefect, which provides task-level observability. In production, the most commonly monitored signals are API request latency, task manager queue depth, database connection pool usage, and disk utilisation. ### How does Infrahub handle authentication and access control?[​](#how-does-infrahub-handle-authentication-and-access-control "Direct link to How does Infrahub handle authentication and access control?") Infrahub supports OAuth2 / OIDC single sign-on (see the [SSO guide](/deploy-manage/user-management/sso/overview.md) and token-based API authentication (see [managing API tokens](/deploy-manage/user-management/managing-api-tokens.md)). The RBAC model provides pre-configured roles (Admin, Standard User, Anonymous User), custom role creation, global permissions, and namespace-scoped object-level permissions with configurable decision types. Enterprise adds workflow-based approval processes with configurable approval thresholds and enforced approval gates. All changes are tracked immutably with actor attribution. Full details are in the [permissions reference](/reference/permissions.md) and the [change approval workflow guide](/change-approval/change-approval-workflow.md). ### If I start with Community, can I move to Enterprise later?[​](#if-i-start-with-community-can-i-move-to-enterprise-later "Direct link to If I start with Community, can I move to Enterprise later?") Yes. Community and Enterprise share the same core engine, data model, and APIs, so moving from one to the other does not require a data migration or re-architecture — your schema, data, integrations, and Generators carry over. Enterprise capabilities are enabled on top of the same platform. See the [Community vs Enterprise](/overview/community-vs-enterprise.md) comparison for the full breakdown of what each edition includes. ### What drives Infrahub performance at scale, and what can I tune?[​](#what-drives-infrahub-performance-at-scale-and-what-can-i-tune "Direct link to What drives Infrahub performance at scale, and what can I tune?") Across production deployments, we have found that Generator execution is more often the build-pipeline bottleneck than GraphQL query latency. The two highest-impact tuning levers are using targeted field selection in your queries rather than eager relationship loading, and increasing pagination size where appropriate. These are the same query patterns any downstream build system should use. Core engine performance is an ongoing investment, not a fixed milestone — recent work in v1.8 moved branch-merge logic into the database layer, delivering about a 4x improvement in merge times on datasets ranging from tens to hundreds of thousands of objects. ### How can I get involved?[​](#how-can-i-get-involved "Direct link to How can I get involved?") We develop Infrahub for customers and with the community. There are a few different ways to get involved with Infrahub: * The best place to engage with the community and OpsMill team is the [Discord](https://discord.gg/opsmill) server. * As you use Infrahub, please submit bugs and feature requests directly on [GitHub](https://github.com/opsmill/infrahub/issues). * [Reach out to OpsMill](https://cal.com/team/opsmill/meet) to set up a user feedback session to share your thoughts with us. * If you are a developer, we are open to pull requests. Please refer to [Contributing Guidelines](https://github.com/opsmill/infrahub/blob/stable/CONTRIBUTING.md) to learn more. ### How do I get help with Infrahub?[​](#how-do-i-get-help-with-infrahub "Direct link to How do I get help with Infrahub?") If you need assistance with Infrahub, you can reach out to the community and the development team through the following channels: * Discord: Join the Infrahub [Discord](https://discord.gg/opsmill) server to ask questions, get support, and engage with other users. * GitHub Issues: Submit issues or questions on [GitHub](https://github.com/opsmill/infrahub/issues). ### Is Infrahub free? Is there a paid support option or an Enterprise version of Infrahub?[​](#is-infrahub-free-is-there-a-paid-support-option-or-an-enterprise-version-of-infrahub "Direct link to Is Infrahub free? Is there a paid support option or an Enterprise version of Infrahub?") Infrahub is built on an open source foundation to allow for broad community access and code transparency. However, many customers want and need higher performance and software QA validation, as well as enterprise support. That's why we offer Infrahub in two distinct versions. * **Infrahub Community** - Infrahub is an open source project that delivers a unified system to model, store, version, validate, and generate infrastructure data and configurations. * **Infrahub Enterprise** - Get SLA-backed support, advanced integrations, and enhanced performance and high-availability. Gain the benefits of mature infrastructure automation with greater velocity and quality assurance. For a detailed comparison of features, use cases, and decision guidance, see our [Community vs Enterprise](/overview/community-vs-enterprise.md) documentation. You can find pricing and feature details on [our website](https://opsmill.com/pricing/). ### Does Infrahub handle network discovery or configuration deployment?[​](#does-infrahub-handle-network-discovery-or-configuration-deployment "Direct link to Does Infrahub handle network discovery or configuration deployment?") No, and it's not part of our vision. Infrahub is focused on being the best infrastructure data platform. We believe in open source and, rather than reinventing the wheel, we provide solid integrations with existing tools like Ansible, Nornir, IP Fabric, Slurp'it, NetPicker, and more. See the **Integrations** dropdown on the documentation website. ### I already have an existing source of truth, can I use Infrahub?[​](#i-already-have-an-existing-source-of-truth-can-i-use-infrahub "Direct link to I already have an existing source of truth, can I use Infrahub?") Yes, Infrahub is built to integrate with existing sources of truth. We offer [infrahub-sync](/sync.md), a tool that lets you synchronize data from other systems into Infrahub. note This synchronization can be set up as a one-time migration or as a recurring task. This is useful in scenarios where part of the data remains in your current system and is synced to Infrahub in a read-only format to serve another purpose. ### Does Infrahub send telemetry? And how can I disable it?[​](#does-infrahub-send-telemetry-and-how-can-i-disable-it "Direct link to Does Infrahub send telemetry? And how can I disable it?") Infrahub collects anonymous usage metrics and sends them to OpsMill servers for analysis and improvement purposes. No personally identifiable information is collected. If you prefer to opt-out of telemetry collection, you can do so by setting the `INFRAHUB_TELEMETRY_OPTOUT` environment variable to `true`: ``` export INFRAHUB_TELEMETRY_OPTOUT=true ``` ### What telemetry does Infrahub send?[​](#what-telemetry-does-infrahub-send "Direct link to What telemetry does Infrahub send?") The following information is included in telemetry: * Infrahub version, deployment type, and platform (Python / OS / architecture) * Anonymous counters about the graph, schema, and features in use * Adoption counts: active accounts, account groups, and open branches * Database, worker, and infrastructure statistics * Daily activity over the previous day: logins, validation checks, artifact generation, branch operations, and webhook deliveries This information is used as aggregated analysis to better understand what and how to improve the project. All information collected is anonymous and the implementation is open source on GitHub. ## About OpsMill[​](#about-opsmill "Direct link to About OpsMill") ### Who is behind this project?[​](#who-is-behind-this-project "Direct link to Who is behind this project?") Infrahub is sponsored and developed by OpsMill, a startup focused on infrastructure management. The OpsMill team is distributed across Europe and the United States, with members who have extensive experience in managing infrastructure and building automation solutions. You can learn more about the OpsMill team and the company on [our website](https://www.opsmill.com/about-us). ### Why is OpsMill's mascot an Otter?[​](#why-is-opsmills-mascot-an-otter "Direct link to Why is OpsMill's mascot an Otter?") Otters are really cool, and they are considered a keystone species, meaning they have a disproportionately large impact on their natural environment relative to their representation. Similarly, Infrahub aims to be a keystone platform for managing and automating infrastructure, providing a central and influential role in shaping and maintaining your infrastructure ecosystem. ### What is the name of the otter?[​](#what-is-the-name-of-the-otter "Direct link to What is the name of the otter?") ![otto](/assets/images/otto-caa90c5c9ee24bb3012ae2e8ae938c80.png) The OpsMill mascot is an otter named Otto. You can send fan mail to Otto at ! ### Can Infrahub make me as cool as Tony Stark?[​](#can-infrahub-make-me-as-cool-as-tony-stark "Direct link to Can Infrahub make me as cool as Tony Stark?") Almost! You'll definitely feel like a tech genius managing your infrastructure with Infrahub! ### Made it this far? We'd love to hear from you[​](#made-it-this-far-wed-love-to-hear-from-you "Direct link to Made it this far? We'd love to hear from you") If you've made it this far, please feel free to reach out on the [Discord](https://discord.gg/opsmill) server to share your thoughts or schedule a customer interview session. The Infrahub team is always eager to receive feedback and engage with the community. --- # Build a generator A Generator queries data and creates new nodes and relationships from the result. The steps below cover how to create one. For conceptual background, see [About Generators](/generators/overview.md). For a step-by-step walkthrough with a running example, see [Build your first generator](/learn/tutorials/generators/build-your-first-generator.md) in the Academy tutorials. > Assumes a working Infrahub instance, a connected Git repository, and `infrahubctl` configured locally. See [Installation](/deploy-manage/install-configure/install/overview.md) and [Connect a repository](/git-integration/connect-repository.md) if you're starting fresh. ## Define the target schema node[​](#define-the-target-schema-node "Direct link to Define the target schema node") Create a schema node for the object kind the Generator operates on. Each instance of this kind becomes a potential Generator target. See [Understanding Schemas](/schema/overview.md) for syntax. ## Create the target group[​](#create-the-target-group "Direct link to Create the target group") Add a `CoreStandardGroup` and add target objects to it as members. The Generator runs once per group member, and members can be any kind you want the Generator to act on. See [Groups](/groups/overview.md) for managing membership. ## Write the GraphQL query[​](#write-the-graphql-query "Direct link to Write the GraphQL query") Create a `.gql` file with a query that fetches the data each Generator run needs. Variables in the query map to per-target attributes via the Generator's `parameters` field. ``` # queries/.gql query MyQuery($name: String!) { YourKind(name__value: $name) { edges { node { # ... the fields your Generator needs } } } } ``` For reusable query fragments shared across Generators or Transformations, see [GraphQL fragments](/development-resources/graphql-fragments.md). ## Implement the Generator class[​](#implement-the-generator-class "Direct link to Implement the Generator class") Create a Python file under `generators/`: generators/your\_generator.py ``` from infrahub_sdk.generator import InfrahubGenerator class YourGenerator(InfrahubGenerator): async def generate(self, data: dict) -> None: # Read per-target data from the response # Create or update objects via self.client ... ``` The class must inherit from `InfrahubGenerator` and implement an async `generate()` method. Whether you read from `data` dictionary or `self.nodes` depends on `convert_query_response` in the Generator definition — see [Query response modes](/generators/overview.md#query-response-modes). ## Register in `.infrahub.yml`[​](#register-in-infrahubyml "Direct link to register-in-infrahubyml") Add the Generator definition to your repository's `.infrahub.yml`: ``` generator_definitions: - name: file_path: "generators/.py" targets: convert_query_response: query: class_name: YourGenerator parameters: : "" queries: - name: file_path: "queries/.gql" ``` The `parameters` mapping defines how Infrahub extracts variables from each target object in the target group. Use `name__value` for a direct attribute or `relationship__name__value` for a related object's attribute. Only cardinality-one relationships are supported in parameter paths. For full `.infrahub.yml` syntax, see [infrahub.yml configuration](/git-integration/infrahub-yml.md). ## Verify the configuration[​](#verify-the-configuration "Direct link to Verify the configuration") ``` infrahubctl generator --list ``` Expected output: ``` Generators defined in repository: 1 (generators/.py::YourGenerator) Target: ``` ## Test locally[​](#test-locally "Direct link to Test locally") Run the Generator against a single target: ``` infrahubctl generator --branch= = ``` This executes the Generator locally on your machine. Verify the Generator produces the expected objects in Infrahub. ## Deploy to Infrahub[​](#deploy-to-infrahub "Direct link to Deploy to Infrahub") Commit your changes to the repository and push them up to your remote. The Generator definition registers with Infrahub when the repository syncs. ## Run the Generator[​](#run-the-generator "Direct link to Run the Generator") Generators can run in several ways: * **Manual trigger** — UI: Actions → Generator Definitions → Run * **Proposed Changes pipeline** — runs automatically when a proposed change affects target group members * **Events and Actions** — configurable rules that trigger on data changes (see [Rules & Actions](/events/overview.md)) For per-target tracking, status, and selective re-runs, see [About Generators → Generator instances](/generators/overview.md#generator-instances). ## Related[​](#related "Direct link to Related") * [About Generators](/generators/overview.md) — concept, architecture, parallelism, and instance tracking * [Modular generators](/generators/modular.md) — chaining Generators across layers * [Modular Generator best practices](/generators/modular-best-practices.md) — patterns for production * [infrahub.yml configuration](/git-integration/infrahub-yml.md) --- # Modular Generators A [Generator](/generators/overview.md) reads data from Infrahub and creates new objects based on the result. A single Generator works well when the scope is contained: one input kind, one set of outputs, no intermediate dependencies. Real-world automation is rarely that contained. A data center fabric has layers: the fabric itself, then pods, then racks, then devices. An enterprise network has sites, buildings, floors, and closets. Each layer depends on objects created by the previous one. Trying to handle all of this in a single Generator leads to a monolithic Python class that is hard to test, impossible to run partially, and painful to maintain. It can also become slow for large datasets. **Modular Generators** solve this by splitting generation across multiple focused Generators, each responsible for one layer or domain. Later Generators depend on objects created by earlier ones, connected through an event-driven signaling mechanism. info For single-Generator fundamentals, see [Generators](/generators/overview.md). For a step-by-step walkthrough, see [Build your first Generator](/learn/tutorials/generators/build-your-first-generator.md) in the Academy tutorials. ## Why split into multiple Generators[​](#why-split-into-multiple-generators "Direct link to Why split into multiple Generators") ### Single responsibility[​](#single-responsibility "Direct link to Single responsibility") Each Generator owns exactly one layer. A fabric Generator creates super spine switches and IP pools. A pod Generator creates spine switches and wires them to the super spines. A rack Generator creates leaf switches and connects them to the spines. Each Generator is small enough to understand at a glance. ### Parallelism[​](#parallelism "Direct link to Parallelism") Infrahub runs one Generator instance per target object. When you split work across layers, each layer can run its targets in parallel. A fabric with 8 pods doesn't generate them sequentially — all 8 pod Generators run concurrently. Each of those pods might have 32 racks, and all 32 rack Generators run concurrently too. This is only possible because each Generator is scoped to a single target. ### Day-two operations[​](#day-two-operations "Direct link to Day-two operations") When a change occurs at one layer — say a pod needs an additional spine switch — only the pod Generator and its downstream dependents need to re-run. The fabric Generator doesn't re-execute. This makes changes faster and reduces the blast radius of updates. ### Testability[​](#testability "Direct link to Testability") Smaller Generators with clear inputs and outputs are easier to develop and debug in isolation. You can run `infrahubctl generator` against a single target to verify one layer without needing the full modular setup to exist. ## When to use modular Generators[​](#when-to-use-modular-generators "Direct link to When to use modular Generators") **Use modular Generators when:** * The automation has a natural hierarchy or stages (physical to logical, site to rack to device, fabric to pod to rack). * Later stages depend on objects or resources created by earlier stages (for example, IP pools allocated by the fabric Generator are consumed by the pod Generator). * You want the ability to re-run a single layer independently for day-two changes, debugging, or partial rebuilds. * The number of target objects at each layer is large enough that parallel execution matters. **A single Generator is fine when:** * The scope is small and self-contained (for example, creating tags or labels based on object properties). * There are no intermediate dependencies and all objects can be created in one pass. * The number of targets is small and parallelism isn't a concern. ## The mental model[​](#the-mental-model "Direct link to The mental model") Modular Generators follow three principles. ### Each Generator owns one layer[​](#each-generator-owns-one-layer "Direct link to Each Generator owns one layer") A Generator reads from a group of target objects and creates or modifies objects that belong to that layer. It does not reach into other layers. The fabric Generator creates fabric-level resources; it does not create pod-level or rack-level objects. ### Each Generator validates its upstream dependencies[​](#each-generator-validates-its-upstream-dependencies "Direct link to Each Generator validates its upstream dependencies") Since there is no central orchestrator, each Generator must be self-protecting. Before doing its work, a Generator checks that the objects it depends on actually exist and are complete. For example, the pod Generator verifies that the expected number of super spine switches are present before creating spine switches and wiring them up. If the upstream layer isn't ready, the Generator fails with a clear error rather than producing incomplete data. ### Each Generator signals its downstream dependents[​](#each-generator-signals-its-downstream-dependents "Direct link to Each Generator signals its downstream dependents") When a Generator finishes, it needs a way to tell Infrahub that the next layer should run. This is done through a signaling mechanism, typically by updating an attribute on the downstream target objects. Infrahub's event framework detects the change and triggers the next Generator. The Generators never call each other directly. They communicate exclusively through the data they create and modify in Infrahub. info The specific mechanism for connecting Generators (the checksum and trigger pattern) is covered in a separate how-to document. ### Modular Generators in practice[​](#modular-generators-in-practice "Direct link to Modular Generators in practice") ``` Generator A (targets: fabrics) → creates fabric-level objects (super spines, IP pools) → signals downstream targets (pods) │ ├─ Generator B (targets: pods) ← runs in parallel per pod │ → validates fabric-level objects exist │ → creates pod-level objects (spines, cabling, IP allocations) │ → signals downstream targets (racks) │ │ │ ├─ Generator C (targets: racks) ← runs in parallel per rack │ │ → validates pod-level objects exist │ │ → creates rack-level objects (leafs, cabling) │ │ │ ├─ Generator C (targets: racks) │ │ → ... │ └─ ... │ ├─ Generator B (targets: pods) │ → ... └─ ... ``` Each level fans out. One fabric triggers N pods, each pod triggers M racks. The total parallelism is multiplicative. ## Trade-offs[​](#trade-offs "Direct link to Trade-offs") Modular Generators are not free. Be aware of the costs. | Benefit | Cost | | ------------------------------- | ----------------------------------------------------------------------------------------------------------- | | Each Generator is simpler | More Generators to define and manage: more `.infrahub.yml` entries, more Python files, more GraphQL queries | | Layers can run independently | Requires a trigger mechanism to connect execution across layers | | Changes are scoped to one layer | Debugging spans multiple Generator runs with no single log showing the full execution | | Parallel execution per target | You need to design clear boundaries between layers and think about what each layer owns | The right question is not "should I always use modular Generators?" but "does my automation have natural layers where the benefits outweigh the overhead?" If the answer is yes, the pattern pays for itself quickly, especially as the number of targets grows. ## Real-world example: data center fabric generation[​](#real-world-example-data-center-fabric-generation "Direct link to Real-world example: data center fabric generation") The DC-AI solution uses modular Generators to build a 5-stage Clos data center fabric: 1. **FabricGenerator**: targets fabric objects. Allocates the top-level IP supernet, creates prefix and loopback pools, and provisions super spine switches. When complete, it signals the pod objects. 2. **PodGenerator**: targets pod objects, runs in parallel per pod. Validates that all super spine switches exist, allocates pod-level IP space from the fabric's pool, creates spine switches, and cables them to the super spines. When complete, it signals the rack objects. 3. **RackGenerator**: targets rack objects, runs in parallel per rack. Validates that all spine switches exist, creates leaf switches using the pod's loopback and prefix pools, and cables them to the spines. A fabric with 4 pods and 32 racks per pod runs 4 pod Generators concurrently, then 128 rack Generators concurrently. Each Generator is independently testable with `infrahubctl generator` and can be re-run for day-two changes without affecting other layers. ## Connection to other concepts[​](#connection-to-other-concepts "Direct link to Connection to other concepts") * [Generators](/generators/overview.md): single-Generator fundamentals, high-level design, and execution methods * [Build a generator](/generators/build.md): recipe-form how-to for creating a Generator * [Build your first generator](/learn/tutorials/generators/build-your-first-generator.md): step-by-step Academy tutorial * [Build chained generators](/learn/tutorials/generators/build-chained-generators.md): checksum-based trigger pattern walkthrough * [Modular Generator best practices](/generators/modular-best-practices.md): idempotency, pool scoping, debugging, and operational guidance * [Groups](/groups/overview.md): Generators use groups to define their targets and track generated objects * [GraphQL queries](/development-resources/graphql/overview.md): each Generator definition includes a query that collects input data * [.infrahub.yml](/git-integration/infrahub-yml.md): configuration file where Generator definitions are declared --- # Best practices for modular Generators The patterns below come from real-world experience building and operating modular Generator cascades in Infrahub. They address problems that are not obvious until you have built a multi-layer cascade and run it in production. For foundational concepts, see [modular Generators](/generators/modular.md). For the chaining mechanism, see [Build chained generators](/learn/tutorials/generators/build-chained-generators.md). ## 1. One Generator, one layer[​](#1-one-generator-one-layer "Direct link to 1. One Generator, one layer") Each Generator should own exactly one layer of your hierarchy. A fabric Generator creates fabric-level objects (super spines, top-level IP pools). A pod Generator creates pod-level objects (spines, pod-level IP allocations). A rack Generator creates rack-level objects (leafs, rack-level cabling). **The temptation:** "The pod Generator already knows about the fabric, so I'll have it create the rack objects too — saves writing a third Generator." **Why to resist it:** * **Parallelism breaks down.** If the pod Generator also creates rack objects, those rack objects are created sequentially within one pod run. With a separate rack Generator, each rack gets its own run and all racks in a pod are created concurrently. * **Day-two scope increases.** If a rack needs to change, you would re-run the entire pod Generator — which also recreates all spine objects and other pod-level resources unnecessarily. * **Testing is harder.** You cannot test rack generation in isolation if it is embedded in the pod Generator. **The rule:** if you are creating objects that belong to a different level of the hierarchy, that is a signal you need a separate Generator. ## 2. Use `allow_upsert=True` for idempotent generation[​](#2-use-allow_upserttrue-for-idempotent-generation "Direct link to 2-use-allow_upserttrue-for-idempotent-generation") Generators must be safely re-runnable. In a cascade, a Generator may be triggered multiple times — by a checksum change, a user edit, or a manual `infrahubctl` run. If re-running a Generator duplicates objects instead of updating them, your data quickly becomes inconsistent. ### The pattern[​](#the-pattern "Direct link to The pattern") Use `allow_upsert=True` when saving objects. This tells the SDK to update the object if it already exists (matched by unique attributes) rather than creating a duplicate: ``` async def generate(self, data: dict) -> None: spine = await self.client.create( kind="NetworkDevice", data={ "name": f"{self.pod_name}-spine-1", "role": "spine", "status": "provisioning", "location": self.pod_id, }, ) await spine.save(allow_upsert=True) ``` ### Without `allow_upsert`[​](#without-allow_upsert "Direct link to without-allow_upsert") If you use `await spine.save()` without the flag, the first run works. The second run fails with a uniqueness constraint error or — worse — creates a duplicate if the unique constraint is not strict enough. Either outcome requires manual cleanup. ### What makes upsert matching work[​](#what-makes-upsert-matching-work "Direct link to What makes upsert matching work") Upsert matching relies on the node's [`human_friendly_id`](/schema/nodes-and-attributes.md#human-friendly-identifier-hfid) (HFID). When the server processes an upsert mutation, it checks whether a node with the same HFID already exists. If it finds a match, it updates the existing node. If no match is found, it creates a new node. This means two things must be in place: 1. **The schema must define `human_friendly_id` on the node type.** Without it, the server has no way to match an incoming upsert against an existing object. ``` nodes: - name: Device namespace: Network human_friendly_id: - hostname__value # ... ``` 2. **The Generator must provide values for all attributes and relationships that compose the HFID.** If any component is missing, the HFID cannot be computed and the match will not work. Beyond the HFID, follow these naming practices to keep matching deterministic: * The HFID attribute values must be deterministic — the same Generator inputs must produce the same values * Use naming conventions that encode the hierarchy: `{fabric}-{pod}-spine-{n}` rather than generic names ### Apply everywhere[​](#apply-everywhere "Direct link to Apply everywhere") Use `allow_upsert=True` on **every** `.save()` call in your Generators, not some of them. Partial idempotency is worse than none — it creates a false sense of re-run safety. ## 3. Make re-runs safe by design[​](#3-make-re-runs-safe-by-design "Direct link to 3. Make re-runs safe by design") Generators in a cascade will re-run. Upstream changes trigger downstream Generators. Users re-run Generators during debugging. Day-two operations re-trigger parts of the cascade. Your Generators need to handle all of this gracefully. ### The three pillars of re-run safety[​](#the-three-pillars-of-re-run-safety "Direct link to The three pillars of re-run safety") | Pillar | Mechanism | What it prevents | | ----------------------- | ------------------------------------------------------ | ---------------------------------------- | | **Idempotent saves** | `allow_upsert=True` on every `.save()` | Duplicate objects on re-run | | **Checksum guards** | `if pod.checksum.value != new_checksum` before writing | Unnecessary downstream triggers | | **Upstream validation** | Count checks at the top of `generate()` | Running against incomplete upstream data | These three mechanisms work together. Idempotency handles the object layer. Checksum guards handle the trigger layer. Upstream validation handles the ordering layer. ### Deterministic output[​](#deterministic-output "Direct link to Deterministic output") For re-runs to be truly safe, the same inputs must produce the same outputs. This means: * **Deterministic naming:** Device names should be derived from their position in the hierarchy (`{pod}-spine-{index}`), not from timestamps or random values. * **Deterministic ordering:** When iterating over objects to create children, use a stable sort (for example, by name or ID) so that index-based naming is consistent across runs. * **No external state dependency:** A Generator should produce the same output from the same Infrahub data, regardless of when it runs. Avoid depending on wall-clock time, external API state, or random values. ### What happens on re-run[​](#what-happens-on-re-run "Direct link to What happens on re-run") When a Generator re-runs with the same inputs: 1. It creates objects with the same names → `allow_upsert=True` updates instead of duplicating 2. It computes the same checksum → the checksum guard prevents writing → no downstream trigger fires 3. The cascade stops naturally — no unnecessary work propagates When inputs change (for example, a new rack is added to a pod): 1. The Generator creates the new objects and updates existing ones 2. The checksum changes → writes to downstream targets → cascade continues 3. Only the affected branch of the cascade re-runs ## 4. Debugging cascades[​](#4-debugging-cascades "Direct link to 4. Debugging cascades") A cascade has no single log showing the full chain. Each Generator runs independently, possibly on different workers, with its own log output. When a cascade stops mid-way or produces unexpected results, here is how to diagnose the problem. ### Start from the symptom[​](#start-from-the-symptom "Direct link to Start from the symptom") | Symptom | Where to look first | | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | Expected objects were not created | Check whether the Generator for that layer ran at all ([Generator instances](/generators/overview.md#generator-instances) in the UI) | | Generator ran but created nothing | Check the Generator's task log — likely an upstream validation failure | | Cascade stopped after layer N | Check whether layer N wrote checksums to layer N+1 targets | | Objects were created but are wrong | Check the query results — the query may be returning unexpected data | | Generator runs repeatedly | Check trigger rules — a trigger may be firing on an attribute the Generator itself modifies | ### Check Generator instances[​](#check-generator-instances "Direct link to Check Generator instances") The Infrahub UI shows [Generator instances](/generators/overview.md#generator-instances) per definition. Each instance corresponds to one target and shows its status: * `ready` — the Generator ran successfully for this target * `error` — the Generator failed for this target (check the task log for details) * `pending` — the Generator has not run yet for this target If a downstream Generator shows no instances at all, the trigger did not fire — check the trigger rule configuration and verify that the upstream Generator wrote the checksum. ### Follow the checksum trail[​](#follow-the-checksum-trail "Direct link to Follow the checksum trail") The checksum attribute on target objects is the cascade's breadcrumb trail: 1. **Check the upstream Generator's target objects** — does each one have a `ready` instance? 2. **Check the downstream target objects** — do they have a checksum value? If not, the upstream Generator did not write it. 3. **Compare checksums** — if the checksum has not changed since the last run, the trigger correctly did not fire. You can query checksums via GraphQL: ``` query { NetworkPod { edges { node { name { value } checksum { value } } } } } ``` ### Common cascade failure patterns[​](#common-cascade-failure-patterns "Direct link to Common cascade failure patterns") #### The "stale checksum" problem[​](#the-stale-checksum-problem "Direct link to The \"stale checksum\" problem") **Symptom:** You changed the upstream Generator's logic, but downstream Generators do not re-run. **Cause:** The upstream Generator produces the same objects (same node IDs) as before, so the checksum is identical. The new logic changed *how* objects are configured, not *which* objects exist. **Fix:** The checksum is based on node IDs, not object contents. If you need downstream re-triggers after logic changes, either: * Temporarily clear the checksum on downstream targets to force a re-trigger * Include a version string in the checksum calculation that you bump when logic changes #### The "partial cascade" problem[​](#the-partial-cascade-problem "Direct link to The \"partial cascade\" problem") **Symptom:** Some targets at layer N+1 ran, others did not. **Cause:** The upstream Generator wrote checksums to some targets but not all — likely because it failed partway through its `update_checksum()` loop, or because it only queries a subset of downstream targets. **Fix:** Ensure the upstream Generator queries **all** downstream targets, not a subset. Check that the `update_checksum()` method completes for all targets before the Generator finishes. #### The "trigger loop" problem[​](#the-trigger-loop-problem "Direct link to The \"trigger loop\" problem") **Symptom:** A Generator runs repeatedly, creating duplicate work or errors. **Cause:** The Generator modifies an attribute on its own target (or an upstream target) that has a trigger rule. This creates a feedback loop: Generator runs → modifies attribute → trigger fires → Generator runs again. **Fix:** Generators should only write checksums to **downstream** targets, never to their own targets or upstream objects. The checksum guard (`if checksum != old`) should also prevent repeated triggers for the same output. ## 5. Design your schema for Generators[​](#5-design-your-schema-for-generators "Direct link to 5. Design your schema for Generators") Generators work best when the schema supports the generation pattern. A few schema design choices make a significant difference. ### Add `GeneratorTarget` to all downstream target nodes[​](#add-generatortarget-to-all-downstream-target-nodes "Direct link to add-generatortarget-to-all-downstream-target-nodes") Any node kind that participates in a cascade as a downstream target should inherit from the `GeneratorTarget` generic (see [Build chained generators](/learn/tutorials/generators/build-chained-generators.md)). This gives it the `checksum` attribute needed for trigger-based chaining. ``` nodes: - name: Pod namespace: Network inherit_from: - GeneratorTarget ``` Do this at schema design time, not as an afterthought. Adding a generic to an existing node kind in production requires a schema migration. ### Define `human_friendly_id` on generated node types[​](#define-human_friendly_id-on-generated-node-types "Direct link to define-human_friendly_id-on-generated-node-types") Generators rely on `allow_upsert=True`, which matches objects by their [`human_friendly_id`](/schema/nodes-and-attributes.md#human-friendly-identifier-hfid). Every node type that a Generator creates must have a `human_friendly_id` defined in the schema: ``` nodes: - name: Device namespace: Network human_friendly_id: - hostname__value ``` Choose HFID components that are deterministic and scoped to the correct level of the hierarchy — for example, `hostname__value` where the hostname encodes the device's position (`{pod}-spine-{index}`). ### Store expected counts in the schema[​](#store-expected-counts-in-the-schema "Direct link to Store expected counts in the schema") Upstream validation compares actual object counts against expected values. Store these expectations as attributes on the parent object: ``` attributes: - name: amount_of_spines kind: Number description: "Expected number of spine switches in this pod" ``` This makes the Generator self-documenting and allows users to change expectations via the UI without modifying Generator code. ## 6. Operational practices[​](#6-operational-practices "Direct link to 6. Operational practices") ### Test each layer independently first[​](#test-each-layer-independently-first "Direct link to Test each layer independently first") Before running the full cascade, test each Generator in isolation with `infrahubctl generator`: ``` # Test the fabric layer infrahubctl generator generate-fabric --branch=test fabric_name=my-fabric # Manually verify fabric objects, then test the pod layer infrahubctl generator generate-pod --branch=test pod_name=pod-1 ``` This catches issues in individual Generators before the complexity of the cascade adds noise. ### Use branches for cascade testing[​](#use-branches-for-cascade-testing "Direct link to Use branches for cascade testing") Always test cascades in a branch, not on the default branch. This lets you: * Inspect generated objects without affecting production * Delete the branch and start over if something goes wrong * Review the cascade output in a proposed change before merging The `branch_scope: "other_branches"` trigger configuration (from [Build chained generators](/learn/tutorials/generators/build-chained-generators.md)) ensures triggers only fire in branches, giving you a controlled testing environment. ### Monitor Generator instances after changes[​](#monitor-generator-instances-after-changes "Direct link to Monitor Generator instances after changes") After modifying a Generator or its schema, run the cascade in a test branch and check: 1. All Generator instances show `ready` status 2. Object counts match expectations at each layer 3. Checksums were written to all downstream targets 4. No unexpected trigger loops occurred ### Version your checksum when logic changes[​](#version-your-checksum-when-logic-changes "Direct link to Version your checksum when logic changes") If you change what a Generator creates (not how it configures objects), the checksum — which is based on node IDs — may not change. Downstream Generators will not re-trigger. To force a full re-cascade after significant logic changes, consider adding a version component to the checksum: ``` GENERATOR_VERSION = "2" # Bump when logic changes require re-cascade def calculate_checksum(self) -> str: related_ids = ( self.client.group_context.related_group_ids + self.client.group_context.related_node_ids ) sorted_ids = sorted(related_ids) joined = f"v{GENERATOR_VERSION}:" + ",".join(sorted_ids) return hashlib.sha256(joined.encode("utf-8")).hexdigest() ``` ## Quick reference checklist[​](#quick-reference-checklist "Direct link to Quick reference checklist") Use this checklist when building a new modular Generator cascade: * [ ] **Each Generator owns one layer** — no cross-layer object creation * [ ] **Every** `.save()` uses `allow_upsert=True` — no exceptions * [ ] **Naming is deterministic** — same inputs produce same names * [ ] **Upstream validation at the top of** `generate()` — fail fast if dependencies are missing * [ ] **Checksums written only to downstream targets** — never to own or upstream targets * [ ] **Checksum guard before writing** — skip if unchanged to prevent unnecessary triggers * [ ] **Trigger rules use** `branch_scope: "other_branches"` — test in branches before production * [ ] **Each layer tested independently** — before running the full cascade * [ ] **Generator instances checked after runs** — verify `ready` status across all targets --- # Generators A `Generator` is a generic plugin that queries data and creates new nodes and relationships based on the result. Examples * Within your [schema](/schema/overview.md) you could create an abstract service object that through a Generator creates other nodes. * Want to read how Generators can be used to create a service catalog? See our blog post on [How to Turn Your Source of Truth into a Service Factory](https://www.opsmill.com/how-to-turn-your-source-of-truth-into-a-service-factory/). ## High level design[​](#high-level-design "Direct link to High level design") Generators are defined as a **Generator definition** within an [.infrahub.yml](/git-integration/infrahub-yml.md) file. A Generator definition consists of a number of related objects. * [Group](/groups/overview.md) of targets - Objects that the Generator will act upon * Generator class - Python code that defines the generation logic * GraphQL Query - Data collection specification ![Generator overview diagram](/assets/images/generator_overview.excalidraw-ddeaf82647838b18a4fb12296687d82b.svg) Running a Generator definition will create new nodes as defined by the Generator, or remove old ones that are no longer required. The removal of obsolete objects is handled using the [SDK tracking feature](/python-sdk/topics/tracking.md) The targets point to a [group](/groups/overview.md) that will consist of objects that are impacted by the Generator. The members of this group can be any type of object within your schema, service objects, devices, contracts or anything you want the Generator to act upon. Generator groups (`CoreGeneratorGroup`) serve as target collections that define which objects trigger Generator execution, while the actual tracking of generated objects is handled by individual Generator instances. The [GraphQL query](/development-resources/graphql/overview.md) defines the data that will be collected when running the Generator. Any object identified in this step is added as a member to a GraphQL query [group](/groups/overview.md) (`CoreGraphQLQueryGroup`). The membership in these groups are then used to determine which Generators need to be executed as part of a proposed change during the pipeline run. The Generator itself is a Python class that is based on the `InfrahubGenerator` class from the SDK. Just like [Transformations](/transformations/overview.md) and [checks](/checks/overview.md), the Generators are user defined. Generators can be executed in several ways, depending on your workflow and where you are in the lifecycle (local development vs. in Infrahub): 1. During development with infrahubctl Use the `infrahubctl generator` command to iterate locally while building and testing your Generator. 2. Manually from the UI From the Infrahub UI, open the Generator Definition detail page (Actions > Generator Definitions) and click Run to trigger the Generator on demand. 3. Automatically via Proposed Changes When you open a Proposed Change that affects the Generator's targets, the Generator runs as part of Infrahub's CI checks. Review the results in the Checks and Data tabs of the Proposed Change. This behavior can also be disabled per Generator in the repository configuration file. 4. Automatically via Events and Actions You can configure Infrahub Event rules and Actions to trigger Generators automatically based on changes in your data. This enables fully automated execution aligned with your workflows. ## Per-target execution model[​](#per-target-execution-model "Direct link to Per-target execution model") Infrahub does **not** run a Generator once for the entire target group. Instead, it creates one independent run per member of the target group. When you trigger a Generator definition, Infrahub: 1. Fetches the target group and enumerates its members. 2. For each member, extracts scoped variables from the target object using the `parameters` mapping. 3. Creates an independent Generator run for that member, passing the scoped variables to the GraphQL query. A Generator definition targeting a group with 10 members produces 10 separate runs. Each run sees only the data relevant to its specific target object. ``` Generator Definition │ ▼ Target Group ├── Member A → Run A (variables from A) ├── Member B → Run B (variables from B) └── Member C → Run C (variables from C) ``` Each run is fully independent — it has its own query variables, its own query results, and its own Generator instance. Runs do not share state. ## Query parameter mapping[​](#query-parameter-mapping "Direct link to Query parameter mapping") The `parameters` field in `.infrahub.yml` controls how Infrahub extracts variables from each target object and passes them to the GraphQL query. This is the mechanism that scopes each run to its target. ### How it works[​](#how-it-works "Direct link to How it works") Given this Generator definition: ``` generator_definitions: - name: widget_generator file_path: "generators/widget_generator.py" targets: widgets query: widget_query class_name: WidgetGenerator parameters: name: "name__value" ``` And this GraphQL query: ``` query Widgets($name: String!) { TestWidget(name__value: $name) { edges { node { name { value } count { value } } } } } ``` For each member of the `widgets` group, Infrahub: 1. Reads the parameter mapping: `name` → `"name__value"` 2. Extracts the value from the target object using the defined path 3. Passes it as a query variable For example: | Target object | Extraction path | Extracted value | Query variable | | ------------- | -------------------- | --------------- | ------------------- | | `widget1` | `widget1.name.value` | `"widget1"` | `$name = "widget1"` | | `widget2` | `widget2.name.value` | `"widget2"` | `$name = "widget2"` | Each run's GraphQL query only returns data for its specific target, keeping runs independent. ### Double-underscore notation[​](#double-underscore-notation "Direct link to Double-underscore notation") The double-underscore (`__`) in parameter values traverses the object hierarchy: * `name__value`: attribute `name`, property `value` * `location__name__value`: relationship `location` (cardinality-one), then attribute `name`, property `value` The first segment is checked against the object's schema. If it matches an attribute, the remaining segments traverse the attribute's properties. If it matches a cardinality-one relationship, Infrahub fetches the related node and continues the traversal recursively. info Only cardinality-one relationships are supported in parameter paths. Cardinality-many relationships cannot be traversed this way. ## Parallel execution[​](#parallel-execution "Direct link to Parallel execution") Because each run is independent — scoped to one target object with no shared state — Infrahub dispatches all runs for a Generator definition concurrently. This means: * **All members of a target group are processed in parallel**, not sequentially. * **Performance scales with available workers**, not with target count. A group with 100 members doesn't take 100x longer than a group with 1 member. * **Different Generator definitions** can also run concurrently when triggered independently. ### What this means for Generator design[​](#what-this-means-for-generator-design "Direct link to What this means for Generator design") Because runs are concurrent: * Your Generator code should not depend on side effects from other runs of the same Generator. * Each run should be self-contained — it reads its scoped data, creates its objects, and finishes. * If you need ordering (layer A must complete before layer B starts), use separate Generator definitions with a trigger mechanism rather than relying on execution order within a single definition. See [modular Generators](/generators/modular.md) for this pattern. ## Generator instances[​](#generator-instances "Direct link to Generator instances") Each per-target run creates or updates a `CoreGeneratorInstance` — a tracking object that links three things together: 1. The **Generator definition** that was run 2. The **target object** (the specific group member) 3. The **status** of that run (`pending`, `ready`, or `error`) Generator instances enable: * **Per-target status tracking**: you can see which targets succeeded and which failed, without needing to inspect logs. * **Selective re-runs**: you can re-run the Generator for a single target object without affecting others. Only the instance for that target gets updated. * **Object lifecycle management**: the instance links the Generator to the objects it created, enabling cleanup when a target is removed. You can view Generator instances in the Infrahub UI under the Generator Definition detail page. ## Designing groups for parallelism[​](#designing-groups-for-parallelism "Direct link to Designing groups for parallelism") Since group structure determines execution structure, how you organize your target groups directly affects parallelism and operational flexibility. ### The principle[​](#the-principle "Direct link to The principle") **Group at the level where you want independent execution.** If racks should generate independently, make racks the target — not pods. If entire sites should generate as a unit, make sites the target. More members in the target group means more parallel runs and better utilization of available workers. ### Example: modular parallelism[​](#example-modular-parallelism "Direct link to Example: modular parallelism") In a modular Generator setup (see [modular Generators](/generators/modular.md)), parallelism increases at each layer: | Layer | Target group | Members | Parallel runs | | ------ | ------------ | --------- | ------------- | | Fabric | `dc_fabrics` | 1 fabric | 1 | | Pod | `dc_pods` | 4-8 pods | 4-8 | | Rack | `dc_racks` | 32+ racks | 32+ | The fabric Generator runs once (1 target). It creates pod objects. The pod Generator runs 4-8 times concurrently. Each pod Generator creates rack objects. The rack Generator runs 32+ times concurrently. Total parallelism is multiplicative across layers. ### Guideline: prefer more, smaller targets[​](#guideline-prefer-more-smaller-targets "Direct link to Guideline: prefer more, smaller targets") A single Generator targeting a group with one member that creates 100 objects runs as one sequential operation. The same work split across 10 targets with 10 objects each runs as 10 concurrent operations — significantly faster on a system with available capacity. ## The relationship between `member_of_groups` and Generator targeting[​](#the-relationship-between-member_of_groups-and-generator-targeting "Direct link to the-relationship-between-member_of_groups-and-generator-targeting") Objects become Generator targets by being members of the group specified in the Generator definition's `targets` field. ### How objects become targets[​](#how-objects-become-targets "Direct link to How objects become targets") You add objects to a target group through the `member_of_groups` relationship, which can be set: * **In the UI**: when creating or editing an object (see [organizing objects with groups](/groups/overview.md)) * **In object files**: define group membership in your YAML object definitions * **Programmatically**: via the SDK or GraphQL mutations ### Dynamic targeting[​](#dynamic-targeting "Direct link to Dynamic targeting") Adding or removing group members changes what gets targeted on the next Generator run: * **Add an object to the group**: it becomes a target and gets its own Generator run next time the definition executes * **Remove an object from the group**: it is no longer targeted (existing generated objects are not automatically cleaned up — see [known limitation #3289](https://github.com/opsmill/infrahub/issues/3289)) ### Standard groups vs. Generator groups[​](#standard-groups-vs-generator-groups "Direct link to Standard groups vs. Generator groups") These two group types serve different purposes and are often confused: | Group type | Purpose | You manage it | | -------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | | `CoreStandardGroup` | Defines which objects are **targeted by** a Generator. Listed in the `targets` field of the Generator definition. | Yes — you create it and control membership | | `CoreGeneratorGroup` | Tracks which objects were **created by** a Generator instance. Managed automatically by the SDK tracking feature. | No — Infrahub manages this automatically | The target group (`CoreStandardGroup`) is an input to the Generator — "run against these objects." The Generator group (`CoreGeneratorGroup`) is an output — "these objects were created by this Generator." ## Query response modes[​](#query-response-modes "Direct link to Query response modes") The `convert_query_response` flag in `.infrahub.yml` controls how the GraphQL query results are delivered to your `generate()` method. This affects how you access data and what SDK features are available. ### Raw dict mode (default)[​](#raw-dict-mode-default "Direct link to Raw dict mode (default)") When `convert_query_response` is `false` (the default), the `data` parameter passed to `generate()` is the raw GraphQL response dictionary: ``` class DeviceGenerator(InfrahubGenerator): async def generate(self, data: dict) -> None: for edge in data["TestDevice"]["edges"]: device_name = edge["node"]["name"]["value"] device_role = edge["node"]["role"]["value"] # ... create objects using self.client ``` This is the more lightweight mode — no conversion overhead, and you work with plain Python dictionaries. You can also use [Pydantic models](https://docs.pydantic.dev/) to parse the response into typed objects for better IDE support and validation. ### SDK object mode[​](#sdk-object-mode "Direct link to SDK object mode") When `convert_query_response` is `true`, Infrahub converts the GraphQL response into `InfrahubNode` SDK objects. These are available via `self.nodes` and `self.store`. You do **not** use the `data` parameter in this mode: ``` class DeviceGenerator(InfrahubGenerator): async def generate(self, data: dict) -> None: for device in self.nodes: device_name = device.name.value device_role = device.role.value # SDK features available: relationships, .save(), .delete() ``` ### When to use each[​](#when-to-use-each "Direct link to When to use each") | Use raw dict mode (`false`) when | Use SDK object mode (`true`) when | | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | Dict access is sufficient for your use case | You need SDK features like `.save()`, `.delete()`, or relationship traversal on the queried objects | | You want minimal overhead | You prefer cleaner attribute access (`node.name.value` vs `node["name"]["value"]`) | | The query returns a flat structure | You want to use `self.store` to look up nodes by ID | | You want to use Pydantic models for type-safe parsing | You want the SDK to handle response parsing automatically | ## Execution lifecycle[​](#execution-lifecycle "Direct link to Execution lifecycle") The `execute_in_proposed_change` and `execute_after_merge` flags in `.infrahub.yml` control **when** a Generator runs in relation to the branch lifecycle. ### `execute_in_proposed_change` (default: `true`)[​](#execute_in_proposed_change-default-true "Direct link to execute_in_proposed_change-default-true") When `true`, the Generator runs during proposed change CI. When `false`, the Generator is skipped entirely in proposed changes. Use this for Generators that are triggered by events (checksum triggers) or when you want to manually run the generator. ### `execute_after_merge` (default: `true`)[​](#execute_after_merge-default-true "Direct link to execute_after_merge-default-true") When `true`, the Generator runs again after the branch has been merged. When `false`, the Generator will not run after the branch has been merged. ## Running a Generator from events[​](#running-a-generator-from-events "Direct link to Running a Generator from events") Inside a proposed change, Generator runs and artifact generation start at the same time, so an artifact that reads a Generator's output can render before that output exists (see [Ordering between Generators and artifacts](/proposed-changes/overview.md#ordering-between-generators-and-artifacts)). Run the Generator on the branch before you open the proposed change. Chain two event rules: 1. A [node trigger rule](/events/event-rules.md#configure-a-node-trigger-rule) matches the mutation that creates the input data and fires a group action that adds the object to the Generator definition's target group. 2. A [group trigger rule](/events/event-rules.md#configure-a-group-trigger-rule) on that group matches the member-added event and fires a [Generator action](/events/event-actions.md#configure-a-generator-action) that runs the Generator against the new member. The Generator runs only for the objects that are members of the Generator definition's target group. If none of the triggering objects are members, the run fails with `Target is not part of the group `. Create both rules on the default branch, and set `branch_scope` to `other_branches` so the chain fires on the branches where you review the result in a proposed change. A rule created on another branch does not fire (see [where trigger rules are stored and evaluated](/events/event-rules.md#where-trigger-rules-are-stored-and-evaluated)). Set `execute_in_proposed_change: false` on the Generator, so that it does not also run inside the proposed-change pipeline. The flag applies only to that pipeline: it does not prevent an event from running the Generator. ## Video guides[​](#video-guides "Direct link to Video guides") The video series below dives into the concept of Generators and services, exploring their significance, structure, and how they can streamline processes for teams. Whether you're a developer or just curious about automation in IT, the series provides a comprehensive understanding of Generators and their applications. The first video will highlight what Generators are and how they can be used to deliver services.
In the second video we will look at how to plan a Generator from coming up with a use case and then finally what the workflow may look like in pseudocode.
In the third video we will look at how a Generator can be created and run in Infrahub. Looking at the `.infrahub.yml` file the GraphQL query the Generator will run against and finally the logic that will be run against Infrahub to create objects and bring the service to life. ## Known limitations[​](#known-limitations "Direct link to Known limitations") * [3289](https://github.com/opsmill/infrahub/issues/3289) deleting a Generator target object should delete the created objects of that target ## Learn by doing[​](#learn-by-doing "Direct link to Learn by doing") For a step-by-step walkthrough that builds a Generator from scratch, see [Build your first Generator](/learn/tutorials/generators/build-your-first-generator.md) in the Academy tutorials. --- # Remove a generator Removing a Generator involves two separate things: the **Generator definition** registered from `.infrahub.yml`, and the **objects the Generator created** in Infrahub. Removing the definition stops the Generator from running, but it does not delete the objects it already created — those stay in Infrahub until you remove them. Handle the objects first, then the definition. ## Delete the objects the Generator created[​](#delete-the-objects-the-generator-created "Direct link to Delete the objects the Generator created") A Generator removes its own obsolete objects only when it runs, through the [SDK tracking feature](/python-sdk/topics/tracking.md). Once the definition is gone the Generator never runs again, so it can no longer remove the objects it created. Delete or reassign those objects — through the UI, GraphQL, or the SDK — while the definition still exists. To identify what a Generator created, see [Generator instances](/generators/overview.md#generator-instances). ## Remove the definition from `.infrahub.yml`[​](#remove-the-definition-from-infrahubyml "Direct link to remove-the-definition-from-infrahubyml") On a branch, delete the Generator's entry from the `generator_definitions` list in your repository's `.infrahub.yml`: ``` generator_definitions: # Remove the block for the Generator you are retiring: - name: file_path: "generators/.py" targets: query: class_name: YourGenerator ``` Remove the Generator's Python file, and any query used only by this Generator. Commit the changes and merge the branch through a [Proposed Change](/proposed-changes/overview.md). Infrahub removes the `CoreGeneratorDefinition` registration on the next repository sync. ## Verify the removal[​](#verify-the-removal "Direct link to Verify the removal") ``` infrahubctl generator --list ``` The retired Generator no longer appears in the list. In the UI, confirm it is gone from Actions → Generator Definitions. ## Related[​](#related "Direct link to Related") * [Build a generator](/generators/build.md) — create a Generator * [About Generators](/generators/overview.md) — concept, instances, and tracking * [infrahub.yml configuration](/git-integration/infrahub-yml.md) — the `generator_definitions` syntax --- # Selective branch synchronization Infrahub automatically creates all branches from a connected Git repository by default. While this ensures full visibility, it can create unnecessary noise when many Git branches are unrelated to Infrahub data or workflows. Selective branch synchronization introduces an optional configuration that allows teams to control which Git branches are imported and synchronized into Infrahub based on custom naming patterns. ## Why selective synchronization matters[​](#why-selective-synchronization-matters "Direct link to Why selective synchronization matters") In real-world Git repositories, teams often maintain many branches for different purposes: * Development branches for features unrelated to infrastructure * Documentation-only branches * Experimental work that doesn't involve Infrahub data * Personal development branches * Archive branches kept for historical reference When all these branches automatically sync to Infrahub, it creates: * **Visual clutter**: Branch lists become difficult to navigate with irrelevant entries * **Performance overhead**: Unnecessary synchronization consumes system resources * **Workflow confusion**: Team members may struggle to identify which branches contain infrastructure changes Selective synchronization addresses these challenges by allowing administrators to define explicit rules for which branches should be imported and maintained in Infrahub. ## How selective synchronization works[​](#how-selective-synchronization-works "Direct link to How selective synchronization works") The feature operates through a set of branch naming rules that determine which Git branches Infrahub will track and synchronize. ### Default behavior[​](#default-behavior "Direct link to Default behavior") By default, every branch created in your Git repository continues to be automatically imported and synchronized into Infrahub. No configuration changes are required for teams who want to maintain the current behavior. This default ensures backward compatibility and zero-configuration operation for existing deployments. ### Enabling selective synchronization[​](#enabling-selective-synchronization "Direct link to Enabling selective synchronization") When you enable selective branch synchronization, Infrahub will only import branches whose names match a set of user-defined rules. Each rule is a regular expression that defines a pattern for matching branch names. info Selective synchronization only affects the Git to Infrahub direction. Branches created in Infrahub with the "sync to Git" option enabled will still synchronize to Git normally. ## Configuring selective synchronization[​](#configuring-selective-synchronization "Direct link to Configuring selective synchronization") Set the `INFRAHUB_GIT_IMPORT_SYNC_BRANCH_NAMES` environment variable (or the equivalent `sync_branch_names` key in `infrahub.toml`) to a JSON array of regular expression patterns: ``` INFRAHUB_GIT_IMPORT_SYNC_BRANCH_NAMES=["^infra/.*$","^feature/infrahub-.*$"] ``` A configuration change requires a restart of the Infrahub server. For detailed instructions on applying configuration changes to your deployment, see [How to configure Infrahub](/deploy-manage/install-configure/configure-infrahub.md). [How to configure Infrahub../deploy-manage/install-configure/configure-infrahub](../deploy-manage/install-configure/configure-infrahub) Validation After setting those variables and restarting, branch synchronization will follow the defined rules. ## Rule processing[​](#rule-processing "Direct link to Rule processing") Infrahub processes the rules in the order they appear in the configuration. The first matching expression determines whether a branch is synchronized. **Processing logic:** 1. When Infrahub discovers a branch in the Git repository, it evaluates the branch name against each rule in sequence 2. The first rule that matches the branch name determines the outcome 3. If a match is found, the branch will be created and synchronized in Infrahub 4. If no match is found after checking all rules, the branch will be ignored This top-to-bottom evaluation allows for fine-grained control over branch synchronization behavior. ### Rule evaluation during updates[​](#rule-evaluation-during-updates "Direct link to Rule evaluation during updates") When you modify the synchronization rules after a repository has been imported, Infrahub evaluates branches differently depending on their current state: **Adding patterns to expand scope:** If a Git branch exists but was not previously imported because it didn't match the old rules, and it now matches the updated rules, Infrahub will import that branch during the next synchronization cycle. ``` # Before sync_branch_names = ["^infra/.*$"] # After — network/topology and similar branches are now imported sync_branch_names = [ "^infra/.*$", "^network/.*$" ] ``` **Removing patterns to restrict scope:** If a branch was already imported into Infrahub and has the "sync with Git" flag enabled, it will continue to synchronize even if it no longer matches the updated rules. This preserves existing work and prevents unexpected data loss. ``` # Before sync_branch_names = [ "^infra/.*$", "^feature/.*$" ] # After — existing feature/* branches in Infrahub continue syncing sync_branch_names = ["^infra/.*$"] ``` **Cleaning up branches manually:** If you want to stop synchronizing branches that no longer match your rules: 1. Navigate to **Branches** in the Infrahub UI 2. Find the branch you want to remove and click the delete option 3. Or delete branches programmatically via the API Permanent deletion Manually deleting branches in Infrahub is permanent. The deletion does not affect the corresponding Git branches, and they will not be re-imported because they no longer match your rules. info The synchronization rules only control initial import of branches from Git to Infrahub. Once a branch exists in Infrahub with Git synchronization enabled, it continues syncing regardless of rule changes. ## Regular expression patterns[​](#regular-expression-patterns "Direct link to Regular expression patterns") Each rule uses standard regular expression syntax to define matching patterns. This approach provides maximum flexibility without enforcing a specific naming convention. Common pattern elements you can use: * `^` - Match the beginning of the branch name * `$` - Match the end of the branch name * `.*` - Match any characters (zero or more) * `[a-z]` - Match any lowercase letter * `[0-9]+` - Match one or more digits * `(feature|fix)` - Match either "feature" or "fix" ### Example 1: infrastructure-only branches[​](#example-1-infrastructure-only-branches "Direct link to Example 1: infrastructure-only branches") Your team uses a naming convention where all infrastructure-related branches start with `infra/`. **Rule:** ``` ["^infra/.*$"] ``` **Results:** | Git Branch Name | Matches Rule? | Imported to Infrahub? | | ---------------------- | ------------- | ---------------------- | | `infra/network-update` | ✅ Yes | ✅ Created in Infrahub | | `infra/add-devices` | ✅ Yes | ✅ Created in Infrahub | | `docs/update-readme` | ❌ No | 🚫 Not imported | | `feature/new-api` | ❌ No | 🚫 Not imported | ### Example 2: multiple team prefixes[​](#example-2-multiple-team-prefixes "Direct link to Example 2: multiple team prefixes") Different teams use different prefixes for their infrastructure work. **Rules:** ``` [ "^infra/.*$", "^network/.*$", "^config/.*$" ] ``` **Results:** | Git Branch Name | Matches Rule? | Imported to Infrahub? | | ------------------- | ------------- | ---------------------- | | `infra/devices` | ✅ Yes | ✅ Created in Infrahub | | `network/topology` | ✅ Yes | ✅ Created in Infrahub | | `config/templates` | ✅ Yes | ✅ Created in Infrahub | | `docs/architecture` | ❌ No | 🚫 Not imported | ### Example 3: feature branches with issue numbers[​](#example-3-feature-branches-with-issue-numbers "Direct link to Example 3: feature branches with issue numbers") Your workflow includes Infrahub-related feature branches that reference issue numbers. **Rule:** ``` ["^feature/infrahub-[0-9]+.*$"] ``` **Results:** | Git Branch Name | Matches Rule? | Imported to Infrahub? | | ------------------------------ | ------------- | ---------------------- | | `feature/infrahub-321` | ✅ Yes | ✅ Created in Infrahub | | `feature/infrahub-123-network` | ✅ Yes | ✅ Created in Infrahub | | `feature/api-update` | ❌ No | 🚫 Not imported | | `feature/device-backup` | ❌ No | 🚫 Not imported | ## Verifying the configuration[​](#verifying-the-configuration "Direct link to Verifying the configuration") To verify that the selective synchronization rules are working correctly, you need a Git repository connected to Infrahub. If you haven't connected a repository yet, see [How to connect external Git repositories](/git-integration/connect-repository.md). ### Test with branches that should NOT be imported[​](#test-with-branches-that-should-not-be-imported "Direct link to Test with branches that should NOT be imported") 1. Create a branch in your Git repository that doesn't match your rules: ``` git checkout -b docs/update-readme git push origin docs/update-readme ``` 2. Wait a few seconds for the synchronization cycle to complete 3. Check the Infrahub UI or API — this branch should **not** appear in Infrahub ### Test with branches that SHOULD be imported[​](#test-with-branches-that-should-be-imported "Direct link to Test with branches that SHOULD be imported") 1. Create a branch in your Git repository that matches your rules: ``` git checkout -b infra/network-update git push origin infra/network-update ``` 2. Wait a few seconds for the synchronization cycle to complete 3. Check the Infrahub UI or API — this branch **should** appear in Infrahub ### Verify using the Infrahub UI[​](#verify-using-the-infrahub-ui "Direct link to Verify using the Infrahub UI") 1. Navigate to the **Branches** section in the Infrahub UI 2. Confirm that only branches matching your rules are displayed ### Verify using the API[​](#verify-using-the-api "Direct link to Verify using the API") Query the GraphQL API to list all branches: ``` query { Branch { edges { node { name } } } } ``` Only branches matching your configured rules should appear in the results. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") ### Branches not appearing after configuration[​](#branches-not-appearing-after-configuration "Direct link to Branches not appearing after configuration") **Possible causes:** * Infrahub services haven't been restarted after configuration changes * Regular expression pattern doesn't match the branch name * Synchronization cycle hasn't completed yet **Solutions:** 1. Verify the configuration is correct 2. Restart Infrahub services 3. Wait 10–15 seconds for the next synchronization cycle 4. Test your regex pattern using an online regex tester ### Branches still appearing after adding rules[​](#branches-still-appearing-after-adding-rules "Direct link to Branches still appearing after adding rules") **Possible causes:** * The branches were already imported before you configured the rules * Branches have the "sync with Git" flag enabled * An environment variable is overriding the configuration file **Solutions:** 1. Check if an environment variable is set and taking precedence 2. Delete existing branches manually if needed — rules only control new imports, not existing branches ### All branches are being imported despite rules[​](#all-branches-are-being-imported-despite-rules "Direct link to All branches are being imported despite rules") **Possible causes:** * Configuration not loaded correctly * Services not restarted * Syntax error in the configuration **Solutions:** 1. Check Infrahub logs for configuration errors 2. Verify the TOML syntax is correct 3. Verify the JSON syntax is correct for environment variables 4. Restart services after any configuration change ## Current scope and limitations[​](#current-scope-and-limitations "Direct link to Current scope and limitations") ### Directional behavior[​](#directional-behavior "Direct link to Directional behavior") Selective branch synchronization only affects the **Git → Infrahub** direction: * Git branches that don't match the rules will not be imported into Infrahub * Branches created in Infrahub with the "sync to Git" option enabled will still synchronize to Git normally * This asymmetric behavior allows teams to use Infrahub as the source of truth while filtering incoming Git branches ### Dynamic updates[​](#dynamic-updates "Direct link to Dynamic updates") When you modify the `sync_branch_names` configuration: * New Git branches will be evaluated against the updated rules during the next synchronization * Git branches that now match the rules will be automatically imported * Existing Infrahub branches with Git synchronization enabled continue syncing regardless of rule changes * Branches no longer matching the rules will **not** be automatically removed from Infrahub * You must manually delete branches in Infrahub if they no longer align with your synchronization policy ### Repository types[​](#repository-types "Direct link to Repository types") Selective branch synchronization applies to standard read-write repositories that support full bidirectional integration. Read-only repositories track a single reference and are not affected by this configuration. ## Further reading[​](#further-reading "Direct link to Further reading") * [Understanding Git repositories in Infrahub](/git-integration/overview.md) - Repository integration concepts and architecture * [How to connect external Git repositories](/git-integration/connect-repository.md) - Step-by-step repository setup guide * [Understanding branching in Infrahub](/branches/overview.md) - Branch isolation, hierarchy, and workflows * [Configuration reference](/reference/configuration.md) - Complete configuration options including Git settings --- # How to connect external Git repositories Connecting an external Git repository will enable many features in Infrahub, such as Transformations, Generators, Checks ... that rely on the repository to store code files. This guide explains how to connect repositories, pull changes, and troubleshoot common issues. Repository vs. Read-only Repository If you want to know more about the differences between these two types, refer to the topic on [Repositories](/git-integration/overview.md). **TL;DR:** * **Repository**: fully integrates with Git version control, including branch tracking and two-way branch synchronization * **Read-only Repository**: links a particular branch in Infrahub to a particular ref in the Git repository. It will only read from the Git repository and never make changes to the external repository. Most of the time, you will want to use a **Repository** connection. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before you begin, you'll need: * An Infrahub instance up and running * Access to a Git repository containing valid Infrahub resources (see [Initializing an Infrahub repository](/development-resources/developer-guide.md#initializing-an-infrahub-repository)) * A [`.infrahub.yml`](/git-integration/overview.md#infrahub-yaml) configuration file in your Git repository * For private repositories and or Read-Write repositories: appropriate credentials or access tokens * Depending on the method you choose, you may need to install the [Infrahub SDK](/python-sdk/introduction.md) or [infrahubctl](/infrahubctl/infrahubctl.md) Authentication for repositories If your repository is private or if you want to set up a Read-Write Repository, you'll need to create an access token for authentication. Depending on your Git provider, the process may vary: * For GitHub repositories, see the [GitHub access token guide](#github-access-token). * For GitLab repositories, see the [GitLab access token guide](#gitlab-access-token). ## Connect a repository[​](#connect-a-repository "Direct link to Connect a repository") ### Step 1: Collect repository information[​](#step-1-collect-repository-information "Direct link to Step 1: Collect repository information") To create a repository connection in Infrahub, you'll need to prepare the following information: * **Name:** The identifier you wish to assign to the repository in Infrahub (for example, `My Git Repository`) * **Repository location:** The URL of the external Git repository (for example, `https://github.com/opsmill/infrahub.git`) For Read-only repositories, you will also need: * **Ref:** A branch, tag, or commit reference to pull (for example, `main`, `v1.0`, or a specific commit hash) For private repositories and/or Read-Write repositories: * **Authentication:** A username/password combination or personal access token ### Step 2: Add the repository[​](#step-2-add-the-repository "Direct link to Step 2: Add the repository") Access token configuration If you are using a **personal access token for authentication**, you should put the token in the `password` field of the credential and leave the `username` field empty. * Via the Web Interface * Via infrahubctl * Via the GraphQL Interface * Via the Infrahub SDK 1. Log in to the Infrahub UI 2. Go to **Integrations > Git Repositories** 3. Click on the **+ Add Git Repository** button 4. Select the repository type (`Repository` or `Read-only Repository`) 5. Complete the required information: * Enter the repository name and location (URL) * Select a credential if connecting to a private repository * For Read-only Repository, specify the ref to pull * Add optional description and tags ![Add a Git Repository](/assets/images/create_repository-546422aa6d5cc1d1861c15f99e3818da.png) Using the [infrahubctl](/infrahubctl/infrahubctl.md) command-line tool, you can add a repository by running the following command: * Repository * Read-only Repository ``` infrahubctl repository add "My Git Repository" "https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git" --username MY_USERNAME --password MY_TOKEN_OR_PASSWORD ``` ``` infrahubctl repository add "My Git Repository" "https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git" --read-only --username MY_USERNAME --password MY_TOKEN_OR_PASSWORD ``` Using the GraphQL API, you can add a `Repository` or `Read-only Repository` via mutations. 1. Open the **GraphQL Interface** GraphQL Sandbox Access the sandbox by clicking your user icon in the bottom left corner and selecting **GraphQL Sandbox**. 2. If relevant, create a Credential object with your username and password/token: ``` mutation { CorePasswordCredentialCreate( data: { name: { value: "my-git-credential" }, username: { value: "MY_USERNAME" }, password: { value: "MY_TOKEN_OR_PASSWORD" } } ) { ok object { hfid } } } ``` 3. Use one of the mutations below based on your repository type: * Repository * Read-only Repository ``` mutation { CoreRepositoryCreate( data: { name: { value: "My Git Repository" }, location: { value: "https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git" }, # Optional: name of the Git branch mapped to Infrahub's default branch, where merges are pushed back (default "main"; set your Git default branch name for a non-main default) default_branch: { value: "main" }, # The HFID returned in step 2 will be used for the credentials credential: { hfid: ["my-git-credential"] } } ) { ok object { id } } } ``` ``` mutation { CoreReadOnlyRepositoryCreate( data: { name: { value: "My Git Repository" }, location: { value: "https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git" }, ref: { value: "BRANCH/TAG/COMMIT_TO_TRACK" }, # Optional: use the credential created earlier credential: { hfid: ["my-git-credential"] } } ) { ok object { id } } } ``` 1. If relevant, create a Credential object with your username and password/token: ``` # Create credential object credential = client.create( "CorePasswordCredential", name="My Git Credential", username="MY_USERNAME", password="MY_TOKEN_OR_PASSWORD", ) credential.save() ``` 2. Create the repository object: * Repository * Read-only Repository ``` # Create repository object repository = client.create( "CoreRepository", name="My Git repository", location="https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git", default_branch="main", # Optional: Git branch mapped to Infrahub's default branch, where merges are pushed back (default "main"; set your Git default branch name for a non-main default) credential=credential, # The credential object created above ) repository.save() ``` ``` # Create repository object repository = client.create( "CoreReadOnlyRepository", name="My Git repository", location="https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git", ref="BRANCH/TAG/COMMIT_TO_TRACK", credential=credential, # Optional: use the credential created above ) repository.save() ``` Check repository connectivity After creation, your new repository should appear under **Integrations > Git Repositories**. If the status shows `Unknown` or an error state, please refer to the [troubleshooting section](#troubleshoot-repository-connections). ## Advanced use cases[​](#advanced-use-cases "Direct link to Advanced use cases") ### Update a read-only repository with remote changes[​](#update-a-read-only-repository-with-remote-changes "Direct link to Update a read-only repository with remote changes") Unlike fully integrated repositories which sync automatically, read-only repositories require manual action to pull in new changes. There are two ways to do this: * **Change the `ref` property**: Point the repository to a different branch, tag, or commit hash * **Import latest commit**: Fetch the latest commit from the remote for the current ref and re-import resources For more details on how each repository type tracks changes, refer to the [Repository Topic](/git-integration/overview.md#read-only-vs-core). #### Change the ref[​](#change-the-ref "Direct link to Change the ref") * Via the Web Interface * Via the GraphQL Interface 1. Log in to the Infrahub UI 2. Go to **Integrations > Git Repositories** 3. Click on the `CoreReadOnlyRepository` record you want to update 4. Click on the **Edit Read-Only Repository** button 5. Update the `ref` field with the new branch, tag, or commit reference to pull 6. Click **Save** to apply the changes 1) Open the **GraphQL Interface** GraphQL Sandbox Access the sandbox by clicking your user icon in the bottom left corner and selecting **GraphQL Sandbox**. 2. Execute the update mutation with the new ref value: ``` mutation { CoreReadOnlyRepositoryUpdate( data: { hfid: ["My Git repository"], ref: { value: "BRANCH/TAG/COMMIT_TO_TRACK" } } ) { ok object { id syncStatus } } } ``` Check synchronization status After updating the ref, the repository will begin synchronizing. You can monitor the process by checking the repository's sync status, which should change to `Syncing` and then to `In sync` when complete. The commit field will also update to reflect the latest commit from the specified ref. #### Import latest commit[​](#import-latest-commit "Direct link to Import latest commit") If the remote repository has new commits on the same ref that your read-only repository is already tracking, you can pull the latest commit without changing the ref. This fetches from the remote, resolves the latest commit for the tracked ref, and re-imports all resources. * Via the Web Interface * Via the GraphQL Interface 1. Log in to the Infrahub UI 2. Go to **Integrations > Git Repositories** 3. Click on the `CoreReadOnlyRepository` record you want to update 4. Click the **More** menu 5. Select **Import latest commit** The repository will fetch the latest changes from the remote, update to the most recent commit on the tracked ref, and re-import resources. 1. Open the **GraphQL Interface** GraphQL Sandbox Access the sandbox by clicking your user icon in the bottom left corner and selecting **GraphQL Sandbox**. 2. Execute the import mutation with the repository ID: ``` mutation { InfrahubReadOnlyRepositoryImportLastCommit( data: { id: "REPOSITORY_ID" } ) { ok task { id } } } ``` The mutation returns a task ID that you can use to monitor the progress of the import operation. Check synchronization status After triggering the import, the repository will fetch from the remote and re-import resources. You can monitor the process by checking the repository's sync status and the commit field, which will update to reflect the latest commit from the tracked ref. ### Troubleshoot repository connections[​](#troubleshoot-repository-connections "Direct link to Troubleshoot repository connections") If you encounter issues with your repository connections, you can use the following steps to diagnose and resolve them: 1. Check the repository status indicators: * **Admin status**: Indicates if Infrahub is actively using the repository * **Operational status**: Shows connectivity between Infrahub and the repository * **Sync status**: Tracks synchronization operations For detailed information about what each status means, refer to the [Repository Status documentation](/git-integration/overview.md#repository-statuses). 2. View detailed operation logs: * Log in to the Infrahub UI * Go to **Integrations > Git Repositories** * Select the relevant repository record * Click on the **Tasks** tab to view detailed logs of Git operations 3. Use repository actions for advanced troubleshooting: * From the repository detail view, click the **More** menu * Select **Check connectivity** to verify authentication and connection * Select **Reimport current commit** to force reimport without changing the reference * For read-only repositories, select **Import latest commit** to fetch and import the latest commit from the remote for the tracked ref Validation A healthy repository connection should show `Online` for operational status and `In sync` for sync status once all operations complete. ### GitHub access token[​](#github-access-token "Direct link to GitHub access token") Please refer to the official GitHub documentation for creating access tokens. GitHub offers two types of tokens: * [Personal Access Token (classic)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic) * [Fine-grained Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-fine-grained-personal-access-token) Personal Access Token vs. Fine-grained Access Token For new repositories, we recommend using the **Fine-grained Access Token** as it allows more granular control over permissions. Grant the token permission: * For a `Repository`, grant **Read/Write** access to the repository content * For a `Read-only Repository`, grant **Read** access only Token created Now that you have created your token, you can use it to connect your GitHub repository to Infrahub. In this case, you will need to put the token in the `password` field of the credential and leave the `username` field empty. ### GitLab access token[​](#gitlab-access-token "Direct link to GitLab access token") Please refer to the official GitLab documentation for creating access tokens. Project Access Tokens We recommend using a [project access token](https://docs.gitlab.com/user/project/settings/project_access_tokens/) which has the advantage of being scoped to a specific project. Security best practice While personal access tokens or username/password authentication might work depending on your setup, we strongly recommend against using them as they are scoped to user accounts rather than specific repositories. Grant the token permission: * For a `Repository`, select both `read_repository` and `write_repository` scopes * For a `Read-only Repository`, select the `read_repository` scope Token created Now that you have created your token, you can use it to connect your GitLab repository to Infrahub. In this case, you will need to put the token in the `password` field of the credential and leave the `username` field empty. ### Connecting to a Git server with a private CA[​](#connecting-to-a-git-server-with-a-private-ca "Direct link to Connecting to a Git server with a private CA") If your Git server presents a certificate signed by a private Certificate Authority (CA), point Infrahub at your CA bundle instead of rebuilding the Docker image: 1. Save your CA certificate and every intermediate certificate in a single PEM file, such as `ca-bundle.pem` 2. Mount the file read-only into the `infrahub-server` and `task-worker` containers, for example at `/opt/infrahub/tls/ca-bundle.pem`, or skip the mount and pass the PEM text through the setting in the next step 3. Set `INFRAHUB_TLS_CA_BUNDLE=/opt/infrahub/tls/ca-bundle.pem` so every component trusts the CA, or `INFRAHUB_GIT_TLS_CA_FILE=/opt/infrahub/tls/ca-bundle.pem` to scope it to git only; both settings also accept the PEM text itself 4. Restart your environment The task worker writes the path as `http.sslCAInfo` into its git configuration file at startup (`INFRAHUB_GIT_GLOBAL_CONFIG_FILE`, default `/opt/infrahub/.gitconfig`), so every clone, fetch and push over HTTPS uses it. The mount and configuration steps for Docker Compose, Docker and Helm are detailed in [Trust a private CA](/deploy-manage/install-configure/production-deployment/private-ca.md). Bundle contents The bundle replaces the system trust store for git. If Infrahub also connects to Git servers with publicly issued certificates, such as GitHub or GitLab, append the public root certificates to the bundle. Validation Run `docker compose exec task-worker git config --file /opt/infrahub/.gitconfig --get http.sslCAInfo`, using the path from `INFRAHUB_GIT_GLOBAL_CONFIG_FILE` if you changed it: it prints the path of your bundle. Read the file explicitly rather than using `--global`: Infrahub selects that file through `GIT_CONFIG_GLOBAL` in its own process only, so an exec shell's `--global` reads a different file. A repository hosted on your Git server now reaches the `Online` operational status instead of failing with `SSL verification failed`. ### Disabling certificate verification (not recommended)[​](#disabling-certificate-verification-not-recommended "Direct link to Disabling certificate verification (not recommended)") Security risk Disabling certificate validation is a bad security practice and is strongly discouraged. Only use this option in controlled development environments. If you absolutely need to disable certificate verification for testing purposes, set `INFRAHUB_GIT_TLS_INSECURE=true` on the task workers and restart your environment. The task worker sets `http.sslVerify` to `false` in its git configuration file, so git skips certificate validation for every HTTPS remote. It takes precedence over `INFRAHUB_GIT_TLS_CA_FILE`, so a configured bundle can stay in place while verification is switched off. Infrahub owns `http.sslVerify` and `http.sslCAInfo` in its git configuration file and rewrites them at every task-worker startup, so a value written into that file by hand does not survive a restart; use the setting instead. Validation Run `docker compose exec task-worker git config --file /opt/infrahub/.gitconfig --get http.sslVerify`: it prints `false`, and Infrahub connects to Git servers without validating their certificates. ### Using a proxy server[​](#using-a-proxy-server "Direct link to Using a proxy server") In some network environments, Infrahub's Git worker containers may need to connect through a proxy server to access external repositories. Connection limitation This method only works for HTTP(S) connections. SSH connections through proxies are not supported in this configuration. Maintenance consideration This process requires building a custom Docker image, which you'll need to rebuild with each Infrahub release. 1. Create a Dockerfile with your proxy configuration (update with your actual proxy details): ``` ARG INFRAHUB_VERSION=latest FROM registry.opsmill.io/opsmill/infrahub:${INFRAHUB_VERSION} RUN git config --global http.proxy http://user:password@internal.proxy:8080 ``` 2. Build the Docker image: ``` INFRAHUB_VERSION=latest && docker build --build-arg INFRAHUB_VERSION=$INFRAHUB_VERSION -f Dockerfile -t custom/infrahub:${INFRAHUB_VERSION} . ``` * Using Docker Compose If you are using Docker Compose, you can override part of your existing `docker-compose.yml` file to use your custom image. See the official [Docker Compose documentation](https://docs.docker.com/compose/how-tos/multiple-compose-files/merge/#how-to-merge-multiple-compose-files) for more details. 3. Create a `docker-compose.override.yml` file in the same directory as your `docker-compose.yml` file: ``` --- services: task-worker: image: custom/infrahub:latest ``` 4. Start your environment using the `docker compose` command. Validation After restarting your environment, Infrahub should now connect through your configured proxy server. ### Customizing Git merge behavior[​](#customizing-git-merge-behavior "Direct link to Customizing Git merge behavior") info This configuration applies only to standard repositories, not read-only repositories. By default, Infrahub performs fast-forward only (ff-only) merges between branches. When merging a branch linked to Git, Infrahub updates the destination branch pointer without creating a merge commit. To maintain consistency or improve auditability, you can configure Infrahub to always create a merge commit. important The configuration process will require a restart of the Infrahub server and depends on your deployment method. For detailed instructions on how to apply configuration changes to your Infrahub instance, see [How to configure Infrahub](/deploy-manage/install-configure/configure-infrahub.md). Essentially you will need to set the following environment variables to the desired values: ``` INFRAHUB_GIT_USE_EXPLICIT_MERGE_COMMIT= # Optional: set user name and email for Git commits INFRAHUB_GIT_USER_NAME="" INFRAHUB_GIT_USER_EMAIL="" ``` So in our case, we will set: ``` INFRAHUB_GIT_USE_EXPLICIT_MERGE_COMMIT=true # Optional: set user name and email for Git commits INFRAHUB_GIT_USER_NAME="infrahub-automation" INFRAHUB_GIT_USER_EMAIL="infrahub-automation@example.com" ``` [How to configure Infrahub../deploy-manage/install-configure/configure-infrahub](../deploy-manage/install-configure/configure-infrahub) Validation Test the merge behavior by merging a branch linked to Git. Infrahub should now create a merge commit in the main branch instead of performing a fast-forward update. ## Verification[​](#verification "Direct link to Verification") After completing this guide, you should have successfully connected external Git repositories to Infrahub. You can verify your repository integration is working by checking that: * The repository shows an "Online" operational status * The repository shows an "In sync" sync status * Branches from your repository appear in Infrahub (for fully integrated repositories) * Resources defined in your `.infrahub.yml` file are accessible in Infrahub ## Related resources[​](#related-resources "Direct link to Related resources") * [Repository concept explanation](/git-integration/overview.md) * [Infrahub configuration file documentation](/git-integration/infrahub-yml.md) --- # Develop changes from a Git repository Change the schemas, objects, Transformations, Generators, and other [`.infrahub.yml`](/git-integration/infrahub-yml.md) content an instance imports from a [read-only repository](/git-integration/overview.md#read-only-vs-core), and land the change on the instance's `main` safely. This is the recommended workflow for a single instance; promoting a change across environments builds on it — see [promote changes between environments](/git-integration/promote-between-environments.md). The instance's `main` tracks a Git branch through the read-only repository's `ref`, and you never import onto `main` directly. Instead you develop on an Infrahub branch and merge it into `main` through a proposed change. This worked example adds a `color` attribute to the built-in Tag. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * An Infrahub instance with the repository connected as a **read-only repository** (see [connect a repository](/git-integration/connect-repository.md)). Its `ref` is the Git branch it tracks — `main` in this example. * A [`.infrahub.yml`](/git-integration/overview.md#infrahub-yaml) file on that branch. * A local checkout of the repository and permission to open a pull request. How imports are triggered A read-only repository does not poll for new commits — there is no background sync. An import runs only when you trigger one: the **Import latest commit** action imports the current `ref`'s latest commit, and changing the repository's `ref` dispatches an import on its own (so you do not also run **Import latest commit**). A mutation that returns `ok: true` means the import was dispatched, not finished — confirm a change landed by checking the repository's recorded commit on the instance. ## Step 1: Create the change in Git[​](#step-1-create-the-change-in-git "Direct link to Step 1: Create the change in Git") Work from a feature branch cut from the tracked branch (`main`). ``` git switch main git switch -c feature/add-tag-color ``` Add a schema file that extends the built-in Tag with a `color` attribute: ``` # schemas/tags.yml --- version: '1.0' extensions: nodes: - kind: BuiltinTag attributes: - name: color kind: Text optional: true ``` Make sure `.infrahub.yml` loads it, then commit and push: ``` # .infrahub.yml --- schemas: - schemas/tags.yml ``` ``` git add schemas/tags.yml .infrahub.yml git commit -m "Add color attribute to Tag" git push -u origin feature/add-tag-color ``` ## Step 2: Test it on an Infrahub branch[​](#step-2-test-it-on-an-infrahub-branch "Direct link to Step 2: Test it on an Infrahub branch") Import your feature branch onto an Infrahub branch and check it there before it reaches `main`. 1. On the instance, create an Infrahub branch, for example `test-tag-color`. 2. On `test-tag-color`, set the repository's `ref` to your `feature/add-tag-color` Git branch. Changing the `ref` triggers the import on its own. Because `ref` is branch-aware, only this branch is affected — `main` is untouched. ``` # against the Infrahub branch, e.g. https:///graphql/test-tag-color mutation { CoreReadOnlyRepositoryUpdate(data: { id: "" ref: { value: "feature/add-tag-color" } }) { ok } } ``` 3. Test the change on `test-tag-color` — confirm the schema has the `color` attribute and behaves as you expect, for example by creating a Tag with a color on this branch. There is no proposed change here; you work on the branch directly and never merge it. This is an **iterative loop**: refine the schema on your Git feature branch, push, run **Import latest commit** on `test-tag-color`, and re-check. Repeat until the feature is ready — delete the branch when you are done. Faster iteration with infrahubctl While developing, you can load a schema file straight onto the branch with [`infrahubctl`](/infrahubctl/infrahubctl.md) instead of importing it from Git — no commit or push required: ``` infrahubctl schema load schemas/tags.yml --branch test-tag-color ``` This shortens the inner loop. The Git import is what makes the change reproducible and promotable, so land the final version through Git as below. ## Step 3: Land the change on `main`[​](#step-3-land-the-change-on-main "Direct link to step-3-land-the-change-on-main") 1. Merge the Git pull request into `main`: ``` gh pr create --base main --head feature/add-tag-color --title "Add color attribute to Tag" --fill gh pr merge --merge ``` 2. On the instance, create an Infrahub branch, set its `ref` to `main` (which now holds the change), and open a proposed change from it to `main`. The `ref` change re-imports on its own. ``` # against the Infrahub branch, e.g. https:///graphql/land-tag-color mutation { CoreReadOnlyRepositoryUpdate(data: { id: "" ref: { value: "main" } }) { ok } } ``` 3. Review the proposed change and merge it. The instance's `main` now has the `color` attribute. Do not import onto `main` directly You can import onto `main` — run **Import latest commit** or set its `ref` there — to skip the Infrahub branch and proposed change. It is faster but not recommended: an import can fail, especially while iterating on a schema, and there is no clean way to undo a bad import on `main`, whereas an Infrahub branch can be discarded. ## Step 4: Verify[​](#step-4-verify "Direct link to Step 4: Verify") Create a Tag with a color to confirm the attribute is on `main`: ``` # against https:///graphql/main mutation { BuiltinTagCreate(data: { name: { value: "ready" } color: { value: "#2ecc71" } }) { ok object { id color { value } } } } ``` ## Related[​](#related "Direct link to Related") * [Promote changes between environments](/git-integration/promote-between-environments.md) — repeat this workflow across dev / staging / production * [Connect a repository](/git-integration/connect-repository.md) — repository registration * [Schema extensions](/schema/extensions.md) — extending existing nodes like Tag * [Proposed changes](/proposed-changes/overview.md) — the review workflow used to merge into `main` --- # Understanding the `.infrahub.yml` configuration file The `.infrahub.yml` file serves as the central manifest that defines how Infrahub integrates with external Git repositories. This topic explains the role of this configuration file and the design philosophy behind its structure. ## Why `.infrahub.yml` exists[​](#why-infrahubyml-exists "Direct link to why-infrahubyml-exists") Infrastructure automation requires both structured data (device inventories, network topologies) and executable code (templates, validation scripts, Generators). The `.infrahub.yml` file solves the fundamental challenge of bridging these two worlds by: * **Declaring intent**: Explicitly stating what resources from a Git repository should be imported into Infrahub * **Enabling selective integration**: Allowing fine-grained control over which files and resources are processed * **Providing metadata**: Supplying the necessary context for Infrahub to properly interpret and use repository contents ## Design philosophy and structure[​](#design-philosophy-and-structure "Direct link to Design philosophy and structure") The `.infrahub.yml` file follows a declarative approach where you specify *what* should happen rather than *how* it should happen. This design choice reflects several key principles: ### Explicit over implicit[​](#explicit-over-implicit "Direct link to Explicit over implicit") Rather than scanning the entire repository and making assumptions about file purposes, the configuration file requires explicit declaration of all resources. This approach: * Prevents unexpected imports of unintended files * Makes the integration surface clear and auditable * Allows for different file organizations across repositories ### Resource types[​](#resource-types "Direct link to Resource types") The configuration organizes resources into distinct categories based on their purpose and processing requirements: * **Executable code**: Python transformations, Generators, and checks * **Templates**: Jinja2 templates for configuration generation * **Schema definitions**: Data models for infrastructure representation * **Data definitions**: Object files for storing infrastructure data * **Query definitions**: GraphQL queries for data retrieval * **Fragment definitions**: Reusable [GraphQL fragments](/development-resources/graphql-fragments.md) for composing queries * **Menu definitions**: UI navigation structure definitions Each category has its own processing pipeline and validation requirements, reflecting the different ways Infrahub handles these resource types. ### Metadata richness[​](#metadata-richness "Direct link to Metadata richness") Beyond basic file paths, the configuration captures essential metadata about how resources should be used: * **Dependencies**: Linking Transformations to their required GraphQL queries * **Targeting**: Specifying which infrastructure groups a resource applies to * **Parameters**: Defining the inputs required for executable resources * **Naming**: Providing human-readable identifiers for UI and API access ## Loading behavior and dependency management[​](#loading-behavior-and-dependency-management "Direct link to Loading behavior and dependency management") The `.infrahub.yml` file controls not only *what* gets loaded, but also *how* and *when* resources are processed: ### Processing order[​](#processing-order "Direct link to Processing order") Infrahub processes configuration sections in a specific order to handle dependencies: 1. **Schemas**: Loaded first to establish the data model 2. **GraphQL queries**: Loaded before resources that depend on them 3. **Objects**: Loaded to populate initial data 4. **Python files**: Check definitions, Python transformations, and Generators are loaded together 5. **Jinja2 transformations**: Loaded after their required queries are available 6. **Artifact definitions**: Loaded last to reference existing transformations ### File vs. directory handling[​](#file-vs-directory-handling "Direct link to File vs. directory handling") When you specify a directory in any resource list: * Infrahub processes all relevant files within that directory * Files are processed in alphabetical order * Subdirectories are processed recursively This behavior enables you to organize complex repositories while maintaining predictable loading order. ### Dependency resolution[​](#dependency-resolution "Direct link to Dependency resolution") The configuration enables Infrahub to resolve dependencies between resources: * Transformations reference their required GraphQL queries by name * Artifact definitions reference Transformations and target groups * Generators specify their data dependencies through query requirements This dependency management ensures that all resources have what they need when they execute. ### Declaring extra dependencies with `watch`[​](#declaring-extra-dependencies-with-watch "Direct link to declaring-extra-dependencies-with-watch") Infrahub automatically detects the files a Transformation or Generator reads: a Python Transformation's or a Generator's own source file, and the templates a Jinja2 Transformation statically includes, imports, or extends. When a file in that detected set changes in a proposed change, only the artifacts of the affected Transformation regenerate, or the instances of the affected Generator re-run. A file that merely sits in the same directory as a Python source is not a detected dependency of it. Some dependencies cannot be detected automatically, such as a template pulled in through a dynamic `{% include some_variable %}`, or any module a Python Transformation or Generator imports - Infrahub does not analyze imports. Declare these with the optional `watch` key on a `jinja2_transforms`, `python_transforms`, or `generator_definitions` entry. Watched files are added to the definition's auto-detected dependencies; they never replace them. `watch` is a strict object. Today it accepts a single key, `files`: * `files` is a list of file or directory paths, relative to the repository root. * A directory entry matches **recursively**: every tracked file beneath it joins the dependency set. * Entries may be written with or without a trailing slash. * Files ignored by Git, `.pyc` files, `__pycache__/` directories, and symlinks are never included. A list at the `watch` key (`watch: [a, b]`), a bare string, or any unknown key is rejected when the repository is imported. Future dependency-related keys (for example `strict` or `exclude`) will live under `watch` as additional keys alongside `files`. Jinja2 example, where a template dynamically includes partials the parser cannot follow: ``` jinja2_transforms: - name: DeviceConfig query: device_config_query template_path: templates/device_config.j2 watch: files: - templates/partials/ ``` Python example, where a Transformation imports helpers from a sibling top-level package and from a module next to it: ``` python_transforms: - name: DeviceNameAttribute class_name: DeviceNameAttribute file_path: transforms/device_name_attribute.py watch: files: - utils/ - shared/helpers.py - transforms/device_name_helpers.py ``` Generator example, where a Generator imports helpers from a sibling top-level package. The `watch` key behaves identically on a `generator_definitions` entry: ``` generator_definitions: - name: device_tags file_path: generators/device_tags.py class_name: DeviceTags query: device_tags_query targets: device_group watch: files: - shared/ - common/constants.py ``` Declaring any `watch.files` entry also marks the definition's dependency closure as complete, which suppresses the conservative regenerate-on-any-change fallback that an incomplete Jinja2 auto-detection would otherwise trigger. On a Python Transformation or Generator, where auto-detection has no references to fail on, the declaration earns its precision the way described in the next section instead. See [the proposed changes overview](/proposed-changes/overview.md#understanding-artifact-regeneration) for what an incomplete closure means and how to resolve it. #### Declaring `watch` with no extra files[​](#declaring-watch-with-no-extra-files "Direct link to declaring-watch-with-no-extra-files") On a Python Transformation or a Generator, declare `watch` with an empty list when the source depends on nothing but its own file: ``` python_transforms: - name: DeviceNameAttribute class_name: DeviceNameAttribute file_path: transforms/device_name_attribute.py watch: files: [] ``` Because imports are never analyzed, Infrahub cannot distinguish a Python source with no further dependencies from one whose dependencies it has no way to see. Until `watch` is present it assumes the latter, and ties the definition's fingerprint to the repository's current commit, so every commit is treated as a possible change to it. The empty list is your statement that the auto-detected closure already names everything, which is what unties the fingerprint from the commit and makes regeneration precise. The key has to carry the `files` list: a bare `watch:` with nothing under it reads as no declaration at all. A Jinja2 Transformation needs no such declaration. Its detection parses the template and follows every reference it declares, and any reference it cannot follow already marks the closure incomplete, so a complete Jinja2 closure is trusted on its own. ## Group targeting[​](#group-targeting "Direct link to Group targeting") Many definitions in `.infrahub.yml` use [groups](/groups/overview.md) to specify their targets: * **Artifact definitions** - The `targets` field references a group name to specify which objects should generate artifacts * **Generator definitions** - The `targets` field specifies the group for generated objects * **Check definitions** - The `targets` field indicates which group to validate Groups provide a flexible way to target operations across multiple objects without hardcoding specific object lists. This design decouples resource definitions from infrastructure changes - you can add or remove objects from groups without modifying the configuration files. Example targeting in `.infrahub.yml`: ``` artifact_definitions: - name: "Router Configuration" targets: "ProductionRouters" # References a group transformation: "router_config_transform" check_definitions: - name: "Interface Validation" targets: "NetworkDevices" # References a different group class_name: "InterfaceCheck" ``` See [organizing objects with groups](/groups/overview.md) for details on creating target groups. ## Version control and branch behavior[​](#version-control-and-branch-behavior "Direct link to Version control and branch behavior") The `.infrahub.yml` file itself is version-controlled, enabling sophisticated integration patterns: ### Branch-specific configuration[​](#branch-specific-configuration "Direct link to Branch-specific configuration") Different Git branches can have different `.infrahub.yml` files: * Development branches can include experimental resources * Production branches can exclude debug or testing resources * Feature branches can temporarily modify resource definitions This flexibility allows teams to maintain different configurations for different environments or development stages. ### Evolution and migration[​](#evolution-and-migration "Direct link to Evolution and migration") As your infrastructure automation evolves, the configuration file supports: * Adding new resource types without affecting existing ones * Deprecating old resources by removing them from the configuration * Refactoring resource organization while maintaining functionality This approach helps teams manage the lifecycle of infrastructure automation components with minimal disruption. ## Future evolution and extensibility[​](#future-evolution-and-extensibility "Direct link to Future evolution and extensibility") The `.infrahub.yml` format is designed to evolve with Infrahub's capabilities: * New resource types can be added without breaking existing configurations * Additional metadata fields can be introduced with backward compatibility * Processing behavior can be enhanced while maintaining existing semantics This future-proof design ensures that your automation investments remain valuable as Infrahub grows. ## Conclusion[​](#conclusion "Direct link to Conclusion") The `.infrahub.yml` configuration file is a crucial bridge between Git repositories and Infrahub's infrastructure management capabilities. By understanding its design principles and structure, you can effectively organize your infrastructure automation resources and leverage Infrahub's full potential to manage your network infrastructure. ## Further reading[​](#further-reading "Direct link to Further reading") * [Repository configuration file reference](/reference/dotinfrahub.md) - Complete syntax and options reference * [How to connect external Git repositories](/git-integration/connect-repository.md) - Step-by-step setup guide * [Understanding Git repositories in Infrahub](/git-integration/overview.md) - Repository integration concepts * [Understanding groups](/groups/overview.md) - Organizational concepts for targeting operations * [How to organize objects with groups](/groups/overview.md) - Creating and managing groups * [Schema development](/schema/overview.md) - Designing infrastructure data models * [Transformations](/transformations/overview.md) - Data processing and template concepts * [Generators](/generators/overview.md) - Infrastructure provisioning automation * [Artifacts](/artifacts/overview.md) - Output generation and delivery --- # Multiple environments from a single repository Run a separate Infrahub instance for each environment — development, staging, production — from one Git repository. Each instance reads its schemas, objects, and automation from a [read-only repository](/git-integration/overview.md#read-only-vs-core) pinned to one long-lived branch. To move a change from one environment to the next, merge it into the next branch on your Git host, then trigger an import on the target instance. Use this pattern when you want environment isolation with a Git-native promotion path: each environment is independent, every promotion is a pull request on your Git host, and a change is reviewed on the Git side and again inside Infrahub before it reaches production. ## What moves through Git[​](#what-moves-through-git "Direct link to What moves through Git") A read-only repository imports what you declare in [`.infrahub.yml`](/git-integration/infrahub-yml.md): * Schemas * Objects defined in object files * GraphQL queries * Jinja2 and Python Transformations * Artifact definitions * Generators * Checks * Menus These are the changes you promote between environments. Data created directly in an instance through the UI or API — the objects your users edit day to day — is not part of this. It lives in each instance and never moves through Git. ## Git and Infrahub terminology[​](#git-and-infrahub-terminology "Direct link to Git and Infrahub terminology") A promotion touches two systems that share vocabulary — both Git and Infrahub have branches, a `main`, and a merge step. The terms below keep them apart: * **Git branch**, **Git pull request** — on your Git host. The long-lived per-environment branches (`develop`, `staging`, `main`) are Git branches; moving a change from one to the next is a pull request. * **Infrahub branch**, **proposed change** — inside a single instance. You import into an Infrahub branch, review it as a proposed change, and merge it into that instance's `main`. Production's Git branch is also named `main`, but it is unrelated to any instance's internal `main`. ## The model[​](#the-model "Direct link to The model") One repository holds one long-lived Git branch per environment. Each branch maps to its own instance through that instance's read-only repository `ref`: | Git branch | Infrahub instance | Read-only repository `ref` | | ---------- | ----------------- | -------------------------- | | `develop` | Development | `develop` | | `staging` | Staging | `staging` | | `main` | Production | `main` | ![Multi-environment topology: one Git repository with a branch per environment, each feeding its own instance through a read-only ref, with changes authored on a feature branch and promoted along the chain](/assets/images/multi-environment-topology.excalidraw-6eff7a0737fdd94ec92cebca4bc3e026.svg) This example uses three environments, but the count is arbitrary — two (`develop` → `main`) or five work the same way, and the steps for each promotion hop are identical. Add or remove an environment by adding or removing a Git branch and the instance that tracks it. Every environment is a **consumer**: its files come from a read-only repository, and no instance holds a read-write [`Repository`](/git-integration/overview.md#read-only-vs-core), so nothing is ever pushed back to Git. You author changes the plain Git way — edit files locally and open a pull request into an environment's branch. Use a read-only repository, not a read-write one Read-write `Repository` connections are not recommended for multi-environment setups: they push merges back to Git and track every branch, which breaks the per-environment isolation this pattern depends on. If you currently connect the repository as a read-write `Repository` on a single instance and want to adopt this pattern, connect it as a read-only repository instead. A read-only repository does not poll the remote for new commits — there is no background sync, unlike a read-write repository. It imports only in response to an explicit action: running the **Import latest commit** action, or changing its `ref` (which dispatches an import on its own). Either way, an import is a deliberate step, not an automatic sync of every new commit on the branch. ## Promoting a change[​](#promoting-a-change "Direct link to Promoting a change") Each instance's `main` tracks its environment branch through the read-only repository's `ref` — the branch name you set. (Infrahub records which commit it imported internally; that is not something you set.) You never import onto `main` directly. Land a change through an Infrahub branch: create the branch, import the change onto it by pointing `ref` at the branch that holds it, review the result as a [proposed change](/proposed-changes/overview.md), and merge — which advances `main`. Because `ref` is branch-aware, working on the Infrahub branch never disturbs `main`. Authoring and promotion differ only in where the change comes from: a Git feature branch you are working on (authoring), or the environment branch after the previous environment merged into it (promotion). In both, `main`'s `ref` ends up on the environment branch. The [guide](/git-integration/promote-between-environments.md) covers both, including how `ref` moves to a feature branch during authoring and returns to the environment branch before the merge. Do not import onto `main` directly Pointing `main`'s `ref` at a branch and importing there skips the Infrahub branch and its proposed change. It is faster but not recommended: an import can fail — especially while a schema is being developed and iterated — and there is no clean way to undo a bad import on `main`, whereas an Infrahub branch can be discarded. ## Known limitations[​](#known-limitations "Direct link to Known limitations") * **Schema retirement is not automatic.** Removing a field or node from the schema files does not remove it from an instance — set `state: absent` on those nodes to retire them (the same as with `infrahubctl schema load`). Deleting the lines is not enough; the removal must be stated explicitly. * **Clearing all object files at once orphans their objects.** Objects imported from an object file are pruned when you remove the file — deleted on the next import. The exception is removing the *last* object file, or clearing the whole `objects:` list, in a single import: with no objects left to track, the removal is not detected and the objects stay, needing manual cleanup. Remove object files one at a time, keeping at least one until the end; an empty file or `data: []` does not force the deletion. * **A proposed change carries the repository's `ref` and `commit`.** Both are branch-aware, so merging an Infrahub branch that changed them updates `main` as well. Point the `ref` back to the environment branch before merging, so `main` keeps tracking it — the [guide](/git-integration/promote-between-environments.md) does this. * **No clean rollback.** A promotion applies forward; reverting the `ref` restores the file-defined content, not the instance state it produced. Recover a bad production promotion from a database backup — see [reaching production safely](#reaching-production-safely) below. * **Data stays per-instance.** Only file-defined content promotes; objects created in the UI or API are not carried between environments. ## Reaching production safely[​](#reaching-production-safely "Direct link to Reaching production safely") Lower environments make promotion safer, but they are not proof. Staging drifts from production over time — different data, a different history of applied changes — so a change that imported cleanly on staging can still behave differently against production's actual state. Importing into an Infrahub branch and merging through a proposed change removes most of that risk: you review the exact diff before it touches production `main`. But the merge of that proposed change can itself fail partway and leave `main` in an unexpected state, and there is no clean rollback — an import applies forward, and reverting the `ref` restores the files, not the instance state the import produced. To be certain a promotion applies cleanly, test it against production's real state first. Provision a pre-production instance on demand from a current production [database backup](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md), run the promotion there, and confirm it applies cleanly before promoting production itself. Because it starts from a fresh backup, it matches production exactly, with none of the drift a long-lived staging environment accumulates. This rehearsal can be added as a stage in the pipeline that merges the promotion's proposed change, so production is touched only after the same change has applied cleanly on a copy of it. ## Further reading[​](#further-reading "Direct link to Further reading") * [Develop changes from a Git repository](/git-integration/develop-changes.md) — the single-instance workflow promotion builds on * [Promote changes between environments](/git-integration/promote-between-environments.md) — the cross-environment how-to * [Git integration](/git-integration/overview.md) — repository types, `ref` tracking, and the import model * [Proposed changes](/proposed-changes/overview.md) — the review and merge workflow used inside each instance * [Database backup and restore](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md) — spinning up a pre-production copy of production --- # Git Integration Connect an external Git repository to keep the code that drives Infrahub — schemas, Transformations, Generators, checks, GraphQL queries, and artifact definitions — under version control, next to your infrastructure data. Infrahub imports that code and keeps it in sync as the repository changes. To set one up, see [connect a repository](/git-integration/connect-repository.md), [configure `.infrahub.yml`](/git-integration/infrahub-yml.md), and [filter which branches sync](/git-integration/branch-synchronization.md). ## Two version-control planes[​](#two-version-control-planes "Direct link to Two version-control planes") Infrahub has two separate version-control systems, and understanding the split is the key to the rest of this section. * **The graph** — your infrastructure data (objects) and schema. Infrahub versions the graph itself: you branch it, change it, review the diff in a [Proposed Change](/proposed-changes/overview.md), and merge. This is Git-like version control, but the store is Infrahub's database, not a Git repository. * **Git** — the code that describes intent: schemas, Transformations, Generators, checks, GraphQL queries, and artifact definitions, declared in [`.infrahub.yml`](/git-integration/infrahub-yml.md). Infrahub reads this code from the repository and imports it. The link between the two runs one way: from Git into Infrahub. Git is where you author code intent; Infrahub imports it and applies it to the graph. Data moves the other way only inside Infrahub — when you merge a branch, the change is applied to the graph, not pushed back to Git. A read-write repository does write one thing back to Git, and only one; the [repository types](#read-only-vs-core) section covers it. ## Why Git integration matters[​](#why-git-integration-matters "Direct link to Why Git integration matters") Modern infrastructure management requires both structured data (network configurations, device inventories) and code (automation scripts, templates, validation logic). By integrating with Git repositories, Infrahub bridges these two worlds, enabling: * **Unified version control**: Both your infrastructure data and automation code follow the same branching and merging workflows * **Familiar developer workflows**: Teams can leverage existing Git practices and tooling * **Separation of concerns**: Infrastructure data resides in Infrahub's graph database while code remains in Git * **Enhanced collaboration**: Multiple teams can work on different aspects of infrastructure without conflicts ## Repository types and their design rationale[​](#read-only-vs-core "Direct link to Repository types and their design rationale") Infrahub supports two distinct repository connection types, each designed for specific use cases and operational requirements. ### Repository: read-write integration[​](#repository-read-write-integration "Direct link to Repository: read-write integration") The standard **Repository** type gives Infrahub read-write access to Git. Infrahub imports code from every branch automatically, and on a Proposed Change merge it updates the linked Git branch — for branches that sync with Git (see [Controlling whether a branch reaches Git](#controlling-whether-a-branch-reaches-git)). **Design principles:** * **Branch parity**: each branch in the Git repository has a corresponding branch in Infrahub * **Automatic import**: Infrahub imports changes from Git in the background, without a manual trigger * **One write-back on merge**: merging a Proposed Change updates the linked branch in Git (see [merging branches](#merging-branches) below); this moves code intent, never your infrastructure data **How it works internally:** When you create a Repository connection, Infrahub: 1. Clones the repository to local storage 2. Creates Git worktrees for each branch to enable parallel operations 3. Tracks all remote branches and creates corresponding Infrahub branches 4. Runs background synchronization tasks every few seconds to detect changes 5. Parses the `.infrahub.yml` file to determine which resources to import When you merge a Proposed Change and the source branch is set to sync with Git, Infrahub merges the corresponding branch in the Git repository and pushes the result to the remote. By default this is a fast-forward when possible, so no merge commit is created (see [merging branches](#merging-branches)). Only the code tracked in Git moves; your infrastructure data stays in the graph. A read-write repository has a `default_branch` attribute (defaulting to `main`) that names the Git branch mapped to Infrahub's default branch. Merges into Infrahub's default branch are pushed back to that Git branch on the remote, and that Git branch is imported as Infrahub's default branch, so a repository can use a non-`main` Git default branch. When `default_branch` is not `main`, a Git branch literally named `main` is not imported. ### Read-only Repository: controlled unidirectional flow[​](#read-only-repository-controlled-unidirectional-flow "Direct link to Read-only Repository: controlled unidirectional flow") The **Read-only Repository** type offers a simpler, unidirectional integration designed for scenarios where you need to consume resources from Git without modifying the external repository. **Design principles:** * **Single reference tracking**: links one specific Git ref (branch, tag, or commit) * **Manual synchronization**: Infrahub does not poll a read-only repository; you update it by changing the `ref` or using the **Import latest commit** action * **Protection of external resources**: guarantees no modifications to the external repository **How it works internally:** Read-only repositories: 1. Track a single Git reference rather than all branches 2. Import changes only when you change the `ref` or run **Import latest commit** — Infrahub does not poll read-only repositories 3. Import resources according to the `.infrahub.yml` configuration 4. Maintain a simpler state model without complex branch synchronization logic ## Architecture and data flow[​](#architecture-and-data-flow "Direct link to Architecture and data flow") Understanding how Infrahub manages Git repositories helps clarify the system's capabilities and constraints. ### Component responsibilities[​](#component-responsibilities "Direct link to Component responsibilities") ![Repository Architecture](/assets/images/repository_architecture.excalidraw-50f688c5b25266dc11ff98e2076593e8.svg) The architecture carefully separates concerns between components: * **Infrahub API Server**: Never directly connects to Git repositories; handles GraphQL mutations and queries * **Task Workers**: Manage all Git operations including cloning, fetching, and pushing * **Message Broker**: Coordinates communication between API server and task workers * **File System Storage**: Maintains local clones and worktrees in `git.repositories_directory` This separation ensures that Git operations don't block API requests and enables horizontal scaling of task workers for improved performance. ### Synchronization mechanisms[​](#synchronization-mechanisms "Direct link to Synchronization mechanisms") **For standard Repositories:** * Background tasks poll for changes multiple times per minute * Each sync operation fetches from origin and compares local vs. remote state * New branches automatically trigger Infrahub branch creation * Updated branches trigger re-import of changed resources **For Read-only Repositories:** * No automatic synchronization occurs * Updates can be triggered by modifying the `ref` property to point to a different branch, tag, or commit * Alternatively, use the **Import latest commit** action to fetch the latest commit for the current ref from the remote without changing the ref * Each update fetches the specified commit and imports resources according to configuration ### Merging branches[​](#merging-branches "Direct link to Merging branches") **This applies only to standard repositories:** By default, Infrahub runs a standard `git merge`, which fast-forwards when the history allows it. When merging a branch linked to Git, Infrahub moves the destination branch pointer forward without creating a merge commit; if the branches have diverged, Git falls back to a merge commit. You can configure Infrahub to always create a merge commit by setting the `INFRAHUB_GIT_USE_EXPLICIT_MERGE_COMMIT` environment variable to `true`. This approach maintains a more explicit commit history and improves auditability. [Enabling explicit merge commits/git-integration/connect-repository#customizing-git-merge-behavior](/git-integration/connect-repository.md#customizing-git-merge-behavior) ### Controlling whether a branch reaches Git[​](#controlling-whether-a-branch-reaches-git "Direct link to Controlling whether a branch reaches Git") Each Infrahub branch has a `sync_with_git` property, chosen when you create the branch. It controls whether a branch **created in Infrahub** is extended to Git: when it is on, Infrahub creates a matching branch in every read-write repository and, on a Proposed Change merge, merges that branch in Git. When it is off, the branch stays inside Infrahub's graph. The name reads as if it governs importing Git branches into Infrahub — it does not. Import always runs from Git to Infrahub regardless of this property; `sync_with_git` only affects branches going the other way, from Infrahub out to Git. Many data-only changes never need a Git branch at all, since that data lives only in the graph. ## The `.infrahub.yml` configuration file[​](#infrahub-yaml "Direct link to infrahub-yaml") The `.infrahub.yml` file acts as a manifest, declaring which resources Infrahub should import from the repository. This declarative approach enables: * Selective importing of repository contents based on specific needs * Clear boundaries between Infrahub-managed and external resources * Version-specific configurations (different branches can have different `.infrahub.yml` files) For detailed information about `.infrahub.yml` configuration and syntax, see [Infrahub configuration file](/git-integration/infrahub-yml.md). ## Status tracking and health monitoring[​](#repository-statuses "Direct link to Status tracking and health monitoring") Repositories track their operational state through three distinct status dimensions, each serving a different monitoring purpose. ### Admin status: lifecycle management[​](#admin-status-lifecycle-management "Direct link to Admin status: lifecycle management") Tracks whether Infrahub is actively using the repository: * **Active**: Repository is in production use * **Inactive**: Repository is disabled or unused * **Staging**: Repository is pending activation via Proposed Change ### Operational status: connectivity health[​](#operational-status-connectivity-health "Direct link to Operational status: connectivity health") Monitors the connection between Infrahub and the Git server: * **Online**: Successfully connected and authenticated * **Credential Error**: Authentication failure * **Connectivity Error**: Network or connectivity issue * **Error**: Other initialization or configuration problems * **Unknown**: Status cannot be determined ### Sync status: data consistency[​](#sync-status-data-consistency "Direct link to Sync status: data consistency") Indicates the state of data synchronization: * **In sync**: All repository resources are current in Infrahub * **Syncing**: Synchronization operation in progress * **Error (import)**: Failed to import resources from repository * **Unknown**: Synchronization state cannot be determined ## Repository actions and operations[​](#repository-actions-and-operations "Direct link to Repository actions and operations") Beyond basic synchronization, repositories support administrative actions for troubleshooting and maintenance: * **Check connectivity**: Validates network access and authentication without modifying repository content. The result is recorded in the repository's operational status: a successful check sets it to **Online**, while a failure sets it to the matching error state (**Credential Error**, **Connectivity Error**, or **Error**) * **Reimport current commit**: Forces re-processing of the current commit without fetching new changes, useful for recovering from import errors * **Import latest commit** (read-only repositories only): Fetches the latest commit from the remote for the tracked ref and imports its resources. This is useful when the remote repository has new changes and you want to pull them without manually updating the `ref` property. Unlike changing the `ref`, this operation keeps the same ref but updates to whatever commit that ref currently points to on the remote ## Design trade-offs and constraints[​](#design-trade-offs-and-constraints "Direct link to Design trade-offs and constraints") The current architecture makes several important trade-offs: ### Branch hierarchy limitations[​](#branch-hierarchy-limitations "Direct link to Branch hierarchy limitations") Infrahub currently supports only single-level branch hierarchy (branches created from main). This simplification: * Reduces complexity in conflict resolution * Ensures predictable merge paths * May limit advanced Git workflows ### Synchronization frequency[​](#synchronization-frequency "Direct link to Synchronization frequency") The polling-based synchronization approach: * Provides near-real-time updates (within seconds) * Avoids webhook complexity and firewall issues * May introduce slight delays compared to push-based systems ## Common misconceptions[​](#common-misconceptions "Direct link to Common misconceptions") * **Merging a change in Infrahub pushes data to Git.** It does not. A data merge is applied to the graph. The only thing a merge writes to Git is the code-intent branch of a read-write repository, updated on a Proposed Change merge. * **`sync_with_git` controls importing Git branches into Infrahub.** It does not. It controls whether a branch created in Infrahub is extended out to Git. Import always runs from Git to Infrahub. * **A read-write repository has an "Import latest commit" button.** It does not. Read-write repositories import automatically in the background; **Import latest commit** is a read-only repository action, used because read-only repositories are not polled. ## Further reading[​](#further-reading "Direct link to Further reading") * [How to connect external Git repositories](/git-integration/connect-repository.md) - Practical guide for setting up repositories * [Infrahub configuration file](/git-integration/infrahub-yml.md) - Detailed `.infrahub.yml` reference * [Selective branch synchronization](/git-integration/branch-synchronization.md) - Controlling which Git branches sync to Infrahub * [Proposed Changes](/proposed-changes/overview.md) - Understanding the merge and review process * [Version control in Infrahub](/immutable-history/overview.md) - Broader context of Infrahub's version control features --- # Promote changes between environments Run a separate Infrahub instance for each environment from one Git repository, and move changes from one environment to the next. This guide covers the **promotion** — taking a change that is already developed on one environment and landing it on the next. To develop and land a change on a single instance, see [develop changes from a Git repository](/git-integration/develop-changes.md); for the concepts, see [multiple environments from a single repository](/git-integration/multi-environment.md). The example uses two environments — **development** (imports the `develop` branch) and **production** (imports the `main` branch) — but the steps repeat for any number. ![The develop-and-promote flow: author and test a change on development, then promote it to production, landing on each instance through a proposed change](/assets/images/develop-promote-flow.excalidraw-b06e790c62f08f5c836b831df37be6fd.svg) ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * One Infrahub instance per environment, each with the repository connected as a **read-only repository** pinned to that environment's Git branch — `develop` on development, `main` on production (see [connect a repository](/git-integration/connect-repository.md)). * A change already developed and merged onto the development instance's `main` (see [develop changes from a Git repository](/git-integration/develop-changes.md)). ## Register the repository[​](#register-the-repository "Direct link to Register the repository") Connect the repository on each instance as a read-only repository, setting `ref` to that environment's Git branch. ``` mutation { CoreReadOnlyRepositoryCreate(data: { name: { value: "infra" } location: { value: "https://github.com//.git" } ref: { value: "develop" } # "main" on the production instance credential: { id: "" } }) { ok } } ``` Each instance's `main` keeps its `ref` on its environment branch from here on; promotion advances the imported commit, never the `ref`. ## Promote a change[​](#promote-a-change "Direct link to Promote a change") A promotion is a Git merge into the next environment's branch, then an import on that environment's instance. The instance-side steps are the same ones you use to [develop a change](/git-integration/develop-changes.md) — the change comes from the upstream branch instead of a feature branch. 1. On your Git host, open a pull request from the source branch to the next (for example `develop` → `main`) and merge it. The target branch's head now holds the change. 2. On the target instance, land the merged change through an Infrahub branch, exactly as in [Develop changes, Step 3](/git-integration/develop-changes.md#step-3-land-the-change-on-main): create a branch, run **Import latest commit** (its `ref` is already the target branch), open a proposed change to `main`, review, and merge. To catch problems before the Git pull request merges, validate first: import the source branch onto a throwaway Infrahub branch (set its `ref` to the source branch) and check it there — see [Develop changes, Step 2](/git-integration/develop-changes.md#step-2-test-it-on-an-infrahub-branch). Repeat the hop for each environment in the chain (for example `develop` → `staging` → `main`). Rehearse a production promotion first Lower environments drift from production over time, so a clean import there does not prove the change applies cleanly to production — and a proposed-change merge that fails partway leaves `main` in an unexpected state, with no clean rollback. Before promoting production, rehearse the promotion against a pre-production instance restored from a current [database backup](/deploy-manage/maintain-upgrade/database-backup/backup-and-restore.md). See [reaching production safely](/git-integration/multi-environment.md#reaching-production-safely). ## Automate the promotion (optional)[​](#automate-the-promotion-optional "Direct link to Automate the promotion (optional)") Experimental — a starting point, not a finished solution The manual steps above are the supported path. Automating the Infrahub side is possible, but treat what follows as inspiration to adapt, not a finished pipeline, and keep production promotions manual until you have validated your own automation end to end. An automated gate follows the same pattern: on a Git pull request into an environment branch, CI creates an Infrahub branch, imports the change onto it, opens a proposed change, and blocks the pull request — as a **required status check** — until the proposed change's checks pass. Checking those from CI uses the proposed change's `validations`, where each reports a `state` (`queued`, `in_progress`, `completed`) and a `conclusion` (`unknown`, `failure`, `success`): ``` query { CoreProposedChange(ids: [""]) { edges { node { validations { edges { node { display_label state { value } conclusion { value } } } } } } } } ``` Poll until every validation's `state` is `completed`, then fail the CI job unless every `conclusion` is `success`. Store each instance's address and an [API token](/deploy-manage/user-management/managing-api-tokens.md) as CI secrets, and authenticate with the `X-INFRAHUB-KEY` header. ## Related[​](#related "Direct link to Related") * [Develop changes from a Git repository](/git-integration/develop-changes.md) — the single-instance workflow this builds on * [Multiple environments from a single repository](/git-integration/multi-environment.md) — the concepts behind this pattern * [Connect a repository](/git-integration/connect-repository.md) — repository registration in depth * [Proposed changes](/proposed-changes/overview.md) — the review workflow used inside each instance --- # Analyze dependencies Use Dependency mode to find every object of a given kind that is reachable from a source object. Use it to discover what depends on a resource, assess blast radius, or map impact before a change or outage. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * You need view permission on the object kinds you want to find. Kinds you cannot view are excluded from results automatically. ## Find dependencies[​](#find-dependencies "Direct link to Find dependencies") 1. From the left navigation, open **Object Management** and select **Path Traversal**. 2. Select the **Dependencies** tab. 3. In **Source Object**, search for and select the starting object. 4. In **Target kinds**, select one or more kinds of object to find — for example, `InfraService` or `InfraDevice`. 5. Optionally, expand **Search options** to adjust: * **Max Depth** — maximum hops to traverse (default: 5, max: 30). * **Max Targets** — maximum distinct target objects to return (default: 50, max: 200). * **Max Paths** — maximum total paths across all targets (default: 500, max: 5000). * **Shortest paths only** — when selected (default), returns only the shortest path to each target. Clear it to return every path within the max depth. 6. Select **Find Dependencies**. ## Read the results[​](#read-the-results "Direct link to Read the results") ![Dependency mode showing every interface with an attached link reachable from a device](/assets/images/dependencies-e54de64918301fa1afc721d1e3ffea15.png) The results list every reachable object of the chosen kinds, each with its depth and the path connecting it back to the source. Objects closer to the source appear first. Select any result to highlight its path in the graph. Right-click a node in the graph to open its detail page, set it as the new source, copy its ID, or exclude its kind from the current view. Use the toolbar at the bottom to zoom, change the layout or edge style, reload, or export the graph. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") **No results** — the source object may have no connections to the chosen kinds within the current max depth. Try increasing Max Depth, or verify the target kinds are connected to the source in your schema. **Query timed out** — reduce Max Depth or Max Targets, or narrow to fewer target kinds. **Select at least one target kind** — Dependency mode requires at least one target kind. ## Next steps[​](#next-steps "Direct link to Next steps") * [Trace a path](/graph-traversal/trace-a-path.md) * [Query traversal with GraphQL](/graph-traversal/query-with-graphql.md) * [Graph Traversal reference](/reference/graph-traversal.md) --- # Graph Traversal Use Graph Traversal to navigate the relationships in your infrastructure data — trace every path between two objects, or find everything that depends on a given object across your entire graph. Infrahub version Graph Traversal was released in 1.10. ## Two modes[​](#two-modes "Direct link to Two modes") Graph Traversal has two modes, each suited to a different question. **Path mode** — select a source and destination object to see how they connect, ordered shortest first with the hop count for each route. By default it returns the shortest path through each intermediate object; you can switch it to return every path instead. Use it when you know both ends and want to understand how they connect. **Dependency mode** — select a source object and the kinds of object you care about to find every reachable object of those kinds. Use it when you know the starting point but not the endpoints — for blast-radius analysis, impact assessment, or dependency discovery. ## When to use each mode[​](#when-to-use-each-mode "Direct link to When to use each mode") | Goal | Mode | You provide | Result | | ---------------------------------------------- | ---------- | -------------------------------------------- | ---------------------------------------------------------------------------- | | Trace the connection between two known objects | Path | A source and a destination object | The paths between them, with hop counts | | Find what an object affects or depends on | Dependency | A source object and one or more target kinds | Every reachable object of those kinds, with the path from the source to each | Use Path mode when you know both ends — for example, tracing connectivity between two devices while investigating packet loss. Use Dependency mode when you know the starting point but not the endpoints — for example, finding every service that depends on a circuit before a maintenance window. ## Branch and time context[​](#branch-and-time-context "Direct link to Branch and time context") When you run Graph Traversal on a branch, the results reflect that branch's state rather than the default branch. Run it against a point in time to see relationships as they were then. ## Permissions[​](#permissions "Direct link to Permissions") Graph Traversal applies your view permissions. If you cannot view a kind of object, paths through that kind are excluded — so you never see objects outside your access. ## What's included in results[​](#whats-included-in-results "Direct link to What's included in results") By default, your results skip Infrahub's internal data model and stay focused on your infrastructure. Nodes in the `Core`, `Internal`, `Builtin`, `Lineage`, `Profile`, and `Template` namespaces are excluded, along with `BuiltinIPNamespace` and the kinds that inherit it. To widen or narrow this, use the namespace and kind filters in the GraphQL API — see the [Graph Traversal reference](/reference/graph-traversal.md). ## Where you can use it[​](#where-you-can-use-it "Direct link to Where you can use it") | Surface | Access | | ------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | Web interface | **Object Management → Path Traversal**; or **Find paths** in any object's Actions menu | | GraphQL API | `InfrahubPathTraversal` and `InfrahubReachableNodes` — see [Query traversal with GraphQL](/graph-traversal/query-with-graphql.md) | | Python SDK | `traverse_paths`, `reachable_nodes`, and `path_exists` — see the [SDK graph traversal guide](/python-sdk/guides/graph_traversal.md) | | MCP server | Coming soon | Because Graph Traversal is also available through the MCP server, an AI agent can trace the same relationships you rely on, grounded in your real data model. ## Related resources[​](#related-resources "Direct link to Related resources") * [Relationships](/schema/relationships.md) * [Trace a path](/graph-traversal/trace-a-path.md) * [Analyze dependencies](/graph-traversal/analyze-dependencies.md) * [Graph Traversal reference](/reference/graph-traversal.md) --- # Query traversal with GraphQL Both Graph Traversal modes are available programmatically — as GraphQL queries (`InfrahubPathTraversal` for Path mode and `InfrahubReachableNodes` for Dependency mode) and through the [Python SDK](/python-sdk/guides/graph_traversal.md). Use them to integrate traversal into scripts, pipelines, or checks. ## Path mode: find paths between two objects[​](#path-mode-find-paths-between-two-objects "Direct link to Path mode: find paths between two objects") * GraphQL * Python SDK ``` query PathBetweenTwoObjects { InfrahubPathTraversal( data: { source_id: "" destination_id: "" max_depth: 5 } ) { count paths { depth hops { node { kind display_label } relationship { from_label to_label } } } } } ``` ``` from infrahub_sdk import InfrahubClient client = InfrahubClient() result = await client.traverse_paths( source="", destination="", max_depth=5, ) print(result.count) for path in result.paths: for hop in path.hops: print(hop.node.kind, hop.node.display_label) ``` `count` is the number of paths found. Each path contains an ordered list of `hops` from source to destination, each with the node visited and the relationship traversed to reach it. ## Dependency mode: find reachable objects[​](#dependency-mode-find-reachable-objects "Direct link to Dependency mode: find reachable objects") * GraphQL * Python SDK ``` query DependenciesOfAnObject { InfrahubReachableNodes( data: { source_id: "" target_kinds: ["InfraService"] max_depth: 5 } ) { count dependencies { depth node { kind display_label } path { depth hops { node { kind display_label } } } } } } ``` ``` result = await client.reachable_nodes( source="", target_kinds=["InfraService"], max_depth=5, ) print(result.count) for dependency in result.dependencies: print(dependency.depth, dependency.node.display_label) ``` `count` is the number of dependency entries returned — one per reachable node. Each entry includes the `depth` from the source, the reachable `node`, and the full `path` back to the source. The SDK also provides `path_exists(source, destination, ...)` for a quick boolean connectivity check. All three methods work in asynchronous and synchronous form and require Infrahub 1.10 or later — see the [SDK graph traversal guide](/python-sdk/guides/graph_traversal.md). ## Branch and time context[​](#branch-and-time-context "Direct link to Branch and time context") Both queries run against the branch and point in time from your request context. To query a specific branch or timestamp, set the branch header or `at` parameter as you would for any Infrahub GraphQL query. ## Use traversal in a check[​](#use-traversal-in-a-check "Direct link to Use traversal in a check") Because the SDK exposes traversal, an Infrahub [check](/checks/overview.md) can validate structural requirements that span relationships and fail a [proposed change](/proposed-changes/overview.md) when they are not met: * **Connectivity** — assert that two objects stay connected, for example that every site keeps a redundant path to the core, using `path_exists` or `traverse_paths`. * **Isolation** — assert that two objects are *not* connected, for example that a production device and a lab device share no path. * **Dependency analysis** — use `reachable_nodes` to find everything that depends on an object and validate it against a policy. Traversal is branch- and time-aware, so a check evaluates connectivity on the proposed change's branch and catches violations before the change merges. See [Build a check](/learn/tutorials/build-a-check.md) to set one up. ## Full argument reference[​](#full-argument-reference "Direct link to Full argument reference") For all arguments, defaults, limits, and filter semantics, see the [Graph Traversal reference](/reference/graph-traversal.md). --- # Trace a path Use Path mode to find the paths between two objects in your infrastructure graph. By default it returns the shortest path through each intermediate object, ordered shortest first with the hop count for each route. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * You need view permission on the object kinds along the path. Kinds you cannot view are excluded from results automatically. ## Find paths between two objects[​](#find-paths-between-two-objects "Direct link to Find paths between two objects") From the left navigation, open **Object Management** and select **Path Traversal**. ![Path Traversal in the Object Management menu](/assets/images/path-traversal-menu-f7848ea0c008078813d0b52ed60b1104.png) 1. Select the **Path** tab if it is not already active. 2. In **Source Object**, search for and select the starting object. 3. In **Destination Object**, search for and select the ending object. * To swap source and destination, select **⇅ Swap**. 4. Optionally, expand **Search options** to adjust: * **Max Depth** — maximum hops to traverse (default: 5, max: 30). * **Max Paths** — maximum paths to return (default: 10, max: 100). * **Kinds to include** — limit traversal to objects of these kinds only. * **Kinds to exclude** — skip objects of these kinds during traversal. * **Shortest paths only** — when selected (default), returns only the shortest path through each intermediate object. Clear it to return all loopless paths within the max depth. 5. Select **Find Paths**. ## Read the results[​](#read-the-results "Direct link to Read the results") ![Path Traversal showing the highlighted path between two leaf devices across different pods, with the list of paths in the side panel](/assets/images/path-traversal-45f07a7c3acba1a52e929f2310c2afff.png) The sidebar shows the number of paths found. Each path includes its hop count and the sequence of objects from source to destination. If the search runs out of time before exploring the whole graph, the sidebar shows a warning with the depth at which it stopped. Paths shorter than that depth are complete, but longer ones may be missing — narrow the search with **Kinds to include** or **Kinds to exclude** to reach deeper. Select a path to highlight it in the graph. To explore from the graph: * **Right-click any node** to access: * **Open details** — open the object's detail page. * **Set as source** — use this node as the new source. * **Set as destination** — use this node as the new destination. * **Copy ID** — copy the node's UUID to the clipboard. * **Exclude ``** — remove all objects of this kind from the current results. Use the toolbar at the bottom to zoom, change the layout or edge style, reload, or export the graph. ## Start from an object's detail page[​](#start-from-an-objects-detail-page "Direct link to Start from an object's detail page") On any object's detail page, select **Find paths** from the **Actions** menu. Path Traversal opens with that object pre-selected as the source. ![The Find paths entry in an object\'s Actions menu](/assets/images/find-paths-action-c59278bd47ff393004f12b93d82ccc05.png) ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") **No paths found** — the two objects may not be connected within the current max depth. Try increasing Max Depth, or check that neither object kind is excluded. **Search stopped early** — if the search runs out of time, it returns the paths found so far and warns about the depth at which it stopped, rather than failing. Reduce Max Depth or Max Paths, or use **Kinds to include**/**Kinds to exclude** to narrow the search space and reach deeper paths. **Source and destination must be different** — select two distinct objects. ## Next steps[​](#next-steps "Direct link to Next steps") * [Analyze dependencies](/graph-traversal/analyze-dependencies.md) * [Query traversal with GraphQL](/graph-traversal/query-with-graphql.md) * [Graph Traversal reference](/reference/graph-traversal.md) --- # Add members to a group Attach existing objects to a Standard group. GraphQL replaces, it does not append `CoreStandardGroupUpdate` with a `members` list **replaces** the current membership. To append, include the current members in the new list. The SDK and the web UI append by default. note This page applies to **Standard** groups. Generator and Query groups have their membership managed automatically by the Generator or query that owns them. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * An existing Standard group and its `hfid` (or name). * The IDs of the objects you want to add. ## Find the object IDs[​](#find-the-object-ids "Direct link to Find the object IDs") Before appending via GraphQL, look up the IDs of the objects you want to add: ``` query { BuiltinTag(name__values: ["", ""]) { edges { node { id display_label } } } } ``` Replace `BuiltinTag` with the kind of object you're adding. ## Add members[​](#add-members "Direct link to Add members") * Web Interface * GraphQL * Python SDK 1. Open the group from the Groups list. 2. Go to the **Members** tab. 3. Click **Add Members** and select the objects. 4. Click **Save**. **Append** — include any existing member IDs plus the new ones: ``` mutation AppendMembers { CoreStandardGroupUpdate( data: { hfid: [""], members: [ {id: ""}, {id: ""}, {id: ""} ] } ) { ok } } ``` If you omit existing IDs, those members will be removed. ``` group = client.get(kind="CoreStandardGroup", name__value="") obj_a = client.get(kind="", name__value="") obj_b = client.get(kind="", name__value="") group.members.add(obj_a) group.members.add(obj_b) group.save() ``` `.members.add()` appends; existing members are preserved. ## Verify[​](#verify "Direct link to Verify") Use [Query group membership](/groups/query-members.md) to confirm the expected objects are attached. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") **Called update but existing members disappeared.** GraphQL update replaces the full list. Re-run with the existing IDs included. **Members don't appear.** Confirm the object IDs are correct and that the objects exist on the same branch as the group. **Permission denied.** You need write access to both the group and the members being attached. ## Next[​](#next "Direct link to Next") * [Query group membership](/groups/query-members.md) * [Remove members from a group](/groups/remove-members.md) * [Use groups in automation](/groups/use-in-automation.md) --- # Create a group Create a Standard group when you want to hand-pick the objects it contains. Which type of group? This page covers **Standard** groups (`CoreStandardGroup`). Generator groups and Query groups are created by the Generator or query that owns them — see the [Groups overview](/groups/overview.md) for when to use each type. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Access to an Infrahub instance. * Permission to create groups on the target branch. ## Create the group[​](#create-the-group "Direct link to Create the group") * Web Interface * GraphQL * Python SDK Navigate to **Object Management** → **Groups** and click **New Group**. Provide: * A name (for example `prod-fabric-devices`). * An optional description. * The group type (`CoreStandardGroup`). ``` mutation CreateGroup { CoreStandardGroupCreate(data: {name: {value: ""}}) { ok object { hfid } } } ``` Save the returned `hfid` — you'll need it when adding members. ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync(address="http://localhost:8000") group = client.create( kind="CoreStandardGroup", name="", ) group.save() ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") **Create returns a permission error.** Check your permissions for group management on the target branch. **The group was created but doesn't appear in the UI.** The group may be in another branch, or you may be viewing with "internal groups" filtered out. See the [visibility model](/groups/overview.md#the-visibility-model). ## Next[​](#next "Direct link to Next") * [Add members to the group](/groups/add-members.md) * [Delete a group](/groups/delete.md) if it was created by mistake --- # Delete a group Remove a group without affecting its member objects. ## What happens when you delete a group[​](#what-happens-when-you-delete-a-group "Direct link to What happens when you delete a group") * **Members** are **not** deleted. They remain in the graph and lose the `member_of_groups` relationship to this group. * **Subscribers** are **not** deleted. They lose the `subscriber_of_groups` relationship. * **Downstream automation** that targeted this group (Artifact Definitions, Transformations, Checks) will stop receiving this group's members. Review those integrations before deleting. Generator and Query groups Deleting a Generator group or Query group that is still referenced by the Generator or query that owns it can break that automation. Delete the owning Generator or query first, or reconfigure it to point elsewhere. ## Delete the group[​](#delete-the-group "Direct link to Delete the group") * Web Interface * GraphQL * Python SDK 1. Open the group from the Groups list. 2. Click **Delete** and confirm. ``` mutation DeleteGroup { CoreStandardGroupDelete(data: {hfid: [""]}) { ok } } ``` ``` group = client.get(kind="CoreStandardGroup", name__value="") group.delete() ``` ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") **Delete returns a reference error.** Something else is pointing at this group (an artifact definition, Transformation, or Check). Locate and update the reference before retrying. **Permission denied.** You need delete permission on groups on this branch. ## Next[​](#next "Direct link to Next") * [Create a group](/groups/create.md) if you need to recreate it. --- # Groups Groups are containers that establish relationships between objects in your infrastructure. They let you create logical collections that span different object types — you might group together devices, locations, and configurations that share a common purpose or characteristic. Unlike traditional static lists, Infrahub groups are first-class objects in the graph database: they have their own attributes, relationships, and lifecycle. This means groups participate in the broader infrastructure model as active entities rather than passive containers. ## The group architecture[​](#the-group-architecture "Direct link to The group architecture") ### CoreGroup — the foundation[​](#coregroup--the-foundation "Direct link to CoreGroup — the foundation") Every group in Infrahub inherits from `CoreGroup`, a generic model that provides the essential structure. The CoreGroup defines two primary relationships: * **Members** — objects that belong to the group. * **Subscribers** — objects that observe or react to changes in the group. This dual-relationship model reflects a key design decision: groups aren't just about membership, they're about creating actionable connections between objects. ### Bidirectional relationships[​](#bidirectional-relationships "Direct link to Bidirectional relationships") When an object becomes a member of a group, Infrahub automatically creates inverse relationships. Every object gains: * `member_of_groups` — a computed relationship showing all groups containing this object. * `subscriber_of_groups` — a computed relationship showing all groups this object subscribes to. These automatic relationships enable powerful graph traversals such as "what groups does this device belong to?" or "what objects are affected by changes to this group?" — see [Query group membership](/groups/query-members.md) for how to use them. ## Types of groups[​](#types-of-groups "Direct link to Types of groups") Infrahub provides three group types, each for a different use case: | Type | Class | Membership managed by | Use when | | ------------- | ----------------------- | -------------------------- | ------------------------------------------------------------------------- | | **Standard** | `CoreStandardGroup` | You | You want to hand-pick members based on business logic | | **Generator** | `CoreGeneratorGroup` | The Generator that owns it | Objects are produced by a Generator and should track back to their origin | | **Query** | `CoreGraphQLQueryGroup` | A live GraphQL query | Membership should change dynamically as the underlying data changes | **Choosing between types:** * If you are manually deciding what goes in the group, use a **Standard** group. * If you have a Generator producing objects, let it own a **Generator** group automatically. * If membership should reflect "everything matching criteria X right now," use a **Query** group. ## The visibility model[​](#the-visibility-model "Direct link to The visibility model") Infrahub distinguishes between two visibility levels for groups: * **Default groups** are user-managed and appear in the standard interface. They represent the organizational structure you actively maintain and reference in daily operations. * **Internal groups** are system-managed and operate behind the scenes, supporting Infrahub's internal processes. You can toggle visibility when needed for debugging or advanced operations. Hiding internal groups by default reduces visual complexity while keeping the underlying power available. ## Where groups are used[​](#where-groups-are-used "Direct link to Where groups are used") Groups serve as a bridge between different Infrahub features, creating a unified way to target operations across the system: * **artifact generation** — an artifact definition targets a group; the definition is decoupled from a specific device list, making it reusable. * **Transformations** — Transformations can operate on entire groups, applying consistent changes across all members. * **Checks** — validation checks target groups, running the same logic across all members without duplicating code. For concrete patterns, see [Use groups in automation](/groups/use-in-automation.md). ## Learn by doing[​](#learn-by-doing "Direct link to Learn by doing") New to groups? Walk through [Organize objects with groups](/learn/tutorials/groups.md) in Academy. It creates a group end-to-end using a running example. ## Reference[​](#reference "Direct link to Reference") * [Groups schema specification](/reference/schema/groups.md) --- # Query group membership Read what's in a group, or find what groups an object belongs to. Use these patterns for auditing, verification, or feeding automation. ## List members of a group[​](#list-members-of-a-group "Direct link to List members of a group") * Web Interface * GraphQL * Python SDK 1. Open the group from the Groups list. 2. Go to the **Members** tab. ``` query { CoreStandardGroup(name__value: "") { edges { node { name { value } members { edges { node { id display_label } } } } } } } ``` ``` group = client.get(kind="CoreStandardGroup", name__value="") for member in group.members.fetch(): print(member.display_label) ``` ## Filter members by type[​](#filter-members-by-type "Direct link to Filter members by type") Groups can hold objects of multiple kinds. To get just one kind, use an inline fragment: ``` query { CoreStandardGroup(name__value: "") { edges { node { members { edges { node { ... on InfraDevice { id name { value } site { node { name { value } } } } } } } } } } } ``` Replace `InfraDevice` with your target kind. ## Find what groups an object belongs to[​](#find-what-groups-an-object-belongs-to "Direct link to Find what groups an object belongs to") Every object gains an automatic `member_of_groups` relationship. Use it to traverse from an object back to its groups: ``` query { InfraDevice(id: "") { edges { node { name { value } member_of_groups { edges { node { name { value } description { value } } } } } } } } ``` The same pattern works with `subscriber_of_groups` when you need to know what's observing a group. ## Next[​](#next "Direct link to Next") * [Add members to a group](/groups/add-members.md) * [Remove members from a group](/groups/remove-members.md) * [Use groups in automation](/groups/use-in-automation.md) --- # Remove members from a group Detach objects from a Standard group without deleting the objects themselves. note Applies to **Standard** groups only. Generator and Query groups manage their own membership. ## How removal works in GraphQL[​](#how-removal-works-in-graphql "Direct link to How removal works in GraphQL") `CoreStandardGroupUpdate` replaces the entire member list. To remove a member, you pass a list containing every member you want to **keep**. This means removal is a two-step operation: list the current members, then update with the new list excluding the ones to remove. ## Remove members[​](#remove-members "Direct link to Remove members") * Web Interface * GraphQL * Python SDK 1. Open the group and go to the **Members** tab. 2. Select the members to remove. 3. Click **Remove from Group** and confirm. **Step 1 — list current members:** ``` query { CoreStandardGroup(name__value: "") { edges { node { members { edges { node { id display_label } } } } } } } ``` **Step 2 — update with the members you want to keep:** ``` mutation RemoveMembers { CoreStandardGroupUpdate( data: { hfid: [""], members: [ {id: ""}, {id: ""} ] } ) { ok } } ``` ``` group = client.get(kind="CoreStandardGroup", name__value="") target = client.get(kind="", id="") group.members.remove(target) group.save() ``` The SDK handles the fetch-and-update internally. ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") **Called update but nothing changed.** If you sent the full current member list back, no members were removed. Omit the IDs of the members you want to remove. **The objects disappeared, not just the membership.** They shouldn't — removing members detaches them; it does not delete them. If objects are missing entirely, look for a cascading delete elsewhere (for example, a Generator that owned them). ## Next[​](#next "Direct link to Next") * [Delete a group](/groups/delete.md) * [Query group membership](/groups/query-members.md) --- # Use groups in automation Target a group from an artifact definition, Transformation, or Check so that the automation applies to every member without hard-coding a list. This is the main reason to use groups: they decouple "what to operate on" from "what to do." Instead of listing 50 devices in an artifact definition, you target one group, and the group's contents can change independently. ## Target a group from an artifact definition[​](#target-a-group-from-an-artifact-definition "Direct link to Target a group from an artifact definition") An artifact definition has a `targets` field that points to a group. Infrahub generates one artifact per member. ``` # .infrahub.yml excerpt artifact_definitions: - name: "switch-base-config" artifact_name: "base-config" parameters: device: "name__value" content_type: "text/plain" targets: "" transformation: "switch-config-template" ``` When the group's membership changes, the next artifact generation run picks up the new members automatically. See [artifacts](/artifacts/overview.md) for the full specification. ## Target a group from a Transformation[​](#target-a-group-from-a-transformation "Direct link to Target a group from a Transformation") Transformations that produce per-object output are wired up through an artifact definition pointing at a group (same pattern as above). The Transformation runs once per group member with that member passed as an input parameter. See [Transformations](/transformations/overview.md) for Jinja2 and Python Transformation details. ## Scope a Check to a group[​](#scope-a-check-to-a-group "Direct link to Scope a Check to a group") A Check Definition can be scoped to a group so the check runs across every member. This lets you write one check and apply it consistently. ``` # .infrahub.yml excerpt check_definitions: - name: "naming-convention" class_name: "NamingConventionCheck" file_path: "checks/naming.py" targets: "" parameters: device: "name__value" ``` See [Checks](/checks/overview.md) for check authoring. ## Use with Generators[​](#use-with-generators "Direct link to Use with Generators") A Generator's `targets` field points to a `CoreStandardGroup`. The members of that group are the Generator's *inputs* — Infrahub creates one Generator run per member. As the Generator runs, Infrahub automatically tracks the objects it produces in a `CoreGeneratorGroup` via the SDK tracking feature. You don't define this group; it's managed for you. | Group | Role | Who manages it | | -------------------- | ------------------------------------------------------ | ---------------------------------------------- | | `CoreStandardGroup` | **Input** — drives Generator runs (one run per member) | You — set in `targets:` and populate the group | | `CoreGeneratorGroup` | **Output** — tracks objects the Generator produced | Infrahub, automatically via SDK tracking | Downstream automation (artifact definitions, Checks) can target either group, depending on whether you want to operate on the inputs or the produced objects. See [Generator overview](/generators/overview.md) for full details, including the difference between Standard and Generator groups. ## Pattern: chain groups through the pipeline[​](#pattern-chain-groups-through-the-pipeline "Direct link to Pattern: chain groups through the pipeline") A typical production setup: 1. A **`CoreStandardGroup`** lists the devices that need a service. You manage its membership directly, or have a workflow keep it in sync with a query. 2. A **Generator** targets that group, producing per-device service objects. Infrahub automatically tracks the produced objects in a `CoreGeneratorGroup`. 3. An **artifact definition** targets either: * the input `CoreStandardGroup` (to render configuration per device), or * the auto-managed `CoreGeneratorGroup` (to render per produced service object). 4. A **Check Definition** targets the input group to validate the fleet. Groups are the glue between each stage — replace any stage without rewiring the rest. ## Next[​](#next "Direct link to Next") * [Query group membership](/groups/query-members.md) to inspect what each group contains at each stage. * [Artifacts](/artifacts/overview.md), [Transformations](/transformations/overview.md), [Checks](/checks/overview.md), [Generators](/generators/overview.md) for feature-specific details. --- # Immutable history Infrahub preserves previous values and relationships as your infrastructure data changes. Earlier versions remain available for queries, so you can inspect what existed at a specific time, compare changes across a period, and trace how an object or relationship changed. This is useful for troubleshooting incidents, reviewing the effect of a change, answering audit questions, and understanding how topology or dependencies evolved. When you specify a timestamp, Infrahub returns the objects, attribute values, relationships, and schema that were valid at that time. ## What you can do with immutable history[​](#what-you-can-do-with-immutable-history "Direct link to What you can do with immutable history") * Query objects, attributes, relationships, and the schema as they existed at a specific point in time. * Compare two timestamps to identify which objects, attributes, or relationships changed. * Trace who performed an operation and when, through the [Activity log](/deploy-manage/run-observe/activity-log.md), and see where a value came from through [object metadata](/objects/metadata.md). * Preserve the history available before a branch was created while recording changes made on that branch. * Use previous infrastructure data for troubleshooting, audits, security investigations, and post-incident analysis. For example, you can answer questions such as: * Which devices, interfaces, and relationships existed for this site during last night's incident? * Which interface attributes changed between the last known-good timestamp and now? * Which services and circuits were related before this topology change? ## Historical data and the Activity log answer different questions[​](#historical-data-and-the-activity-log-answer-different-questions "Direct link to Historical data and the Activity log answer different questions") The Activity log records operations: which object was changed, when that change occurred and who made it, and the sequence of actions. Immutable history preserves the versions of data produced by those changes. If you need to know who changed an interface and when, use the [Activity log](/deploy-manage/run-observe/activity-log.md). If you need to know which interfaces, addresses, and relationships existed at 14:00 during an incident, query the data for that timestamp. When you specify a timestamp, Infrahub returns the objects, attribute values, and relationships that were valid then, so you query that data directly rather than reconstructing it from a backup and the changes recorded after it. ## How Infrahub preserves history[​](#how-infrahub-preserves-history "Direct link to How Infrahub preserves history") Each change in Infrahub creates a new version instead of modifying the previous value in place. Every version is associated with a timestamp, and earlier values remain available for queries that specify an earlier time. History is recorded at four levels: objects, attributes, relationships, and the schema. An object records the period during which it existed, so a query for an earlier time returns objects that have since been deleted and omits objects that did not yet exist. Attributes are versioned individually — when one field changes, Infrahub records a new value for that field alone, which is what makes it possible to identify the specific fields that changed between two timestamps. Relationships are versioned the same way, so a query for a past time returns the connections between objects as they stood then rather than the current ones. This matters for topology and dependency questions, where the connections are part of the answer. The schema is versioned as well. If the schema changed after the timestamp you request, Infrahub loads the schema as it was at that point, so the query sees the attributes and relationships the schema defined then rather than the ones it defines now. A graph that stores validity times for both its values and its relationships is a **temporal graph**. The period during which each value and connection applied is part of the stored data, which is why you can answer a question about past topology with a query. ## How branches use immutable history[​](#how-branches-use-immutable-history "Direct link to How branches use immutable history") When you create a branch, it starts from the data and history available on the default branch at its branch point. Changes made on that branch create new versions there without changing the data on the default branch, so you can develop and review several changes independently while retaining the history needed to compare and merge them. See [Branches](/branches/overview.md) for the branch creation, diff, and merge workflow. Three branch operations change which history remains available, and each one affects what a historical query can return. **Merging** records the merged changes on the default branch with the timestamp of the merge, not the timestamps they had on the branch. If someone creates an object on a branch on Monday and the branch merges on Friday, the default branch shows that object as created on Friday. The default branch's history therefore holds the result of the branch rather than the sequence of changes that produced it, and those intermediate versions stay readable on the branch itself for as long as the branch exists. **Rebasing** moves the timestamps of changes made on a branch up to the rebase time, so a change recorded on the branch before a rebase is no longer readable at its original timestamp. **Deleting a branch** removes the data and history recorded on it permanently. Versions on the default branch are unaffected, and the [Activity log](/deploy-manage/run-observe/activity-log.md) still records the operations that were performed on the branch, but the versions themselves cannot be recovered. If a branch's history matters for an audit or an investigation, read it before the branch is deleted. ## Query data at a specific time[​](#query-data-at-a-specific-time "Direct link to Query data at a specific time") By default, queries return the latest data on the selected branch. Set a time when you need the state of the data at an earlier time. You can specify a time through the web interface, GraphQL API, REST API, and Python SDK. See [Query historical data](/immutable-history/query-historical-data.md) for the available interfaces, comparing two timestamps, supported time formats, and branch history limits. ## Related[​](#related "Direct link to Related") * [Query historical data](/immutable-history/query-historical-data.md) — read the graph at an earlier timestamp, and compare two timestamps to see what changed * [Branches](/branches/overview.md) — how branches diverge, share history, and merge * [Proposed Changes](/proposed-changes/overview.md) — compare a branch with its base, with review, validation, and checks * [Activity log](/deploy-manage/run-observe/activity-log.md) — which operations occurred, when, and by whom --- # Query historical data You can read infrastructure data as it existed at previous points in time, and compare how it changed between them. A timestamped query answers what current data cannot: which devices and interfaces existed for a site during last night's incident, which interface attributes changed between the last known-good time and now, or which prefixes were assigned to a site on a given date. Infrahub offers three ways to ask about the past, and they answer different questions: | What you need | How to ask | Where it is covered | | --------------------------------------------------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | The state of the data at one point in time | `at` and `branch` on a read | [Query data at a specific time](#query-data-at-a-specific-time) | | Which objects and attributes changed between two points | `DiffUpdate`, then `DiffTree` | [Compare changes between two timestamps](#compare-changes-between-two-timestamps) | | Which operations were performed between two points, and by whom | `since` and `until` on an event query | [Querying the activity log over the API](/deploy-manage/run-observe/activity-log.md#querying-the-activity-log-over-the-api) | The first two read the data itself. The third reads the record of operations that produced it — see [Immutable history](/immutable-history/overview.md) for which question each one answers. ## Specify a branch and a time[​](#specify-a-branch-and-a-time "Direct link to Specify a branch and a time") To read the state of the data at a specific point in time, specify a branch and a timestamp. Infrahub returns the objects, attribute values, and related objects that were valid at that time. If you do not set either one, Infrahub returns the current data from the default branch. | State to read | Branch | Time | | ------------------------------------------------- | ------------ | -------------------------- | | Production as it stands | default | now | | Production during last night's incident | default | a time within the incident | | The change someone is proposing | their branch | now | | The default branch just before that change merged | default | a time before the merge | ## How historical queries work[​](#how-historical-queries-work "Direct link to How historical queries work") When you add a timestamp to a query, Infrahub evaluates each requested object using the attribute values and relationships that were valid at that time. You use the same query structure as you do for current data; the timestamp changes which versions Infrahub returns. Infrahub can return those versions because a change records a new value alongside the previous one rather than replacing it. History is recorded per object, per attribute, and per relationship, which is what lets a comparison identify the specific fields or connections that changed and what they held before. See [How Infrahub preserves history](/immutable-history/overview.md) for the storage model. The schema is evaluated for the same point in time. If the schema changed after the timestamp you request, Infrahub loads the schema as it was then, so the query sees the objects, attributes, and relationships that existed at that point rather than the current ones. A timestamp applies to read operations only — queries, in GraphQL terms. If a query document contains a mutation, Infrahub ignores the timestamp and applies the mutation at the current time. Once Infrahub records a version, a change made after it does not alter that version. See [Immutable history](/immutable-history/overview.md) for more detail on how Infrahub preserves those versions and how immutable history relates to branches and the Activity log. ## Query data at a specific time[​](#query-data-at-a-specific-time "Direct link to Query data at a specific time") Use a timestamp when you need the objects, attributes, and relationships that existed at a known point — for example, during an incident or before a change. You can specify a time when viewing or querying Infrahub data through the web interface, GraphQL, REST API, or Python SDK. Use the web interface for interactive investigation and the APIs or SDK when you need a repeatable or programmatic query. * Web interface * GraphQL * REST API * Python SDK Select the time selector — the calendar and clock icon beside the branch selector — and choose a date and time. The picker uses your browser's time zone rather than UTC, and offers past dates and times only. Until you set a time, the selector displays only the icon; once you set one, the bar beside it displays **Current view time** with your selection. Select the × beside the displayed time to return to the current time. The selected time stays applied as you navigate, and it covers more than object data. Attribute values, relationships, the schema, and the navigation menu are all loaded for that timestamp, so an object's fields and the navigation tree can differ from what the current schema defines. Diffs, Proposed Changes, tasks, and events have no time-aware query, so those screens show current data even while **Current view time** is displayed. The selected time is part of the page URL as the `at` parameter, and it is preserved as you navigate, so you can bookmark a historical view or share it as a link. warning Infrahub does not prevent editing while a past time is applied, and it does not warn you. A change you save — including a deletion — applies to the current data at the current time, not to the time you are viewing. Treat a view with **Current view time** set as read-only. ![The time selector, with a past time applied](/assets/images/tutorial_2_historical-c96d49d6acece4d5ea5d72ac120af396.png) Use the `at` URL parameter to specify the timestamp for a GraphQL query. `at` goes on the endpoint, not in the query document, so the document itself is unchanged from the one you use for current data: ``` http://localhost:8000/graphql/main?at=2026-03-09T14:00:00Z ``` Read a device as it existed at an earlier time ``` query DeviceAtTime { InfraDevice(name__value: "ord1-edge1") { edges { node { name { value } description { value } status { value } } } } } ``` The REST API accepts `at` on the endpoints that run stored objects, not on ad-hoc object reads: | Endpoint | Returns | | -------------------------------------- | ------------------------------------------------------ | | `/api/query/{query_id}` | The result of a stored `CoreGraphQLQuery` at that time | | `/api/artifact/{artifact_id}` | An artifact rendered from the data valid at that time | | `/api/transform/python/{transform_id}` | A Python transform run against that data | | `/api/transform/jinja2/{transform_id}` | A Jinja2 transform rendered from that data | To read objects at a past time over REST, save the read as a `CoreGraphQLQuery` and execute it by name: ``` curl "http://localhost:8000/api/query/device-status?branch=main&at=2026-03-09T14:00:00Z" \ -H "X-INFRAHUB-KEY: $INFRAHUB_API_TOKEN" ``` Pass the `at` argument to the SDK query methods when you need a node or set of nodes as they existed at a specific time. `all()`, `get()`, and `filters()` take a `Timestamp`; `execute_graphql()` also accepts a string. ``` from infrahub_sdk import InfrahubClient from infrahub_sdk.timestamp import Timestamp client = InfrahubClient(address="http://localhost:8000") device = await client.get( kind="InfraDevice", name__value="ord1-edge1", at=Timestamp("2026-03-09T14:00:00Z"), ) ``` ## Compare changes between two timestamps[​](#compare-changes-between-two-timestamps "Direct link to Compare changes between two timestamps") Comparing two timestamps takes two steps: Infrahub calculates the diff, then you retrieve it. A diff is stored rather than computed when you ask for it, which is why a query for a period that has never been calculated returns `null` instead of a result. Send the `DiffUpdate` mutation to calculate the diff. A custom period requires both the time range and a `name` to store it under — without a name, Infrahub rejects the request. Pass `wait_until_completion: true` when the mutation should return only once the calculation has finished, rather than starting a background task. Calculate a diff for a period ``` mutation CalculateIncidentDiff { DiffUpdate( data: { branch: "main" name: "incident-2026-03-09" from_time: "2026-03-09T00:00:00Z" to_time: "2026-03-10T00:00:00Z" } wait_until_completion: true ) { ok } } ``` Then retrieve it with the same branch and time range you calculated it for, using `DiffTree` for the changed nodes or `DiffTreeSummary` when you only need counts. The timestamps do not need to align with a branch point or a Proposed Change, so you can compare any useful period, such as the hours around an incident. Retrieve a custom period by that period rather than by its name. `DiffTree` looks for a diff covering the range it computes by default, from the branch point to the present, so it does not find one stored for a different range and returns `null`. `DiffTreeSummary` does not accept `name` at all and returns an error. A name is still worth setting, because `DiffUpdate` requires one for a custom period. Retrieve the calculated diff ``` query IncidentChanges { DiffTree( branch: "main" from_time: "2026-03-09T00:00:00Z" to_time: "2026-03-10T00:00:00Z" ) { num_added num_updated num_removed nodes { kind label status attributes { name status } } } } ``` Behavior to expect: * Omit `from_time` and the comparison starts at the branch's `branched_from` timestamp. * Omit `to_time` and the comparison runs to the present. * Narrow the result with `filters`, which accepts `ids`, `kind`, `namespace`, and `status`. The `kind`, `namespace`, and `status` filters each take `includes` and `excludes` lists. * Page through results with `limit` and `offset`. * The base of the comparison is the default branch. Name a feature branch to compare it with the default branch across the requested period, or name the default branch to compare it with itself over time. * Pass `proposed_change_id` instead of a branch and period to retrieve the diff for a Proposed Change. * A `null` result means that no diff covering the requested period has been calculated, not that nothing changed. In the web interface, a branch's Branch view shows its diff and lets you refresh it, whether or not a Proposed Change exists for that branch, and a Proposed Change shows the diff between its source and target branches. When you need review, validation, and checks alongside the comparison, use a [Proposed Change](/proposed-changes/overview.md). ## Choose an absolute timestamp or relative offset[​](#choose-an-absolute-timestamp-or-relative-offset "Direct link to Choose an absolute timestamp or relative offset") Use a relative offset when you are investigating from the current time. Use an absolute timestamp when the query needs to resolve to the same time each time it runs — for example, for an audit answer, post-incident report, or reproducible analysis. | Form | Example | Resolves to | | ------------------------------ | ----------------------------------------------------- | -------------------------------------------- | | ISO 8601 with a zone or offset | `2026-03-09T14:00:00Z` or `2026-03-09T15:00:00+01:00` | The specified instant | | ISO 8601 without a zone | `2026-03-09T14:00:00` | The same wall-clock time, interpreted as UTC | | Date only | `2026-03-09` | 12:00 UTC on that day | | Offset from now | `30s`, `45m`, `6h`, `2h30m` | That interval before the current time | A date without a time resolves to midday rather than midnight, so include an explicit time when a day boundary matters. Relative offsets support seconds, minutes, and hours, including combined values such as `2h30m`. They do not support day or week units; use an absolute timestamp for those intervals. ## Understand branch history limits[​](#understand-branch-history-limits "Direct link to Understand branch history limits") Every branch reads history through the default branch, so the earliest time you can query is the default branch's creation time — the point at which Infrahub was first initialized. That floor is the same on a feature branch as on the default branch, because creating a branch records where it diverged rather than copying the dataset. A branch created this morning can still be read back to the default branch's creation, with the changes recorded on the branch since it diverged added to that history. If you request an earlier time than that, Infrahub rejects the query rather than returning partial data: ``` Requested time '2026-01-05T00:00:00Z' is before branch 'main' was created at '2026-02-01T09:14:22.481000Z'. ``` Infrahub validates the earliest time only. If you request a time later than the current time, Infrahub accepts it and returns the current data, because every attribute value and relationship that is valid now is also valid at a later time. In the web interface, the time selector offers past dates and times; through GraphQL, the REST API, and the Python SDK, you can request a future time. If you build the timestamp from a variable or a calculation, confirm it resolves to a past time — nothing in the response indicates that a future timestamp was used. ## Related[​](#related "Direct link to Related") * [Immutable history](/immutable-history/overview.md) — how Infrahub preserves previous values and relationships * [Branches](/branches/overview.md) — how branches diverge, share history, and merge * [Activity log](/deploy-manage/run-observe/activity-log.md) — which operations occurred, when, and by whom * [Proposed Changes](/proposed-changes/overview.md) — compare a branch with its base with review, validation, and checks --- ## [🔗Ansible Integration ↗](https://docs.infrahub.app/ansible) --- # Allocate IPs and prefixes ## When to allocate from a pool[​](#when-to-allocate-from-a-pool "Direct link to When to allocate from a pool") Allocate from a Resource Pool when something other than a person assigns the address — a provisioning pipeline creating devices, a Generator rendering configuration, a CI/CD job reserving space for a new service. If you are only recording and querying IP data that already exists, the IPAM nodes alone are enough. ## Use a resource pool[​](#use-a-resource-pool "Direct link to Use a resource pool") Resource Pool is the recommended way to allocate IP addresses and prefixes. A `CoreIPAddressPool` or `CoreIPPrefixPool` takes your IPAM prefixes as its `resources` and returns the next available address or subnet on request. ``` mutation { CoreIPAddressPoolCreate(data: { name: {value: "Management IP Pool"}, default_address_type: {value: "IpamIPAddress"}, default_prefix_length: {value: 24}, resources: [{id: ""}], ip_namespace: {id: "default"} }) { ok object { id } } } ``` The `resources` field references the `IpamIPPrefix` objects the pool allocates from — this is the connection point between IPAM and Resource Manager. Passing the same `identifier` on repeated allocation requests returns the existing allocation instead of creating a new one, so a provisioning script can run more than once without allocating the same address or subnet twice. Each pool also takes an exclusive lock scoped to itself while allocating, so two concurrent requests against the same pool never return the same resource. For the full allocation walkthrough — creating the source prefix, creating the pool, and allocating directly or during node creation — see [Allocate IP addresses](/resource-manager/allocate-ip-address.md) and [Allocate IP prefixes](/resource-manager/allocate-ip-prefix.md). ## Allocate from generators[​](#allocate-from-generators "Direct link to Allocate from generators") A Generator has the same Python SDK client available anywhere else, so allocating from a Resource Pool inside a Generator is a regular SDK call: ``` device_loopback = await client.allocate_next_ip_address( resource_pool=loopback_pool, identifier=device_name, data={"interface": interface.id}, branch=branch, ) interconnect_subnet = await client.allocate_next_ip_prefix( resource_pool=interconnection_pool, kind=IpamIPPrefix, identifier=network.identifier, branch=branch, ) ``` `allocate_next_ip_address` and `allocate_next_ip_prefix` take the same `resource_pool` and `identifier` arguments as the underlying `InfrahubIPAddressPoolGetResource` and `InfrahubIPPrefixPoolGetResource` mutations — passing a stable `identifier` (for example, the device name) keeps a Generator idempotent across reruns. The [infrastructure-edge topology model](https://github.com/opsmill/infrahub/blob/stable/models/infrastructure_edge.py) is a worked example that allocates loopback and management addresses, and interconnection subnets, this way. ## Advanced: next-available GraphQL primitives[​](#advanced-next-available-graphql-primitives "Direct link to Advanced: next-available GraphQL primitives") `InfrahubIPAddressGetNextAvailable` and `InfrahubIPPrefixGetNextAvailable` are read-only GraphQL queries that return the next free address or subnet from a given prefix. Unlike a Resource Pool allocation, they don't create a record, and they don't provide the pool's cross-branch allocation guarantee — two callers on different branches can compute the same "next available" value independently and collide. Use Resource Pool for production workflows. Use these queries only to look up a value without persisting it. --- # Build your IPAM schema Infrahub includes the `IpamNamespace` node and the `BuiltinIPNamespace`, `BuiltinIPPrefix`, and `BuiltinIPAddress` generics by default — but no concrete IP prefix or address node until you load one. ## Load the IPAM schema[​](#load-the-ipam-schema "Direct link to Load the IPAM schema") Already have IPAM in the menu? If the `IPAM` entry already appears in the left-side menu, a schema is already loaded — this is the case after completing a quick start. You can skip the rest of this section. Otherwise, load the base collection from the Marketplace: ``` infrahubctl marketplace get infrahub/base-schemas --collection infrahubctl schema load schemas/ ``` To inspect the IPAM schema alone without loading it, fetch it with `--stdout`: ``` infrahubctl marketplace get infrahub/ipam --stdout ``` See [Infrahub Marketplace](/schema/marketplace.md) for the full fetch-and-load workflow. ## Define your own IPAM nodes[​](#define-your-own-ipam-nodes "Direct link to Define your own IPAM nodes") The Marketplace collection speeds this up by giving you a working schema to start from — building it yourself means inheriting directly from the built-in generics. Inheriting from `BuiltinIPPrefix` and `BuiltinIPAddress` gets you hierarchy, utilization, and the other generic behavior for free: schemas/ipam/prefix-and-address.yml ``` nodes: - name: IPPrefix namespace: Ipam inherit_from: - BuiltinIPPrefix description: IPv4 or IPv6 network icon: mdi:ip-network label: IP Prefix menu_placement: IpamNamespace - name: IPAddress namespace: Ipam inherit_from: - BuiltinIPAddress description: IPv4 or IPv6 address icon: mdi:ip label: IP Address menu_placement: IpamNamespace ``` Load it the same way as a Marketplace schema: ``` infrahubctl schema load schemas/ ``` An `IpamNamespace` node inheriting from `BuiltinIPNamespace` already exists by default — define your own only if you need attributes beyond `name` and `description`. ## VLAN, VRF, RIR, and other extensions[​](#vlan-vrf-rir-and-other-extensions "Direct link to VLAN, VRF, RIR, and other extensions") VLAN, VRF, RIR, and aggregate are Marketplace extensions, not part of the base IPAM schema — fetch them individually: ``` infrahubctl marketplace get infrahub/vrf infrahubctl marketplace get infrahub/vlan ``` Browse [marketplace.infrahub.app](https://marketplace.infrahub.app) for the full catalog of extensions. Load extension schemas on a branch, not `main` — see [Create and load schema](/schema/create-and-load.md). ## IPHost, IPNetwork, and IPAddress attribute kinds[​](#iphost-ipnetwork-and-ipaddress-attribute-kinds "Direct link to IPHost, IPNetwork, and IPAddress attribute kinds") `IPHost`, `IPNetwork`, and `IPAddress` are attribute kinds, not IPAM nodes. Attach one directly to any schema node for a validated IP value that does not belong in your address plan — a monitoring target, a syslog destination, an NTP server address. The value behaves like any other attribute: it lives on the node you attach it to, it does not appear in `IpamIPPrefix` hierarchy or utilization, and it cannot be allocated from a Resource Manager pool. | Kind | Stores | Example | | ----------- | ------------------------------------------------------------------------------------------------------------------- | -------------- | | `IPAddress` | A bare IP address, with no prefix or netmask. A value carrying prefix notation is rejected. | `192.0.2.1` | | `IPHost` | An IP address with a prefix length. A bare value is stored with a host prefix — `192.0.2.1` becomes `192.0.2.1/32`. | `192.0.2.1/24` | | `IPNetwork` | A network in CIDR notation. | `192.0.2.0/24` | Use `IPAddress` when the value has no subnet context and a prefix would be misleading. Use `IPHost` when the prefix is part of the data but the address itself is not tracked in IPAM — for example, a value mirrored from an external inventory system. Use `IPNetwork` for a subnet reference that is not itself a tracked `IpamIPPrefix`. An address that belongs to infrastructure you manage — an interface's IP, for example — is not a fit for any of these three kinds. Infrahub's base schema relates `InfraInterfaceL3` to `IpamIPAddress` through a `Component` relationship instead, so each address gets its own hierarchy position, utilization accounting, and pool allocation history. See [When to model an IPAM node instead](#when-to-model-an-ipam-node-instead) below. ### Example: NTP and syslog addresses on a region[​](#example-ntp-and-syslog-addresses-on-a-region "Direct link to Example: NTP and syslog addresses on a region") A region's NTP and syslog servers are addresses other devices point to, not address space to manage — they need no hierarchy, no utilization, and no allocation of their own. Add them as `IPAddress` attributes on the node that uses them: schemas/location/region.yml ``` nodes: - name: Region namespace: Location attributes: - name: name kind: Text unique: true - name: ntp_server kind: IPAddress optional: true - name: syslog_server kind: IPAddress optional: true ``` `192.168.1.1/24` is rejected on either attribute — only a bare address is accepted. Read the value directly, with no netmask to strip: ``` region = await client.get(kind="LocationRegion", name__value="us-east") print(region.ntp_server.value) # "192.168.1.1" ``` ### When to model an IPAM node instead[​](#when-to-model-an-ipam-node-instead "Direct link to When to model an IPAM node instead") The `Region` example above fits a third-party NTP or syslog service — the address is not yours to manage, and Infrahub only needs it to render into device configuration. The `IPAddress` attribute is the right choice for that: a validated value to read back at generation time, nothing more. Use an `IpamIPAddress` node instead of an attribute kind when the address itself needs to: * Participate in prefix hierarchy or utilization calculations * Be allocated from a `CoreIPAddressPool` * Be shared across more than one object, or carry its own relationships, lifecycle, or namespace If the NTP server is infrastructure you track and allocate addresses for — not a third-party value another device just points to — model it as an `IpamIPAddress` and relate it to `Region` following the pattern in [Extend IPAM with a relationship to another object](#extend-ipam-with-a-relationship-to-another-object) below. ## Extend IPAM with a relationship to another object[​](#extend-ipam-with-a-relationship-to-another-object "Direct link to Extend IPAM with a relationship to another object") Relationships from IPAM to other Infrahub objects are ordinary schema relationships, not metadata fields. The [Service Catalog demo](https://github.com/opsmill/infrahub-demo-service-catalog) relates `IpamIPPrefix` and `IpamIPAddress` to a `ServiceGeneric` node this way, declaring both directions and pairing them by `identifier`. Declare the relationship from the `Service` side first — inbound, paired by `identifier`: example/service.yml ``` nodes: - name: Service namespace: Customer inherit_from: - InfraService attributes: - name: service_identifier kind: Text unique: true relationships: - name: assigned_prefix label: "Assigned prefix" peer: IpamIPPrefix optional: false kind: Attribute direction: inbound identifier: service_prefix cardinality: one ``` To add the relationship from the IPAM side instead — so you can query and filter from `IpamIPPrefix` or `IpamIPAddress` directly — extend the IPAM kind with an `extensions.nodes` block. Adapted from opsmill/infrahub-demo-service-catalog: schemas/service/service.yml ``` extensions: nodes: - kind: IpamIPPrefix relationships: - name: service peer: ServiceGeneric cardinality: one direction: outbound identifier: service_prefix ``` ## Next[​](#next "Direct link to Next") * [Use namespaces](/ipam/use-namespaces.md) — isolating the prefixes and addresses this schema holds * [Allocate IPs and prefixes](/ipam/allocate-ips-and-prefixes.md) — assigning addresses and subnets from a pool * [Query IPAM data](/ipam/query-ipam-data.md) — reading computed values and the hierarchy --- # IP Address Management Use Infrahub IPAM to manage IP prefixes, IP addresses, and namespaces alongside the rest of your infrastructure data. Prefixes and addresses are ordinary Infrahub objects, so you can add relationships to devices, interfaces, and services the same way you would on any other node. Infrahub computes hierarchy automatically and calculates utilization on read. Every change is branch-aware. Both IPv4 and IPv6 are supported. ![IPAM prefix hierarchy](/assets/images/ipam-efda16d682cca91155d4434c3b1663a3.png) Migrating from another system? Once your IPAM schema is in place, use [Infrahub Sync](https://docs.infrahub.app/sync) to bring in your existing IP data from a prior source of truth. ## Built-in generics[​](#built-in-generics "Direct link to Built-in generics") Infrahub provides three built-in generics to inherit from when building your IPAM schema: * `BuiltinIPNamespace` — isolates a set of IP prefixes and addresses. Comparable to a VRF or routing instance. * `BuiltinIPPrefix` — models an IPv4 or IPv6 network in CIDR notation. * `BuiltinIPAddress` — models a single IPv4 or IPv6 host address. Infrahub includes a schema node called `IpamNamespace` that inherits from `BuiltinIPNamespace`, and creates a `default` namespace object automatically on first start. info Building an IPAM with these generics serves a different purpose from using the `IPHost`, `IPNetwork`, or `IPAddress` attribute kinds on other nodes. The IPAM generics use the `IPHost` and `IPNetwork` attribute kinds internally. See [Build your IPAM schema](/ipam/build-your-ipam-schema.md#iphost-ipnetwork-and-ipaddress-attribute-kinds) for when to use an attribute kind instead of a full IPAM node. ## Hierarchy[​](#hierarchy "Direct link to Hierarchy") IP prefixes and addresses form a tree based on network containment — a prefix can be the parent or child of another prefix, and an address always belongs to the most specific prefix that contains it. Infrahub reconciles this tree automatically on every create, update, or delete, so parent, child, and address-to-prefix relationships stay correct without manual intervention. See [Query IPAM data](/ipam/query-ipam-data.md) for the relationships exposed on prefixes and addresses. ## Utilization[​](#utilization "Direct link to Utilization") Every IP prefix exposes a read-only `utilization` attribute, computed on read from the graph, using one of two formulas based on `member_type`: * **`prefix` mode** — utilization is the address space of the child prefixes, divided by the address space of the prefix itself. For example, `192.0.2.0/24` with one `192.0.2.0/26` child reports 25%. * **`address` mode** — utilization is the number of IP addresses assigned to the prefix, divided by the number of usable addresses in the prefix. For example, `192.0.2.0/26` holds up to 62 usable addresses (excluding network and broadcast); with 20 addresses assigned, it reports 32%. Network and broadcast addresses count toward the usable total when `is_pool` is `true`, when an IPv4 prefix is `/31` or longer (per [RFC 3021](https://datatracker.ietf.org/doc/html/rfc3021)), and for every IPv6 prefix. ## Namespaces[​](#namespaces "Direct link to Namespaces") Every IP prefix and IP address belongs to exactly one IP namespace. Namespaces isolate IP space — the same prefix or address can exist in multiple namespaces without conflict, similar to how a VRF isolates a routing table. Use additional namespaces when you need that isolation, for example between customers or business units with overlapping address ranges. Deleting a namespace deletes every IP prefix and IP address it contains. The `default` namespace cannot be deleted. ## Branching[​](#branching "Direct link to Branching") Creating, updating, or deleting an IP namespace, prefix, or address is supported on any branch. Hierarchy reconciliation runs immediately after each mutation, and again when the branch is merged or rebased into another branch — Infrahub reads the diff between the two branches, identifies which prefixes and addresses changed, and reconciles only those. See [Plan changes on a branch](/ipam/plan-changes-on-a-branch.md) for the full workflow, including concurrency control for concurrent branch-based changes. ## Resource Manager[​](#resource-manager "Direct link to Resource Manager") Resource Pool is the recommended way to allocate IP addresses and prefixes in automation — provisioning pipelines, generators, and CI/CD workflows. A `CoreIPAddressPool` or `CoreIPPrefixPool` draws from your IPAM prefixes as its resources and returns the next available address or subnet. Passing the same identifier on repeated calls returns the same allocation, so a provisioning script can run more than once without allocating the same resource twice. Infrahub also exposes lower-level GraphQL queries — `InfrahubIPAddressGetNextAvailable` and `InfrahubIPPrefixGetNextAvailable` — that return the next free address or subnet without creating a record and without the pool's cross-branch allocation guarantees. Use Resource Pool for production workflows; use these only to look up a value without persisting it. --- # Plan changes on a branch Use a branch to draft IP prefix and address changes, review exactly what will change, and merge once you're confident — the same version-control workflow Infrahub applies to the rest of your infrastructure data. ## The workflow[​](#the-workflow "Direct link to The workflow") Create a branch, then create, update, or delete IP namespaces, prefixes, and addresses on it through the web interface, GraphQL, or the Python SDK — the same way you would on the default branch. See [Create a branch](/branches/create.md) if you haven't created one before. Once your changes are ready, open a [Proposed Change](/proposed-changes/overview.md) to review them. The Data tab shows exactly which prefixes, addresses, and namespaces were added, modified, or deleted, and which attributes changed on each. Merge once the review is complete — see [Merge a branch](/branches/merge.md) for what happens at merge time. ## What reconciles on merge[​](#what-reconciles-on-merge "Direct link to What reconciles on merge") IPAM hierarchy reconciliation runs immediately after each mutation on a branch, so the branch's own prefix and address tree stays consistent while you work. When the branch merges or is rebased, Infrahub reads the diff between the two branches, identifies which prefixes and addresses changed, and reconciles only those — reassigning parent prefixes, child prefixes, and address-to-prefix associations as needed on the target branch. ## Concurrency[​](#concurrency "Direct link to Concurrency") Allocating from a resource pool is guarded by lock-based concurrency control (`InfrahubMultiLock`) on any branch — two clients requesting the next address or prefix from the same pool at the same time cannot receive the same value. Creating or updating a prefix with an explicit value is locked on any branch as well, so two clients cannot create the same prefix concurrently. Creating or updating an address with an explicit value is only locked on the default branch — on other branches, concurrent clients can create the same address value without contention. ## Worked example: redesign a subnet on a branch[​](#worked-example-redesign-a-subnet-on-a-branch "Direct link to Worked example: redesign a subnet on a branch") Continuing the ISP example from [Use namespaces](/ipam/use-namespaces.md): Customer A's network team needs to split their `10.0.0.0/16` supernet into two `/17` subnets ahead of a datacenter migration. 1. Create a branch: `customer-a-subnet-redesign`. 2. On the branch, create the two `10.0.0.0/17` and `10.0.128.0/17` prefixes in the `customer-a` namespace, both with `10.0.0.0/16` as their parent. 3. Move the existing `/24` and `/26` prefixes under whichever new `/17` contains them — Infrahub re-parents each one automatically based on its network containment. 4. Open a Proposed Change and review the Data tab: the new `/17` prefixes appear as additions, and each moved prefix shows its `parent` relationship changing from `10.0.0.0/16` to the matching `/17`. 5. Merge. Infrahub reconciles the hierarchy of the `customer-a` namespace on the default branch to match. ## Related[​](#related "Direct link to Related") * [Branches](/branches/overview.md) — branch lifecycle and isolation modes * [Proposed Changes](/proposed-changes/overview.md) — the review and merge workflow * [Allocate IPs and prefixes](/ipam/allocate-ips-and-prefixes.md) — allocating on a branch during provisioning --- # Query IPAM data Read the computed values on IP prefixes and addresses, or follow the hierarchy relationships. Use these patterns for reporting, automation, or verification. ## Query a prefix's computed values[​](#query-a-prefixs-computed-values "Direct link to Query a prefix's computed values") Every `IpamIPPrefix` exposes `utilization`, `netmask`, `hostmask`, `network_address`, and `broadcast_address` as read-only attributes, computed on read: * GraphQL * Python SDK ``` query { IpamIPPrefix(prefix__value: "192.0.2.0/24") { edges { node { prefix { value } utilization { value } netmask { value } hostmask { value } network_address { value } broadcast_address { value } } } } } ``` ``` prefix = await client.get(kind="IpamIPPrefix", prefix__value="192.0.2.0/24") print(prefix.utilization.value, prefix.netmask.value, prefix.hostmask.value) print(prefix.network_address.value, prefix.broadcast_address.value) ``` ## Query an address's computed values[​](#query-an-addresss-computed-values "Direct link to Query an address's computed values") `IpamIPAddress` has no equivalent sibling attributes — instead, the `address` attribute itself (kind `IPHost`) exposes computed sub-fields: `ip`, `netmask`, `hostmask`, `prefixlen`, and `version`. * GraphQL * Python SDK ``` query { IpamIPAddress(address__value: "192.0.2.10/24") { edges { node { address { value ip netmask hostmask prefixlen version } } } } } ``` ``` address = await client.get(kind="IpamIPAddress", address__value="192.0.2.10/24") value = address.address.value # an ipaddress.IPv4Interface or IPv6Interface print(value.ip, value.netmask, value.hostmask, value.version) ``` The Python SDK maps an `IPHost` attribute's value to the standard library's `ipaddress.IPv4Interface` or `IPv6Interface` — `netmask` and `hostmask` come from that object, not from a separate attribute. ## Navigate the hierarchy[​](#navigate-the-hierarchy "Direct link to Navigate the hierarchy") IP prefixes expose `parent`, `children`, `ancestors`, and `descendants` relationships. They are read-only — Infrahub manages them automatically. * GraphQL * Python SDK ``` query { IpamIPPrefix(prefix__value: "10.0.0.0/16") { edges { node { prefix { value } parent { node { prefix { value } } } children { edges { node { prefix { value } } } } ancestors { edges { node { prefix { value } } } } descendants { edges { node { prefix { value } } } } } } } } ``` ``` prefix = await client.get(kind="IpamIPPrefix", prefix__value="10.0.0.0/16") for child in await prefix.children.fetch(): print(child.prefix.value) for descendant in await prefix.descendants.fetch(): print(descendant.prefix.value) ``` ## Next[​](#next "Direct link to Next") * [IP Address Management](/ipam/overview.md) — the built-in generics and the hierarchy they participate in * [Use namespaces](/ipam/use-namespaces.md) — isolating the prefixes and addresses these queries run against * [Allocate IPs and prefixes](/ipam/allocate-ips-and-prefixes.md) — creating prefixes and addresses instead of reading them --- # Use namespaces Use IP namespaces to isolate sets of IP prefixes and addresses within the same Infrahub instance — for example, to give each customer or business unit its own IP space, even when their ranges overlap. ## Understanding IP namespaces[​](#understanding-ip-namespaces "Direct link to Understanding IP namespaces") If you are familiar with networking concepts, an IP namespace is analogous to: * A VRF (Virtual Routing and Forwarding) instance * A routing table or routing instance * A separate IP address space in a multi-tenant environment Just as a VRF lets a single router maintain multiple independent routing tables, IP namespaces let Infrahub manage multiple independent sets of IP resources. Each namespace is a boundary: IP prefixes and addresses in one namespace are separate from those in another. ## The default namespace[​](#the-default-namespace "Direct link to The default namespace") Infrahub creates a namespace called `default` automatically on first start. This namespace holds IP resources when you don't specify a namespace. For deployments managing a single network or organization, the default namespace is usually sufficient — create IP prefixes and addresses without configuring a namespace first. ## When to use multiple namespaces[​](#when-to-use-multiple-namespaces "Direct link to When to use multiple namespaces") * **Multi-tenant environments** — create a namespace per customer. Each customer can use the same RFC 1918 range (for example `10.0.0.0/8`) without conflict. * **Overlapping IP ranges** — track multiple isolated networks that reuse the same IP ranges, each in its own namespace. * **Organizational boundaries** — namespace per business unit, region, or data center to keep ownership and separation clear. * **Network segmentation** — model networks with separate routing domains, such as production versus lab, as distinct namespaces. ## How namespaces work[​](#how-namespaces-work "Direct link to How namespaces work") Every IP prefix and IP address belongs to exactly one namespace. An IP resource created without a namespace is assigned to `default`. Within a namespace, IP prefixes form hierarchical trees based on their network containment, and IP addresses are associated automatically with their most specific containing prefix. A prefix such as `10.0.0.0/24` can exist in multiple namespaces at the same time without conflict — the same applies to IP addresses. warning Deleting a namespace deletes every IP prefix and IP address it contains. The `default` namespace cannot be deleted. ## Practical example[​](#practical-example "Direct link to Practical example") An Internet Service Provider runs managed network services for three customers, and each customer uses `10.0.0.0/8` internally. A single IPAM without namespaces cannot hold three copies of the same range. With one namespace per customer, each range is tracked independently: * **customer-a** — Customer A's `10.0.0.0/8` and its subnets * **customer-b** — Customer B's `10.0.0.0/8` and its subnets * **customer-c** — Customer C's `10.0.0.0/8` and its subnets Each namespace has its own hierarchy, utilization figures, and address assignments, so a query scoped to one customer never returns another customer's addresses. ## Switch the active namespace[​](#switch-the-active-namespace "Direct link to Switch the active namespace") The IPAM section of the web interface scopes the prefix and address tree to one namespace at a time. Use the **IP Namespace** selector in the IPAM header to switch — the tree, filters, and utilization figures update to the selected namespace. The Python SDK and GraphQL API don't have an equivalent "active namespace" concept — every query or mutation for a prefix or address takes the namespace explicitly, either as an `ip_namespace` filter or as part of the object's data. --- # Build a check in Infrahub By the end of this tutorial you will have built, deployed, and validated a custom check that enforces a naming convention on tags. You will set up sample data, write a GraphQL query, implement the check logic in Python, configure it in `.infrahub.yml`, deploy it via a connected Git repository, and verify it runs against a proposed change. Checks in Infrahub allow you to validate data based on custom business logic. They are defined in [external repositories](/git-integration/overview.md) and can be developed and tested locally using [infrahubctl check](/infrahubctl/infrahubctl-check.md). Global vs. Targeted Checks If you want to know more about the differences between these two types, refer to the topic on [Checks & Validation](/checks/overview.md). **TL;DR:** * **Global Checks**: Apply to all instances of a specific object type in the database * **Targeted Checks**: Apply only to specific instances that you select (by adding them to a group) ## What you'll build[​](#what-youll-build "Direct link to What you'll build") In this guide, you'll build a check that: 1. Reads information about tag objects from the database 2. Validates that the tags follow specific naming conventions using regex patterns 3. Automatically runs when changes to tags are proposed ## Steps overview[​](#steps-overview "Direct link to Steps overview") 1. Prepare sample data in your Infrahub instance 2. Create a GraphQL query to fetch data from Infrahub 3. Implement a Python check that validates the data 4. Configure the check in an `.infrahub.yml` file 5. Test the check locally with infrahubctl 6. Deploy the check to Infrahub 7. Validate that the check works in Infrahub's CI pipeline ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before you begin, you'll need: * Basic knowledge of Infrahub, Python, GraphQL, YAML, and Git * Familiarity with Checks and how they work in Infrahub (see [Checks & Validation](/checks/overview.md)) * An Infrahub instance running locally or remotely * A Git repository connected to Infrahub (see [Connect a repository](/git-integration/connect-repository.md)) * [infrahubctl](/infrahubctl/infrahubctl.md) installed and configured locally * The repository cloned where you'll develop the check ## Setting up your workspace[​](#setting-up-your-workspace "Direct link to Setting up your workspace") Version Control & Branching We recommend creating a new branch in your Git repository for your check development to avoid affecting the main branch. In this guide, we'll use a branch named `create-tag-check`. ``` git checkout -b create-tag-check git push --set-upstream origin create-tag-check ``` ## Step 1: Preparing sample data[​](#step-1-preparing-sample-data "Direct link to Step 1: Preparing sample data") First, let's create some sample tag data to test our check against. warning This step is only necessary if you don't already have the objects affected by the check in your instance. important For targeted checks you will need to create a group that the check will target. Create three tags with different naming patterns: * `color-red` - Follows the "color-\[name]" pattern (will pass validation) * `color-green` - Follows the "color-\[name]" pattern (will pass validation) * `blue` - Doesn't follow the pattern (will fail our validation check) You can create these tags using the Infrahub UI or in batch using object file: * Targeted Check * Global Check For targeted checks, you will also need to create a group that includes the tags you want to validate. * Using Object File * Using the UI 1. Create a file named `groups.yml` in the `objects` directory of your repository: objects/groups.yml ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: CoreStandardGroup data: - name: colored_tags description: Group for colored tags ``` 2. Create a file named `tags.yml` in the `objects` directory of your repository: objects/tags.yml ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: BuiltinTag data: - name: "color-red" description: "The red tag" member_of_groups: - ["colored_tags"] - name: "color-green" description: "The green tag" member_of_groups: - ["colored_tags"] - name: "blue" description: "The blue tag" member_of_groups: - ["colored_tags"] ``` 2. Use infrahubctl to load the objects: ``` infrahubctl object load objects/groups.yml --branch=create-tag-check infrahubctl object load objects/tags.yml --branch=create-tag-check ``` 1. Open the **Infrahub UI** in your browser 2. Select the branch `create-tag-check` 3. Navigate to the **Groups** view (Object Management > Groups) 4. Create a group * **Name**: `colored_tags`, **Description**: `Group for colored tags` 5. Navigate to the **Tags** view (Other > Tags) 6. Create the tags * **Name**: `color-red`, **Description**: `The red tag`, **Member of Group**: `colored_tags` * **Name**: `color-green`, **Description**: `The green tag`, **Member of Group**: `colored_tags` * **Name**: `blue`, **Description**: `The blue tag`, **Member of Group**: `colored_tags` * Using Object File * Using the UI 1. Create a file named `tags.yml` in the `objects` directory of your repository: objects/tags.yml ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: BuiltinTag data: - name: "color-red" description: "The red tag" - name: "color-green" description: "The green tag" - name: "blue" description: "The blue tag" ``` 2. Use infrahubctl to load the objects: ``` infrahubctl object load objects/tags.yml --branch=create-tag-check ``` 1. Open the **Infrahub UI** in your browser 2. Select the branch `create-tag-check` 3. Navigate to the **Tags** view (Other > Tags) 4. Create the tags * **Name**: `color-red`, **Description**: `The red tag` * **Name**: `color-green`, **Description**: `The green tag` * **Name**: `blue`, **Description**: `The blue tag` success To verify the tags were created successfully, navigate to the Tags view in the Infrahub UI (Other > Tags), select the `create-tag-check` branch, and check that the tags appear in the list. ## Step 2: Create the GraphQL query[​](#step-2-create-the-graphql-query "Direct link to Step 2: Create the GraphQL query") Next, create a GraphQL query that fetches the data your check needs to process. ### Create a query to fetch the tag data[​](#create-a-query-to-fetch-the-tag-data "Direct link to Create a query to fetch the tag data") Create a `tags_query.gql` file in the `queries` directory of your repository: * Targeted Check * Global Check queries/tags\_query.gql ``` query TagsQuery($name: String!) { BuiltinTag(name__value: $name) { edges { node { name { value } description { value } } } } } ``` queries/tags\_query.gql ``` query TagsQuery { BuiltinTag { edges { node { name { value } description { value } } } } } ``` ### Test the query[​](#test-the-query "Direct link to Test the query") To test the query you can use **Infrahub's GraphQL Sandbox**. GraphQL Sandbox Access the sandbox by clicking your user icon in the bottom left corner and selecting **GraphQL Sandbox**. This tool allows you to run GraphQL queries and provides an interactive way to explore the schema. * Targeted Check * Global Check 1. Copy the above GraphQL query in the main section 2. In the **variables** section, add: ``` { "name": "blue" } ``` 3. Click the **Execute** button, this should return a response like: ``` { "data": { "BuiltinTag": { "edges": [ { "node": { "name": { "value": "blue" }, "description": { "value": "The blue tag" } } } ] } } } ``` 1. Copy the above GraphQL query in the main section 2. Click the **Execute** button, this should return a response like: ``` { "data": { "BuiltinTag": { "edges": [ { "node": { "name": { "value": "blue" }, "description": { "value": "The blue tag" } } }, { "node": { "name": { "value": "color-green" }, "description": { "value": "The green tag" } } }, { "node": { "name": { "value": "color-red" }, "description": { "value": "The red tag" } } } ] } } } ``` success You now have a working GraphQL query that retrieves the necessary data from Infrahub. You might want to keep the result of the query for later reference, as it will be useful when implementing the check. ## Step 3: Implement the check logic[​](#step-3-implement-the-check-logic "Direct link to Step 3: Implement the check logic") Now create a Python class that implements your check logic. The check makes sure that every tag's name follows the pattern `color-[name]`. The class must: * Inherit from `InfrahubCheck` from the [Python SDK](/python-sdk/introduction.md) * Define a `query` class attribute that references your GraphQL query by name * Implement a `validate()` method that processes data from your GraphQL query * Log errors using `log_error()` when validation fails 1. Create a file called `tags_check.py` in your `checks` directory: checks/tags\_check.py ``` import re from infrahub_sdk.checks import InfrahubCheck RE_TAG = re.compile(r"^color-[a-z]+") class ColorTagsCheck(InfrahubCheck): query = "tags_query" # This references the query name in .infrahub.yml def validate(self, data): for tag in data["BuiltinTag"]["edges"]: if not RE_TAG.match(tag["node"]["name"]["value"]): self.log_error( message=f"Invalid tag name: {tag['node']['name']['value']}. Tag names must follow pattern 'color-[name]'", object_id=tag["node"]["name"]["value"], object_type="BuiltinTag" ) ``` ## Step 4: Configure the check in the .infrahub.yml file[​](#step-4-configure-the-check-in-the-infrahubyml-file "Direct link to Step 4: Configure the check in the .infrahub.yml file") Now that you have your GraphQL query and Python check, edit your [.infrahub.yml](/git-integration/infrahub-yml.md) file to tie everything together. 1. Add the following configuration to the file `.infrahub.yml`: * Targeted Check * Global Check .infrahub.yml ``` --- queries: - name: tags_query file_path: queries/tags_query.gql check_definitions: - name: check_color_tags_name class_name: ColorTagsCheck file_path: checks/tags_check.py targets: colored_tags # This specifies the group to target parameters: name: "name__value" ``` .infrahub.yml ``` --- queries: - name: tags_query file_path: queries/tags_query.gql check_definitions: - name: check_color_tags_name class_name: ColorTagsCheck file_path: checks/tags_check.py ``` For a complete explanation of the `.infrahub.yml` file format, see the [infrahub.yml topic](/git-integration/infrahub-yml.md). 2. Verify the configuration Check that your `.infrahub.yml` file is correctly formatted by listing available checks: ``` infrahubctl check --list ``` success If successful, you'll see output like: * Targeted Check * Global Check ``` Python checks defined in repository: 1 check_color_tags_name (checks/tags_check.py::ColorTagsCheck) Target: colored_tags ``` ``` Python checks defined in repository: 1 check_color_tags_name (checks/tags_check.py::ColorTagsCheck) Target: -global- ``` ## Step 5: Test the check locally[​](#step-5-test-the-check-locally "Direct link to Step 5: Test the check locally") Before deploying your check to Infrahub, test it locally using the `infrahubctl` command-line tool. 1. First, verify that your check is correctly configured: ``` infrahubctl check --list ``` 2. Run the check against your branch by specifying its name: ``` infrahubctl check check_color_tags_name --branch=create-tag-check ``` success If successful, you'll see output like: * Targeted Check * Global Check ``` INFO HTTP Request: GET http://localhost:8000/api/schema?branch=create-tag-check2 "HTTP/1.1 200 OK" INFO HTTP Request: POST http://localhost:8000/graphql/create-tag-check2 "HTTP/1.1 200 OK" INFO HTTP Request: POST http://localhost:8000/graphql/create-tag-check2 "HTTP/1.1 200 OK" INFO HTTP Request: POST http://localhost:8000/graphql/create-tag-check2 "HTTP/1.1 200 OK" INFO HTTP Request: POST http://localhost:8000/graphql/create-tag-check2 "HTTP/1.1 200 OK" INFO tags_check::ColorTagsCheck: PASSED - {'name': 'color-red'} INFO HTTP Request: POST http://localhost:8000/graphql/create-tag-check2 "HTTP/1.1 200 OK" INFO tags_check::ColorTagsCheck: PASSED - {'name': 'color-green'} INFO HTTP Request: POST http://localhost:8000/graphql/create-tag-check2 "HTTP/1.1 200 OK" ERROR tags_check::ColorTagsCheck: FAILED - {'name': 'blue'} ERROR Invalid tag name: blue. Tag names must follow pattern 'color-[name]' ``` ``` INFO HTTP Request: POST http://localhost:8000/graphql/create-tag-check "HTTP/1.1 200 OK" ERROR tags_check::ColorTagsCheck: FAILED ERROR Invalid tag name: blue. Tag names must follow pattern 'color-[name]' ``` The error message shows that our check correctly identified the tag named "blue" as invalid because it doesn't match our required naming pattern. ## Step 6: Deploy the check to Infrahub[​](#step-6-deploy-the-check-to-infrahub "Direct link to Step 6: Deploy the check to Infrahub") Now that you've tested your check and confirmed it works locally, deploy it to Infrahub by pushing your code to the repository. ### Verify repository structure[​](#verify-repository-structure "Direct link to Verify repository structure") Ensure your repository has the following structure before committing: ``` your-repository/ ├── .infrahub.yml ├── checks/ │ └── tags_check.py └── queries/ └── tags_query.gql ``` note Depending on your organization, you might also have `schemas` and `objects` directories in your repository. ### Commit and push your code[​](#commit-and-push-your-code "Direct link to Commit and push your code") Upload your check code to the repository: ``` git add . git commit -m "Add tags naming convention check" git push ``` ### Confirm the check is imported[​](#confirm-the-check-is-imported "Direct link to Confirm the check is imported") After pushing your changes, confirm that the check is imported correctly by checking the Infrahub UI: 1. Open the **Infrahub UI** in your browser 2. Select the `create-tag-check` branch 3. Navigate to the **Check Definition view** (Actions > Check Definitions) 4. You should see your `check_color_tags_name` check listed there warning If you don't see your check, verify the repository status in the **Repository view** (Integrations > Git Repositories) and ensure the sync status is `synced`. success Your check is now successfully deployed to Infrahub. ## Step 7: Validate that the check works[​](#step-7-validate-that-the-check-works "Direct link to Step 7: Validate that the check works") Let's verify that your check correctly validates data by creating a proposed change from your development branch `create-tag-check` to `main`. ### Create a proposed change[​](#create-a-proposed-change "Direct link to Create a proposed change") 1. In the Infrahub UI, navigate to the **Proposed Changes** view 2. Click **New proposed change** 3. Create the **Proposed Change** with the following parameters: * Source branch: `create-tag-check` * Destination branch: `main` * Name: `Add tag naming convention check` ### Verify check results[​](#verify-check-results "Direct link to Verify check results") After the proposed change is created: 1. Navigate to the **Checks** tab of your proposed change 2. Wait for the **User** CI check to complete success You should see that the check has failed because the tag "blue" doesn't match the expected pattern. ![Check details in a proposed change](/assets/images/check_proposed_change-9eae6433a402959650c2febf94c00388.png) ### Fix the validation issue and merge[​](#fix-the-validation-issue-and-merge "Direct link to Fix the validation issue and merge") Now let's fix the issue with the invalid tag and complete the merge: 1. Navigate to **Tags** view (Other > Tags) 2. Select the `create-tag-check` branch 3. Delete the tag `blue` (as it doesn't follow the naming convention) * Alternatively, you could rename it to `color-blue` to match the pattern * For a targeted check you could also remove the tag `blue` from the `colored_tags` group 4. Return to the **Proposed Change** view 5. Navigate to the **Data** and **Files** tabs to review the changes 6. Navigate to the **Checks** 7. Hit the **Retry** button to re-run the check 8. Verify that all checks now pass 9. On the **Overview** tab, click the **Merge** button to merge your changes to main success Your check is now deployed to Infrahub in the `main` branch and ready to be used. Any future tag creations will be validated against your naming convention rule. ## Next steps[​](#next-steps "Direct link to Next steps") Now that you've created a basic check for data validation, you can expand your validation capabilities: * Create more complex checks that validate relationships between objects * Build checks that validate business logic across multiple object types * Add detailed error messages with recommendations to help users understand and fix validation failures success Checks are a powerful way to enforce business rules in your data. By implementing validation rules that match your organization's requirements, you ensure data consistency and prevent configuration errors before they reach production environments. --- # Build your first schema By the end of this tutorial you will have a working schema for network devices and interfaces, covering nodes, attributes, relationships, and generic abstractions. You'll load each version into a branch and verify the results. This tutorial uses a network device example to demonstrate key schema concepts. Real-world schemas involve additional complexity, but the patterns here apply broadly. For conceptual background on schemas and design patterns, see [About schema](/schema/overview.md). ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") * Infrahub running locally or in your environment * [`infrahubctl`](/infrahubctl/infrahubctl.md) command-line tool installed * Basic understanding of YAML syntax Schema validation Enable [schema validation](/reference/schema-validation.md) in your editor to catch errors while developing schema files. ## Alternative learning resources[​](#alternative-learning-resources "Direct link to Alternative learning resources") ### Hands-on lab[​](#hands-on-lab "Direct link to Hands-on lab") Try the interactive [Infrahub Schema Introduction](https://opsmill.instruqt.com/pages/labs) lab for hands-on practice with schemas. ### Video tutorials[​](#video-tutorials "Direct link to Video tutorials") Learn advanced schema concepts in this livestream recording: ## Step 1: Create basic nodes with attributes[​](#step-1-create-basic-nodes-with-attributes "Direct link to Step 1: Create basic nodes with attributes") Create a file named `schema_guide.yml` in a location you can access easily, such as `/tmp/schema_guide.yml`. Start by defining two basic nodes in the `Network` namespace: * `Device`: Represents network devices with hostname and model attributes * `Interface`: Represents network interfaces with name and description attributes ``` --- version: "1.0" nodes: - name: Device namespace: Network human_friendly_id: ['hostname__value'] attributes: - name: hostname kind: Text unique: true - name: model kind: Text - name: Interface namespace: Network attributes: - name: name kind: Text - name: description kind: Text optional: true ``` Human-friendly IDs The `human_friendly_id` on the `hostname` attribute lets you use the hostname value instead of the system-generated ID in queries and mutations. ### Load the schema into Infrahub[​](#load-the-schema-into-infrahub "Direct link to Load the schema into Infrahub") Create a new branch for your schema changes: ``` infrahubctl branch create network-device-schema ``` Load the schema into the branch: ``` infrahubctl schema load --branch network-device-schema /tmp/schema_guide.yml ``` View your schema in the [Web UI](http://localhost:8000/schema?branch=network-device-schema) under Object Management > Schemas. ![Schema page screenshot](/assets/images/create_schema_1-0c77748efef18604d3451c6b3609991d.png) ### Create test data[​](#create-test-data "Direct link to Create test data") Test your schema by creating a device and interface: * GraphQL * Web interface * cURL Open the GraphQL sandbox (bottom left of the web interface) and execute: ``` mutation { NetworkDeviceCreate(data: {hostname: {value: "atl1-edge1"}, model: {value: "Cisco ASR1002-HX"}}) { ok object { id } } NetworkInterfaceCreate(data: {name: {value: "Ethernet1"}, description: {value: "WAN interface"}}) { ok object { id } } } ``` 1. Navigate to **Objects > Device** in the left menu 2. Click **Add Device** 3. Enter `atl1-edge1` as Hostname and `Cisco ASR1002-HX` as Model 4. Click **Save** 5. Navigate to **Objects > Interface** 6. Create an interface with `Ethernet1` as Name and `WAN interface` as Description Replace the IP address and API key with your actual values: ``` curl -X POST http://localhost:8000/graphql/network-device-schema \ -H "Content-Type: application/json" \ -H "X-INFRAHUB-KEY: 1802eed5-eeb7-cc45-2e4d-c51de9d66cba" \ -d '{"query": "mutation { NetworkDeviceCreate(data: {hostname: {value: \"atl1-edge1\"}, model: {value: \"Cisco ASR1002-HX\"}}) { ok object { id } } NetworkInterfaceCreate(data: {name: {value: \"Ethernet1\"}, description: {value: \"WAN interface\"}}) { ok object { id } } }"}' ``` Verify your objects were created by navigating to **Objects** and selecting **Device** or **Interface**. ## Step 2: Add relationships between nodes[​](#step-2-add-relationships-between-nodes "Direct link to Step 2: Add relationships between nodes") Connect your device and interface nodes with relationships. This creates meaningful associations between your data. Add these relationships to your schema: 1. **Device to interfaces**: One device can have many interfaces (component relationship) 2. **Interface to device**: Each interface belongs to one device (parent relationship) Replace your `schema_guide.yml` content with: ``` --- version: "1.0" nodes: - name: Device namespace: Network human_friendly_id: ['hostname__value'] attributes: - name: hostname kind: Text unique: true - name: model kind: Text relationships: - name: interfaces cardinality: many peer: NetworkInterface kind: Component - name: Interface namespace: Network attributes: - name: name kind: Text - name: description kind: Text optional: true relationships: - name: device cardinality: one peer: NetworkDevice optional: false kind: Parent ``` ### Load the updated schema[​](#load-the-updated-schema "Direct link to Load the updated schema") Create a new branch for the relationship changes: ``` infrahubctl branch create network-device-relations ``` Load the updated schema: ``` infrahubctl schema load --branch network-device-relations /tmp/schema_guide.yml ``` ### Test the relationships[​](#test-the-relationships "Direct link to Test the relationships") Create connected objects using this GraphQL mutation: ``` mutation { NetworkDeviceCreate(data: {hostname: {value: "atl1-edge1"}, model: {value: "Cisco ASR1002-HX"}}) { ok object { id } } NetworkInterfaceCreate(data: {name: {value: "Ethernet1"}, description: {value: "WAN interface"}, device: {hfid: "atl1-edge1"}}) { ok object { id } } } ``` In the Web UI, view the device details to see the relationship to the Ethernet1 interface. ![Schema page screenshot](/assets/images/create_schema_3-cc070529d09bc7eb1df4e51fe3a556eb.png) ## Step 3: Create generic nodes for abstraction[​](#step-3-create-generic-nodes-for-abstraction "Direct link to Step 3: Create generic nodes for abstraction") Real network devices have different interface types with shared and unique characteristics. Use generic nodes to model this abstraction. Physical interfaces (like Ethernet1) have properties like speed and cable connections. Logical interfaces (like Vlan1) don't have these physical properties but share common attributes like name and description. ### Create a generic interface[​](#create-a-generic-interface "Direct link to Create a generic interface") Replace your schema with this structure using generics: ``` --- version: "1.0" generics: - name: Interface namespace: Network attributes: - name: name kind: Text - name: description kind: Text optional: true relationships: - name: device cardinality: one peer: NetworkDevice kind: Parent optional: false nodes: - name: Device namespace: Network human_friendly_id: ['hostname__value'] attributes: - name: hostname kind: Text unique: true - name: model kind: Text relationships: - name: interfaces cardinality: many peer: NetworkInterface kind: Component - name: PhysicalInterface namespace: Network inherit_from: - NetworkInterface attributes: - name: speed kind: Number - name: LogicalInterface namespace: Network inherit_from: - NetworkInterface ``` ### Load the generic schema[​](#load-the-generic-schema "Direct link to Load the generic schema") Create a new branch for generic changes: ``` infrahubctl branch create network-device-generics ``` Load the schema: ``` infrahubctl schema load --branch network-device-generics /tmp/schema_guide.yml ``` ### Test different interface types[​](#test-different-interface-types "Direct link to Test different interface types") Create both physical and logical interfaces: ``` mutation { NetworkDeviceCreate(data: {hostname: {value: "atl1-edge1"}, model: {value: "Cisco ASR1002-HX"}}) { ok object { id } } NetworkPhysicalInterfaceCreate(data: {name: {value: "Ethernet1"}, description: {value: "WAN interface"}, speed: {value: 1000000000}, device: {hfid: "atl1-edge1"}}) { ok object { id } } NetworkLogicalInterfaceCreate(data: {name: {value: "Vlan1"}, description: {value: "SVI for VLAN 1"}, device: {hfid: "atl1-edge1"}}) { ok object { id } } } ``` View the device in the Web UI to see both interface types connected to the same device. ![Schema page screenshot](/assets/images/create_schema_generics-eecd428f91e064e125bd10e3edfe2fcd.png) ## Step 4: Improve the schema with migrations[​](#step-4-improve-the-schema-with-migrations "Direct link to Step 4: Improve the schema with migrations") Refine your schema using Infrahub's schema migration features. These changes demonstrate how to evolve schemas while preserving existing data. Make these improvements: 1. Add `mtu` and `enabled` attributes to the generic `NetworkInterface` 2. Remove the `description` attribute from the generic `NetworkInterface` 3. Set a default value for the `speed` attribute of `NetworkPhysicalInterface` 4. Rename the `model` attribute to `device_type` 5. Add labels for better user experience 6. Define uniqueness constraints ### Get the attribute ID for migration[​](#get-the-attribute-id-for-migration "Direct link to Get the attribute ID for migration") Attribute ID required To rename an attribute, you need its current ID. Find the `model` attribute ID on the [NetworkDevice schema page](http://localhost:8000/schema?branch=network-device-generics\&kind=NetworkDevice) in the Web UI. Replace your schema content (update the `id` value with the actual model attribute ID): ``` --- version: "1.0" generics: - name: Interface namespace: Network attributes: - name: name kind: Text label: Name - name: description state: absent kind: Text optional: true label: Description - name: mtu kind: Number label: MTU optional: false default_value: 1500 - name: enabled label: Enabled kind: Boolean optional: false default_value: false relationships: - name: device label: Device cardinality: one peer: NetworkDevice kind: Parent optional: false nodes: - name: Device namespace: Network human_friendly_id: ['hostname__value'] attributes: - name: hostname kind: Text label: Hostname unique: true - name: device_type label: Device Type kind: Text id: 17bcf8a7-9c03-4a6a-3295-c51345cb1c33 relationships: - name: interfaces label: Interfaces cardinality: many peer: NetworkInterface kind: Component - name: PhysicalInterface namespace: Network uniqueness_constraints: - ["device", "name__value"] inherit_from: - NetworkInterface attributes: - name: speed label: Speed (bps) kind: Number default_value: 1000000000 - name: LogicalInterface namespace: Network uniqueness_constraints: - ["device", "name__value"] inherit_from: - NetworkInterface ``` ### Preview schema changes[​](#preview-schema-changes "Direct link to Preview schema changes") Use the check command to see what changes will be applied: ``` infrahubctl schema check --branch network-device-generics /tmp/schema_guide.yml ``` This shows a diff of changes before applying them. ### Apply the migration[​](#apply-the-migration "Direct link to Apply the migration") Load the improved schema: ``` infrahubctl schema load --branch network-device-generics /tmp/schema_guide.yml ``` Migration complete Infrahub automatically migrates existing data to match the new schema structure. View the updated schema in the [Web UI](http://localhost:8000/schema?branch=network-device-generics). ## Next steps[​](#next-steps "Direct link to Next steps") Now that you've created a basic schema: * Learn about [schema validation](/reference/schema-validation.md) for development workflows * Explore [Nodes & attributes](/schema/nodes-and-attributes.md) for advanced concepts * Explore [Relationships](/schema/relationships.md) for relationship kinds and direction * Learn how to [create and load schema files](/schema/create-and-load.md) from existing sources * Set up [external repositories](/git-integration/connect-repository.md) to manage schemas with Git --- # Build chained generators By the end of this tutorial you will have wired two layers of [modular Generators](/generators/modular.md) together — fabric → pod → rack — using a checksum attribute and Infrahub's [event framework](/events/overview.md). You will have added a `GeneratorTarget` generic with a `checksum` attribute to your downstream node kinds, implemented a `GeneratorMixin` that writes the checksum to downstream targets as the last step of `generate()`, built downstream Generators that validate upstream completeness before doing any work, and created `CoreGeneratorAction` and `CoreNodeTriggerRule` objects so the chain runs automatically when an upstream layer finishes. You will leave with a concrete model of how a checksum makes Generator chaining idempotent — re-runs that produce the same output do not re-trigger the next layer. **Prerequisites:** * Familiarity with [Generators](/generators/overview.md) and [Build your first generator](/learn/tutorials/generators/build-your-first-generator.md) * Understanding of the [modular Generators concept](/generators/modular.md) * Basic understanding of [event rules and actions](/events/overview.md) ## How the pattern works[​](#how-the-pattern-works "Direct link to How the pattern works") **The problem:** Generator A creates objects that Generator B depends on. Generator B's targets exist before A runs, but B should only execute after A has finished its work. **The solution:** Generator A computes a hash (checksum) of all the node IDs it created or touched during execution. It writes this checksum to an attribute on Generator B's target objects. A `CoreNodeTriggerRule` watches for checksum changes on those targets and fires a `CoreGeneratorAction` to run Generator B. **Why a checksum?** It's idempotent. If Generator A runs again and produces the same output, the checksum doesn't change, so Generator B doesn't re-trigger. If the output differs (new nodes, removed nodes), the checksum changes and execution continues to the next layer. This makes re-runs safe by default. ``` Generator A finishes → calculates checksum of all created/touched node IDs → writes checksum to each downstream target (e.g., pod.checksum = "abc123") → Infrahub emits infrahub.node.updated event for each target → CoreNodeTriggerRule matches: kind=NetworkPod, attribute=checksum, value_match=any → CoreGeneratorAction runs Generator B for the updated target ``` ## Step 1: Add a checksum attribute to target objects[​](#step-1-add-a-checksum-attribute-to-target-objects "Direct link to Step 1: Add a checksum attribute to target objects") To signal downstream Generators, the target objects need an attribute that the upstream Generator can write to. The recommended approach is to define a schema generic with a `checksum` attribute and have target nodes inherit from it. ### Define the generic[​](#define-the-generic "Direct link to Define the generic") ``` # schemas/generator.yml version: "1.0" generics: - name: Target namespace: Generator include_in_menu: false attributes: - name: checksum kind: Text optional: true ``` ### Inherit from the generic[​](#inherit-from-the-generic "Direct link to Inherit from the generic") Any node that an upstream Generator should be able to trigger inherits from `GeneratorTarget`: ``` # schemas/logical_design.yml nodes: - name: Pod namespace: Network inherit_from: - NetworkBuildingBlock - GeneratorTarget # Adds the checksum attribute # ... rest of the node definition ``` **Why a generic?** It keeps the pattern reusable. Any node kind that participates in a modular Generator setup just inherits from `GeneratorTarget` — no need to manually add checksum attributes to each schema. ## Step 2: Calculate and write the checksum in your Generator[​](#step-2-calculate-and-write-the-checksum-in-your-generator "Direct link to Step 2: Calculate and write the checksum in your Generator") After the Generator creates or modifies its objects, it needs to compute a checksum and write it to the downstream targets. This should be the **last step** in your `generate()` method — only signal completion after all work is done. ### The checksum mixin[​](#the-checksum-mixin "Direct link to The checksum mixin") The tracking context (`self.client.group_context`) automatically collects the IDs of all nodes and groups the Generator interacted with during execution. These make an ideal checksum input. ``` # src/your_bundle/generator.py import hashlib class GeneratorMixin: def calculate_checksum(self) -> str: """Compute a checksum from all node IDs touched during this generator run.""" related_ids = ( self.client.group_context.related_group_ids + self.client.group_context.related_node_ids ) sorted_ids = sorted(related_ids) joined = ",".join(sorted_ids) return hashlib.sha256(joined.encode("utf-8")).hexdigest() ``` ### Using the mixin in a Generator[​](#using-the-mixin-in-a-generator "Direct link to Using the mixin in a Generator") ``` # generators/generate_fabric.py from infrahub_sdk.generator import InfrahubGenerator from your_bundle.generator import GeneratorMixin from your_bundle.protocols import NetworkPod class FabricGenerator(InfrahubGenerator, GeneratorMixin): async def generate(self, data: dict) -> None: fabric_id = data["NetworkFabric"]["edges"][0]["node"]["id"] # ... create fabric-level objects (super spines, IP pools, etc.) ... # Last step: signal downstream targets await self.update_checksum(fabric_id) async def update_checksum(self, fabric_id: str) -> None: pods = await self.client.filters(kind=NetworkPod, parent__ids=[fabric_id]) checksum = self.calculate_checksum() for pod in pods: if pod.checksum.value != checksum: pod.checksum.value = checksum await pod.save(allow_upsert=True) ``` **Key points:** * The `if pod.checksum.value != checksum` guard prevents unnecessary saves — and unnecessary triggers — when the Generator is re-run with the same result. * `update_checksum()` must be the **last step** in `generate()`. Only signal downstream after all work is complete. * The checksum naturally reflects everything the Generator touched, because it's derived from the SDK tracking context. ## Step 3: Validate upstream dependencies before generating[​](#step-3-validate-upstream-dependencies-before-generating "Direct link to Step 3: Validate upstream dependencies before generating") In a modular Generator setup there is no central orchestrator sequencing execution. A checksum change fires a trigger per target — which means a downstream Generator could start before the upstream Generator has finished creating all its objects. Each Generator must therefore protect itself by validating that upstream work is complete before proceeding. Without this validation, a Generator might run against partial data and produce incomplete results or cryptic errors. ### The pattern[​](#the-pattern "Direct link to The pattern") Before doing its work, a downstream Generator should: 1. Query for expected upstream objects (for example, spine switches created by the pod Generator) 2. Compare the actual count against the expected count from configuration 3. Raise a clear `RuntimeError` if validation fails — don't proceed with partial data ### Example: PodGenerator validating the fabric layer[​](#example-podgenerator-validating-the-fabric-layer "Direct link to Example: PodGenerator validating the fabric layer") ``` # generators/generate_pod.py from infrahub_sdk.generator import InfrahubGenerator from your_bundle.generator import GeneratorMixin from your_bundle.protocols import NetworkDevice class PodGenerator(InfrahubGenerator, GeneratorMixin): async def generate(self, data: dict) -> None: # ... extract pod data from query ... # Guard: skip unsupported pod roles if self.pod_role in EXCLUDED_POD_ROLES: raise ValueError( f"Cannot run pod generator on {self.pod_name}: " f"{self.pod_role} is not supported by the generator!" ) # Validate upstream: are all super spine switches present? super_spine_switches = await self.client.filters( kind=NetworkDevice, pod__ids=[fabric_pod_id], role__value="super_spine" ) if self.expected_super_spine_count != len(super_spine_switches): raise RuntimeError( f"Cannot start pod generator on {self.pod_name}: " f"the fabric doesn't seem to be fully generated yet!" ) # Validate required configuration if not self.pod_spine_switch_template: raise RuntimeError( f"Cannot start pod generator on {self.pod_name}: " f"no spine switch template defined!" ) # ... proceed with generation ... # Last step: write checksum to downstream targets await self.update_checksum(self.pod_id) ``` ### Example: RackGenerator validating the pod layer[​](#example-rackgenerator-validating-the-pod-layer "Direct link to Example: RackGenerator validating the pod layer") ``` # generators/generate_rack.py from infrahub_sdk.generator import InfrahubGenerator from your_bundle.protocols import NetworkDevice class RackGenerator(InfrahubGenerator): async def generate(self, data: dict) -> None: # ... extract rack data from query ... # Validate upstream: are all spine switches present? spine_switches = await self.client.filters( kind=NetworkDevice, pod__ids=[self.pod_id], role__value="spine" ) if self.expected_spine_count != len(spine_switches): raise RuntimeError( f"Cannot start rack generator on {self.rack_name}: " f"the pod doesn't seem to be fully generated!" ) # ... proceed with generation ... ``` ### Why this matters[​](#why-this-matters "Direct link to Why this matters") * **Checksum fires per-target.** A pod's checksum could update before the fabric Generator has created all super spines. Without validation, the pod Generator would run against incomplete data. * **Compare count vs. expected.** The target's schema typically stores the expected count (for example, `amount_of_super_spines` on the fabric node). Query for actual objects and compare. * **Fail with a clear error.** Use `RuntimeError` with a message that identifies the target and what's missing. This shows up in task logs and makes debugging straightforward. * **This replaces orchestration.** Each Generator polices itself. No need for a central controller to sequence execution — if the upstream layer isn't ready, the Generator fails safely and will succeed on the next trigger. ### Two types of validation[​](#two-types-of-validation "Direct link to Two types of validation") | Type | What to check | Example | | -------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------ | | **Upstream completeness** | Are the expected objects present? | Compare `len(super_spine_switches)` against `expected_super_spine_count` | | **Required configuration** | Are necessary templates, pools, or settings defined? | Check that `pod_spine_switch_template` is not `None` | Both checks should happen at the top of `generate()`, before any objects are created. ## Step 4: Create trigger rules and actions[​](#step-4-create-trigger-rules-and-actions "Direct link to Step 4: Create trigger rules and actions") With the checksum being written to downstream targets, you need trigger rules to detect the change and fire the next Generator. You can define these as object files in your repository (recommended) or create them via the UI. ### Using object files (recommended)[​](#using-object-files-recommended "Direct link to Using object files (recommended)") Object files make trigger configuration version-controlled alongside your Generators. #### Define Generator actions[​](#define-generator-actions "Direct link to Define Generator actions") ``` # objects/triggers.yml --- apiVersion: infrahub.app/v1 kind: Object spec: kind: CoreGeneratorAction data: - name: run-pod-generator generator: generate-pod - name: run-rack-generator generator: generate-rack ``` #### Define trigger rules[​](#define-trigger-rules "Direct link to Define trigger rules") ``` # objects/triggers.yml (continued) --- apiVersion: infrahub.app/v1 kind: Object spec: kind: CoreNodeTriggerRule data: - name: trigger-pod-generator-checksum branch_scope: "other_branches" node_kind: NetworkPod mutation_action: "updated" action: run-pod-generator matches: kind: CoreNodeTriggerAttributeMatch data: - attribute_name: checksum value_match: any - name: trigger-rack-generator-checksum branch_scope: "other_branches" node_kind: LocationRack mutation_action: "updated" action: run-rack-generator matches: kind: CoreNodeTriggerAttributeMatch data: - attribute_name: checksum value_match: any ``` #### Reference in .infrahub.yml[​](#reference-in-infrahubyml "Direct link to Reference in .infrahub.yml") ``` # .infrahub.yml objects: - file_path: objects/triggers.yml ``` **Key configuration explained:** | Field | Value | Why | | ----------------- | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `branch_scope` | `"other_branches"` | **Recommended.** Triggers only fire on non-default branches (within proposed changes). This means all Generators run in a branch where results can be reviewed before merging. Alternatives: `"all_branches"` (fires everywhere) or `"default_branch"` (fires only on main). | | `mutation_action` | `"updated"` | Fires when the target node is updated (the checksum write counts as an update). | | `value_match` | `any` | Fires on any change to the attribute, regardless of the new value. | | `action` | name of the action | References the `CoreGeneratorAction` by name. | Trigger rules loaded from object files are **active by default** — no separate activation step is needed. ### Beyond checksums: triggering on user-facing attributes[​](#beyond-checksums-triggering-on-user-facing-attributes "Direct link to Beyond checksums: triggering on user-facing attributes") The checksum handles the connection between Generators, but you may also want Generators to re-run when a user changes a relevant attribute directly. For example, if a user changes the number of spines on a pod: ``` - name: trigger-pod-generator-amount-of-spines branch_scope: "other_branches" node_kind: NetworkPod mutation_action: "updated" action: run-pod-generator matches: kind: CoreNodeTriggerAttributeMatch data: - attribute_name: amount_of_spines value_match: any - name: trigger-pod-generator-role branch_scope: "other_branches" node_kind: NetworkPod mutation_action: "updated" action: run-pod-generator matches: kind: CoreNodeTriggerAttributeMatch data: - attribute_name: role value_match: any ``` This gives you a complete event-driven system: downstream Generators run automatically when upstream Generators finish, **and** when users make relevant changes in the UI. ### Repeat for each layer[​](#repeat-for-each-layer "Direct link to Repeat for each layer") For each pair of (upstream Generator, downstream Generator), define one `CoreGeneratorAction` and one or more `CoreNodeTriggerRule` entries. A three-layer setup (fabric to pod to rack) needs two actions and at least two trigger rules. ### Using the UI (alternative)[​](#using-the-ui-alternative "Direct link to Using the UI (alternative)") If you prefer to create trigger rules via the Infrahub UI, follow the [creating event trigger rules and actions](/events/overview.md) guide. The configuration is the same: 1. Create a **Generator Action** (Actions > Create > Generator Action) pointing to the downstream Generator 2. Create a **Node Trigger Rule** (Trigger Rules > Create > Node Trigger) for the downstream target kind, with mutation action `updated` 3. Add an **Attribute Match** on the `checksum` attribute with value match `any` 4. Set the trigger to **active** UI-created triggers work identically but aren't version-controlled. ## Step 5: Disable CI-based execution (optional)[​](#step-5-disable-ci-based-execution-optional "Direct link to Step 5: Disable CI-based execution (optional)") Generators that are entirely event-driven should have their CI execution disabled to avoid double-runs: ``` # .infrahub.yml generator_definitions: - name: generate-pod file_path: "./generators/generate_pod.py" query: generate_pod targets: pods parameters: pod_name: name__value class_name: PodGenerator convert_query_response: false execute_in_proposed_change: false # Triggered by events, not CI execute_after_merge: false # Triggered by events, not CI ``` Set both `execute_in_proposed_change` and `execute_after_merge` to `false` for any Generator that's triggered via the checksum/trigger pattern. This prevents the Generator from running both as a CI check and as an event-triggered action. info This is a design choice. Some teams keep CI execution enabled for the first Generator (the entry point) and only use event triggers for downstream Generators. ## Step 6: Test the modular setup[​](#step-6-test-the-modular-setup "Direct link to Step 6: Test the modular setup") ### Run the first Generator manually[​](#run-the-first-generator-manually "Direct link to Run the first Generator manually") ``` infrahubctl generator generate-fabric --branch=my-test-branch fabric_name=my-fabric ``` ### Verify the checksum was written[​](#verify-the-checksum-was-written "Direct link to Verify the checksum was written") Check the downstream targets in the UI or via GraphQL: ``` query { NetworkPod(parent__name__value: "my-fabric") { edges { node { name { value } checksum { value } } } } } ``` You should see a checksum value on each pod. ### Verify downstream Generators triggered[​](#verify-downstream-generators-triggered "Direct link to Verify downstream Generators triggered") 1. Check if the downstream Generator ran by looking at the objects it should have created 2. Inspect the task logs in Infrahub for Generator execution entries 3. For each subsequent layer, verify that: * The upstream Generator wrote checksums to downstream targets * The trigger fired and the next Generator ran * The expected objects were created ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") | Symptom | Likely cause | Fix | | ------------------------------------------ | ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Trigger doesn't fire | Trigger rule is not active, or `branch_scope` doesn't match the branch you're working on | Check trigger rule status. If `branch_scope` is `other_branches`, the trigger won't fire on the default branch — work in a non-default branch. | | Generator runs but creates nothing | Downstream Generator's upstream validation is failing | Check Generator logs. The Generator likely detects that upstream objects are incomplete and raises an error. | | Generator runs in a loop | The downstream Generator modifies an attribute that triggers the upstream Generator | Ensure Generators only write checksums to *downstream* targets, never upstream. The checksum guard (`if checksum != old`) should also prevent repeated triggers for the same output. | | Checksum unchanged after re-run | Generator produced the same output as before | This is **expected behavior** — execution correctly stops when nothing changed. | | Trigger fires but Generator errors | Generator definition name doesn't match the action's Generator reference | Verify that the `generator` field in `CoreGeneratorAction` matches the `name` field in your Generator definition in `.infrahub.yml`. | | Generator fails with "not fully generated" | Upstream validation caught incomplete data — the upstream Generator hasn't finished yet | This is the validation from Step 3 working as intended. The Generator will succeed on the next trigger once the upstream layer is complete. | ## What you learned[​](#what-you-learned "Direct link to What you learned") You now have a working two-stage Generator chain: * A `GeneratorTarget` generic that adds a `checksum` attribute to any downstream node kind * A `GeneratorMixin` that derives a checksum from the SDK tracking context and writes it to downstream targets only as the last step of `generate()` * Downstream Generators that validate upstream completeness before doing any work, failing fast and clearly when the upstream layer isn't ready * `CoreGeneratorAction` and `CoreNodeTriggerRule` objects loaded from object files, scoped to non-default branches so cascades run inside proposed changes ## Next steps[​](#next-steps "Direct link to Next steps") * See [best practices for modular Generators](/generators/modular-best-practices.md) for additional debugging techniques, pool scoping strategies, and operational guidance for running cascades in production * Revisit [modular Generators](/generators/modular.md) for the conceptual framing of why splitting Generators across layers is worth the overhead --- # Build your first generator By the end of this tutorial you will have built a working Generator end-to-end: modeled two object kinds, written a GraphQL query, implemented a Python Generator class, registered it in `.infrahub.yml`, run it locally with `infrahubctl`, and verified it runs automatically as part of a proposed change. You will leave with a `Widget` and `Resource` schema loaded into Infrahub and a Generator that creates `Resource` objects from each `Widget`'s `count` attribute. For conceptual background, see [Generators](/generators/overview.md). For a recipe-form how-to without the running example, see [Build a generator](/generators/build.md). ## What you'll build[​](#what-youll-build "Direct link to What you'll build") You'll build a basic Generator that: 1. Reads information about Widget objects from the database 2. Creates a number of Resource objects based on each Widget's count property 3. Automatically runs when changes to Widget objects are proposed ## Steps overview[​](#steps-overview "Direct link to Steps overview") 1. Prepare the data model by creating a schema and objects 2. Create a GraphQL query to fetch data from Infrahub 3. Implement a Python Generator that creates the data 4. Tie everything together in an `.infrahub.yml` file 5. Test the Generator locally with infrahubctl 6. Deploy the Generator to Infrahub 7. Validate that the Generator works in Infrahub's CI pipeline ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before you begin, you'll need: * Basic knowledge of Infrahub, Python, GraphQL, YAML, and Git * Familiarity with Generators and how they work in Infrahub (see [Generators](/generators/overview.md)) * An Infrahub instance running locally or remotely * A Git repository connected to Infrahub (see [Connect a repository](/git-integration/connect-repository.md)) * [infrahubctl](/infrahubctl/infrahubctl.md) installed and configured locally * The repository cloned locally where you'll develop the Generator ## Step 1: Setting up the data model[​](#step-1-setting-up-the-data-model "Direct link to Step 1: Setting up the data model") In this step, you'll create the data structures needed for your Generator, including a schema, group, and sample objects that the Generator will process. ### Create a new branch[​](#create-a-new-branch "Direct link to Create a new branch") Version Control & Branching We recommend creating a new branch in your Git repository for your Generator development to avoid affecting the main branch. In this tutorial, we'll use a branch named `create-widget-generator`. ``` git checkout -b create-widget-generator ``` ### Create group[​](#create-group "Direct link to Create group") Object file or UI If you prefer a "as code" approach, use the object file method. Otherwise, you can create the group directly in the Infrahub UI. * Using Object File * Using the UI 1. Create a file named `groups.yml` in the `objects` directory of your repository: objects/groups.yml ``` --- apiVersion: infrahub.app/v1 kind: Object spec: kind: CoreStandardGroup data: - name: "widgets" ``` 2. Update the `.infrahub.yml` file to include the new object file: .infrahub.yml ``` --- objects: - file_path: objects/groups.yml ``` 3. Commit the changes to your repository: ``` git add objects/groups.yml .infrahub.yml git commit -m "Add widgets group" git push ``` 1. Open the **Infrahub UI** in your browser 2. Select the branch `create-widget-generator` 3. Navigate to the **Groups** view (Object Management > Groups) 4. Create a **Standard group** named `widgets` success To verify the group was created successfully, navigate to the Groups view in the Infrahub UI and check that the `widgets` group appears in the list. ### Create and load schema[​](#create-and-load-schema "Direct link to Create and load schema") Schema Creation This step is only necessary if you don't already have the schemas affected by the Generator in your instance. 1. Create a `widgets.yml` file in the `schemas` directory of your repository: schemas/widgets.yml ``` # yaml-language-server: $schema=https://schema.infrahub.app/infrahub/schema/latest.json --- version: '1.0' nodes: - name: Widget namespace: Test label: Widget human_friendly_id: - name__value display_label: "{{ name__value }}" attributes: - name: name kind: Text unique: true - name: count kind: Number - name: Resource namespace: Test label: Resource human_friendly_id: - name__value display_label: "{{ name__value }}" attributes: - name: name kind: Text unique: true ``` Schema Loading Choose the method you already use to manage schema changes in your Infrahub deployment. * Using Git Integration * Using infrahubctl 2. Update the `.infrahub.yml` file to include the new schema file: .infrahub.yml ``` --- schemas: - schemas/widgets.yml ``` 3. Commit and push the changes to your repository 2) Load the following schema using the [infrahubctl schema](/infrahubctl/infrahubctl-schema.md) command. ``` infrahubctl schema load schemas/widgets.yml --branch=create-widget-generator ``` ``` schema 'widgets.yml' loaded successfully 1 schema processed in 8.453 seconds. ``` success You should see new entries for `Widget` and `Resource` in the left-hand side menu of the Infrahub UI. ### Create widget objects[​](#create-widget-objects "Direct link to Create widget objects") Now that you have the schema loaded, you need to create objects that will be used by the Generator: Object Creation This step is only necessary if you don't already have the objects affected by the Generator in your instance. Object Creation Method You can also use an object file to create the widgets, though it's less relevant here than for the group creation. 1. Open the **Infrahub UI** in your browser 2. Select the branch `create-widget-generator` 3. Navigate to the **Widget** view 4. Create a new widget: * Name: `widget1` * Count: `1` * Member of groups: `widgets` 5. Create a second widget: * Name: `widget2` * Count: `2` * Member of groups: `widgets` ## Step 2: Create the GraphQL query[​](#step-2-create-the-graphql-query "Direct link to Step 2: Create the GraphQL query") Next, create a GraphQL query that fetches the data your Generator needs to process. ### Create a query file[​](#create-a-query-file "Direct link to Create a query file") Create a `widget_query.gql` file in the `queries` directory of your repository: queries/widget\_query.gql ``` query Widgets($name: String!) { TestWidget(name__value: $name) { edges { node { __typename id name { value } count { value } } } } } ``` ### Test the query[​](#test-the-query "Direct link to Test the query") To test the query you can use **Infrahub's GraphQL Sandbox**. GraphQL Sandbox Access the sandbox by clicking your user icon in the bottom left corner and selecting **GraphQL Sandbox**. This tool allows you to run GraphQL queries and provides an interactive way to explore the schema. 1. Copy the above GraphQL query in the main section 2. In the **variables** section, add: ``` { "name": "widget1" } ``` 3. Click the **Execute** button, this should return a response like: ``` { "data": { "TestWidget": { "edges": [ { "node": { "__typename": "TestWidget", "id": "185526eb-2114-ce20-390a-c51aac78460a", "name": { "value": "widget1" }, "count": { "value": 1 } } } ] } } } ``` success You now have a working GraphQL query that retrieves the necessary data from Infrahub. You might want to keep the result of the query for later reference, as it will be useful when implementing the Generator. ## Step 3: Implement the Generator[​](#step-3-implement-the-generator "Direct link to Step 3: Implement the Generator") Now create a Python class that implements your Generator logic. The Generator creates `TestResource` objects equal to the widget's count value. The class must: * Inherit from `InfrahubGenerator` * Implement an async `generate()` method that processes data from your GraphQL query info If you aren't using `convert_query_response`, you can access the widget data directly from the `data` parameter passed to the `generate()` method. If you're working with `protocols`, you can create `TestResource` objects using strict typing. 1. Create a file named `widget_generator.py` in your `generators` directory: generators/widget\_generator.py ``` from infrahub_sdk.generator import InfrahubGenerator class WidgetGenerator(InfrahubGenerator): async def generate(self, data: dict) -> None: # Access the widget as an SDK object widget = self.nodes[0] # or self.store.get(data["TestWidget"]["edges"][0]["node"]["id"]) widget_name: str = widget.name.value widget_count: int = widget.count.value # Create resources based on widget count for count in range(1, widget_count + 1): payload = { "name": f"{widget_name.lower()}-{count}", } obj = await self.client.create(kind="TestResource", data=payload) await obj.save(allow_upsert=True) ``` ## Step 4: Tie everything together in the .infrahub.yml file[​](#step-4-tie-everything-together-in-the-infrahubyml-file "Direct link to Step 4: Tie everything together in the .infrahub.yml file") Now that you have your GraphQL query and Python Generator, edit your [.infrahub.yml](/git-integration/infrahub-yml.md) file to tie everything together. 1. Add the following configuration to the file `.infrahub.yml`: .infrahub.yml ``` --- generator_definitions: - name: widget_generator file_path: "generators/widget_generator.py" targets: widgets query: widget_query convert_query_response: true class_name: WidgetGenerator parameters: name: "name__value" queries: - name: widget_query file_path: "queries/widget_query.gql" ``` For a complete explanation of the `.infrahub.yml` file format, see the [infrahub.yml configuration](/git-integration/infrahub-yml.md). 2. Verify the configuration Check that your `.infrahub.yml` file is correctly formatted by listing available Generators: ``` infrahubctl generator --list ``` success If successful, you'll see output like: ``` Generators defined in repository: 1 widget_generator (generators/widget_generator.py::WidgetGenerator) Target: widgets ``` ## Step 5: Test the Generator locally[​](#step-5-test-the-generator-locally "Direct link to Step 5: Test the Generator locally") Before deploying your Generator to Infrahub, test it locally using the `infrahubctl` command-line tool. ### Run the Generator[​](#run-the-generator "Direct link to Run the Generator") Run the Generator on both of your widget objects: ``` infrahubctl generator widget_generator --branch=create-widget-generator name=widget1 infrahubctl generator widget_generator --branch=create-widget-generator name=widget2 ``` ### Verify the results[​](#verify-the-results "Direct link to Verify the results") 1. Select the branch `create-widget-generator` from the branch selector 2. Navigate to **Resource** objects 3. You should see: * One resource object for `widget1` (named `widget1-1`) * Two resource objects for `widget2` (named `widget2-1` and `widget2-2`) ![Generated resource objects](/assets/images/generator_pc_3-cdfaa19101c8ed4238c45fec7213a7a4.png) warning You might want to cleanup this development data before merging your changes into the main branch. success Your Generator worked as expected, it's now time to deploy it to Infrahub. ## Step 6: Deploy to Infrahub[​](#step-6-deploy-to-infrahub "Direct link to Step 6: Deploy to Infrahub") Now that you've tested your Generator, deploy it to Infrahub. ### Verify repository structure[​](#verify-repository-structure "Direct link to Verify repository structure") Ensure your repository has the following structure: Schemas & Objects Depending on your organization, you might also have `schemas` and `objects` directories in your repository. ``` your-repository/ ├── .infrahub.yml ├── generators/ │ └── widget_generator.py └── queries/ └── widget_query.gql ``` ### Commit and push your code[​](#commit-and-push-your-code "Direct link to Commit and push your code") Upload your Generator code to the repository: ``` git add . git commit -m "Add widget generator" git push ``` ### Confirm the Generator is imported[​](#confirm-the-generator-is-imported "Direct link to Confirm the Generator is imported") After pushing your changes, confirm that the Generator is imported correctly by checking the Infrahub UI. 1. Open the **Infrahub UI** in your browser 2. Select the `create-widget-generator` branch 3. Navigate to the **Generators Definition view** (Actions > Generator Definitions) 4. You should see your `widget_generator` listed there If you don't see it verify the status of the repository in the **Repository view** and ensure the sync status is `synced`. ### Proposed changes and merge[​](#proposed-changes-and-merge "Direct link to Proposed changes and merge") Now merge your development branch `create-widget-generator` into `main`. Merging Options We recommend using Infrahub's **Proposed Changes** feature to merge your changes. This allows you to review both data and code changes before merging. 1. Create a **Proposed Change** * Source branch: `create-widget-generator` * Destination branch: `main` * Name: `Add widget generator` 2. Navigate to the **Data** and **Files** tabs to review the changes 3. Back to the **Overview** tab, click the **Merge** button success Your Generator is now deployed to Infrahub in the `main` branch and ready to be used. ## Step 7: Run the Generator in Infrahub's CI pipeline[​](#step-7-run-the-generator-in-infrahubs-ci-pipeline "Direct link to Step 7: Run the Generator in Infrahub's CI pipeline") Now let's verify that your Generator automatically runs when new widgets are created: 1. Create a new branch named `add-widget-3` 2. Navigate to the **Widget** objects in the Infrahub UI 3. Create a new widget: * Name: `widget3` * Count: `3` * Member of groups: `widgets` 4. Create a **Proposed Change** * Source branch: `add-widget-3` * Destination branch: `main` * Name: `Add widget3` ### Check Generator results[​](#check-generator-results "Direct link to Check Generator results") 1. Navigate to the **Checks** tab of your proposed change 2. Wait for the **Generator** CI check to complete ![Generator CI Check](/assets/images/generator_pc_1-806e4bf23f6b989bd1dffb62ee863b2b.png) 3. Navigate to the **Data** tab 4. Click the **Refresh diff** button to see the resources created by your Generator ![Data Refresh showing generated resources](/assets/images/generator_pc_2-029c5257939535f8409cb76669efcc2b.png) success You should see three new resources (`widget3-1`, `widget3-2`, and `widget3-3`) automatically created by your Generator. ## What you learned[​](#what-you-learned "Direct link to What you learned") You now have a working Generator end-to-end: * A `Widget` and `Resource` schema loaded into Infrahub via the Git integration * A GraphQL query that selects a single Widget by name and reads its `count` * A Python `WidgetGenerator` class that creates one `Resource` per count, using `allow_upsert=True` for safe re-runs * A `.infrahub.yml` definition that wires the query, the Python class, and the `widgets` target group together * Confirmed behavior both via `infrahubctl generator` locally and via the Generator CI check on a proposed change ## Next steps[​](#next-steps "Direct link to Next steps") Now that you've created a basic Generator, you can: * Build chained Generators that span multiple layers of a hierarchy — see [Build chained generators](/learn/tutorials/generators/build-chained-generators.md) * Read the [best practices for modular Generators](/generators/modular-best-practices.md) for idempotency, deterministic naming, and debugging cascades * Add unit tests to your Generator to ensure it behaves as expected * Harden your Generator by adding error handling and logging Real-World Example Want to see how Generators can be used in production? Read our blog post on [How to Turn Your Source of Truth into a Service Factory](https://www.opsmill.com/how-to-turn-your-source-of-truth-into-a-service-factory/). --- # Organize objects with groups By the end of this tutorial you will have created a group, added two objects to it, and queried the result end-to-end. You'll leave with a concrete mental model of how groups work and where to go next. This tutorial uses `BuiltinTag` objects so you can follow along without any special schema. The same steps apply to any object type. ## What you will need[​](#what-you-will-need "Direct link to What you will need") * A running Infrahub instance (local or remote). * Permission to create and modify groups. * Two tags to add as members. If you don't have tags yet, create two called `red` and `blue` before starting. ## Step 1 — Create a new group[​](#step-1--create-a-new-group "Direct link to Step 1 — Create a new group") You'll create a Standard group named `TagConfigGroup`. A Standard group is the general-purpose type you create and manage yourself. * Web Interface * GraphQL * Python SDK Navigate to **Object Management** → **Groups** in the left menu. Click **New Group** and provide: * Name: `TagConfigGroup` * An optional description. * Group type: `CoreStandardGroup`. ![New Group](/assets/images/group_tagconfig_grp_new_grp-930aefb484e8655e133dde2fc40a67d5.png) Open the GraphQL interface at `http://localhost:8000/graphql` and run: ``` mutation CreateGroup { CoreStandardGroupCreate(data: {name: {value: "TagConfigGroup"}}) { ok object { hfid } } } ``` Save the `hfid` returned — you'll use it in the next step. ``` from infrahub_sdk import InfrahubClientSync client = InfrahubClientSync(address="http://localhost:8000") group = client.create(kind="CoreStandardGroup", name="TagConfigGroup") group.save() ``` ## Step 2 — Add two tags as members[​](#step-2--add-two-tags-as-members "Direct link to Step 2 — Add two tags as members") Attach your `red` and `blue` tags to the group. * Web Interface * GraphQL * Python SDK 1. Open `TagConfigGroup` from the Groups list. 2. Go to the **Members** tab. 3. Click **Add Members** and select `red` and `blue`. 4. Click **Save**. ![Adding members in group](/assets/images/group_tagconfig_grp_adding_members-8485bf45b9a1b55d2199c2e9e490ff45.png) First, look up the tag IDs: ``` query { BuiltinTag(name__values: ["red", "blue"]) { edges { node { id display_label } } } } ``` Then update the group with those IDs as members: ``` mutation UpdateGroupMembers { CoreStandardGroupUpdate( data: { hfid: ["TagConfigGroup"], members: [ {id: ""}, {id: ""} ] } ) { ok } } ``` ``` group = client.get(kind="CoreStandardGroup", name__value="TagConfigGroup") red_tag = client.get(kind="BuiltinTag", name__value="red") blue_tag = client.get(kind="BuiltinTag", name__value="blue") group.members.add(red_tag) group.members.add(blue_tag) group.save() ``` ## Step 3 — Verify[​](#step-3--verify "Direct link to Step 3 — Verify") Confirm both tags are now in the group. * Web Interface * GraphQL * Python SDK Open `TagConfigGroup` and check the **Members** tab. Both `red` and `blue` should appear. ![Group members](/assets/images/group_tagconfig_grp_new_members-824f11e2fd33298231d83de3662bd7ac.png) ``` query { CoreStandardGroup(name__value: "TagConfigGroup") { edges { node { name { value } members { edges { node { display_label } } } } } } } ``` ``` group = client.get(kind="CoreStandardGroup", name__value="TagConfigGroup") for member in group.members.fetch(): print(f"Member: {member.display_label}") ``` ## What you learned[​](#what-you-learned "Direct link to What you learned") * Groups are first-class objects in the graph with their own attributes and relationships. * A Standard group is manually managed — you decide what goes in it. * Membership is a relationship, so changes flow through the graph and can be queried from either side. ## Where to next[​](#where-to-next "Direct link to Where to next") * [Groups overview](/groups/overview.md) — concepts, architecture, and when to choose each group type. * [Add more members to a group](/groups/add-members.md) — how-to reference. * [Use groups in automation](/groups/use-in-automation.md) — target groups from Artifacts, Transformations, and Checks. --- # Tutorials The most direct way to learn a specific Infrahub workflow is to run a tutorial. Each one walks you through a real task end to end — modeling a schema, writing a check, building a Generator — against your own Infrahub instance, using the same GraphQL, Python, and schema snippets you'd write in production. You follow the steps directly in these docs, at your own pace, with no external platform involved. What you take away is a working example you built yourself, not a copy-pasted snippet: each tutorial explains why each step matters, so you leave with the concepts as well as the commands. Use a tutorial when: * You already have Infrahub running and want a guided, hands-on introduction to a specific feature * You learn best by building something real rather than reading a conceptual overview * You want a working example you can adapt for your own use case ## Schema[​](#schema "Direct link to Schema") * **[Build your first schema](/learn/tutorials/build-your-first-schema.md)** — model network devices and interfaces (nodes, attributes, relationships, and generics), then load each version into a branch. Take this tutorial first if you're new to schema design. ## Creating objects[​](#creating-objects "Direct link to Creating objects") * **[Organize objects with groups](/learn/tutorials/groups.md)** — create a group, add objects to it, and query the result end to end. Take this tutorial before using groups to target checks, Generators, or other automation. ## Data validation[​](#data-validation "Direct link to Data validation") * **[Build a check](/learn/tutorials/build-a-check.md)** — build, deploy, and validate a custom check that enforces a naming convention, from GraphQL query to proposed change. Take this tutorial once you have a schema in place and want to enforce data-quality rules on it. ## Transformations[​](#transformations "Direct link to Transformations") * **[Build a Jinja2 Transformation](/learn/tutorials/transformations/build-a-jinja2-transformation.md)** — render a device configuration snippet from a GraphQL query, test it locally, and call it through the render API. * **[Build a Python Transformation](/learn/tutorials/transformations/build-a-python-transformation.md)** — implement a Transformation class that returns JSON, test it locally, and call it through the REST API. Take this tutorial to see the Python and Jinja2 approaches side by side and pick the one that fits your use case. ## Generators[​](#generators "Direct link to Generators") * **[Build your first generator](/learn/tutorials/generators/build-your-first-generator.md)** — model two object kinds, write a GraphQL query, and implement a Generator that creates objects automatically whenever a change is proposed. * **[Build chained generators](/learn/tutorials/generators/build-chained-generators.md)** — wire two layers of modular Generators together with a checksum attribute so a downstream Generator only runs once its upstream dependency has finished. Take this tutorial once you're comfortable with a single Generator and need to model a multi-stage pipeline. --- # Build a Jinja2 Transformation By the end of this tutorial you will have built a working Jinja2 Transformation end-to-end: loaded a small network-device schema, created a few sample devices, written a GraphQL query that filters by device name, written a Jinja template that renders a configuration snippet from the result, registered it in `.infrahub.yml`, tested it locally with `infrahubctl render`, added the repository to Infrahub, and called the render API. You'll leave with a `device_config_transform` you can call against any device by name. For conceptual background, see [Transformations](/transformations/overview.md). For a recipe-form how-to without the running example, see [Write a Jinja2 transformation](/transformations/jinja2.md). Within Infrahub a [Transformation](/transformations/overview.md) is defined in an [external repository](/git-integration/connect-repository.md). However, during development and troubleshooting it is easiest to start from your local computer and run the render using [infrahubctl render](/infrahubctl/infrahubctl-render.md). The tutorial follows these steps: 1. Identify the relevant data you want to extract from the database using a [GraphQL query](/development-resources/graphql/overview.md), that can take an input parameter to filter the data 2. Write a Jinja2 file that uses the GraphQL query to read information from the system and render the data into a new format 3. Create an entry for the Jinja2 Transformation within an `.infrahub.yml` file 4. Create a Git repository 5. Test the Transformation rendering with `infrahubctl` 6. Add the repository to Infrahub as an external repository 7. Validate that the Transformation works using the render API endpoint ## 1. Loading a schema[​](#1-loading-a-schema "Direct link to 1. Loading a schema") This tutorial uses a very simplistic network device model. The rendered template won't be very useful on its own — the goal is to show how Jinja rendering works. Once you've mastered the basics you'll be ready to create more advanced templates. ``` --- version: "1.0" nodes: - name: Device namespace: Network display_label: "{{ name__value }}" attributes: - name: name kind: Text label: Name optional: false unique: true - name: description kind: Text label: Description optional: true ``` Store the schema as a YAML file on your local disk, and load the schema into Infrahub using the following command ``` infrahubctl schema load /path/to/schema.yml ``` More information on loading schema files into Infrahub can be found in the [schema import guide](/schema/create-and-load.md#load-a-schema-file). ## 2. Creating a query to collect the desired data[​](#2-creating-a-query-to-collect-the-desired-data "Direct link to 2. Creating a query to collect the desired data") As the first step we need some data in the database to actually query. Create three devices, called "switch1", "switch2", "switch3", either using the frontend or by submitting three GraphQL mutations as per below (swapping out the name each time). ``` mutation CreateDevice { NetworkDeviceCreate( data: {name: {value: "switch1"}, description: {value: "This is device switch1"}} ) { ok object { id } } } ``` The next step is to create a query that returns the data we created above. The rest of this tutorial assumes that the following query will return a response similar to the response below the query. ``` query DeviceQuery { NetworkDevice { edges { node { name { value } description { value } } } } } ``` Response to the query: ``` { "data": { "NetworkDevice": { "edges": [ { "node": { "name": { "value": "switch1" }, "description": { "value": "This is device switch1" } } }, { "node": { "name": { "value": "switch2" }, "description": { "value": "This is device switch2" } } }, { "node": { "name": { "value": "switch3" }, "description": { "value": "This is device switch3" } } } ] } } } ``` While it's possible to create a Transformation that targets all of these devices — for example to create a report — the goal here is to focus on one device at a time. Modify the query above to take an input parameter so that we can filter the result. Single-target query requirement For proper artifact detection, your query must target a **unique node** using a unique attribute or ID. This ensures Infrahub only regenerates the necessary artifacts instead of regenerating all artifacts unnecessarily. **Requirements for a valid single-target query:** * Must filter on a unique identifier (ID or unique attribute like `name`) * Must use a required variable, for example, `$name: String!` * Must use exact match filters, for example, `name__value: $name`, not list filters, for example, `name__values: $name` **Valid example:** ``` query BuiltinTag($name: String!) { BuiltinTag(name__value: $name) { edges { node { id } } } } ``` **Invalid examples (will cause excessive artifact generation):** No filter at all: ``` query BuiltinTag { BuiltinTag { edges { node { id } } } } ``` Filtering on a non-unique attribute: ``` query BuiltinTag($description: String!) { BuiltinTag(description__value: $description) { edges { node { id } } } } ``` To learn more about single-target queries and why they are important, see the [GraphQL topic](/development-resources/graphql/single-target-queries.md). Create a local directory on your computer. ``` mkdir device_config_render ``` Then save the below query as a text file named `device_config.gql`. ``` query DeviceQuery($name: String!) { NetworkDevice(name__value: $name) { edges { node { name { value } description { value } } } } } ``` The query requires an input parameter called `$name` that will refer to the name of each device. When we want to query for device switch1, the input variables to the query would look like this: ``` { "name": "switch1" } ``` ## 3. Create the Jinja template[​](#3-create-the-jinja-template "Direct link to 3. Create the Jinja template") The next step is to create the actual Jinja Template file. Create a file called `device_config.j2`. ``` {% if data.NetworkDevice.edges and data.NetworkDevice.edges is iterable %} {% for device in data["NetworkDevice"]["edges"] %} {% set device_name = device.node.name.value %} {% set device_description = device.node.description.value %} hostname {{ device_name }} description "{{ device_description }}" end {% endfor %} {% endif %} ``` note In your template, you can utilize most of the **filters** provided by **Jinja2** and **Netutils**! For more information, see the [SDK Templating Reference](/python-sdk/reference/templating.md). ## 4. Create a .infrahub.yml file[​](#4-create-a-infrahubyml-file "Direct link to 4. Create a .infrahub.yml file") In the `.infrahub.yml` file you define what Transformations you have in your repository that you want to make available for Infrahub. Create a `.infrahub.yml` file in the root of the directory. ``` --- jinja2_transforms: - name: device_config_transform # Unique name for your Transformation description: "device config Transformation" # (optional) query: "device_config_query" # Name or ID of the GraphQLQuery template_path: "device_config.j2" # Path to the main Jinja2 template queries: - name: device_config_query # Name of the GraphQLQuery file_path: "device_config.gql" # Path to the main Jinja2 template ``` > The main Jinja2 template can import other templates Three parts here are required: first the `name` of the Transformation which should be unique across Infrahub, `query` the GraphqlQuery linked to our Transformation, and also the `template_path` that should point to the Jinja2 file within the repository. ## 5. Create a Git repository[​](#5-create-a-git-repository "Direct link to 5. Create a Git repository") Within the `device_config_render` folder you should now have three files: * `device_config.gql`: Contains the GraphQL query * `device_config.j2`: Contains the Jinja2 Template * `.infrahub.yml`: Contains the definition for the Transformation Before we can test our Transformation we must add the files to a local Git repository. ``` git init --initial-branch=main git add . git commit -m "First commit" ``` ## 6. Test the render using infrahubctl[​](#6-test-the-render-using-infrahubctl "Direct link to 6. Test the render using infrahubctl") Using `infrahubctl` you can first verify that the `.infrahub.yml` file is formatted correctly by listing available Transformations. ``` Usage: infrahubctl render [OPTIONS] TRANSFORM_NAME [VARIABLES]... Render a local Jinja2 Transformation for debugging purpose. ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────╮ │ * transform_name TEXT [default: None] [required] │ │ variables [VARIABLES]... Variables to pass along with the query. Format │ │ key=value key=value. │ │ [default: None] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────╯ ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────╮ │ --branch TEXT Branch on which to render the Transformation. [default: None] │ │ --debug --no-debug [default: no-debug] │ │ --config-file TEXT [env var: INFRAHUBCTL_CONFIG] [default: infrahubctl.toml] │ │ --help Show this message and exit. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────╯ ``` Examples ``` infrahubctl render my-param=XXXXX my-other-param=YYYYY ``` note If `--branch` is not provided it will automatically use the name of the local branch. ## 7. Adding the repository to Infrahub[​](#7-adding-the-repository-to-infrahub "Direct link to 7. Adding the repository to Infrahub") To avoid repeating the same instructions, see [Connect a repository](/git-integration/connect-repository.md) for syncing the repository you created and making it available within Infrahub. ## 8. Accessing the Transformation from the API[​](#8-accessing-the-transformation-from-the-api "Direct link to 8. Accessing the Transformation from the API") A Transformation can be rendered on demand via the REST API with the endpoint: `https:///api/transform/jinja2/` This endpoint is branch-aware and it accepts the name of the branch and/or the time as URL parameters. * `https:///api/transform/jinja2/?branch=main` * `https:///api/transform/jinja2/?branch=main&at=