Microsoft Graph adapter

What is Microsoft Graph?

Microsoft Graph is the API that applications use to read data from Microsoft cloud services. Microsoft Entra ID, previously Azure Active Directory, is the Microsoft cloud directory of users, groups and applications, and one of the services that Graph serves.

The msgraphsync adapter reads users, groups and group membership from Microsoft Entra ID through Microsoft Graph, and loads them as nodes of the kinds defined in your Infrahub schema. It sends GET requests only.

The adapter does not:

• Read devices or any Graph resource other than users and groups.
• Read only the changes since the last run (Graph delta queries). Every run reads the full collections.
• Write to Microsoft Entra ID. The adapter can only be the source of a sync.

Requirements

Python packages

The adapter uses msgraph-sdk and azure-identity. Neither package is installed with Infrahub Sync. Install both in the environment that runs infrahub-sync:

pip install msgraph-sdk azure-identity

With uv:

uv pip install msgraph-sdk azure-identity

The unit tests of the adapter ran with msgraph-sdk 1.65.0 and azure-identity 1.27.0, against a mocked Microsoft Graph.

note

In a development checkout of Infrahub Sync, uv sync removes both packages because they are not in uv.lock. Install them again after each uv sync.

Application registration and Microsoft Graph permissions

The adapter reads the directory as an application, so the tenant needs an application registration with Microsoft Graph application permissions:

1. In the Microsoft Entra admin center, register an application.
2. Under API permissions, add these Microsoft Graph permissions of type Application (not Delegated):
   • User.Read.All, to read users.
   • Group.Read.All, to read groups and their members. GroupMember.Read.All can replace Group.Read.All: it also allows the adapter to read groups and their members, with access to fewer group properties.
   • Member.Read.Hidden, only when the configuration maps group members and the tenant has groups with hidden membership. Without it, Graph refuses the member read of such a group with HTTP 403 and the sync stops.
3. Grant admin consent for the tenant. Application permissions have no effect until an administrator grants consent.
4. For auth: client_secret, create a client secret under Certificates & secrets. Note the Directory (tenant) ID and the Application (client) ID shown on the overview page of the application.

With auth: default, the identity that signs in (a user, a managed identity or a workload identity) needs read access to users and groups instead.

When Microsoft Graph refuses a request, the error message explains the status. HTTP 401 means that Graph rejected the access token: the tenant or the credential is wrong, or graph_url does not match the audience of the token. HTTP 403 means that the token is valid but a permission is missing. See HTTP 401 or 403 from Microsoft Graph.

Sync directions supported

• Microsoft Entra ID → Infrahub

note

The adapter is source-only. When msgraphsync is set as the destination, the sync stops before any request with: The Microsoft Graph adapter can only be the source of a sync, but it is configured as the destination. Move 'msgraphsync' to the 'source' block of the configuration.

Configuration

The source block selects the adapter with name: msgraphsync. Every setting is optional. This block shows each setting with its default value or an example:

source:
  name: msgraphsync
  settings:
    auth: client_secret            # client_secret (default) or default
    tenant_id: "00000000-0000-0000-0000-000000000000"   # MSGRAPH_TENANT_ID overrides it
    client_id: "11111111-1111-1111-1111-111111111111"   # MSGRAPH_CLIENT_ID overrides it
    graph_url: "https://graph.microsoft.com/v1.0"       # default
    page_size: 999                 # 1 to 999
    timeout: 60                    # seconds per request
    max_retries: 5                 # 0 to 10, after transient HTTP or transport errors
    membership: direct             # direct (default) or transitive
    select:                        # extra Graph properties per resource
      users: [department]
    collections:
      accounts:
        resource: users
        filter: "accountEnabled eq true"
      security_groups:
        resource: groups
        filter: "securityEnabled eq true and NOT groupTypes/any(c:c eq 'Unified')"
        advanced_query: true

Authentication modes

The auth setting selects how the adapter gets an access token for Microsoft Graph:

• client_secret (default): the adapter authenticates as the registered application with its tenant ID, client ID and client secret. Use it in CI pipelines and on servers.
• default: the adapter uses DefaultAzureCredential from azure-identity. It tries, in turn, the AZURE_* environment variables, a workload identity, a managed identity and the Azure CLI login (az login), among others. Use it on a laptop after az login, or on an Azure host with a managed identity. The identity that signs in needs read access to users and groups. The tenant_id and client_id settings and the MSGRAPH_* variables are ignored in this mode.

The client secret is never read from the configuration file. A client_secret key in settings stops the sync with: Setting 'client_secret' is not accepted in the configuration file. Set the environment variable MSGRAPH_CLIENT_SECRET instead.

Environment variables

Variable	Used with	Content
MSGRAPH_TENANT_ID	auth: client_secret	Directory (tenant) ID. Overrides the tenant_id setting.
MSGRAPH_CLIENT_ID	auth: client_secret	Application (client) ID. Overrides the client_id setting.
MSGRAPH_CLIENT_SECRET	auth: client_secret	Client secret. This variable is the only accepted source of the secret.
AZURE_* and the Azure CLI login	auth: default	Read by DefaultAzureCredential, for example AZURE_TENANT_ID, AZURE_CLIENT_ID and AZURE_CLIENT_SECRET.

With auth: client_secret, a missing tenant ID, client ID or secret stops the sync before any request, and the message names the missing variables:

auth 'client_secret' is missing values. Set the environment variable(s): MSGRAPH_CLIENT_SECRET. The tenant and client IDs can also come from the 'tenant_id' and 'client_id' settings.

Settings reference

Setting	Type	Default	Allowed values	Description
auth	string	client_secret	client_secret, default	How the adapter gets an access token. See Authentication modes.
tenant_id	string	none	Tenant ID	Directory (tenant) ID for auth: client_secret. MSGRAPH_TENANT_ID overrides it. Ignored with auth: default.
client_id	string	none	Client ID	Application (client) ID for auth: client_secret. MSGRAPH_CLIENT_ID overrides it. Ignored with auth: default.
graph_url	string	https://graph.microsoft.com/v1.0	An https:// URL	Microsoft Graph endpoint and version. The token scope is derived from its scheme and host (https://<host>/.default), and the token is sent to that host only. A graph_url of a Microsoft cloud other than the public one is not tested: the credential still signs in through the default (public cloud) authority.
page_size	integer	999	1 to 999	Records per page, sent as $top on every request.
timeout	number	60	Greater than 0	Seconds to wait for each attempt. Timeouts are retried up to max_retries times.
max_retries	integer	5	0 to 10	Retries of one request after HTTP 429, 502, 503, 504, a timeout or a dropped connection. All failures share this count. Without Retry-After, the wait between attempts doubles at each retry, with a random part, up to 60 seconds.
membership	string	direct	direct, transitive	Which group members the adapter reads. See Group membership.
select	mapping	none	Keys users and groups; each value a property name or a list of names	Extra Graph properties to request, beyond those that field mappings, filters and transforms read. See Properties the adapter requests.
collections	mapping	none	Collection names mapped to definitions	Named, optionally filtered reads of users or groups. See Collections and advanced queries.

The adapter refuses any other key and lists the allowed settings in the message, for example: Unknown msgraphsync setting(s): page-size. Allowed settings: auth, tenant_id, client_id, graph_url, page_size, timeout, max_retries, membership, select, collections.

Collections and advanced queries

The mapping of each schema_mapping entry names the collection the entry reads:

• users or groups: every user or every group, without a filter.
• A name defined under settings.collections: a read of users or groups with an optional filter.

A collection definition accepts four keys:

Key	Required	Default	Content
resource	Yes	none	users or groups.
filter	No	none	An OData $filter expression, sent to Microsoft Graph unchanged.
advanced_query	No	false	true sends the request as a Graph advanced query.
member_of	No	none	A non-empty group object ID. Only valid with resource: users; reads the users that belong to this group.

A collection cannot be called users or groups: these names already mean a read of every user or every group.

Use member_of to read only a group's users, optionally narrowed with an OData filter:

collections:
  room_accounts:
    resource: users
    member_of: 00000000-0000-0000-0000-000000000000
    filter: "startswith(userPrincipalName,'room')"

With membership: direct (the default), this collection reads GET /groups/{member_of}/members/microsoft.graph.user. With membership: transitive, it reads GET /groups/{member_of}/transitiveMembers/microsoft.graph.user, including users in nested groups. Both reads use the collection's $select, $filter and page size, and follow every page. They require GroupMember.Read.All (or Group.Read.All); hidden membership also requires Member.Read.Hidden.

Microsoft Graph requires an advanced query for the microsoft.graph.user cast that both reads use. For every member_of collection, the adapter adds $count=true and sends ConsistencyLevel: eventual on every page, even with advanced_query: false. The eventual-consistency delay described below applies. If the group does not exist, the sync stops with an error naming the collection and group ID.

The adapter reads each collection once per run, even when several schema_mapping entries name it, and it does not read collections that no entry names. Each entry receives its own copy of the records, so the filters and transforms of one entry do not change the records of another entry. In the example below, one accounts collection feeds two kinds: employees and service accounts.

Advanced queries and their delay

Microsoft Graph accepts some filters only as advanced queries, for example a filter that uses NOT, ne or endsWith. For a collection with advanced_query: true, the adapter adds $count=true to the first request and sends the header ConsistencyLevel: eventual with the first request and with every next page.

Advanced queries read an index that Microsoft Graph updates some time after each change in the directory (eventual consistency). A user or group that was created, changed or deleted shortly before a run can be missing from that run, or appear with its previous values. Later runs see the change after the index catches up. When a collection must show changes at once, use a filter that Graph accepts without advanced_query, and apply further selection with the filters of the schema_mapping entry.

Schema mapping

The adapter follows the schema mapping rules of every adapter (see the schema mapping reference), with these conventions:

• The mapping of a field names a Graph property as Graph returns it in JSON, for example displayName, userPrincipalName or accountEnabled.
• Dots reach nested properties, for example onPremisesExtensionAttributes.extensionAttribute1.
• filters and transforms apply to the records of the entry's collection, as with every adapter.
• The Graph id is a stable identifier: it does not change when a user or group is renamed. The example maps it to an attribute entra_id and uses that attribute as the identifier of each kind. The adapter also uses id to resolve membership references.

Properties the adapter requests

The adapter requests only the properties it needs, with $select. For each collection, the list contains, in this order:

• id.
• The top-level property of every field mapping, including fields with a reference, unless a transform of the entry writes that key.
• The top-level property of every filter. Filters run before the transforms, so they read Graph properties.
• The variables that each transform expression reads, unless an earlier transform of the entry writes them. For example, {{ department }} / {{ jobTitle }} adds department and jobTitle, and {{ displayName | upper }} written to displayName adds displayName.
• The properties of settings.select for that resource.

The list never contains members on groups: the adapter reads members with separate requests.

Add a property with settings.select when Graph must return a property that none of these reads.

A transform expression that is not valid Jinja stops the sync before any request, with a message that names the entry. A filter on a key that a transform writes, where no transform reads a Graph property of that name, logs a warning before the reads: the filter reads the Graph property of that name, not the transform output, and Graph refuses the read when no such property exists.

Some Graph properties hold a list, for example businessPhones or proxyAddresses. Mapping such a property to an attribute that holds one value stops the sync with: schema_mapping 'IdentityEmployee', field 'phone': the Graph property 'businessPhones' is a list, but the attribute holds one value. Turn the list into one value with a transform of this entry and map the transformed key.

Group membership

To load the members of each group, add a field with mapping: members and a reference to the users kind on a groups entry:

  - name: IdentityGroup
    mapping: security_groups
    identifiers: ["entra_id"]
    fields:
      - name: entra_id
        mapping: id
      - name: members
        mapping: members
        reference: IdentityEmployee

members is not a Graph property: the adapter reads the members of every group of the collection and stores their IDs under that key. The membership setting selects the Graph request:

membership	Request per group	Members kept
direct (default)	GET /groups/{id}/members/microsoft.graph.user	Users that are members of the group.
transitive	GET /groups/{id}/transitiveMembers/microsoft.graph.user	Users that are members of the group or of any group nested in it.

The microsoft.graph.user cast makes Graph return only users: members of other types, such as groups, devices and service principals, are not read. Graph requires $count=true and the header ConsistencyLevel: eventual for the cast, so the adapter sends both on every member read. A user reached through several nested groups appears once.

The adapter sends these requests only when an entry maps members, and only for the groups that pass the filters of at least one such entry. When one of these entries has no filters, or filters on members, the adapter reads the members of every group of the collection. It sends at least one request per group and reads four groups at a time, so on a tenant with many groups the member reads make up most of the requests of a run. A filter on the groups collection, or filters on the entry, reduce their number.

A mapping: members field without a reference stops the sync before any request: schema_mapping 'IdentityGroup', field 'members': 'mapping: members' holds the IDs of the group's user members and needs a 'reference' to the kind that loads those users.

Members that are not loaded

The adapter resolves each member ID to a node of the referenced kind loaded earlier in the same run. It drops the member IDs that match no loaded node, for example:

• Users that the filter of the users collection or the filters of the users entry leave out.
• Users loaded as another users kind. In the example, a group member that is a service account loads as IdentityServiceAccount, while IdentityGroup.members references IdentityEmployee, so that member is dropped.

After loading a kind, the adapter logs one warning per field with dropped references, with the count:

WARNING | infrahub_sync.adapters.msgraphsync | event='references to objects that are not loaded were dropped' kind='IdentityGroup' field='members' dropped=3

To keep those members, reference a kind that loads them.

Load order

When the configuration has no order, Infrahub Sync computes it from the reference fields, so the users kind loads before the groups kind that references it. When the configuration sets order, list every users kind before the groups kind that references it. If the groups kind loads first, every member reference is dropped and the warning shows the count.

Example: Entra ID to Infrahub

Entra ID to Infrahub examplehttps://github.com/opsmill/infrahub-sync/tree/main/examples/msgraphsync_to_infrahub

The example directory contains an Infrahub schema (schema.yml) with the kinds IdentityEmployee, IdentityServiceAccount and IdentityGroup, the sync configuration (config.yml) and the generated code. The configuration:

• Reads enabled user accounts once, in the accounts collection, and splits them into employees and service accounts. User principal names that start with svc- load as IdentityServiceAccount; the others load as IdentityEmployee.
• Reads security groups that are not Microsoft 365 groups, with a filter that needs advanced_query: true.
• Loads the direct user members of each group as IdentityGroup.members, a relationship to IdentityEmployee.

---
name: from-msgraphsync

source:
  name: msgraphsync
  settings:
    auth: client_secret
    membership: direct
    collections:
      accounts:
        resource: users
        filter: "accountEnabled eq true"
      security_groups:
        resource: groups
        filter: "securityEnabled eq true and NOT groupTypes/any(c:c eq 'Unified')"
        advanced_query: true

destination:
  name: infrahub
  settings:
    url: "http://localhost:8000"

schema_mapping:
  - name: IdentityEmployee
    mapping: accounts
    identifiers: ["entra_id"]
    filters:
      - field: userPrincipalName
        operation: regex
        value: "(?i)^(?!svc-)"
    transforms:
      - field: name_lower
        expression: "{{ userPrincipalName | lower }}"
    fields:
      - name: entra_id
        mapping: id
      - name: name
        mapping: name_lower
      - name: display_name
        mapping: displayName
      - name: email
        mapping: mail
      - name: enabled
        mapping: accountEnabled

  - name: IdentityServiceAccount
    mapping: accounts
    identifiers: ["entra_id"]
    filters:
      - field: userPrincipalName
        operation: regex
        value: "(?i)^svc-"
    fields:
      - name: entra_id
        mapping: id
      - name: name
        mapping: userPrincipalName
      - name: display_name
        mapping: displayName
      - name: email
        mapping: mail
      - name: enabled
        mapping: accountEnabled

  - name: IdentityGroup
    mapping: security_groups
    identifiers: ["entra_id"]
    fields:
      - name: entra_id
        mapping: id
      - name: name
        mapping: displayName
      - name: description
        mapping: description
      - name: members
        mapping: members
        reference: IdentityEmployee

Running the example

1. Load the example schema into Infrahub:

   infrahubctl schema load examples/msgraphsync_to_infrahub/schema.yml

2. Set the credentials of the application registration:

   export MSGRAPH_TENANT_ID="00000000-0000-0000-0000-000000000000"
   export MSGRAPH_CLIENT_ID="11111111-1111-1111-1111-111111111111"
   export MSGRAPH_CLIENT_SECRET="<client secret>"

3. Generate the Python models from the configuration. The generate command reads the schema from the destination Infrahub, so Infrahub must be running with the schema loaded. Run it again after each change to config.yml or to the schema:

   infrahub-sync generate --name from-msgraphsync --directory examples/

4. View the differences, then run the sync:

   infrahub-sync diff --name from-msgraphsync --directory examples/
   infrahub-sync sync --name from-msgraphsync --directory examples/

At the default log level, the adapter logs one line per collection read (records, requests, member requests and retries), one summary line after all reads, and one line per kind loaded (records loaded and records read).

Common issues and troubleshooting

The adapter checks the settings, the collection names of the schema_mapping entries, the members fields and the sync direction when it starts, before any request. The CLI shows the errors of these checks prefixed with Error initializing MsgraphsyncAdapter: , followed by the message shown on this page. Errors found later, while the adapter reads Microsoft Graph or loads the records, have no prefix.

ModuleNotFoundError when the sync starts

The optional packages are not installed. The missing module named in the message depends on which package is missing, for example:

ModuleNotFoundError: No module named 'azure'
ModuleNotFoundError: No module named 'kiota_abstractions'
ModuleNotFoundError: No module named 'msgraph'

Install both packages with pip install msgraph-sdk azure-identity in the environment that runs infrahub-sync.

Access token request fails

The credential could not get a token. The message names the auth mode, the request and what to check:

Could not get an access token for Microsoft Graph (auth 'client_secret') while requesting /v1.0/users?$select=id,...&$top=999. Check the environment variables MSGRAPH_TENANT_ID, MSGRAPH_CLIENT_ID and MSGRAPH_CLIENT_SECRET. Details: ...

• With auth: client_secret, check the tenant ID, the client ID, and that the client secret is the secret value (not its ID) and has not expired.
• With auth: default, the message asks to check that an Azure CLI login (az login), a managed identity or a workload identity is available to DefaultAzureCredential.

When the token endpoint cannot be reached, for example because of a proxy or a firewall, the message adds The token endpoint could not be reached or refused the request (ServiceRequestError) and asks to check the network connection to the Microsoft sign-in service.

HTTP 401 or 403 from Microsoft Graph

The adapter got a token, but Microsoft Graph refused the request:

• HTTP 401: Graph rejected the token. Check the tenant and the credential, and that graph_url matches the audience of the token.
• HTTP 403: the token is valid, but a permission is missing.

Microsoft Graph refused /v1.0/groups/.../members/microsoft.graph.user?$select=id&$top=999&$count=true with HTTP 403 (<code>: <message>). HTTP 401 means Graph rejected the access token: check the tenant, the credential, and that 'graph_url' matches the token audience. HTTP 403 means a permission is missing. The application needs the Microsoft Graph application permissions User.Read.All and Group.Read.All (or GroupMember.Read.All instead of Group.Read.All), with admin consent.

With auth: default, the last sentence says that the signed-in identity (user, managed identity or workload identity) needs read access to users and groups.

When Graph refuses a member read, the message also says that reading the members of a group with hidden membership needs Member.Read.Hidden.

Check that the permissions are of type Application, not Delegated, and that an administrator granted consent for the tenant. See Application registration and Microsoft Graph permissions.

Page without a value list

Microsoft Graph returns each page of a list as a JSON object with a value list. When a response has an empty body, or a JSON object without value, the adapter stops instead of reading zero records, so that the sync does not plan to delete the nodes it did not read:

Microsoft Graph returned a page without a 'value' list for /v1.0/users?$select=id,...&$top=999. The read stops so that the sync does not treat the missing records as deleted.

A proxy between the adapter and Microsoft Graph can cause such responses. A page with an empty value list is valid and reads zero records.

Several peers for a relationship that holds one peer

A field with a reference to a relationship that holds one peer reads a Graph property that resolves to more than one loaded node. The adapter stops and names the entry and the field:

schema_mapping 'IdentityServiceAccount', field 'owner': the Graph property 'ownerIds' resolves to 2 loaded 'IdentityEmployee' objects, but the relationship holds one peer. Map a property that holds one ID, or use a relationship that holds many peers.

Throttling and gateway errors: HTTP 429, 502, 503 or 504

Microsoft Graph limits the request rate of each application and tenant. On HTTP 429, 502, 503 or 504, the adapter waits for the number of seconds in the Retry-After header, then sends the same request again. Requests that time out or lose their connection are also retried. Without Retry-After, the upper bound of the wait starts at 5 seconds and doubles at each retry, up to 60 seconds. The adapter picks a random wait between 1 second and that bound. TLS certificate verification failures, invalid URLs or protocols, authentication failures and other HTTP 4xx errors are not retried. It logs one warning per wait:

WARNING | infrahub_sync.adapters.msgraphsync | event='graph request throttled, waiting before retry' status=429 path='/v1.0/groups/.../members/microsoft.graph.user?$select=id&$top=999&$count=true' wait_seconds=5.0 retry=1 max_retries=5

max_retries counts retries per request: HTTP and transport failures share one count, and each request starts with its own count, so a run can wait more times in total than max_retries. After the last retry of one request, the sync stops:

Microsoft Graph returned HTTP 429 for /v1.0/groups/.../members/microsoft.graph.user?$select=id&$top=999&$count=true after 5 retries (<code>: <message>). Run the sync again later, or raise the setting 'max_retries'.

When Retry-After asks for more than 300 seconds, the adapter does not retry earlier than Graph allows. It stops instead:

Microsoft Graph returned HTTP 429 for /v1.0/users?$select=id,...&$top=999 and asked to wait 900 seconds before the next request, more than the adapter waits (300 seconds). Run the sync again later.

Member reads send the most requests. Run the sync again later, raise max_retries, or reduce the number of groups with a filter.

Request timeout

Microsoft Graph request /v1.0/users?$select=id,...&$top=999 timed out after 60 seconds (setting 'timeout') after 5 retries.

The adapter retries a request that timed out up to max_retries times, using the same request URL for each attempt. If retries are exhausted, raise timeout, or lower page_size so that each page takes less time.

Unknown collection

An entry names a collection that is not defined under settings.collections and is not users or groups. The sync stops before any request, and the message lists the valid names:

schema_mapping 'IdentityGroup': Unknown Microsoft Graph collection 'security-groups'. Valid names: accounts, security_groups, users, groups.

HTTP 400 for a filter that needs an advanced query

A filter that Microsoft Graph accepts only as an advanced query, sent from a collection without advanced_query: true, gets HTTP 400. The message carries the Graph error code and the Graph message, which names the unsupported query:

Microsoft Graph returned HTTP 400 for /v1.0/groups?$filter=...&$select=id,...&$top=999 (<code>: <message>).

Set advanced_query: true on the collection. See Advanced queries and their delay.

Groups without some or all of their members

The adapter dropped member references to users that are not loaded as the referenced kind, and logged the count in a warning. See Members that are not loaded and Load order.

Related

• Choose an adapter
• Schema mapping reference
• Sync instance configuration
• Run a sync