Bus event handling

Under Construction

This page is still under construction and is not available yet.
Please reach out on Discord if you have some questions about the Bus events

Architecture

Infrahub requires the following features from the message bus:

1. Retry messages with a delay
2. Broadcast messages to multiple consumers
3. Asynchronous RPC
4. Set priorities to messages

Depending on the message bus system, we implement them using the following:

RabbitMQ

1. We create a delayed exchange and multiple queues per TTL variation. When there is an error on message processing, the worker will publish the failed message in the delayed exchange with the delay TTL set. Note: for non-acked (NAK, expired, dropped, or lost) messages, we make use of a dead-letter exchange (DLX). Note bis: events and callback messages do not require acknowledgment.
2. We create one events queue per worker binding on the required broadcast messages using their routing keys.
3. Requests are sent to the exchange and processed by an unique queue where each worker consume messages. We also create one callback queue per worker and use the reply-to header to send reply messages to this queue.
4. Priorities are handled using the priority field of message properties.

NATS JetStream

1. We NAK the message with a delay.
2. We create an events stream with retention policy set to INTEREST and create an ephemeral consumer for each worker.
3. Requests are sent to the rpcs stream with retention policy set to WORK_QUEUE and create a consumer group on which each worker will subscribe. We also create one callback stream per worker and publish to this stream for the replies.
4. We do not handle priorities.