Queue topology and message routing
Message broker architecture with topic subscriptions and dead-letter handling.
Event-driven systems rely on message brokers — Kafka, RabbitMQ, AWS SNS/SQS, Google Pub/Sub — to decouple services. A message broker topology determines how messages flow: does one subscriber get each message, or do many subscribers each get a copy? What happens if a subscriber crashes or is slow?
This template shows a modern pub-sub topology: an application publishes events to a topic, multiple subscribers pick them up in parallel, successful processing sends an ACK (acknowledgment), failed processing sends a NACK, and after retries, poison messages move to a dead-letter queue. This is the foundation of resilient event-driven systems.
When to use this template
- Architecture design — when adding async workflows (emails, notifications, analytics), map them on this diagram to show message flow and error handling.
- On-call runbook — when a subscriber fails, this diagram shows the recovery path: check the DLQ, investigate why messages failed, fix the code, replay from the DLQ.
- Compliance and data flow — map every topic and subscriber to document what data flows where and who has access.
How to adapt it
Customize it to your message broker and add your specific workflows:
- Name your topics — replace "topic" with "orders-created", "user-signup", "payment-processed", etc.
- Add multiple topics — show how one application publishes to many topics and how different subscribers consume different topics.
- Add request-reply — for synchronous workflows, add a subscriber that publishes a response back to a reply topic after processing.
- Add filtering — between the broker and subscribers, add a filter node to show topic filtering or message selectors (e.g., "only process orders > $100").
Visual edits regenerate clean code, so you can model your exact message flow.
Mermaid code
Copy it anywhere Mermaid is supported — GitHub, Notion, or your docs.
sequenceDiagram
participant App as Application
participant Broker as Message Broker
participant Sub1 as Subscriber 1
participant Sub2 as Subscriber 2
participant DLQ as Dead Letter Queue
App->>Broker: Publish event to topic
activate Broker
Broker->>Sub1: Deliver message
Broker->>Sub2: Deliver message
deactivate Broker
activate Sub1
Sub1->>Sub1: Process message
Sub1-->>Broker: ACK success
deactivate Sub1
activate Sub2
Sub2->>Sub2: Try to process
alt Processing fails
Sub2-->>Broker: NACK / error
Broker->>DLQ: Move to DLQ after retries
else Processing succeeds
Sub2-->>Broker: ACK success
end
deactivate Sub2
Frequently asked questions
- What is a message queue and why not just call the API directly?
- A message queue decouples producers from consumers. The app publishes an event and moves on; subscribers pick it up asynchronously. If a subscriber is slow or down, the queue buffers messages so they don't get lost. Direct API calls fail if the receiver is unavailable — queues make your system resilient to outages and let you scale consumers independently.
- What's the difference between a topic and a queue?
- A queue is point-to-point: one message, one consumer (first come, first served). A topic is pub-sub: one message, many subscribers. Most modern systems use topics (e.g., Kafka, RabbitMQ exchanges) because they enable fan-out: the same event triggers multiple independent workflows — email, analytics, notifications — without coupling them.
- What is a dead-letter queue and when do I use it?
- A dead-letter queue (DLQ) is where messages go after they fail to process (after N retries). It's a holding pen for poison messages — corrupted data, messages targeting services that don't exist, or logic bugs. Instead of dropping the message and losing data, send it to the DLQ, alert ops, and fix it later. This prevents message loss.
- How do I handle out-of-order messages?
- In a topic with many consumers, messages can arrive out of order. If order matters, use a partition key or shard key (e.g., by user ID or tenant). Messages with the same key go to the same partition and preserve order. If order doesn't matter, consumers can process in parallel. Document which events are order-sensitive and which are not.
Related templates
Real-time notification delivery system
Web sockets, message broker, and delivery handlers in action.
DNS resolution sequence
Client query through recursive resolvers to authoritative nameservers.
Message queue retry logic
Publish-subscribe with exponential backoff and dead-letter queues.