Webhook event processor state machine
Event ingestion, processing, retry, and idempotency states.
Every application that integrates with external services — Stripe, GitHub, Twilio — receives webhooks. A naive handler causes duplicates when the sender retries, loses events when the handler crashes, and creates support incidents when a transient error goes unretried. This template maps the full lifecycle: ingestion, signature verification, idempotency checking, execution, and the branching paths for retryable vs. fatal errors.
The state machine makes visible the overlooked states: AlreadyProcessed (when your own retry or the sender's retry re-delivers the same event), Queued (when you accept the event but defer execution), and DeadLetter (when all retries are exhausted and a human must investigate).
When to use this template
- Webhook handler design — before writing the handler, sketch the state machine so engineers see all the branches: success, transient failure with retry, fatal error with DLQ, and duplicate detection.
- Incident postmortems — when a webhook integration breaks, trace which state the system got stuck in. Did it not retry? Did it retry forever and crash?
- Compliance audits — show auditors the idempotency and audit-trail machinery, proving you don't lose or duplicate financial transactions.
How to adapt it
Start by naming your real webhook provider and error types. Common extensions:
- Add a batching state before execution if you collect multiple events and process them as a transaction (common in payment reconciliation).
- Insert a manual review state between DeadLetter and [*] if a human must approve the final outcome (compliance requirement for some industries).
- Add a circuit breaker state that stops retries if the target service is down for an extended period, then resumes later (prevents retry storms).
Visual edits regenerate clean Mermaid code, so you can build your team's webhook policy directly in the diagram without writing state syntax.
Mermaid code
Copy it anywhere Mermaid is supported — GitHub, Notion, or your docs.
stateDiagram-v2
[*] --> Received
Received --> Validating: parse payload
Validating --> Verified: signature OK
Validating --> InvalidSignature: bad signature
InvalidSignature --> [*]
Verified --> CheckIdempotent: look up by event ID
CheckIdempotent --> AlreadyProcessed: event seen before
CheckIdempotent --> Processing: new event
AlreadyProcessed --> [*]
Processing --> Executing: call handler
Executing --> Success: ✓ completed
Executing --> Failure: ✗ error
Success --> Persisting: log success
Persisting --> [*]
Failure --> Retryable: retriable error?
Failure --> Fatal: fatal error
Retryable --> Queued: schedule retry
Queued --> Delay: wait (backoff)
Delay --> Executing: retry attempt
Fatal --> DeadLetter: move to DLQ
DeadLetter --> [*]
Frequently asked questions
- Why do webhook processors need a state machine?
- Webhooks are unreliable by nature — the sender may retry (causing duplicates), the network may fail, or the handler may crash partway through. A state machine forces your team to think through each state: what if the signature is invalid, what if we have already seen this event, when should we retry vs. give up?
- What is idempotency in webhook processing?
- Idempotency means processing the same event twice produces the same result as processing it once. Webhook providers (Stripe, GitHub, etc.) send an event ID; if you see the same ID twice, skip it or return the cached result. This protects against duplicates when your client retries after a network failure.
- How long should I retry a failed webhook?
- Most providers retry for 3–7 days with exponential backoff, retrying more frequently at first (after 1 min, then 5 min, then 1 hour). Use a queuing system (SQS, RabbitMQ, Bull) to schedule retries rather than blocking the handler. If all retries fail, move the event to a dead-letter queue for manual investigation.
- How do I catch idempotency bugs?
- In tests, replay the same webhook event three times and verify the result is identical — the charge is not tripled, the user is not re-registered, etc. Visual edits regenerate clean Mermaid code, so you can sketch your team's retry and idempotency policy directly in the diagram, then implement it and use this diagram as your runbook.
Related templates
Event-driven notification system
Event publisher, queue, consumers, and retry/failure handling.
Message queue retry logic
Publish-subscribe with exponential backoff and dead-letter queues.
Multi-provider API integration with fallbacks
Try primary provider, fallback to secondary, log failures for audit.