Feature flag emergency rollback
Detect problems, decide action, roll back or kill the feature.
Every feature flag comes with an invisible contract: "if something goes wrong, we can turn this off in seconds." This template maps that contract into states: from healthy steady state through alert to rollback decision, then either back to serving traffic or forward to a fix. The key divergence happens at Triage — is the feature the culprit, or is the problem elsewhere? That decision determines rollback or continued investigation. Once disabled, the feature can be investigated safely while the old code serves production again.
The pattern reflects real production incidents: metrics spike, on-call is paged, triage happens in parallel with feature flag disable (taking 30 seconds), and only after rollback completes do engineers investigate root cause. Rollback-first saves users immediately; investigation happens later.
When to use this template
- Feature flag strategy documentation — show engineers and product managers that every flag has an escape hatch, and define when the on-call will pull it.
- On-call runbooks — during an incident, this diagram is a decision tree: if your new feature is suspected, how fast can you disable it?
- Blameless post-mortems — trace the incident from alert through triage to rollback, document why triage took time, and capture what would speed up the decision next time.
How to adapt it
Adapt the trigger and decision points to your real systems and SLOs:
- Replace Healthy metrics with your specific SLO — 99.5% availability, <500ms p99 latency, <0.1% error rate — so teams know exactly what threshold triggers an alert.
- Add A/B test branches if your feature is behind a percentage rollout — show whether you disable gradually (reduce flag to 1%) or instantly (disable to 0%).
- Insert stakeholder notification states between Flag_Disabled and Traffic_Drained to show when product, customer success, and status-page get notified.
Visual edits regenerate clean Mermaid code, so you can model your actual incident response and export the diagram into your runbooks and wiki.
Mermaid code
Copy it anywhere Mermaid is supported — GitHub, Notion, or your docs.
stateDiagram-v2
[*] --> Feature_Enabled
Feature_Enabled --> Monitoring: Live in production
Monitoring --> Healthy: Metrics normal
Healthy --> Feature_Enabled: Continue serving
Monitoring --> Alert_Triggered: Error rate spikes
Alert_Triggered --> Triage: On-call reviews
Triage --> Flag_Disabled: Problem confirmed in feature
Triage --> Investigating: Unclear root cause
Investigating --> Flag_Disabled: Feature is culprit
Investigating --> Monitoring: Issue in unrelated system
Flag_Disabled --> Traffic_Drained: Rollback signal sent
Traffic_Drained --> Feature_Disabled: Old code serving all users
Feature_Disabled --> Investigation_Ongoing: Post-incident analysis
Investigation_Ongoing --> Fix_Deployed: Bug fixed and tested
Fix_Deployed --> Feature_Enabled: Roll forward to production
Feature_Disabled --> [*]: Decision: deprecate feature
Related templates
Deployment rollback decision tree
Incident detection, severity assessment, and rollback trigger criteria.
Incident state machine
States and transitions during an incident from detection to postmortem.
Release readiness checklist
State machine for go-live criteria and rollout progress tracking.