An SLO is burning its error budget too fast
criticalreviewed 2026-10-13What it meansmoreMulti-window burn-rate alerts from the SLOs in slos/ (generated by generators/gen-slo-rules.py). Page: the budget burns fast enough to be gone within days (14.4x over 1 h, or 6x over 6 h). Ticket: slower burns that would still exhaust it within the 28-day window (3x over 1 d, or 1x over 3 d). The slo label says which one. The alert says users are hurt; the cause is in the component alerts that usually fire with it.
ImpactmorePage: users are affected now; SEV2 by default (the SLO's own impact decides). Ticket: no page, plan the work. Once a budget is spent, only reliability work goes in for that service (freeze in the CHG checklist).
Quick mitigation0/3
Read the
slo label and open the incident that holds the component alerts firing with itFollow that component's runbook: settlement-freshness →
finalization-stalled or prover-oom-loop; block-production → sequencer-stalled; tx-inclusion or rpc → l2-node-lagging; deposits or withdrawals → l2-bridge-messagesIf no component alert fires, the SLI itself may be wrong: check its query and the l2-exporter (
l2-telemetry-missing)Diagnosis2 queries
Error budget left per SLO (1 = full, below 0 = spent):
slo:error_budget_remaining:ratioError rate over the last hour per SLO:
slo:sli_error:ratio_rate1hFix0/2
Write the follow-ups that stop the same burn: the postmortem's action items for the component that caused it
If the SLO was wrong (too strict, measured the wrong thing), change it in
slos/ with a CHG, not by silencingEscalation
A page-level burn that is not explained by a component incident within 30 minutes: page the IC and treat the SLO itself as the incident until the cause is known.
Seen in INC-2026-10-11-11, INC-2026-10-13-12 · SLOs settlement-freshness, block-production, tx-inclusion-5s, tx-inclusion-60s, rpc-availability, rpc-server-errors, withdrawal-1h, deposit-2m, deposit-10m · owner s1ns3nz0