Loading…

AI SRE OpsUTC …

Overview Infrastructure Detection Incidents Handoff Reliability Postmortems AI

Demo data Sample data for a portfolio demo; not real incidents.

Change in progressCHG-2026-10-13-04 Pause synthetic traffic during finalization catch-up (low risk, 1 component)Details

Incident queue

Loading…

3 alerts firingLinethL2MessagesStuck (warning)Open incident #3

Prover requests failing

cause: prover · 2 open follow-ups

SEV2Mitigated⚠ change…

Bridge messages in flight but nothing claimed for 45 minutes

SEV3Declared2 firing…

Sequencer liveness txs missing

SEV3suggested⚠ change1 firing…

#4SEV3suggested⚠ Suspected change

Sequencer liveness txs missing

Probable cause traffic

Detected
Declared
Mitigated
Resolved
Closed

Merge into another incident

Its alert moves to the chosen incident and this candidate is dismissed. Both timelines record it.

Declare incident

…

open for

1

alerts firing

—

checklist

—

next update

none

paging

Summary Response Updates0 Alerts1 Roles Runbook1

Preview for SEV3 (suggested). Declaring the incident creates these checklists; items can be checked after that.

Assign roles

Leave the name empty to hand the role back to the commander.

Incident commander

0/4

Owns the incident and every role nobody else holds.

Unassigned

▶Right after declaring0/2

Confirm the severity and write the impact in Current status
Set the next status update time

▶Before resolving0/1

Confirm the affected SLIs have been normal for 30 minutes

▶Before closing0/1

Decide whether this incident needs a postmortem

Ops lead

0/6

Works the system: diagnosis and mitigation.

Held by the commander

▶Right after declaring0/2

Check recent changes (open CHGs, Flux reconciles) on the affected components
Open the runbook for each firing alert

▶Before mitigation0/2

Apply the runbook's quick mitigation and post the result as an update
Record each mitigation as a decision, with when to revert it

▶Before resolving0/2

Verify the fix on the dashboard and that the alerts have cleared
Revert temporary mitigations, or schedule the revert

Comms lead

0/1

Keeps people outside the response informed.

Held by the commander

▶Before resolving0/1

Post the all-clear

Planning lead

0/1

Tracks follow-ups, handoffs and the longer view.

Held by the commander

▶Before mitigation0/1

Note follow-up work as it comes up

Severity

You become incident commander unless someone already holds the role.

Incident commander

Ops lead

Comms lead

Planning lead