Postmortems
Blameless write-ups of what the system and the process allowed, with action items that prevent, detect or mitigate a recurrence. The GitHub repository is the single source of truth; drafts open as pull requests.
Loading…
Blameless write-ups of what the system and the process allowed, with action items that prevent, detect or mitigate a recurrence. The GitHub repository is the single source of truth; drafts open as pull requests.
Loading…
Contributing factors across 16 postmortems. Select one to filter.
What the postmortems asked for, and how much is done.
56%done
Postmortems per day, by severity.
The sequencer logs ERROR LineaLivenessService Error building bundle for sequencer uptime transactions java.io.IOException: Web3Signer request was interrupted about 190 times an hour. The first guess was that ~24 % of liveness attempts failed. Measuring the outcome showed none failed: Maru re-sends forkchoiceUpdated with new payload attributes, Besu cancels the superseded block build while it is waiting for Web3Signer, and logs the interruption at ERROR. The new build signs again and succeeds; every block that needs liveness txs has them.
linea_getBlockTracesCountersV2 returns INTERNAL_ERROR on the public RPCSince the T1 drill restarted the public RPC node (l2-node-besu), some calls to linea_getBlockTracesCountersV2 fail with INTERNAL_ERROR. The coordinator uses this call to count trace lines per block when it cuts batches; it retries the block after 1 s and gets an answer, so no batch has been lost. The stack trace points into the line counter's fork lookup ( in ). The error survived the 10-12 chain reset, so it is not tied to old chain data.
Fork.getForkFromBesuBlockchainServiceLineCounterGenerator.createLineCountingTracer