Detection management
A GitHub repository is the single source of truth for alert rules, SLOs and runbooks. Edits here open a pull request; they apply after review, merge and the GitOps sync.
Loading…
A GitHub repository is the single source of truth for alert rules, SLOs and runbooks. Edits here open a pull request; they apply after review, merge and the GitOps sync.
Loading…
Where each metric comes from and what uses it. Broken: referenced by a rule or dashboard but not collected. Unused: collected but referenced nowhere. Metric names are read from PromQL, so a rare template expression can be missed.
| Metric | Labels | Service | Used by |
|---|---|---|---|
| besu_blockchain_chain_head_timestamp | clustercomponentinstancejoblayer+2 | besu | 1 reference |
| besu_blockchain_chain_head_transaction_count | clustercomponentinstancejoblayer+2 | besu | 1 reference |
| besu_rpc_errors_count_total | clustercomponenterrorTypeinstancejob+4 | besu | 7 references |
| besu_rpc_request_time_count | clustercomponentinstancejoblayer+3 | besu | 7 references |
| container_cpu_usage_seconds_total | containercpuendpointidimage+8 | cAdvisor | 11 references |
| container_memory_rss | containerendpointidimageinstance+7 | cAdvisor | 4 references |
| container_memory_working_set_bytes | containerendpointidimageinstance+7 | cAdvisor | 13 references |
| eth2_slashingprotection_prevented_signings_total | clustercomponentinstancejoblayer+2 | web3signer | 2 references |
| ethereum_blockchain_height | clustercomponentinstancejoblayer+2 | besu | 5 references |
| ethereum_peer_count | clustercomponentinstancejoblayer+2 | besu | 2 references |
| http_secp256k1_malformed_request_count_total | clustercomponentinstancejoblayer+2 | http (various) | 1 reference |
| jvm_buffer_memory_used_bytes | clustercomponentidinstancejob+3 | JVM runtime | 3 references |
| jvm_buffer_pool_capacity_bytes | clustercomponentinstancejoblayer+3 | JVM runtime | 6 references |
| jvm_buffer_pool_used_bytes | clustercomponentinstancejoblayer+3 | JVM runtime | 6 references |
| jvm_buffer_total_capacity_bytes | clustercomponentidinstancejob+3 | JVM runtime | 3 references |
| jvm_gc_collection_seconds_count | clustercomponentgcinstancejob+3 | JVM runtime | 6 references |
| jvm_gc_collection_seconds_sum | clustercomponentgcinstancejob+3 | JVM runtime | 6 references |
| jvm_gc_pause_seconds_count | actioncauseclustercomponentgc+5 | JVM runtime | 3 references |
| jvm_gc_pause_seconds_sum | actioncauseclustercomponentgc+5 | JVM runtime | 3 references |
| jvm_memory_committed_bytes | areaclustercomponentidinstance+4 | JVM runtime | 9 references |
| jvm_memory_max_bytes | areaclustercomponentidinstance+4 | JVM runtime | 16 references |
| jvm_memory_used_bytes | areaclustercomponentidinstance+4 | JVM runtime | 12 references |
| jvm_threads_deadlocked | clustercomponentinstancejoblayer+2 | JVM runtime | 6 references |
| jvm_threads_live_threads | clustercomponentinstancejoblayer+2 | JVM runtime | 3 references |
| jvm_threads_state | clustercomponentinstancejoblayer+3 | JVM runtime | 6 references |
| jvm_threads_states_threads | clustercomponentinstancejoblayer+3 | JVM runtime | 3 references |
| kube_cronjob_status_last_successful_time | containercronjobendpointinstancejob+3 | kube-state-metrics | 1 reference |
| kube_pod_container_resource_limits | containerendpointinstancejobnamespace+6 | kube-state-metrics | 13 references |
| kube_pod_container_status_last_terminated_reason | containerendpointinstancejobnamespace+4 | kube-state-metrics | 13 references |
| kube_pod_container_status_last_terminated_timestamp | containerendpointinstancejobnamespace+3 | kube-state-metrics | 13 references |
| kube_pod_container_status_restarts_total | containerendpointinstancejobnamespace+3 | kube-state-metrics | 12 references |
| lab_beacon_collection_success | clustercomponentinstancejoblayer+2 | lab exporters | 7 references |
| lab_beacon_el_offline | clustercomponentinstancejoblayer+2 | lab exporters | 3 references |
| lab_beacon_finality_collection_success | clustercomponentinstancejoblayer+2 | lab exporters | 5 references |
| lab_beacon_finalized_epoch | clustercomponentinstancejoblayer+2 | lab exporters | 3 references |
| lab_beacon_head_slot | clustercomponentinstancejoblayer+2 | lab exporters | 3 references |
| lab_beacon_is_optimistic | clustercomponentinstancejoblayer+2 | lab exporters | 3 references |
| lab_execution_collection_success | clustercomponentinstancejoblayer+2 | lab exporters | 5 references |
| lab_execution_head_block | clustercomponentinstancejoblayer+2 | lab exporters | 1 reference |
| lab_execution_head_timestamp_seconds | clustercomponentinstancejoblayer+2 | lab exporters | 3 references |
| lab_sync_exporter_collection_timestamp_seconds | clustercomponentinstancejoblayer+2 | lab exporters | 11 references |
| linea_aggregation_highest_submitted_on_l1 | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_aggregation_proven_highest_consecutive_block_number | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_aggregation_prover_waiting | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_aggregation_submission_delay_seconds_bucket | clustercomponentinstancejoblayer+3 | linea coordinator / postman | 1 reference |
| linea_batch_proven_highest_consecutive_block_number | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 2 references |
| linea_batch_prover_pendingproofs | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_blob_highest_submitted_on_l1 | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 2 references |
| linea_blob_proven_highest_consecutive_block_number | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_blob_prover_pendingproofs | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_blob_submission_delay_seconds_bucket | clustercomponentinstancejoblayer+3 | linea coordinator / postman | 1 reference |
| linea_conflation_last_block_number | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 3 references |
| linea_gas_price_cap_l1_blobsubmission_maxfeeperblobgascap | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_gas_price_cap_l1_blobsubmission_maxfeepergascap | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_gas_price_cap_l1_finalization_maxfeepergascap | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_l2_pricing_ethgasprice | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_l2_pricing_variablecost | clustercomponentinstancejoblayer+2 | linea coordinator / postman | 1 reference |
| linea_postman_messages | appclustercomponentdirectioninstance+5 | linea coordinator / postman | 3 references |
| lineth_l2_exporter_last_collection_timestamp_seconds | clustercomponentinstancejoblayer+2 | lineth exporter | 1 reference |
| lineth_l2_exporter_target_up | clustercomponentinstancejoblayer+3 | lineth exporter | 1 reference |
| lineth_l2_head_block | clustercomponentexported_nodeinstancejob+3 | lineth exporter | 2 references |
| lineth_l2_liveness_included_blocks_total | clustercomponentinstancejoblayer+2 | lineth exporter | 2 references |
| lineth_l2_liveness_needed_blocks_total | clustercomponentinstancejoblayer+2 | lineth exporter | 2 references |
| lineth_l2_rollup_finalized_block | clustercomponentinstancejobl1_block_tag+3 | lineth exporter | 4 references |
| lineth_l2_rpc_txpool_oldest_age_seconds | clustercomponentinstancejoblayer+2 | lineth exporter | 2 references |
| lineth_prover_last_response_mode | clustercomponentinstancejoblayer+4 | lineth workflows | 1 reference |
| lineth_prover_oldest_request_age_seconds | clustercomponentinstancejoblayer+4 | lineth workflows | 2 references |
| lineth_prover_requests | clustercomponentinstancejoblayer+4 | lineth workflows | 3 references |
| maru_consensus_block_latency_bucket | clustercomponentinstancejoblayer+5 | maru | 1 reference |
| maru_engine_api_request_latency_seconds_bucket | clustercomponentendpointforkinstance+8 | maru | 1 reference |
| maru_executor_queue_size | clustercomponentinstancejoblayer+4 | maru | 1 reference |
| maru_metadata_cl_block_height | clustercomponentinstancejoblayer+3 | maru | 1 reference |
| node_filesystem_avail_bytes | containerdeviceendpointfstypeinstance+5 | node exporter | 1 reference |
| node_filesystem_size_bytes | containerdeviceendpointfstypeinstance+5 | node exporter | 1 reference |
| pg_stat_database_numbackends | clustercomponentdatiddatnameinstance+4 | postgres exporter | 2 references |
| process_cpu_seconds_total | clustercomponentcontainerendpointinstance+9 | JVM runtime | 6 references |
| process_resident_memory_bytes | clustercomponentcontainerendpointinstance+9 | JVM runtime | 6 references |
| shomei_blockchain_head | clustercomponentinstancejoblayer+2 | shomei | 2 references |
| signing_secp256k1_missing_identifier_count_total | clustercomponentinstancejoblayer+2 | web3signer | 1 reference |
| signing_secp256k1_signing_duration_count | clustercomponentinstancejoblayer+2 | web3signer | 1 reference |
| signing_secp256k1_signing_duration_sum | clustercomponentinstancejoblayer+2 | web3signer | 1 reference |
| signing_signers_loaded_count | clustercomponentinstancejoblayer+2 | web3signer | 4 references |
| slo:error_budget_remaining:ratio | slo | SLO rules | 1 reference |
| slo:objective:ratio | slo | SLO rules | 1 reference |
| slo:sli_error:ratio_rate1d | slo | SLO rules | 1 reference |
| slo:sli_error:ratio_rate1h | slo | SLO rules | 2 references |
| slo:sli_error:ratio_rate28d | slo | SLO rules | 1 reference |
| slo:sli_error:ratio_rate2h | slo | SLO rules | 1 reference |
| slo:sli_error:ratio_rate30m | slo | SLO rules | 1 reference |
| slo:sli_error:ratio_rate3d | slo | SLO rules | 1 reference |
| slo:sli_error:ratio_rate5m | slo | SLO rules | 2 references |
| slo:sli_error:ratio_rate6h | slo | SLO rules | 3 references |
| slo:sli_good:bool1m | slo | SLO rules | 7 references |
| traces_spanmetrics_calls_total | __metrics_gen_instancenetworkservicesourcespan_kind+4 | tempo | 3 references |
| traces_spanmetrics_latency_bucket | __metrics_gen_instancelenetworkservicesource+5 | tempo | 8 references |
| up | clustercomponentcontainerendpointinstance+9 | prometheus | 34 references |
| validator_external_signer_requests_total | clustercomponentinstancejoblayer+3 | teku | 2 references |
| validator_local_validator_counts | clustercomponentinstancejoblayer+3 | teku | 3 references |
| vertx_pool_in_use | clustercomponentinstancejoblayer+4 | JVM runtime | 2 references |
| vertx_pool_queue_pending | clustercomponentinstancejoblayer+4 | JVM runtime | 2 references |
| :node_memory_MemAvailable_bytes:sum | — | other | — |
| aggregator_discovery_aggregation_count_total | endpointinstancejobmetrics_pathnamespace+2 | kubernetes control plane | — |
| aggregator_unavailable_apiservice | endpointinstancejobnamenamespace+1 | kubernetes control plane | — |
| alertmanager_alerts | containerendpointinstancejobnamespace+3 | alertmanager | — |
| alertmanager_alerts_invalid_total | containerendpointinstancejobnamespace+3 | alertmanager | — |
| alertmanager_alerts_per_alert_limit | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_alerts_received_total | containerendpointinstancejobnamespace+4 | alertmanager | — |
| alertmanager_alerts_subscriber_channel_writes_total | containerendpointinstancejobnamespace+3 | alertmanager | — |
| alertmanager_build_info | branchcontainerendpointgoarchgoos+9 | alertmanager | — |
| alertmanager_cluster_enabled | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_config_hash | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_config_last_reload_success_timestamp_seconds | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_config_last_reload_successful | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_dispatcher_aggregation_group_creation_given_up_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_dispatcher_aggregation_group_creation_retries_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_dispatcher_aggregation_group_limit_reached_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_dispatcher_aggregation_groups | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_dispatcher_alert_processing_duration_seconds_count | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_dispatcher_alert_processing_duration_seconds_sum | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_http_concurrency_limit_exceeded_total | containerendpointinstancejobmethod+3 | alertmanager | — |
| alertmanager_http_request_duration_seconds_bucket | codecontainerendpointhandlerinstance+6 | alertmanager | — |
| alertmanager_http_request_duration_seconds_count | codecontainerendpointhandlerinstance+5 | alertmanager | — |
| alertmanager_http_request_duration_seconds_sum | codecontainerendpointhandlerinstance+5 | alertmanager | — |
| alertmanager_http_requests_in_flight | containerendpointinstancejobmethod+3 | alertmanager | — |
| alertmanager_http_response_size_bytes_bucket | containerendpointhandlerinstancejob+5 | alertmanager | — |
| alertmanager_http_response_size_bytes_count | containerendpointhandlerinstancejob+4 | alertmanager | — |
| alertmanager_http_response_size_bytes_sum | containerendpointhandlerinstancejob+4 | alertmanager | — |
| alertmanager_inhibition_rules | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_integrations | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_gc_duration_seconds_count | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_gc_duration_seconds_sum | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_gossip_messages_propagated_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_maintenance_errors_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_maintenance_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_queries_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_query_duration_seconds_bucket | containerendpointinstancejoble+3 | alertmanager | — |
| alertmanager_nflog_query_duration_seconds_count | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_query_duration_seconds_sum | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_query_errors_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_snapshot_duration_seconds_count | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_snapshot_duration_seconds_sum | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_nflog_snapshot_size_bytes | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_notification_latency_seconds_bucket | containerendpointinstanceintegrationjob+4 | alertmanager | — |
| alertmanager_notification_latency_seconds_count | containerendpointinstanceintegrationjob+3 | alertmanager | — |
| alertmanager_notification_latency_seconds_sum | containerendpointinstanceintegrationjob+3 | alertmanager | — |
| alertmanager_notification_requests_failed_total | containerendpointinstanceintegrationjob+3 | alertmanager | — |
| alertmanager_notification_requests_total | containerendpointinstanceintegrationjob+3 | alertmanager | — |
| alertmanager_notifications_failed_total | containerendpointinstanceintegrationjob+4 | alertmanager | — |
| alertmanager_notifications_suppressed_total | containerendpointinstancejobnamespace+3 | alertmanager | — |
| alertmanager_notifications_total | containerendpointinstanceintegrationjob+3 | alertmanager | — |
| alertmanager_receivers | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences | containerendpointinstancejobnamespace+3 | alertmanager | — |
| alertmanager_silences_gc_duration_seconds_count | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_gc_duration_seconds_sum | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_gc_errors_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_gossip_messages_propagated_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_maintenance_errors_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_maintenance_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_matcher_compile_errors_total | containerendpointinstancejobnamespace+3 | alertmanager | — |
| alertmanager_silences_matcher_index_size | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_queries_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_query_duration_seconds_bucket | containerendpointinstancejoble+3 | alertmanager | — |
| alertmanager_silences_query_duration_seconds_count | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_query_duration_seconds_sum | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_query_errors_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_query_silences_scanned_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_query_silences_skipped_total | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_snapshot_duration_seconds_count | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_snapshot_duration_seconds_sum | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_snapshot_size_bytes | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_state_size | containerendpointinstancejobnamespace+2 | alertmanager | — |
| alertmanager_silences_version_index_size | containerendpointinstancejobnamespace+2 | alertmanager | — |
| ALERTS | alertmanageralertnamealertstateareaburn_rate+41 | other | — |
| ALERTS_FOR_STATE | alertmanageralertnameareaburn_ratecluster+40 | other | — |
| alloy_build_info | branchcontainerendpointgoarchgoos+9 | alloy / otel collector | — |
| alloy_component_controller_evaluating | containercontroller_idcontroller_pathendpointinstance+4 | alloy / otel collector | — |
| alloy_component_controller_running_components | containercontroller_pathendpointhealth_typeinstance+4 | alloy / otel collector | — |
| alloy_component_dependencies_wait_seconds_bucket | containercontroller_idcontroller_pathendpointinstance+5 | alloy / otel collector | — |
| alloy_component_dependencies_wait_seconds_count | containercontroller_idcontroller_pathendpointinstance+4 | alloy / otel collector | — |
| alloy_component_dependencies_wait_seconds_sum | containercontroller_idcontroller_pathendpointinstance+4 | alloy / otel collector | — |
| alloy_component_evaluation_queue_size | containercontroller_idcontroller_pathendpointinstance+4 | alloy / otel collector | — |
| alloy_component_evaluation_seconds_bucket | containercontroller_idcontroller_pathendpointinstance+5 | alloy / otel collector | — |
| alloy_component_evaluation_seconds_count | containercontroller_idcontroller_pathendpointinstance+4 | alloy / otel collector | — |
| alloy_component_evaluation_seconds_sum | containercontroller_idcontroller_pathendpointinstance+4 | alloy / otel collector | — |
| alloy_component_evaluation_slow_seconds | component_idcontainercontroller_pathendpointinstance+4 | alloy / otel collector | — |
| alloy_component_graph_connection | containercontroller_pathendpointfrominstance+5 | alloy / otel collector | — |
| alloy_config_hash | containerendpointinstancejobnamespace+3 | alloy / otel collector | — |
| alloy_config_last_load_success_timestamp_seconds | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_config_last_load_successful | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_config_load_failures_total | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_labelstore_global_ids_count | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_labelstore_last_stale_check_timestamp | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_resources_machine_rx_bytes_total | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_resources_machine_tx_bytes_total | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_resources_process_cpu_seconds_total | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_resources_process_resident_memory_bytes | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_resources_process_start_time_seconds | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| alloy_resources_process_virtual_memory_bytes | containerendpointinstancejobnamespace+2 | alloy / otel collector | — |
| apiextensions_apiserver_validation_ratcheting_seconds_bucket | endpointinstancejoblenamespace+1 | kubernetes control plane | — |
| apiextensions_apiserver_validation_ratcheting_seconds_count | endpointinstancejobnamespaceservice | kubernetes control plane | — |