Alert rules

This page contains a markdown version of the alert rules described in the mysql-operator repository.

See the source of truth on GitHub for the latest information, or an older version:

alert_rules/prometheus/

MySQL General Alerts

Alert	Severity	Notes
MySQLDown		MySQL instance is down. Please check if the MySQL process is running and the network connectivity.
MySQLMetricsScrapeError		MySQL Exporter encountered a metrics scrape error. Check the MySQL Exporter logs for more details.
MySQLTooManyConnections(>90%)		MySQL instance is using > 90% of max_connections. Consider checking the client application responsible for generating those additional connections.
MySQLHighThreadsRunning		MySQL instance is actively using > 80% of max_connections. Consider reviewing the value of the max-connections config parameter or allocate more resources to your database server.
MySQLHighPreparedStatementsUtilization(>80%)		MySQL instance is using > 80% of max_prepared_stmt_count. Too many prepared statements might consume a lot of memory.
MySQLSlowQueries		MySQL instance has slow queries. Consider optimizing the query by reviewing its execution plan, then rewrite the query and add any relevant indexes.
MySQLInnoDBLogWaits		MySQL instance has long InnoDB log waits. MySQL InnoDB log writes might be stalling. Check I/O activity on your nodes to find the responsible process or query.
MySQLRestarted		MySQL instance restarted. MySQL restarted less than one minute ago. If the restart was unplanned or frequent, check Loki logs (e.g. error.log).
MySQLConnectionErrors		MySQL instance has connection errors. Connection errors might indicate network issues, authentication problems, or resource limitations. Check the MySQL logs for more details.

MySQL Replication Alerts

Alert	Severity	Notes
MySQLClusterUnitOffline		MySQL cluster member is marked offline. The process might still be running, but the member is excluded from the cluster.
MySQLClusterNoPrimary		No primary in the cluster. The cluster will likely be in a Read-Only state. Check cluster health and logs.
MySQLClusterTooManyPrimaries		More than one primary detected. This can indicate a split-brain situation. Refer to troubleshooting docs.
MySQLNoReplication		No secondary members in the cluster. The cluster is not redundant and failure of the primary will cause downtime.
MySQLGroupReplicationReduced		The number of ONLINE members in the replication group has reduced compared to the maximum observed in the last 6 hours. Check cluster health and logs.
MySQLGroupReplicationConflicts		Conflicts detected in Group Replication. Indicates concurrent writes to the same rows/keys across members. Investigate logs and cluster status.
MySQLGroupReplicationQueueSizeHigh		High number of transactions in Group Replication queue (>100). May indicate network issues or overloaded nodes. Investigate cluster performance.