Feb 15 10:28:30 clouddb1001 mysqld[2018]: 2022-02-15 10:28:30 140283553928960 [Warning] mysqld: Disk is full writing '/srv/labsdb/tmp/#sql_7e2_9.MAD' (Errcode: 28 "No space left on device"). Waiting for someone to free space... (Expect up to 60 secs delay for server to continue after freeing disk space)

MariaDB [(none)]> show grants for 'prometheus'@'localhost'\G
*************************** 1. row ***************************
Grants for prometheus@localhost: GRANT PROCESS, REPLICATION CLIENT ON *.* TO 'prometheus'@'localhost' IDENTIFIED VIA unix_socket WITH MAX_USER_CONNECTIONS 5
*************************** 2. row ***************************
Grants for prometheus@localhost: GRANT SELECT ON `heartbeat`.`heartbeat` TO 'prometheus'@'localhost'
2 rows in set (0.00 sec)

# found the next binlog/position to start from
# was stuck at log.275246 event 15499142
# next event is log.275247 position 4 (binlogs start at 4)
# You can use mysqlbinlog to verify
# on the slave
sudo mysql
mysql> STOP SLAVE;
# verify that it stopped
mysql> SHOW SLAVE STATUS\G
# check Slave_IO_Running and Slave_SQL_Running should be no
mysql> CHANGE MASTER TO MASTER_LOG_FILE='log.275247', MASTER_LOG_POS=4;
mysql> START SLAVE;
# Then see that the slave picks up
mysql> SHOW SLAVE STATUS\G
# check Slave_IO_Running and Slave_SQL_Running should be yes
# and Seconds_Behind_Master should be going down bit by bit