Description
Details
Related Objects
Event Timeline
Applied to db1070- the reason why it was not active is because it was a former master; so it had the configuration for a normal master.
Trying to detect other potentially problematic servers:
root@neodymium:~$ sudo salt -C 'G@mysql_role:slave and G@site:eqiad and G@mysql_group:core' cmd.run 'mysql --defaults-file=/root/.my.cnf --batch --skip-column-names -e "SHOW EVENTS FROM ops"' db1071.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978778 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978778 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978778 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978778 utf8 utf8_general_ci binary db1042.eqiad.wmnet: db1086.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101622 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101622 utf8 utf8_general_ci binary db1070.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2016-10-21 00:00:01NULL ENABLED 171978777 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2016-10-21 00:00:00 NULL ENABLED 171978777 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2016-10-21 00:00:05 NULL ENABLED 171978777 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2016-10-21 00:00:03NULL ENABLED 171978777 utf8 utf8_general_ci binary db1077.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171966483 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171966483 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171966483 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171966483 utf8 utf8_general_ci binary db1094.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101622 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101622 utf8 utf8_general_ci binary db1081.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101629 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101629 utf8 utf8_general_ci binary db1090.eqiad.wmnet: ops wmf_master_purge root@208.80.154.151 SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00 NULL ENABLED 101613 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@208.80.154.151 SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101613 utf8 utf8_general_ci binary ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970573 utf8 utf8_general_ci binary db1084.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101629 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101629 utf8 utf8_general_ci binary db1019.eqiad.wmnet: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2 "No such file or directory") db1082.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 103228 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 103228 utf8 utf8_general_ci binary db1092.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 103228 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 103228 utf8 utf8_general_ci binary db1089.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171974771 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171974771 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171974771 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171974771 utf8 utf8_general_ci binary db1078.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970576 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970576 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970576 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970576 utf8 utf8_general_ci binary db1093.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101612 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101612 utf8 utf8_general_ci binary db1074.eqiad.wmnet: ops wmf_master_purge root@208.80.154.151 SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00 NULL ENABLED 101613 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@208.80.154.151 SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101613 utf8 utf8_general_ci binary ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970573 utf8 utf8_general_ci binary db1083.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970661 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970661 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970661 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970661 utf8 utf8_general_ci binary db1079.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101622 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101622 utf8 utf8_general_ci binary db1076.eqiad.wmnet: ops wmf_master_purge root@208.80.154.151 SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00 NULL ENABLED 101613 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@208.80.154.151 SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101613 utf8 utf8_general_ci binary ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970573 utf8 utf8_general_ci binary db1080.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171966556 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171966556 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171966556 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171966556 utf8 utf8_general_ci binary db1064.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978771 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978771 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978771 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978771 utf8 utf8_general_ci binary db1087.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 103228 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 103228 utf8 utf8_general_ci binary db1088.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101612 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101612 utf8 utf8_general_ci binary db1073.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978780 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978780 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978780 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978780 utf8 utf8_general_ci binary db1091.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101629 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101629 utf8 utf8_general_ci binary db1053.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978771 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978771 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978771 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978771 utf8 utf8_general_ci binary db1085.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101612 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101612 utf8 utf8_general_ci binary db1068.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978775 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978775 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978775 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978775 utf8 utf8_general_ci binary db1021.eqiad.wmnet: db1059.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171974685 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171974685 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171974685 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171974685 utf8 utf8_general_ci binary db1060.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171974686 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171974686 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171974686 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171974686 utf8 utf8_general_ci binary db1067.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978774 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978774 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978774 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978774 utf8 utf8_general_ci binary db1061.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978766 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978766 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978766 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978766 utf8 utf8_general_ci binary db1038.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101627 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101627 utf8 utf8_general_ci binary db1063.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978768 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978768 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978768 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978768 utf8 utf8_general_ci binary db1062.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978767 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978767 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978767 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978767 utf8 utf8_general_ci binary db1028.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970577 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970577 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970577 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970577 utf8 utf8_general_ci binary db1036.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970585 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970585 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970585 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970585 utf8 utf8_general_ci binary db1072.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978779 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978779 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978779 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978779 utf8 utf8_general_ci binary db1054.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171974680 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171974680 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171974680 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171974680 utf8 utf8_general_ci binary db1029.eqiad.wmnet: db1026.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970575 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970575 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970575 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970575 utf8 utf8_general_ci binary db1056.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171974682 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171974682 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171974682 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171974682 utf8 utf8_general_ci binary db1052.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 103222 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 103222 utf8 utf8_general_ci binary es1013.eqiad.wmnet: db1066.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978773 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978773 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978773 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978773 utf8 utf8_general_ci binary db1024.eqiad.wmnet: ops wmf_master_purge root@208.80.154.151 SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00 NULL ENABLED 101613 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@208.80.154.151 SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101613 utf8 utf8_general_ci binary ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970573 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970573 utf8 utf8_general_ci binary db1044.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970593 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970593 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970593 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970593 utf8 utf8_general_ci binary es1011.eqiad.wmnet: es1014.eqiad.wmnet: es1017.eqiad.wmnet: db1015.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171966483 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171966483 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171966483 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171966483 utf8 utf8_general_ci binary db1039.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970588 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970588 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970588 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970588 utf8 utf8_general_ci binary db1055.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171974681 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171974681 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171974681 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171974681 utf8 utf8_general_ci binary db1022.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970571 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970571 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970571 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970571 utf8 utf8_general_ci binary db1045.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970594 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970594 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970594 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970594 utf8 utf8_general_ci binary db1035.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970584 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970584 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970584 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970584 utf8 utf8_general_ci binary db1023.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101612 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101612 utf8 utf8_general_ci binary db1033.eqiad.wmnet: ops wmf_master_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-26 00:00:00NULL ENABLED 101622 utf8 utf8_general_ci binary ops wmf_master_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-26 00:00:05 NULL ENABLED 101622 utf8 utf8_general_ci binary db1037.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970586 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970586 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970586 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970586 utf8 utf8_general_ci binary db1051.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171974677 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171974677 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171974677 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171974677 utf8 utf8_general_ci binary db1034.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970583 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970583 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970583 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970583 utf8 utf8_general_ci binary db1030.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171970579 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171970579 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171970579 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171970579 utf8 utf8_general_ci binary db1065.eqiad.wmnet: ops wmf_slave_overload root@localhost SYSTEM RECURRING NULL 10 SECOND 2014-08-13 00:00:01NULL ENABLED 171978772 utf8 utf8_general_ci binary ops wmf_slave_purge root@localhost SYSTEM RECURRING NULL 15 MINUTE 2014-08-13 00:00:00 NULL ENABLED 171978772 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_sleep root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:05 NULL ENABLED 171978772 utf8 utf8_general_ci binary ops wmf_slave_wikiuser_slow root@localhost SYSTEM RECURRING NULL 30 SECOND 2014-08-13 00:00:03NULL ENABLED 171978772 utf8 utf8_general_ci binary
Change 317122 had a related patch set uploaded (by Jcrespo):
dbtools: binarly log and drop fixes on the events
Pending hosts:
- es hosts- it will not be applied
- Old masters: needs to change its events to slaves
- New masters: needs to change its events to masters
- db1019.eqiad.wmnet: it is decomissioned- needs to reset its salt grains
- Many of the >db1073, which were cloned from old masters
- In addition, db1029.eqiad.wmnet and db1021.eqiad.wmnet
Change 317123 had a related patch set uploaded (by Jcrespo):
dbtools: disable binary log for all events (not only on creation)
Change 317124 had a related patch set uploaded (by Jcrespo):
dbtools: explicitly set definer as root@localhost
Change 317123 merged by Jcrespo:
dbtools: disable binary log for all events (not only on creation)
Mentioned in SAL (#wikimedia-operations) [2016-10-21T09:13:31Z] <jynus> reviewing and applying new watchdog events to all core dbs T148790
I have applied master events to all masters, eqiad and codfw, slave events to all slaves (on the core s1-s7 shards). For x1, I have applied the slave ones to all, as all handle lots of slave connections. es shards should continue having no events.