Change Details

Due to the failure of cloudvirt1018 last month both of our database instances at the time suffered disk corruption and the work to deal get stuff working again happened in {T216404} and {T216067}. The current situation looking at db-labs.php is that: * deployment-db03 is unused (theoretically, I haven't dug into what else could be talking to this beyond MW so don't rely on it) * deployment-db04 is the master * deployment-db05 is a fresh slave which we should keep and possibly make the new master When investigating puppet failures today I noticed that deployment-db04 had all sorts of nonsense like puppet function files overwritten with apt data and junk. I do not trust it and we should delete it. db03 is in a similar position though it's at least seemingly unused.

Due to the failure of cloudvirt1018 last month both of our database instances at the time suffered disk corruption and the work to deal get stuff working again happened in {T216404} and {T216067}. The current situation looking at db-labs.php is that: * deployment-db03 is unused (theoretically, I haven't dug into what else could be talking to this beyond MW so don't rely on it) * deployment-db04 is the master * deployment-db05 is a fresh slave which we should keep and possibly make the new master When investigating puppet failures today I noticed that deployment-db04 had all sorts of nonsense like puppet function files overwritten with apt data and junk. I do not trust it and we should delete it. db03 is in a similar position though it's at least seemingly unused. Added to which they're jessie and {T218729}.

Due to the failure of cloudvirt1018 last month both of our database instances at the time suffered disk corruption and the work to deal get stuff working again happened in {T216404} and {T216067}. The current situation looking at db-labs.php is that: * deployment-db03 is unused (theoretically, I haven't dug into what else could be talking to this beyond MW so don't rely on it) * deployment-db04 is the master * deployment-db05 is a fresh slave which we should keep and possibly make the new master When investigating puppet failures today I noticed that deployment-db04 had all sorts of nonsense like puppet function files overwritten with apt data and junk. I do not trust it and we should delete it. db03 is in a similar position though it's at least seemingly unused. Added to which they're jessie and {T218729}.