on our salt master, neodymium ,we have a bunch of unaccepted keys. if all of them are recent (re)-installs we should accept them, but probably not just blindly?
Unaccepted Keys: aluminium.wikimedia.org cp3022.esams.wmnet db2001.codfw.wmnet db2002.codfw.wmnet db2003.codfw.wmnet db2004.codfw.wmnet db2005.codfw.wmnet db2008.codfw.wmnet db2009.codfw.wmnet mw1291.eqiad.wmnet mw2087.codfw.wmnet
These are servers to be decomission, but I cannot access all of them to shut them down T125827:
db2001.codfw.wmnet db2002.codfw.wmnet db2003.codfw.wmnet db2004.codfw.wmnet db2005.codfw.wmnet db2008.codfw.wmnet db2009.codfw.wmnet
The keys for mw* were all caused by broken IPMI settings on the servers: The wmf-reimage script removes the Salt key in the initial step, but then fails to initiate the PXE reboot due failing IPMI commands. The IPMI settings are in the process of being fixed and I've fixed up the Salt keys.
The salt key for mw1291 was removed; mw129[12] were image scalers, which were repurposed as Thumbor servers.
> $ salt-key -l pre > Unaccepted Keys: > aluminium.wikimedia.org >
Checked aluminium and accepted the key. Seems like I had forgotten it back then.
Also accepted the key cp3022, which was recently reinstalled. All done now. We should have an Icinga check for this, I'll open a Phab task.