Page MenuHomePhabricator

decom furud
Closed, ResolvedPublic

Description

remove furud from all the places, dont forget to shutdown/destroy the VM, stored configs/ icinga, salt...DNS..

Event Timeline

Dzahn created this task.Jun 7 2016, 4:50 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJun 7 2016, 4:50 PM
Dzahn added a comment.Jun 7 2016, 5:08 PM

furud was meant to replace antimony but plans changed and it's not going to be used now

https://gerrit.wikimedia.org/r/#/c/292940/

https://gerrit.wikimedia.org/r/#/c/292971/

related: T123718, T111465, T137224

Change 293129 had a related patch set uploaded (by Dzahn):
decom furud

https://gerrit.wikimedia.org/r/293129

Dzahn added a comment.Jun 7 2016, 5:36 PM

root@palladium:~# puppetstoredconfigclean.rb furud.codfw.wmnet
Killing furud.codfw.wmnet...done.

[palladium:~] $ sudo puppet cert clean furud.codfw.wmnet
Notice: Revoked certificate with serial 1602

[neodymium:~] $ sudo salt-key -d furud.codfw.wmnet
The following keys are going to be deleted:

root@furud:~# shutdown -h now
W: molly-guard: SSH session detected!

[ganeti2001:~] $ sudo gnt-instance remove furud.codfw.wmnet
This will remove the volumes of the instance furud.codfw.wmnet
(including mirrors), thus removing all the data of the instance.

Dzahn added a comment.EditedJun 7 2016, 6:15 PM

Continue?
y/[n]/?: y

Tue Jun 7 17:50:41 2016 - WARNING: Could not remove disk 1 on node ganeti2003.codfw.wmnet, continuing anyway: Error 28: Operation timed out after 900023 milliseconds w
ith 0 bytes received
Failure: command execution error:
Can't remove instance's disks
[ganeti2001:~] $

furud.codfw.wmnet kvm debootstrap+default ganeti2003.codfw.wmnet ERROR_down -

Dzahn added a subscriber: akosiaris.Jun 7 2016, 6:18 PM

@akosiaris have you seen the problem above before when deleting VMs? ^

RobH added a subscriber: RobH.Jun 7 2016, 10:05 PM

furud is still showing pending salt key acceptance on the salt master.

Dzahn added a comment.Jun 7 2016, 10:47 PM

@RobH thanks, the accepted key was deleted, then this got recreated because it was still running. it's in an "ERROR_down" state now after the error above.. hmm

Dzahn added a comment.Jun 8 2016, 7:18 PM

i simply repeated the same gnt-instance remove command today and it was immediately done, no issues at all.. shrug

Change 293129 merged by Dzahn:
decom furud

https://gerrit.wikimedia.org/r/293129

Dzahn closed this task as Resolved.Jun 8 2016, 7:26 PM

@akosiaris have you seen the problem above before when deleting VMs? ^

No, but then again we haven't been deleting VMs much yet. I am not sure what that is, but it might be a network issue. Perhaps some port ferm misconfiguration. I 'll have a look.