Page MenuHomePhabricator

New VMs for ML staging cluster in codfw
Closed, ResolvedPublic

Description

Site/Location: CODFW
Number of systems: 3
Service: ml-etcd-staging
Networking Requirements: internal network
Processor Requirements: 1 core
Memory: 3G
Disks: 1x20G for /
Other Requirements: Row diversity (each machine on a different row)

Machine names: ml-etcd-staging2001 - ml-etcd-staging2003

Event Timeline

klausman renamed this task from New VMs for ML staging cluster in eqiad to New VMs for ML staging cluster in codfw.Feb 24 2022, 3:55 PM

Correction: we should use ml-staging-etcdXXX as names for consistency

cookbooks.sre.hosts.decommission executed by klausman@cumin2002 for hosts: ml-etcd-staging2001

  • ml-etcd-staging2001 (FAIL)
    • Host steps raised exception:

ERROR: some step on some host failed, check the bolded items above

$ sudo cookbook sre.ganeti.makevm --vcpus 1 --memory 3 --disk 20 codfw_A ml-staging-etcd2001
[...]
+ml-staging-etcd2001                      1H IN A 10.192.0.37
+ml-staging-etcd2001                      1H IN AAAA 2620:0:860:101:10:192:0:37
[...]
$ sudo cookbook sre.ganeti.makevm --vcpus 1 --memory 3 --disk 20 codfw_B ml-staging-etcd2002
[...]
+ml-staging-etcd2002                      1H IN A 10.192.16.92                                                                                                                                           
+ml-staging-etcd2002                      1H IN AAAA 2620:0:860:102:10:192:16:92                                                                                                                         

Change 767194 had a related patch set uploaded (by Klausman; author: Klausman):

[operations/puppet@production] Add DHCP and partman info for ML staging etcd VMs

https://gerrit.wikimedia.org/r/767194

Change 767194 merged by Klausman:

[operations/puppet@production] Add DHCP and partman info for ML staging etcd VMs

https://gerrit.wikimedia.org/r/767194

Change 767221 had a related patch set uploaded (by Klausman; author: Klausman):

[operations/puppet@production] Add insetup role for ML staging etcd VMs

https://gerrit.wikimedia.org/r/767221

Change 767221 merged by Klausman:

[operations/puppet@production] Add insetup role for ML staging etcd VMs

https://gerrit.wikimedia.org/r/767221

All three VMs have been created, installed and have had at least one puppet run.

klausman claimed this task.

Change 767478 had a related patch set uploaded (by Klausman; author: Klausman):

[operations/puppet@production] Add entries for ML staging control plane VMs

https://gerrit.wikimedia.org/r/767478