Page MenuHomePhabricator
Create Task
Maniphest T261724

cloudgw: evaluate / validate setup in codfw1dev
Closed, ResolvedPublic
Actions

Description

This task is to track the work of evaluating the cloudgw setup in codfw1dev, as described in the 2020 network refresh project:

https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/2020_Network_refresh

  • introduce basic puppet support
  • reimage labtestvirt2003 as Debian Buster
  • enable new puppet code in labtestvirt2003
  • test basic network bits
  • test relocating neutron functions to cloudgw
  • repurpose/rename labtestvirt2003 as cloudgw2001-dev,

Details

SubjectRepoBranchLines +/-
templates/57.15.185.in-addr.arpa: add missing PTR record for neutron virtual addressoperations/dnsmaster+1 -0
wikimediacloud.org: refresh codfw1dev addresses with cloudgw changesoperations/dnsmaster+8 -5
cloudgw: install latest nftables packageoperations/puppetproduction+3 -1
cloudgw: force installation of latest kerneloperations/puppetproduction+9 -2
cloudgw: fix unset variable in interfaces templateoperations/puppetproduction+1 -0
openstack: pdns recursor: allow querys from extra CIDRsoperations/puppetproduction+19 -4
hieradata: labtestvirt2003: cloudgw: fix NIC namesoperations/puppetproduction+2 -2
cloudgw: add support for selecting NICs using hieraoperations/puppetproduction+23 -15
cloudgw: add routes for internal VM addressingoperations/puppetproduction+4 -0
neutron: add option to disable customizations if cloudgw is enabledoperations/puppetproduction+11 -1
cloudgw: basefirewall: accept ICMP packetsoperations/puppetproduction+1 -0
cloudgw: refresh network configurationoperations/puppetproduction+13 -6
cloudgw: refresh routing configurationoperations/puppetproduction+12 -4
openstack: codfw1dev: refresh external connection for neutronoperations/puppetproduction+7 -7
cloudgw: interfaces: don't bring up interfaces recursivelyoperations/puppetproduction+0 -2
cloudgw: refresh network config for the PoCoperations/puppetproduction+14 -15
openstack: cloudgw: replace dots with colons when building host IPv6operations/puppetproduction+2 -2
openstack: cloudgw: introduce native vlan for easier reimagingoperations/puppetproduction+4 -12
openstack: cloudgw: add hiera keys for basic network bitsoperations/puppetproduction+25 -17
openstack: cloudgw: drop unused orig_nic hiera keyoperations/puppetproduction+0 -2
hiera: labtestvirt2003: fix hiera key namesoperations/puppetproduction+3 -3
cloudgw: hardcode mapped IPv6 interfaceoperations/puppetproduction+3 -2
cloudgw: interfaces: template fixesoperations/puppetproduction+1 -1
cloudgw: interfaces: more template fixesoperations/puppetproduction+3 -1
cloudgw: interfaces: avoid augeas loopoperations/puppetproduction+0 -2
cloudgw: rt_table: fix newline at the end of fileoperations/puppetproduction+1 -1
cloudgw: add templated network interfaces fileoperations/puppetproduction+61 -27
labtestvirt2003: hiera: fix addressingoperations/puppetproduction+8 -7
cloudgw: bonding: specify original NICoperations/puppetproduction+4 -1
cloudgw: introduce bonding+trunkingoperations/puppetproduction+88 -8
nftables: basefirewall: allow monitoring hostsoperations/puppetproduction+28 -0
cloudgw: protect sets in templates against empty puppet varsoperations/puppetproduction+7 -1
nftables: basefirewall: allow common input trafficoperations/puppetproduction+6 -0
cloudgw: use newer nftables and kerneloperations/puppetproduction+16 -2
labtestvirt2003: reimage as busteroperations/puppetproduction+0 -1
nftables: drop service paramsoperations/puppetproduction+3 -6
nftables: use wmflib::ensure for the service parameteroperations/puppetproduction+13 -13
nftables: reorder masking operations to don't race with service state changesoperations/puppetproduction+9 -9
nftables: mask the service if not runningoperations/puppetproduction+9 -0
nftables: fix service unit overrideoperations/puppetproduction+2 -0
nftables: override systemd service fileoperations/puppetproduction+1 -0
nftables: add infrastructure for customizing the rulesetoperations/puppetproduction+56 -2
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
gerritbot added a comment.Sep 22 2020, 2:48 PM

Change 629143 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: rt_table: fix newline at the end of file

https://gerrit.wikimedia.org/r/629143

gerritbot added a comment.Sep 22 2020, 2:50 PM

Change 629143 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: rt_table: fix newline at the end of file

https://gerrit.wikimedia.org/r/629143

gerritbot added a comment.Sep 22 2020, 2:56 PM

Change 629146 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: interfaces: avoid augeas loop

https://gerrit.wikimedia.org/r/629146

gerritbot added a comment.Sep 22 2020, 3:00 PM

Change 629146 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: interfaces: avoid augeas loop

https://gerrit.wikimedia.org/r/629146

gerritbot added a comment.Sep 22 2020, 3:10 PM

Change 629150 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: interfaces: more template fixes

https://gerrit.wikimedia.org/r/629150

gerritbot added a comment.Sep 22 2020, 3:15 PM

Change 629150 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: interfaces: more template fixes

https://gerrit.wikimedia.org/r/629150

gerritbot added a comment.Sep 22 2020, 3:19 PM

Change 629151 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: interfaces: template fixes

https://gerrit.wikimedia.org/r/629151

gerritbot added a comment.Sep 22 2020, 3:20 PM

Change 629151 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: interfaces: template fixes

https://gerrit.wikimedia.org/r/629151

gerritbot added a comment.Sep 22 2020, 4:10 PM

Change 629164 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: hardcode mapped IPv6 interface

https://gerrit.wikimedia.org/r/629164

gerritbot added a comment.Sep 22 2020, 4:12 PM

Change 629164 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: hardcode mapped IPv6 interface

https://gerrit.wikimedia.org/r/629164

gerritbot added a comment.Sep 23 2020, 9:06 AM

Change 629323 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] hiera: labtestvirt2003: fix hiera key names

https://gerrit.wikimedia.org/r/629323

gerritbot added a comment.Sep 23 2020, 9:12 AM

Change 629323 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] hiera: labtestvirt2003: fix hiera key names

https://gerrit.wikimedia.org/r/629323

gerritbot added a comment.Sep 29 2020, 10:04 AM

Change 630778 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] openstack: cloudgw: drop unused orig_nic hiera key

https://gerrit.wikimedia.org/r/630778

gerritbot added a comment.Sep 29 2020, 10:05 AM

Change 630778 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] openstack: cloudgw: drop unused orig_nic hiera key

https://gerrit.wikimedia.org/r/630778

gerritbot added a comment.Sep 29 2020, 11:10 AM

Change 630809 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] openstack: cloudgw: add hiera keys for basic network bits

https://gerrit.wikimedia.org/r/630809

gerritbot added a comment.Sep 29 2020, 11:15 AM

Change 630809 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] openstack: cloudgw: add hiera keys for basic network bits

https://gerrit.wikimedia.org/r/630809

ops-monitoring-bot added a comment.Sep 29 2020, 11:18 AM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202009291118_aborrero_29762_labtestvirt2003_codfw_wmnet.log.

ops-monitoring-bot added a comment.Sep 29 2020, 11:35 AM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

Of which those FAILED:

['labtestvirt2003.codfw.wmnet']
gerritbot added a comment.Sep 29 2020, 11:47 AM

Change 630812 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] openstack: cloudgw: introduce native vlan for easier reimaging

https://gerrit.wikimedia.org/r/630812

gerritbot added a comment.Oct 1 2020, 3:33 PM

Change 630812 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] openstack: cloudgw: introduce native vlan for easier reimaging

https://gerrit.wikimedia.org/r/630812

ops-monitoring-bot added a comment.Oct 1 2020, 3:34 PM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202010011534_aborrero_15026_labtestvirt2003_codfw_wmnet.log.

ops-monitoring-bot added a comment.Oct 1 2020, 3:34 PM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

Of which those FAILED:

['labtestvirt2003.codfw.wmnet']
ops-monitoring-bot added a comment.Oct 1 2020, 3:35 PM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202010011535_aborrero_15187_labtestvirt2003_codfw_wmnet.log.

ops-monitoring-bot added a comment.Oct 1 2020, 3:42 PM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

Of which those FAILED:

['labtestvirt2003.codfw.wmnet']
gerritbot added a comment.Oct 1 2020, 4:04 PM

Change 631500 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] openstack: cloudgw: replace dots with colons when building host IPv6

https://gerrit.wikimedia.org/r/631500

gerritbot added a comment.Oct 1 2020, 4:25 PM

Change 631500 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] openstack: cloudgw: replace dots with colons when building host IPv6

https://gerrit.wikimedia.org/r/631500

ops-monitoring-bot added a comment.Oct 2 2020, 9:16 AM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202010020916_aborrero_15594_labtestvirt2003_codfw_wmnet.log.

ops-monitoring-bot added a comment.Oct 2 2020, 9:16 AM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

Of which those FAILED:

['labtestvirt2003.codfw.wmnet']
Stashbot added a comment.Oct 2 2020, 9:16 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-02T09:16:54Z] <arturo> [codfw1dev] trying the labtestvirt2003 (cloudgw) reimage again (T261724)

ops-monitoring-bot added a comment.Oct 2 2020, 9:17 AM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202010020917_aborrero_15759_labtestvirt2003_codfw_wmnet.log.

ops-monitoring-bot added a comment.Oct 2 2020, 10:17 AM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

Of which those FAILED:

['labtestvirt2003.codfw.wmnet']
ops-monitoring-bot added a comment.Oct 8 2020, 10:28 AM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202010081028_aborrero_31551_labtestvirt2003_codfw_wmnet.log.

Stashbot added a comment.Oct 8 2020, 10:28 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-08T10:28:30Z] <arturo> [codfw1dev] reimaging labtestvirt2003 (cloudgw) T261724

ops-monitoring-bot added a comment.Oct 8 2020, 11:22 AM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

Of which those FAILED:

['labtestvirt2003.codfw.wmnet']
gerritbot added a comment.Oct 8 2020, 12:21 PM

Change 632904 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] hieradata: labtestvirt2003: refresh network data for cloudgw PoC with latest allocations

https://gerrit.wikimedia.org/r/632904

gerritbot added a comment.Oct 8 2020, 1:15 PM

Change 632904 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: refresh network config for the PoC

https://gerrit.wikimedia.org/r/632904

gerritbot added a comment.Oct 8 2020, 2:54 PM

Change 632945 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: interfaces: don't bring up interfaces recursively

https://gerrit.wikimedia.org/r/632945

gerritbot added a comment.Oct 8 2020, 2:57 PM

Change 632945 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: interfaces: don't bring up interfaces recursively

https://gerrit.wikimedia.org/r/632945

Stashbot added a comment.Oct 9 2020, 8:47 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-09T08:47:40Z] <arturo> [codfw1dev] root@cloudcontrol2001-dev:~# openstack subnet delete a5ab5362-4ffb-4059-9ff7-391e22dcf3bc (T261724)

Stashbot added a comment.Oct 9 2020, 8:49 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-09T08:49:24Z] <arturo> [codfw1dev] root@cloudcontrol2001-dev:~# openstack subnet create --network wan-transport-codfw --gateway 185.15.57.9 --no-dhcp --subnet-range 185.15.57.8/30 cloud-gw-transport-codfw (T261724)

Stashbot added a comment.Oct 9 2020, 9:10 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-09T09:10:33Z] <arturo> [codfw1dev] root@cloudcontrol2001-dev:~# openstack router set --external-gateway wan-transport-codfw --fixed-ip subnet=cloud-gw-transport-codfw,ip-address=185.15.57.10 cloudinstances2b-gw (T261724)

Stashbot added a comment.Oct 9 2020, 9:12 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-09T09:12:48Z] <arturo> [codfw1dev] root@cloudcontrol2001-dev:~# openstack subnet delete 31214392-9ca5-4256-bff5-1e19a35661de (cloud-instances-transport1-b-codfw - 208.80.153.184/29) (T261724)

gerritbot added a comment.Oct 9 2020, 9:17 AM

Change 633155 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] openstack: codfw1dev: refresh external connection for neutron

https://gerrit.wikimedia.org/r/633155

gerritbot added a comment.Oct 9 2020, 9:18 AM

Change 633155 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] openstack: codfw1dev: refresh external connection for neutron

https://gerrit.wikimedia.org/r/633155

Stashbot added a comment.Oct 9 2020, 9:22 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-09T09:22:44Z] <arturo> [codfwd1dev] rebooting cloudnet boxes for bridge and vlan changes (T261724)

Stashbot added a comment.Oct 9 2020, 10:15 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-09T10:15:10Z] <arturo> [codfwd1ev] root@cloudcontrol2001-dev:~# openstack router set --disable-snat cloudinstances2b-gw --external-gateway wan-transport-codfw (T261724)

• ayounsi closed subtask T263622: codfw: more vlans setup changes in the cloudgw PoC as Resolved.Oct 13 2020, 7:23 AM
gerritbot added a comment.Oct 13 2020, 11:01 AM

Change 633711 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: refresh routing configuration

https://gerrit.wikimedia.org/r/633711

gerritbot added a comment.Oct 13 2020, 11:05 AM

Change 633711 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: refresh routing configuration

https://gerrit.wikimedia.org/r/633711

gerritbot added a comment.Oct 13 2020, 3:53 PM

Change 633775 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: refresh network configuration

https://gerrit.wikimedia.org/r/633775

gerritbot added a comment.Oct 13 2020, 3:55 PM

Change 633775 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: refresh network configuration

https://gerrit.wikimedia.org/r/633775

gerritbot added a comment.Oct 15 2020, 11:47 AM

Change 634209 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: basefirewall: accept ICMP packets

https://gerrit.wikimedia.org/r/634209

gerritbot added a comment.Oct 15 2020, 11:49 AM

Change 634209 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: basefirewall: accept ICMP packets

https://gerrit.wikimedia.org/r/634209

gerritbot added a comment.Oct 15 2020, 12:50 PM

Change 634223 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] neutron: add option to disable customizations if cloudgw is enabled

https://gerrit.wikimedia.org/r/634223

gerritbot added a comment.Oct 15 2020, 12:58 PM

Change 634223 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] neutron: add option to disable customizations if cloudgw is enabled

https://gerrit.wikimedia.org/r/634223

Stashbot added a comment.Oct 15 2020, 1:56 PM

Mentioned in SAL (#wikimedia-cloud) [2020-10-15T13:56:16Z] <arturo> [codfw1dev] drop neutron l3 agent hacks in cloudnet2002/2003-dev (T261724)

aborrero updated the task description. (Show Details)Oct 15 2020, 1:56 PM
Stashbot added a comment.Oct 15 2020, 3:17 PM

Mentioned in SAL (#wikimedia-cloud) [2020-10-15T15:17:30Z] <arturo> [codfw1dev] try cleaning up anything related to address scopes in the neutron database (T261724)

gerritbot added a comment.Oct 16 2020, 8:35 AM

Change 634470 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: add routes for internal VM addressing

https://gerrit.wikimedia.org/r/634470

gerritbot added a comment.Oct 16 2020, 8:36 AM

Change 634470 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: add routes for internal VM addressing

https://gerrit.wikimedia.org/r/634470

Stashbot added a comment.Oct 16 2020, 9:01 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-16T09:01:16Z] <arturo> [codfw1dev] basic network connectivity seems stable after cleaning up everything related to address scopes (T261724)

Stashbot added a comment.Oct 16 2020, 9:18 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-16T09:18:19Z] <arturo> [codfw1dev] live-hacked cloudservices2002-dev /etc/powerdns/recursor.conf file to include cloud-codfw1dev-floating CIDR (185.15.57.0/29) while https://gerrit.wikimedia.org/r/c/operations/puppet/+/634050 is in review, so VMs with a floating IP can query the DNS recursor (T261724)

gerritbot added a comment.Oct 16 2020, 11:50 AM

Change 634490 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: add support for selecting NICs using hiera

https://gerrit.wikimedia.org/r/634490

aborrero updated the task description. (Show Details)Oct 16 2020, 11:51 AM
gerritbot added a comment.Oct 16 2020, 11:52 AM

Change 634490 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: add support for selecting NICs using hiera

https://gerrit.wikimedia.org/r/634490

gerritbot added a comment.Oct 16 2020, 11:54 AM

Change 634492 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] hieradata: labtestvirt2003: cloudgw: fix NIC names

https://gerrit.wikimedia.org/r/634492

gerritbot added a comment.Oct 16 2020, 11:55 AM

Change 634492 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] hieradata: labtestvirt2003: cloudgw: fix NIC names

https://gerrit.wikimedia.org/r/634492

ops-monitoring-bot added a comment.Oct 20 2020, 2:48 PM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202010201448_aborrero_17241_labtestvirt2003_codfw_wmnet.log.

Stashbot added a comment.Oct 20 2020, 2:49 PM

Mentioned in SAL (#wikimedia-cloud) [2020-10-20T14:49:06Z] <arturo> [codfw1dev] reimaging labtestvirt2003 (cloudgw) to test puppet code (T261724)

gerritbot added a comment.Oct 20 2020, 3:26 PM

Change 635314 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] openstack: pdns recursor: allow querys from extra CIDRs

https://gerrit.wikimedia.org/r/635314

gerritbot added a comment.Oct 20 2020, 3:39 PM

Change 635314 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] openstack: pdns recursor: allow querys from extra CIDRs

https://gerrit.wikimedia.org/r/635314

Stashbot added a comment.Oct 20 2020, 3:47 PM

Mentioned in SAL (#wikimedia-cloud) [2020-10-20T15:47:05Z] <arturo> changing DNS recursor ACLs (https://gerrit.wikimedia.org/r/c/operations/puppet/+/635314) this can be reverted any time if it causes problems (T261724)

ops-monitoring-bot added a comment.Oct 20 2020, 4:04 PM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

Of which those FAILED:

['labtestvirt2003.codfw.wmnet']
ops-monitoring-bot added a comment.Oct 21 2020, 8:55 AM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202010210855_aborrero_18426_labtestvirt2003_codfw_wmnet.log.

Stashbot added a comment.Oct 21 2020, 8:56 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-21T08:56:38Z] <arturo> [codfw1dev] reimaging labtestvirt2003 (cloudgw) to test puppet code (T261724)

gerritbot added a comment.Oct 21 2020, 9:51 AM

Change 635514 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: fix unset variable in interfaces template

https://gerrit.wikimedia.org/r/635514

gerritbot added a comment.Oct 21 2020, 9:56 AM

Change 635514 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: fix unset variable in interfaces template

https://gerrit.wikimedia.org/r/635514

ops-monitoring-bot added a comment.Oct 21 2020, 10:01 AM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

and were ALL successful.

gerritbot added a comment.Oct 21 2020, 10:20 AM

Change 635519 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: force installation of latest kernel

https://gerrit.wikimedia.org/r/635519

gerritbot added a comment.Oct 21 2020, 10:29 AM

Change 635519 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: force installation of latest kernel

https://gerrit.wikimedia.org/r/635519

ops-monitoring-bot added a comment.Oct 21 2020, 10:31 AM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202010211030_aborrero_5050_labtestvirt2003_codfw_wmnet.log.

Stashbot added a comment.Oct 21 2020, 10:31 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-21T10:31:23Z] <arturo> [codfw1dev] reimaging labtestvirt2003 (cloudgw) to test puppet code (T261724)

ops-monitoring-bot added a comment.Oct 21 2020, 11:49 AM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

Of which those FAILED:

['labtestvirt2003.codfw.wmnet']
gerritbot added a comment.Oct 21 2020, 2:51 PM

Change 635555 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudgw: install latest nftables package

https://gerrit.wikimedia.org/r/635555

gerritbot added a comment.Oct 21 2020, 2:53 PM

Change 635555 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudgw: install latest nftables package

https://gerrit.wikimedia.org/r/635555

ops-monitoring-bot added a comment.Oct 22 2020, 11:06 AM

Script wmf-auto-reimage was launched by aborrero on cumin2001.codfw.wmnet for hosts:

labtestvirt2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202010221106_aborrero_17826_labtestvirt2003_codfw_wmnet.log.

ops-monitoring-bot added a comment.Oct 22 2020, 11:54 AM

Completed auto-reimage of hosts:

['labtestvirt2003.codfw.wmnet']

Of which those FAILED:

['labtestvirt2003.codfw.wmnet']
gerritbot added a comment.Oct 23 2020, 9:56 AM

Change 635965 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/dns@master] wikimediacloud.org: refresh codfw1dev addresses with cloudgw changes

https://gerrit.wikimedia.org/r/635965

gerritbot added a comment.Oct 23 2020, 10:07 AM

Change 635965 merged by Arturo Borrero Gonzalez:
[operations/dns@master] wikimediacloud.org: refresh codfw1dev addresses with cloudgw changes

https://gerrit.wikimedia.org/r/635965

Stashbot added a comment.Oct 23 2020, 10:09 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-23T10:09:04Z] <arturo> [codf1dev] doing DNS changes for the cloudgw PoC, including designate and https://gerrit.wikimedia.org/r/c/operations/dns/+/635965 (T261724)

gerritbot added a comment.Oct 23 2020, 10:20 AM

Change 635990 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/dns@master] templates/57.15.185.in-addr.arpa: add missing PTR record for neutron virtual address

https://gerrit.wikimedia.org/r/635990

gerritbot added a comment.Oct 23 2020, 10:24 AM

Change 635990 merged by Arturo Borrero Gonzalez:
[operations/dns@master] templates/57.15.185.in-addr.arpa: add missing PTR record for neutron virtual address

https://gerrit.wikimedia.org/r/635990

Stashbot added a comment.Oct 23 2020, 11:30 AM

Mentioned in SAL (#wikimedia-cloud) [2020-10-23T11:30:56Z] <arturo> [codfw1dev] openstack --os-project-id cloudinfra-codfw1dev recordset create --type PTR --record nat.cloudgw.codfw1dev.wikimediacloud.org. --description "created by hand" 0-29.57.15.185.in-addr.arpa. 1.0-29.57.15.185.in-addr.arpa. (T261724)

aborrero mentioned this in T265288: Enable L3 routing on cloudsw nodes.Oct 30 2020, 10:26 AM
aborrero added a subtask: Unknown Object (Task).Nov 30 2020, 10:47 AM
aborrero closed this task as Resolved.Dec 22 2020, 2:22 PM

We consider this evaluation to be successfully completed.

aborrero mentioned this in T270704: cloud: introduce new edge network architecture for eqiad1 and codfw1dev.Dec 22 2020, 2:25 PM
aborrero removed a subtask: Unknown Object (Task).
aborrero mentioned this in T271519: codfw1dev: repurpose/rename labtestvirt2003.codfw.wmnet as cloudgw2001-dev.codfw.wmnet.Jan 8 2021, 10:21 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits