Page MenuHomePhabricator

Ferm leftovers on labtestnet2003
Closed, ResolvedPublic

Description

The labnet* hosts don't use Ferm, as Nova configures it's own set of iptables rules and the same applies to the labtestnet* hosts.

While testing the new Stretch-based Cumin master cumin2001 with debdeploy I noticed that it failed to connect to labtestnet2003. Puppet runs were successful, but the new IP address of cumin2001 wasn't added to the $CUMIN_MASTERS macro in /etc/ferm/conf.d/00_defs.

The reason is that in commit 9f0d55323c5 the role of that host was switched from role(test) (which has profile::base::firewall) to role(wmcs:openstack::labtest::net_standby) (which doesn't have profile::base::firewall due to Nova). However, the old ferm installation (and the stale config files which were formerly added via Puppet) is still around and blocks Cumin.

While it would be possible to manually remove ferm this seems a bit brittle, so a reimage of labtestnet2003 is probably the cleanest solution given that it's a test host.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 18 2018, 9:28 AM

hey @GTirloni would you like to do this reimage? I can guide you in the process.

Script wmf-auto-reimage was launched by gtirloni on sarin.codfw.wmnet for hosts:

labtestnet2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/201809201317_gtirloni_14566_labtestnet2003_codfw_wmnet.log.

Mentioned in SAL (#wikimedia-operations) [2018-09-20T13:20:34Z] <gtirloni> T204667 reimage labtestnet2003

Completed auto-reimage of hosts:

['labtestnet2003.codfw.wmnet']

Of which those FAILED:

['labtestnet2003.codfw.wmnet']

Reimage completed. The error was caused by me running Puppet manually instead of letting wmf-auto-reimage do it.

@MoritzMuehlenhoff anything else that needs to be done for this task?

MoritzMuehlenhoff closed this task as Resolved.Sep 20 2018, 3:34 PM
MoritzMuehlenhoff claimed this task.

@MoritzMuehlenhoff anything else that needs to be done for this task?

No, all good now.