Page MenuHomePhabricator

Upgrade codfw,ulsfo,esams LVS to jessie
Closed, ResolvedPublic

Description

codfw is on trusty, the rest are precise. eqiad hardware is being replaced soon and can wait until then. Should be fairly trivial thanks to testing on jessie cache nodes + codfw trusty LVS nodes already.

Note: will need an extra reboot after initial puppetization, in order for puppet-configured ethernet kernel module params to take effect for performance.

  • package pybal for jessie
  • codfw lvs200[456]
  • codfw lvs200[123]
  • ulsfo lvs400[34]
  • ulsfo lvs400[12]
  • esams lvs300[34]
  • esams lvs300[12]

Event Timeline

BBlack raised the priority of this task from to Needs Triage.
BBlack updated the task description. (Show Details)
BBlack added a project: acl*sre-team.
BBlack subscribed.
BBlack set Security to None.
BBlack added subscribers: faidon, Joe.

@fgiunchedi packaged up pybal (Thanks!).

Mostly what's left here is taking a stab at the installs themselves. At each site the higher-numbered half are the inactive ones, and codfw in particular doesn't see "real" traffic, so the order in the list at the top of the task is appropriate, with some testing and validation before affecting the live, active stuff.

Change 234260 had a related patch set uploaded (by BBlack):
codfw LVS installer -> jessie

https://gerrit.wikimedia.org/r/234260

Change 234260 merged by BBlack:
codfw LVS installer -> jessie

https://gerrit.wikimedia.org/r/234260

lvs2006 reinstall went fine. They need reboots after successful puppetization for bnx2x params, but that aside this should go pretty smoothly I think.

All codfw LVS are on jessie. Should investigate eth params (blocking task) before doing the other DCs or putting huge loads of traffic here.

Change 238161 had a related patch set uploaded (by BBlack):
switch lvs[34]00x installer to jessie

https://gerrit.wikimedia.org/r/238161

Change 238161 merged by BBlack:
switch lvs[34]00x installer to jessie

https://gerrit.wikimedia.org/r/238161

Backups LVSes in ulsfo and esams upgraded. Should fail over to them for testing before upgrading the primaries.

ulsfo: tested backups on jessie most of today, and converted primaries this evening (whole site now on jessie LVS)
esams: will likely do the same tomorrow

Change 239416 had a related patch set uploaded (by BBlack):
define interface::manual for jessie LVS eth1-3 tweaks

https://gerrit.wikimedia.org/r/239416

Change 239416 merged by BBlack:
define interface::manual for jessie LVS eth1-3 tweaks

https://gerrit.wikimedia.org/r/239416

BBlack claimed this task.
BBlack updated the task description. (Show Details)
BBlack moved this task from Actively Servicing to Done on the Traffic board.