Page MenuHomePhabricator

cloudvps: phlogiston project trusty deprecation
Closed, ResolvedPublic

Description

Ubuntu Trusty is no longer available in Cloud VPS since Nov 2017 for new instances. However, the EOL of Trusty is approaching in 2019 and we need to move to Debian Stretch before that date.

All instances in the phlogiston project needs to upgrade as soon as possible.

The list of affected VMs is:

  • phlogiston-2.phlogiston.eqiad.wmflabs
  • phlogiston-3.phlogiston.eqiad.wmflabs

Listed administrator are:

More info in openstack browser: https://tools.wmflabs.org/openstack-browser/project/phlogiston

Work Breakdown

  • Get more capacity (Done, see comment below)
  • build dev dev server
    • configure
    • update install docs
    • migrate DNS
    • complete a full data run on new server
    • validate data
    • bring current with incrementals, and automate
    • validate function over several days
  • build new production server
    • configure
    • complete a full data run
    • update install docs
    • complete a full data run on new server
    • validate data
    • bring current with incrementals, and automate
    • validate function over several days
    • migrate DNS
  • Notify that old servers are not needed

Details

Related Gerrit Patches:

Event Timeline

Krenair triaged this task as Normal priority.Sep 17 2018, 4:28 PM
Krenair created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 17 2018, 4:28 PM
Krenair updated the task description. (Show Details)Sep 17 2018, 4:28 PM

Hey @JAufrecht @MBinder_WMF @Niharika! Just a friendly reminder that you should get rid of your Trusty instances as described in https://wikitech.wikimedia.org/wiki/News/Trusty_deprecation#Cloud_VPS_projects. The deadline is 2018-12-18. Please get in contact if you need help. Also please assign this task to an individual.

@bd808 @Krenair Could I temporarily get more capacity so I can throw in a new server for rolling upgrade? Same size as phlogiston-3 should be fine.

I expect so, I think right now this happens under the existing quota request/review process at Cloud-VPS (Quota-requests) - note the meeting time there

The easiest thing for more quota right now is to start using the "eqiad1-r" region where each project has the same amount of quota as they have in the legacy eqiad region. I think that @Andrew is still using an allow list to flag which projects have access to the new region via Horizon, so that is probably the only change that we need to make to give @JAufrecht space to build a new instance that would solve both the Trusty depreciation and Neutron migration problems for his project.

Change 469441 had a related patch set uploaded (by Alex Monk; owner: Alex Monk):
[operations/puppet@production] horizon: Change phlogiston's enabled region to eqiad1-r

https://gerrit.wikimedia.org/r/469441

Change 469441 merged by Andrew Bogott:
[operations/puppet@production] horizon: Change phlogiston's enabled region to eqiad1-r

https://gerrit.wikimedia.org/r/469441

You should now be able to create new VMs in eqiad1-r. Let me know if you run into any trouble.

JAufrecht updated the task description. (Show Details)

Another ping. Deadline is approaching next month (2018-12-18).

JAufrecht updated the task description. (Show Details)Nov 26 2018, 5:15 PM

Status update: New Dev server has been set up; everything works except for a few R commands that perhaps have been deprecated or changed. They have been deprecated.

It now needs a complete end-to-end run for all scopes, which could take a few days.

JAufrecht updated the task description. (Show Details)Nov 26 2018, 5:23 PM
JAufrecht updated the task description. (Show Details)Dec 3 2018, 11:06 PM

Ping. Today is Friday and the deadline is Tuesday.

JAufrecht updated the task description. (Show Details)Dec 16 2018, 5:38 AM

The new production server has been set up and is populating. The remaining server on eqiad, phlogiston-3, can be removed. All phlogiston services are running on eqiad1-r, and there is no possibility of data loss if phlogiston-3 is removed.

JAufrecht closed this task as Resolved.Dec 17 2018, 5:27 PM

Mentioned in SAL (#wikimedia-cloud) [2018-12-17T17:57:33Z] <andrewbogott> deleting phlogiston-3 as per https://phabricator.wikimedia.org/T204551