Page MenuHomePhabricator

globaleducation outreachdashboard instance has been found to be running an unsupported operating system
Closed, ResolvedPublic

Description

Ubuntu Trusty is no longer available in Cloud VPS since Nov 2017 for new instances. The migration path is to replace such instances with Debian Stretch (or possibly Jessie), no new Ubuntu versions are supported for Wikimedia use. Around April 2019 Ubuntu Trusty support will be removed, and with it any accidental support for other versions of Ubuntu such as Xenial or Bionic.

The following VM has been detected as having been in-place-upgraded to an unsupported Ubuntu version instead of being replaced with Debian:

outreachdashboard.globaleducation.eqiad.wmflabs

Listed administrators are:

@Andrew
@AndyRussG
@awight
@dduvall
@jeremyb
@JeroenDeDauw
@Kjschiroo
@Mattfordham
@MusikAnimal
@Nischayn22
@Ragesoss
@thcipriani
@Thehelpfulone

More info in openstack browser: https://tools.wmflabs.org/openstack-browser/project/globaleducation

Event Timeline

Thanks @Krenair. What will it mean for Ubuntu support to be removed?

Migrating before then will probably not be a huge problem, but I'd like to understand the worst case scenario if there are unanticipated problems.

What will it mean for Ubuntu support to be removed?

At some point "soon" (probably not actually until April 2019) code explicitly supporting Ubuntu will be removed from ops/puppet.git. Once this happens it is likely that Puppet runs on any remaining Ubuntu instances will start failing. That will put the instances at risk of not being accessible via ssh, NFS mounts failing, and not being reconfigured for other core infrastructure changes that may happen in the larger Cloud VPS environment. The worst case scenario is fundamentally the instance becoming an unrecoverable brick.

We are skipping this project in today's operation due to short notice period.

Wes and I spent today getting a new debian server spun up, configured, and set for deployment. We got the app up and running, so what remains to be done is:

  • Put up a notice about pending downtime
  • Disable the old server
  • Create a mysql dump
  • Load the dump on new server
  • Redirect outreachdashboard.wmflabs.org to the new server
  • Verify that things are working smoothly
  • Destroy the old server

Our all-staff meeting is next week and we want to have plenty of wiggle room in case things go wrong, so we are planning to complete these final steps the week of February 12.

Ragesoss claimed this task.

Done! We had the server up and configured two weeks ago and did the switchover, and I just did the final step of destroying the Ubuntu instance.