Page MenuHomePhabricator

In-place VM upgrades for Debian Stretch math cloud VPS instances
Closed, ResolvedPublic

Description

The following request reached us

WMCS is reaching out to you again because your project is still running on Debian Stretch.

Debian Stretch reached its end-of-life on July 1, 2022.
In order to keep our platforms secure we need to move to Debian Bullseye (or Buster).

All instances in the math project need to upgrade as soon as possible.

Instances not upgraded may be shut down at any moment and deleted by March 1st unless prior arrangements for an extended deadline has been approved by the Cloud VPS administration team.

As the math instances are heavily linked to wmflabs.org domains see e.g, T327297 it seems problematic to recreate the instances. There is no valuable data on the instances as such, however, a significant amount of configuration has been done. As supporters of FAIR research, we really want to avoid broken links as long as possible.

Therefore, it would be easier to update the instances via apt full-upgrade and friends.

We were wondering why

In-place VM upgrades are not encouraged

?

Event Timeline

taavi removed komla as the assignee of this task.
taavi added subscribers: komla, taavi.

There are three main reasons why we try to discourage in-place upgrades. First, since both the Debian base system and the Cloud VPS base setup occasionally change in a way that is not possible to automatically apply to existing systems, it's common that in-place upgrades leave behind cruft that would not be there with fresh installs, which in turn makes troubleshooting and making further changes even more difficult. Second, periodically rebuilding instances encourages having good documentation or automation for the instance setup, which helps onboard new maintainers to the project and makes it easier to re-create the instances in case something goes wrong (which is rare but not impossible, especially with in-place upgrades). And third, the fact that VMs don't change OS versions after initial installation makes the migration tracking much easier on our side.

As the math instances are heavily linked to wmflabs.org domains see e.g, T327297 it seems problematic to recreate the instances

As I said T131301#8535338 it's completely possible to move wmflabs.org domains from a single backend to another as long as you do it before the old backend is deleted.

I'm closing this task as I don't see anything actionable here (especially since all math instances have been already migrated). In the future please take these kinds of general questions up on our support channels.

Thank you. This makes a lot of sense to me. However, I am wondering why I got the email

WMCS is reaching out to you again because your project is still running on Debian Stretch.

and a follow up

Can you create a ticket so the Cloud Services team can look at this?

So maybe this is something to back-propagate to the support channels?

Thank you. This makes a lot of sense to me. However, I am wondering why I got the email

So maybe this is something to back-propagate to the support channels?

Sorry about the confusion.
I reached out to admins that did not claim their projects on the VPS purge page.

It turns out, some projects upgraded but did not claim their projects on that page.
So we thought they were still on Stretch or abandoned.

The messaging should have been clearer.