Page MenuHomePhabricator

Cloud VPS Debian Bullseye deprecation
Open, Needs TriagePublic

Description

Trixie is out, which means it's time to start pushing Bullseye users to migrate.

See also:

Related Objects

StatusSubtypeAssignedTask
OpenNone
ResolvedAndrew
Resolvedtaavi
Resolvedkomla
ResolvedPhysikerwelt
Resolvedkomla
OpenNone
Resolvedstwalkerster
ResolvedLeaderboard
Resolved• bd808
Duplicatetchin
Opentchin
OpenNone
OpenNone
OpenNone
DuplicateNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
ResolvedAddshore
ResolvedXover
ResolvedJdlrobson-WMF
OpenNone
ResolvedRoySmith
OpenEevans
OpenNone
OpenNone
ResolvedAmandaNP
OpenNone
OpenNone
OpenNone

Event Timeline

fnegri triaged this task as Medium priority.Aug 20 2025, 2:28 PM

This deprecation has been not been properly communicated.

  • Nothing posted on cloud-announce
  • Timeline is TBD on Wikitech
  • Nag emails that started 2026-06-17 claimed "migration from Debian Bullseye has been underway across Cloud VPS projects for a while now" while setting a deadline of 2026-07-31.
Andrew raised the priority of this task from Medium to Unbreak Now!.Jul 30 2026, 1:08 PM

Nothing posted on cloud-announce

I think this is because communication has been in targeted per-project emails rather than broadcast. @komla probably has the correspondence history.

@Andrew , yes it is targeted.

@JJMC89 a bit of history on this.
The task was created last year.

On August 13th, a [Cloud-announce] Debian 13 "Trixie" now available on Toolforge and Cloud VPS was sent to cloud-announce encouraging a move away from Bullseye

On February 5th, Luca sent a Please plan for the Bullseye deprecation (August 2026) email.

I started reaching out to individual project owners in June.

I will clarify the messaging around the timeline on Wikitech.

Thank you.

The Trixie availability email mentions that deprecation would start in the future and does not constitute an announcement of the deprecation itself (which should preferably include timelines).
Luca's email was not sent to cloud-announce.

You need public announcements of things like this, not private communications from an individual, who can have their email blocked (as I do for sending emails nagging me to do things I had already done).

@JJMC89 well noted. Thanks for the feedback.

I missed the email sent on Jun 17 (though I'm not sure it would have changed things much); I only became aware of this after Wednesday's (Jul 29) email.

Of the instances that (I assume) are associated with me: deployment-echostore02.deployment-prep, deployment-restbase05.deployment-prep, and deployment-sessionstore06.deployment-prep (at least) are definitely going to cause deployment-prep breakage if you shut them down, and I will definitely not have any time to do anything with them before the deadline. And, I'm not comfortable committing to when I would be able to; My first priority at the moment is production machines still running Bullseye (I'm sure you can understand why I cannot pause that to spend time here).

More broadly, I think we need to get clarity on the ownership of these. It's seems —at least implicitly— that this falls to me (or Data Persistence), but I don't think that has been made explicit. I'm always happy to help, but I'm not sure that best-effort, or as-needed, is cutting it here.

I missed the email sent on Jun 17 (though I'm not sure it would have changed things much); I only became aware of this after Wednesday's (Jul 29) email.

Let's take the Beta Cluster specific discussion over to T401839: Migrate deployment-prep away from Debian Bullseye to Bookworm/Trixie.

All Cyberbot VMs are now upgraded to Trixie

All Cyberbot VMs are now upgraded to Trixie

It seems like an in-place upgrade was done there instead of the usual recreate-and-delete operation? In that case please see https://wikitech.wikimedia.org/wiki/News/2025_Cloud_VPS_Bullseye_deprecation#Can_I_upgrade_an_existing_VM_in-place,_instead_of_re-creating_it_with_a_newer_Debian_, especially since I believe outbound v6 connectivity for a tool like IABot that does a lot of outbound HTTP requests to the internet.

All Cyberbot VMs are now upgraded to Trixie

It seems like an in-place upgrade was done there instead of the usual recreate-and-delete operation? In that case please see https://wikitech.wikimedia.org/wiki/News/2025_Cloud_VPS_Bullseye_deprecation#Can_I_upgrade_an_existing_VM_in-place,_instead_of_re-creating_it_with_a_newer_Debian_, especially since I believe outbound v6 connectivity for a tool like IABot that does a lot of outbound HTTP requests to the internet.

I read in a different ticket somewhere that in place upgrades were acceptable. These VMs are going to be scrapped in the foreseeable future, but I won’t be ready to do so before the deadlines.

Hi. Can I ask if the project wlm-it-visual is OK? I ask because I've received a notification from the kind Komla Sapaty, but in Horizon everything seems already to trixie. Thanks!

taavi lowered the priority of this task from Unbreak Now! to Needs Triage.Aug 5 2026, 4:02 PM

Given that an in-place upgrade was apparently performed for my VM (T433570), I'm now pretty confused. Is it that VMs that are not already migrated will be force-updated? I don't see this described anywhere in the documentation.

Given that an in-place upgrade was apparently performed for my VM (T433570), I'm now pretty confused.

From https://wikitech.wikimedia.org/wiki/News/2025_Cloud_VPS_Bullseye_deprecation#Can_I_upgrade_an_existing_VM_in-place,_instead_of_re-creating_it_with_a_newer_Debian_release?

Can I upgrade an existing VM in-place, instead of re-creating it with a newer Debian release?

We are using the OS upgrades as an opportunity to roll out new defaults (e.g. more modern network options), which does not work if instances are upgraded in-place instead of created from scratch. The migration tracking tooling also relies on the image that an instance was created with to detect if it needs upgrading or not and your instance might be wrongly flagged for deletion. If you have specific reasons why you can not re-create the instance, please contact us.

Basically an in-place upgrade is not automatically tracked so your instance will keep showing up on automated reports.

Is it that VMs that are not already migrated will be force-updated? I don't see this described anywhere in the documentation.

No, it is not expected for WMCS to force update any instances. I see the discussion on T433570: Debian Bullseye deprecation extension for statanalyser, and I don't have an answer for who may have initiated the dist-upgrade that seems to have happened there.

Are you still trying to move the project into Toolforge? If so, that is probably the best action right now. Second best would be actually building a new instance with a base image that will be seen by our tooling as being something other than Bullseye. Third is to stay the course while making sure that @komla records that your instance is a false positive somewhere that will be noticed before anyone attempts to shut it down.

Are you still trying to move the project into Toolforge? If so, that is probably the best action right now

I am still working on that but this might take 2 - 3 weeks.

Mentioned in SAL (#wikimedia-cloud-feed) [2026-08-12T12:44:51Z] <dcaro@cloudcumin1001> START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-bastion-6 (T401804)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-08-12T12:45:45Z] <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-bastion-6 (T401804)