Page MenuHomePhabricator
Create Task
Maniphest T306098

Cloud VPS "swift" project Stretch deprecation
Closed, ResolvedPublic
Actions

Description

The end of life of Debian Stretch is approaching in 2022 and we need to move to Debian Bullseye (or Buster) before that date.

All instances in the swift project need to upgrade as soon as possible. Instances not upgraded by 2022-05-01 may be subject to deletion unless prior arrangements for an extended deadline has been approved by the Cloud VPS administration team.

Remaining Debian Stretch instances (live report):

Listed administrators are:

See also:

More info on current project instances is available via openstack browser.

Details

Due Date
Apr 30 2022, 11:59 PM

Related Objects
Search...

Event Timeline

StrikerBot triaged this task as Medium priority.Apr 13 2022, 5:00 PM
StrikerBot created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 13 2022, 5:00 PM
Zabe changed the edit policy from "Custom Policy" to "All Users".Apr 13 2022, 5:30 PM
MatthewVernon added a parent task: T279637: Upgrade Swift ms cluster to Bullseye and revisit mkfs.xfs options.Apr 14 2022, 1:29 PM
MatthewVernon removed a parent task: T279637: Upgrade Swift ms cluster to Bullseye and revisit mkfs.xfs options.
MatthewVernon added a subtask: T279637: Upgrade Swift ms cluster to Bullseye and revisit mkfs.xfs options.
MatthewVernon added a comment.Apr 14 2022, 1:32 PM

I think it makes sense to keep these available until we've finished upgrading production swift from stretch (in case we need a test system in a hurry) - that work is T279637.

Is that OK? Do I need to raise some ticket elsewhere to get that approved?

MatthewVernon added a subscriber: Andrew.Apr 14 2022, 1:34 PM

@Andrew I think you own T289883 so are you the person to ask about extensions, please?

Andrew added a comment.Apr 19 2022, 8:58 PM

Delaying until the prod systems are upgraded is OK. I would've expected the deadline for prod servers to be before the cloud-vps deadline though!

Krinkle mentioned this in T313195: Cloud VPS Project Stretch Deprecation - Exemptions.Oct 13 2022, 8:43 PM
Stashbot added a comment.Oct 19 2022, 8:44 PM

Mentioned in SAL (#wikimedia-cloud) [2022-10-19T20:44:04Z] <andrewbogott> restarting ms-be-10 and ms-fe-10 due to comments on T306098

Andrew added a comment.Oct 19 2022, 8:44 PM

Just checking in here -- is this still blocked on prod upgrades?

Andrew added a comment.Oct 31 2022, 3:28 PM

?

MatthewVernon added a comment.Nov 1 2022, 1:58 PM

Sorry!
We do still have a couple of stretch hosts in production, alas; hopefully gone by end of this quarter. Then I have to work out how to migrated the virtual swift cluster without breaking it...

Andrew added a comment.Dec 6 2022, 6:19 PM

Hi! Any progress on these upgrades?

MatthewVernon added a comment.Dec 8 2022, 11:22 AM

Well, I completed T322547 (upload rclone >=1.60 to Debian) today, which is the next step on T299125 (replacing swiftrepl with rclone), which is the thing blocking T279637 (getting the last swift nodes off stretch).
That turned out to be (much) more work than anticipated (a fiddly problem with pristine-tar, about 15 new build-dependencies, quite a lot of packaging work); still aiming to have the work for T299125 done this quarter, but I suspect the deployment freeze means it'll now get actually rolled out in the start of 2023 :-/

Andrew added a comment.Jan 17 2023, 10:06 PM

*nudge*

MatthewVernon added a comment.Jan 20 2023, 2:23 PM

Still working on this, I'm afraid - rclone deployed OK, but then we hit T327253 and T327269 ; fixing these issues and then resolving T299125 and T279637 is one of my KRs for this quarter.

Andrew added a comment.Mar 7 2023, 1:42 PM

Hello! According to https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2022_Purge, the 'swift' project is now abandoned. Komla and I have sent many emails to the project admins without response.

And yet, this ticket implies that those VMs are still in use. Can you clarify?

Andrew edited subscribers, added: komla; removed: faidon.Mar 7 2023, 1:43 PM
MatthewVernon added a comment.Mar 7 2023, 2:10 PM

Sorry, have updated that page now.

Andrew added a comment.Mar 8 2023, 3:47 PM
In T306098#8672661, @MatthewVernon wrote:

Sorry, have updated that page now.

thanks!

Andrew added a comment.Apr 3 2023, 7:41 PM

Hello! I may be misreading but it looks like this task (and the associated task for bare metal hosts) has been stalled since January. Is there a plan to resume work?

MatthewVernon added a comment.Apr 4 2023, 7:34 AM

I have been working on T327253 around everything else.

Andrew added a comment.May 24 2023, 1:26 PM

Hello again! I'm just checking in that someone is still tasked with resolving this after our recent team shuffles.

MatthewVernon added a comment.May 24 2023, 1:48 PM

Yes, and it's still me.

MatthewVernon closed subtask T279637: Upgrade Swift ms cluster to Bullseye and revisit mkfs.xfs options as Resolved.May 31 2023, 10:58 AM
MatthewVernon added a project: SRE-swift-storage.May 31 2023, 1:12 PM
MatthewVernon added a subscriber: faidon.
MatthewVernon closed this task as Resolved.May 31 2023, 3:44 PM
MatthewVernon claimed this task.

I've now completed the bullseye upgrade for the remaining stretch nodes, and have decommissioned the virtual machines accordingly. Thanks for your patience!

• taavi awarded a token.May 31 2023, 4:02 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits