Page MenuHomePhabricator
Create Task
Maniphest T322231

Create new deployment-ms-be instances running Debian Bullseye
Closed, ResolvedPublic
Actions

Description

Current ms-be instances in the deployment-prep project have been shutdown cause those are running debian 9. The current production instances are running already in Bullseye so it should be possible to spawn new instances running Debian 11

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

Vgutierrez created this task.Nov 2 2022, 4:10 PM
taavi merged a task: T298253: Upgrade deployment-prep Swift cluster to Debian Buster or newer.Nov 2 2022, 4:18 PM
taavi added subscribers: taavi, LSobanski, MatthewVernon.
Vgutierrez triaged this task as Medium priority.Nov 2 2022, 4:26 PM
Vgutierrez mentioned this in T306068: Cloud VPS "deployment-prep" project Stretch deprecation.
gerritbot added a comment.Nov 2 2022, 5:10 PM

Change 852245 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] deployment-prep: Add ms-be0[78] as storagehosts

https://gerrit.wikimedia.org/r/852245

gerritbot added a project: Patch-For-Review.Nov 2 2022, 5:10 PM
Vgutierrez changed the task status from Open to In Progress.Nov 2 2022, 5:11 PM
gerritbot added a comment.Nov 2 2022, 7:47 PM

Change 852245 merged by Vgutierrez:

[operations/puppet@production] deployment-prep: Add ms-be0[78] as storagehosts

https://gerrit.wikimedia.org/r/852245

gerritbot added a comment.Nov 2 2022, 8:04 PM

Change 852281 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] swift: Add ms-be0[78] to deployment-prep cluster

https://gerrit.wikimedia.org/r/852281

gerritbot added a comment.Nov 2 2022, 8:12 PM

Change 852281 merged by Vgutierrez:

[operations/puppet@production] swift: Add ms-be0[78] to deployment-prep cluster

https://gerrit.wikimedia.org/r/852281

gerritbot added a comment.Nov 2 2022, 8:28 PM

Change 852290 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] swift: Drain ms-be0[56]@deployment-prep cluster

https://gerrit.wikimedia.org/r/852290

gerritbot added a comment.Nov 2 2022, 8:30 PM

Change 852290 merged by Vgutierrez:

[operations/puppet@production] swift: Drain ms-be0[56]@deployment-prep cluster

https://gerrit.wikimedia.org/r/852290

Maintenance_bot removed a project: Patch-For-Review.Nov 2 2022, 8:30 PM
gerritbot added a comment.Nov 2 2022, 8:35 PM

Change 852293 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] swift: Fix syntax error on deployment-prep config file

https://gerrit.wikimedia.org/r/852293

gerritbot added a project: Patch-For-Review.Nov 2 2022, 8:35 PM

Change 852293 merged by Vgutierrez:

[operations/puppet@production] swift: Fix syntax error on deployment-prep config file

https://gerrit.wikimedia.org/r/852293

Maintenance_bot removed a project: Patch-For-Review.Nov 2 2022, 9:30 PM
gerritbot added a comment.Nov 3 2022, 9:46 AM

Change 852740 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] cluster: Add deployment-prep swift cluster definition

https://gerrit.wikimedia.org/r/852740

gerritbot added a project: Patch-For-Review.Nov 3 2022, 9:46 AM
gerritbot added a comment.Nov 3 2022, 9:49 AM

Change 852740 abandoned by Vgutierrez:

[operations/puppet@production] cluster: Add deployment-prep swift cluster definition

Reason:

https://gerrit.wikimedia.org/r/852740

gerritbot added a comment.Nov 3 2022, 10:08 AM

Change 852767 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] puppetmaster::standalone: Sync swift rings

https://gerrit.wikimedia.org/r/852767

gerritbot added a comment.Nov 3 2022, 11:41 AM

Change 852767 merged by Vgutierrez:

[operations/puppet@production] puppetmaster::standalone: Sync swift rings

https://gerrit.wikimedia.org/r/852767

Vgutierrez changed the task status from In Progress to Stalled.Nov 3 2022, 11:56 AM
vgutierrez@deployment-cumin:~$ sudo -i cumin 'project:deployment-prep name:ms-be' 'swift-ring-builder /etc/swift/object.builder'
4 hosts will be targeted:
deployment-ms-be[05-08].deployment-prep.eqiad1.wikimedia.cloud
Ok to proceed on 4 hosts? Enter the number of affected hosts to confirm or "q" to quit 4
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |   0% (0/4) [00:00<?, ?hosts/s]
===== NODE GROUP =====                                                                                                                                                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |   0% (0/4) [00:01<?, ?hosts/s]
(2) deployment-ms-be[07-08].deployment-prep.eqiad1.wikimedia.cloud                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                                                         
                                                                                                                                                                                                                                                                                                                                         |   0% (0/4) [00:01<?, ?hosts/s]
----- OUTPUT of 'swift-ring-build...t/object.builder' -----                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                         |   0% (0/4) [00:01<?, ?hosts/s]
/etc/swift/object.builder, build version 39, id (not assigned)                                                                                                                                                                                                                                                                                                           65536 partitions, 2.000000 replicas, 1 regions, 3 zones, 4 devices, 0.02 balance, 0.00 dispersion
The minimum number of hours before a partition can be reassigned is 3 (2:15:51 remaining)
The overload factor is 10.00% (0.100000)
Ring file /etc/swift/object.ring.gz is obsolete
Devices:   id region zone   ip address:port replication ip:port  name weight partitions balance flags meta
            0      1    1 172.16.5.167:6010   172.16.5.167:6010  sdb1  40.00       2622    0.02       
            1      1    1  172.16.6.12:6010    172.16.6.12:6010  sdb1  40.00       2621   -0.02       
            2      1    3 172.16.7.114:6000   172.16.7.114:6000 lv-a1 960.00      62915    0.00       
            3      1    4 172.16.7.115:6000   172.16.7.115:6000 lv-a1 960.00      62914   -0.00       
                                                                                                                                                                                                                                                                                                                                                                         
                                                                                                                                                                                                                                                                                                                                         |   0% (0/4) [00:01<?, ?hosts/s]
===== NODE GROUP =====                                                                                                                                                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                                                                                                                                         
                                                                                                                                                                                                                                                                                                                                         |   0% (0/4) [00:01<?, ?hosts/s]
(2) deployment-ms-be[05-06].deployment-prep.eqiad1.wikimedia.cloud                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                                                         
                                                                                                                                                                                                                                                                                                                                         |   0% (0/4) [00:01<?, ?hosts/s]
----- OUTPUT of 'swift-ring-build...t/object.builder' -----                                                                                                                                                                                                                                                                                                              
                                                                                                                                                                                                                                                                                                                                                                         
                                                                                                                                                                                                                                                                                                                                         |   0% (0/4) [00:01<?, ?hosts/s]
/etc/swift/object.builder, build version 39                                                                                                                                                                                                                                                                                                                              
65536 partitions, 2.000000 replicas, 1 regions, 3 zones, 4 devices, 0.02 balance, 0.00 dispersion
The minimum number of hours before a partition can be reassigned is 3 (2:15:51 remaining)
The overload factor is 10.00% (0.100000)
Ring file /etc/swift/object.ring.gz is up-to-date
Devices:   id region zone   ip address:port replication ip:port  name weight partitions balance flags meta
            0      1    1 172.16.5.167:6010   172.16.5.167:6010  sdb1  40.00       2622    0.02       
            1      1    1  172.16.6.12:6010    172.16.6.12:6010  sdb1  40.00       2621   -0.02       
            2      1    3 172.16.7.114:6000   172.16.7.114:6000 lv-a1 960.00      62915    0.00       
            3      1    4 172.16.7.115:6000   172.16.7.115:6000 lv-a1 960.00      62914   -0.00

waiting while the old nodes get drained

gerritbot added a comment.Nov 3 2022, 3:20 PM

Change 852912 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] swift: Ramp up ms-be07 balance

https://gerrit.wikimedia.org/r/852912

gerritbot added a comment.Nov 3 2022, 3:22 PM

Change 852912 merged by Vgutierrez:

[operations/puppet@production] swift: Ramp up ms-be07 balance

https://gerrit.wikimedia.org/r/852912

gerritbot added a comment.Nov 4 2022, 7:35 PM

Change 853401 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] swift: Ramp up ms-be08 rebalance

https://gerrit.wikimedia.org/r/853401

gerritbot added a comment.Nov 4 2022, 7:38 PM

Change 853401 merged by Vgutierrez:

[operations/puppet@production] swift: Ramp up ms-be08 rebalance

https://gerrit.wikimedia.org/r/853401

gerritbot added a comment.Nov 5 2022, 3:58 PM

Change 853475 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] swift: Ramp up ms-be05 draining

https://gerrit.wikimedia.org/r/853475

gerritbot added a comment.Nov 5 2022, 4:04 PM

Change 853475 merged by Vgutierrez:

[operations/puppet@production] swift: Ramp up ms-be05 draining

https://gerrit.wikimedia.org/r/853475

gerritbot added a comment.Nov 7 2022, 8:26 AM

Change 853917 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] swift: drain ms-be06 ASAP and remove ms-be05

https://gerrit.wikimedia.org/r/853917

gerritbot added a comment.Nov 7 2022, 8:29 AM

Change 853917 merged by Vgutierrez:

[operations/puppet@production] swift: drain ms-be06 ASAP and remove ms-be05

https://gerrit.wikimedia.org/r/853917

Stashbot added a comment.Nov 7 2022, 3:57 PM

Mentioned in SAL (#wikimedia-releng) [2022-11-07T15:57:06Z] <vgutierrez> shutting down deployment-ms-be05 - T322231

Vgutierrez added a parent task: T278641: Migrate deployment-prep away from Debian Stretch to Buster/Bullseye.Nov 7 2022, 4:01 PM
Vgutierrez mentioned this in T278641: Migrate deployment-prep away from Debian Stretch to Buster/Bullseye.
gerritbot added a comment.Nov 7 2022, 4:24 PM

Change 854041 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] deployment-prep: Remove ms-be05

https://gerrit.wikimedia.org/r/854041

gerritbot added a comment.Nov 7 2022, 4:35 PM

Change 854041 merged by Vgutierrez:

[operations/puppet@production] deployment-prep: Remove ms-be05

https://gerrit.wikimedia.org/r/854041

gerritbot added a comment.Nov 7 2022, 8:05 PM

Change 854094 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] swift: Remove ms-be06 from deployment-prep cluster

https://gerrit.wikimedia.org/r/854094

gerritbot added a comment.Nov 7 2022, 8:06 PM

Change 854094 merged by Vgutierrez:

[operations/puppet@production] swift: Remove ms-be06 from deployment-prep cluster

https://gerrit.wikimedia.org/r/854094

Stashbot added a comment.Nov 8 2022, 6:39 AM

Mentioned in SAL (#wikimedia-releng) [2022-11-08T06:39:41Z] <vgutierrez> delete deployment-ms-be05 - T322231

gerritbot added a comment.Nov 8 2022, 6:44 AM

Change 854416 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] deployment-prep: Move swift expirer duties to ms-be07

https://gerrit.wikimedia.org/r/854416

gerritbot added a comment.Nov 8 2022, 6:47 AM

Change 854416 merged by Vgutierrez:

[operations/puppet@production] deployment-prep: Move swift expirer duties to ms-be07

https://gerrit.wikimedia.org/r/854416

gerritbot added a comment.Nov 8 2022, 3:14 PM

Change 854533 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] deployment-prep: Remove ms-be06 from swift::storagehosts

https://gerrit.wikimedia.org/r/854533

gerritbot added a comment.Nov 8 2022, 3:16 PM

Change 854533 merged by Vgutierrez:

[operations/puppet@production] deployment-prep: Remove ms-be06 from swift::storagehosts

https://gerrit.wikimedia.org/r/854533

Stashbot added a comment.Nov 8 2022, 3:21 PM

Mentioned in SAL (#wikimedia-releng) [2022-11-08T15:21:37Z] <vgutierrez> shutdown deployment-ms-be06 - T322231

Stashbot added a comment.Nov 8 2022, 3:26 PM

Mentioned in SAL (#wikimedia-releng) [2022-11-08T15:26:15Z] <vgutierrez> delete deployment-ms-be06 - T322231

Vgutierrez closed this task as Resolved.Nov 8 2022, 3:26 PM
Vgutierrez claimed this task.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits