Page MenuHomePhabricator
Create Task
Maniphest T265643

Upgrade MediaWiki's Redis cluster to Debian Buster
Closed, ResolvedPublic
Actions

Description

Mediawiki's Redis cluster is co-hosted with our memcached cluster (mc* hosts). It is important we migrate this cluster to Debian buster ideally by the end of Q2. It has been an issue for quite some time, how we will upgrade this Redis cluster.

Our options are:

  • Use Redis 5.0, the one that ships with debian buster
  • Use Redis 2.8, the version we are currently running, but we need to port it to debian buster (and maintain) ourselves.

Something to additionally consider is, in case of an upgrade, how this will affect the Redis Lock Manager.

Details

SubjectRepoBranchLines +/-
Swap Redis Lock hosts with upgraded onesoperations/mediawiki-configmaster+4 -4
Swap mc1019 with mc1031 for Redis lock manageroperations/mediawiki-configmaster+6 -6
Swap mc1033 with mc1034 for Redis lock manageroperations/mediawiki-configmaster+1 -1
hiera: install redis on shard16operations/puppetproduction+8 -0
redis: define redis version on buster for multidcoperations/puppetproduction+64 -52
redis: define redis version on busteroperations/puppetproduction+30 -11
Customize query in gerrit

Related Objects
Search...

Event Timeline

jijiki triaged this task as Medium priority.Oct 15 2020, 5:57 PM
jijiki created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 15 2020, 5:57 PM
Krinkle updated the task description. (Show Details)Oct 15 2020, 7:22 PM
Krinkle added a subtask: T252391: Reimage one memcached shard per DC to Buster.
Krinkle added a parent task: T247045: Migrate all of production metal and VMs to Buster or later.
jijiki added a parent task: T213089: Upgrade memcached cluster to Debian Buster.Oct 20 2020, 4:00 PM
jijiki added a project: User-jijiki.
jijiki moved this task from Incoming🐅 to In Progress 🏋️‍♀️ on the User-jijiki board.Oct 20 2020, 4:44 PM
• Clarakosi moved this task from Inbox to Tracking/Watching on the Platform Engineering board.Oct 20 2020, 8:40 PM
jijiki mentioned this in T213089: Upgrade memcached cluster to Debian Buster.Oct 20 2020, 8:44 PM
jijiki updated the task description. (Show Details)Oct 28 2020, 5:28 PM
jijiki added a subscriber: Ladsgroup.
jijiki moved this task from In Progress 🏋️‍♀️ to Next up 🥌 on the User-jijiki board.Nov 2 2020, 11:20 AM
jijiki mentioned this in T267581: Phase out "redis_sessions" cluster and away from memcached cluster.Nov 9 2020, 5:16 PM
jijiki added a parent task: T267581: Phase out "redis_sessions" cluster and away from memcached cluster.
gerritbot added a comment.Dec 7 2020, 12:10 PM

Change 646638 had a related patch set uploaded (by Effie Mouzeli; owner: Effie Mouzeli):
[operations/puppet@production] redis: define redis version on buster

https://gerrit.wikimedia.org/r/646638

gerritbot added a project: Patch-For-Review.Dec 7 2020, 12:10 PM
gerritbot added a comment.Dec 9 2020, 9:19 AM

Change 647197 had a related patch set uploaded (by Effie Mouzeli; owner: Effie Mouzeli):
[operations/puppet@production] redis: define redis version on buster for multidc

https://gerrit.wikimedia.org/r/647197

gerritbot added a comment.Dec 9 2020, 9:19 AM

Change 646638 abandoned by Effie Mouzeli:
[operations/puppet@production] redis: define redis version on buster

Reason:
abandoned for 647197

https://gerrit.wikimedia.org/r/646638

gerritbot added a comment.Dec 9 2020, 9:47 AM

Change 647204 had a related patch set uploaded (by Effie Mouzeli; owner: Effie Mouzeli):
[operations/puppet@production] hiera: install redis on mc1036

https://gerrit.wikimedia.org/r/647204

Stashbot added a comment.Dec 10 2020, 9:11 AM

Mentioned in SAL (#wikimedia-operations) [2020-12-10T09:11:12Z] <effie> disable puppet on all hosts running redis - T265643

gerritbot added a comment.Dec 10 2020, 9:12 AM

Change 647197 merged by Effie Mouzeli:
[operations/puppet@production] redis: define redis version on buster for multidc

https://gerrit.wikimedia.org/r/647197

Stashbot added a comment.Dec 10 2020, 9:26 AM

Mentioned in SAL (#wikimedia-operations) [2020-12-10T09:26:31Z] <effie> disable puppet on all mw* hosts for 647204 - T265643

Stashbot added a comment.Dec 10 2020, 9:28 AM

Mentioned in SAL (#wikimedia-operations) [2020-12-10T09:28:19Z] <effie> disable puppet on all hosts running nutcracker for 647204 - T265643

gerritbot added a comment.Dec 10 2020, 10:03 AM

Change 647204 merged by Effie Mouzeli:
[operations/puppet@production] hiera: install redis on shard16

https://gerrit.wikimedia.org/r/647204

jijiki closed subtask T252391: Reimage one memcached shard per DC to Buster as Resolved.Dec 10 2020, 11:39 AM
gerritbot added a comment.Dec 15 2020, 10:17 AM

Change 649602 had a related patch set uploaded (by Effie Mouzeli; owner: Effie Mouzeli):
[operations/mediawiki-config@master] Swap mc1033 with mc1034 for Redis lock manager

https://gerrit.wikimedia.org/r/649602

gerritbot added a comment.Dec 15 2020, 11:01 AM

Change 649602 merged by jenkins-bot:
[operations/mediawiki-config@master] Swap mc1033 with mc1034 for Redis lock manager

https://gerrit.wikimedia.org/r/649602

Stashbot added a comment.Dec 15 2020, 11:06 AM

Mentioned in SAL (#wikimedia-operations) [2020-12-15T11:06:15Z] <jiji@deploy1001> Synchronized wmf-config/ProductionServices.php: Swap mc1033 with mc1034 for Redis lock manager - T265643 (duration: 00m 59s)

jijiki moved this task from Next up 🥌 to In Progress 🏋️‍♀️ on the User-jijiki board.Dec 15 2020, 11:25 AM
gerritbot added a comment.Dec 16 2020, 10:19 AM

Change 649836 had a related patch set uploaded (by Effie Mouzeli; owner: Effie Mouzeli):
[operations/mediawiki-config@master] Swap mc1019 with mc1031 for Redis lock manager

https://gerrit.wikimedia.org/r/649836

gerritbot added a comment.Dec 16 2020, 11:15 AM

Change 649836 merged by jenkins-bot:
[operations/mediawiki-config@master] Swap mc1019 with mc1031 for Redis lock manager

https://gerrit.wikimedia.org/r/649836

Stashbot added a comment.Dec 16 2020, 11:19 AM

Mentioned in SAL (#wikimedia-operations) [2020-12-16T11:19:16Z] <jiji@deploy1001> Synchronized wmf-config/ProductionServices.php: Swap mc1019 with mc1031 for Redis lock manager - T265643 (duration: 01m 17s)

gerritbot added a comment.Dec 16 2020, 3:47 PM

Change 649901 had a related patch set uploaded (by Effie Mouzeli; owner: Effie Mouzeli):
[operations/mediawiki-config@master] Swap Redis Lock hosts with upgraded ones

https://gerrit.wikimedia.org/r/649901

gerritbot added a comment.Dec 16 2020, 4:00 PM

Change 649901 merged by jenkins-bot:
[operations/mediawiki-config@master] Swap Redis Lock hosts with upgraded ones

https://gerrit.wikimedia.org/r/649901

Stashbot added a comment.Dec 16 2020, 4:03 PM

Mentioned in SAL (#wikimedia-operations) [2020-12-16T16:03:29Z] <jiji@deploy1001> Synchronized wmf-config/ProductionServices.php: Swap Redis lock managers with upgraded ones - T265643 (duration: 01m 03s)

jijiki closed this task as Resolved.Jan 13 2021, 3:37 PM
jijiki claimed this task.

We ported version 2.8 to Buster, and all servers were upgraded as part of T213089.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits