Page MenuHomePhabricator

Evaluate and decide the future of relational datastore at WMF after the upgrade of MariaDB 10.1 is finished
Closed, ResolvedPublic

Description

  • This will likely be paired with the release of Debian Buster. What available relational databases will be included on the distribution?
  • Which are well-supported, stable databases to run Wikipedia and its sister projects?
  • What is the commitment to free software and other shared values with the foundation?
  • Among all possibilities, which have the best set of features for WMF-specific needs
  • What is the level of past commitment/support to reported bugs by WMF users

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 27 2018, 9:30 AM
jcrespo triaged this task as Low priority.Apr 27 2018, 9:30 AM
jcrespo moved this task from Triage to Meta/Epic on the DBA board.

(Low for now, likely to get a boost for the 15-year plan)

Vvjjkkii renamed this task from Evaluate and decide the future of relational datastore at WMF after the upgrade of MariaDB 10.1 is finished to z3daaaaaaa.Jul 1 2018, 1:13 AM
Vvjjkkii raised the priority of this task from Low to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
Marostegui renamed this task from z3daaaaaaa to Evaluate and decide the future of relational datastore at WMF after the upgrade of MariaDB 10.1 is finished.Jul 1 2018, 7:34 AM
Marostegui lowered the priority of this task from High to Low.
Marostegui updated the task description. (Show Details)
Marostegui added a subscriber: Aklapper.

Change 449120 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] Test MariaDB 10.3 on core test hosts

https://gerrit.wikimedia.org/r/449120

Change 449120 merged by Jcrespo:
[operations/puppet@production] Test MariaDB 10.3 on core test hosts

https://gerrit.wikimedia.org/r/449120

Change 449196 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Test MariaDB 10.3 on db1118 (core test host)

https://gerrit.wikimedia.org/r/449196

Change 449196 merged by Jcrespo:
[operations/puppet@production] mariadb: Test MariaDB 10.3 on db1118 (core test host)

https://gerrit.wikimedia.org/r/449196

Change 449234 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Better support MariaDB 10.2 and 10.3 config on production

https://gerrit.wikimedia.org/r/449234

Change 449413 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] Test MySQL 8.0.12 on test-s1 host db1118

https://gerrit.wikimedia.org/r/449413

Change 449413 merged by Jcrespo:
[operations/puppet@production] Test MySQL 8.0.12 on test-s1 host db1118

https://gerrit.wikimedia.org/r/449413

Change 449234 merged by Jcrespo:
[operations/puppet@production] mariadb: Better support MariaDB 10.2 and 10.3 config on production

https://gerrit.wikimedia.org/r/449234

CDanis added a subscriber: CDanis.Nov 21 2018, 6:09 PM

Change 494236 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Change the default arguments for buster

https://gerrit.wikimedia.org/r/494236

Change 494469 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] install_server: Reimage db1114 to buster

https://gerrit.wikimedia.org/r/494469

Change 494469 merged by Jcrespo:
[operations/puppet@production] install_server: Reimage db1114 to buster

https://gerrit.wikimedia.org/r/494469

Change 494236 merged by Jcrespo:
[operations/puppet@production] mysqld-prometheus-exporter: Change the default arguments for buster

https://gerrit.wikimedia.org/r/494236

Change 494759 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mysqld-prometheus-exporter: Fix typo on configuration

https://gerrit.wikimedia.org/r/494759

Change 494759 merged by Jcrespo:
[operations/puppet@production] mysqld-prometheus-exporter: Fix typo on configuration

https://gerrit.wikimedia.org/r/494759

Change 519066 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Upgrade db1114 (testing buster host) to MariaDB 10.3

https://gerrit.wikimedia.org/r/519066

Change 519066 merged by Jcrespo:
[operations/puppet@production] mariadb: Upgrade db1114 (testing buster host) to MariaDB 10.3

https://gerrit.wikimedia.org/r/519066

db1114: Version 10.3.16-MariaDB-log, Uptime 937s, read_only: True, 122.17 QPS, connection latency: 0.003587s, query latency: 0.000728s

Change 519073 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Prepare core for buster

https://gerrit.wikimedia.org/r/519073

db1114: Version 10.3.16-MariaDB-log, Uptime 937s, read_only: True, 122.17 QPS, connection latency: 0.003587s, query latency: 0.000728s

Nice! Is it replicating from the master? It would be nice to let it replicate for a few weeks and see if we get some unexpected issues

Nice! Is it replicating from the master?

It is, although tendril doesn't seem to like it.

@jcrespo you ok if I copy dewiki.logging into db1114? I would like to see the behaviour of 10.3 optimizer in regards to the query planner bug observed at T71222: list=logevents slow for users with last log action long time ago

@jcrespo you ok if I copy dewiki.logging into db1114

Sure, if you do it in its own separate schema.

@jcrespo you ok if I copy dewiki.logging into db1114

Sure, if you do it in its own separate schema.

Of course :)

jcrespo added a subscriber: Anomie.Jun 26 2019, 7:07 AM

Ping @Anomie We have temporarily setup db1114 with MariaDB 10.3 and load it with an up-to-date replica of enwiki. While we will do proper regression testing if we decide to go for 10.3, we are notifying you in case that may help testing some of the other specific problematic queries to see if they do better in this version and/or give us any feedback. Note this testing is temporary, as db1114's real purpose is to test backups, not Mediawiki.

Other interesting features people may want to play/experiment with:

Change 519073 merged by Jcrespo:
[operations/puppet@production] mariadb: Prepare core for buster

https://gerrit.wikimedia.org/r/519073

daniel moved this task from Inbox to Watching on the TechCom board.Jul 31 2019, 5:12 AM

db1114 is now running percona-server 8.0, if anyone wants to test it.

Change 550647 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb-client: Install 10.4 on buster, unblock os upgrade

https://gerrit.wikimedia.org/r/550647

Change 550647 merged by Jcrespo:
[operations/puppet@production] mariadb-client: Install 10.4 on buster, unblock os upgrade

https://gerrit.wikimedia.org/r/550647

db1107 is now running the latest 10.3.20 from MariaDB replicating from s1 master and db1114 (which runs percona-server 8.0) replicates from it.

Change 554240 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Add wmf-mariadb104 as a possibility

https://gerrit.wikimedia.org/r/554240

Change 554240 merged by Marostegui:
[operations/puppet@production] mariadb: Add wmf-mariadb104,percona 8.0 as a possibility

https://gerrit.wikimedia.org/r/554240

Rxy added a subscriber: Rxy.Dec 3 2019, 12:34 PM

Change 561264 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] control-mariadb*: Upgrade version

https://gerrit.wikimedia.org/r/561264

I have upgraded db1107 from 10.4.10 to the latest version: 10.4.11

Change 561264 merged by jenkins-bot:
[operations/software@master] control-mariadb*: Upgrade version

https://gerrit.wikimedia.org/r/561264

Change 575483 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Add example percona configuration for a core host

https://gerrit.wikimedia.org/r/575483

Change 575483 merged by Jcrespo:
[operations/puppet@production] mariadb: Add example percona configuration for a core host

https://gerrit.wikimedia.org/r/575483

Change 575496 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mysql: Fix mysql server configuration for the percona flavour

https://gerrit.wikimedia.org/r/575496

Change 575496 merged by Jcrespo:
[operations/puppet@production] mysql: Fix mysql server configuration for the percona flavour

https://gerrit.wikimedia.org/r/575496

jcrespo closed this task as Resolved.Apr 20 2020, 9:23 AM
jcrespo assigned this task to Marostegui.

As per meetings, the decision was: no decision. Closing this in favour of the epic T250666.