We'll get this for "free" when we complete T335766: Migrate Quibble images from buster to bullseye and then T362705: Migrate Quibble images from bullseye to bookworm – buster comes with 10.3, bullseye comes with 10.5, and it's not until bookworm that we get to or over 10.6 (specifically, to 10.11). Meh.

In all those years, I don't think I have cared much about MariaDB in the CI image beside:

• setting sql_mode = 'TRADITIONAL' T119371
• Using binary character set and collation to match production T193222

If production went to 10.6 and Debian does not provide it, I imagine SRE went to build 1.6 version for Bullseye or Bookworm. If our CI images are based on the same Debian system, we can install that MariaDB instead of the stock Debian version, else I guess we can ignore?

In production we run 10.6. We have packages for Bullseye, but we encourage to go to Bookworm directly if possible.

Change #1133241 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package

https://gerrit.wikimedia.org/r/1133241

Change #1133241 merged by jenkins-bot:

[integration/config@master] Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package

https://gerrit.wikimedia.org/r/1133241

Mentioned in SAL (#wikimedia-releng) [2025-04-01T20:34:17Z] <James_F> Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package, for T366646

Change #1133903 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Upgrade quibble bullseye jobs to MariaDB 10.6

https://gerrit.wikimedia.org/r/1133903

Change #1133912 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Docker: [quibble-bullseye] Revert MardiaDB to 10.5, Wikimedia package lacks mysql_install_db

https://gerrit.wikimedia.org/r/1133912

Change #1133903 abandoned by Jforrester:

[integration/config@master] jjb: Upgrade quibble bullseye jobs to MariaDB 10.6

Reason:

Sadly this won't work.

https://gerrit.wikimedia.org/r/1133903

Change #1133912 merged by jenkins-bot:

[integration/config@master] Docker: [quibble-bullseye] Revert MardiaDB to 10.5

https://gerrit.wikimedia.org/r/1133912

Mentioned in SAL (#wikimedia-releng) [2025-04-03T13:33:15Z] <James_F> Docker: [quibble-bullseye] Revert MardiaDB to 10.5, for (against) T366646

In https://gerrit.wikimedia.org/r/c/integration/config/+/1133912 Revert MardiaDB to 10.5, Wikimedia package lacks mysql_install_db you state in the commit message:

The Wikimedia package lacks mysql_install_db, so sadly we're going to have to wait for the bookworm switch, or get Wikimedia SRE to package more of MariaDB than they use. Either way, let's revert now so we can do more things with Quibble.

I do not know why DBA came with a custom package name wmf-mariadb106 since usually the way to go is to use an apt component (similar to our custom php 7.4 package coming from bullseye-wikimedia/component/php74) which should probably be reported to them / filed as a task. But I imagine they have their reason.

There is no obvious reason for mysql_install_db to be missing, short of it being renamed upstream (could it have become mariadb_install_db? Has it been superseded by something else?)

I went to explore the package:

$ docker run --rm -it --entrypoint=bash --user=root docker-registry.wikimedia.org/releng/quibble-bullseye-php74:1.13.0-s1
# dpkg -L wmf-mariadb106|grep install.db
/opt/wmf-mariadb106/man/man1/mariadb-install-db.1
/opt/wmf-mariadb106/man/man1/mysql_install_db.1
/opt/wmf-mariadb106/scripts/mariadb-install-db
/opt/wmf-mariadb106/scripts/mysql_install_db

The mysql_install_db script is present, a renamed copy exists mariadb-install-db but our Debian package does not install it under /usr/bin. I guess it is a rather simple fix in their packaging and that is worse filing a task for them. Meanwhile you can:

COPY /opt/wmf-mariadb106/scripts/mysql_install_db /usr/local/bin/mysql_install_db

Problem solved.

For the MariaDB upgrade, beware of the mariadb config in quibble-bullseye/mariadb.cnf but hopefully it is stable by now (T193222 was a change to use utf8mb4 when MediaWiki prod prefers binary for historical reasons/lot of languages).

And MediaWiki DevelopmentSetting.php has:

// Enable MariaDB/MySQL strict mode (T108255)
$wgSQLMode = 'STRICT_ALL_TABLES,ONLY_FULL_GROUP_BY';

Change #1134167 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: apply rebuild of revert to Mariadb 10.5

https://gerrit.wikimedia.org/r/1134167

Change #1134167 merged by jenkins-bot:

[integration/config@master] jjb: apply rebuild of revert to Mariadb 10.5

https://gerrit.wikimedia.org/r/1134167

I think this can be revisited again since the subticket is now resolved?

Change #1159443 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package, again

https://gerrit.wikimedia.org/r/1159443

Change #1159443 merged by jenkins-bot:

[integration/config@master] Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package, again

https://gerrit.wikimedia.org/r/1159443

Mentioned in SAL (#wikimedia-releng) [2025-06-16T14:31:59Z] <James_F> Docker: [quibble-bullseye] Switch MariaDB to 10.6 Wikimedia package, again, for T366646

Change #1159480 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Switch to Quibble images with MariaDB 10.6

https://gerrit.wikimedia.org/r/1159480

Change #1159501 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Docker: [quibble-bullseye] Add the MariaDB binaries to our path

https://gerrit.wikimedia.org/r/1159501

Change #1159501 merged by jenkins-bot:

[integration/config@master] Docker: [quibble-bullseye] Add the MariaDB binaries to our path

https://gerrit.wikimedia.org/r/1159501

Mentioned in SAL (#wikimedia-releng) [2025-06-16T15:15:26Z] <James_F> Docker: [quibble-bullseye] Add the MariaDB binaries to our path T366646

Change #1159509 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Docker: [quibble-bullseye] Copy the my_print_defaults script to main bin too

https://gerrit.wikimedia.org/r/1159509

Change #1159480 abandoned by Jforrester:

[integration/config@master] jjb: Switch to Quibble images with MariaDB 10.6

https://gerrit.wikimedia.org/r/1159480

Change #1159509 merged by jenkins-bot:

[integration/config@master] Docker: [quibble-bullseye] Revert MardiaDB to 10.5, Wikimedia package lacks

https://gerrit.wikimedia.org/r/1159509

Thank you!

In T366646#10922328, @Ladsgroup wrote:

Thank you!

Sadly not done. :-( Effectively, unless SRE re-packages the Wikimedia version of MariaDB on bullseye, we're going to have to wait for:

• the 8.1 -> 8.3 migration in Wikimedia prod (T360995),
• the dropping of CI testing (in practice, MW support) for PHP 8.1 (T358666), and then
• switching CI base images from bullseye to bookworm (T383337)

This is because:

• Bullseye ships with MariaDB 10.5.
• Bookworm is the first OS that ships with MariaDB 10.6.
• The Wikimedia MariaDB package is missing most of the stuff we use in CI; after four attempts to hack things in, we've given up.
• The Wikimedia PHP 8.1 package isn't available on bookworm (T397075), and apparently won't be due to ICU issues, so we have to wait.

Alternative options, all of them quite bad:

1. Someone puts in the (? large) effort to package WMF PHP 8.1 for bookworm.
2. Someone puts in the (? large) effort to package WMF MariaDB for bullseye (and others for the future!).
3. We stop testing with MariaDB in quibble and only use SQLite (very far from prod) for PHP 8.1 on bullseye, and test other PHP versions with MariaDB on bookworm (!!!)

Another alternative is to have multiple containers:

• Quibble in a container with Bullseye + php7.4
• MariaDB with Bookworm + MariaDB 10.6

That will be possible with the new Zuul version but Quibble lacks the instrumentation to specify a different DB though.

Someone puts in the (? large) effort to package WMF PHP 8.1 for bookworm.

Packaging PHP itself is not that complicated in my experience. The devil is:

• ensure the set of extensions works and are at the expected versions
• we need specific versions of libicu and libpcre

And of course there might be other libs that we end up exploding on.

I don't think that is automated.

The Wikimedia MariaDB package is missing most of the stuff we use in CI; after four attempts to hack things in, we've given up.

I wonder what is missing. For reference Quibble relies on the following:

From mariadb-client:

From mariadb-server-core:

We can make adjustment in Quibble if scripts got renamed between 10.5 and 10.6, then locally I have mariadb-server-core 10.11.11 from Debian Bookworm and Quibble works fine. Maybe our Debian package does not place the files at the proper place? Do you have an example, some traces, a task?

FWIW, we are already switching most of our of production databases to 10.11