Page MenuHomePhabricator

[EPIC] Upgrade MediaWiki-Vagrant to Debian Buster
Open, MediumPublic

Description

The Vagrant base image is currently Stretch (Debian 9), which is EOL in 2020-07-06 (although the Debian LTS project supports it until 2022 summer). Wikimedia production and CI (which Vagrant is meant to be compatible with) are moving to Buster (Debian 10; T247045: Migrate all of production metal and VMs to Buster or later, T252432: Drop MediaWiki testing in stretch and instead test only in buster), MediaWiki-Vagrant should too.

See also:


It's preferable to just recreate your Vagrant boxes from scratch (cattle not pets) but if you really need to upgrade an existing Stretch-based box, here are the steps:

  • dump DB with mysqldump --add-drop-database --databases `mysql -rNe 'show databases' | ack -v '^(mysql|information_schema|performance_schema)$'` > /vagrant/dump.sql
  • update Vagrant repo
  • run vagrant destroy -f; vagrant up
  • restore dump with mysql < /vagrant/dump.sql
  • clear caches, e-g- by running vagrant reload
  • regenerate all data wasn't stored in the DB:
    • if the cirrussearch role is enabled, run mwscript /vagrant/mediawiki/extensions/CirrusSearch/maintenance/ForceSearchIndex.php

Related Objects

Event Timeline

I led the Precise -> Trusty -> Jessie -> Stretch migrations for MediaWiki-Vagrant. I would be happy to see someone else take that lead role in a Stretch -> Buster migration. See T181353: [EPIC] Migrate MediaWiki-Vagrant base image to Debian Stretch for the last tracking task of this type.

bd808 renamed this task from Upgrade MediaWiki-Vagrant to Debian Buster to [EPIC] Upgrade MediaWiki-Vagrant to Debian Buster.Jul 14 2020, 11:43 PM
bd808 triaged this task as Medium priority.
bd808 added a project: Epic.
bd808 updated the task description. (Show Details)

I'm interested in theory, would have to dig myself out from below a mountain of other tasks first, though.

I'm also interested in helping out with this. Over in fundraising-tech land, we have a few mw-vagrant users and with our production stack recently moving to buster I'm keen to see what it would take to move the base vagrant to buster.

T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) seems to be at the stage of upgrading prod MW servers to Buster, so in theory this task is ready to move forward.

$ git push --set-upstream origin buster-migration
Total 0 (delta 0), reused 0 (delta 0)
remote: Processing changes: refs: 1, done
To ssh://gerrit/mediawiki/vagrant
 * [new branch]        buster-migration -> buster-migration
Branch 'buster-migration' set up to track remote branch 'buster-migration' from 'origin'.

Change 655298 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[mediawiki/vagrant@buster-migration] buster: Update uwsgi-plugin-rack-ruby version

https://gerrit.wikimedia.org/r/655298

Change 655298 merged by jenkins-bot:
[mediawiki/vagrant@buster-migration] buster: Update uwsgi-plugin-rack-ruby version

https://gerrit.wikimedia.org/r/655298

Should the migration to PHP 7.4 be part of buster-migration or part of stretch? With the changes to MediaWiki, master now requires PHP 7.4 (T261872). Buster provides 7.3 and would be outdated already.

Should the migration to PHP 7.4 be part of buster-migration or part of stretch? With the changes to MediaWiki, master now requires PHP 7.4 (T261872). Buster provides 7.3 and would be outdated already.

The reason this is important is because there are no stretch packages for PHP 7.4. This means that at this point, stretch is broken with MediaWiki master (REL1_ branches already were broken) and buster is too until it is upgraded to PHP 7.4. Changing stretch to use a custom repository (say Ondrej's) is a wasted effort when everyone should move to a buster-based box soon. (and bullseye later...)

I can't find PHP 7.4 in stretch-backports and Ondrej's PPA does not provide for stretch anymore, so that leaves us with manually compiling the packages. For an outdated distro, that's not worth the effort.

Change 837165 had a related patch set uploaded (by Mainframe98; author: Mainframe98):

[mediawiki/vagrant@buster-migration] Merge remote-tracking branch 'origin/master' into buster-migration

https://gerrit.wikimedia.org/r/837165

Change 837166 had a related patch set uploaded (by Mainframe98; author: Mainframe98):

[mediawiki/vagrant@master] Switch to PHP 7.4

https://gerrit.wikimedia.org/r/837166

Change 837166 abandoned by Mainframe98:

[mediawiki/vagrant@master] Switch to PHP 7.4

Reason:

This patch is targeting the wrong branch and needs to be redone after buster-migration has been rebased on top of master

https://gerrit.wikimedia.org/r/837166

Change 837165 abandoned by Mainframe98:

[mediawiki/vagrant@buster-migration] Merge remote-tracking branch 'origin/master' into buster-migration

Reason:

Rebasing buster-migration on master and then merging the branch locally is less messy

https://gerrit.wikimedia.org/r/837165

Change 842963 had a related patch set uploaded (by Mainframe98; author: Bryan Davis):

[mediawiki/vagrant@master] buster: Update uwsgi-plugin-rack-ruby version

https://gerrit.wikimedia.org/r/842963

Change 842964 had a related patch set uploaded (by Mainframe98; author: Mainframe98):

[mediawiki/vagrant@master] Switch to PHP 7.4

https://gerrit.wikimedia.org/r/842964

Just noting that T319432: Migrate WMF production from PHP 7.4 to PHP 8.1 is in progress, not sure if that affects how you go about updating Vagrant for PHP 7.4, but wanted to mention it in case it has some bearing in the approach you take.

Change 842963 merged by jenkins-bot:

[mediawiki/vagrant@master] buster: Update uwsgi-plugin-rack-ruby version

https://gerrit.wikimedia.org/r/842963

Ideally we'd match the production setup (which also means doing T319167: [EPIC] Upgrade to Debian Bullseye, probably). I'd expect the production migration to take a while, though.

Change 842964 merged by jenkins-bot:

[mediawiki/vagrant@master] Switch to PHP 7.4

https://gerrit.wikimedia.org/r/842964

Change 850614 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/vagrant@master] Fix wikidiff2 package name

https://gerrit.wikimedia.org/r/850614

Change 850614 merged by jenkins-bot:

[mediawiki/vagrant@master] Fix wikidiff2 package name

https://gerrit.wikimedia.org/r/850614

Change 850639 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/vagrant@master] Add --allow-releaseinfo-change to apt update

https://gerrit.wikimedia.org/r/850639

Change 850639 merged by jenkins-bot:

[mediawiki/vagrant@master] Add --allow-releaseinfo-change to apt update

https://gerrit.wikimedia.org/r/850639

Change 851177 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/vagrant@master] Update various PHP extensions to 7.4

https://gerrit.wikimedia.org/r/851177

Change 851177 merged by jenkins-bot:

[mediawiki/vagrant@master] Update various PHP extensions to 7.4

https://gerrit.wikimedia.org/r/851177

Steps to upgrade an existing box:

  • dump DB with mysqldump --add-drop-database --databases `mysql -rNe 'show databases' | ack -v '^(mysql|information_schema|performance_schema)$'` > /vagrant/dump.sql
  • update Vagrant repo
  • run vagrant destroy -f; vagrant up
  • restore dump with mysql < /vagrant/dump.sql
  • clear caches, e-g- by running vagrant reload to
  • regenerate all data wasn't stored in the DB:
    • if the cirrussearch role is enabled, run mwscript /vagrant/mediawiki/extensions/CirrusSearch/maintenance/ForceSearchIndex.php

Change 930609 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/vagrant@master] Buster migration: Link to upgrading help page

https://gerrit.wikimedia.org/r/930609

List of errors from a recent upgrade:

  1. ATP cache write error:
==> default: W: chown to _apt:root of directory /vagrant/cache/apt/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
==> default: W: chown to root:root of file /vagrant/cache/apt/partial/logstash_1%3a7.1.1-1_all.deb failed - 201::URIDone (1: Operation not permitted)

(T183150: /vagrant/cache/apt permissions warnings with Stretch sandboxed apt settings - long-standing issue, I don't think it causes any problems, just distracting)

  1. libruby2.5 downgrade
==> default: The following packages will be DOWNGRADED:
==> default:   libruby2.5
==> default: 1 upgraded, 4 newly installed, 1 downgraded, 0 to remove and 102 not upgraded.
==> default: E: Packages were downgraded and -y was used without --allow-downgrades.
==> default: Error: /Stage[main]/Packages::Ruby_dev/Package[ruby-dev]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install ruby-dev' returned 100: Reading package lists...
  1. Logstash OOM
==> default: Setting up logstash (1:7.1.1-1) ...
==> default: Using provided startup.options file: /etc/logstash/startup.options
==> default: OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000d4cc0000, 724828160, 0) failed; error='Cannot allocate memory' (errno=12)
==> default: /usr/share/logstash/bin/system-install: line 88: #: command not found
==> default: Unable to install system startup script for Logstash.
==> default: chmod: cannot access '/etc/default/logstash': No such file or directory
==> default: dpkg: error processing package logstash (--configure):
==> default:  installed logstash package post-installation script subprocess returned error exit status 1

This then somehow breaks all other apt package installs to, they all have the same

==> default: Using provided startup.options file: /etc/logstash/startup.options
==> default: OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000d4cc0000, 724828160, 0) failed; error='Cannot allocate memory' (errno=12)

lines + dpkg error.

Change 930609 merged by jenkins-bot:

[mediawiki/vagrant@master] Buster migration: Link to upgrading help page

https://gerrit.wikimedia.org/r/930609