Ouch.

Wikimedia\Rdbms\DBQueryError from line 1699 of /usr/share/mediawiki/includes/libs/rdbms/database/Database.php: Error 1054: Unknown column 'ipb_reason_id' in 'where clause' (127.0.0.1)
Function: MigrateComments::migrate
Query: SELECT  ipb_id,ipb_reason  FROM `wikimini_stockwiki`.`ipblocks`    WHERE ipb_reason_id = 0 AND (1=1)  ORDER BY ipb_id LIMIT 100  

#0 /usr/share/mediawiki/includes/libs/rdbms/database/Database.php(1683): Wikimedia\Rdbms\Database->getQueryException('Unknown column ...', 1054, 'SELECT  ipb_id,...', 'MigrateComments...')
#1 /usr/share/mediawiki/includes/libs/rdbms/database/Database.php(1658): Wikimedia\Rdbms\Database->getQueryExceptionAndLog('Unknown column ...', 1054, 'SELECT  ipb_id,...', 'MigrateComments...')
#2 /usr/share/mediawiki/includes/libs/rdbms/database/Database.php(1227): Wikimedia\Rdbms\Database->reportQueryError('Unknown column ...', 1054, 'SELECT  ipb_id,...', 'MigrateComments...', false)
#3 /usr/share/mediawiki/includes/libs/rdbms/database/Database.php(1907): Wikimedia\Rdbms\Database->query('SELECT  ipb_id,...', 'MigrateComments...', 32)
#4 /usr/share/mediawiki/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->select('ipblocks', Array, Array, 'MigrateComments...', Array)
#5 /usr/share/mediawiki/includes/libs/rdbms/database/DBConnRef.php(313): Wikimedia\Rdbms\DBConnRef->__call('select', Array)
#6 /usr/share/mediawiki/maintenance/migrateComments.php(162): Wikimedia\Rdbms\DBConnRef->select('ipblocks', Array, Array, 'MigrateComments...', Array)
#7 /usr/share/mediawiki/maintenance/migrateComments.php(55): MigrateComments->migrate('ipblocks', Array, 'ipb_reason')
#8 /usr/share/mediawiki/maintenance/includes/LoggedUpdateMaintenance.php(45): MigrateComments->doDBUpdates()
#9 /usr/share/mediawiki/includes/installer/DatabaseUpdater.php(1348): LoggedUpdateMaintenance->execute()
#10 /usr/share/mediawiki/includes/installer/DatabaseUpdater.php(554): DatabaseUpdater->migrateComments()
#11 /usr/share/mediawiki/includes/installer/DatabaseUpdater.php(517): DatabaseUpdater->runUpdates(Array, false)
#12 /usr/share/mediawiki/maintenance/update.php(181): DatabaseUpdater->doUpdates(Array)
#13 /usr/share/mediawiki/maintenance/doMaintenance.php(107): UpdateMediaWiki->execute()
#14 /usr/share/mediawiki/maintenance/update.php(253): require_once('/usr/share/medi...')
#15 {main}

While in theory this should work, it is probably not well tested. Going via 1.31 should help

I am seeing the same error migrating ipblocks, probably due to $wgSharedTables having ipblocks in it. I think the solution will be to unshare the tables.

Thank you so much @tstarling. I can surely try it this week.

I would suggest temporarily removing ipblocks from $wgSharedTables during the upgrade. Re-add it once all wikis are upgraded.

I kind of restarted this upgrade from scratch last week. I will be a bit verbose soon.

Thanks to T277880: Deploy a testing Wikimini now the Wikimini.org community can test things somewhere.

Downloaded everything off-site with mysqldump and rsync. The off-site backup consists in 13GB of filesystem stuff and 1.9GB of .sql.gz dumps.

Relevant upgrade notes:

Do not upgrade from a MediaWiki version older than 1.33 to MediaWiki 1.39.1, or you may lose data! Upgrade to 1.35 first. See task T326071.

• https://www.mediawiki.org/wiki/Manual:Upgrading

Also relevant:

Since Version 1.36, MediaWiki only commits to supporting upgrades from two LTS releases ago (see phab:T259771). Upgrades from older versions of MediaWiki will have to be performed in multiple steps. This means that if you want to upgrade to 1.41 from 1.34 or earlier, you'll first have to upgrade your 1.34 wiki to 1.35 (or 1.39), and, from 1.35 (or 1.39), you'll be able to upgrade to 1.41.

So we are trying to upgrade from 1.28.0 to 1.35.14 instead of going directly to 1.39. If this fails, we will try from 1.31 as suggested in https://phabricator.wikimedia.org/T278033#6933946

https://releases.wikimedia.org/mediawiki/1.35/mediawiki-1.35.14.tar.gz

md5    2bc2ebfd0f2ae03f0d4ad38d525f9721
sha1   ba74290829705eb6b7cc5390306d0f3b48d19430
sha256 4e1ca6b7bd9feb07e705897ef0c2ca044b6c7ec98c8e98c1e1e4ebf787074558

Deployed in /var/www/wikimini.org/www.testing/w

I will verbosily update the description with extensions info

First of all:

This time I've disabled some shared tables (from WikiminiSettings/WikiFamilySettings.php) as suggested by dear Tim Starling

Crash:

PHP Warning:  Use of undefined constant NS_IMAGE - assumed 'NS_IMAGE' (this will throw an Error in a future version of PHP) in /var/www/wikimini.org/www.testing/w/WikiminiSettings/ExtensionSettings/ExtensionsFrench.php on line 174
PHP Warning:  Use of undefined constant NS_IMAGE_TALK - assumed 'NS_IMAGE_TALK' (this will throw an Error in a future version of PHP) in /var/www/wikimini.org/www.testing/w/WikiminiSettings/ExtensionSettings/ExtensionsFrench.php on line 175
PHP Warning:  Use of undefined constant NS_IMAGE - assumed 'NS_IMAGE' (this will throw an Error in a future version of PHP) in /var/www/wikimini.org/www.testing/w/WikiminiSettings/ExtensionSettings/ExtensionsFrench.php on line 196
PHP Warning:  Use of undefined constant NS_IMAGE_TALK - assumed 'NS_IMAGE_TALK' (this will throw an Error in a future version of PHP) in /var/www/wikimini.org/www.testing/w/WikiminiSettings/ExtensionSettings/ExtensionsFrench.php on line 197

https://www.mediawiki.org/wiki/Release_notes/1.34

The constants NS_IMAGE and NS_IMAGE_TALK, deprecated in 1.14, have been removed. Use NS_FILE and NS_FILE_TALK respectively.

Replaced entries from ./ExtensionSettings/Extensions*.php from:

• Arabian
• English
• French
• Italian
• Stock
• Swedish

Finally the page renders! With crash errors but it renders without epic failures.

So I'm trying the first upgrade.

sudo -u www-data HTTP_HOST=fr.demo.wikimini.gitpull.it php maintenance/update.php

It's running since minutes.

Upgrade crash:

User name "MisseArt" is usable, cannot create an anonymous actor for it. Run maintenance/cleanupUsersWithNoId.php to fix this situation.

Maybe I should upgrade the central tables first to avoid these 🤔 Will retry later from there from "stock" wiki directly.

And well, we have "BossEveryone" that is taking care of our legacy choices in database charsets:

Wikimedia\Rdbms\DBQueryError from line 1713 of /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php: Error 1300: Invalid utf8 character string: 'F09F98' (127.0.0.1)
Function: MigrateActors::addActorsForRows
Query: SELECT  actor_id,actor_name  FROM `actor`    WHERE actor_name = '⚽😎💪BossEveryone💪😎⚽'  

#0 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php(1697): Wikimedia\Rdbms\Database->getQueryException('Invalid utf8 ch...', 1300, 'SELECT  actor_i...', 'MigrateActors::...')
#1 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php(1672): Wikimedia\Rdbms\Database->getQueryExceptionAndLog('Invalid utf8 ch...', 1300, 'SELECT  actor_i...', 'MigrateActors::...')
#2 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php(1241): Wikimedia\Rdbms\Database->reportQueryError('Invalid utf8 ch...', 1300, 'SELECT  actor_i...', 'MigrateActors::...', false)
#3 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php(1921): Wikimedia\Rdbms\Database->query('SELECT  actor_i...', 'MigrateActors::...', 32)
#4 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->select('actor', Array, Array, 'MigrateActors::...')
#5 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/DBConnRef.php(313): Wikimedia\Rdbms\DBConnRef->__call('select', Array)
#6 /var/www/wikimini.org/www.testing/w/maintenance/includes/MigrateActors.php(228): Wikimedia\Rdbms\DBConnRef->select('actor', Array, Array, 'MigrateActors::...')
#7 /var/www/wikimini.org/www.testing/w/maintenance/includes/MigrateActors.php(412): MigrateActors->addActorsForRows(Object(Wikimedia\Rdbms\MaintainableDBConnRef), 'rev_user_text', Array, Array, 244287)
#8 /var/www/wikimini.org/www.testing/w/maintenance/includes/MigrateActors.php(105): MigrateActors->migrateToTemp('revision', 'rev_id', Array, 'rev_user', 'rev_user_text', 'revactor_rev', 'revactor_actor')
#9 /var/www/wikimini.org/www.testing/w/maintenance/includes/LoggedUpdateMaintenance.php(45): MigrateActors->doDBUpdates()
#10 /var/www/wikimini.org/www.testing/w/includes/installer/DatabaseUpdater.php(1387): LoggedUpdateMaintenance->execute()
#11 /var/www/wikimini.org/www.testing/w/includes/installer/DatabaseUpdater.php(554): DatabaseUpdater->migrateActors()
#12 /var/www/wikimini.org/www.testing/w/includes/installer/DatabaseUpdater.php(517): DatabaseUpdater->runUpdates(Array, false)
#13 /var/www/wikimini.org/www.testing/w/maintenance/update.php(181): DatabaseUpdater->doUpdates(Array)
#14 /var/www/wikimini.org/www.testing/w/maintenance/doMaintenance.php(107): UpdateMediaWiki->execute()
#15 /var/www/wikimini.org/www.testing/w/maintenance/update.php(253): require_once('/var/www/wikimi...')
#16 {main}

We can start from this point next soon.

Attempt number 4.

I've pulled again databases from production to demo.

MAX WAIT: 1H.

Sub-task: we had mixed charsets and collations. Some latin1 (legacy), some utf8mb4 (default in MariaDB in our Debian distribution)

Tried to migrate legacy latin1:

SELECT CONCAT ('ALTER TABLE ', TABLE_SCHEMA, '.', TABLE_NAME, ' CONVERT TO CHARACTER SET binary ')  FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_COLLATION LIKE '%latin%' AND TABLE_SCHEMA LIKE '%beta%';

Result saved in a file. 257 commands. Executing:

mysql --verbose < migrate-latin-to-binary.sql

Some minutes then crashed:

$ ALTER TABLE wikimini_beta_frwiki.watchlist CONVERT TO CHARACTER SET binary
ERROR 1406 (22001) at line 11: Data too long for column 'wl_title' at row 30045

Note table creation:

$ SHOW CREATE TABLE wikimini_beta_frwiki.watchlist
CREATE TABLE `awc_f_sessions` (
  `ses_name` varchar(255) NOT NULL DEFAULT '0',
  `ses_id` int(11) DEFAULT 0,
  `ses_where` varchar(255) DEFAULT '',
  `ses_guest` int(11) NOT NULL DEFAULT 1,
  `ses_when` int(10) unsigned NOT NULL DEFAULT 0,
  `ses_browser` varchar(255) DEFAULT '',
  `ses_bot` int(11) DEFAULT 1,
  `ses_type` varchar(255) DEFAULT '',
  `ses_perm` varchar(255) DEFAULT '',
  PRIMARY KEY (`ses_name`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci

Probably caused by MyISAM instead of InnoDB. Confirmed that we have mixed engines.

Sub-sub-task. Trying to avoid MyISAM everywhere:

SELECT  CONCAT('ALTER TABLE ', TABLE_SCHEMA, '.', table_name, ' ENGINE=InnoDB;') AS sql_statements FROM information_schema.tables AS tb WHERE table_schema LIKE '%beta%' AND ENGINE = 'MyISAM' AND TABLE_TYPE = 'BASE TABLE';

Result saved on file. Executing that:

mysql --verbose < innodb.sql

MAX WAIT: 1H.

Also altered all 11 beta databases with

ALTER DATABASE wikimini_beta_testwiki CHARACTER SET binary;

Tried to upgrade stock wiki but immediately crashed because of WikiSEO. See description. Disabled from all positions:

./ExtensionSettings/ExtensionsItalian.php:#require_once "$wgOtherExtensions/WikiSEO/WikiSEO.php";
./ExtensionSettings/ExtensionsStock.php:#require_once "$wgOtherExtensions/WikiSEO/WikiSEO.php";
./ExtensionSettings/ExtensionsSwedish.php:require_once "$wgOtherExtensions/WikiSEO/WikiSEO.php";
./ExtensionSettings/ExtensionsFrench.php://require_once "$wgOtherExtensions/WikiSEO/WikiSEO.php";
./ExtensionSettings/ExtensionsFrench.php:#wfLoadExtension( 'WikiSEO' );
./ExtensionSettings/ExtensionsEnglish.php:require_once "$wgOtherExtensions/WikiSEO/WikiSEO.php";
./ExtensionSettings/ExtensionsArabian.php:#require_once "$wgOtherExtensions/WikiSEO/WikiSEO.php";
./ExtensionSettings/ExtensionsSpanish.php:#require_once "$wgOtherExtensions/WikiSEO/WikiSEO.php";

Tried again from stock wiki:

sudo -u www-data HTTP_HOST=stock.demo.wikimini.gitpull.it php maintenance/update.php

MAX WAIT: 15 minutes

Success!

Tried for fr:

sudo -u www-data HTTP_HOST=fr.demo.wikimini.gitpull.it    php maintenance/update.php

Wikimedia\Rdbms\DBQueryError from line 1713 of /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php: Error 1300: Invalid utf8 character string: 'F09F98' (127.0.0.1)
Function: MigrateActors::addActorsForRows
Query: SELECT  actor_id,actor_name  FROM `actor`    WHERE actor_name = '⚽😎💪BossEveryone💪😎⚽'

OK still crash on the same point. Strange.

Database definition:

CREATE DATABASE `wikimini_beta_frwiki` /*!40100 DEFAULT CHARACTER SET binary */

Table definition:

CREATE TABLE `actor` (
  `actor_id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `actor_user` int(10) unsigned DEFAULT NULL,
  `actor_name` varchar(255) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
  PRIMARY KEY (`actor_id`),
  UNIQUE KEY `actor_name` (`actor_name`),
  UNIQUE KEY `actor_user` (`actor_user`)
) ENGINE=InnoDB AUTO_INCREMENT=167 DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci

Why the actor table is not binary? 🤔 I don't know.

Maybe we can just nuke that specific row.

Premise: now at least stock demo works (temporary domain)

https://stock.demo.wikimini.gitpull.it/wiki/Main_Page

But I encountered some problems while trying to convert everything to binary (I'm doing this since MediaWiki loves binary and since we have legacy stuff in case insensitive - https://www.mediawiki.org/wiki/Manual:$wgDBTableOptions )

So I started looking for legacy stuff in utf8mb4_general_ci:

SELECT CONCAT ('ALTER TABLE ', TABLE_SCHEMA, '.', TABLE_NAME, ' CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;') FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_COLLATION LIKE '%general_ci%' AND TABLE_SCHEMA LIKE '%beta%' AND TABLE_TYPE = 'BASE TABLE';

So:

ALTER TABLE wikimini_beta_labwiki.searchindex CONVERT TO CHARACTER SET binary;
ALTER TABLE wikimini_beta_labwiki.sites CONVERT TO CHARACTER SET binary;
-- etc.

But:

ERROR 1283 (HY000) at line 91: Column 'si_title' cannot be part of FULLTEXT index

Caused by this table:

https://www.mediawiki.org/wiki/Manual:Searchindex_table

I'm quite confused. Why does MediaWiki suggests binary, and why the documentation about the searchindex table does not says that it does not support binary? Anyway, trying on something FULLTEXT-friendly but still case-sensitive, like utf8mb4_bin.

ALTER TABLE wikimini_beta_labwiki.searchindex CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;
-- etc.

Accepted. Trying again on fr:

• ar
• es
• fr
• it
• lab
• stock
• sv
• test

sudo -u www-data HTTP_HOST=fr.demo.wikimini.gitpull.it php maintenance/update.php

Running. (Still getting a pletora of User name "Haron Moussaoui" is usable, cannot create an anonymous actor for it. Run maintenance/cleanupUsersWithNoId.php to fix this situation. and similar things)

Finally arrived at this point:

Completed migration, inserted 713 row(s) with 0 new actor(s), 713 error(s)
errors were encountered.
Modifying rev_text_id field of table revision ...done.
Modifying table site_stats ...done.
Populating ar_rev_id.
Populating ar_rev_id...
Completed ar_rev_id population, 0 rows updated.
Making ar_rev_id not nullable ...done.
Adding index rc_namespace_title_timestamp to table recentchanges ...done.
Creating change_tag_def table ...done.
Populating el_index_60 field, printing progress markers. For large
databases, you may want to hit Ctrl-C and do this manually with
maintenance/populateExternallinksIndex60.php.
Populating externallinks.el_index_60...

Also reached this point for the first time:

Updating externallinks table index fields
el_id 1 - 10001 of 28695
el_id 10001 - 20001 of 28695
el_id 20001 - 28695 of 28695
Done, 77 rows updated, 80 deleted.
Set the local repo temp zone container to be private.
Purging caches...done.

Done in 5 min 4 s.

DONE - AFTER 3 WEEKS \o/

Proceeding with ar wiki that should be easier.

$ sudo -u www-data HTTP_HOST=ar.demo.wikimini.gitpull.it php maintenance/update.php
PHP Warning:  require_once(/var/www/wikimini.org/www.testing/w/extensions/SpecialForm/SpecialForm.setup.php): failed to open stream: No such file or directory in /var/www/wikimini.org/www.testing/w/WikiminiSettings/ExtensionSettings/ExtensionsArabian.php on line 38
PHP Fatal error:  require_once(): Failed opening required '/var/www/wikimini.org/www.testing/w/extensions/SpecialForm/SpecialForm.setup.php' (include_path='/var/www/wikimini.org/www.testing/w/vendor/pear/console_getopt:/var/www/wikimini.org/www.testing/w/vendor/pear/mail:/var/www/wikimini.org/www.testing/w/vendor/pear/mail_mime:/var/www/wikimini.org/www.testing/w/vendor/pear/net_smtp:/var/www/wikimini.org/www.testing/w/vendor/pear/net_socket:/var/www/wikimini.org/www.testing/w/vendor/pear/pear-core-minimal/src:/var/www/wikimini.org/www.testing/w/vendor/pear/pear_exception:.:/usr/share/php') in /var/www/wikimini.org/www.testing/w/WikiminiSettings/ExtensionSettings/ExtensionsArabian.php on line 38

Disabling SpecialForm from WikiminiSettings/ExtensionSettings/ExtensionsArabian.php

Trying again. Completed! \o/

• ar
• es
• fr
• it
• lab
• stock
• sv
• test

Trying with es:

$ sudo -u www-data HTTP_HOST=es.demo.wikimini.gitpull.it php maintenance/update.php
PHP Warning:  require_once(/var/www/wikimini.org/www.testing/w/extensions/SpecialForm/SpecialForm.setup.php): failed to open stream: No such file or directory in /var/www/wikimini.org/www.testing/w/WikiminiSettings/ExtensionSettings/ExtensionsSpanish.php on line 38
PHP Fatal error:  require_once(): Failed opening required '/var/www/wikimini.org/www.testing/w/extensions/SpecialForm/SpecialForm.setup.php' (include_path='/var/www/wikimini.org/www.testing/w/vendor/pear/console_getopt:/var/www/wikimini.org/www.testing/w/vendor/pear/mail:/var/www/wikimini.org/www.testing/w/vendor/pear/mail_mime:/var/www/wikimini.org/www.testing/w/vendor/pear/net_smtp:/var/www/wikimini.org/www.testing/w/vendor/pear/net_socket:/var/www/wikimini.org/www.testing/w/vendor/pear/pear-core-minimal/src:/var/www/wikimini.org/www.testing/w/vendor/pear/pear_exception:.:/usr/share/php') in /var/www/wikimini.org/www.testing/w/WikiminiSettings/ExtensionSettings/ExtensionsSpanish.php on line 38

Disabling SpecialForm from WikiminiSettings/ExtensionSettings/ExtensionsSpanish.php

Retrying. Done! \o/

• ar
• es
• fr
• it
• lab
• stock
• sv
• test

Proceeding with it.

Same crash. Disabling SpecialForms from WikiminiSettings/ExtensionSettings/ExtensionsItalian.php. Retrying it.

Wow! A new crash!

MediaWiki 1.35.14 Updater

Your composer.lock file is up to date with current dependencies!
Going to run database updates for wikimini_beta_itwiki
Depending on the size of your database this may take a while!
Abort with control-c in the next five seconds (skip this countdown with --quick) ... 0
...have ipb_id field in ipblocks table.
...have ipb_expiry field in ipblocks table.
...already have interwiki table
...indexes seem up to 20031107 standards.
...have rc_type field in recentchanges table.
...index new_name_timestamp already set on recentchanges table.
...user table does not exist, skipping new field patch.
...querycache table already exists.
...objectcache table already exists.
...categorylinks table already exists.
...have pagelinks; skipping old links table updates
...il_from OK
...have rc_ip field in recentchanges table.
...index PRIMARY already set on image table.
...have rc_id field in recentchanges table.
...have rc_patrolled field in recentchanges table.
...logging table already exists.
...user table does not exist, skipping new field patch.
...watchlist table does not exist, skipping new field patch.
Wikimedia\Rdbms\DBQueryError from line 1713 of /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php: Error 1146: Table 'wikimini_beta_itwiki.watchlist' doesn't exist (127.0.0.1)
Function: MysqlUpdater::doWatchlistUpdate
Query: (SELECT  wlsubject.wl_user AS `wl_user`,wlsubject.wl_namespace | 1 AS `wl_namespace`,wlsubject.wl_title AS `wl_title`,wlsubject.wl_notificationtimestamp AS `wl_notificationtimestamp`  FROM `watchlist` `wlsubject` LEFT JOIN `watchlist` `wltalk` ON ((wltalk.wl_user = wlsubject.wl_user) AND (wltalk.wl_namespace = (wlsubject.wl_namespace | 1)) AND (wltalk.wl_title = wlsubject.wl_title))   WHERE (NOT (wlsubject.wl_namespace & 1)) AND (wltalk.wl_namespace IS NULL)  ) UNION ALL (SELECT  wltalk.wl_user AS `wl_user`,wltalk.wl_namespace & ~1 AS `wl_namespace`,wltalk.wl_title AS `wl_title`,wltalk.wl_notificationtimestamp AS `wl_notificationtimestamp`  FROM `watchlist` `wltalk` LEFT JOIN `watchlist` `wlsubject` ON ((wlsubject.wl_user = wltalk.wl_user) AND (wlsubject.wl_namespace = (wltalk.wl_namespace & ~1)) AND (wlsubject.wl_title = wltalk.wl_title))   WHERE (wltalk.wl_namespace & 1) AND (wlsubject.wl_namespace IS NULL)  )

#0 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php(1697): Wikimedia\Rdbms\Database->getQueryException('Table 'wikimini...', 1146, '(SELECT  wlsubj...', 'MysqlUpdater::d...')
#1 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php(1672): Wikimedia\Rdbms\Database->getQueryExceptionAndLog('Table 'wikimini...', 1146, '(SELECT  wlsubj...', 'MysqlUpdater::d...')
#2 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/Database.php(1241): Wikimedia\Rdbms\Database->reportQueryError('Table 'wikimini...', 1146, '(SELECT  wlsubj...', 'MysqlUpdater::d...', false)
#3 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->query('(SELECT  wlsubj...', 'MysqlUpdater::d...', 0)
#4 /var/www/wikimini.org/www.testing/w/includes/libs/rdbms/database/DBConnRef.php(286): Wikimedia\Rdbms\DBConnRef->__call('query', Array)
#5 /var/www/wikimini.org/www.testing/w/includes/installer/MysqlUpdater.php(612): Wikimedia\Rdbms\DBConnRef->query('(SELECT  wlsubj...', 'MysqlUpdater::d...')
#6 /var/www/wikimini.org/www.testing/w/includes/installer/DatabaseUpdater.php(554): MysqlUpdater->doWatchlistUpdate()
#7 /var/www/wikimini.org/www.testing/w/includes/installer/DatabaseUpdater.php(517): DatabaseUpdater->runUpdates(Array, false)
#8 /var/www/wikimini.org/www.testing/w/maintenance/update.php(181): DatabaseUpdater->doUpdates(Array)
#9 /var/www/wikimini.org/www.testing/w/maintenance/doMaintenance.php(107): UpdateMediaWiki->execute()
#10 /var/www/wikimini.org/www.testing/w/maintenance/update.php(253): require_once('/var/www/wikimi...')
#11 {main}

Interestingly the testing wikimini_beta_itwiki has 67 tables and the last one in alphabetical order is text - and instead production has 83 tables. Something is broken in the ./pull-production-in-testing.sh (?)

Re-executing again ./pull-production-in-testing.sh.

For some unknown reasons the import ends with success but without terminating 🤔 nothing in standard output, error, nothing in /var/log/mysql/error.log. The text table is imported with ~100 entries instead of 683113 rows.

Seems related:

https://bugs.mysql.com/bug.php?id=77879

Trying with --hex-blob.

I began investigating a bit: the extension WhosOnline is seemingly similar to CurrentUsers according to the description, with a minor difference in the display of anons and bots, and it works fine on MW 1.43 (after some routine maintenance about one deprecated method deleted after 1.40), so it should work fine on MW 1.35 and 1.39. It displays a bullet-points list of the logged-in readers in the last 60 minutes, on a special page and possibly on some wiki page like the main page like attached screenshot (I enabled the display of anon users, it’s a parameter).

Sorry, I prepared the following comment around 30 December 2024, I expected to fix the minor details before posting, but I was busy then. I will post an updated comment just after.

I worked today [30 December 2024] on upgrading Wikimini (fr and stock), it’s long but it works so far. On my computer, where I have PHP 7.0 and 7.4 (and others), I created transitional installations 1.31 and 1.35 where these installations have only 2 extensions (LiquidThreads and CheckUser), since only these two extensions have custom database tables, all other extensions should not be needed during transition. I commented all activations of extensions in these transitional installations.

I upgraded without issues the upgrade 1.28 → 1.31 for FR and STOCK with PHP 7.0, although I set $wgSharedDB = null; as discussed above, I hope the usernames were not mixed if actors are attributed differently accross wikis (I have to check this, and I will possibly re-launch the upgrade).

Then the upgrade 1.31 → 1.35 of stock.wikimini with PHP 7.4, no issues reported. A small maintenance was needed in config as said above for NS_IMAGE → NS_FILE.

In the past for other customers, I tried upgrading from 1.27/1.28 to 1.35, but I experienced a lot of issues mainly about actors but also other DB transitions around 1.31/1.32/1.34, so now I always do a transition through 1.31 (at least, possibly 1.32 and possibly 1.34 in case of issues, some old wikis should have specific data behaving badly).

That's a very good 2025 news, thanks Seb!

The migration by multiple steps seems good. I'm curious about the upgrade with $wgSharedDB on the right value. Fortunately, wikimini_frwiki.user seems to be empty, while sv has only user but it's just WikiSysopMini, and probably so on; another good news to have less mixed usernames. Keep us updated 👍

EDIT: added a 5th option for the skin.

The next things planned are:

• upload prepared installations of MW 1.31 and 1.35 on test server, with the adapted LocalSettings; there is PHP 7.3, it should be fine both for 1.31 and 1.35
• (I would prefer to launch upgrades for all wikis with the shared DB, even if I did otherwise in my tests, to be sure users ↔ actors are correctly attributed, so I plan to keep the original $wgSharedDB in the next upgrades)
• on 1.31, be sure the images directories are correctly configured for all wikis (possibly copy the original images directories somewhere else for the upgrades)
• on 1.31, launch maintenance/cleanupUsersWithNoId.php --assign --prefix 'stock' on stock
• on 1.31, launch maintenance/update.php --quick on stock then on all other wikis
• on 1.35, be sure the images directories are correctly configured for all wikis
• on 1.35, launch maintenance/update.php --quick on stock then on all other wikis

An independant thing to work on is the skin: the current code on 1.28 is a fork of MonoBook, and it doesn’t work out-of-the-box on 1.35, so a decision has to be maid (roughly sorted by increasing difficulty):

1. use a standard skin: MonoBook, Vector, Minerva, Timeless, Chameleon…
2. upgrade the code a minima for 1.35, probably quite quick for 1.35 but it will be more difficult for 1.39 and 1.43 since skins features are quite evolving
3. use MonoBook 1.35 and reproduce the current style mainly with CSS rules as far as possible (to avoid big changes and facilitate the next upgrades)
4. use Chameleon and reproduce the current style, it could be more perennial (chameleon is not used by WMF but a lot of wikis use it)
5. [NEW] joker option, the resulting skin may vary more or less from the existing skin, possibly related to the complexity/time spent of some parts and/or if you or the community wants to make evolve some things

We could estimate the time needed for each, but do you have preferred options for now?

UPDATED 2025-04-25 with update to 1.39

I just launched on mores-demo-testing the updates to 1.31 for: stock, ar, en, lab, sv, fr, it, es (by increasing size), then 1.35 for: stock, ar, en, lab, sv, fr, it, es

Be sure that:

• on 1.31, 1.35, 1.39:
  • $wgSharedTables includes actor but do not include ipblocks
  • $wgSharedDB is null for stock.wikimini.org, and is wikimini_stockwiki elsewhere

Actual operations done are:

• uploaded prepared 1.31 and 1.35 on /var/www/wikimini.org/upgrade_wikivalley
• made a bind mount from /var/www/wikimini.org/images to /var/www/wikimini.org/upgrade_wikivalley/1.3*/images
• sudo chown -R www-data 1.3*/cache 1.3*/images LocalSettings* WikiminiSettings
• the DB eswiki has a broken table searchindex (*), it can be removed and recreated: sudo mysql wikimini_eswiki then: DROP TABLE searchindex; and recreate this table and its indexes from their definitions in /var/www/wikimini.org/www/w/maintenance/tables.sql
• made DB dumps to be able to restart in case of issue: sudo mysqldump wikimini_itwiki|gzip -7 >wikimini_itwiki-2025-01-20.sql.gz (in /var/www/wikimini.org/upgrade_wikivalley/dumps)
• NB: wgSharedDB is kept as null on stock and non-null elsewhere (value 'wikimini_stockwiki')
• on 1.31 and 1.35, be sure that no extension except AbuseFilter and LiquidThread is activated
• in a browser, open the test Wikimini after setting 192.168.128.50 fr.wikimini.org (and similar for stock) in local /etc/hosts (NB: on Firefox, an exception is needed for these domains if a "secure DNS/DoH" is enabled), check the website works (in 1.28), and check in the DevTools that the called IP is this one and not the official one,
• installed the apt package screen to launch updates in a screen session, all following PHP commands are in a screen,
• on 1.31, launched sudo -u www-data HTTP_HOST=stock.wikimini.org php7.3 maintenance/update.php --quick on stock,
• on 1.31, launched sudo -u www-data HTTP_HOST=stock.wikimini.org php7.3 maintenance/cleanupUsersWithNoId.php --assign --prefix 'stock' on stock,
• on 1.31, launch sudo -u www-data HTTP_HOST=<language>.wikimini.org php7.3 maintenance/eval.php on ar, en, lab, sv, fr, it, es, to check there is no warning before launching update.php
• on 1.31, launch sudo -u www-data HTTP_HOST=<language>.wikimini.org php7.3 maintenance/update.php --quick on ar, en, lab, sv, fr, it, es,
• in /var/www/wikimini.org/www, move w to w.1-28 (to keep old 1.28) then sudo ln -s ../../upgrade_wikivalley/1.31, then sudo systemctl restart apache2.service
• in a browser, open some Wikimini website (check it is really 192.168.128.50), and particularly Special:Version, it should be 1.31
• on 1.35, launched sudo -u www-data HTTP_HOST=stock.wikimini.org php7.3 maintenance/update.php --quick on stock,
• on 1.35, launch sudo -E -u www-data HTTP_HOST=<language>.wikimini.org php7.3 maintenance/eval.php on ar, en, lab, sv, fr, it, es, to check there is no warning before launching update.php
• on 1.35, launch sudo -u www-data HTTP_HOST=<language>.wikimini.org php7.3 maintenance/update.php --quick on ar, en, lab, sv, fr, it, es
• on 1.35, for eswiki (only, I don’t know why),
  • execute in the database (sudo mysql wikimini_eswiki): INSERT INTO slot_roles(role_name) VALUES ("main"); (something I proposed the patch some time ago in T286578 and still not merged, I should insist or create a fork with this patch, I’m still upgrading pre-1.32 wikis like now, probably one or two per year and some have this issue)
  • then execute sudo -u www-data HTTP_HOST=es.wikimini.org php7.3 maintenance/populateContentTables.php
  • then re-launch the update for eswiki
• in /var/www/wikimini.org/www : sudo ln -s ../../upgrade_wikivalley/1.35, then sudo systemctl restart apache2.service
• on 1.39, launched sudo -u www-data HTTP_HOST=stock.wikimini.org php8.2 maintenance/update.php --quick on stock,
• on 1.39, launch sudo -E -u www-data HTTP_HOST=<language>.wikimini.org php8.2 maintenance/eval.php on ar, en, lab, sv, fr, it, es, to check there is no warning before launching update.php
• on 1.39, launch sudo -u www-data HTTP_HOST=<language>.wikimini.org php8.2 maintenance/update.php --quick on ar, en, lab, sv, fr, it, es
• in /var/www/wikimini.org/www : sudo ln -s ../../upgrade_wikivalley/1.39, then sudo systemctl restart apache2.service

At the end:

• you can re-enable ipblocks in $wgSharedTables
• remove APT packages php7.3*
• restore Debian version of php-common (instead of sury)
• remove APT package debsuryorg-archive-keyring and APT pinning /etc/apt/preferences.d/lower-php-sury.pref

(*) mysqldump: Got error: 144: "Table './wikimini_eswiki/searchindex' is marked as crashed and last (automatic?) repair failed" when using LOCK TABLES

Yuh! Thanks (About searchindex I confirm I had the same problem before. I guess we can nuke it, as you said).

The current state is that all Wikiminis are upgraded to 1.35 on mores-demo-testing, currently almost all extensions are disabled, I will re-enable most for our call of Friday, and list remaining issues.

To view these wikis, you have to:

• activate the WMCH VPN,
• write in your /etc/hosts (C:\Windows\System32\drivers\etc\hosts on Windows) : 192.168.128.50 fr.wikimini.org and similar for other Wikiminis
• be sure your are not using "secure DNS"/DoH/DoT in your browser, else /etc/hosts is not taken into account (on Firefox, it is possible to add exceptions for these domains in Parameters > Privacy & Security > DNS over HTTPS)

An alternative would be to configure in some WMCH router (I don’t know where) to redirect some domain (e.g. .test.wikimini.org) to this internal IP, but it should be emitted SSL certificates, so it’s some work.

I feel it would not a lot of supplementary work to upgrade on 1.39 (EOL in November 2025), possibly I may work on it quickly, we could discuss about it Friday.

But there is some work still about the skin, see my comment T278033#10469888 to be discussed Friday also.

Hello everyone and thank you so much @Seb35 for everything you've already done! It looks really promising!

Concerning the skin, it's been an integral part of Wikimini's identity since its creation: simple and colorful, the reduced width (which facilitates reading and highlights the children's short texts), the use of icons, the removal of certain elements to simplify the interface, etc. In my opinion, it would be sad to lose this.

We also invested a lot of time in updating this skin during the previous major Wikimini update.

Of course, the skin is already a few years old, and there's certainly room for improvement. Some features no longer work, the icons have aged a little, it would be interesting to enlarge the text size, add some spaces, etc.

I'm not familiar with the Chamaleon skin. But I believe the current Wikimini skin is also based on Bootstrap (since the last major Wikimini update). Option 5 (Joker) is of course super tempting, as it would also “refresh” the look of the site, But we'd need someone qualified to take care of it.

Thanks again for all your hard work!

Thanks Lorangeo. P.S. I must be completely idiot O.o I completely forgot this work that seems promising!

https://gitlab.wikimedia.org/ashley/mediawiki-skins-Wikimini

T356324: Adopt the Wikimini ashley's skin (at least in testing-Wikimini)

We tested the skin of Ashley on the 1.35 installation, it does not work since it requires MediaWiki 1.39 (we tried to go back in the history, but without success, there are other issues and it did not work).

So one solution is to continue upgrading Wikimini to 1.39, did I continue on this path? My guess is it should be quite easy (but there may be surprises).

So one solution is to continue upgrading Wikimini to 1.39, did I continue on this path? My guess is it should be quite easy (but there may be surprises).

Thank you so much Seb35, yes please, let's try to adopt 1.39 to unlock the new skin 👍 fingers crossed

I prepared the version 1.39, it is in /var/www/wikimini.org/upgrade_wikivalley/1.39, but it requires PHP 7.4 (we are currently in PHP 7.3). I was about to propose to install it from deb.sury.org, but version for Debian buster is no more available.

@ValerioBoz-WMCH : May we organise a call to discuss about it? I am available the next week. In the meantime, I test Wikimini 1.39 in my local environment.

Yep unfortunately sury.org cannot save our souls right now. Probably I can prepare a super-dirty workaround by creating a "simple" Docker Compose file with a bind mount, so to do not require yet another migration. So we have a confined process inside a Docker to be exposed via proxypass to Apache and it should fit the job (horribly) and gaining some time to postpone the OS upgrade.

Sound's reasonable? I can boost this tomorrow in case probably

In T278033#10679701, @valerio.bozzolan wrote:

Sound's reasonable? I can boost this tomorrow in case probably

Yes, preferably with the directory /var/www/wikimini.org as bind mount in the container in the same directory (there are various things like images that must be accessible by MW). You can let the Dockerfile/docker-compose.yml somewhere, I may need to adjust things.

I tested yesterday locally on my computer the upgrade of sv.wikimini.org and stock.wikimini.org with the skin Wikimini of @ashley, it works globally, there are still some things to fix (like the "image of the day").

Wait a minute. We are talking about the server "mores-demo-testing" 192.168.128.50 that is not used for anything so what about just upgrading from Debian 11 to Debian 12? Then it would be easier to support older PHP versions, if ever needed, thanks to sury.org

Would it help you? Any downside in your workflow? @Seb35

(I received a confirmation from Seb35 off-site to start the Debian upgrade - nice 👍 I can start today at 3PM CEST)

Upgrade of server mores-demo-testing (Wikimini testing) from Debian 10 → to Debian 11 → to Debian 12 concluded. Please tell me if you need anything else to speedup the upgrade 👍

P.S. I had strange problem in Apache2 qos like:

apache2_reload: apache2: Syntax error on line 146 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/qos.load: Cannot load /usr/lib/apache2/modules/mod_qos.so into server: /usr/lib/apache2/modules/mod_qos.so: undefined symbol: pcre_free

I have no idea about what's going on in qos. Problem not solved even after re-installing libapache2-mod-qos so I've disabled it at the moment...

If you need older PHP version feel free to install sury.org as usual.

I deployed today 1.39 on mores-demo-testing for stock, fr, sv with the skin of @ashley. I works globally, and we will do a more extensive check on next Wednesday.

I am re-trying to convert three wikis from the initial state 1.28 to 1.39 (stockwiki, svwiki, frwiki). For that I installed PHP 7.3 with deb.sury.org (it can be removed as soon as the upgrade is finished). To install PHP 7.3 with deb.sury.org, I upgrade the package php-common to the sury version else it is not possible to install PHP 7.3; it can be downgraded to the Debian-provided version when PHP 7.3 will be removed.

stock.wikimini.org is correctly upgraded to 1.39. I have issues on fr and sv because of the ipb_reason_id problem (first comment), I may unshare the tables but I’m afraid it would introduce mixes between user/actor (or user/actor should be copied temporarily from the shared stock.wikimini); I will investigate a bit more this issue.

It’s finally working fine in 1.39 for:

• https://stock.wikimini.org
• https://fr.wikimini.org
• https://sv.wikimini.org

(you have to set /etc/hosts to view these preproduction wikis, see T278033#10480573)

I updated the comment T278033#10476037 with the step for 1.39.

By browsing on some pages on these 3 wikis, I remarked these things to be changed in the wikitext:

• https://fr.wikimini.org/wiki/Accueil : {{formatnum:{{NUMBEROFARTICLES}}}} → {{NUMBEROFARTICLES}} (else hidden category "Pages avec des arguments non numériques dans formatnum", and not needed to use formatnum since {{NUMBEROFARTICLES}} is already formatted with separators)
• https://sv.wikimini.org/wiki/Huvudsida : idem
• https://stock.wikimini.org/wiki/Main_Page/en : https://stock.wikimini.org/wiki/resources/assets/stock-carousel/ → https://stock.wikimini.org/w/resources/assets/stock-carousel/ (multiple occurences)

Other details:

• The visual editor is working fine but there is a warning "Your skin is incompatible with VisualEditor. See https://www.mediawiki.org/wiki/Extension:VisualEditor/Skin_requirements for the requirements.", I may check in more details how to improve this integration.

@valerio.bozzolan: can you check if you see blocking issues before production?

I may re-run the upgrades on all languages, but I don’t find the SQL dumps for other languages (ar, en, es, it, lab) on mores-demo-testing, can you create these dumps from the current production (and by the way also: fr, stock, sv)?

In T278033#10769139, @Seb35 wrote:

• The visual editor is working fine but there is a warning "Your skin is incompatible with VisualEditor. See https://www.mediawiki.org/wiki/Extension:VisualEditor/Skin_requirements for the requirements.", I may check in more details how to improve this integration.

I no longer have a 1.39 box to test with, but this might be resolvable with this VE patch (which was backported to REL1_39 as well). If memory serves me correctly, I did have VE working under 1.39 just fine; it definitely works under MW 1.43.0 with no related changes to the skin. (For 1.43, there are two tiny PHP-level changes needed to get rid of some deprecation notices and whatnot, and a whole bunch of LESS changes related to the .background-image mixin to unbreak the skin for 1.43.0; again, as you're targeting 1.39, it should just work.)

See T396901 for the testing of 1.39 on the domain https://wikimaxi.org