Page MenuHomePhabricator

After upgrade from MediaWiki 1.32 to 1.33 some pages display no text (revision_comment_temp/comment)
Open, HighPublic

Description

First attempt, update.php was consistently failing with (putting it here in case it is relevant):

[2020-04-10 09:46:14] exception.ERROR: [f71d03190b56338b7c9bdc30] [no req]   MediaWiki\Revision\RevisionAccessException from line 131 of /srv/mediawiki/workdir/includes/Revision/RevisionStoreCacheRecord.php: Unable to load fresh row for rev_id: 21917 {"exception":"[object] (MediaWiki\\Revision\\RevisionAccessException(code: 0): Unable to load fresh row for rev_id: 21917 at /srv/mediawiki/workdir/includes/Revision/RevisionStoreCacheRecord.php:131)
[stacktrace]
#0 /srv/mediawiki/workdir/includes/Revision/RevisionStoreCacheRecord.php(94): MediaWiki\\Revision\\RevisionStoreCacheRecord->loadFreshRow()
#1 /srv/mediawiki/workdir/includes/Revision.php(806): MediaWiki\\Revision\\RevisionStoreCacheRecord->getUser(1, NULL)
#2 /srv/mediawiki/workdir/includes/page/WikiPage.php(857): Revision->getUser(1, NULL)
#3 /srv/mediawiki/workdir/extensions/SemanticMediaWiki/src/Importer/ContentCreators/TextContentCreator.php(153): WikiPage->getUser()
#4 /srv/mediawiki/workdir/extensions/SemanticMediaWiki/src/Importer/ContentCreators/TextContentCreator.php(93): SMW\\Importer\\ContentCreators\\TextContentCreator->isCreatorLastEditor(Object(WikiPage))
#5 /srv/mediawiki/workdir/extensions/SemanticMediaWiki/src/Importer/ContentCreators/DispatchingContentCreator.php(75): SMW\\Importer\\ContentCreators\\TextContentCreator->create(Object(SMW\\Importer\\ImportContents))
#6 /srv/mediawiki/workdir/extensions/SemanticMediaWiki/src/Importer/Importer.php(133): SMW\\Importer\\ContentCreators\\DispatchingContentCreator->create(Object(SMW\\Importer\\ImportContents))
#7 /srv/mediawiki/workdir/extensions/SemanticMediaWiki/src/Importer/Importer.php(109): SMW\\Importer\\Importer->doImportContents(Object(SMW\\Importer\\ImportContents))
#8 /srv/mediawiki/workdir/extensions/SemanticMediaWiki/src/MediaWiki/Hooks.php(1352): SMW\\Importer\\Importer->doImport()
#9 /srv/mediawiki/workdir/includes/Hooks.php(174): SMW\\MediaWiki\\Hooks->onAfterCreateTablesComplete(Object(SMW\\SQLStore\\TableBuilder\\MySQLTableBuilder), Object(Onoi\\MessageReporter\\ObservableMessageReporter), Object(SMW\\Options))
#10 /srv/mediawiki/workdir/includes/Hooks.php(202): Hooks::callHook('SMW::SQLStore::...', Array, Array, NULL)
#11 /srv/mediawiki/workdir/extensions/SemanticMediaWiki/src/SQLStore/Installer.php(191): Hooks::run('SMW::SQLStore::...', Array)
#12 /srv/mediawiki/workdir/extensions/SemanticMediaWiki/src/SQLStore/SQLStore.php(437): SMW\\SQLStore\\Installer->install(Object(SMW\\Options))
#13 /srv/mediawiki/workdir/extensions/SemanticMediaWiki/src/Store.php(506): SMW\\SQLStore\\SQLStore->setup(Object(SMW\\Options))
#14 /srv/mediawiki/workdir/includes/installer/DatabaseUpdater.php(489): SMW\\Store::setupStore(true, Object(SMW\\Options))
#15 /srv/mediawiki/workdir/includes/installer/DatabaseUpdater.php(457): DatabaseUpdater->runUpdates(Array, true)
#16 /srv/mediawiki/workdir/maintenance/update.php(203): DatabaseUpdater->doUpdates(Array)
#17 /srv/mediawiki/workdir/maintenance/doMaintenance.php(96): UpdateMediaWiki->execute()
#18 /srv/mediawiki/workdir/maintenance/update.php(275): require_once('/srv/mediawiki/...')
#19 {main}

I was able to work around this by restarting memcached before running update.php.

Second problem is that many pages show no text. Trying to edit them gives this warning:

The revision #0 of the page named "Etusivu" does not exist.

Accessing them via history works though. Purging or restarting memcached does not help.

I was able to track this down to RevisionStore::fetchRevisionRowFromConds which does the following query:

SELECT  rev_id,rev_page,rev_timestamp,rev_minor_edit,rev_deleted,rev_len,rev_parent_id,rev_sha1,comment_rev_comment.comment_text AS `rev_comment_text`,comment_rev_comment.comment_data AS `rev_comment_data`,comment_rev_comment.comment_id AS `rev_comment_cid`,actor_rev_user.actor_user AS `rev_user`,actor_rev_user.actor_name AS `rev_user_text`,temp_rev_user.revactor_actor AS `rev_actor`,page_namespace,page_title,page_id,page_latest,page_is_redirect,page_len,user_name
FROM `revision`
JOIN `revision_comment_temp` `temp_rev_comment` ON ((temp_rev_comment.revcomment_rev = rev_id))
JOIN `comment` `comment_rev_comment` ON ((comment_rev_comment.comment_id = temp_rev_comment.revcomment_comment_id))
JOIN `revision_actor_temp` `temp_rev_user` ON ((temp_rev_user.revactor_rev = rev_id))
JOIN `actor` `actor_rev_user` ON ((actor_rev_user.actor_id = temp_rev_user.revactor_actor))
JOIN `page` ON ((page_id = rev_page)) LEFT JOIN `user` ON ((actor_rev_user.actor_user != 0) AND (user_id = actor_rev_user.actor_user))
WHERE rev_id = '23650' ;

Empty set (0.00 sec)

This returns no rows. If I change the query to remove some joins, it returns rows:

SELECT  rev_id,rev_page,rev_timestamp,rev_minor_edit,rev_deleted,rev_len,rev_parent_id,rev_sha1,actor_rev_user.actor_user AS `rev_user`,actor_rev_user.actor_name AS `rev_user_text`,temp_rev_user.revactor_actor AS `rev_actor`,page_namespace,page_title,page_id,page_latest,page_is_redirect,page_len,user_name
FROM `revision`
JOIN `revision_actor_temp` `temp_rev_user` ON ((temp_rev_user.revactor_rev = rev_id))
JOIN `actor` `actor_rev_user` ON ((actor_rev_user.actor_id = temp_rev_user.revactor_actor))
JOIN `page` ON ((page_id = rev_page)) LEFT JOIN `user` ON ((actor_rev_user.actor_user != 0) AND (user_id = actor_rev_user.actor_user))
WHERE rev_id = '23650' ;

+--------+----------+----------------+----------------+-------------+---------+---------------+---------------------------------+----------+---------------+-----------+----------------+------------+---------+-------------+------------------+----------+-----------+
| rev_id | rev_page | rev_timestamp  | rev_minor_edit | rev_deleted | rev_len | rev_parent_id | rev_sha1                        | rev_user | rev_user_text | rev_actor | page_namespace | page_title | page_id | page_latest | page_is_redirect | page_len | user_name |
+--------+----------+----------------+----------------+-------------+---------+---------------+---------------------------------+----------+---------------+-----------+----------------+------------+---------+-------------+------------------+----------+-----------+
|  23650 |        1 | 20191221193750 |              0 |           0 |    1943 |         22260 | 9bo17oxqd88yv96s5vbpikmddgz8cr9 |        1 | Ylläpito      |         1 |              0 | Etusivu    |       1 |       23650 |                0 |     1943 | Ylläpito  |
+--------+----------+----------------+----------------+-------------+---------+---------------+---------------------------------+----------+---------------+-----------+----------------+------------+---------+-------------+------------------+----------+-----------+

So it seems update.php has botched something with revision_comment_temp or comment table. I have backups so I can easily test and compare before/after.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 10 2020, 10:47 AM
Nikerabbit moved this task from Backlog to Needing on the User-Nikerabbit board.Apr 10 2020, 10:47 AM
Anomie added a subscriber: Anomie.

update.php is supposed to have run maintenance/migrateComments.php to populate that data. That would have happened at any time with MediaWiki 1.30 if you set $wgCommentTableSchemaMigrationStage to a stage indicating that the comment table should be used, or in 1.33 if you never set $wgCommentTableSchemaMigrationStage to such a stage.

I have not set $wgCommentTableSchemaMigrationStage. Only thing I had set was $wgChangeTagsSchemaMigrationStage = MIGRATION_NEW which I also removed. Curiously, highest revision id in revision_comment_temp is 19600, so looks like the table was populated at some point, but not updated after that.

holger.knust triaged this task as High priority.Apr 21 2020, 8:39 PM
daniel added a subscriber: daniel.Apr 21 2020, 8:41 PM

@Nikerabbit What site is this from? What version of SMW was involved here?

MediaWiki 1.32.6
PHP 7.2.24-0ubuntu0.18.04.4 (fpm-fcgi)
MariaDB 10.1.44-MariaDB-0ubuntu0.18.04.1
ICU 60.2
Semantic MediaWiki 3.1.0

This was a testing instance of https://wiki.aineetonkulttuuriperinto.fi/wiki/Toiminnot:Versio - I'm currently blocked updating the production site until I can get the upgrade to work on my testing instance.

@Nikerabbit thank you for the details. It would be very useful to know if this also happens without SMW enabled. Can you test that? This sounds a bit scary, but the wiki seems small, so maybe you can just clone the database?

Sure, I'll do that later this week. Cloning the database to a test environment is how I test upgrades before doing them in production.

I lied, took a bit longer.

The main issue of pages not being visible seems to be fixed by running php maintenance/migrateComments.php --force. I have no idea if this is something other sites may encounter. I am not sure why the table would be only partially populated.

Also, that script must be run before update.php is run, otherwise the comments will be lost forever, at least for recent changes (which causes similar issue there... nothing displayed).

Also, could not reproduce SMW error in my new environment.

I think MediaWiki could and should be more robust and replace the JOIN comment with LEFT JOIN comment (maybe similarly for actor?) with graceful fallback.

Aklapper removed a subscriber: Anomie.Oct 16 2020, 5:01 PM