Page MenuHomePhabricator

Inefficient query in TranslatablePage::getTranslationPages
Closed, ResolvedPublic4 Estimated Story PointsPRODUCTION ERROR

Description

Error
  • mwversion: 1.38.0-wmf.2
  • reqId: f92cae26-fd3c-4207-a348-93e6585783c5
normalized_message
Expectation (readQueryRows <=) 10000 by MediaWiki::main not met (actual: {actual}):
{query}
exception.trace
from /srv/mediawiki/php-1.38.0-wmf.2/includes/libs/rdbms/TransactionProfiler.php(444)
#0 /srv/mediawiki/php-1.38.0-wmf.2/includes/libs/rdbms/TransactionProfiler.php(272): Wikimedia\Rdbms\TransactionProfiler->reportExpectationViolated(string, Wikimedia\Rdbms\GeneralizedSql, integer)
#1 /srv/mediawiki/php-1.38.0-wmf.2/includes/libs/rdbms/database/Database.php(1516): Wikimedia\Rdbms\TransactionProfiler->recordQueryCompletion(Wikimedia\Rdbms\GeneralizedSql, double, boolean, integer)
#2 /srv/mediawiki/php-1.38.0-wmf.2/includes/libs/rdbms/database/Database.php(1398): Wikimedia\Rdbms\Database->executeQueryAttempt(string, string, boolean, string, integer)
#3 /srv/mediawiki/php-1.38.0-wmf.2/includes/libs/rdbms/database/Database.php(1323): Wikimedia\Rdbms\Database->executeQuery(string, string, integer)
#4 /srv/mediawiki/php-1.38.0-wmf.2/includes/libs/rdbms/database/Database.php(2012): Wikimedia\Rdbms\Database->query(string, string, integer)
#5 /srv/mediawiki/php-1.38.0-wmf.2/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->select(string, array, array, string)
#6 /srv/mediawiki/php-1.38.0-wmf.2/includes/libs/rdbms/database/DBConnRef.php(324): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#7 /srv/mediawiki/php-1.38.0-wmf.2/extensions/Translate/tag/TranslatablePage.php(461): Wikimedia\Rdbms\DBConnRef->select(string, array, array, string)
#8 /srv/mediawiki/php-1.38.0-wmf.2/extensions/Translate/tag/TranslatablePage.php(553): TranslatablePage->getTranslationPages()
#9 /srv/mediawiki/php-1.38.0-wmf.2/extensions/Translate/tag/PageTranslationHooks.php(1147): TranslatablePage->getTranslationPercentages()
#10 /srv/mediawiki/php-1.38.0-wmf.2/extensions/Translate/tag/PageTranslationHooks.php(1049): PageTranslationHooks::translationPageHeader(RequestContext, TranslatablePage)
#11 /srv/mediawiki/php-1.38.0-wmf.2/includes/HookContainer/HookContainer.php(338): PageTranslationHooks::translatablePageHeader(Article, NULL, boolean)
#12 /srv/mediawiki/php-1.38.0-wmf.2/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#13 /srv/mediawiki/php-1.38.0-wmf.2/includes/HookContainer/HookRunner.php(846): MediaWiki\HookContainer\HookContainer->run(string, array)
#14 /srv/mediawiki/php-1.38.0-wmf.2/includes/page/Article.php(571): MediaWiki\HookContainer\HookRunner->onArticleViewHeader(Article, NULL, boolean)
#15 /srv/mediawiki/php-1.38.0-wmf.2/includes/page/Article.php(509): Article->generateContentOutput(User, ParserOptions, integer, OutputPage, array)
#16 /srv/mediawiki/php-1.38.0-wmf.2/includes/actions/ViewAction.php(74): Article->view()
#17 /srv/mediawiki/php-1.38.0-wmf.2/includes/MediaWiki.php(538): ViewAction->show()
#18 /srv/mediawiki/php-1.38.0-wmf.2/includes/MediaWiki.php(320): MediaWiki->performAction(Article, Title)
#19 /srv/mediawiki/php-1.38.0-wmf.2/includes/MediaWiki.php(925): MediaWiki->performRequest()
#20 /srv/mediawiki/php-1.38.0-wmf.2/includes/MediaWiki.php(559): MediaWiki->main()
#21 /srv/mediawiki/php-1.38.0-wmf.2/index.php(53): MediaWiki->run()
#22 /srv/mediawiki/php-1.38.0-wmf.2/index.php(46): wfIndexMain()
#23 /srv/mediawiki/w/index.php(3): require(string)
#24 {main}
Impact

Rare case of an inefficient query, that slows down uncached page views.

Notes

The issue is TranslatablePage::getTranslationPages, which loads all subpages and then checks which of them are translation pages. A better way could be to use a LinkBatch for all known languages.

The example page in question has a lot of subpages: https://commons.wikimedia.org/wiki/Commons:Deletion_requests

Details

Request URL
https://commons.wikimedia.org/wiki/Commons:Deletion_requests

Event Timeline

Nikerabbit created this task.
Nikerabbit raised the priority of this task from Low to High.Oct 6 2021, 6:44 AM

This is actually one of the leading issues on our Logstash dashboard currently.

Nikerabbit set the point value for this task to 4.Feb 9 2022, 2:16 PM
Nikerabbit lowered the priority of this task from High to Medium.Mar 1 2022, 9:39 AM
abi_ added a subscriber: abi_.

As per discussion in the board run on 15th March, moving this to the next sprint to make time for other time sensitive tasks.

Change 798660 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Use link batch when fetching translation pages

https://gerrit.wikimedia.org/r/798660

Change 798660 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Use link batch when fetching translation pages

https://gerrit.wikimedia.org/r/798660

Unable to reproduce error after fix.

QA PASS

This warning is only visible in Logstash. The fix is going with the deployment train this week. I recommend to look at this Logstash query in a few days to confirm there is full or considerable reduction in there.

I did not notice any issues when moving translatable pages that uses the code that was modified. Will leave open to review Logstash to identify if the change has had the expected impact.