Page MenuHomePhabricator
Create Task
Maniphest T267316

FileImporter throws backend errors on Special:SpecialPages
Closed, ResolvedPublic2 Estimated Story Points
Actions

Description

Special:ImportFile is constructed on the SpecialPages index, throwing the following error deep in its factories. It should never be constructed. This happens c. 1000 times per day, so causing significant log spam and resource use.

#0 /srv/mediawiki/php-1.36.0-wmf.14/includes/libs/rdbms/TransactionProfiler.php(189): Wikimedia\Rdbms\TransactionProfiler->reportExpectationViolated(string, string, integer)
#1 /srv/mediawiki/php-1.36.0-wmf.14/includes/libs/rdbms/loadbalancer/LoadBalancer.php(943): Wikimedia\Rdbms\TransactionProfiler->recordConnection(string, string, boolean)
#2 /srv/mediawiki/php-1.36.0-wmf.14/includes/libs/rdbms/loadbalancer/LoadBalancer.php(899): Wikimedia\Rdbms\LoadBalancer->getServerConnection(integer, string, integer)
#3 /srv/mediawiki/php-1.36.0-wmf.14/extensions/FileImporter/src/ServiceWiring.php(134): Wikimedia\Rdbms\LoadBalancer->getConnection(integer)
#4 /srv/mediawiki/php-1.36.0-wmf.14/vendor/wikimedia/services/src/ServiceContainer.php(447): Wikimedia\Services\ServiceContainer->FileImporter\{closure}(MediaWiki\MediaWikiServices)
#5 /srv/mediawiki/php-1.36.0-wmf.14/vendor/wikimedia/services/src/ServiceContainer.php(416): Wikimedia\Services\ServiceContainer->createService(string)
#6 /srv/mediawiki/php-1.36.0-wmf.14/extensions/FileImporter/src/ServiceWiring.php(89): Wikimedia\Services\ServiceContainer->getService(string)
#7 /srv/mediawiki/php-1.36.0-wmf.14/vendor/wikimedia/services/src/ServiceContainer.php(447): Wikimedia\Services\ServiceContainer->FileImporter\{closure}(MediaWiki\MediaWikiServices)
#8 /srv/mediawiki/php-1.36.0-wmf.14/vendor/wikimedia/services/src/ServiceContainer.php(416): Wikimedia\Services\ServiceContainer->createService(string)
#9 /srv/mediawiki/php-1.36.0-wmf.14/extensions/FileImporter/src/SpecialImportFile.php(95): Wikimedia\Services\ServiceContainer->getService(string)
#10 /srv/mediawiki/php-1.36.0-wmf.14/vendor/wikimedia/object-factory/src/ObjectFactory.php(184): FileImporter\SpecialImportFile->__construct()
#11 /srv/mediawiki/php-1.36.0-wmf.14/vendor/wikimedia/object-factory/src/ObjectFactory.php(102): Wikimedia\ObjectFactory::getObjectFromSpec(array, array)
#12 /srv/mediawiki/php-1.36.0-wmf.14/includes/specialpage/SpecialPageFactory.php(853): Wikimedia\ObjectFactory->createObject(string, array)
#13 /srv/mediawiki/php-1.36.0-wmf.14/includes/specialpage/SpecialPageFactory.php(885): MediaWiki\SpecialPage\SpecialPageFactory->getPage(string)
#14 /srv/mediawiki/php-1.36.0-wmf.14/includes/specials/SpecialSpecialpages.php(62): MediaWiki\SpecialPage\SpecialPageFactory->getUsablePages(User)
#15 /srv/mediawiki/php-1.36.0-wmf.14/includes/specials/SpecialSpecialpages.php(51): SpecialSpecialpages->getPageGroups()
#16 /srv/mediawiki/php-1.36.0-wmf.14/includes/specialpage/SpecialPage.php(600): SpecialSpecialpages->execute(NULL)
#17 /srv/mediawiki/php-1.36.0-wmf.14/includes/specialpage/SpecialPageFactory.php(1018): SpecialPage->run(NULL)
#18 /srv/mediawiki/php-1.36.0-wmf.14/includes/MediaWiki.php(307): MediaWiki\SpecialPage\SpecialPageFactory->executePath(Title, RequestContext)
#19 /srv/mediawiki/php-1.36.0-wmf.14/includes/MediaWiki.php(940): MediaWiki->performRequest()
#20 /srv/mediawiki/php-1.36.0-wmf.14/includes/MediaWiki.php(543): MediaWiki->main()
#21 /srv/mediawiki/php-1.36.0-wmf.14/index.php(53): MediaWiki->run()
#22 /srv/mediawiki/php-1.36.0-wmf.14/index.php(46): wfIndexMain()
#23 /srv/mediawiki/w/index.php(3): require(string)
#24 {main}

Discovered during T243220: Create logstash monitoring dashboard for Technical Wishes products.

Details

SubjectRepoBranchLines +/-
NullRevisionCreator: connect to DB only when neededmediawiki/extensions/FileImportermaster+30 -19
Customize query in gerrit

Related Objects

Event Timeline

awight created this task.Nov 5 2020, 1:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 5 2020, 1:18 PM
awight updated the task description. (Show Details)Nov 5 2020, 1:18 PM
awight removed a parent task: T243220: Create logstash monitoring dashboard for Technical Wishes products.
awight updated the task description. (Show Details)
awight moved this task from Proposed to Maintenance Backlog on the WMDE-TechWish board.Nov 6 2020, 9:03 AM
awight added a project: MediaWiki-extensions-Maintenance (by Skizzerz).
thiemowmde removed a project: MediaWiki-extensions-Maintenance (by Skizzerz).Nov 6 2020, 9:05 AM
awight added a project: Wikimedia-production-error.Nov 6 2020, 12:46 PM
Umherirrender subscribed.Nov 6 2020, 11:41 PM

Service NullRevisionCreator is using a database connection to DB_MASTER, pass just the load balancer and instance the connection when needed? Special:SpecialPages is supposed to be "read only" and not touching the master

gerritbot added a comment.Nov 10 2020, 11:34 AM

Change 640395 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] NullRevisionCreator: connect to DB only when needed

https://gerrit.wikimedia.org/r/640395

gerritbot added a project: Patch-For-Review.Nov 10 2020, 11:34 AM
thiemowmde claimed this task.Nov 10 2020, 11:36 AM
thiemowmde moved this task from Maintenance Backlog to Currently in sprint on the WMDE-TechWish board.
thiemowmde set the point value for this task to 2.
thiemowmde added projects: Unplanned-Sprint-Work, WMDE-QWERTY-Sprint-2020-11-04.
thiemowmde moved this task from Sprint Backlog to Review on the WMDE-QWERTY-Sprint-2020-11-04 board.
gerritbot added a comment.Nov 10 2020, 11:59 AM

Change 640395 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] NullRevisionCreator: connect to DB only when needed

https://gerrit.wikimedia.org/r/640395

Maintenance_bot removed a project: Patch-For-Review.Nov 10 2020, 12:10 PM
thiemowmde mentioned this in rEFLI702451b8f82d: NullRevisionCreator: connect to DB only when needed.Nov 10 2020, 12:23 PM
ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.18; 2020-11-17).Nov 10 2020, 1:00 PM
awight moved this task from Currently in sprint to Watching/Blocked/External on the WMDE-TechWish board.Nov 10 2020, 1:02 PM

This is currently scheduled for deployment the week of November 17th. Let's monitor the logs and then close.

thiemowmde moved this task from Review to Demo on the WMDE-QWERTY-Sprint-2020-11-04 board.Nov 11 2020, 11:38 AM
thiemowmde closed this task as Resolved.Nov 17 2020, 1:09 PM
awight reopened this task as Open.EditedNov 18 2020, 9:50 AM

I don't understand why this was closed, there are still errors as of today (Nov 18th):

image.png (153×872 px, 24 KB)

#0 /srv/mediawiki/php-1.36.0-wmf.16/includes/libs/rdbms/TransactionProfiler.php(189): Wikimedia\Rdbms\TransactionProfiler->reportExpectationViolated(string, string, integer)
#1 /srv/mediawiki/php-1.36.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(943): Wikimedia\Rdbms\TransactionProfiler->recordConnection(string, string, boolean)
#2 /srv/mediawiki/php-1.36.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(899): Wikimedia\Rdbms\LoadBalancer->getServerConnection(integer, string, integer)
#3 /srv/mediawiki/php-1.36.0-wmf.16/extensions/FileImporter/src/ServiceWiring.php(134): Wikimedia\Rdbms\LoadBalancer->getConnection(integer)
#4 /srv/mediawiki/php-1.36.0-wmf.16/vendor/wikimedia/services/src/ServiceContainer.php(447): Wikimedia\Services\ServiceContainer->FileImporter\{closure}(MediaWiki\MediaWikiServices)
#5 /srv/mediawiki/php-1.36.0-wmf.16/vendor/wikimedia/services/src/ServiceContainer.php(416): Wikimedia\Services\ServiceContainer->createService(string)
#6 /srv/mediawiki/php-1.36.0-wmf.16/extensions/FileImporter/src/ServiceWiring.php(89): Wikimedia\Services\ServiceContainer->getService(string)
#7 /srv/mediawiki/php-1.36.0-wmf.16/vendor/wikimedia/services/src/ServiceContainer.php(447): Wikimedia\Services\ServiceContainer->FileImporter\{closure}(MediaWiki\MediaWikiServices)
#8 /srv/mediawiki/php-1.36.0-wmf.16/vendor/wikimedia/services/src/ServiceContainer.php(416): Wikimedia\Services\ServiceContainer->createService(string)
#9 /srv/mediawiki/php-1.36.0-wmf.16/extensions/FileImporter/src/SpecialImportFile.php(95): Wikimedia\Services\ServiceContainer->getService(string)
#10 /srv/mediawiki/php-1.36.0-wmf.16/vendor/wikimedia/object-factory/src/ObjectFactory.php(184): FileImporter\SpecialImportFile->__construct()
#11 /srv/mediawiki/php-1.36.0-wmf.16/vendor/wikimedia/object-factory/src/ObjectFactory.php(102): Wikimedia\ObjectFactory::getObjectFromSpec(array, array)
#12 /srv/mediawiki/php-1.36.0-wmf.16/includes/specialpage/SpecialPageFactory.php(1002): Wikimedia\ObjectFactory->createObject(string, array)
#13 /srv/mediawiki/php-1.36.0-wmf.16/includes/specialpage/SpecialPageFactory.php(1035): MediaWiki\SpecialPage\SpecialPageFactory->getPage(string)
#14 /srv/mediawiki/php-1.36.0-wmf.16/includes/specials/SpecialSpecialpages.php(62): MediaWiki\SpecialPage\SpecialPageFactory->getUsablePages(User)
#15 /srv/mediawiki/php-1.36.0-wmf.16/includes/specials/SpecialSpecialpages.php(51): SpecialSpecialpages->getPageGroups()
#16 /srv/mediawiki/php-1.36.0-wmf.16/includes/specialpage/SpecialPage.php(607): SpecialSpecialpages->execute(NULL)
#17 /srv/mediawiki/php-1.36.0-wmf.16/includes/specialpage/SpecialPageFactory.php(1168): SpecialPage->run(NULL)
#18 /srv/mediawiki/php-1.36.0-wmf.16/includes/MediaWiki.php(310): MediaWiki\SpecialPage\SpecialPageFactory->executePath(Title, RequestContext)
#19 /srv/mediawiki/php-1.36.0-wmf.16/includes/MediaWiki.php(943): MediaWiki->performRequest()
#20 /srv/mediawiki/php-1.36.0-wmf.16/includes/MediaWiki.php(546): MediaWiki->main()
#21 /srv/mediawiki/php-1.36.0-wmf.16/index.php(53): MediaWiki->run()
#22 /srv/mediawiki/php-1.36.0-wmf.16/index.php(46): wfIndexMain()
#23 /srv/mediawiki/w/index.php(3): require(string)
#24 {main}

The fix hasn't been deployed yet, it's in MW-1.36-notes (1.36.0-wmf.18; 2020-11-17)

thiemowmde closed this task as Resolved.Nov 18 2020, 1:37 PM
WMDE-Fisch moved this task from Demo to Done on the WMDE-QWERTY-Sprint-2020-11-04 board.Nov 18 2020, 1:56 PM
Krinkle moved this task from Untriaged to Nov 2020 on the Wikimedia-production-error board.Jul 29 2022, 11:19 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits