Page MenuHomePhabricator
Create Task
Maniphest T241057

Cannot access the database: No working replica DB server: Unknown error (10.64.32.113)
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

MediaWiki version: 1.35.0-wmf.11

message
Cannot access the database: No working replica DB server: Unknown error (10.64.32.113)

Impact

Hundreds of errors a minute. Structured data edits seemed to be broken. Train blocker.

Notes

Details

Request ID
XfolhQpAADoAAD-ybd8AAAAX
Request URL
https://commons.wikimedia.org/w/index.php?title=Image:H%C3%B4tel_de_ville_de_Pont-%C3%A0-Mousson-7814.jpg&action=render&uselang=en
Stack Trace
exception.trace
#0 /srv/mediawiki/php-1.35.0-wmf.11/includes/libs/rdbms/loadbalancer/LoadBalancer.php(469): Wikimedia\Rdbms\LoadBalancer->reportConnectionError()
#1 /srv/mediawiki/php-1.35.0-wmf.11/includes/libs/rdbms/loadbalancer/LoadBalancer.php(897): Wikimedia\Rdbms\LoadBalancer->getConnectionIndex(integer, array, string)
#2 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(645): Wikimedia\Rdbms\LoadBalancer->getConnection(integer)
#3 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(688): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->getPageInfoForType(string)
#4 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(759): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->getPageInfo()
#5 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(259): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->findRedirects()
#6 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(808): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->resolveRedirects()
#7 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/data-access/src/ByTypeDispatchingEntityInfoBuilder.php(42): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->collectEntityInfo(array, array)
#8 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/data-access/src/ByTypeDispatchingEntityInfoBuilder.php(42): Wikibase\DataAccess\ByTypeDispatchingEntityInfoBuilder->collectEntityInfo(array, array)
#9 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/ParserOutput/FullEntityParserOutputGenerator.php(211): Wikibase\DataAccess\ByTypeDispatchingEntityInfoBuilder->collectEntityInfo(array, array)
#10 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/ParserOutput/FullEntityParserOutputGenerator.php(160): Wikibase\Repo\ParserOutput\FullEntityParserOutputGenerator->getEntityInfo(ParserOutput)
#11 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/ParserOutput/StatsdTimeRecordingEntityParserOutputGenerator.php(48): Wikibase\Repo\ParserOutput\FullEntityParserOutputGenerator->getParserOutput(Wikibase\Lib\Store\EntityRevision, boolean)
#12 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/Content/EntityContent.php(288): Wikibase\Repo\ParserOutput\StatsdTimeRecordingEntityParserOutputGenerator->getParserOutput(Wikibase\Lib\Store\EntityRevision, boolean)
#13 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/Content/EntityContent.php(227): Wikibase\EntityContent->getParserOutputFromEntityView(integer, ParserOptions, boolean)
#14 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RenderedRevision.php(267): Wikibase\EntityContent->getParserOutput(Title, integer, ParserOptions, boolean)
#15 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RenderedRevision.php(236): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(Wikibase\MediaInfo\Content\MediaInfoContent, boolean)
#16 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RevisionRenderer.php(230): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
#17 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RevisionRenderer.php(152): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#18 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#19 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RenderedRevision.php(198): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#20 /srv/mediawiki/php-1.35.0-wmf.11/includes/poolcounter/PoolWorkArticleView.php(196): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#21 /srv/mediawiki/php-1.35.0-wmf.11/includes/poolcounter/PoolCounterWork.php(125): PoolWorkArticleView->doWork()
#22 /srv/mediawiki/php-1.35.0-wmf.11/includes/page/Article.php(794): PoolCounterWork->execute()
#23 /srv/mediawiki/php-1.35.0-wmf.11/includes/page/ImagePage.php(94): Article->view()
#24 /srv/mediawiki/php-1.35.0-wmf.11/includes/page/Article.php(1776): ImagePage->view()
#25 /srv/mediawiki/php-1.35.0-wmf.11/includes/actions/RenderAction.php(39): Article->render()
#26 /srv/mediawiki/php-1.35.0-wmf.11/includes/MediaWiki.php(514): RenderAction->show()
#27 /srv/mediawiki/php-1.35.0-wmf.11/includes/MediaWiki.php(304): MediaWiki->performAction(ImagePage, Title)
#28 /srv/mediawiki/php-1.35.0-wmf.11/includes/MediaWiki.php(967): MediaWiki->performRequest()
#29 /srv/mediawiki/php-1.35.0-wmf.11/includes/MediaWiki.php(530): MediaWiki->main()
#30 /srv/mediawiki/php-1.35.0-wmf.11/index.php(46): MediaWiki->run()
#31 /srv/mediawiki/w/index.php(3): require(string)
#32 {main}
ProjectBranchLines +/-Subject
operations/mediawiki-configmaster+1 -1train: Roll Commons forward to 1.35.0-wmf.11 again
mediawiki/extensions/Wikibasewmf/1.35.0-wmf.11+17 -5Fix DatabaseEntityInfoBuilder on federated repos
mediawiki/extensions/Wikibasemaster+17 -5Fix DatabaseEntityInfoBuilder on federated repos
Customize query in gerrit

Related Objects
Search...

Event Timeline

Jdforrester-WMF created this task.Dec 18 2019, 1:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 18 2019, 1:14 PM
Jdforrester-WMF triaged this task as Unbreak Now! priority.Dec 18 2019, 1:14 PM
Jdforrester-WMF added a parent task: T233859: 1.35.0-wmf.11 deployment blockers.
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptDec 18 2019, 1:14 PM
Addshore added a subscriber: Addshore.Dec 18 2019, 1:16 PM
• Ladsgroup claimed this task.Dec 18 2019, 1:17 PM
• Ladsgroup edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed WikibaseMediaInfo.
• Ladsgroup added a subscriber: • Ladsgroup.

This is caused by yours truly as part of T236681: SqlEntityInfoBuilder reads and writes from the old term store regardless of the config

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptDec 18 2019, 1:17 PM
• Ladsgroup added a comment.Dec 18 2019, 1:25 PM

It's not possible to reproduce it in beta cluster or test commons as they both reside in the same host, it might break other client wikis soon.

gerritbot added a comment.Dec 18 2019, 1:35 PM

Change 559057 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Fix DatabaseEntityInfoBuilder on federated repos

https://gerrit.wikimedia.org/r/559057

gerritbot added a project: Patch-For-Review.Dec 18 2019, 1:35 PM
• Ladsgroup moved this task from To Do (prioritised from top to bottom) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Dec 18 2019, 1:38 PM
gerritbot added a comment.Dec 18 2019, 1:59 PM

Change 559064 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.11] Fix DatabaseEntityInfoBuilder on federated repos

https://gerrit.wikimedia.org/r/559064

gerritbot added a comment.Dec 18 2019, 2:09 PM

Change 559057 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Fix DatabaseEntityInfoBuilder on federated repos

https://gerrit.wikimedia.org/r/559057

gerritbot added a comment.Dec 18 2019, 2:34 PM

Change 559064 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.11] Fix DatabaseEntityInfoBuilder on federated repos

https://gerrit.wikimedia.org/r/559064

gerritbot added a comment.Dec 18 2019, 2:44 PM

Change 559089 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[operations/mediawiki-config@master] train: Roll Commons forward to 1.35.0-wmf.11 again

https://gerrit.wikimedia.org/r/559089

Stashbot added a comment.Dec 18 2019, 2:45 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-18T14:45:23Z] <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.11/extensions/Wikibase: [[gerrit:559064|Fix DatabaseEntityInfoBuilder on federated repos (T241057)]] (duration: 01m 07s)

gerritbot added a comment.Dec 18 2019, 2:47 PM

Change 559089 merged by jenkins-bot:
[operations/mediawiki-config@master] train: Roll Commons forward to 1.35.0-wmf.11 again

https://gerrit.wikimedia.org/r/559089

• Ladsgroup moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Dec 18 2019, 2:48 PM

works now: https://test-commons.wikimedia.org/wiki/File:1024px-Spotted_fritillary_(Melitaea_didyma)_underside_Macedonia.jpg

Stashbot mentioned this in T233859: 1.35.0-wmf.11 deployment blockers.Dec 18 2019, 2:49 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-18T14:49:54Z] <jforrester@deploy1001> rebuilt and synchronized wikiversions files: train: Rolling Commons foward to 1.35.0-wmf.11 T233859 T241057

Jdforrester-WMF closed this task as Resolved.Dec 18 2019, 2:53 PM

Thank you!

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.11; 2019-12-17).Dec 18 2019, 3:00 PM
Maintenance_bot removed a project: Patch-For-Review.Dec 18 2019, 3:10 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL