Page MenuHomePhabricator
Create Task
Maniphest T241057

Cannot access the database: No working replica DB server: Unknown error (10.64.32.113)
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

MediaWiki version: 1.35.0-wmf.11

message
Cannot access the database: No working replica DB server: Unknown error (10.64.32.113)

Impact

Hundreds of errors a minute. Structured data edits seemed to be broken. Train blocker.

Notes

Details

Request ID
XfolhQpAADoAAD-ybd8AAAAX
Request URL
https://commons.wikimedia.org/w/index.php?title=Image:H%C3%B4tel_de_ville_de_Pont-%C3%A0-Mousson-7814.jpg&action=render&uselang=en
Stack Trace
exception.trace
#0 /srv/mediawiki/php-1.35.0-wmf.11/includes/libs/rdbms/loadbalancer/LoadBalancer.php(469): Wikimedia\Rdbms\LoadBalancer->reportConnectionError()
#1 /srv/mediawiki/php-1.35.0-wmf.11/includes/libs/rdbms/loadbalancer/LoadBalancer.php(897): Wikimedia\Rdbms\LoadBalancer->getConnectionIndex(integer, array, string)
#2 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(645): Wikimedia\Rdbms\LoadBalancer->getConnection(integer)
#3 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(688): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->getPageInfoForType(string)
#4 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(759): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->getPageInfo()
#5 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(259): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->findRedirects()
#6 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseEntityInfoBuilder.php(808): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->resolveRedirects()
#7 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/data-access/src/ByTypeDispatchingEntityInfoBuilder.php(42): Wikibase\Lib\Store\Sql\Terms\DatabaseEntityInfoBuilder->collectEntityInfo(array, array)
#8 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/data-access/src/ByTypeDispatchingEntityInfoBuilder.php(42): Wikibase\DataAccess\ByTypeDispatchingEntityInfoBuilder->collectEntityInfo(array, array)
#9 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/ParserOutput/FullEntityParserOutputGenerator.php(211): Wikibase\DataAccess\ByTypeDispatchingEntityInfoBuilder->collectEntityInfo(array, array)
#10 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/ParserOutput/FullEntityParserOutputGenerator.php(160): Wikibase\Repo\ParserOutput\FullEntityParserOutputGenerator->getEntityInfo(ParserOutput)
#11 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/ParserOutput/StatsdTimeRecordingEntityParserOutputGenerator.php(48): Wikibase\Repo\ParserOutput\FullEntityParserOutputGenerator->getParserOutput(Wikibase\Lib\Store\EntityRevision, boolean)
#12 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/Content/EntityContent.php(288): Wikibase\Repo\ParserOutput\StatsdTimeRecordingEntityParserOutputGenerator->getParserOutput(Wikibase\Lib\Store\EntityRevision, boolean)
#13 /srv/mediawiki/php-1.35.0-wmf.11/extensions/Wikibase/repo/includes/Content/EntityContent.php(227): Wikibase\EntityContent->getParserOutputFromEntityView(integer, ParserOptions, boolean)
#14 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RenderedRevision.php(267): Wikibase\EntityContent->getParserOutput(Title, integer, ParserOptions, boolean)
#15 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RenderedRevision.php(236): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(Wikibase\MediaInfo\Content\MediaInfoContent, boolean)
#16 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RevisionRenderer.php(230): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
#17 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RevisionRenderer.php(152): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#18 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#19 /srv/mediawiki/php-1.35.0-wmf.11/includes/Revision/RenderedRevision.php(198): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#20 /srv/mediawiki/php-1.35.0-wmf.11/includes/poolcounter/PoolWorkArticleView.php(196): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#21 /srv/mediawiki/php-1.35.0-wmf.11/includes/poolcounter/PoolCounterWork.php(125): PoolWorkArticleView->doWork()
#22 /srv/mediawiki/php-1.35.0-wmf.11/includes/page/Article.php(794): PoolCounterWork->execute()
#23 /srv/mediawiki/php-1.35.0-wmf.11/includes/page/ImagePage.php(94): Article->view()
#24 /srv/mediawiki/php-1.35.0-wmf.11/includes/page/Article.php(1776): ImagePage->view()
#25 /srv/mediawiki/php-1.35.0-wmf.11/includes/actions/RenderAction.php(39): Article->render()
#26 /srv/mediawiki/php-1.35.0-wmf.11/includes/MediaWiki.php(514): RenderAction->show()
#27 /srv/mediawiki/php-1.35.0-wmf.11/includes/MediaWiki.php(304): MediaWiki->performAction(ImagePage, Title)
#28 /srv/mediawiki/php-1.35.0-wmf.11/includes/MediaWiki.php(967): MediaWiki->performRequest()
#29 /srv/mediawiki/php-1.35.0-wmf.11/includes/MediaWiki.php(530): MediaWiki->main()
#30 /srv/mediawiki/php-1.35.0-wmf.11/index.php(46): MediaWiki->run()
#31 /srv/mediawiki/w/index.php(3): require(string)
#32 {main}
SubjectRepoBranchLines +/-
train: Roll Commons forward to 1.35.0-wmf.11 againoperations/mediawiki-configmaster+1 -1
Fix DatabaseEntityInfoBuilder on federated reposmediawiki/extensions/Wikibasewmf/1.35.0-wmf.11+17 -5
Fix DatabaseEntityInfoBuilder on federated reposmediawiki/extensions/Wikibasemaster+17 -5
Customize query in gerrit

Related Objects
Search...

Event Timeline

Jdforrester-WMF created this task.Dec 18 2019, 1:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 18 2019, 1:14 PM
Jdforrester-WMF triaged this task as Unbreak Now! priority.Dec 18 2019, 1:14 PM
Jdforrester-WMF added a parent task: T233859: 1.35.0-wmf.11 deployment blockers.
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptDec 18 2019, 1:14 PM
Addshore subscribed.Dec 18 2019, 1:16 PM
Ladsgroup claimed this task.Dec 18 2019, 1:17 PM
Ladsgroup edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed WikibaseMediaInfo.
Ladsgroup subscribed.

This is caused by yours truly as part of T236681: SqlEntityInfoBuilder reads and writes from the old term store regardless of the config

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptDec 18 2019, 1:17 PM
Ladsgroup added a comment.Dec 18 2019, 1:25 PM

It's not possible to reproduce it in beta cluster or test commons as they both reside in the same host, it might break other client wikis soon.

gerritbot added a comment.Dec 18 2019, 1:35 PM

Change 559057 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Fix DatabaseEntityInfoBuilder on federated repos

https://gerrit.wikimedia.org/r/559057

gerritbot added a project: Patch-For-Review.Dec 18 2019, 1:35 PM
Ladsgroup moved this task from To Do (prioritised from top to bottom) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Dec 18 2019, 1:38 PM
gerritbot added a comment.Dec 18 2019, 1:59 PM

Change 559064 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.11] Fix DatabaseEntityInfoBuilder on federated repos

https://gerrit.wikimedia.org/r/559064

gerritbot added a comment.Dec 18 2019, 2:09 PM

Change 559057 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Fix DatabaseEntityInfoBuilder on federated repos

https://gerrit.wikimedia.org/r/559057

gerritbot added a comment.Dec 18 2019, 2:34 PM

Change 559064 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.11] Fix DatabaseEntityInfoBuilder on federated repos

https://gerrit.wikimedia.org/r/559064

gerritbot added a comment.Dec 18 2019, 2:44 PM

Change 559089 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[operations/mediawiki-config@master] train: Roll Commons forward to 1.35.0-wmf.11 again

https://gerrit.wikimedia.org/r/559089

Stashbot added a comment.Dec 18 2019, 2:45 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-18T14:45:23Z] <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.11/extensions/Wikibase: [[gerrit:559064|Fix DatabaseEntityInfoBuilder on federated repos (T241057)]] (duration: 01m 07s)

gerritbot added a comment.Dec 18 2019, 2:47 PM

Change 559089 merged by jenkins-bot:
[operations/mediawiki-config@master] train: Roll Commons forward to 1.35.0-wmf.11 again

https://gerrit.wikimedia.org/r/559089

Ladsgroup moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Dec 18 2019, 2:48 PM

works now: https://test-commons.wikimedia.org/wiki/File:1024px-Spotted_fritillary_(Melitaea_didyma)_underside_Macedonia.jpg

Stashbot mentioned this in T233859: 1.35.0-wmf.11 deployment blockers.Dec 18 2019, 2:49 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-18T14:49:54Z] <jforrester@deploy1001> rebuilt and synchronized wikiversions files: train: Rolling Commons foward to 1.35.0-wmf.11 T233859 T241057

Jdforrester-WMF closed this task as Resolved.Dec 18 2019, 2:53 PM

Thank you!

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.11; 2019-12-17).Dec 18 2019, 3:00 PM
Maintenance_bot removed a project: Patch-For-Review.Dec 18 2019, 3:10 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL