Page MenuHomePhabricator
Create Task
Maniphest T249533

testwikidata wiki is broken with "Cannot access the database"
Closed, ResolvedPublic
Actions

Description

See: https://archive.is/dun82

Cannot access the database: Cannot access the database: Unknown error (10.64.32.136)

Details

ProjectBranchLines +/-Subject
operations/mediawiki-configmaster+1 -1Follow-up 4d11e15ed0: Fix database name for repo in testwikidata
Customize query in gerrit

Related Objects
Search...

Event Timeline

Bugreporter created this task.Apr 6 2020, 6:51 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 6 2020, 6:51 PM
Bugreporter triaged this task as Unbreak Now! priority.Apr 6 2020, 6:52 PM
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptApr 6 2020, 6:52 PM
Bugreporter mentioned this in T249535: VE and Flow fail with "Error contacting the Parsoid/RESTBase server (HTTP 404)" / "…(HTTP 411)" on officewiki.Apr 6 2020, 7:11 PM
Framawiki renamed this task from Test Wikidata broken to testwikidata wiki is broken with "Cannot access the database".Apr 6 2020, 7:13 PM
Framawiki updated the task description. (Show Details)
Framawiki added a subscriber: Framawiki.Apr 6 2020, 7:16 PM
Lydia_Pintscher added a project: Wikidata-Campsite.Apr 6 2020, 7:43 PM
Lydia_Pintscher added a subscriber: Michael.
cscott added a subscriber: cscott.Apr 6 2020, 7:48 PM

T249535 seems not related. More likely the cause: the changes associated with 4d11e15ed098a0658719687557bb899c7cbf3711

Bugreporter mentioned this in T249565: Wikidata's wb_items_per_site table has suddenly disappeared, creating DBQueryErrors on page views.Apr 6 2020, 11:13 PM
Krinkle added a subscriber: Krinkle.EditedApr 7 2020, 1:06 AM

This started at 2020-04-06T14:51:00 and is still on-going.

https://logstash.wikimedia.org/goto/52e6eca628f5380e1dbb3b5783cd4300

Screenshot 2020-04-07 at 02.04.32.png (464×1 px, 70 KB)

one sample:

Cannot access the database: Unknown error (10.64.0.205)

#0 /srv/mediawiki/php-1.35.0-wmf.26/includes/libs/rdbms/loadbalancer/LoadBalancer.php(934): Wikimedia\Rdbms\LoadBalancer->reportConnectionError()
#1 /srv/mediawiki/php-1.35.0-wmf.26/includes/libs/rdbms/loadbalancer/LoadBalancer.php(901): Wikimedia\Rdbms\LoadBalancer->getServerConnection(integer, string, integer)
#2 /srv/mediawiki/php-1.35.0-wmf.26/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1031): Wikimedia\Rdbms\LoadBalancer->getConnection(integer, array, string, integer)
#3 /srv/mediawiki/php-1.35.0-wmf.26/includes/dao/DBAccessBase.php(63): Wikimedia\Rdbms\LoadBalancer->getConnectionRef(integer, array, string)
#4 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityMetaDataLookup.php(289): DBAccessBase->getConnection(integer)
#5 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityMetaDataLookup.php(89): Wikibase\Lib\Store\Sql\WikiPageEntityMetaDataLookup->selectRevisionInformationMultiple(array, integer)
#6 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/Sql/TypeDispatchingWikiPageEntityMetaDataAccessor.php(88): Wikibase\Lib\Store\Sql\WikiPageEntityMetaDataLookup->loadRevisionInformation(array, string)
#7 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/Sql/PrefetchingWikiPageEntityMetaDataAccessor.php(267): Wikibase\Lib\Store\Sql\TypeDispatchingWikiPageEntityMetaDataAccessor->loadRevisionInformation(array, string)
#8 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/Sql/PrefetchingWikiPageEntityMetaDataAccessor.php(189): Wikibase\Lib\Store\Sql\PrefetchingWikiPageEntityMetaDataAccessor->doFetch(string)
#9 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityRevisionLookup.php(178): Wikibase\Lib\Store\Sql\PrefetchingWikiPageEntityMetaDataAccessor->loadRevisionInformation(array, string)
#10 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/TypeDispatchingEntityRevisionLookup.php(69): Wikibase\Lib\Store\Sql\WikiPageEntityRevisionLookup->getLatestRevisionId(Wikibase\DataModel\Entity\ItemId, string)
#11 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/data-access/src/ByTypeDispatchingEntityRevisionLookup.php(64): Wikibase\Lib\Store\TypeDispatchingEntityRevisionLookup->getLatestRevisionId(Wikibase\DataModel\Entity\ItemId, string)
#12 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/TypeDispatchingEntityRevisionLookup.php(69): Wikibase\DataAccess\ByTypeDispatchingEntityRevisionLookup->getLatestRevisionId(Wikibase\DataModel\Entity\ItemId, string)
#13 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/CacheRetrievingEntityRevisionLookup.php(108): Wikibase\Lib\Store\TypeDispatchingEntityRevisionLookup->getLatestRevisionId(Wikibase\DataModel\Entity\ItemId, string)
#14 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/CachingEntityRevisionLookup.php(84): Wikibase\Lib\Store\CacheRetrievingEntityRevisionLookup->getEntityRevisionFromCache(Wikibase\DataModel\Entity\ItemId, integer, string)
#15 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/CachingEntityRevisionLookup.php(104): Wikibase\Lib\Store\CachingEntityRevisionLookup->getEntityRevision(Wikibase\DataModel\Entity\ItemId, integer, string)
#16 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/lib/includes/Store/CachingEntityRevisionLookup.php(87): Wikibase\Lib\Store\CachingEntityRevisionLookup->fetchEntityRevision(Wikibase\DataModel\Entity\ItemId, integer, string)
#17 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/repo/includes/Api/GetEntities.php(275): Wikibase\Lib\Store\CachingEntityRevisionLookup->getEntityRevision(Wikibase\DataModel\Entity\ItemId)
#18 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/repo/includes/Api/GetEntities.php(257): Wikibase\Repo\Api\GetEntities->getEntityRevision(Wikibase\DataModel\Entity\ItemId, boolean)
#19 /srv/mediawiki/php-1.35.0-wmf.26/extensions/Wikibase/repo/includes/Api/GetEntities.php(136): Wikibase\Repo\Api\GetEntities->getEntityRevisionsFromEntityIds(array, boolean)
#20 /srv/mediawiki/php-1.35.0-wmf.26/includes/api/ApiMain.php(1591): Wikibase\Repo\Api\GetEntities->execute()
#21 /srv/mediawiki/php-1.35.0-wmf.26/includes/api/ApiMain.php(523): ApiMain->executeAction()
#22 /srv/mediawiki/php-1.35.0-wmf.26/includes/api/ApiMain.php(494): ApiMain->executeActionWithErrorHandling()
#23 /srv/mediawiki/php-1.35.0-wmf.26/api.php(84): ApiMain->execute()
#24 /srv/mediawiki/w/api.php(3): require(string)
#25 {main}
  • reqId: 4d9f57d2-5293-43ae-8a83-7b368b1c6a0a

... and cross-referenced the reqId on type:mediawiki to find the DBConnection channel entry:

Error connecting to 10.64.0.205 as user wikiuser: :real_connect(): (HY000/1049): Unknown database 'testwikidata'
Krinkle added a project: Wikimedia-production-error.Apr 7 2020, 1:06 AM
Krinkle added a project: Wikimedia-Incident.
• Addshore added a subscriber: • Addshore.Apr 7 2020, 1:08 AM
Jdforrester-WMF added a parent task: T247774: 1.35.0-wmf.27 deployment blockers.Apr 7 2020, 1:12 AM
Ladsgroup claimed this task.Apr 7 2020, 4:07 AM
Ladsgroup edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed Wikidata-Campsite.
Ladsgroup added a subscriber: Ladsgroup.

This is caused by entity source work: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/586366

I fix it now.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptApr 7 2020, 4:07 AM
gerritbot added a comment.Apr 7 2020, 4:11 AM

Change 586488 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Fix database name for repo in testwikidata

https://gerrit.wikimedia.org/r/586488

gerritbot added a project: Patch-For-Review.Apr 7 2020, 4:11 AM
Maintenance_bot moved this task from incoming to in progress on the Wikidata board.Apr 7 2020, 4:15 AM
Maintenance_bot moved this task from Incoming to In progress on the User-Ladsgroup board.
Ladsgroup moved this task from To Do (prioritised from top to bottom) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Apr 7 2020, 4:28 AM
gerritbot added a comment.Apr 7 2020, 5:31 AM

Change 586488 merged by jenkins-bot:
[operations/mediawiki-config@master] Follow-up 4d11e15ed0: Fix database name for repo in testwikidata

https://gerrit.wikimedia.org/r/586488

Stashbot added a comment.Apr 7 2020, 5:37 AM

Mentioned in SAL (#wikimedia-operations) [2020-04-07T05:37:03Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:586488|Fix database name for repo in testwikidata (T249533)]] (duration: 01m 00s)

Ladsgroup closed this task as Resolved.Apr 7 2020, 5:38 AM
Ladsgroup moved this task from Peer Review to Done on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
Stashbot added a comment.Apr 7 2020, 5:38 AM

Mentioned in SAL (#wikimedia-operations) [2020-04-07T05:38:21Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:586488|Fix database name for repo in testwikidata (T249533)]], take II (duration: 00m 58s)

WMDE-leszek added a subscriber: WMDE-leszek.Apr 7 2020, 5:45 AM
Maintenance_bot removed a project: Patch-For-Review.Apr 7 2020, 6:10 AM
Maintenance_bot moved this task from In progress to Done on the User-Ladsgroup board.Apr 7 2020, 6:15 AM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL