Page MenuHomePhabricator
Create Task
Maniphest T230065

DBQueryError "Commands out of sync" from Rdbms\Database::close
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

Request ID: XUsofwpAIDAAAHn6l7kAAABD
Request URL: /wiki/User:ImageResizeBot/500000-full

message
PHP Warning: Destructor threw an object exception: exception 'Wikimedia\Rdbms\DBQueryError' with message 'A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
Query: ROLLBACK
Function: Wikimedia\Rdbms\Database::close
Error: 2014 Commands out of sync; you can't run this command now (10.64.48.11:3314)
' in /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php:1596
trace
Stack trace:
#0 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(1567): Wikimedia\Rdbms\Database->getQueryExceptionAndLog()
#1 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(1156): Wikimedia\Rdbms\Database->reportQueryError()
#2 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(4178): Wikimedia\Rdbms\Database->query()
#3 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(4127): Wikimedia\Rdbms\Database->doRollback()
#4 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(917): Wikimedia\Rdbms\Database->rollback()
#5 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1505): Wikimedia\Rdbms\Database->close()
#6 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(2009): Closure$Wikimedia\Rdbms\LoadBalancer::closeAll()
#7 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1506): Wikimedia\Rdbms\LoadBalancer->forEachOpenConnection()
#8 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1495): Wikimedia\Rdbms\LoadBalancer->closeAll()
#9 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactory.php(230): Wikimedia\Rdbms\LoadBalancer->disable()
#10 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactoryMulti.php(368): Closure$Wikimedia\Rdbms\LBFactory::forEachLBCallMethod()
#11 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactory.php(233): Wikimedia\Rdbms\LBFactoryMulti->forEachLB()
#12 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactory.php(164): Wikimedia\Rdbms\LBFactory->forEachLBCallMethod()
#13 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactory.php(758): Wikimedia\Rdbms\LBFactory->destroy()
#14 (): Wikimedia\Rdbms\LBFactory->__destruct()
#15 {main}
Impact

Unclear.

Notes

Cropped up in mediawiki-new-errors shortly after rolling wmf.17 forward to group1; presumably unrelated since it's in wmf.16.

Likely to be transient?

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
rdbms: make LBFactory close/rollback dangling handles like LoadBalancermediawiki/coremaster+0 -1
Customize query in gerrit

Related Objects

Event Timeline

brennen created this task.Aug 7 2019, 7:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 7 2019, 7:53 PM
Reedy added a project: MediaWiki-libs-Rdbms.Aug 7 2019, 9:01 PM
brennen added a comment.Aug 7 2019, 11:08 PM

I'm assuming related:

Error

Request ID: XUtW3gpAIDwAAK3cxFAAAACT
Request URL: https://de.wikipedia.org/w/api.php (POST)

message
[XUtW3gpAIDwAAK3cxFAAAACT] /w/api.php   ErrorException from line 46 of /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/DatabaseMysqli.php: PHP Warning: mysqli::query(): Error reading result set's header
trace
#0 [internal function]: MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/DatabaseMysqli.php(46): mysqli->query(string)
#2 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(1298): Wikimedia\Rdbms\DatabaseMysqli->doQuery(string)
#3 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(1215): Wikimedia\Rdbms\Database->executeQueryAttempt(string, string, boolean, string, integer)
#4 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(1152): Wikimedia\Rdbms\Database->executeQuery(string, string, integer)
#5 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(4178): Wikimedia\Rdbms\Database->query(string, string, integer)
#6 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(4127): Wikimedia\Rdbms\Database->doRollback(string)
#7 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/database/Database.php(917): Wikimedia\Rdbms\Database->rollback(string, string)
#8 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1505): Wikimedia\Rdbms\Database->close()
#9 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(2009): Wikimedia\Rdbms\LoadBalancer->Wikimedia\Rdbms\{closure}(Wikimedia\Rdbms\DatabaseMysqli)
#10 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1506): Wikimedia\Rdbms\LoadBalancer->forEachOpenConnection(Closure)
#11 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1495): Wikimedia\Rdbms\LoadBalancer->closeAll()
#12 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactory.php(230): Wikimedia\Rdbms\LoadBalancer->disable()
#13 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactoryMulti.php(368): Wikimedia\Rdbms\LBFactory->Wikimedia\Rdbms\{closure}(Wikimedia\Rdbms\LoadBalancer, string, array)
#14 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactory.php(232): Wikimedia\Rdbms\LBFactoryMulti->forEachLB(Closure, array)
#15 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactory.php(164): Wikimedia\Rdbms\LBFactory->forEachLBCallMethod(string)
#16 /srv/mediawiki/php-1.34.0-wmf.16/includes/libs/rdbms/lbfactory/LBFactory.php(758): Wikimedia\Rdbms\LBFactory->destroy()
#17 [internal function]: Wikimedia\Rdbms\LBFactory->__destruct()
#18 {main}
• mobrovac triaged this task as Medium priority.Aug 8 2019, 9:38 AM
• mobrovac edited projects, added Platform Team Workboards (Clinic Duty Team), Performance-Team; removed Platform Engineering.
• mobrovac added a subscriber: aaron.
aaron claimed this task.Aug 12 2019, 7:41 PM
aaron moved this task from Inbox, needs triage to Doing (old) on the Performance-Team board.
• mobrovac moved this task from Inbox to Backlog on the Platform Team Workboards (Clinic Duty Team) board.Aug 13 2019, 9:24 AM
• WDoranWMF moved this task from Backlog to Inbox on the Platform Team Workboards (Clinic Duty Team) board.Aug 13 2019, 6:22 PM
• kchapman subscribed.Aug 15 2019, 1:28 AM

@aaron since you claimed this task do you need anything from CPT?

Krinkle moved this task from Untriaged to Aug2019/1.34.wmf.17+ on the Wikimedia-production-error board.Aug 16 2019, 11:14 AM
aaron added a comment.Aug 17 2019, 7:04 PM

I don't think so.

gerritbot added a comment.Aug 17 2019, 7:05 PM

Change 529199 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] rdbms: make LBFactory close/rollback dangling handles like LoadBalancer

https://gerrit.wikimedia.org/r/529199

gerritbot added a project: Patch-For-Review.Aug 17 2019, 7:05 PM
• mobrovac moved this task from Inbox to External Code Review Needed on the Platform Team Workboards (Clinic Duty Team) board.Aug 19 2019, 12:06 PM
gerritbot added a comment.Aug 24 2019, 11:15 PM

Change 529199 merged by jenkins-bot:
[mediawiki/core@master] rdbms: make LBFactory close/rollback dangling handles like LoadBalancer

https://gerrit.wikimedia.org/r/529199

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.20; 2019-08-27).Aug 25 2019, 12:03 AM
Maintenance_bot removed a project: Patch-For-Review.Aug 25 2019, 12:10 AM
Krinkle mentioned this in T231443: Uncaught Wikimedia\Rdbms\DBUnexpectedError: Wikimedia\Rdbms\Database::close: mass commit/rollback of peer transaction required (DBO_TRX set).Aug 28 2019, 10:12 PM
• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:06 PM
Krinkle moved this task from Untriaged to Rdbms library on the MediaWiki-libs-Rdbms board.Aug 29 2019, 10:35 AM
Anomie removed a project: Platform Team Workboards (Clinic Duty Team).Sep 6 2019, 6:54 PM
Restricted Application added a project: Platform Engineering. · View Herald TranscriptSep 6 2019, 6:54 PM
Anomie removed a project: Platform Engineering.Sep 9 2019, 2:48 PM
Krinkle renamed this task from 'Wikimedia\Rdbms\DBQueryError' with message 'A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? to DBQueryError "Commands out of sync" from Rdbms\Database::close.Oct 12 2019, 11:21 PM
Krinkle closed this task as Resolved.Oct 12 2019, 11:25 PM
Krinkle subscribed.

No longer seen in Logstash in recent weeks.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits