Page MenuHomePhabricator
Create Task
Maniphest T171370

ERROR: "LBFactory::getEmptyTransactionTicket: WikiPageUpdater::injectRCRecords does not have outer scope"
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Started with wmf.10. Presumably caused by https://gerrit.wikimedia.org/r/364094 / rEWBA24d45d6d0cfc: Wait for replication after adding RC entries in WikiPageUpdater

Logstash:
https://logstash.wikimedia.org/goto/709280746172b68115f62db346b06201

Screen Shot 2017-07-21 at 19.27.46.png (334×1 px, 49 KB)

Wikimedia\Rdbms\LBFactory::getEmptyTransactionTicket: Wikibase\Client\Changes\WikiPageUpdater::injectRCRecords does not have outer scope.
#0 /srv/mediawiki/php-1.30.0-wmf.10/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/WikiPageUpdater.php(161): Wikimedia\Rdbms\LBFactory->getEmptyTransactionTicket()
#1 /srv/mediawiki/php-1.30.0-wmf.10/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/ChangeHandler.php(250): Wikibase\Client\Changes\WikiPageUpdater->injectRCRecords()
#2 /srv/mediawiki/php-1.30.0-wmf.10/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/ChangeHandler.php(169): Wikibase\Client\Changes\ChangeHandler->applyUpdateAction()
#3 /srv/mediawiki/php-1.30.0-wmf.10/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/ChangeHandler.php(137): Wikibase\Client\Changes\ChangeHandler->handleChange()
#4 /srv/mediawiki/php-1.30.0-wmf.10/extensions/Wikidata/extensions/Wikibase/client/includes/ChangeNotificationJob.php(99): Wikibase\Client\Changes\ChangeHandler->handleChanges()
#5 /srv/mediawiki/php-1.30.0-wmf.10/includes/jobqueue/JobRunner.php(293): Wikibase\Client\ChangeNotificationJob->run()
#6 /srv/mediawiki/php-1.30.0-wmf.10/includes/jobqueue/JobRunner.php(193): JobRunner->executeJob()
#7 /srv/mediawiki/rpc/RunJobs.php(47): JobRunner->run()
#8 {main}

See also:

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Fix handling of coalesced changes in InjectRCRecordsJobmediawiki/extensions/Wikibasemaster+427 -21
WB: Move injection of RC records to a separate job.mediawiki/extensions/Wikidatawmf/1.30.0-wmf.10+689 -195
Move injection of RC records to a separate job.mediawiki/extensions/Wikibasewmf/1.30.0-wmf.10+590 -100
Move injection of RC records to a separate job.mediawiki/extensions/Wikibasemaster+590 -100
Customize query in gerrit

Related Objects
Search...

Event Timeline

Krinkle created this task.Jul 22 2017, 2:30 AM
Restricted Application added a project: Wikidata. · View Herald TranscriptJul 22 2017, 2:30 AM
Restricted Application added subscribers: PokestarFan, Aklapper. · View Herald Transcript
Krinkle mentioned this in T164173: Cache invalidations coming from the JobQueue are causing lag on several wikis.Jul 22 2017, 2:30 AM
Krinkle added a parent task: T168050: MW 1.30.0-wmf.10 deployment blockers.
Krinkle updated the task description. (Show Details)Jul 22 2017, 2:32 AM
greg added a parent task: T168051: 1.30.0-wmf.11 deployment blockers.Jul 22 2017, 4:21 AM
gerritbot subscribed.Jul 23 2017, 4:02 PM

Change 367328 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/extensions/Wikibase@master] Move injection of RC records to a separate job.

https://gerrit.wikimedia.org/r/367328

gerritbot added a project: Patch-For-Review.Jul 23 2017, 4:02 PM
jcrespo subscribed.Jul 24 2017, 7:16 AM
daniel triaged this task as High priority.Jul 25 2017, 4:19 PM
daniel raised the priority of this task from High to Unbreak Now!.
daniel added a project: Wikidata-Former-Sprint-Board.
daniel moved this task from Proposed to Review on the Wikidata-Former-Sprint-Board board.
Restricted Application added subscribers: Liuxinyu970226, Jay8g, TerraCodes. · View Herald TranscriptJul 25 2017, 4:19 PM
greg subscribed.Jul 26 2017, 6:15 PM
daniel added subscribers: aaron, daniel.Jul 26 2017, 7:44 PM

Help! This thing is holding up 1.30.0-wmf.11 and needs review! review from folks who know about the JobQueue and deferred updates. @Krinkle? @aaron? The Wikidata team has little experience in this, especially with Addshore and Thiemo on vacation.

daniel awarded a token.Jul 26 2017, 7:44 PM
gerritbot added a comment.Jul 26 2017, 8:30 PM

Change 367328 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Move injection of RC records to a separate job.

https://gerrit.wikimedia.org/r/367328

Lucas_Werkmeister_WMDE moved this task from Review to Done on the Wikidata-Former-Sprint-Board board.Jul 27 2017, 10:06 AM
gerritbot added a comment.Jul 27 2017, 5:56 PM

Change 368224 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/extensions/Wikibase@wmf/1.30.0-wmf.10] Move injection of RC records to a separate job.

https://gerrit.wikimedia.org/r/368224

gerritbot added a comment.Jul 27 2017, 6:15 PM

Change 368224 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.30.0-wmf.10] Move injection of RC records to a separate job.

https://gerrit.wikimedia.org/r/368224

thcipriani subscribed.Jul 27 2017, 7:40 PM

Removing as blocker for wmf.11, does need a backport, but started with wmf.10, per IRC:

<DanielK_WMDE_> if wmf.10 of wikidata has been live without the patch for a week, there's no point in blocking on it now.
<DanielK_WMDE_> the error causes log spam, and database transactions get a bit strange, but nothing should be loast
<DanielK_WMDE_> *lost
thcipriani removed a parent task: T168051: 1.30.0-wmf.11 deployment blockers.Jul 27 2017, 7:42 PM
Krinkle added a subscriber: • demon.Jul 27 2017, 8:13 PM
In T171370#3476108, @gerritbot wrote:

Change 367328 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Move injection of RC records to a separate job.

https://gerrit.wikimedia.org/r/367328

At https://gerrit.wikimedia.org/r/367328, @demon wrote:

Where do we need to cherry-pick this to so we can get it live? I see wmf.7 is the current Wikidata branch.

In T171370#3479130, @gerritbot wrote:

Change 368224 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.30.0-wmf.10] Move injection of RC records to a separate job.

https://gerrit.wikimedia.org/r/368224

The Wikibase extension isn't deployed. The Wikidata extension is, which needs to be rebuild in the wmf.10 branch (or something?) for it to take effect. And the wmf.10 branch in mediawiki would need to be updated to point to it.

I don't see it yet at: https://github.com/wikimedia/mediawiki-extensions-Wikidata/tree/wmf/1.30.0-wmf.10/extensions/Wikibase/client/includes/Changes

Current Wikidata commit in wmf.10 of mediawiki-core: 41db3da4c

Also not at https://github.com/wikimedia/mediawiki-extensions-Wikidata/tree/41db3da4c/extensions/Wikibase/client/includes/Changes.

Krinkle lowered the priority of this task from Unbreak Now! to High.Jul 27 2017, 8:13 PM
gerritbot added a comment.Jul 27 2017, 9:41 PM

Change 368312 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikidata@wmf/1.30.0-wmf.10] WB: Move injection of RC records to a separate job.

https://gerrit.wikimedia.org/r/368312

Addshore added a parent task: T165316: Wikidata build jenkins failure: Command "test" is not defined and don't run composer update in the job.Jul 28 2017, 9:06 AM
Addshore removed a parent task: T165316: Wikidata build jenkins failure: Command "test" is not defined and don't run composer update in the job.
Addshore subscribed.

This is blocked by T165316 which is the issue we see in CI with the above patch.

Addshore added a comment.Jul 28 2017, 1:28 PM

Looks like jenkins is giving the patch a +2 now

• mmodell added a parent task: T168053: 1.30.0-wmf.12 deployment blockers.Jul 31 2017, 6:56 PM
• mmodell subscribed.

The patch appears to be merged on master but the wmf.10 cherry-pick is still open? I've added this as a blocker to wmf.12 since it's still open and I need to verify that the issue isn't affecting wmf.12 after I cut the branch tomorrow.

daniel added a comment.Jul 31 2017, 7:05 PM

@mmodell the cherry pick is merged https://gerrit.wikimedia.org/r/#/c/368224/, but the patch for the Wikidata build hasn't been https://gerrit.wikimedia.org/r/#/c/368312/3. I don't know why - I guess it got stuck on jenkins.

To avoid the confusion that held up the train last week: that this is about branches of the wikidata build and the wikibase extension. This does not block core updates.

• mmodell added a comment.Jul 31 2017, 7:25 PM

@daniel: Thanks, removing it from blocking

• mmodell removed a parent task: T168053: 1.30.0-wmf.12 deployment blockers.Jul 31 2017, 7:25 PM
Krinkle added a parent task: T164173: Cache invalidations coming from the JobQueue are causing lag on several wikis.Aug 2 2017, 7:59 PM
Krinkle removed projects: Wikidata-Former-Sprint-Board, Patch-For-Review.
gerritbot added a comment.Aug 2 2017, 8:01 PM

Change 368312 abandoned by Krinkle:
WB: Move injection of RC records to a separate job.

https://gerrit.wikimedia.org/r/368312

Krinkle added a subtask: T172320: Error in Wikibase/client/includes/Changes/InjectRCRecordsJob.php line 120: Bad value for parameter $params: $params['change'] not set..Aug 2 2017, 8:01 PM
Krinkle mentioned this in T172447: Investigate 2017-08-02 Save Timing regression (+40-60%).Aug 4 2017, 10:49 PM
Addshore mentioned this in T125976: Run mediawiki::maintenance scripts in Beta Cluster.Aug 9 2017, 7:17 PM
gerritbot added a comment.Aug 9 2017, 7:46 PM

Change 369881 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Fix handling of coalesced changes in InjectRCRecordsJob

https://gerrit.wikimedia.org/r/369881

thcipriani added a comment.Aug 16 2017, 5:09 PM

Just to be clear: it looks like there are fixes for this in wikibase master that made it into wikidata wmf.14 which made the branch cut for core wmf.14, so this shouldn't block wmf.14 train going to group1. This is the assumption I will be working under when I run the train in an hour or so.

greg edited parent tasks, added: T170632: 1.30.0-wmf.14 deployment blockers; removed: T168050: MW 1.30.0-wmf.10 deployment blockers.Aug 16 2017, 5:44 PM
Ladsgroup closed subtask T172320: Error in Wikibase/client/includes/Changes/InjectRCRecordsJob.php line 120: Bad value for parameter $params: $params['change'] not set. as Resolved.Aug 18 2017, 9:01 AM
Liuxinyu970226 awarded a token.Aug 18 2017, 12:46 PM
Krinkle closed this task as Resolved.Aug 23 2017, 12:11 AM

Fixed. The hit rate dropped to 0 for this warning in Logstash as of August 17. (Presumably due to all wikis to 1.30.0-wmf.14)

Liuxinyu970226 unsubscribed.Aug 23 2017, 4:34 AM
Krinkle mentioned this in T225098: Exception shutdown "transaction round 'Wikibase\Client\Changes\InjectRCRecordsJob::run' still running".Jun 5 2019, 2:29 PM
• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:10 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits