Maniphest T220127

Reuse JobExecutor within a JobRunner
Closed, DeclinedPublic
Actions

Assigned To

Authored By

	• Pchelolo
	Apr 4 2019, 4:45 PM

Description

Currently there are 2 ways of executing the jobs in MediaWiki:

JobRunner used for all job queue implementation except Kafka implementation
JobExecutor used to execute a single job for the Kafka queue.

JobExecutor is mostly a copy-paste of certain parts of the JobRunner, so the two should be either merged together, or the JobRunner has to delegate actual job execution to the JobExecutor.

Details

	Subject	Repo	Branch	Lines +/-
	Add test for JobRunner	mediawiki/core	master	+86 -0

Customize query in gerrit

Related Objects

Mentioned In: T243096: Jobrunner monitoring still calles /rpc/runJobs.php
T243492: RunSingleJob: MediaWiki::restInPeace: transaction round 'LinksUpdate::doUpdate' still running
Mentioned Here: T244826: Create rest endpoint for executing jobs instead of /rpc/RunSingleJob
T175146: JobQueue: Unify JobRunner entry points

Event Timeline

• Pchelolo created this task.Apr 4 2019, 4:45 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 4 2019, 4:45 PM

• mobrovac triaged this task as Medium priority.Apr 4 2019, 6:08 PM

• mobrovac edited projects, added Platform Team Legacy (Next), Platform Engineering (Needs Cleaning - Security, stability, performance, and scalability (TEC1)); removed Platform Team Legacy.

Change 501597 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/core@master] [WIP] Reuse JobExecutor within a JobRunner

https://gerrit.wikimedia.org/r/501597

gerritbot added a project: Patch-For-Review.Apr 5 2019, 3:18 PM

• Pchelolo removed projects: Platform Team Legacy (Next), Services (next).Jul 16 2019, 7:21 PM

Krinkle moved this task from Untriaged to EventBus infra on the WMF-JobQueue board.Sep 9 2019, 9:03 PM

CCicalese_WMF edited projects, added Platform Team Workboards (Clinic Duty Team); removed Platform Engineering (Needs Cleaning - Security, stability, performance, and scalability (TEC1)).Oct 1 2019, 6:41 PM

daniel moved this task from Inbox to Backlog on the Platform Team Workboards (Clinic Duty Team) board.Oct 8 2019, 9:56 AM

• WDoranWMF reassigned this task from • Pchelolo to • Clarakosi.Jan 9 2020, 7:47 PM

• WDoranWMF moved this task from Backlog to Ready (WIP:5) on the Platform Team Workboards (Clinic Duty Team) board.

• Clarakosi moved this task from Ready (WIP:5) to Doing(WIP:5) on the Platform Team Workboards (Clinic Duty Team) board.Jan 23 2020, 9:38 PM

As reminder, there is a similar story between SpecialRunJob and (wmf-rpc)/RunSingleJob. See also T175146 about consolidating those.

Krinkle mentioned this in T243492: RunSingleJob: MediaWiki::restInPeace: transaction round 'LinksUpdate::doUpdate' still running.Jan 29 2020, 4:14 AM

• Pchelolo mentioned this in T243096: Jobrunner monitoring still calles /rpc/runJobs.php.Feb 7 2020, 1:57 AM

Change 501597 merged by jenkins-bot:
[mediawiki/core@master] Add test for JobRunner

https://gerrit.wikimedia.org/r/501597

Maintenance_bot removed a project: Patch-For-Review.Feb 7 2020, 5:10 PM

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.19; 2020-02-11).Feb 7 2020, 6:00 PM

• Pchelolo added a subtask: T244770: Enable RunSingleJobHandler endpoint on Job Runner Cluster.Feb 11 2020, 4:35 AM

• Pchelolo removed a subtask: T244770: Enable RunSingleJobHandler endpoint on Job Runner Cluster.

In light of T244826 we are not really going to do this, instead remove the JobExecutor entirely.

Reuse JobExecutor within a JobRunnerClosed, DeclinedPublicActions

Description

Details

Related Objects

Event Timeline

Reuse JobExecutor within a JobRunner
Closed, DeclinedPublic
Actions