Page MenuHomePhabricator

Enable RunSingleJobHandler endpoint on Job Runner Cluster
Closed, ResolvedPublic

Description

As discussed in the parent RFC, we want to get rid of the /rpc code in the mediawiki-config that's currently responsible for the job execution, and introduce a rest endpoint instead.

The endpoint will be protected against random job submissions by verifying the event signature, signed with mediawiki private key, like Special:RunSingleJob does. Also, since we currently don't maintain any kind of an automatic listing of REST endpoints in core, this essentially will be an internal endpoint.

However, to protect even more, it would be good to only enable it on jobrunner and videoscaler clusters and not on public-facing web or api clusters. We can control introduce a global configuration variable to selectively enable/disable the endpoint.

Event Timeline

Pchelolo updated the task description. (Show Details)
Pchelolo added a subscriber: Joe.

I've poked around mediawiki-config, and I don't really see a none-hacky way of setting a variable depending on MW cluster. set-time-limit.php has an implementation, but it seems a bit hacky. Is there a better way of getting a name of the MW cluster within MW config?

I've poked around mediawiki-config, and I don't really see a none-hacky way of setting a variable depending on MW cluster. set-time-limit.php has an implementation, but it seems a bit hacky. Is there a better way of getting a name of the MW cluster within MW config?

There is an env variable set by apache you can use, SERVER_GROUP.

For instance we check for the parsoid cluster as follows:

if ( ( $_SERVER['SERVERGROUP'] ?? null ) === 'parsoid' ) {
...
}

Change 572994 had a related patch set uploaded (by Clarakosi; owner: Clarakosi):
[operations/mediawiki-config@master] Enable EventBus Run Job API on only jobrunner clusters

https://gerrit.wikimedia.org/r/572994

Change 572994 merged by jenkins-bot:
[operations/mediawiki-config@master] Add EventBus Run Job API permissions

https://gerrit.wikimedia.org/r/572994

Mentioned in SAL (#wikimedia-operations) [2020-03-02T19:15:36Z] <ppchelko@deploy1001> Synchronized wmf-config/CommonSettings-labs.php: Enable REST run jobs endpoint on jobrunners T244770 (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2020-03-02T19:17:09Z] <ppchelko@deploy1001> Synchronized wmf-config/CommonSettings.php: Enable REST run jobs endpoint on jobrunners T244770 (duration: 00m 56s)

Pchelolo claimed this task.