Page MenuHomePhabricator

Move video transcoding to use Shellbox
Open, HighPublic

Description

This needs the following to happen:

  • Create a new shellbox deployment called shellbox-video or similar
  • Create a new flavour of the shellbox image, including ffmpeg and fluidsynth at least, to use in that deployment
  • Convert the TimedMediaHandler extension to use BoxedCommand instead of UnboxedCommand (via wfShellExec)
  • Configure MediaWiki in production to use the remote shellbox installation

Related Objects

Event Timeline

Joe triaged this task as High priority.Jan 31 2024, 7:46 AM
Joe updated the task description. (Show Details)

Change #1043812 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/mediawiki-config@master] Add shellbox-video vars/config, enable on testwiki

https://gerrit.wikimedia.org/r/1043812

Change #1043812 merged by jenkins-bot:

[operations/mediawiki-config@master] Add shellbox-video vars/config, enable on beta

https://gerrit.wikimedia.org/r/1043812

Mentioned in SAL (#wikimedia-operations) [2024-06-26T14:21:24Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Started scap: Backport for [[gerrit:1043812|Add shellbox-video vars/config, enable on beta (T356241)]]

Mentioned in SAL (#wikimedia-operations) [2024-06-26T14:24:07Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 hnowlan, lucaswerkmeister-wmde: Backport for [[gerrit:1043812|Add shellbox-video vars/config, enable on beta (T356241)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-06-26T14:29:42Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1043812|Add shellbox-video vars/config, enable on beta (T356241)]] (duration: 08m 22s)

Change #1049970 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/mediawiki-config@master] testwiki: use shellbox-video for scaling video

https://gerrit.wikimedia.org/r/1049970

Change #1049970 merged by jenkins-bot:

[operations/mediawiki-config@master] testwiki: use shellbox-video for scaling video

https://gerrit.wikimedia.org/r/1049970

Mentioned in SAL (#wikimedia-operations) [2024-06-27T13:08:07Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:1038742|[CheckUser] Stop writing old for event tables migration on all wikis (T360685)]], [[gerrit:1049970|testwiki: use shellbox-video for scaling video (T356241)]], [[gerrit:1049886|Add VK namespace alias to Azerbaijani Wikibooks (T368237)]]

Mentioned in SAL (#wikimedia-operations) [2024-06-27T13:10:46Z] <urbanecm@deploy1002> urbanecm, dreamrimmer, hnowlan, dreamyjazz: Backport for [[gerrit:1038742|[CheckUser] Stop writing old for event tables migration on all wikis (T360685)]], [[gerrit:1049970|testwiki: use shellbox-video for scaling video (T356241)]], [[gerrit:1049886|Add VK namespace alias to Azerbaijani Wikibooks (T368237)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-06-27T13:24:55Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:1038742|[CheckUser] Stop writing old for event tables migration on all wikis (T360685)]], [[gerrit:1049970|testwiki: use shellbox-video for scaling video (T356241)]], [[gerrit:1049886|Add VK namespace alias to Azerbaijani Wikibooks (T368237)]] (duration: 16m 48s)

Change #1050375 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] shellbox-video: increase replicas, namespace resources

https://gerrit.wikimedia.org/r/1050375

Change #1050378 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/mediawiki-config@master] group0, group1: enable shellbox-video

https://gerrit.wikimedia.org/r/1050378

Change #1052325 had a related patch set uploaded (by Kamila Součková; author: Kamila Součková):

[operations/mediawiki-config@master] Add $wgMaxShellWallClockTime setting for shellbox

https://gerrit.wikimedia.org/r/1052325

Change #1050375 merged by jenkins-bot:

[operations/deployment-charts@master] shellbox-video: increase replicas, namespace resources

https://gerrit.wikimedia.org/r/1050375

Change #1052957 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] shellbox-video: reduce replicas

https://gerrit.wikimedia.org/r/1052957

Change #1052325 merged by jenkins-bot:

[operations/mediawiki-config@master] Add $wgMaxShellWallClockTime setting for shellbox

https://gerrit.wikimedia.org/r/1052325

Mentioned in SAL (#wikimedia-operations) [2024-07-09T13:05:31Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Started scap sync-world: Backport for [[gerrit:1052325|Add $wgMaxShellWallClockTime setting for shellbox (T356241)]]

Mentioned in SAL (#wikimedia-operations) [2024-07-09T13:08:04Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 kamila, lucaswerkmeister-wmde: Backport for [[gerrit:1052325|Add $wgMaxShellWallClockTime setting for shellbox (T356241)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-07-09T13:13:59Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1052325|Add $wgMaxShellWallClockTime setting for shellbox (T356241)]] (duration: 08m 28s)

Change #1052957 merged by jenkins-bot:

[operations/deployment-charts@master] shellbox-video: reduce replicas

https://gerrit.wikimedia.org/r/1052957

Change #1053300 had a related patch set uploaded (by Kamila Součková; author: Kamila Součková):

[operations/deployment-charts@master] shellbox-video: increase mesh timeout

https://gerrit.wikimedia.org/r/1053300

Change #1053300 merged by Kamila Součková:

[operations/deployment-charts@master] shellbox-video: increase mesh timeout

https://gerrit.wikimedia.org/r/1053300

Change #1053621 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] shellbox-video: set log level to debug temporarily

https://gerrit.wikimedia.org/r/1053621

Change #1053621 merged by jenkins-bot:

[operations/deployment-charts@master] shellbox-video: set log level to debug temporarily

https://gerrit.wikimedia.org/r/1053621