Page MenuHomePhabricator
Create Task
Maniphest T352515

RuntimeException: firejail is enabled, but cannot be found
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error
labels.normalized_message
[{reqId}] {exception_url}   RuntimeException: firejail is enabled, but cannot be found
error.stack_trace
from /srv/mediawiki/php-1.42.0-wmf.7/includes/shell/CommandFactory.php(125)
#0 /srv/mediawiki/php-1.42.0-wmf.7/includes/shell/CommandFactory.php(160): MediaWiki\Shell\CommandFactory->getLocalShellboxOptions()
#1 /srv/mediawiki/php-1.42.0-wmf.7/includes/shell/Shell.php(129): MediaWiki\Shell\CommandFactory->create()
#2 /srv/mediawiki/php-1.42.0-wmf.7/includes/GlobalFunctions.php(1597): MediaWiki\Shell\Shell::command(array)
#3 /srv/mediawiki/php-1.42.0-wmf.7/includes/media/DjVuImage.php(263): wfShellExec(string)
#4 /srv/mediawiki/php-1.42.0-wmf.7/includes/media/DjVuHandler.php(317): DjVuImage->retrieveMetaData()
#5 /srv/mediawiki/php-1.42.0-wmf.7/includes/media/MediaHandler.php(227): DjVuHandler->getSizeAndMetadata(TrivialMediaHandlerState, string)
#6 /srv/mediawiki/php-1.42.0-wmf.7/includes/utils/MWFileProps.php(89): MediaHandler->getSizeAndMetadataWithFallback(FSFile, string)
#7 /srv/mediawiki/php-1.42.0-wmf.7/includes/upload/UploadBase.php(546): MWFileProps->getPropsFromPath(string, string)
#8 /srv/mediawiki/php-1.42.0-wmf.7/includes/upload/UploadBase.php(478): UploadBase->verifyPartialFile()
#9 /srv/mediawiki/php-1.42.0-wmf.7/includes/upload/UploadBase.php(401): UploadBase->verifyFile()
#10 /srv/mediawiki/php-1.42.0-wmf.7/includes/upload/UploadFromFile.php(99): UploadBase->verifyUpload()
#11 /srv/mediawiki/php-1.42.0-wmf.7/includes/specials/SpecialUpload.php(521): UploadFromFile->verifyUpload()
#12 /srv/mediawiki/php-1.42.0-wmf.7/includes/specials/SpecialUpload.php(248): MediaWiki\Specials\SpecialUpload->processUpload()
#13 /srv/mediawiki/php-1.42.0-wmf.7/includes/specialpage/SpecialPage.php(727): MediaWiki\Specials\SpecialUpload->execute(NULL)
#14 /srv/mediawiki/php-1.42.0-wmf.7/includes/specialpage/SpecialPageFactory.php(1644): MediaWiki\SpecialPage\SpecialPage->run(NULL)
#15 /srv/mediawiki/php-1.42.0-wmf.7/includes/MediaWiki.php(359): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext)
#16 /srv/mediawiki/php-1.42.0-wmf.7/includes/MediaWiki.php(962): MediaWiki->performRequest()
#17 /srv/mediawiki/php-1.42.0-wmf.7/includes/MediaWiki.php(615): MediaWiki->main()
#18 /srv/mediawiki/php-1.42.0-wmf.7/index.php(50): MediaWiki->run()
#19 /srv/mediawiki/php-1.42.0-wmf.7/index.php(46): wfIndexMain()
#20 /srv/mediawiki/w/index.php(3): require(string)
#21 {main}
Impact
Notes

The URL is Special:Upload , the 3 errors logged so far are all for frwikisource and all related to DjVuHandler.php(317): DjVuImage->retrieveMetaData() which does MediaWiki\Shell\CommandFactory->getLocalShellboxOptions(). My guess is the Kubernetes image does not have firejail or the DjVu handler should use a remote shellbox?

Details

Request URL
https://fr.wikisource.org/wiki/Sp%C3%A9cial:T%C3%A9l%C3%A9verser
SubjectRepoBranchLines +/-
Explicitly disable all local imagescaling on k8soperations/mediawiki-configmaster+3 -0
jobqueue: restore media handling jobs to k8soperations/deployment-chartsmaster+0 -8
Always process media files via shellbox on k8soperations/mediawiki-configmaster+12 -0
Use shellbox for djvu handling on kubernetesoperations/mediawiki-configmaster+9 -1
changeprop-jobqueue: move PublishStashedFile back to metal temporarilyoperations/deployment-chartsmaster+5 -1
changeprop-jobqueue: move AssembleUploadChunks back to metal temporarilyoperations/deployment-chartsmaster+4 -0
Clean up DjVuImage::retrieveMetaData including shellbox supportmediawiki/coremaster+145 -87
media: Port DjVuImage::retrieveMetaData() to use BoxedCommandmediawiki/coremaster+142 -14
Customize query in gerrit

Related Objects

Event Timeline

hashar created this task.Dec 1 2023, 9:48 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 1 2023, 9:48 AM
Joe triaged this task as High priority.Dec 1 2023, 10:31 AM
Joe subscribed.Dec 1 2023, 10:42 AM

The problem is apparently https://phabricator.wikimedia.org/T289228#7588412

Joe raised the priority of this task from High to Unbreak Now!.Dec 1 2023, 11:09 AM

Changing to UBN! because even if the frequency is low enough right now, this is a user-visible error.

Joe added a comment.Dec 1 2023, 12:19 PM

So the context is that there was a problem on CLI when running maint scripts with the move of DjVu to shellbox. One quick stopgap would be to reintroduce the code in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/720143 under a feature flag, and flip it to true only when PHP_SAPI is not cli in mediawiki-config for now, while a proper fix for the issue is found.

MSantos added subscribers: AntiCompositeNumber, Legoktm, TheDJ.Dec 1 2023, 12:29 PM
Joe added a comment.EditedDec 1 2023, 1:03 PM

According to logstash, this has happened 9 times in the last month, all on wikisource. So it is a user-facing problem but probably not as severe as I feared.

Jdforrester-WMF lowered the priority of this task from Unbreak Now! to High.Dec 1 2023, 3:07 PM
Jdforrester-WMF subscribed.
In T352515#9374813, @Joe wrote:

According to logstash, this has happened 9 times in the last month, all on wikisource. So it is a user-facing problem but probably not as severe as I feared.

Agreed.

MSantos added a project: Structured-Data-Backlog.Dec 1 2023, 3:55 PM
MSantos subscribed.

Adding Structured-Data-Backlog as per https://www.mediawiki.org/wiki/Developers/Maintainers

gerritbot added a comment.Dec 1 2023, 4:05 PM

Change 979386 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[mediawiki/core@master] media: Port DjVuImage::retrieveMetaData() to use BoxedCommand

https://gerrit.wikimedia.org/r/979386

gerritbot added a project: Patch-For-Review.Dec 1 2023, 4:05 PM
gerritbot added a comment.Dec 1 2023, 9:19 PM

Change 979386 merged by jenkins-bot:

[mediawiki/core@master] media: Port DjVuImage::retrieveMetaData() to use BoxedCommand

https://gerrit.wikimedia.org/r/979386

Maintenance_bot removed a project: Patch-For-Review.Dec 1 2023, 9:31 PM
ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.9; 2023-12-12).Dec 1 2023, 10:00 PM
tstarling subscribed.Dec 3 2023, 11:17 PM

Please wait until my forthcoming cleanup patch is merged before deploying this.

gerritbot added a comment.Dec 4 2023, 12:41 AM

Change 979483 had a related patch set uploaded (by Tim Starling; author: Tim Starling):

[mediawiki/core@master] Clean up DjVuImage::retrieveMetaData including shellbox support

https://gerrit.wikimedia.org/r/979483

gerritbot added a project: Patch-For-Review.Dec 4 2023, 12:41 AM
gerritbot added a comment.Dec 6 2023, 10:40 AM

Change 979483 merged by jenkins-bot:

[mediawiki/core@master] Clean up DjVuImage::retrieveMetaData including shellbox support

https://gerrit.wikimedia.org/r/979483

Maintenance_bot removed a project: Patch-For-Review.Dec 6 2023, 11:10 AM
TheresNoTime subscribed.Dec 12 2023, 11:38 AM
hashar unsubscribed.Dec 13 2023, 8:19 AM
brennen added a project: User-brennen.Dec 13 2023, 7:19 PM
brennen moved this task from Backlog to Logs/Train on the User-brennen board.
brennen subscribed.Dec 14 2023, 4:18 PM

Seeing this also for PdfHandler:

Error
labels.normalized_message
[{reqId}] {exception_url}   RuntimeException: firejail is enabled, but cannot be found
error.stack_trace
from /srv/mediawiki/php-1.42.0-wmf.9/includes/shell/CommandFactory.php(125)
#0 /srv/mediawiki/php-1.42.0-wmf.9/includes/shell/CommandFactory.php(189): MediaWiki\Shell\CommandFactory->getLocalShellboxOptions()
#1 /srv/mediawiki/php-1.42.0-wmf.9/extensions/PdfHandler/includes/PdfImage.php(110): MediaWiki\Shell\CommandFactory->createBoxed(string)
#2 /srv/mediawiki/php-1.42.0-wmf.9/extensions/PdfHandler/includes/PdfHandler.php(280): MediaWiki\Extension\PdfHandler\PdfImage->retrieveMetaData()
#3 /srv/mediawiki/php-1.42.0-wmf.9/includes/media/MediaHandler.php(227): MediaWiki\Extension\PdfHandler\PdfHandler->getSizeAndMetadata(TrivialMediaHandlerState, string)
#4 /srv/mediawiki/php-1.42.0-wmf.9/includes/utils/MWFileProps.php(89): MediaHandler->getSizeAndMetadataWithFallback(FSFile, string)
#5 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadBase.php(546): MWFileProps->getPropsFromPath(string, string)
#6 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadBase.php(478): UploadBase->verifyPartialFile()
#7 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadBase.php(401): UploadBase->verifyFile()
#8 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadFromFile.php(99): UploadBase->verifyUpload()
#9 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadFromChunks.php(183): UploadFromFile->verifyUpload()
#10 /srv/mediawiki/php-1.42.0-wmf.9/includes/jobqueue/jobs/AssembleUploadChunksJob.php(67): UploadFromChunks->concatenateChunks()
#11 /srv/mediawiki/php-1.42.0-wmf.9/extensions/EventBus/includes/JobExecutor.php(80): AssembleUploadChunksJob->run()
#12 /srv/mediawiki/rpc/RunSingleJob.php(67): MediaWiki\Extension\EventBus\JobExecutor->execute(array)
#13 {main}
brennen moved this task from Untriaged to Nov 2023 on the Wikimedia-production-error board.Dec 14 2023, 4:31 PM
gerritbot added a comment.Dec 19 2023, 11:38 AM

Change 984140 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] changeprop-jobqueue: move AssembleUploadChunks back to metal temporarily

https://gerrit.wikimedia.org/r/984140

gerritbot added a project: Patch-For-Review.Dec 19 2023, 11:38 AM
gerritbot added a comment.Dec 19 2023, 11:56 AM

Change 984140 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop-jobqueue: move AssembleUploadChunks back to metal temporarily

https://gerrit.wikimedia.org/r/984140

Clement_Goubert subscribed.Dec 19 2023, 12:00 PM
In T352515#9407128, @brennen wrote:

Seeing this also for PdfHandler:

Error
labels.normalized_message
[{reqId}] {exception_url}   RuntimeException: firejail is enabled, but cannot be found
error.stack_trace
from /srv/mediawiki/php-1.42.0-wmf.9/includes/shell/CommandFactory.php(125)
#0 /srv/mediawiki/php-1.42.0-wmf.9/includes/shell/CommandFactory.php(189): MediaWiki\Shell\CommandFactory->getLocalShellboxOptions()
#1 /srv/mediawiki/php-1.42.0-wmf.9/extensions/PdfHandler/includes/PdfImage.php(110): MediaWiki\Shell\CommandFactory->createBoxed(string)
#2 /srv/mediawiki/php-1.42.0-wmf.9/extensions/PdfHandler/includes/PdfHandler.php(280): MediaWiki\Extension\PdfHandler\PdfImage->retrieveMetaData()
#3 /srv/mediawiki/php-1.42.0-wmf.9/includes/media/MediaHandler.php(227): MediaWiki\Extension\PdfHandler\PdfHandler->getSizeAndMetadata(TrivialMediaHandlerState, string)
#4 /srv/mediawiki/php-1.42.0-wmf.9/includes/utils/MWFileProps.php(89): MediaHandler->getSizeAndMetadataWithFallback(FSFile, string)
#5 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadBase.php(546): MWFileProps->getPropsFromPath(string, string)
#6 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadBase.php(478): UploadBase->verifyPartialFile()
#7 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadBase.php(401): UploadBase->verifyFile()
#8 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadFromFile.php(99): UploadBase->verifyUpload()
#9 /srv/mediawiki/php-1.42.0-wmf.9/includes/upload/UploadFromChunks.php(183): UploadFromFile->verifyUpload()
#10 /srv/mediawiki/php-1.42.0-wmf.9/includes/jobqueue/jobs/AssembleUploadChunksJob.php(67): UploadFromChunks->concatenateChunks()
#11 /srv/mediawiki/php-1.42.0-wmf.9/extensions/EventBus/includes/JobExecutor.php(80): AssembleUploadChunksJob->run()
#12 /srv/mediawiki/rpc/RunSingleJob.php(67): MediaWiki\Extension\EventBus\JobExecutor->execute(array)
#13 {main}

We've moved the affected job (AssembleUploadChunks) back to metal temporarily.

There are still some of these errors popping up for frwikisource Special:Téléverser (Special:Upload), but not on other wikis as far as I can tell.

Maintenance_bot removed a project: Patch-For-Review.Dec 19 2023, 12:30 PM
thcipriani subscribed.Dec 21 2023, 4:47 PM
In T352515#9415206, @Clement_Goubert wrote:

We've moved the affected job (AssembleUploadChunks) back to metal temporarily.

There are still some of these errors popping up for frwikisource Special:Téléverser (Special:Upload), but not on other wikis as far as I can tell.

Note that some of the PDFHandler and PagedTiffHandler errors are still showing up on the mw-jobrunner k8s hosts. They seem to be coming from the PublishStashedFileJob. Mostly on commonswiki.

For example:

Error
normalized_message
[{reqId}] {exception_url}   RuntimeException: firejail is enabled, but cannot be found
exception.trace
from /srv/mediawiki/php-1.42.0-wmf.10/includes/shell/CommandFactory.php(125)
#0 /srv/mediawiki/php-1.42.0-wmf.10/includes/shell/CommandFactory.php(189): MediaWiki\Shell\CommandFactory->getLocalShellboxOptions()
#1 /srv/mediawiki/php-1.42.0-wmf.10/extensions/PdfHandler/includes/PdfImage.php(110): MediaWiki\Shell\CommandFactory->createBoxed(string)
#2 /srv/mediawiki/php-1.42.0-wmf.10/extensions/PdfHandler/includes/PdfHandler.php(280): MediaWiki\Extension\PdfHandler\PdfImage->retrieveMetaData()
#3 /srv/mediawiki/php-1.42.0-wmf.10/includes/media/MediaHandler.php(227): MediaWiki\Extension\PdfHandler\PdfHandler->getSizeAndMetadata(TrivialMediaHandlerState, string)
#4 /srv/mediawiki/php-1.42.0-wmf.10/includes/utils/MWFileProps.php(89): MediaHandler->getSizeAndMetadataWithFallback(FSFile, string)
#5 /srv/mediawiki/php-1.42.0-wmf.10/includes/upload/UploadBase.php(546): MWFileProps->getPropsFromPath(string, string)
#6 /srv/mediawiki/php-1.42.0-wmf.10/includes/upload/UploadBase.php(478): UploadBase->verifyPartialFile()
#7 /srv/mediawiki/php-1.42.0-wmf.10/includes/upload/UploadBase.php(401): UploadBase->verifyFile()
#8 /srv/mediawiki/php-1.42.0-wmf.10/includes/jobqueue/jobs/PublishStashedFileJob.php(66): UploadBase->verifyUpload()
#9 /srv/mediawiki/php-1.42.0-wmf.10/extensions/EventBus/includes/JobExecutor.php(80): PublishStashedFileJob->run()
#10 /srv/mediawiki/rpc/RunSingleJob.php(60): MediaWiki\Extension\EventBus\JobExecutor->execute(array)
#11 {main}
gerritbot added a comment.Dec 21 2023, 4:54 PM

Change 984865 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/deployment-charts@master] changeprop-jobqueue: move PublishStashedFile back to metal temporarily

https://gerrit.wikimedia.org/r/984865

gerritbot added a project: Patch-For-Review.Dec 21 2023, 4:54 PM
gerritbot added a comment.Dec 21 2023, 5:01 PM

Change 984865 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop-jobqueue: move PublishStashedFile back to metal temporarily

https://gerrit.wikimedia.org/r/984865

Clement_Goubert added a comment.Dec 21 2023, 5:06 PM
In T352515#9421751, @thcipriani wrote:
In T352515#9415206, @Clement_Goubert wrote:

We've moved the affected job (AssembleUploadChunks) back to metal temporarily.

There are still some of these errors popping up for frwikisource Special:Téléverser (Special:Upload), but not on other wikis as far as I can tell.

Note that some of the PDFHandler and PagedTiffHandler errors are still showing up on the mw-jobrunner k8s hosts. They seem to be coming from the PublishStashedFileJob. Mostly on commonswiki.

For example:

Error
normalized_message
[{reqId}] {exception_url}   RuntimeException: firejail is enabled, but cannot be found
exception.trace
from /srv/mediawiki/php-1.42.0-wmf.10/includes/shell/CommandFactory.php(125)
#0 /srv/mediawiki/php-1.42.0-wmf.10/includes/shell/CommandFactory.php(189): MediaWiki\Shell\CommandFactory->getLocalShellboxOptions()
#1 /srv/mediawiki/php-1.42.0-wmf.10/extensions/PdfHandler/includes/PdfImage.php(110): MediaWiki\Shell\CommandFactory->createBoxed(string)
#2 /srv/mediawiki/php-1.42.0-wmf.10/extensions/PdfHandler/includes/PdfHandler.php(280): MediaWiki\Extension\PdfHandler\PdfImage->retrieveMetaData()
#3 /srv/mediawiki/php-1.42.0-wmf.10/includes/media/MediaHandler.php(227): MediaWiki\Extension\PdfHandler\PdfHandler->getSizeAndMetadata(TrivialMediaHandlerState, string)
#4 /srv/mediawiki/php-1.42.0-wmf.10/includes/utils/MWFileProps.php(89): MediaHandler->getSizeAndMetadataWithFallback(FSFile, string)
#5 /srv/mediawiki/php-1.42.0-wmf.10/includes/upload/UploadBase.php(546): MWFileProps->getPropsFromPath(string, string)
#6 /srv/mediawiki/php-1.42.0-wmf.10/includes/upload/UploadBase.php(478): UploadBase->verifyPartialFile()
#7 /srv/mediawiki/php-1.42.0-wmf.10/includes/upload/UploadBase.php(401): UploadBase->verifyFile()
#8 /srv/mediawiki/php-1.42.0-wmf.10/includes/jobqueue/jobs/PublishStashedFileJob.php(66): UploadBase->verifyUpload()
#9 /srv/mediawiki/php-1.42.0-wmf.10/extensions/EventBus/includes/JobExecutor.php(80): PublishStashedFileJob->run()
#10 /srv/mediawiki/rpc/RunSingleJob.php(60): MediaWiki\Extension\EventBus\JobExecutor->execute(array)
#11 {main}

Moved back to metal as well.

Maintenance_bot removed a project: Patch-For-Review.Dec 21 2023, 5:30 PM
gerritbot added a comment.Jan 2 2024, 8:34 AM

Change 987032 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/mediawiki-config@master] Use shellbox for djvu handling on kubernetes

https://gerrit.wikimedia.org/r/987032

gerritbot added a project: Patch-For-Review.Jan 2 2024, 8:34 AM

Change 987033 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/mediawiki-config@master] Always process media files via shellbox on k8s

https://gerrit.wikimedia.org/r/987033

akosiaris subscribed.Jan 2 2024, 11:31 AM
gerritbot added a comment.Jan 3 2024, 2:44 PM

Change 987432 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/mediawiki-config@master] Explicitly disable all local imagescaling on k8s

https://gerrit.wikimedia.org/r/987432

gerritbot added a comment.Jan 9 2024, 9:03 AM

Change 987032 merged by jenkins-bot:

[operations/mediawiki-config@master] Use shellbox for djvu handling on kubernetes

https://gerrit.wikimedia.org/r/987032

Stashbot added a comment.Jan 9 2024, 9:03 AM

Mentioned in SAL (#wikimedia-operations) [2024-01-09T09:03:25Z] <oblivian@deploy2002> Started scap: Backport for [[gerrit:987032|Use shellbox for djvu handling on kubernetes (T352515)]]

Stashbot added a comment.Jan 9 2024, 9:05 AM

Mentioned in SAL (#wikimedia-operations) [2024-01-09T09:05:17Z] <oblivian@deploy2002> oblivian: Backport for [[gerrit:987032|Use shellbox for djvu handling on kubernetes (T352515)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Stashbot added a comment.Jan 9 2024, 9:27 AM

Mentioned in SAL (#wikimedia-operations) [2024-01-09T09:27:22Z] <oblivian@deploy2002> Finished scap: Backport for [[gerrit:987032|Use shellbox for djvu handling on kubernetes (T352515)]] (duration: 23m 56s)

gerritbot added a comment.Jan 9 2024, 9:42 AM

Change 987033 merged by jenkins-bot:

[operations/mediawiki-config@master] Always process media files via shellbox on k8s

https://gerrit.wikimedia.org/r/987033

Stashbot added a comment.Jan 9 2024, 9:43 AM

Mentioned in SAL (#wikimedia-operations) [2024-01-09T09:42:59Z] <oblivian@deploy2002> Started scap: Backport for [[gerrit:987033|Always process media files via shellbox on k8s (T352515)]]

Stashbot added a comment.Jan 9 2024, 9:44 AM

Mentioned in SAL (#wikimedia-operations) [2024-01-09T09:44:41Z] <oblivian@deploy2002> oblivian: Backport for [[gerrit:987033|Always process media files via shellbox on k8s (T352515)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Stashbot added a comment.Jan 9 2024, 9:54 AM

Mentioned in SAL (#wikimedia-operations) [2024-01-09T09:54:02Z] <oblivian@deploy2002> Finished scap: Backport for [[gerrit:987033|Always process media files via shellbox on k8s (T352515)]] (duration: 11m 03s)

gerritbot added a comment.Jan 9 2024, 11:06 AM

Change 989128 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] jobqueue: restore media handling jobs to k8s

https://gerrit.wikimedia.org/r/989128

gerritbot added a comment.Jan 9 2024, 11:15 AM

Change 989128 merged by jenkins-bot:

[operations/deployment-charts@master] jobqueue: restore media handling jobs to k8s

https://gerrit.wikimedia.org/r/989128

hnowlan subscribed.Jan 9 2024, 5:45 PM

After moving these tasks to shellbox and pointing the jobqueue back to k8s jobrunners, these errors have not reappeared.

hnowlan closed this task as Resolved.Jan 16 2024, 9:43 AM
tstarling mentioned this in T295679: Shellbox getFileContents dooesn't return the file in production CLI.Feb 20 2024, 4:52 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL