shellbox/execvp: Permission denied on `MWScript.php importImages.php`
Open, Needs TriagePublicPRODUCTION ERROR
Actions

Assigned To

None

Authored By

	TheresNoTime
	Dec 15 2022, 11:33 PM

Description

Error

mwversion: 1.40.0-wmf.14
reqId: 4e5da06f74b9078274ee9a29
Find reqId in Logstash

normalized_message

Error running {command}: {error}

cli_argv

/srv/mediawiki-staging/multiversion/MWScript.php importImages.php --wiki=commonswiki --comment-ext=txt --user=Coffeeandcrumbs /home/samtar/imports

command

Error running /bin/bash '/srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/bin/firejail'\'' '\''--quiet'\'' '\''--profile=/srv/mediawiki/php-1.40.0-wmf.14/includes/shell/firejail.profile'\'' '\''--seccomp'\'' '\''--net=none'\'' '\''--env=TIFF_USETIFFINFO=yes'\'' '\''--env=TIFF_TIFFINFO=/usr/bin/tiffinfo'\'' '\''--env=TIFF_IDENTIFY=/usr/bin/identify'\'' '\''--env=TIFF_USEEXIV=no'\'' '\''--env=TIFF_EXIV2=/usr/bin/exiv2'\'' -- '\''/bin/sh'\'' '\''scripts/retrieveMetaData.sh'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=50; SB_CGROUP='\''/sys/fs/cgroup/memory/mediawiki/job'\''; SB_MEM_LIMIT=1073741824; SB_FILE_SIZE_LIMIT=536870912; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes': execvp: Permission denied

exception.trace

from /srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/UnboxedExecutor.php(430)
#0 /srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/LocalBoxedExecutor.php(41): Shellbox\Command\UnboxedExecutor->execute(Shellbox\Command\BoxedCommand)
#1 /srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/BoxedExecutor.php(20): Shellbox\Command\LocalBoxedExecutor->executeValid(Shellbox\Command\BoxedCommand)
#2 /srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/BoxedCommand.php(183): Shellbox\Command\BoxedExecutor->execute(Shellbox\Command\BoxedCommand)
#3 /srv/mediawiki/php-1.40.0-wmf.14/extensions/PagedTiffHandler/includes/PagedTiffImage.php(157): Shellbox\Command\BoxedCommand->execute()
#4 /srv/mediawiki/php-1.40.0-wmf.14/extensions/PagedTiffHandler/includes/PagedTiffHandler.php(597): MediaWiki\Extension\PagedTiffHandler\PagedTiffImage->retrieveMetaData()
#5 /srv/mediawiki/php-1.40.0-wmf.14/includes/media/MediaHandler.php(225): MediaWiki\Extension\PagedTiffHandler\PagedTiffHandler->getSizeAndMetadata(TrivialMediaHandlerState, string)
#6 /srv/mediawiki/php-1.40.0-wmf.14/includes/utils/MWFileProps.php(89): MediaHandler->getSizeAndMetadataWithFallback(FSFile, string)
#7 /srv/mediawiki/php-1.40.0-wmf.14/maintenance/importImages.php(326): MWFileProps->getPropsFromPath(string, string)
#8 /srv/mediawiki/php-1.40.0-wmf.14/maintenance/includes/MaintenanceRunner.php(309): ImportImages->execute()
#9 /srv/mediawiki/php-1.40.0-wmf.14/maintenance/doMaintenance.php(85): MediaWiki\Maintenance\MaintenanceRunner->run()
#10 /srv/mediawiki/php-1.40.0-wmf.14/maintenance/importImages.php(537): require_once(string)
#11 /srv/mediawiki/multiversion/MWScript.php(120): require_once(string)
#12 {main}

Impact

Notes

Caused while attempting to do T325330: Server-side upload request for Coffeeandcrumbs (see also https://commons.wikimedia.org/wiki/File:Eliza_Clark_Garrett_1858_Engraving.tiff)

Related Objects

Mentioned In: T325330: Server-side upload request for Coffeeandcrumbs
Mentioned Here: T325330: Server-side upload request for Coffeeandcrumbs

Event Timeline

TheresNoTime created this task.Dec 15 2022, 11:33 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 15 2022, 11:33 PM

TheresNoTime mentioned this in T325330: Server-side upload request for Coffeeandcrumbs.Dec 15 2022, 11:40 PM

Might be an issue with the restrictions in the shell out in PagedTiffHandler? Though pretty sure it worked when I implemented this...

Well I can reproduce it at least:

legoktm@mwmaint1002:~/tmp$ sudo -u www-data /bin/bash '/srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/bin/firejail'\'' '\''--quiet'\'' '\''--profile=/srv/mediawiki/php-1.40.0-wmf.14/includes/shell/firejail.profile'\'' '\''--seccomp'\'' '\''--net=none'\'' '\''--env=TIFF_USETIFFINFO=yes'\'' '\''--env=TIFF_TIFFINFO=/usr/bin/tiffinfo'\'' '\''--env=TIFF_IDENTIFY=/usr/bin/identify'\'' '\''--env=TIFF_USEEXIV=no'\'' '\''--env=TIFF_EXIV2=/usr/bin/exiv2'\'' -- '\''/bin/sh'\'' '\''/srv/mediawiki/php-1.40.0-wmf.14/extensions/PagedTiffHandler/scripts/retrieveMetaData.sh'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=50; SB_CGROUP='\''/sys/fs/cgroup/memory/mediawiki/job'\''; SB_MEM_LIMIT=1073741824; SB_FILE_SIZE_LIMIT=536870912; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
execvp: Permission denied

It's something specific to www-data I think. When I run it as mwdeploy, it works fine...

legoktm@mwmaint1002:/tmp/lego-test$ sudo chown -R mwdeploy:mwdeploy .
legoktm@mwmaint1002:/tmp/lego-test$ sudo -u mwdeploy /bin/bash '/srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/bin/firejail'\'' '\''--trace'\''  '\''--quiet'\'' '\''--profile=/srv/mediawiki/php-1.40.0-wmf.14/includes/shell/firejail.profile'\'' '\''--seccomp'\'' '\''--net=none'\'' '\''--env=TIFF_USETIFFINFO=yes'\'' '\''--env=TIFF_TIFFINFO=/usr/bin/tiffinfo'\'' '\''--env=TIFF_IDENTIFY=/usr/bin/identify'\'' '\''--env=TIFF_USEEXIV=no'\'' '\''--env=TIFF_EXIV2=/usr/bin/exiv2'\'' -- '\''/bin/sh'\'' '\''/srv/mediawiki/php-1.40.0-wmf.14/extensions/PagedTiffHandler/scripts/retrieveMetaData.sh'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=50; SB_CGROUP='\''/sys/fs/cgroup/memory/mediawiki/job'\''; SB_MEM_LIMIT=1073741824; SB_FILE_SIZE_LIMIT=536870912; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
5:bash:exec /usr/bin/bash:0
5:bash:open /dev/tty:3
5:sh:exec /usr/bin/dash:0
5:sh:open /srv/mediawiki/php-1.40.0-wmf.14/extensions/PagedTiffHandler/scripts/retrieveMetaData.sh:3
5:sh:open64 info:3
legoktm@mwmaint1002:/tmp/lego-test$ cat info 
6:tiffinfo:exec /usr/bin/tiffinfo:0
6:tiffinfo:open file.tiff:3
TIFF Directory at offset 0x19f9b9e (27237278)
  Subfile Type: (0 = 0x0)
  Image Width: 2746 Image Length: 4114
  Resolution: 600, 600 pixels/inch
  Bits/Sample: 8
  Compression Scheme: AdobeDeflate
  Photometric Interpretation: RGB color
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Rows/Strip: 1
  Planar Configuration: single image plane
  Software: Adobe Photoshop CC 2017 (Windows)
  DateTime: 2018:11:06 15:33:18
legoktm@mwmaint1002:/tmp/lego-test$ rm info 
rm: remove write-protected regular file 'info'? y
rm: cannot remove 'info': Permission denied
legoktm@mwmaint1002:/tmp/lego-test$ sudo rm info 
legoktm@mwmaint1002:/tmp/lego-test$ sudo chown -R www-data:www-data .
legoktm@mwmaint1002:/tmp/lego-test$ sudo -u www-data /bin/bash '/srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/bin/firejail'\'' '\''--trace'\''  '\''--quiet'\'' '\''--profile=/srv/mediawiki/php-1.40.0-wmf.14/includes/shell/firejail.profile'\'' '\''--seccomp'\'' '\''--net=none'\'' '\''--env=TIFF_USETIFFINFO=yes'\'' '\''--env=TIFF_TIFFINFO=/usr/bin/tiffinfo'\'' '\''--env=TIFF_IDENTIFY=/usr/bin/identify'\'' '\''--env=TIFF_USEEXIV=no'\'' '\''--env=TIFF_EXIV2=/usr/bin/exiv2'\'' -- '\''/bin/sh'\'' '\''/srv/mediawiki/php-1.40.0-wmf.14/extensions/PagedTiffHandler/scripts/retrieveMetaData.sh'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=50; SB_CGROUP='\''/sys/fs/cgroup/memory/mediawiki/job'\''; SB_MEM_LIMIT=1073741824; SB_FILE_SIZE_LIMIT=536870912; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
execvp: Permission denied

And then a simpler case, taking all the tiff stuff out of the equation:

legoktm@mwmaint1002:/tmp/lego-test$ sudo -u mwdeploy /bin/bash '/srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/bin/firejail'\'' '\''--trace'\'' '\''/bin/sh'\'''
Reading profile /etc/firejail/default.profile
Reading profile /etc/firejail/disable-common.inc
Reading profile /etc/firejail/disable-passwdmgr.inc
Reading profile /etc/firejail/disable-programs.inc
Warning: networking feature is disabled in Firejail configuration file

** Note: you can use --noprofile to disable default.profile **

Parent pid 12152, child pid 12160
Warning: failed to clean up /etc/group
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Child process initialized in 72.49 ms
5:bash:exec /usr/bin/bash:0
5:bash:open /dev/tty:3
5:sh:exec /usr/bin/dash:0
5:sh:open /dev/tty:3
$ 
/bin/sh: 0: Cannot set tty process group (No such process)

Parent is shutting down, bye...
legoktm@mwmaint1002:/tmp/lego-test$ sudo -u www-data /bin/bash '/srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/bin/firejail'\'' '\''--trace'\'' '\''/bin/sh'\'''
Reading profile /etc/firejail/default.profile
Reading profile /etc/firejail/disable-common.inc
Reading profile /etc/firejail/disable-passwdmgr.inc
Reading profile /etc/firejail/disable-programs.inc
Warning: networking feature is disabled in Firejail configuration file

** Note: you can use --noprofile to disable default.profile **

Parent pid 12224, child pid 12225
Warning: failed to clean up /etc/group
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Child process initialized in 60.30 ms
execvp: Permission denied

Parent is shutting down, bye...

Setting --shell=/bin/dash with firejail worked. https://github.com/netblue30/firejail/issues/5196 suggests it's using the default login shell, which for www-data is /sbin/nologin and obviously fails (whereas mwdeploy gets a bash shell). If I'm right (feeling like 70% confident) I still don't know why 1) this isn't more broadly broken for normal uploads, 2) what caused this to break now.

legoktm@mwmaint1002:/tmp/lego-test$ sudo -u www-data /bin/bash '/srv/mediawiki/php-1.40.0-wmf.14/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/bin/firejail'\'' '\''--trace'\''  '\''--quiet'\'' '\''--profile=/srv/mediawiki/php-1.40.0-wmf.14/includes/shell/firejail.profile'\'' '\''--seccomp'\'' '\''--net=none'\'' '\''--shell=/bin/dash'\'' '\''--env=TIFF_USETIFFINFO=yes'\'' '\''--env=TIFF_TIFFINFO=/usr/bin/tiffinfo'\'' '\''--env=TIFF_IDENTIFY=/usr/bin/identify'\'' '\''--env=TIFF_USEEXIV=no'\'' '\''--env=TIFF_EXIV2=/usr/bin/exiv2'\'' -- '\''/bin/sh'\'' '\''/srv/mediawiki/php-1.40.0-wmf.14/extensions/PagedTiffHandler/scripts/retrieveMetaData.sh'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=50; SB_CGROUP='\''/sys/fs/cgroup/memory/mediawiki/job'\''; SB_MEM_LIMIT=1073741824; SB_FILE_SIZE_LIMIT=536870912; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
5:dash:exec /usr/bin/dash:0
6:sh:exec /usr/bin/dash:0
6:sh:open /srv/mediawiki/php-1.40.0-wmf.14/extensions/PagedTiffHandler/scripts/retrieveMetaData.sh:3
6:sh:open64 info:3
legoktm@mwmaint1002:/tmp/lego-test$ ls
file.tiff  info
legoktm@mwmaint1002:/tmp/lego-test$ cat info 
7:tiffinfo:exec /usr/bin/tiffinfo:0
7:tiffinfo:open file.tiff:3
TIFF Directory at offset 0x19f9b9e (27237278)
  Subfile Type: (0 = 0x0)
  Image Width: 2746 Image Length: 4114
  Resolution: 600, 600 pixels/inch
  Bits/Sample: 8
  Compression Scheme: AdobeDeflate
  Photometric Interpretation: RGB color
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Rows/Strip: 1
  Planar Configuration: single image plane
  Software: Adobe Photoshop CC 2017 (Windows)
  DateTime: 2018:11:06 15:33:18

(I figured out the nologin shell via sudo -u www-data firejail --noprofile --trace /bin/sh)

thcipriani moved this task from Untriaged to Dec 2022 on the Wikimedia-production-error board.Jan 5 2023, 5:10 PM

• tstarling moved this task from Backlog to Usage and deployment on the Shellbox board.Feb 20 2024, 4:31 AM

shellbox/execvp: Permission denied on `MWScript.php importImages.php`Open, Needs TriagePublicPRODUCTION ERRORActions