Page MenuHomePhabricator

php-composer-security-docker currently failing with "curl: command not found" error
Closed, ResolvedPublic

Description

php-composer-security-docker has been failing for a while as the docker it uses (docker-registry.wikimedia.org/releng/composer-php72:0.2.1-s1) can't seem to find curl. Error output we're receiving:

+ curl -i -H 'Accept: text/plain' https://php-security-checker.wmflabs.org/check_lock -F lock=@composer.lock -o results.check
/srv/composer/security-check: line 14: curl: command not found
Build step 'Execute shell' marked build as failure

And:

$ docker run --rm -ti --entrypoint /bin/bash docker-registry.wikimedia.org/releng/composer-php72:0.2.1-s1
nobody@cde023c63389:/$ which curl
nobody@cde023c63389:/$ command -v curl
nobody@cde023c63389:/$

Does curl need to be explicitly installed during the apt-get install for this image now? Seemed to work fine in the past.

Details

Related Gerrit Patches:

Event Timeline

sbassett created this task.Oct 10 2019, 9:10 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 10 2019, 9:10 PM
sbassett triaged this task as High priority.Oct 10 2019, 9:11 PM
sbassett added a comment.EditedOct 10 2019, 9:26 PM

Seems to fix it locally -
Dockerfile:

FROM docker-registry.wikimedia.org/releng/composer-php72:0.2.1-s1

USER root
RUN apt-get update && \
  apt-get install --yes \
  curl

Test:

$ docker run --rm -ti --entrypoint /bin/bash test235221
root@9a5219a953d9:/# which curl
/usr/bin/curl

So maybe curl just needs to be explicitly installed within php72 or composer-php72?

Probably a side effect of T234623, due to the new image not having curl installed.

Change 543169 had a related patch set uploaded (by SBassett; owner: SBassett):
[integration/config@master] Disable email reporting to security admin feed

https://gerrit.wikimedia.org/r/543169

Change 543169 merged by jenkins-bot:
[integration/config@master] Disable email reporting to security admin feed

https://gerrit.wikimedia.org/r/543169

Jdlrobson added a subtask: Restricted Task.EditedMon, Nov 25, 6:24 PM

This is also blocking T236589 during the npm run docs command.
The error can be seen here: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/MobileFrontend/+/550516/

hashar added a subscriber: Jdforrester-WMF.

The security check job relies on a container having the entry point:

security-check.sh
#!/usr/bin/env bash

umask 002

set -euxo pipefail

cd /src

# If there's no lock file, then generate one
if [ ! -f "composer.lock" ]; then
    composer install --no-progress --prefer-dist
fi

curl -i -H "Accept: text/plain" https://php-security-checker.wmflabs.org/check_lock -F lock=@composer.lock -o results.check
cat results.check && grep -iF "X-Alerts: 0" results.check

Originally that got introduced in the container releng/composer-php70 which has:

COPY security-check.sh /srv/composer/security-check

RUN {{ "jq curl" | apt_install }}

For the php 7.2 migration, we went creating a container releng/composer-php72 which only copy the entry point script but miss the additional packages:

FROM {{ "composer-php70" | image_tag }} as composer
...
COPY --from=composer /srv/composer /srv/composer

RUN {{ "jq" | apt_install }}

And thus we are missing curl :D

Fun thing releng/composer-php70 had curl explicitly added with ebe0aec56614d61797a599d4825ecf9803c1a7b1

Change 553102 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: add curl for composer-security-check

https://gerrit.wikimedia.org/r/553102

Change 553103 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Fix php-composer-security-docker missing curl

https://gerrit.wikimedia.org/r/553103

hashar claimed this task.Tue, Nov 26, 12:57 PM

Thanks, @hashar. Should we file a separate bug for mwext-node10-rundoc-docker? Which is also failing for this same reason, per T235221#5690746 above.

Change 553102 merged by jenkins-bot:
[integration/config@master] docker: add curl for composer-security-check

https://gerrit.wikimedia.org/r/553102

Mentioned in SAL (#wikimedia-releng) [2019-11-26T15:50:51Z] <James_F> Docker: Pushing composer-php72:0.2.2 and composer-php73:0.1.7 for T235221

Change 553103 merged by jenkins-bot:
[integration/config@master] Fix php-composer-security-docker missing curl

https://gerrit.wikimedia.org/r/553103

The job is now working; do you want the notifications re-instated?

Thanks, @hashar. Should we file a separate bug for mwext-node10-rundoc-docker? Which is also failing for this same reason, per T235221#5690746 above.

Yes please, that's nothing to do with this (and I'm not fully convinced it's a good idea).

sbassett added a comment.EditedTue, Nov 26, 4:20 PM

The job is now working; do you want the notifications re-instated?

Sure, I can probably do that.

Yes please, that's nothing to do with this (and I'm not fully convinced it's a good idea).

Filed: T239246. I guess more discussion needs to happen on this topic.

Also, with the revert now merged, I think this task can probably be resolved.

Jdforrester-WMF closed this task as Resolved.Tue, Nov 26, 7:24 PM