Page MenuHomePhabricator

New Service Request Shellbox
Open, HighPublic

Description

Description: T260330: RFC: PHP microservice for containerized shell execution
Timeline: <A desired timeline>
Diagram: <Link to an architectural diagram>
Technologies: PHP, bash
Point persons: @tstarling, @Legoktm


https://gerrit.wikimedia.org/r/q/topic:%22shellbox%22+(status:open%20OR%20status:merged)

Event Timeline

Dzahn added a subscriber: Dzahn.

A new namespace "shellbox" has been created in environments "staging-codfw" and "staging-eqiad".

root@deploy1002:~# kube_env admin staging-eqiad
root@deploy1002:~# kubectl get ns | grep shell
shellbox                       Active   66m
Dzahn triaged this task as High priority.May 4 2021, 10:11 PM

Change 692736 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/deployment-charts@master] Add helmfile.d for shellbox

https://gerrit.wikimedia.org/r/692736

Change 693957 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/dns@master] Add shellbox.svc.{codfw,eqiad}.wmnet

https://gerrit.wikimedia.org/r/693957

Change 693957 merged by Legoktm:

[operations/dns@master] Add shellbox.svc.{codfw,eqiad}.wmnet

https://gerrit.wikimedia.org/r/693957

Change 693959 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/puppet@production] Add shellbox to LVS

https://gerrit.wikimedia.org/r/693959

Change 693960 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/puppet@production] service: Switch shellbox to lvs_setup

https://gerrit.wikimedia.org/r/693960

Change 693961 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/puppet@production] service: Switch shellbox to monitoring_setup

https://gerrit.wikimedia.org/r/693961

Change 693962 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/puppet@production] service: Switch shellbox to production

https://gerrit.wikimedia.org/r/693962

Change 693965 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/dns@master] Add shellbox to discovery

https://gerrit.wikimedia.org/r/693965

Mentioned in SAL (#wikimedia-operations) [2021-05-26T17:16:36Z] <legoktm@deploy1002> Synchronized private/PrivateSettings.php: Set $wgShellboxSecretKey - T281423 (duration: 01m 14s)

Change 692736 merged by jenkins-bot:

[operations/deployment-charts@master] Add helmfile.d for shellbox

https://gerrit.wikimedia.org/r/692736

legoktm@deploy1002:~$ curl https://staging.svc.eqiad.wmnet:4008/index.php
File not found.
legoktm@deploy1002:~$ curl https://staging.svc.eqiad.wmnet:4008/index.php -I
HTTP/1.1 404 Not Found
date: Tue, 01 Jun 2021 21:37:02 GMT
server: wikimedia
x-powered-by: PHP/7.2.31-1+0~20200514.41+debian9~1.gbpe2a56b+wmf1+buster1
backend-timing: D=999 t=1622583422508263
content-type: text/html; charset=UTF-8
x-envoy-upstream-service-time: 1
transfer-encoding: chunked

Progress, I guess.

legoktm@deploy1002:~$ curl https://staging.svc.eqiad.wmnet:4008/index.php
File not found.
legoktm@deploy1002:~$ curl https://staging.svc.eqiad.wmnet:4008/index.php -I
HTTP/1.1 404 Not Found
date: Tue, 01 Jun 2021 21:37:02 GMT
server: wikimedia
x-powered-by: PHP/7.2.31-1+0~20200514.41+debian9~1.gbpe2a56b+wmf1+buster1
backend-timing: D=999 t=1622583422508263
content-type: text/html; charset=UTF-8
x-envoy-upstream-service-time: 1
transfer-encoding: chunked

Progress, I guess.

I think the problem is with the score image:

$ docker run --rm --entrypoint /bin/bash docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox:score  -c "ls -la /srv/app"
total 8
drwxr-xr-x 1 composer composer 4096 May 18 17:50 .
drwxr-xr-x 1 root     root     4096 May 18 17:50 ..

The php code is not included in the container, so obviously that file isn't found.

So I found the problem: in https://gerrit.wikimedia.org/r/c/mediawiki/libs/Shellbox/+/673136 we removed the run stanza from builder, thus making the requirements useless and discarded. As a consequence, nothing was actually copied to the prod variant 😿

The solution was to just tell blubber to copy the files directly instead.

Change 698581 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[mediawiki/libs/Shellbox@master] Fix the blubber.yaml file

https://gerrit.wikimedia.org/r/698581

Change 698581 merged by jenkins-bot:

[mediawiki/libs/Shellbox@master] Fix the blubber.yaml file

https://gerrit.wikimedia.org/r/698581