Page MenuHomePhabricator

Check/Rebuild all docker-pkg build docker images running on kubernetes
Closed, ResolvedPublic

Description

To ensure images are not running as root in kubernetes clusters we need the images to have numeric UIDs rather then user names set in the "USER" instructions (via Dockerfile).

We need to check and potentially rebuild all production-images that are running in our cluster (snapshot as of 2021-02-09):

  • coredns:1.5.2-1 (currently binds to port 53, needs root and is running in kube-system)
  • eventrouter:0.3.0-4
  • tiller:2.16.7-wmf1
  • nutcracker:latest
  • prometheus-statsd-exporter:0.0.5
  • prometheus-statsd-exporter:0.0.7
  • envoy:1.15.1-2

The following ones are only used in api-gateway, so they should be merged and deployed together in https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/664523

  • envoy-future:1.16.0
  • ratelimit:1.5.1
  • fluent-bit:1.5.3-0

What kubernetes basically does for validation is:

docker inspect $IMAGE_ID | jq '.[].Config.User' | { read u; echo $u;  if ! [[ "$u" =~ ^[0-9]+(:[0-9]+)?$ ]]; then echo "Nono"; fi; }

Details

SubjectRepoBranchLines +/-
operations/deployment-chartsmaster+6 -0
operations/deployment-chartsmaster+0 -1
operations/deployment-chartsmaster+0 -13
operations/puppetproduction+2 -1
operations/puppetproduction+2 -1
operations/deployment-chartsmaster+2 -1
operations/deployment-chartsmaster+3 -0
operations/deployment-chartsmaster+4 -4
operations/docker-images/production-imagesmaster+9 -2
operations/docker-images/production-imagesmaster+7 -1
operations/docker-images/production-imagesmaster+9 -3
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+2 -2
operations/docker-images/production-imagesmaster+7 -1
operations/docker-images/production-imagesmaster+8 -2
operations/docker-images/production-imagesmaster+7 -1
operations/docker-images/production-imagesmaster+9 -1
Show related patches Customize query in gerrit

Event Timeline

JMeybohm renamed this task from Check/Rebuild all dpcker-pkg build docker images running on kubernetes to Check/Rebuild all docker-pkg build docker images running on kubernetes.Feb 10 2021, 6:01 PM
JMeybohm triaged this task as High priority.
JMeybohm updated the task description. (Show Details)

Change 664095 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/docker-images/production-images@master] tiller: Run tiller as user nobody

https://gerrit.wikimedia.org/r/664095

Change 664096 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/docker-images/production-images@master] eventrouter: Use numeric UID

https://gerrit.wikimedia.org/r/664096

Change 664097 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/docker-images/production-images@master] fluent-bit: Use numeric UID

https://gerrit.wikimedia.org/r/664097

Change 664098 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/docker-images/production-images@master] ratelimit: Use numeric UID

https://gerrit.wikimedia.org/r/664098

Change 664095 merged by JMeybohm:
[operations/docker-images/production-images@master] tiller: Run tiller as user nobody

https://gerrit.wikimedia.org/r/664095

Change 664097 merged by JMeybohm:
[operations/docker-images/production-images@master] fluent-bit: Use numeric UID

https://gerrit.wikimedia.org/r/664097

Change 664096 merged by JMeybohm:
[operations/docker-images/production-images@master] eventrouter: Use numeric UID

https://gerrit.wikimedia.org/r/664096

Change 664098 merged by JMeybohm:
[operations/docker-images/production-images@master] ratelimit: Use numeric UID

https://gerrit.wikimedia.org/r/664098

Change 664522 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] eventrouter: Bump image version to 0.3.0-5

https://gerrit.wikimedia.org/r/664522

Change 664523 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] api-gateway: Update fluent-bit and ratelimit images

https://gerrit.wikimedia.org/r/664523

Change 664525 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] admin_ng: Update tiller image to 2.16.7-3

https://gerrit.wikimedia.org/r/664525

Change 664526 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] admin: Update tiller image to 2.16.7-3

https://gerrit.wikimedia.org/r/664526

Change 664522 merged by jenkins-bot:
[operations/deployment-charts@master] eventrouter: Bump image version to 0.3.0-5

https://gerrit.wikimedia.org/r/664522

Change 664525 merged by JMeybohm:
[operations/deployment-charts@master] admin_ng: Update tiller image to 2.16.7-3

https://gerrit.wikimedia.org/r/664525

Change 664526 merged by JMeybohm:
[operations/deployment-charts@master] admin: Update tiller image to 2.16.7-3

https://gerrit.wikimedia.org/r/664526

Change 664818 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] initialize_cluster: Update tiller image to 2.16.7-3

https://gerrit.wikimedia.org/r/664818

Change 664818 merged by jenkins-bot:
[operations/deployment-charts@master] initialize_cluster: Update tiller image to 2.16.7-3

https://gerrit.wikimedia.org/r/664818

Change 664853 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/docker-images/production-images@master] envoy: Run as user nobody

https://gerrit.wikimedia.org/r/664853

Change 664864 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/docker-images/production-images@master] prometheus-statsd-exporter: Run as nobody

https://gerrit.wikimedia.org/r/664864

Change 664865 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/docker-images/production-images@master] nutcracker: Run as user nobody

https://gerrit.wikimedia.org/r/664865

Change 666116 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] deployment_server: Update default envoy image to 1.15.1-4

https://gerrit.wikimedia.org/r/666116

Change 666117 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] deployment_server: Update default prometheus-statsd-exporter image to 0.0.9

https://gerrit.wikimedia.org/r/666117

Change 666122 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] wikifeeds: Update envoy to 1.15.1-4 and statsd exporter to 0.0.9

https://gerrit.wikimedia.org/r/666122

Change 666123 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] Remove deployment local definitions of statsd exporter version

https://gerrit.wikimedia.org/r/666123

Change 666124 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] wikifeeds: Switch back to the global tls.image_version

https://gerrit.wikimedia.org/r/666124

Change 664853 merged by JMeybohm:
[operations/docker-images/production-images@master] envoy: Run as user nobody

https://gerrit.wikimedia.org/r/664853

Change 664864 merged by JMeybohm:
[operations/docker-images/production-images@master] prometheus-statsd-exporter: Run as nobody

https://gerrit.wikimedia.org/r/664864

Change 664865 merged by JMeybohm:
[operations/docker-images/production-images@master] nutcracker: Run as user nobody

https://gerrit.wikimedia.org/r/664865

Change 664523 merged by jenkins-bot:
[operations/deployment-charts@master] api-gateway: Update mutiple sidecar images

https://gerrit.wikimedia.org/r/664523

Change 666148 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] api-gateway: Pin nutcracker version to 0.0.4

https://gerrit.wikimedia.org/r/666148

Change 666148 merged by jenkins-bot:
[operations/deployment-charts@master] api-gateway: Pin nutcracker version to 0.0.4

https://gerrit.wikimedia.org/r/666148

Change 666122 merged by jenkins-bot:
[operations/deployment-charts@master] wikifeeds: Update envoy to 1.15.1-4 and statsd exporter to 0.0.9

https://gerrit.wikimedia.org/r/666122

All images build.
api-gateway is running updated envoy-future, nutcracker, ratelimit, fluent-bit and statsd-exporter since yesterday.
wikifeeds is running updated envoy and statsd-exporter since ~ 10 minutes.

Will keep it that way for now and switch the default later today or tomorrow morning (followed by a deploy of all services).

Change 666116 merged by JMeybohm:
[operations/puppet@production] deployment_server: Update default envoy image to 1.15.1-4

https://gerrit.wikimedia.org/r/666116

Change 666117 merged by JMeybohm:
[operations/puppet@production] deployment_server: Update default prometheus-statsd-exporter image to 0.0.9

https://gerrit.wikimedia.org/r/666117

Change 666123 merged by jenkins-bot:
[operations/deployment-charts@master] Remove deployment local definitions of statsd exporter version

https://gerrit.wikimedia.org/r/666123

Change 666124 merged by jenkins-bot:
[operations/deployment-charts@master] wikifeeds: Switch back to the global tls.image_version

https://gerrit.wikimedia.org/r/666124

Change 666612 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] changeprop(-jobqueue): Don't run nutcracker:latest

https://gerrit.wikimedia.org/r/666612

Change 666612 merged by jenkins-bot:
[operations/deployment-charts@master] changeprop(-jobqueue): Don't run nutcracker:latest

https://gerrit.wikimedia.org/r/666612

Updated images have been rolled out to all clusters as of today.