Page MenuHomePhabricator

Make pipelinelib able to update deployment chart image tags
Closed, ResolvedPublic

Description

It would be nice if developers didn't have to update the deployment-charts/helmfile.d values files every time they make a change to their containerized application.

Add a feature to pipeline lib so that after publishing a production image, it could upload a patchset with the newly published image tag to the service's values.yaml in each of helmfile.d/services/eqiad, helmfile.d/services/codfw, and helmfile.d/services/staging in the deployment-charts repo . This could be part of the publish step.

The publish pipeline of config.yaml could have a chart property added to it to define which chart's values to update:

publish:
    blubberfile: blubber.yaml
    stages:
      - name: test
      - name: candidate
        build: production
        publish:
          image: true
      - name: rehearsal
        deploy:
          chart: https://releases.wikimedia.org/charts/blubberoid-0.0.9.tgz
          image: '${candidate.imageName}'
          tag: '${candidate.imageTag}'
          test: true
      - name: production
        publish:
          image:
            id: '${candidate.imageID}'
            tags: [stable]
          chart: "blubberoid"

Event Timeline

That sounds nice!
I would suggest to update the image version in the helmfiles.d values (e.g. https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/helmfile.d/services/staging/blubberoid/values.yaml#33) instead of the chart itself, though. In general a new chart release is (or at least should be) only needed when substantial changes have been made to the containerized application (changes that would change the way the container is deployed/run, not changes to what is run inside the container).

That sounds nice!
I would suggest to update the image version in the helmfiles.d values (e.g. https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/helmfile.d/services/staging/blubberoid/values.yaml#33) instead of the chart itself, though. In general a new chart release is (or at least should be) only needed when substantial changes have been made to the containerized application (changes that would change the way the container is deployed/run, not changes to what is run inside the container).

Oh, yes, that is what we intended. I'll make it clearer in the task detail. Thanks!

jeena updated the task description. (Show Details)

Change 619833 had a related patch set uploaded (by Jeena Huneidi; owner: Jeena Huneidi):
[operations/deployment-charts@master] [WIP] Script to update image versions

https://gerrit.wikimedia.org/r/619833

We decided to add a new step to pipelinelib called 'promote' to do this task. It will clone the deployment-charts repo and run an update script to update the versions, then push a patch to gerrit.

Change 622231 had a related patch set uploaded (by Jeena Huneidi; owner: Jeena Huneidi):
[integration/pipelinelib@master] Enable deployment-charts update for published image

https://gerrit.wikimedia.org/r/622231

Change 619833 merged by jenkins-bot:
[operations/deployment-charts@master] Script to update image versions

https://gerrit.wikimedia.org/r/619833

Change 624963 had a related patch set uploaded (by Jeena Huneidi; owner: Jeena Huneidi):
[operations/deployment-charts@master] Make update_version.py work with python 3.4

https://gerrit.wikimedia.org/r/624963

Change 624972 had a related patch set uploaded (by Jeena Huneidi; owner: Jeena Huneidi):
[operations/puppet@production] ci/pipeline/builder.pp: Add ruamel package

https://gerrit.wikimedia.org/r/624972

Change 624963 merged by jenkins-bot:
[operations/deployment-charts@master] Make update_version.py work with python 3.5

https://gerrit.wikimedia.org/r/624963

Change 624972 merged by Dzahn:
[operations/puppet@production] ci/pipeline/builder.pp: Add ruamel package

https://gerrit.wikimedia.org/r/624972

Change 622231 merged by jenkins-bot:
[integration/pipelinelib@master] Enable deployment-charts update for published image

https://gerrit.wikimedia.org/r/622231

Change 627893 had a related patch set uploaded (by Jeena Huneidi; owner: Jeena Huneidi):
[blubber@master] .pipeline/config.yaml: add promote step

https://gerrit.wikimedia.org/r/627893

Change 627893 merged by jenkins-bot:
[blubber@master] .pipeline/config.yaml: add promote step

https://gerrit.wikimedia.org/r/627893

Change 627900 had a related patch set uploaded (by Jeena Huneidi; owner: Jeena Huneidi):
[integration/pipelinelib@master] fix typo

https://gerrit.wikimedia.org/r/627900

Change 627900 merged by jenkins-bot:
[integration/pipelinelib@master] fix typo

https://gerrit.wikimedia.org/r/627900

Change 629449 had a related patch set uploaded (by Jeena Huneidi; owner: Jeena Huneidi):
[operations/puppet@production] CI profile: move ruamel requirement to publisher

https://gerrit.wikimedia.org/r/629449

Change 629449 merged by Dzahn:
[operations/puppet@production] CI profile: move ruamel requirement to publisher

https://gerrit.wikimedia.org/r/629449