Page MenuHomePhabricator

Come up with a decent method of declaring helm chart path/version in service repo
Closed, ResolvedPublic

Description

Deployment charts for all services are currently maintained in a central repo (operations/deployment-charts) external from each individual service repo, creating an external dependency problem with regard to testing deployments in CI. We should figure out a sane method for declaring which chart (path/version) should be used to deploy the service and maintain that declaration in the service repo.

Some possibilities:

  1. Since helm supports packaging, we could maintain a simple chart repo and update it upon each merge to operations/deployment-charts. Versioned chart URLs (e.g. https://deployment-charts.wikimedia.org/mathoid-1.0.0.tgz) could then be specified within some YAML file in the already established dist/pipeline directory—where each repo's blubber.yaml currently lives.
  2. Continue to use deploy repos that encapsulate both the service repo and the charts repo as submodules. This seems like a more heavy handed approach.
  3. Rethink the way we're managing charts, and instead of keeping them all central, allow for them to be defined directly within application repos. Define a central "cluster" chart that specifies all dependencies for the cluster. This eliminates the external dependency issue for testing individual repos, punting it downstream where a nice chart dependency mechanism is already provided by helm.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 3 2018, 6:42 PM
dduvall triaged this task as Medium priority.
dduvall moved this task from Backlog to Deployment Tooling on the Release Pipeline board.
dduvall updated the task description. (Show Details)Apr 3 2018, 7:30 PM

Added a third option.

demon added a comment.Apr 3 2018, 7:47 PM

Option 3 is what I was learning towards in my initial work.

I think we should seize this opportunity to incorporate the idea of getting rid of deploy repos altogether, so I'd be against option 2. Option 3 is a no-go in my mind as well since we want our software to be reused by other people as well, in which case I don't think having the chart there is conceptually good.

The current charts in the deployment-charts repo are exposed via an API, so are easy to get and that should be the case for CI as well. I guess my vote would go for option 1.

demon added a comment.Apr 4 2018, 7:58 PM

(1) works for me too, actually.

So if we're following option #1, we'll need to expose packaged charts somewhere central. Does https://integration.wikimedia.org/charts/ seem like an appropriate location for these, at least for the purposes of pre-prod helm deployments?

In today's SSD (aka CD Pipeline, aka Release Pipeline) Meeting, we agreed to implement option #1 after charts were published to https://releases.wikimedia.org/charts/ (done!).

Change 425914 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[mediawiki/services/mathoid@master] Define Helm chart used in pipeline deployments

https://gerrit.wikimedia.org/r/425914

Change 425914 merged by Mobrovac:
[mediawiki/services/mathoid@master] Define Helm chart used in pipeline deployments

https://gerrit.wikimedia.org/r/425914

Change 429234 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/mathoid@master] Helm Chart: Use v0.0.5 of the chart

https://gerrit.wikimedia.org/r/429234

Change 429234 merged by Mobrovac:
[mediawiki/services/mathoid@master] Helm Chart: Use v0.0.5 of the chart

https://gerrit.wikimedia.org/r/429234

dduvall closed this task as Resolved.Apr 27 2018, 9:47 PM