Page MenuHomePhabricator

Come up with a decent method of declaring helm chart path/version in service repo
Closed, ResolvedPublic

Description

Deployment charts for all services are currently maintained in a central repo (operations/deployment-charts) external from each individual service repo, creating an external dependency problem with regard to testing deployments in CI. We should figure out a sane method for declaring which chart (path/version) should be used to deploy the service and maintain that declaration in the service repo.

Some possibilities:

  1. Since helm supports packaging, we could maintain a simple chart repo and update it upon each merge to operations/deployment-charts. Versioned chart URLs (e.g. https://deployment-charts.wikimedia.org/mathoid-1.0.0.tgz) could then be specified within some YAML file in the already established dist/pipeline directory—where each repo's blubber.yaml currently lives.
  2. Continue to use deploy repos that encapsulate both the service repo and the charts repo as submodules. This seems like a more heavy handed approach.
  3. Rethink the way we're managing charts, and instead of keeping them all central, allow for them to be defined directly within application repos. Define a central "cluster" chart that specifies all dependencies for the cluster. This eliminates the external dependency issue for testing individual repos, punting it downstream where a nice chart dependency mechanism is already provided by helm.

Event Timeline

dduvall triaged this task as Medium priority.
dduvall moved this task from Backlog to Deployment Tooling on the Release Pipeline board.
dduvall moved this task from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.

Option 3 is what I was learning towards in my initial work.

I think we should seize this opportunity to incorporate the idea of getting rid of deploy repos altogether, so I'd be against option 2. Option 3 is a no-go in my mind as well since we want our software to be reused by other people as well, in which case I don't think having the chart there is conceptually good.

The current charts in the deployment-charts repo are exposed via an API, so are easy to get and that should be the case for CI as well. I guess my vote would go for option 1.

(1) works for me too, actually.

So if we're following option #1, we'll need to expose packaged charts somewhere central. Does https://integration.wikimedia.org/charts/ seem like an appropriate location for these, at least for the purposes of pre-prod helm deployments?

In today's SSD (aka CD Pipeline, aka Release Pipeline) Meeting, we agreed to implement option #1 after charts were published to https://releases.wikimedia.org/charts/ (done!).

Change 425914 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[mediawiki/services/mathoid@master] Define Helm chart used in pipeline deployments

https://gerrit.wikimedia.org/r/425914

Change 425914 merged by Mobrovac:
[mediawiki/services/mathoid@master] Define Helm chart used in pipeline deployments

https://gerrit.wikimedia.org/r/425914

Change 429234 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/mathoid@master] Helm Chart: Use v0.0.5 of the chart

https://gerrit.wikimedia.org/r/429234

Change 429234 merged by Mobrovac:
[mediawiki/services/mathoid@master] Helm Chart: Use v0.0.5 of the chart

https://gerrit.wikimedia.org/r/429234