Page MenuHomePhabricator

Move helm chart repository out of git
Closed, ResolvedPublic

Description

It's a pain to always have to rebase CRs because the index.yaml has changed.

My suggestion is either have CI build the index or, even better, have CI do "helm package" as well.

We could also run an instance of https://chartmuseum.com/ (with swift backend?) and use the "helm push" plugin to package and push helm charts (authenticated) via CI.

Things that do not look nice:

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+2 -0
operations/puppetproduction+0 -17
operations/deployment-chartsmaster+0 -5 K
operations/puppetproduction+3 -3
operations/deployment-chartsmaster+0 -4
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+5 -5
operations/deployment-chartsmaster+0 -200
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+2 -23
operations/puppetproduction+15 -2
operations/puppetproduction+1 -1
operations/puppetproduction+4 -4
operations/puppetproduction+3 -3
operations/puppetproduction+58 -0
operations/puppetproduction+7 -0
operations/puppetproduction+5 -0
operations/dnsmaster+4 -0
operations/puppetproduction+2 -2
operations/puppetproduction+4 -0
operations/puppetproduction+1 -1
operations/puppetproduction+1 -1
labs/privatemaster+0 -0
labs/privatemaster+8 -0
operations/dnsmaster+8 -0
operations/puppetproduction+25 -0
labs/privatemaster+3 -0
operations/puppetproduction+224 -1
operations/debs/chartmuseummaster+528 -0
operations/debs/chartmuseummaster+511 -0
integration/configmaster+5 -0
Show related patches Customize query in gerrit

Event Timeline

akosiaris triaged this task as Medium priority.May 29 2020, 2:17 PM
akosiaris added a subscriber: akosiaris.

My suggestion is either have CI build the index or, even better, have CI do "helm package" as well.

+1

We could also run an instance of https://chartmuseum.com/ (with swift backend?) and use the "helm push" plugin to package and push helm charts (authenticated) via CI.

+1 as well, altough we probably don't even need the helm push plugin (a bit less code to maintain, nothing against the plugin per se) and just do a curl --data-binary "@mychart-0.1.0.tgz" http://chartmuseum.discovery.wmnet:8080/api/charts

Change 605940 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/debs/chartmuseum@master] Initial commit of debian directory

https://gerrit.wikimedia.org/r/605940

Change 605945 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[integration/config@master] Debian glue for operations/debs/chartmuseum

https://gerrit.wikimedia.org/r/605945

Change 605945 merged by jenkins-bot:
[integration/config@master] Debian glue for operations/debs/chartmuseum

https://gerrit.wikimedia.org/r/605945

Change 605940 merged by jenkins-bot:
[operations/debs/chartmuseum@master] Initial commit of debian directory

https://gerrit.wikimedia.org/r/605940

Change 606956 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] WIP: chartmusum: Add initial module, profile and role

https://gerrit.wikimedia.org/r/606956

I need to make decisions regarding TLS and storage:

Do we want to use envoy here (ChartMuseum is able to TLS termination as well)? I think it might be desirable as it prevents us from writing/maintaining TLS stuff in the ChartMuseum class/profile and the overhead is probably irrelevant here.

Storage:
It would be nice to have ChartMuseum active/active and it supports Swift as backend. I've asked @fgiunchedi about the multi-dc swift cluster that is used for thanos.

If that's not possible for whatever reasons (mostly reliability I guess), we would need to go active/passive and periodically sync the charts between the DCs.

@akosiaris what do you think?

I need to make decisions regarding TLS and storage:

Do we want to use envoy here (ChartMuseum is able to TLS termination as well)? I think it might be desirable as it prevents us from writing/maintaining TLS stuff in the ChartMuseum class/profile and the overhead is probably irrelevant here.

I 'd go with the envoy TLS termination for:

  • consistency's sake
  • one less part of the chartmuseum codebase then we end up exercising and having to deal with in terms of maintenance, bugs, etc.

Storage:
It would be nice to have ChartMuseum active/active and it supports Swift as backend. I've asked @fgiunchedi about the multi-dc swift cluster that is used for thanos.

If that's not possible for whatever reasons (mostly reliability I guess), we would need to go active/passive and periodically sync the charts between the DCs.

@akosiaris what do you think?

/me hopes we do active/active as it sounds preferable on our side.

Turns out our swift cluster does only support Swift V1 Auth, which ChartMuseum does not. I've tried the S3 API as well but that only supports "v2 signatures" which ChartMuseum ... does not (because the official aws-sdk-go only supports v4 signatures).

Fortunately the library ChartMuseum uses for OpenStack/Swift, supports V1 Auth (although not "automatically" as v2 and v3). Implementation looked pretty straight forward, so I ended up with:

Change 608088 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/debs/chartmuseum@master] Add patches for swift auth and bind interface

https://gerrit.wikimedia.org/r/608088

Change 608088 merged by jenkins-bot:
[operations/debs/chartmuseum@master] Add patches for swift auth and bind interface

https://gerrit.wikimedia.org/r/c/operations/debs/chartmuseum/ /608088

Mentioned in SAL (#wikimedia-operations) [2020-07-02T09:28:46Z] <jayme> imported chartmuseum_0.12.0-2 to buster-wikimedia - T253843

Change 606956 merged by JMeybohm:
[operations/puppet@production] chartmuseum: Add initial module, profile and role

https://gerrit.wikimedia.org/r/c/operations/puppet/ /606956

Change 609121 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[labs/private@master] secret: add dummy key for helm-charts (chartmuseum)

https://gerrit.wikimedia.org/r/c/labs/private/ /609121

Change 609122 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] Add certificate for helm-charts (chartmuseum)

https://gerrit.wikimedia.org/r/c/operations/puppet/ /609122

Change 609164 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/dns@master] Introduce chartmuseum[12]001

https://gerrit.wikimedia.org/r/c/operations/dns/ /609164

Change 609165 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/dns@master] Add helm-charts discovery records

https://gerrit.wikimedia.org/r/c/operations/dns/ /609165

Change 609121 merged by JMeybohm:
[labs/private@master] secret: add dummy key for helm-charts (chartmuseum)

https://gerrit.wikimedia.org/r/609121

Change 609122 merged by JMeybohm:
[operations/puppet@production] Add certificate for helm-charts (chartmuseum)

https://gerrit.wikimedia.org/r/609122

Change 609164 merged by JMeybohm:
[operations/dns@master] Introduce chartmuseum[12]001

https://gerrit.wikimedia.org/r/609164

Change 609760 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] Switch role for chartmuseum hosts to chartmuseum

https://gerrit.wikimedia.org/r/609760

Change 609781 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[labs/private@master] Add fake secrets for chartmuseum

https://gerrit.wikimedia.org/r/609781

Change 609781 merged by JMeybohm:
[labs/private@master] Add fake secrets for chartmuseum

https://gerrit.wikimedia.org/r/609781

Change 609785 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[labs/private@master] Fix chartmuseum hiera path (common/profile -> role/common)

https://gerrit.wikimedia.org/r/609785

Change 609785 merged by JMeybohm:
[labs/private@master] Fix chartmuseum hiera path (common/profile -> role/common)

https://gerrit.wikimedia.org/r/609785

Change 609760 merged by JMeybohm:
[operations/puppet@production] Switch role for chartmuseum hosts to chartmuseum

https://gerrit.wikimedia.org/r/609760

Change 609792 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] chartmuseum: Ensure envoy connects via IPv4

https://gerrit.wikimedia.org/r/609792

Change 609792 merged by JMeybohm:
[operations/puppet@production] chartmuseum: Ensure envoy connects via IPv4

https://gerrit.wikimedia.org/r/609792

Change 609829 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] prometheus: Switch chartmuseum scrape target to HTTPS

https://gerrit.wikimedia.org/r/609829

Change 609829 merged by JMeybohm:
[operations/puppet@production] prometheus: Switch chartmuseum scrape target to HTTPS

https://gerrit.wikimedia.org/r/609829

Change 609984 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] chartmuseum: fix typo in config template

https://gerrit.wikimedia.org/r/609984

Change 609984 merged by JMeybohm:
[operations/puppet@production] chartmuseum: fix typo in config template

https://gerrit.wikimedia.org/r/609984

Change 609403 had a related patch set uploaded (by JMeybohm; owner: Alexandros Kosiaris):
[operations/puppet@production] Add discovery records for chartmuseum

https://gerrit.wikimedia.org/r/609403

Change 613634 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] modules/systemd: Allow to define EnvironmentFile for timers

https://gerrit.wikimedia.org/r/613634

Change 613635 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] charmuseum: Add systemd timer to package and push charts

https://gerrit.wikimedia.org/r/613635

Change 609165 merged by JMeybohm:
[operations/dns@master] Add helm-charts discovery record

https://gerrit.wikimedia.org/r/609165

Change 616447 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] ATS: Add backend for helm-charts.wikimedia.org

https://gerrit.wikimedia.org/r/616447

Change 616447 merged by JMeybohm:
[operations/puppet@production] ATS: Add backend for helm-charts.wikimedia.org

https://gerrit.wikimedia.org/r/616447

Change 613634 merged by JMeybohm:
[operations/puppet@production] modules/systemd: Allow to define EnvironmentFile for timers

https://gerrit.wikimedia.org/r/613634

Change 613635 merged by JMeybohm:
[operations/puppet@production] chartmuseum: Add systemd timer to package and push charts

https://gerrit.wikimedia.org/r/613635

Change 616473 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] chartmuseum: Run timer command (ExecStart) in a shell

https://gerrit.wikimedia.org/r/616473

Change 616473 merged by JMeybohm:
[operations/puppet@production] chartmuseum: Run timer command (ExecStart) in a shell

https://gerrit.wikimedia.org/r/616473

Change 617455 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] chartmuseum: Fix inverted hostname and IP

https://gerrit.wikimedia.org/r/617455

Change 617455 merged by JMeybohm:
[operations/puppet@production] chartmuseum: Fix inverted hostname and IP

https://gerrit.wikimedia.org/r/617455

Change 617652 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] Switch helmfiles to use chartmuseum repository

https://gerrit.wikimedia.org/r/617652

Change 617659 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] chartmuseum: Change repository name to stable

https://gerrit.wikimedia.org/r/617659

Change 617659 merged by JMeybohm:
[operations/puppet@production] chartmuseum: Change repository name to stable

https://gerrit.wikimedia.org/r/617659

Change 617673 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] helm: Allow multiple helm repositories

https://gerrit.wikimedia.org/r/617673

Change 617673 merged by JMeybohm:
[operations/puppet@production] helm: Allow multiple helm repositories

https://gerrit.wikimedia.org/r/617673

Change 617693 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] Remove the repository definition from helmfiles

https://gerrit.wikimedia.org/r/617693

Change 617694 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] changeprop: Update repository URL in requirements

https://gerrit.wikimedia.org/r/617694

Change 617695 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] eventgate: Update repository URL in requirements

https://gerrit.wikimedia.org/r/617695

Change 617652 merged by jenkins-bot:
[operations/deployment-charts@master] Remove the repository definition from helmfiles

https://gerrit.wikimedia.org/r/617652

Change 617699 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] mathoid: Change staging chart back to stable

https://gerrit.wikimedia.org/r/617699

Change 617699 merged by jenkins-bot:
[operations/deployment-charts@master] mathoid: Change staging chart back to stable

https://gerrit.wikimedia.org/r/617699

Change 617693 merged by jenkins-bot:
[operations/deployment-charts@master] Remove the repository definition from helmfiles

https://gerrit.wikimedia.org/r/617693

Change 617695 merged by JMeybohm:
[operations/deployment-charts@master] eventgate: Update repository URL in requirements

https://gerrit.wikimedia.org/r/617695

Change 617694 merged by jenkins-bot:
[operations/deployment-charts@master] changeprop: Update repository URL in requirements

https://gerrit.wikimedia.org/r/617694

Change 618347 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] blubberoid: remove out-dated repositories definition

https://gerrit.wikimedia.org/r/618347

Change 618352 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] releases: Remove deployment-charts repo

https://gerrit.wikimedia.org/r/618352

Change 618347 merged by jenkins-bot:
[operations/deployment-charts@master] blubberoid: remove out-dated repositories definition

https://gerrit.wikimedia.org/r/618347

Change 618352 merged by JMeybohm:
[operations/puppet@production] releases: Remove deployment-charts repo

https://gerrit.wikimedia.org/r/618352

Change 619433 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/deployment-charts@master] Remove helm repo (index.yaml and chart tars) from git

https://gerrit.wikimedia.org/r/619433

Change 619433 merged by jenkins-bot:
[operations/deployment-charts@master] Remove helm repo (index.yaml and chart tars) from git

https://gerrit.wikimedia.org/r/619433

Change 619434 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] releases: Remove absend ressources

https://gerrit.wikimedia.org/r/619434

Finally removed the chart tgz's from git and the mirror from releases. Closing.

Change 619434 merged by Dzahn:
[operations/puppet@production] releases: Remove absent resources

https://gerrit.wikimedia.org/r/619434

Change 623958 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] configmaster: add helm-charts to disc_desired_state.py

https://gerrit.wikimedia.org/r/623958

Change 623958 merged by JMeybohm:
[operations/puppet@production] configmaster: add helm-charts to disc_desired_state.py

https://gerrit.wikimedia.org/r/623958