Page MenuHomePhabricator
Create Task
Maniphest T256020

Access to the thanos-swift cluster for ChartMuseum
Closed, ResolvedPublic
Actions

Description

I would like to use the thanos swift cluster as storage back-end for chartmuseum (https://chartmuseum.com/docs/#using-with-openstack-object-storage). If the S3 API is a better choice, that's fine too.

  • ChartMuseum will be deployed active/active in both DCs.
  • ChartMuseum will act as proxy to swift, so no direct client connections.
  • Currently we have ~291 objects with a total of ~3.5MB which I expect to slowly increase.
  • I/O will be mostly reads, writes will happen only on push of a new chart.
  • The data that will be stored can be (with some effort) regenerated from git, but swift will be the "authoritative" storage for the artifacts (chart tgz's).

Details

SubjectRepoBranchLines +/-
profile: thanos::swift::frontend add account for chartmuseumoperations/puppetproduction+6 -0
thanos::swift add chartmuseum account keylabs/privatemaster+1 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

JMeybohm created this task.Jun 22 2020, 4:11 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 22 2020, 4:11 PM
fgiunchedi added a comment.Jun 23 2020, 7:45 AM

Generally LGTM as a use case.

Is there PII/private data in the charts or you expect to? I'm pointing this out because while connections to the swift frontend must use https, the multi-site replication happens in plain text.

re: API, both S3 (with "v2 signatures") and swift should work. For the latter we are using "tempauth" authentication (not keystone) in case that makes a difference on the ChartMuseum side. The easiest is likely to set things up and give both a try though!

JMeybohm added a comment.Jun 23 2020, 8:01 AM

We don't expect private data in the charts at all.
In addition, they are already publicly accessible via https://releases.wikimedia.org/charts/ and https://gerrit.wikimedia.org/g/operations/deployment-charts ofc.

I'm still working on the puppet integration for ChartMuseum but I'm happy to test beforehand to make a decision on what API to use/configure. Do you need anything from me/how do we proceed?

fgiunchedi added a comment.Jun 23 2020, 10:48 AM
In T256020#6247552, @JMeybohm wrote:

We don't expect private data in the charts at all.
In addition, they are already publicly accessible via https://releases.wikimedia.org/charts/ and https://gerrit.wikimedia.org/g/operations/deployment-charts ofc.

I'm still working on the puppet integration for ChartMuseum but I'm happy to test beforehand to make a decision on what API to use/configure. Do you need anything from me/how do we proceed?

The easiest way is to create an account at hieradata/common/profile/thanos/swift.yaml and the corresponding private material both in private.git on the puppetmaster and on the public/labs private.git. After that is merged I can help with deployment (essentially a rolling restart of swift-proxy on thanos-fe*).

Then you can create containers with the credentials above, there's SSD-based storage too available for containers which we can do in this case.

gerritbot added a comment.Jun 24 2020, 10:52 AM

Change 607467 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[operations/puppet@production] profile: thanos::swift::frontend add account for chartmuseum

https://gerrit.wikimedia.org/r/607467

gerritbot added a project: Patch-For-Review.Jun 24 2020, 10:52 AM
gerritbot added a comment.Jun 24 2020, 11:09 AM

Change 607468 had a related patch set uploaded (by JMeybohm; owner: JMeybohm):
[labs/private@master] thanos::swift add chartmuseum account key

https://gerrit.wikimedia.org/r/607468

JMeybohm added a comment.Jun 24 2020, 11:16 AM

Commit in private is e427c266f2d6ac0a937bf5d972b759933a9f9a18

gerritbot added a comment.Jun 24 2020, 2:51 PM

Change 607468 merged by JMeybohm:
[labs/private@master] thanos::swift add chartmuseum account key

https://gerrit.wikimedia.org/r/607468

gerritbot added a comment.Jun 24 2020, 2:52 PM

Change 607467 merged by JMeybohm:
[operations/puppet@production] profile: thanos::swift::frontend add account for chartmuseum

https://gerrit.wikimedia.org/r/607467

JMeybohm mentioned this in rLPRI3f7f7e1f91f0: thanos::swift add chartmuseum account key.Jun 24 2020, 2:52 PM
Maintenance_bot removed a project: Patch-For-Review.Jun 24 2020, 3:10 PM
Stashbot added a comment.Jun 24 2020, 3:20 PM

Mentioned in SAL (#wikimedia-operations) [2020-06-24T15:20:40Z] <jayme> rolling restart of swift-proxy on thanos-fe[2001-2003].codfw.wmnet,thanos-fe[1001-1003].eqiad.wmnet - T256020

JMeybohm closed this task as Resolved.Jun 26 2020, 3:25 PM
JMeybohm claimed this task.

This is done and the account is working, thanks @fgiunchedi !

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits