Page MenuHomePhabricator

Support Openstack Swift APIs via the radosgw
Open, Needs TriagePublic

Description

The Trove docs assume the presence of an object store (Swift). It looks like this is only used for backups but I also haven't found any clear evidence that Trove can run in a setup without object stores.

This is a good excuse to get Swift up and running with Trove its initial user.

Event Timeline

Apparently we can continue to put all our eggs in the ceph basket in Swift as well https://keithtenzer.com/2017/03/30/openstack-swift-integration-with-ceph/

Apparently we can continue to put all our eggs in the ceph basket in Swift as well https://keithtenzer.com/2017/03/30/openstack-swift-integration-with-ceph/

That's what I was thinking, although it's a bit questionable to have the primary databases on ceph-hosted VMs and then 'back them up' to ceph-backed swift when it's the same ceph.

A separate pool is one level of separation, but it's not physical. It seems a second cluster for some uses would make sense for that, which would reduce our skill needs without sacrificing redundancy.

I don't think we need to actually run swift in wmcs; radosgw supports swift APIs.

https://docs.ceph.com/en/latest/radosgw/index.html

And it can consume Keystone for auth:

https://docs.ceph.com/en/latest/radosgw/keystone/

Of course VMs can't actually talk to ceph APIs, but maybe with haproxy that doesn't matter?

Change 682317 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] cloud-vps ceph: initial config for adding radosgw to control nodes

https://gerrit.wikimedia.org/r/682317

Change 682317 merged by Andrew Bogott:

[operations/puppet@production] cloud-vps ceph: initial config for adding radosgw to control nodes

https://gerrit.wikimedia.org/r/682317

Change 682729 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Ceph: Try to re-centralize radosgw keyrings

https://gerrit.wikimedia.org/r/682729

Change 682729 merged by Andrew Bogott:

[operations/puppet@production] Ceph: Try to re-centralize radosgw keyrings

https://gerrit.wikimedia.org/r/682729

Andrew renamed this task from Swift before Trove? to Support Openstack Swift APIs via the radosgw.Wed, Apr 28, 4:18 PM
Andrew updated the task description. (Show Details)

We discussed this in our weekly meeting today. Issues to sort out before radosgw goes in eqiad1 are:

  1. Ceph pg audit to make sure we can support the new pools that radosgw needs
  1. Moving haproxy (and possibly radosgw) onto separate hardware so that the Swift bandwidth doesn't overwhelm the other openstack APIs.

I recommend we look at erasure coding the pools for Swift, since I don't know if I've said it directly related to this yet https://ceph.io/planet/erasure-coding-in-ceph/

That would give us more storage for the number of OSDs involved. Erasure coding for RBD was relatively new when we deployed ceph, so we avoided it for the safer replicated version, but it's quite mature and common in a radosgw config.

I recommend we look at erasure coding the pools for Swift, since I don't know if I've said it directly related to this yet https://ceph.io/planet/erasure-coding-in-ceph/

I think that's pretty simple -- radosgw creates the pools but you can convert them to erasure after the fact. Do you have an opinion about what ratio we should use?

Nope, I'd probably google around for what "seems good", unfortunately.