Page MenuHomePhabricator
Create Task
Maniphest T293752

cloud ceph: refactor rbd client puppet profiles
Closed, ResolvedPublic
Actions

Description

To better support diversity of use cases, we decided a new layout for the puppet code that supports ceph rbd clients.

See https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/ceph_client_refactor for details.

deploymentceph identityidentity consumertypestatepatches
codfw1devcomputenova-compute @ cloudvirtsnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/737936
eqiad1computenova-compute @ cloudvirtsnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/737936
codfw1devglance-imagesglance-api @ cloudcontrolnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/737925
eqiad1glance-imagesglance-api @ cloudcontrolnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/737936
codfw1devcindercinder-api @ cloudcontrolnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/738411 https://gerrit.wikimedia.org/r/c/operations/puppet/+/738412
eqiad1cindercinder-api @ cloudcontrolnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/738413 https://gerrit.wikimedia.org/r/c/operations/puppet/+/738414
codfw1devcinderlibvirt @ cloudvirtlibvirt secret[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/739474 https://gerrit.wikimedia.org/r/c/operations/puppet/+/739516
eqiad1cinderlibvirt @ cloudvirtlibvirt secret[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/739474 https://gerrit.wikimedia.org/r/c/operations/puppet/+/739516
codfw1devcinder-backupscinder-backups @ cloudbackupnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/736201
eqiad1cinder-backupscinder-backups @ cloudbackupnormal keyring[x] done(not in use, no patch)
codfw1devadminceph @ codfw (mons, osds), backy2 @ cloudvirt, glance-backup @ cloudcontrolnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/738904 https://gerrit.wikimedia.org/r/c/operations/puppet/+/738963
eqiad1adminceph @ eqiad (mons, osds) backy2 @ cloudvirt, glance-backup @ cloudcontrolnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/738904 https://gerrit.wikimedia.org/r/c/operations/puppet/+/738963
codfw1devradosgwradosgw @ cloudcontrolnormal keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/739522
eqiad1radosgwnot in use yetnormal keyring[x] done(not in use, no patch)
codfw1devbootstrap.*ceph @ codfwnormal keyrings[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/742132
eqiad1bootstrap.*ceph @ eqiadnormal keyrings[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/742132
codfw1devmon.*cephmon @ codfwtemp keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/742176
eqiad1mon.*cephmon @ eqiadtemp keyring[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/742176
codfw1devmgr.*cephmon @ codfwnormal keyrings ?[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/744808
eqiad1mgr.*cephmon @ eqiadnormal keyrings ?[x] donehttps://gerrit.wikimedia.org/r/c/operations/puppet/+/744808

Details

SubjectRepoBranchLines +/-
ceph: auth: introduce function to calculate keyring_pathoperations/puppetproduction+10 -1
p:ceph::client::rbd_cloudcontrol: remove keyring generationoperations/puppetproduction+1 -39
ceph: auth: eqiad: fix rgw client nameoperations/puppetproduction+1 -1
ceph: auth: load_all: don't fail if only keydata is definedoperations/puppetproduction+2 -1
ceph: auth: load_all: fix input datatypeoperations/puppetproduction+1 -1
ceph: mgr: migrate keyring to new auth abstractionoperations/puppetproduction+72 -130
hiera: ceph: add dummy caps for mgr auth entrieslabs/privatemaster+36 -0
hiera: ceph: add mgr keyrings placeholderslabs/privatemaster+12 -0
ceph: mgr: fix typo in relationshipoperations/puppetproduction+1 -1
ceph: migrate mon auth to the new abstractionoperations/puppetproduction+92 -52
ceph: auth: introduce new parameter 'import_to_ceph'operations/puppetproduction+9 -1
ceph: auth: introduce keydata for mon.xxxx entrieslabs/privatemaster+12 -0
ceph: move bootstrap keyring into new auth abstractionoperations/puppetproduction+7 -11
hieradata: ceph: refresh bootstrap authlabs/privatemaster+4 -2
openstack: radosgw: migrate to new ceph auth abstractionoperations/puppetproduction+8 -24
hieradata: codfw: ceph: add dummy keydata for radosgwlabs/privatemaster+2 -0
cloud: don't deploy cinder keyring in cloudvirtsoperations/puppetproduction+0 -5
cloud: ceph: client: rbd_libvirt: enable ceph::auth::confoperations/puppetproduction+39 -7
openstack: nova: factorize libvirt secrets managementoperations/puppetproduction+53 -49
cloud: ceph: libvirt: migrate to new ceph auth abstractionoperations/puppetproduction+28 -90
ceph: auth: introduce datatype for configuration hashoperations/puppetproduction+6 -5
p:{osd,b_g_images,backy2}: require c::a::deploy when checking keyringsoperations/puppetproduction+3 -0
ceph::auth::keyring: Generate keyring_path if not passedoperations/puppetproduction+35 -51
ceph::auth::keyring: allow passing the full client nameoperations/puppetproduction+12 -1
ceph:mon/osd: remove admin classoperations/puppetproduction+0 -12
cloud: ceph: reorder osd/auth profile declarationoperations/puppetproduction+1 -1
cloud: ceph: auth: enable admin clientoperations/puppetproduction+20 -30
hieradata: ceph: auth: add dummy keydata for the admin clientlabs/privatemaster+4 -0
hieradata: ceph: auth: add dummy keydata for the admin clientlabs/privatemaster+4 -0
cloud: ceph: migrate admin keyring to new abstractionoperations/puppetproduction+55 -105
ceph::auth: skip keys with no keydataoperations/puppetproduction+64 -11
openstack: eqiad1: enable cinder keyring generation on control nodesoperations/puppetproduction+1 -0
openstack: eqiad1: Remove cinder key generation from cloudcontrolsoperations/puppetproduction+0 -12
openstack: codfw1dev: enable cinder key generationoperations/puppetproduction+1 -0
openstack: codfw1dev: remove cinder keyringoperations/puppetproduction+0 -12
cloud: eqiad1: deploy cinder ceph auth credsoperations/puppetproduction+3 -1
ceph::control: enable auth deploy on eqiad and remove unused varsoperations/puppetproduction+45 -56
openstack::control: remove other old keyrings and enable for codfwoperations/puppetproduction+41 -54
ceph::client::rbd_glance: remove keyring generationoperations/puppetproduction+6 -24
ceph::auth: enable loading the keys on all clusters/dcsoperations/puppetproduction+2 -4
ceph::auth: Add deploy for eqiad virt_ceph hosts toooperations/puppetproduction+62 -11
ceph::auth::keyring: Add possibility to override file permissionsoperations/puppetproduction+47 -1
ceph::auth: Move codfw1dev params to the right yamloperations/puppetproduction+4 -3
ceph::auth::delpoy: allow not passing the keyring pathoperations/puppetproduction+22 -3
ceph::auth::deploy: enable on virt_ceph nodes on codfwoperations/puppetproduction+3 -0
ceph::auth: remove some unneeded data from codfwoperations/puppetproduction+0 -3
libvirt|ceph: small refactor and remove keyringsoperations/puppetproduction+17 -26
Add codfw cloudvirts ceph::auth::deploy profileoperations/puppetproduction+9 -8
ceph::auth::load_all: allow generating the keyring pathoperations/puppetproduction+19 -2
ceph::auth: Add codfw1dev-compute client keyoperations/puppetproduction+6 -1
ceph::auth: add deploy profile and classesoperations/puppetproduction+148 -6
ceph: introduce auth load abstractionoperations/puppetproduction+216 -20
cloud: ceph: refactor rbd client profile for cloudcontroloperations/puppetproduction+64 -109
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
gerritbot added a comment.Nov 12 2021, 2:59 PM

Change 738413 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] openstack::eqiad: Remove cinder key generation from cloudcontrols

https://gerrit.wikimedia.org/r/738413

gerritbot added a comment.Nov 12 2021, 2:59 PM

Change 738414 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] openstack::eqiad: enable cinder keyring generation on control nodes

https://gerrit.wikimedia.org/r/738414

dcaro changed the task status from In Progress to Open.Nov 15 2021, 8:58 AM
dcaro moved this task from Doing to Today on the User-dcaro board.
dcaro moved this task from Today to Doing on the User-dcaro board.
dcaro moved this task from Doing to Today on the User-dcaro board.
gerritbot added a comment.Nov 15 2021, 11:20 AM

Change 738411 merged by David Caro:

[operations/puppet@production] openstack: codfw1dev: remove cinder keyring

https://gerrit.wikimedia.org/r/738411

gerritbot added a comment.Nov 15 2021, 11:26 AM

Change 738412 merged by David Caro:

[operations/puppet@production] openstack: codfw1dev: enable cinder key generation

https://gerrit.wikimedia.org/r/738412

aborrero updated the task description. (Show Details)Nov 15 2021, 12:34 PM
dcaro changed the task status from Open to In Progress.Nov 15 2021, 12:45 PM
dcaro moved this task from Today to Doing on the User-dcaro board.
dcaro updated the task description. (Show Details)Nov 15 2021, 12:49 PM
gerritbot added a comment.Nov 15 2021, 1:01 PM

Change 738413 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] openstack: eqiad1: Remove cinder key generation from cloudcontrols

https://gerrit.wikimedia.org/r/738413

gerritbot added a comment.Nov 15 2021, 1:05 PM

Change 738414 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] openstack: eqiad1: enable cinder keyring generation on control nodes

https://gerrit.wikimedia.org/r/738414

aborrero updated the task description. (Show Details)Nov 15 2021, 1:28 PM
aborrero updated the task description. (Show Details)Nov 15 2021, 1:30 PM
gerritbot added a comment.Nov 15 2021, 1:42 PM

Change 738903 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] ceph::auth::keyring: allow passing the full client name

https://gerrit.wikimedia.org/r/738903

gerritbot added a comment.Nov 15 2021, 1:44 PM

Change 738904 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] cloud: ceph: migrate admin keyring to new abstraction

https://gerrit.wikimedia.org/r/738904

gerritbot added a comment.Nov 15 2021, 1:59 PM

Change 738908 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] ceph::auth::keyring: Generate keyring_path if not passed

https://gerrit.wikimedia.org/r/738908

gerritbot added a comment.Nov 15 2021, 4:38 PM

Change 738904 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] cloud: ceph: migrate admin keyring to new abstraction

https://gerrit.wikimedia.org/r/738904

gerritbot added a comment.Nov 15 2021, 4:46 PM

Change 738963 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] cloud: ceph: auth: enable admin client

https://gerrit.wikimedia.org/r/738963

gerritbot added a comment.Nov 15 2021, 4:52 PM

Change 738964 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[labs/private@master] hieradata: ceph: auth: add dummy keydata for the admin client

https://gerrit.wikimedia.org/r/738964

gerritbot added a comment.Nov 15 2021, 4:54 PM

Change 738964 merged by Arturo Borrero Gonzalez:

[labs/private@master] hieradata: ceph: auth: add dummy keydata for the admin client

https://gerrit.wikimedia.org/r/738964

gerritbot added a comment.Nov 15 2021, 5:08 PM

Change 738969 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[labs/private@master] hieradata: ceph: auth: add dummy keydata for the admin client

https://gerrit.wikimedia.org/r/738969

gerritbot added a comment.Nov 15 2021, 5:09 PM

Change 738969 merged by Arturo Borrero Gonzalez:

[labs/private@master] hieradata: ceph: auth: add dummy keydata for the admin client

https://gerrit.wikimedia.org/r/738969

aborrero mentioned this in rLPRIf33f1fff217c: hieradata: ceph: auth: add dummy keydata for the admin client.Nov 15 2021, 5:14 PM
aborrero mentioned this in rLPRId5d21b4a1de6: hieradata: ceph: auth: add dummy keydata for the admin client.
aborrero updated the task description. (Show Details)Nov 15 2021, 5:56 PM
gerritbot added a comment.Nov 15 2021, 6:14 PM

Change 738963 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] cloud: ceph: auth: enable admin client

https://gerrit.wikimedia.org/r/738963

gerritbot added a comment.Nov 15 2021, 6:21 PM

Change 738982 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] cloud: ceph: reorder osd/auth profile declaration

https://gerrit.wikimedia.org/r/738982

gerritbot added a comment.Nov 15 2021, 6:23 PM

Change 738982 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] cloud: ceph: reorder osd/auth profile declaration

https://gerrit.wikimedia.org/r/738982

aborrero updated the task description. (Show Details)Nov 15 2021, 6:28 PM
dcaro changed the task status from In Progress to Open.Nov 16 2021, 8:30 AM
dcaro moved this task from Doing to Today on the User-dcaro board.
gerritbot added a comment.Nov 16 2021, 9:43 AM

Change 737401 abandoned by David Caro:

[operations/puppet@production] ceph:mon/osd: remove admin class

Reason:

https://gerrit.wikimedia.org/r/737401

gerritbot added a comment.Nov 16 2021, 10:24 AM

Change 738903 merged by David Caro:

[operations/puppet@production] ceph::auth::keyring: allow passing the full client name

https://gerrit.wikimedia.org/r/738903

gerritbot added a comment.Nov 16 2021, 10:24 AM

Change 738908 merged by David Caro:

[operations/puppet@production] ceph::auth::keyring: Generate keyring_path if not passed

https://gerrit.wikimedia.org/r/738908

gerritbot added a comment.Nov 16 2021, 10:39 AM

Change 739127 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] ceph::auth: require load_all when checking keyrings

https://gerrit.wikimedia.org/r/739127

gerritbot added a comment.Nov 16 2021, 11:07 AM

Change 739127 merged by David Caro:

[operations/puppet@production] p:{osd,b_g_images,backy2}: require c::a::deploy when checking keyrings

https://gerrit.wikimedia.org/r/739127

gerritbot added a comment.Nov 16 2021, 12:26 PM

Change 739223 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] openstack: nova: factorize libvirt secrets management

https://gerrit.wikimedia.org/r/739223

gerritbot added a comment.Nov 16 2021, 12:57 PM

Change 739228 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: auth: introduce datatype for configuration hash

https://gerrit.wikimedia.org/r/739228

gerritbot added a comment.Nov 16 2021, 1:48 PM

Change 739235 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] cloud: ceph: libvirt: migrate to new ceph auth abstraction

https://gerrit.wikimedia.org/r/739235

gerritbot added a comment.Nov 16 2021, 3:30 PM

Change 739228 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: auth: introduce datatype for configuration hash

https://gerrit.wikimedia.org/r/739228

gerritbot added a comment.Nov 17 2021, 9:31 AM

Change 739474 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] cloud: ceph: client: rbd_libvirt: enable ceph::auth::conf

https://gerrit.wikimedia.org/r/739474

gerritbot added a comment.Nov 17 2021, 9:34 AM

Change 739235 abandoned by Arturo Borrero Gonzalez:

[operations/puppet@production] cloud: ceph: libvirt: migrate to new ceph auth abstraction

Reason:

trying with https://gerrit.wikimedia.org/r/c/operations/puppet/+/739474 instead

https://gerrit.wikimedia.org/r/739235

dcaro reassigned this task from dcaro to aborrero.Nov 17 2021, 10:43 AM
dcaro removed a project: User-dcaro.
gerritbot added a comment.Nov 17 2021, 10:56 AM

Change 739223 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] openstack: nova: factorize libvirt secrets management

https://gerrit.wikimedia.org/r/739223

gerritbot added a comment.Nov 17 2021, 11:09 AM

Change 739474 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] cloud: ceph: client: rbd_libvirt: enable ceph::auth::conf

https://gerrit.wikimedia.org/r/739474

gerritbot added a comment.Nov 17 2021, 11:20 AM

Change 739516 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] cloud: don't deploy cinder keyring in cloudvirts

https://gerrit.wikimedia.org/r/739516

gerritbot added a comment.Nov 17 2021, 11:23 AM

Change 739516 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] cloud: don't deploy cinder keyring in cloudvirts

https://gerrit.wikimedia.org/r/739516

aborrero updated the task description. (Show Details)Nov 17 2021, 11:52 AM
gerritbot added a comment.Nov 17 2021, 12:09 PM

Change 739522 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] openstack: radosgw: migrate to new ceph auth abstraction

https://gerrit.wikimedia.org/r/739522

gerritbot added a comment.Nov 17 2021, 12:11 PM

Change 739523 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[labs/private@master] hieradata: codfw: ceph: add dummy keydata for radosgw

https://gerrit.wikimedia.org/r/739523

gerritbot added a comment.Nov 17 2021, 12:13 PM

Change 739523 merged by Arturo Borrero Gonzalez:

[labs/private@master] hieradata: codfw: ceph: add dummy keydata for radosgw

https://gerrit.wikimedia.org/r/739523

aborrero mentioned this in rLPRI575b28cc5f26: hieradata: codfw: ceph: add dummy keydata for radosgw.Nov 17 2021, 12:15 PM
gerritbot added a comment.Nov 17 2021, 12:19 PM

Change 739522 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] openstack: radosgw: migrate to new ceph auth abstraction

https://gerrit.wikimedia.org/r/739522

aborrero updated the task description. (Show Details)Nov 17 2021, 12:47 PM
dcaro updated the task description. (Show Details)Nov 17 2021, 1:09 PM
gerritbot added a comment.Nov 26 2021, 12:40 PM

Change 742132 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: move bootstrap keyring into new auth abstraction

https://gerrit.wikimedia.org/r/742132

gerritbot added a comment.Nov 26 2021, 12:43 PM

Change 742133 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[labs/private@master] hieradata: ceph: refresh bootstrap auth

https://gerrit.wikimedia.org/r/742133

aborrero updated the task description. (Show Details)Nov 26 2021, 1:17 PM
gerritbot added a comment.Nov 26 2021, 6:19 PM

Change 742175 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: auth: introduce new parameter 'import_to_ceph'

https://gerrit.wikimedia.org/r/742175

gerritbot added a comment.Nov 26 2021, 6:19 PM

Change 742176 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: migrate mon auth to the new abstraction

https://gerrit.wikimedia.org/r/742176

aborrero updated the task description. (Show Details)Nov 26 2021, 6:20 PM
gerritbot added a comment.Nov 29 2021, 10:59 AM

Change 742133 merged by Arturo Borrero Gonzalez:

[labs/private@master] hieradata: ceph: refresh bootstrap auth

https://gerrit.wikimedia.org/r/742133

aborrero mentioned this in rLPRI5ffc7c4acd0b: hieradata: ceph: refresh bootstrap auth.Nov 29 2021, 11:29 AM
gerritbot added a comment.Nov 29 2021, 12:28 PM

Change 742132 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: move bootstrap keyring into new auth abstraction

https://gerrit.wikimedia.org/r/742132

gerritbot added a comment.Nov 30 2021, 11:36 AM

Change 742699 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[labs/private@master] ceph: auth: introduce keydata for mon.xxxx entries

https://gerrit.wikimedia.org/r/742699

gerritbot added a comment.Nov 30 2021, 12:32 PM

Change 742709 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: auth: introduce function to calculate keyring_path

https://gerrit.wikimedia.org/r/742709

gerritbot added a comment.Nov 30 2021, 1:23 PM

Change 742699 merged by Arturo Borrero Gonzalez:

[labs/private@master] ceph: auth: introduce keydata for mon.xxxx entries

https://gerrit.wikimedia.org/r/742699

gerritbot added a comment.Nov 30 2021, 1:26 PM

Change 742175 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: auth: introduce new parameter 'import_to_ceph'

https://gerrit.wikimedia.org/r/742175

gerritbot added a comment.Nov 30 2021, 1:26 PM

Change 742709 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: auth: introduce function to calculate keyring_path

https://gerrit.wikimedia.org/r/742709

gerritbot added a comment.Nov 30 2021, 1:29 PM

Change 742176 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: migrate mon auth to the new abstraction

https://gerrit.wikimedia.org/r/742176

aborrero mentioned this in rLPRIf42b60479b78: ceph: auth: introduce keydata for mon.xxxx entries.Nov 30 2021, 1:35 PM
aborrero updated the task description. (Show Details)Nov 30 2021, 1:53 PM
gerritbot added a comment.Dec 7 2021, 2:00 PM

Change 744784 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: mgr: fix typo in relationship

https://gerrit.wikimedia.org/r/744784

gerritbot added a comment.Dec 7 2021, 2:34 PM

Change 744808 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: mgr: migrate keyring to new auth abstraction

https://gerrit.wikimedia.org/r/744808

gerritbot added a comment.Dec 7 2021, 2:39 PM

Change 744784 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: mgr: fix typo in relationship

https://gerrit.wikimedia.org/r/744784

aborrero updated the task description. (Show Details)Dec 7 2021, 2:45 PM
gerritbot added a comment.Dec 9 2021, 9:34 AM

Change 745477 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[labs/private@master] hiera: ceph: add mgr keyrings placeholders

https://gerrit.wikimedia.org/r/745477

gerritbot added a comment.Dec 9 2021, 9:35 AM

Change 745477 merged by Arturo Borrero Gonzalez:

[labs/private@master] hiera: ceph: add mgr keyrings placeholders

https://gerrit.wikimedia.org/r/745477

aborrero mentioned this in rLPRI2863d679d944: hiera: ceph: add mgr keyrings placeholders.Dec 9 2021, 9:39 AM
gerritbot added a comment.Dec 9 2021, 9:43 AM

Change 745478 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[labs/private@master] hiera: ceph: add dummy caps for mgr auth entries

https://gerrit.wikimedia.org/r/745478

gerritbot added a comment.Dec 9 2021, 9:44 AM

Change 745478 merged by Arturo Borrero Gonzalez:

[labs/private@master] hiera: ceph: add dummy caps for mgr auth entries

https://gerrit.wikimedia.org/r/745478

aborrero mentioned this in rLPRId7c65f621172: hiera: ceph: add dummy caps for mgr auth entries.Dec 9 2021, 9:45 AM
gerritbot added a comment.Dec 9 2021, 2:50 PM

Change 744808 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: mgr: migrate keyring to new auth abstraction

https://gerrit.wikimedia.org/r/744808

gerritbot added a comment.Dec 10 2021, 10:32 AM

Change 745767 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: auth: load_all: fix input datatype

https://gerrit.wikimedia.org/r/745767

gerritbot added a comment.Dec 10 2021, 10:32 AM

Change 745768 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: auth: load_all: don't fail if only keydata is defined

https://gerrit.wikimedia.org/r/745768

gerritbot added a comment.Dec 10 2021, 10:34 AM

Change 745767 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: auth: load_all: fix input datatype

https://gerrit.wikimedia.org/r/745767

gerritbot added a comment.Dec 10 2021, 10:38 AM

Change 745768 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: auth: load_all: don't fail if only keydata is defined

https://gerrit.wikimedia.org/r/745768

gerritbot added a comment.Dec 10 2021, 11:06 AM

Change 745779 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] ceph: auth: eqiad: fix rgw client name

https://gerrit.wikimedia.org/r/745779

gerritbot added a comment.Dec 10 2021, 11:10 AM

Change 745779 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] ceph: auth: eqiad: fix rgw client name

https://gerrit.wikimedia.org/r/745779

aborrero updated the task description. (Show Details)Dec 10 2021, 11:13 AM
aborrero closed this task as Resolved.Dec 10 2021, 11:22 AM

After two months, a deep rabbit hole to climb, some frictions within the WMCS team. more than 50 patches, and a severe sidetrack in the WMCS roadmap, I think we could consider this refactor completed.
On the other side, we are (or at least I am) wiser now, gained a lot of knowledge on how ceph works, a couple of interesting code patterns in puppet, and in general, a more robust ceph abstraction. Worth it :-)

Pretty sure there are things that can be improved, but let's do them in a future iteration. Also, the day we bootstrap again a ceph cluster from scratch we may or may not discover a few potential bugs, or race conditions with how keyrings are deployed and how soon ceph needs them in the bootstrap process.

For the future: figure out a way to cleanup old unused keyrings from the ceph internal DB.

gerritbot added a comment.Dec 13 2021, 12:34 PM

Change 737869 abandoned by David Caro:

[operations/puppet@production] p:ceph::client::rbd_cloudcontrol: remove keyring generation

Reason:

obsolete

https://gerrit.wikimedia.org/r/737869

BTullis awarded a token.Feb 7 2023, 10:41 PM
BTullis subscribed.
BTullis mentioned this in T364105: Streamline the creation of ceph storage cluster user accounts.May 3 2024, 11:27 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits