Page MenuHomePhabricator
Create Task
Maniphest T285858

Install the new ceph osd machines cloudcephosd10(1[6-9]|20) using cookbooks
Closed, ResolvedPublic
Actions

Description

Write the description below

Now that we have the new machines there (T274945), install and incorporate them to the eqiad cluster using cookbooks.

Note that there's some part of it that has to be done manually:

  • Check if the disk hardware model has support to disable write cache, if so:
    • Add entry to profile::ceph::osd::disk_models_without_write_cache
  • Change host role and add puppet hiera info:
    • manifests/site.pp (to the wmcs::ceph::osd role)
    • hieradata/eqiad/profile/ceph/osd.yaml

The above will add the packages, configuration and credentials necesary to configure the OSD.

Details

SubjectRepoBranchLines +/-
wmcs.ceph: add cloudcephosd1018 as osdoperations/puppetproduction+10 -2
wmcs.ceph: add cookbook to bootstrap and add OSDsoperations/cookbookswmcs+283 -2
wmcs.ceph: rename the ceph controller to CephClusterControlleroperations/cookbookswmcs+27 -17
ceph.keyring: make sure the bootstrap dir existsoperations/puppetproduction+8 -0
ceph.keyring: ensure that the bootstrap dir existsoperations/puppetproduction+14 -0
wmcs.ceph: Add the new 17, 19 and 20 OSDsoperations/puppetproduction+28 -4
wmcs.ceph: Add cloudcephosd1016 to the ceph osd roleoperations/puppetproduction+2 -2
wmcs.ceph: update disk models should disable write cacheoperations/puppetproduction+5 -3
Customize query in gerrit

Related Objects
Search...

Event Timeline

dcaro triaged this task as High priority.Jun 30 2021, 1:15 PM
dcaro created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 30 2021, 1:15 PM
dcaro added a parent task: T274945: (Need By: TBD) rack/setup/install cloudcephosd10[16-20].eqiad.wmnet.Jun 30 2021, 1:15 PM
dcaro removed a parent task: T274945: (Need By: TBD) rack/setup/install cloudcephosd10[16-20].eqiad.wmnet.Jun 30 2021, 1:16 PM
dcaro renamed this task from Install the new ceph osd machines cloudcephosd101[6-9] using cookbooks to Install the new ceph osd machines cloudcephosd10(1[6-9]|20) using cookbooks.Jun 30 2021, 1:16 PM
dcaro added a subtask: T274945: (Need By: TBD) rack/setup/install cloudcephosd10[16-20].eqiad.wmnet.
gerritbot added a comment.Jun 30 2021, 1:36 PM

Change 702370 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] wmcs.ceph: update disk models should disable write cache

https://gerrit.wikimedia.org/r/702370

gerritbot added a project: Patch-For-Review.Jun 30 2021, 1:36 PM
gerritbot added a comment.Jun 30 2021, 1:41 PM

Change 702374 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] wmcs.ceph: Add cloudcephosd1016 to the ceph osd role

https://gerrit.wikimedia.org/r/702374

dcaro updated the task description. (Show Details)Jun 30 2021, 1:43 PM
gerritbot added a comment.Jun 30 2021, 2:41 PM

Change 702370 merged by David Caro:

[operations/puppet@production] wmcs.ceph: update disk models should disable write cache

https://gerrit.wikimedia.org/r/702370

gerritbot added a comment.Jun 30 2021, 2:41 PM

Change 702374 merged by David Caro:

[operations/puppet@production] wmcs.ceph: Add cloudcephosd1016 to the ceph osd role

https://gerrit.wikimedia.org/r/702374

Stashbot added a comment.Jul 1 2021, 10:40 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T10:40:43Z] <wm-bot> Adding new OSD cloudcephosd1016.eqiad.wmnet to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 10:41 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T10:41:25Z] <wm-bot> Adding new OSD cloudcephosd1016.eqiad.wmnet to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 10:42 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T10:42:45Z] <wm-bot> Adding new OSD cloudcephosd1016.eqiad.wmnet to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 10:44 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T10:44:52Z] <wm-bot> Adding new OSD cloudcephosd1016.eqiad.wmnet to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 10:47 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T10:47:46Z] <wm-bot> Adding new OSD cloudcephosd1016.eqiad.wmnet to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 10:58 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T10:58:53Z] <dcaro> rebooting cloudcephosd1016 (T285858)

Stashbot added a comment.Jul 1 2021, 11:13 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T11:13:32Z] <wm-bot> Adding new OSD cloudcephosd1016.eqiad.wmnet to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 11:16 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T11:16:17Z] <wm-bot> Added new OSD node cloudcephosd1016.eqiad.wmnet (T285858) - cookbook ran by dcaro@vulcanus

gerritbot added a comment.Jul 1 2021, 1:33 PM

Change 702655 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] wmcs.ceph: Add the new 17, 19 and 20 OSDs

https://gerrit.wikimedia.org/r/702655

gerritbot added a comment.Jul 1 2021, 2:06 PM

Change 702655 merged by David Caro:

[operations/puppet@production] wmcs.ceph: Add the new 17, 19 and 20 OSDs

https://gerrit.wikimedia.org/r/702655

Stashbot added a comment.Jul 1 2021, 2:17 PM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T14:16:58Z] <wm-bot> Adding new OSDs ['cloudcephosd1017.eqiad.wmnet', 'cloudcephosd1019.eqiad.wmnet', 'cloudcephosd1020.eqiad.wmnet'] to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 2:17 PM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T14:17:04Z] <wm-bot> Adding OSD cloudcephosd1017.eqiad.wmnet... (1/3) (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 2:20 PM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T14:20:33Z] <wm-bot> Adding new OSDs ['cloudcephosd1017.eqiad.wmnet', 'cloudcephosd1019.eqiad.wmnet', 'cloudcephosd1020.eqiad.wmnet'] to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 2:20 PM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T14:20:37Z] <wm-bot> Adding OSD cloudcephosd1017.eqiad.wmnet... (1/3) (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 2:25 PM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T14:25:52Z] <wm-bot> Added OSD cloudcephosd1017.eqiad.wmnet... (1/3) (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 1 2021, 2:25 PM

Mentioned in SAL (#wikimedia-cloud) [2021-07-01T14:25:57Z] <wm-bot> Adding OSD cloudcephosd1019.eqiad.wmnet... (2/3) (T285858) - cookbook ran by dcaro@vulcanus

gerritbot added a comment.Jul 1 2021, 3:02 PM

Change 702677 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] ceph.keyring: ensure that the bootstrap dir exists

https://gerrit.wikimedia.org/r/702677

gerritbot added a comment.Jul 2 2021, 9:03 AM

Change 702677 merged by David Caro:

[operations/puppet@production] ceph.keyring: ensure that the bootstrap dir exists

https://gerrit.wikimedia.org/r/702677

Stashbot added a comment.Jul 2 2021, 9:13 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-02T09:13:53Z] <wm-bot> Adding new OSDs ['cloudcephosd1019.eqiad.wmnet', 'cloudcephosd1020.eqiad.wmnet'] to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 2 2021, 9:13 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-02T09:13:58Z] <wm-bot> Adding OSD cloudcephosd1019.eqiad.wmnet... (1/2) (T285858) - cookbook ran by dcaro@vulcanus

gerritbot added a comment.Jul 2 2021, 9:52 AM

Change 702897 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] ceph.keyring: make sure the bootstrap dir exists

https://gerrit.wikimedia.org/r/702897

gerritbot added a comment.Jul 2 2021, 10:00 AM

Change 702897 merged by David Caro:

[operations/puppet@production] ceph.keyring: make sure the bootstrap dir exists

https://gerrit.wikimedia.org/r/702897

Stashbot added a comment.Jul 2 2021, 10:01 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-02T10:01:54Z] <wm-bot> Adding new OSDs ['cloudcephosd1019.eqiad.wmnet', 'cloudcephosd1020.eqiad.wmnet'] to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 2 2021, 10:01 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-02T10:01:58Z] <wm-bot> Adding OSD cloudcephosd1019.eqiad.wmnet... (1/2) (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 2 2021, 10:07 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-02T10:07:02Z] <wm-bot> Added OSD cloudcephosd1019.eqiad.wmnet... (1/2) (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 2 2021, 10:07 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-02T10:07:06Z] <wm-bot> Adding OSD cloudcephosd1020.eqiad.wmnet... (2/2) (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 2 2021, 10:12 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-02T10:12:13Z] <wm-bot> Added OSD cloudcephosd1020.eqiad.wmnet... (2/2) (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 2 2021, 10:12 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-02T10:12:19Z] <wm-bot> Added 2 new OSDs ['cloudcephosd1019.eqiad.wmnet', 'cloudcephosd1020.eqiad.wmnet'] (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Jul 2 2021, 10:12 AM

Mentioned in SAL (#wikimedia-cloud) [2021-07-02T10:12:35Z] <wm-bot> The cluster is not rebalance after adding the new OSDs ['cloudcephosd1019.eqiad.wmnet', 'cloudcephosd1020.eqiad.wmnet'] (T285858) - cookbook ran by dcaro@vulcanus

gerritbot added a comment.Jul 2 2021, 12:39 PM

Change 702930 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/cookbooks@wmcs] wmcs.ceph: add cookbook to bootstrap and add OSDs

https://gerrit.wikimedia.org/r/702930

gerritbot added a comment.Jul 2 2021, 12:56 PM

Change 702929 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/cookbooks@wmcs] wmcs.ceph: rename the ceph controller to CephClusterController

https://gerrit.wikimedia.org/r/702929

gerritbot added a comment.Jul 16 2021, 10:27 AM

Change 702929 merged by jenkins-bot:

[operations/cookbooks@wmcs] wmcs.ceph: rename the ceph controller to CephClusterController

https://gerrit.wikimedia.org/r/702929

gerritbot added a comment.Jul 16 2021, 10:27 AM

Change 702930 merged by jenkins-bot:

[operations/cookbooks@wmcs] wmcs.ceph: add cookbook to bootstrap and add OSDs

https://gerrit.wikimedia.org/r/702930

dcaro moved this task from Inbox to Blocked on the cloud-services-team (Kanban) board.Jul 16 2021, 10:28 AM
dcaro moved this task from Blocked to Doing on the cloud-services-team (Kanban) board.Aug 10 2021, 2:15 PM
Stashbot added a comment.Aug 11 2021, 1:47 PM

Mentioned in SAL (#wikimedia-cloud) [2021-08-11T13:47:45Z] <wm-bot> Adding new OSDs ['cloudcephosd1018.eqiad.wmnet'] to the cluster (T285858) - cookbook ran by dcaro@vulcanus

Stashbot added a comment.Aug 11 2021, 1:47 PM

Mentioned in SAL (#wikimedia-cloud) [2021-08-11T13:47:50Z] <wm-bot> Adding OSD cloudcephosd1018.eqiad.wmnet... (1/1) (T285858) - cookbook ran by dcaro@vulcanus

gerritbot added a comment.Aug 11 2021, 1:54 PM

Change 711499 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] wmcs.ceph: add cloudcephosd1018 as osd

https://gerrit.wikimedia.org/r/711499

gerritbot added a comment.Aug 18 2021, 4:04 PM

Change 711499 merged by Andrew Bogott:

[operations/puppet@production] wmcs.ceph: add cloudcephosd1018 as osd

https://gerrit.wikimedia.org/r/711499

dcaro reassigned this task from dcaro to Andrew.Aug 25 2021, 9:53 AM

Re-assign to me once the new OSD is deployed to followup with the docs.

Andrew reassigned this task from Andrew to dcaro.Aug 30 2021, 4:18 PM
Andrew subscribed.

nodes are installed -- back to you!

dcaro closed this task as Resolved.Sep 1 2021, 3:21 PM

Docs updated

dcaro mentioned this in rCCKBce9b933458fd: wmcs.ceph: rename the ceph controller to CephClusterController.Dec 14 2022, 3:32 PM
dcaro mentioned this in rCCKBdd8adffc42ab: wmcs.ceph: add cookbook to bootstrap and add OSDs.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits