Page MenuHomePhabricator
Create Task
Maniphest T314870

Setup cloudcephosd10[25-34] into the ceph eqiad cluster
Closed, ResolvedPublic
Actions

Description

We have to set them up (configure and such) and make them join the eqiad ceph cluster.

Note that these hosts will have a different network setup than usual, having their private network in the ranges:

cloudcephosd10[25-29] (row E) -> 192.168.5.0/24
cloudcephosd10[30-34] (row F) -> 192.168.6.0/24

So there might be more changes than usual needed.

For a guideline see:
https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Ceph#Adding_OSDs

Progress

  • cloudcephosd1025
  • cloudcephosd1026
  • cloudcephosd1027
  • cloudcephosd1028
  • cloudcephosd1029
  • cloudcephosd1030
  • cloudcephosd1031
  • cloudcephosd1032
  • cloudcephosd1033
  • cloudcephosd1034

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
node_pinger: use jumbo framesoperations/puppetproduction+183 -71
Add cloudcephosd103[1-4] to the Ceph pooloperations/puppetproduction+1 -6
ceph.bootstrap_and_add: add support to change the osd class typeoperations/cookbookswmcs+215 -3
Add cloudcephosd1030 to the Ceph pooloperations/puppetproduction+2 -2
Add cloudcephosd1029 to the Ceph pooloperations/puppetproduction+2 -2
Add cloudcephosd1028 to the Ceph pooloperations/puppetproduction+2 -2
ceph.bootstrapp_and_add: don't rely on sda/sdb being the os disksoperations/cookbookswmcs+11 -16
Add cloudcephosd1027 to the Ceph pooloperations/puppetproduction+2 -2
ceph::osd: add new disks model to disable write caches foroperations/puppetproduction+1 -0
p:ceph::osd: get the os disks by sizeoperations/puppetproduction+82 -94
cloud: reformat cloud.yaml with prettieroperations/puppetproduction+96 -97
ceph.bootstrap_and_add: add --force optionoperations/cookbookswmcs+19 -3
ceph: add roll_restart_osd_daemons cookbookoperations/cookbookswmcs+125 -0
ceph,opensatck: use the inventory to get the nodes domainoperations/cookbookswmcs+6 -13
ceph: use the correct codfw ceph mon hostsoperations/cookbookswmcs+3 -3
ceph: use human-readable names for ceph clustersoperations/cookbookswmcs+2 -2
ceph: use cluster_name instead of control nodeoperations/cookbookswmcs+276 -280
Openstack: use cluster_name instead of control nodeoperations/cookbookswmcs+90 -142
global: add inventory moduleoperations/cookbookswmcs+699 -201
Add cloudcephosd1026 to the Ceph pooloperations/puppetproduction+2 -2
cloudcephosd1025 through 1034: rename ip interfaces for Bullseyeoperations/puppetproduction+20 -20
ceph: use many cluster and public networksoperations/puppetproduction+443 -348
Use broader network for Ceph clusteroperations/puppetproduction+2 -2
p:ceph::osd: also install the ceph-osd packageoperations/puppetproduction+2 -0
p:ceph::osd: bring the cluster interface upoperations/puppetproduction+15 -6
p:ceph::osd: add the routes only after the interfaceoperations/puppetproduction+2 -1
Add cloudcephosd1025 to Ceph pooloperations/puppetproduction+2 -2
Add new Ceph hosts in racks E4-F4operations/puppetproduction+86 -3
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Stashbot added a comment.Aug 23 2022, 1:24 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-23T13:24:26Z] <wm-bot2> Rebooting node cloudcephosd1027.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 23 2022, 1:27 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-23T13:27:49Z] <wm-bot2> Finished rebooting node cloudcephosd1027.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 23 2022, 1:46 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-23T13:46:28Z] <wm-bot2> Added OSD cloudcephosd1027.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 23 2022, 1:46 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-23T13:46:33Z] <wm-bot2> Added 1 new OSDs ['cloudcephosd1027.eqiad.wmnet'] (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

gerritbot added a comment.Aug 24 2022, 8:19 AM

Change 826208 had a related patch set uploaded (by FNegri; author: FNegri):

[operations/puppet@production] Add cloudcephosd1028 to the Ceph pool

https://gerrit.wikimedia.org/r/826208

gerritbot added a comment.Aug 24 2022, 9:59 AM

Change 826208 merged by FNegri:

[operations/puppet@production] Add cloudcephosd1028 to the Ceph pool

https://gerrit.wikimedia.org/r/826208

Stashbot added a comment.Aug 24 2022, 10:33 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-24T10:33:29Z] <wm-bot2> Adding new OSDs ['cloudcephosd1028.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 24 2022, 10:33 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-24T10:33:33Z] <wm-bot2> Adding OSD cloudcephosd1028.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 24 2022, 10:34 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-24T10:34:08Z] <wm-bot2> Rebooting node cloudcephosd1028.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 24 2022, 10:37 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-24T10:37:15Z] <wm-bot2> Finished rebooting node cloudcephosd1028.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 24 2022, 10:45 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-24T10:45:38Z] <wm-bot2> Added OSD cloudcephosd1028.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 24 2022, 10:45 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-24T10:45:42Z] <wm-bot2> Added 1 new OSDs ['cloudcephosd1028.eqiad.wmnet'] (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

fnegri moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.Aug 25 2022, 2:12 PM
gerritbot added a comment.Aug 25 2022, 2:24 PM

Change 826577 had a related patch set uploaded (by FNegri; author: FNegri):

[operations/puppet@production] Add cloudcephosd1029 to the Ceph pool

https://gerrit.wikimedia.org/r/826577

gerritbot added a comment.Aug 25 2022, 2:51 PM

Change 826577 merged by FNegri:

[operations/puppet@production] Add cloudcephosd1029 to the Ceph pool

https://gerrit.wikimedia.org/r/826577

Stashbot added a comment.Aug 25 2022, 2:58 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-25T14:58:20Z] <wm-bot2> Adding new OSDs ['cloudcephosd1029.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 25 2022, 2:58 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-25T14:58:25Z] <wm-bot2> Adding OSD cloudcephosd1029.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 25 2022, 2:59 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-25T14:59:31Z] <wm-bot2> Rebooting node cloudcephosd1029.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 25 2022, 3:02 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-25T15:02:36Z] <wm-bot2> Finished rebooting node cloudcephosd1029.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 25 2022, 3:14 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-25T15:14:16Z] <wm-bot2> Added OSD cloudcephosd1029.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 25 2022, 3:14 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-25T15:14:21Z] <wm-bot2> Added 1 new OSDs ['cloudcephosd1029.eqiad.wmnet'] (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

gerritbot added a comment.Aug 26 2022, 12:30 PM

Change 826843 had a related patch set uploaded (by FNegri; author: FNegri):

[operations/puppet@production] Add cloudcephosd1030 to the Ceph pool

https://gerrit.wikimedia.org/r/826843

gerritbot added a comment.Aug 30 2022, 9:08 AM

Change 826843 merged by FNegri:

[operations/puppet@production] Add cloudcephosd1030 to the Ceph pool

https://gerrit.wikimedia.org/r/826843

Stashbot added a comment.Aug 30 2022, 9:10 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T09:10:34Z] <wm-bot2> Adding new OSDs ['cloudcephosd1030.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 9:10 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T09:10:38Z] <wm-bot2> Adding OSD cloudcephosd1030.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 9:11 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T09:11:44Z] <wm-bot2> Rebooting node cloudcephosd1030.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 9:14 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T09:14:56Z] <wm-bot2> Finished rebooting node cloudcephosd1030.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 9:32 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T09:32:17Z] <wm-bot2> Adding new OSDs ['cloudcephosd1030.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 9:32 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T09:32:21Z] <wm-bot2> Adding OSD cloudcephosd1030.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 9:32 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T09:32:57Z] <wm-bot2> Rebooting node cloudcephosd1030.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 9:39 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T09:39:00Z] <wm-bot2> Finished rebooting node cloudcephosd1030.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 10:38 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T10:38:52Z] <wm-bot2> Adding new OSDs ['cloudcephosd1030.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 10:38 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T10:38:55Z] <wm-bot2> Adding OSD cloudcephosd1030.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 10:39 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T10:39:31Z] <wm-bot2> Rebooting node cloudcephosd1030.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Aug 30 2022, 10:43 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-08-30T10:43:02Z] <wm-bot2> Finished rebooting node cloudcephosd1030.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

gerritbot added a comment.Sep 2 2022, 7:39 AM

Change 824153 merged by jenkins-bot:

[operations/cookbooks@wmcs] ceph.bootstrap_and_add: add support to change the osd class type

https://gerrit.wikimedia.org/r/824153

gerritbot added a comment.Sep 6 2022, 10:31 AM

Change 830147 had a related patch set uploaded (by FNegri; author: FNegri):

[operations/puppet@production] Add cloudcephosd103[1-4] to the Ceph pool

https://gerrit.wikimedia.org/r/830147

gerritbot added a comment.Sep 6 2022, 10:48 AM

Change 830147 merged by FNegri:

[operations/puppet@production] Add cloudcephosd103[1-4] to the Ceph pool

https://gerrit.wikimedia.org/r/830147

fnegri added a subtask: T317127: hw troubleshooting: power supply alert for cloudcephosd1031.eqiad.wmnet.Sep 6 2022, 1:43 PM
fnegri updated the task description. (Show Details)Sep 6 2022, 1:59 PM
Stashbot added a comment.Sep 7 2022, 9:41 AM

Mentioned in SAL (#wikimedia-cloud) [2022-09-07T09:41:52Z] <dhinus> Temporarily removing cloudcephosd1030 from Ceph cluster (https://phabricator.wikimedia.org/T314870)

fnegri added a comment.EditedSep 7 2022, 9:42 AM

cloudcephosd1030 and cloudcephosd1031 have hardware issues (see related tasks). I'm going to remove osd.231, osd.232 and osd.233 from the cluster (they're in cloudcephosd1030)

Following these instructions: https://docs.ceph.com/en/latest/rados/operations/add-or-rm-osds/#removing-osds-manual

$ sudo ceph osd out osd.231
marked out osd.231.
$ sudo ceph osd out osd.232
marked out osd.232.
$ sudo ceph osd out osd.233
marked out osd.233.

$ sudo systemctl stop ceph-osd@231.service
$ sudo systemctl stop ceph-osd@232.service
$ sudo systemctl stop ceph-osd@233.service

$ sudo ceph osd purge osd.231 --yes-i-really-mean-it
purged osd.231
$ sudo ceph osd purge osd.232 --yes-i-really-mean-it
purged osd.232
$ sudo ceph osd purge osd.233 --yes-i-really-mean-it
purged osd.233
Stashbot added a comment.Sep 7 2022, 10:10 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-07T10:10:07Z] <wm-bot2> Adding new OSDs ['cloudcephosd1032.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Sep 7 2022, 10:10 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-07T10:10:10Z] <wm-bot2> Adding OSD cloudcephosd1032.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Sep 7 2022, 10:10 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-07T10:10:51Z] <wm-bot2> Rebooting node cloudcephosd1032.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Sep 7 2022, 10:14 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-07T10:14:01Z] <wm-bot2> Finished rebooting node cloudcephosd1032.eqiad.wmnet (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Stashbot added a comment.Sep 7 2022, 10:18 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-07T10:18:01Z] <wm-bot2> Added OSD cloudcephosd1032.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@Francesco’s-MacBook-Pro

Jclark-ctr closed subtask T317127: hw troubleshooting: power supply alert for cloudcephosd1031.eqiad.wmnet as Resolved.Sep 7 2022, 1:30 PM
fnegri changed the status of subtask T317219: Ceph cookbook fails on node with empty children list from Open to In Progress.Sep 13 2022, 10:22 AM
fnegri updated the task description. (Show Details)Sep 14 2022, 9:08 AM
fnegri updated the task description. (Show Details)Sep 14 2022, 9:11 AM
fnegri closed subtask T317219: Ceph cookbook fails on node with empty children list as Resolved.Sep 14 2022, 4:11 PM
Jclark-ctr closed subtask T316673: hw troubleshooting: one disk not working properly in cloudcephosd1034.eqiad.wmnet as Resolved.Sep 14 2022, 4:54 PM
fnegri changed the status of subtask T318680: Ceph cookbook fails when checking jumbo frames from Open to In Progress.Sep 27 2022, 10:12 AM
fnegri removed a project: Patch-For-Review.
fnegri removed a subscriber: cmooney.
Stashbot added a comment.Sep 27 2022, 10:32 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-27T10:32:03Z] <wm-bot2> Adding new OSDs ['cloudcephosd1030.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Sep 27 2022, 10:32 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-27T10:32:07Z] <wm-bot2> Adding OSD cloudcephosd1030.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Sep 27 2022, 10:32 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-27T10:32:41Z] <wm-bot2> Rebooting node cloudcephosd1030.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Sep 27 2022, 10:35 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-27T10:35:56Z] <wm-bot2> Finished rebooting node cloudcephosd1030.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Sep 27 2022, 10:48 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-27T10:48:13Z] <wm-bot2> Added OSD cloudcephosd1030.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@wmf3169

fnegri changed the status of subtask T318723: Ceph cookbook fails waiting for OSDs to show up from Open to In Progress.Sep 27 2022, 3:27 PM
fnegri updated the task description. (Show Details)
Stashbot added a comment.Sep 30 2022, 12:41 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-30T12:41:25Z] <wm-bot2> Adding new OSDs ['cloudcephosd1031.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Sep 30 2022, 12:41 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-30T12:41:28Z] <wm-bot2> Adding OSD cloudcephosd1031.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Sep 30 2022, 12:42 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-30T12:42:07Z] <wm-bot2> Rebooting node cloudcephosd1031.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Sep 30 2022, 12:45 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-09-30T12:45:36Z] <wm-bot2> Finished rebooting node cloudcephosd1031.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169

fnegri updated the task description. (Show Details)Sep 30 2022, 1:48 PM
fnegri closed subtask T318723: Ceph cookbook fails waiting for OSDs to show up as Resolved.Sep 30 2022, 3:17 PM
fnegri closed subtask T318680: Ceph cookbook fails when checking jumbo frames as Declined.Oct 4 2022, 11:09 AM
Stashbot added a comment.Oct 4 2022, 2:30 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-04T14:30:19Z] <wm-bot2> Adding new OSDs ['cloudcephosd1033.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 4 2022, 2:30 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-04T14:30:22Z] <wm-bot2> Adding OSD cloudcephosd1033.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 4 2022, 2:30 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-04T14:30:56Z] <wm-bot2> Rebooting node cloudcephosd1033.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 4 2022, 2:34 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-04T14:34:05Z] <wm-bot2> Finished rebooting node cloudcephosd1033.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 4 2022, 4:40 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-04T16:40:05Z] <wm-bot2> Added OSD cloudcephosd1033.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 4 2022, 4:40 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-04T16:40:09Z] <wm-bot2> Added 1 new OSDs ['cloudcephosd1033.eqiad.wmnet'] (T314870) - cookbook ran by fran@wmf3169

fnegri updated the task description. (Show Details)Oct 4 2022, 5:16 PM
Stashbot added a comment.Oct 5 2022, 12:58 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-05T12:58:11Z] <wm-bot2> Adding new OSDs ['cloudcephosd1034.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 5 2022, 12:58 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-05T12:58:15Z] <wm-bot2> Adding OSD cloudcephosd1034.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 5 2022, 12:58 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-05T12:58:55Z] <wm-bot2> Rebooting node cloudcephosd1034.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 5 2022, 1:02 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-05T13:02:16Z] <wm-bot2> Finished rebooting node cloudcephosd1034.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 5 2022, 1:11 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-05T13:11:16Z] <wm-bot2> Added OSD cloudcephosd1034.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@wmf3169

Stashbot added a comment.Oct 5 2022, 1:11 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2022-10-05T13:11:19Z] <wm-bot2> Added 1 new OSDs ['cloudcephosd1034.eqiad.wmnet'] (T314870) - cookbook ran by fran@wmf3169

fnegri closed this task as Resolved.Oct 5 2022, 1:26 PM
fnegri updated the task description. (Show Details)
dcaro mentioned this in rCCKBbb4b585faddc: global: add inventory module.Dec 14 2022, 3:33 PM
dcaro mentioned this in rCCKB387bcc19b05a: Openstack: use cluster_name instead of control node.
dcaro mentioned this in rCCKB076bce7a6ed7: ceph: use cluster_name instead of control node.
dcaro mentioned this in rCCKB485aaf1da8c1: ceph: use human-readable names for ceph clusters.
dcaro mentioned this in rCCKB8900fc810ee4: ceph: use the correct codfw ceph mon hosts.
dcaro mentioned this in rCCKBda9a6fac759e: ceph,opensatck: use the inventory to get the nodes domain.
dcaro mentioned this in rCCKB132ac529962b: ceph: add roll_restart_osd_daemons cookbook.
dcaro mentioned this in rCCKB1b19649648ff: ceph.bootstrap_and_add: add --force option.
dcaro mentioned this in rCCKB5bbdfaa4beef: ceph.bootstrapp_and_add: don't rely on sda/sdb being the os disks.
dcaro mentioned this in rCCKBa9a4d42ac902: ceph.bootstrap_and_add: add support to change the osd class type.
dcaro mentioned this in T329535: Cloud Ceph outage 2023-02-13.Feb 15 2023, 4:43 PM
gerritbot added a comment.Feb 21 2023, 11:09 AM

Change 824202 merged by David Caro:

[operations/puppet@production] node_pinger: use jumbo frames

https://gerrit.wikimedia.org/r/824202

dcaro mentioned this in T375204: [cloudceph] Improve downtime when a switch goes down.Sep 19 2024, 2:30 PM
taavi added a project: Cloud-VPS.Sep 28 2024, 12:22 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits