Page MenuHomePhabricator
Create Task
Maniphest T395910

cloudcephosd10[48-52] service implementation
Closed, ResolvedPublic
Actions

Description

  • cloudcephosd1048
  • cloudcephosd1049 -- ready to be pooled
  • cloudcephosd1050 -- ready to be pooled
  • cloudcephosd1051 -- ready to be pooled
  • cloudcephosd1052

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Make cloudcephosd1052 an osd nodeoperations/puppetproduction+10 -5
Cloudcephosd1048: Configure ceph with a single nicoperations/puppetproduction+26 -3
cloudceph: add new OSDs: cloudcephosd1042-1051operations/puppetproduction+60 -5
Customize query in gerrit

Related Objects
Search...

Event Timeline

Andrew created this task.Jun 3 2025, 3:06 PM
Restricted Application edited projects, added cloud-services-team; removed cloud-services-team (Hardware). · View Herald TranscriptJun 3 2025, 3:06 PM
Andrew claimed this task.Jun 3 2025, 3:06 PM
Andrew added a subscriber: Jclark-ctr.
Stashbot added a comment.Jun 3 2025, 5:37 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-06-03T17:37:01Z] <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.bootstrap_and_add (T395910)

Stashbot added a comment.Jun 3 2025, 5:42 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-06-03T17:42:08Z] <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=99) (T395910)

Stashbot added a comment.Jun 3 2025, 5:50 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-06-03T17:50:23Z] <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.bootstrap_and_add (T395910)

Stashbot added a comment.Jun 3 2025, 5:55 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-06-03T17:55:42Z] <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=99) (T395910)

Stashbot added a comment.Jun 3 2025, 6:29 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-06-03T18:29:01Z] <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.bootstrap_and_add (T395910)

Stashbot added a comment.Jun 3 2025, 6:34 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-06-03T18:34:14Z] <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=99) (T395910)

Andrew changed the task status from Open to Stalled.Jun 6 2025, 4:50 PM
Andrew triaged this task as Medium priority.
gerritbot added a comment.Jul 9 2025, 11:23 PM

Change #1167708 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Cloudcephosd1048: Configure ceph with a single nic

https://gerrit.wikimedia.org/r/1167708

gerritbot added a project: Patch-For-Review.Jul 9 2025, 11:23 PM
cmooney subscribed.Jul 10 2025, 5:54 PM

We may need to hold off on this for now.

The requirement for jumbo frames poses a difficulty for the plan as the parent interface is configured for regular 1500 byte frames, which I don't think Linux will allow.

I will put a note about this on T399180 also.

cmooney added a comment.Jul 17 2025, 4:58 PM

Regarding the jumbo-frame complication on the plan to move to one link we are arranging to connect a second 25G on each of these new hosts for the storage vlan. See below tasks:

T394333: Q4:rack/setup/install cloudcephosd10[48-51]
{T399869}

wiki_willy removed a project: ops-eqiad.Jul 29 2025, 9:13 PM
RobH unsubscribed.Aug 7 2025, 3:59 PM
Andrew changed the task status from Stalled to In Progress.Aug 20 2025, 9:22 PM
gerritbot added a comment.Aug 21 2025, 2:05 AM

Change #1180693 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] cloudceph: add new OSDs: cloudcephosd1042-1051

https://gerrit.wikimedia.org/r/1180693

gerritbot added a comment.Aug 21 2025, 2:10 AM

Change #1180693 merged by Andrew Bogott:

[operations/puppet@production] cloudceph: add new OSDs: cloudcephosd1042-1051

https://gerrit.wikimedia.org/r/1180693

fnegri added a subtask: T402475: KernelErrors .Aug 21 2025, 9:05 AM
Stashbot added a comment.Aug 22 2025, 2:15 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-08-22T14:15:16Z] <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.bootstrap_and_add (T395910)

Stashbot added a comment.Aug 22 2025, 2:20 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-08-22T14:20:17Z] <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=99) (T395910)

Stashbot added a comment.Aug 22 2025, 2:24 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-08-22T14:24:03Z] <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.bootstrap_and_add (T395910)

fnegri closed subtask T402475: KernelErrors as Resolved.Aug 22 2025, 3:03 PM
gerritbot added a comment.Aug 22 2025, 7:37 PM

Change #1167708 abandoned by Andrew Bogott:

[operations/puppet@production] Cloudcephosd1048: Configure ceph with a single nic

Reason:

This turns out to be a whole thing, on account of needing jumbo frames on one nic but not the other

https://gerrit.wikimedia.org/r/1167708

Maintenance_bot removed a project: Patch-For-Review.Aug 22 2025, 8:31 PM
Andrew renamed this task from cloudcephosd10[48-51] service implementation to cloudcephosd10[48-52] service implementation.Aug 26 2025, 3:36 PM
Andrew updated the task description. (Show Details)
gerritbot added a comment.Aug 26 2025, 3:36 PM

Change #1182172 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Make cloudcephosd1052 an osd node

https://gerrit.wikimedia.org/r/1182172

gerritbot added a project: Patch-For-Review.Aug 26 2025, 3:36 PM

Change #1182172 merged by Andrew Bogott:

[operations/puppet@production] Make cloudcephosd1052 an osd node

https://gerrit.wikimedia.org/r/1182172

Andrew updated the task description. (Show Details)Aug 26 2025, 4:03 PM
Maintenance_bot removed a project: Patch-For-Review.Aug 26 2025, 4:31 PM
Andrew updated the task description. (Show Details)Aug 29 2025, 4:00 PM
taavi added a project: Cloud-VPS.Aug 29 2025, 4:05 PM
Stashbot added a comment.Sep 4 2025, 4:32 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-09-04T16:32:03Z] <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.bootstrap_and_add (T395910)

Stashbot added a comment.Sep 4 2025, 4:36 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2025-09-04T16:36:16Z] <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=99) (T395910)

fnegri edited projects, added cloud-services-team (FY2025/26-Q1-Q2); removed cloud-services-team.Sep 5 2025, 8:40 AM
fnegri moved this task from Backlog to In progress on the cloud-services-team (FY2025/26-Q1-Q2) board.
Andrew updated the task description. (Show Details)Sep 24 2025, 3:06 PM
Andrew added a comment.Sep 24 2025, 3:11 PM

1050 and 1051 won't be pooled immediately, they're being reserved for T405478

fgiunchedi closed this task as Resolved.Oct 27 2025, 9:23 AM
fgiunchedi closed subtask T405478: Experiment with cloudcephosd1050 and cloudcephosd1051 in single-nic configuration as Resolved.
fgiunchedi updated the task description. (Show Details)
fgiunchedi subscribed.

Completed

fnegri moved this task from In progress to Done on the cloud-services-team (FY2025/26-Q1-Q2) board.Oct 27 2025, 2:37 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits