Page MenuHomePhabricator
Create Task
Maniphest T324660

Install Ceph Cluster for Data Platform Engineering
Closed, ResolvedPublic
Actions

Description

We have now acquired the Ceph servers identified in T310195: Ceph Data Infrastructure Request and they have been racked in the task {T311869}

This task is intended to track all of the steps required to install and configure these servers, such that they can provide the storage services identified in the design document.

Sub-tasks will be created for a number of these steps, but an initial list will include:

  • Create partman recipe for cephosd servers
  • Install operating system to cephosd servers
  • Decide on ceph version for data-engineering cluster
  • Package ceph appropriately for data-engineering cluster
  • Deploy ceph mon processes to data-engineering cluster
  • Deploy ceph mgr processes to data-engineering cluster
  • Deploy ceph osd processes to data-engineering cluster
  • Deploy ceph radosgw processes to data-engineering cluster
  • Configure load-balancing approriate for ceph radosgw services on the data-engineering cluster

Details

SubjectRepoBranchLines +/-
Use an LVM volume for /var/lib/ceph on cephosd nodesoperations/puppetproduction+9 -9
Add dummy keydata for the new ceph OSD serviceslabs/privatemaster+12 -0
Add keydata for ceph mgr daemonslabs/privatemaster+10 -0
Move ceph profile authentication token to the rolelabs/privatemaster+10 -74
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedGehelT327267 Create a DSE Kubernetes cluster with support for persistent storage from Ceph
 ResolvedGehelT324660 Install Ceph Cluster for Data Platform Engineering
 ResolvedBTullisT324670 Create partman recipe for cephosd servers
 ResolvedBTullisT326945 Decide on installation details for new ceph cluster
 ResolvedBTullisT328123 Create puppet profiles for the new ceph cluster
 ResolvedBTullisT330149 Deploy ceph mon and mgr processes to data-engineering cluster
 ResolvedBTullisT330151 Deploy ceph osd processes to data-engineering cluster
 DeclinedBTullisT363558 Switch the DPE Ceph cluster to use cephadm management
 ResolvedBTullisT363559 eqiad: 1 VMs requested for ceph cluster administration (cephadm)
 ResolvedBTullisT330153 Configure Anycast load-balancing ceph radosgw services on the data-engineering cluster
 ResolvedBTullisT374447 Test the S3 and swift interfaces of rgw.eqiad.dpe.anycast.wmnet
 ResolvedBTullisT330152 Deploy ceph radosgw processes to data-engineering cluster
 ResolvedBTullisT369634 Decide how to do DAG logging on dse-k8s
 ResolvedbrouberolT372787 Implement S3 based logging for Airflow tasks on dse-k8s
 DeclinedNoneT372788 Implement elasticsearch/opensearch based logging for Airflow on dse-k8s
 ResolvedbkingT373034 Build Opensearch2 Docker image for testing
 ResolvedbrouberolT372281 Configure scheduled backups and WAL archiving to use our S3 endpoint
 ResolvedNoneT376401 Enable the cephfs services on the DPE Ceph cluster
 ResolvedbrouberolT376402 Create cephx authentication keys for the mds servers
 ResolvedbrouberolT376404 Deploy mds services to the cephosd cluster
 ResolvedBTullisT376405 Create the pools required for cephfs
 ResolvedbrouberolT376406 Import the upstream ceph-csi-cephfs chart and adapt it to our needs
 ResolvedbrouberolT376408 Deploy the ceph-csi-cephfs chart to the dse-k8s-cluster
 ResolvedbrouberolT376407 Create cephx authentication keys for the ceph-csi-cephfs plugin

Event Timeline

BTullis created this task.Dec 7 2022, 12:15 PM
JArguello-WMF moved this task from Backlog to EQ2 Kanban (Sprints 04-07) on the Shared-Data-Infrastructure board.Dec 7 2022, 1:39 PM
JArguello-WMF edited projects, added Shared-Data-Infrastructure (EQ2 Kanban (Sprints 04-07)); removed Shared-Data-Infrastructure.
BTullis changed the status of subtask T324670: Create partman recipe for cephosd servers from Open to In Progress.Dec 7 2022, 1:49 PM
• EChetty edited projects, added Data-Engineering-Planning; removed Data-Engineering.Jan 6 2023, 11:01 AM
• EChetty moved this task from Backlog to Shared Data Infra on the Data-Engineering-Planning board.Jan 6 2023, 12:45 PM
• EChetty edited projects, added Shared-Data-Infrastructure; removed Shared-Data-Infrastructure (EQ2 Kanban (Sprints 04-07)).
• EChetty assigned this task to BTullis.Jan 9 2023, 11:33 AM
• EChetty moved this task from Backlog to Epics on the Shared-Data-Infrastructure board.
MatthewVernon subscribed.Jan 9 2023, 5:16 PM
BTullis closed subtask T324670: Create partman recipe for cephosd servers as Resolved.Jan 11 2023, 3:33 PM
BTullis updated the task description. (Show Details)Jan 11 2023, 3:44 PM
BTullis added a parent task: T310195: Ceph Data Infrastructure Request.Jan 11 2023, 3:47 PM
BTullis mentioned this in T310195: Ceph Data Infrastructure Request.Jan 11 2023, 3:55 PM
• EChetty added a parent task: T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.Jan 18 2023, 12:15 PM
BTullis updated the task description. (Show Details)Jan 27 2023, 12:12 PM
BTullis triaged this task as High priority.Jan 27 2023, 12:45 PM
JArguello-WMF mentioned this in T330149: Deploy ceph mon and mgr processes to data-engineering cluster.Feb 21 2023, 1:19 PM
JArguello-WMF added a subtask: T330151: Deploy ceph osd processes to data-engineering cluster.Feb 21 2023, 1:28 PM
JArguello-WMF mentioned this in T330152: Deploy ceph radosgw processes to data-engineering cluster.Feb 21 2023, 1:30 PM
JArguello-WMF mentioned this in T330153: Configure Anycast load-balancing ceph radosgw services on the data-engineering cluster.Feb 21 2023, 1:33 PM
gerritbot added a comment.Feb 27 2023, 10:52 AM

Change 892380 had a related patch set uploaded (by Btullis; author: Btullis):

[labs/private@master] Move ceph profile authentication token to the role

https://gerrit.wikimedia.org/r/892380

gerritbot added a project: Patch-For-Review.Feb 27 2023, 10:52 AM
gerritbot added a comment.Feb 27 2023, 10:55 AM

Change 892380 merged by Btullis:

[labs/private@master] Move ceph profile authentication token to the role

https://gerrit.wikimedia.org/r/892380

BTullis mentioned this in rLPRI772c227f3d87: Move ceph profile authentication token to the role.Feb 27 2023, 10:55 AM
Maintenance_bot removed a project: Patch-For-Review.Feb 27 2023, 11:11 AM
gerritbot added a comment.Feb 27 2023, 11:15 AM

Change 892388 had a related patch set uploaded (by Btullis; author: Btullis):

[labs/private@master] Add keydata for ceph mgr daemons

https://gerrit.wikimedia.org/r/892388

gerritbot added a project: Patch-For-Review.Feb 27 2023, 11:15 AM

Change 892388 merged by Btullis:

[labs/private@master] Add keydata for ceph mgr daemons

https://gerrit.wikimedia.org/r/892388

BTullis mentioned this in rLPRI04d7059b694b: Add keydata for ceph mgr daemons.Feb 27 2023, 11:17 AM
Maintenance_bot removed a project: Patch-For-Review.Feb 27 2023, 11:30 AM
BTullis closed subtask T328123: Create puppet profiles for the new ceph cluster as Resolved.Mar 14 2023, 4:04 PM
gerritbot added a comment.Mar 24 2023, 4:34 PM

Change 902752 had a related patch set uploaded (by Btullis; author: Btullis):

[labs/private@master] Add dummy keydata for the new ceph OSD services

https://gerrit.wikimedia.org/r/902752

gerritbot added a project: Patch-For-Review.Mar 24 2023, 4:34 PM
gerritbot added a comment.Mar 24 2023, 4:36 PM

Change 902752 merged by Btullis:

[labs/private@master] Add dummy keydata for the new ceph OSD services

https://gerrit.wikimedia.org/r/902752

BTullis mentioned this in rLPRI731c09708cfe: Add dummy keydata for the new ceph OSD services.Mar 24 2023, 4:36 PM
Maintenance_bot removed a project: Patch-For-Review.Mar 24 2023, 5:10 PM
BTullis updated the task description. (Show Details)Mar 24 2023, 5:17 PM
JArguello-WMF closed subtask T330149: Deploy ceph mon and mgr processes to data-engineering cluster as Resolved.Mar 31 2023, 1:41 PM
hashar subscribed.May 24 2023, 9:43 AM
BTullis added a project: Data-Platform-SRE.Jun 9 2023, 11:56 AM
JArguello-WMF removed a project: Shared-Data-Infrastructure.Jun 29 2023, 1:43 PM
JArguello-WMF removed a project: Data-Engineering-Planning.Jun 29 2023, 4:14 PM
Ottomata mentioned this in T307959: [Event Platform] Design and Implement realtime enrichment pipeline for MW page change with content.Jun 30 2023, 4:52 PM
BTullis removed a parent task: T327259: Enable the Container Storage Interface (CSI) and the Ceph CSI plugin on dse-k8s cluster.Jul 18 2023, 10:41 AM
BTullis edited parent tasks, added: T327267: Create a DSE Kubernetes cluster with support for persistent storage from Ceph; removed: T310195: Ceph Data Infrastructure Request, T308317: Data Infrastructure as a Service MVP.Jul 18 2023, 10:59 AM
Gehel moved this task from Incoming to Epics on the Data-Platform-SRE board.Jul 19 2023, 3:40 PM
BTullis mentioned this in T330151: Deploy ceph osd processes to data-engineering cluster.Jul 20 2023, 10:49 AM
BTullis updated the task description. (Show Details)Jul 24 2023, 10:01 PM
BTullis closed subtask T330151: Deploy ceph osd processes to data-engineering cluster as Resolved.Aug 1 2023, 3:29 PM
Gehel closed subtask T326945: Decide on installation details for new ceph cluster as Resolved.Aug 8 2023, 3:45 PM
Gehel lowered the priority of this task from High to Low.Oct 11 2023, 8:48 AM
BTullis removed BTullis as the assignee of this task.Mar 15 2024, 5:58 PM
hashar unsubscribed.Mar 18 2024, 3:01 PM
gerritbot added a comment.Apr 26 2024, 2:04 PM

Change #1024695 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Use an LVM volume for /var/lib/ceph on cephosd nodes

https://gerrit.wikimedia.org/r/1024695

gerritbot added a project: Patch-For-Review.Apr 26 2024, 2:04 PM
gerritbot added a comment.Apr 29 2024, 9:16 AM

Change #1024695 merged by Btullis:

[operations/puppet@production] Use an LVM volume for /var/lib/ceph on cephosd nodes

https://gerrit.wikimedia.org/r/1024695

Maintenance_bot removed a project: Patch-For-Review.Apr 29 2024, 9:32 AM
BTullis closed subtask T363558: Switch the DPE Ceph cluster to use cephadm management as Declined.Apr 30 2024, 4:47 PM
Gehel closed subtask T330153: Configure Anycast load-balancing ceph radosgw services on the data-engineering cluster as Declined.May 7 2024, 8:51 AM
Gehel closed subtask T330152: Deploy ceph radosgw processes to data-engineering cluster as Declined.
Gehel closed this task as Resolved.May 7 2024, 8:55 AM
Gehel claimed this task.
Gehel subscribed.

We're not introducing object storage at this point, this task is completed

Gehel reopened subtask T330152: Deploy ceph radosgw processes to data-engineering cluster as Open.May 21 2024, 8:55 AM
Gehel removed a subtask: T330152: Deploy ceph radosgw processes to data-engineering cluster.
Gehel removed a subtask: T330153: Configure Anycast load-balancing ceph radosgw services on the data-engineering cluster.
BTullis renamed this task from Install Ceph Cluster for Data Engineering to Install Ceph Cluster for Data Platform Engineering.Sep 7 2024, 9:08 AM
BTullis updated the task description. (Show Details)
BTullis updated the task description. (Show Details)Sep 10 2024, 1:35 PM
BTullis mentioned this in T374447: Test the S3 and swift interfaces of rgw.eqiad.dpe.anycast.wmnet.Sep 10 2024, 1:44 PM
BTullis added a subtask: T330153: Configure Anycast load-balancing ceph radosgw services on the data-engineering cluster.
BTullis added a subtask: T374447: Test the S3 and swift interfaces of rgw.eqiad.dpe.anycast.wmnet.Sep 10 2024, 1:46 PM
BTullis added a subtask: T330152: Deploy ceph radosgw processes to data-engineering cluster.Sep 10 2024, 2:06 PM
BTullis closed subtask T374447: Test the S3 and swift interfaces of rgw.eqiad.dpe.anycast.wmnet as Resolved.Sep 13 2024, 2:23 PM
brouberol closed subtask T376401: Enable the cephfs services on the DPE Ceph cluster as Resolved.Oct 17 2024, 4:08 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits