Page MenuHomePhabricator

titan100[12] ram/ssd upgrade coordination
Closed, ResolvedPublic

Description

This task will coordinate the scheduling and installation of RAM and SSD upgrades into titan100[12]

These upgrades were initially requested for order on T359449, but checking on-site decommission servers/spare parts has shown we have the SSDs available from there, but need to order the RAM via T359449.

titan1001:

  • - onsite: allocate (2) 480GB SSDs
  • - onsite: receive in order T359449
  • - onsite & service owners: schedule downtime for host
  • - service owners: depool host for maint window and put into maint in icinga when services are halted
  • - service owners: power down host for onsite work (RAM update requires full downtime)
  • - onsite: install SSD additions (do not remove existing disks, this is appending more storage not replacing)
  • - onsite: install RAM additions and ensure it is detected during post
  • - onsite & service owners: ensure system is remotely accessible for service owners to return to service.
  • - service owners: return host to service

titan1002:

  • - onsite: allocate (2) 480GB SSDs
  • - onsite: receive in order T359449
  • - onsite & service owners: schedule downtime for host
  • - service owners: depool host for maint window and put into maint in icinga when services are halted
  • - service owners: power down host for onsite work (RAM update requires full downtime)
  • - onsite: install SSD additions (do not remove existing disks, this is appending more storage not replacing)
  • - onsite: install RAM additions and ensure it is detected during post
  • - onsite & service owners: ensure system is remotely accessible for service owners to return to service.
  • - service owners: return host to service

Related Objects

StatusSubtypeAssignedTask
ResolvedVRiley-WMF

Event Timeline

RobH triaged this task as Medium priority.Mar 28 2024, 4:09 PM
RobH created this task.
RobH added a parent task: Unknown Object (Task).
RobH mentioned this in Unknown Object (Task).

We currently have all these parts needed for this upgrade. @fgiunchedi do you have an estimated time for these upgrades to take place? Please let us know and we'll schedule it. Thanks!

Hey @VRiley-WMF, I'll help out with this one for the o11y side.

Schedule-wise would this Thursday (4/11) sometime after 11a Eastern work?

Thanks!

That works for me. I'll be there to assist with it. Thank you!

Mentioned in SAL (#wikimedia-operations) [2024-04-11T16:03:41Z] <herron> beginning rolling hardware upgrades for titan100[12] T361251

VRiley-WMF updated the task description. (Show Details)

Worked with @herron and upgraded these servers. They came back properly and making this ticket as resolved. Thanks!