Page MenuHomePhabricator

replacement hardware for iridium (phabricator)
Closed, ResolvedPublic

Description

The current production phabricator machine is "iridium" on trusty.

Then we have "phab2001" in codfw on jessie, using the same role but obviously not in prod.

What we want is "phab1001" in eqiad, on jessie, to go along with phab2001.

There was an original plan to make phab2001 the prod machine and then reinstall iridium and rename it to phab1001 while prod traffic is not on it.

But since we can't just switch a single service over to codfw while others stay in eqiad yet (due to lack of per-service routing in VCL), the prod system has to stay in eqiad for the time being.

Checking the warranty expiration of iridium i see it is soon on 2017-03-23.

Given these things i suggest we get new hardware for eqiad, install it as "phab1001" with jessie, confirm the phabricator role works fine on jessie (T158434), then switch prod from iridium to phab1001, both being in eqiad, and finally decom/reclaim iridium as a spare and call the "rename" ticket T152129 resolved.

So this is for a physical machine that is similar to iridium., HW type: Dell PowerEdge R420, wmf4547

https://racktables.wikimedia.org/index.php?page=object&tab=default&object_id=2161

Event Timeline

The advantages here (over reinstalling iridium): - less maintenance/downtime for end users - don't have to fix everything at once in a single maintenance period (this is also a switch from trusty to jessie) - reverting is trivial, after a reinstall it's not - avoid running on hardware out of warranty by the end of March ...

procurement ticket for iridium was https://rt.wikimedia.org/Ticket/Display.html?id=6772

they (12 misc servers) were listed as "dual cpu, 64GB, dual 500GB HDD "

RobH added a subscriber: mark.

iridium's (current eqiad phab host) specs:

  • 1U system
  • Dual Intel Xeon CPU E5-2450 v2 @ 2.5 GHz / 8 cores
  • 64GB RAM
  • Dual 500GB SATA (sw raid)

phab2001's (codfw phab host) specs:

  • 1U system
  • Dual Intel Xeon CPU E5-2640 v3 @ 2.60GHz / 8 cores
  • 64GB RAM
  • Dual 1TB SATA (sw raid)

The proposed replacement system for iridium is wmf4747, which will have the hostname phab1001 (since if this is approved, iridium won't be reimaged to that hostname and upgrade to debian from ubuntu, it'll just decommission)

  • 1U
  • Dual Intel Xeon CPU E5-2640 v3 @ 2.60GHz / 8 cores
  • 64GB RAM
  • Dual 1TB SATA (sw raid)
  • Purchased on 2013-03-24 via T130738

I've assigned this task to @mark for his approval from our spare allocations in eqiad. We currently have 5 spares of this identical build, and three other assorted spares for spare allocation. (Viewable on spares sheet.)

Please comment and assign back to me for follow up. Thanks!

For testing phabricator on Jessie, I've setup phabricator-01.wmflabs.org. All looks ok at the moment.

Migrations complete for phab-01 -> phabricator (labs instance). Phabricator officially currently works on jessie now. I haven't found any problems yet.

Bump, any update on this please?

RobH added a subscriber: faidon.

I'm assigning this to @faidon since @mark is away at present.

@faidon: Basically I need mgmt approval to allocate one of the spare systems in eqiad, we have 5 of these spares, plus a few other of other types. Please review and comment/assign back to me.

Thanks!

That allocation sounds temporary, right? Sounds fine.

That allocation sounds temporary, right? Sounds fine.

Not temp no, its for a permanent allocation of spare system wmf4747, which has a hw warranty end date of 2019-03-24. It will replace iridium, which is active and cannot really be reinstalled without downtime. Iridium then would return to spares, but it is an older machine which just ended its warranty last month.

Your approval seems dependent on temp, and it isn't so please advise.

Resolving this task, as there is now a setup task for the system.