Page MenuHomePhabricator

eqiad cache cluster re-arrangements
Closed, ResolvedPublic

Description

(see also T125485 for related more-complicated situation in esams)

cp1056, cp1057, cp1069, cp1070 - The only non-SSD still use in eqiad - former bits cluster, currently misc cluster

The upload cluster is over-provisioned (as with esams, we were being conservative with how TLS rollout would go on CPU) and could easily lose 2x of its older (but still decent) spec SSD machines. The parsoid cluster is due for termination in the next few weeks and also has 2x decent spec SSD machines to give up. We can send these 4 machines to the misc cluster and decom the non-SSD ones there.

Steps:

  • 1. Decom cache_parsoid, freeing cp1045 + cp1058
  • 2. Remove cp10[56]1 from cache_upload (excess capacity)
  • 3. Add the 4x machines above into cache_misc
  • 4. Remove the current 4x non-SSD from cache_misc for decom/reclaim/spare
  • 5. Actually sort out the final disposition of those 4 unused machines

Event Timeline

BBlack raised the priority of this task from to Needs Triage.
BBlack updated the task description. (Show Details)
BBlack added projects: Traffic, ops-eqiad.
BBlack added subscribers: BBlack, mark, faidon, ema.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald Transcript
BBlack set Security to None.

Change 272323 had a related patch set uploaded (by BBlack):
decom cache_parsoid

https://gerrit.wikimedia.org/r/272323

Change 272723 had a related patch set uploaded (by BBlack):
cache_misc: remove cp1056, cp1069

https://gerrit.wikimedia.org/r/272723

Change 272725 had a related patch set uploaded (by BBlack):
cp10[56]1: upload->misc

https://gerrit.wikimedia.org/r/272725

Change 272726 had a related patch set uploaded (by BBlack):
cache_misc: remove cp1057, cp1070

https://gerrit.wikimedia.org/r/272726

Change 272723 merged by BBlack:
cache_misc: remove cp1056, cp1069

https://gerrit.wikimedia.org/r/272723

Change 272725 merged by BBlack:
cp10[56]1: upload->misc

https://gerrit.wikimedia.org/r/272725

Change 272726 merged by BBlack:
cache_misc: remove cp1057, cp1070

https://gerrit.wikimedia.org/r/272726

Change 272792 had a related patch set uploaded (by BBlack):
2layer: remove dead nodes storage_size

https://gerrit.wikimedia.org/r/272792

Change 272792 merged by BBlack:
2layer: remove dead nodes storage_size

https://gerrit.wikimedia.org/r/272792

BBlack updated the task description. (Show Details)

Status update: this is basically-done other than sorting out decom/reclaim.

ema reassigned this task from Christopher to Cmjohnson.
ema triaged this task as Medium priority.
ema added a subscriber: Christopher.
ema edited subscribers, added: Cmjohnson; removed: Christopher.