Page MenuHomePhabricator

eqiad: Move maps1001 same rack A4
Closed, ResolvedPublic

Description

@elukey moving this server to make room for an-workers. Can I do this Monday 8 Feb @1515UTC. It will be the same rack so network ports will stay the same, just need 5 minutes of downtime to relocate.

Event Timeline

Adding @hnowlan to understand if the time window is ok for the host (we briefly had a chat about it on IRC).

The idea is to:

  1. shutdown the node
  2. move it to a different rack within the same row (no ip change, no vlan change)
  3. boot it up again

It should take max 30 mins but it may also depend on how busy Chris is in the DC, if there are emergencies etc..

Since this will free space up for Analytics Hadoop workers, thanks a lot!

Sounds good, I will have everything ready at 1500. Thanks for the heads-up!

@hnowlan just a heads up that it looks like the depool of maps1001 left maps@eqiad underprovisioned:
https://grafana.wikimedia.org/d/000000305/maps-performances?orgId=1&from=1612777514651&to=1612819057362

The still-pooled servers have been at 100% CPU for some hours, and we also had a page due to some HTTP requests timing out.

Thanks for the heads up @CDanis - I've repooled. It appears there were some issues with the weights of other maps hosts that should have prevented this having an impact, I've rectified that now too.