Page MenuHomePhabricator

Expand HTTP frontend clusters with new hardware
Closed, ResolvedPublic

Description

We'll soon need:

  • Extra resources, mainly CPU for all of our HTTP frontend (Varnish, nginx) clusters, due to an increased use of HTTPS & IPsec
  • Clusters for codfw

As HTTPS scalability is one of the quarterly goals for FY Q3 2015 and there's a lead time for procurement and site installation (plus an extra delay for esams which needs an upgrade the most, due to not having a dedicated field tech), we'll need to do this procurement very soon.

The current blocker for this is testing of our new target platform (post- T86648 & T81543) as to have more accurate figures for proceeding with procurement. Current agreement on planning suggests:

  • 2nd week of January + 2-3 first weeks of February for limited performance testing
  • Late February/early March for proceeding with procurement in parallel to reinstallation of the current systems
  • Rest of March: setup & install of the new servers

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 202964 had a related patch set uploaded (by BBlack):
T86663 4.3: pool 3044; depool 3017,3013,amssq4[78]

https://gerrit.wikimedia.org/r/202964

Change 202964 merged by BBlack:
T86663 4.3: pool 3044; depool 3017,3013,amssq4[78]

https://gerrit.wikimedia.org/r/202964

Change 202967 had a related patch set uploaded (by BBlack):
T86663 4.3: shuffle roles, no repool yet

https://gerrit.wikimedia.org/r/202967

Change 202967 merged by BBlack:
T86663 4.3: shuffle roles, no repool yet

https://gerrit.wikimedia.org/r/202967

Change 202969 had a related patch set uploaded (by BBlack):
T86663 4.3: repool cp301[37]

https://gerrit.wikimedia.org/r/202969

Change 202970 had a related patch set uploaded (by BBlack):
T86663 4.4: pool 3045; depool 3018,3014,amssq49-50

https://gerrit.wikimedia.org/r/202970

Change 202970 merged by BBlack:
T86663 4.4: pool 3045; depool 3018,3014,amssq49-50

https://gerrit.wikimedia.org/r/202970

Change 202972 had a related patch set uploaded (by BBlack):
T86663 4.4: shuffle roles, no repool yet

https://gerrit.wikimedia.org/r/202972

Change 202972 merged by BBlack:
T86663 4.4: shuffle roles, no repool yet

https://gerrit.wikimedia.org/r/202972

Change 202973 had a related patch set uploaded (by BBlack):
T86663 4.4: repool cp301[48]

https://gerrit.wikimedia.org/r/202973

Based on the overnight esams traffic graphs up through present, the Step 4 moves last night were probably slightly more aggressive than ideal, but still within reason. Given that the moves in Step 5 are slightly-less-impactful (no mobile), it should be possible to slot them into the same time window over two EU evenings (one set of 4 moves starting circa 20:00 UTC Apr 9th, and the final set either Fri Apr 10 or Mon Apr 13, depending how we feel on risking it just before a weekend).

Change 203231 had a related patch set uploaded (by BBlack):
T86663 5.1: pool 3036; depool 3003,amssq5[12]

https://gerrit.wikimedia.org/r/203231

Change 203231 merged by BBlack:
T86663 5.1: pool 3036; depool 3003,amssq5[12]

https://gerrit.wikimedia.org/r/203231

Change 203233 had a related patch set uploaded (by BBlack):
T86663 5.1: switch cp3003 role

https://gerrit.wikimedia.org/r/203233

Change 203235 had a related patch set uploaded (by BBlack):
T86663 5.1: repool cp3003

https://gerrit.wikimedia.org/r/203235

Change 203238 had a related patch set uploaded (by BBlack):
T86663 5.1: pool 3037; depool 3004,amssq5[34]

https://gerrit.wikimedia.org/r/203238

Change 203238 merged by BBlack:
T86663 5.1: pool 3037; depool 3004,amssq5[34]

https://gerrit.wikimedia.org/r/203238

Change 203239 had a related patch set uploaded (by BBlack):
T86663 5.2: switch cp3004 role

https://gerrit.wikimedia.org/r/203239

Change 203240 had a related patch set uploaded (by BBlack):
T86663 5.2: repool cp3004

https://gerrit.wikimedia.org/r/203240

Change 203264 had a related patch set uploaded (by BBlack):
T86663 5.5: pool 3046; depool 3007,amssq59-60

https://gerrit.wikimedia.org/r/203264

Change 203264 merged by BBlack:
T86663 5.5: pool 3046; depool 3007,amssq59-60

https://gerrit.wikimedia.org/r/203264

Change 203267 had a related patch set uploaded (by BBlack):
T86663 5.5: switch cp3007 role

https://gerrit.wikimedia.org/r/203267

Change 203270 had a related patch set uploaded (by BBlack):
T86663 5.2: repool cp3007

https://gerrit.wikimedia.org/r/203270

Change 203271 had a related patch set uploaded (by BBlack):
T86663 5.6: pool 3047; depool 3008,amssq6[12]

https://gerrit.wikimedia.org/r/203271

Change 203271 merged by BBlack:
T86663 5.6: pool 3047; depool 3008,amssq6[12]

https://gerrit.wikimedia.org/r/203271

Change 203274 had a related patch set uploaded (by BBlack):
T86663 5.6: switch cp3008 role

https://gerrit.wikimedia.org/r/203274

Change 203276 had a related patch set uploaded (by BBlack):
T86663 5.6: repool cp3008

https://gerrit.wikimedia.org/r/203276

Change 203316 had a related patch set uploaded (by BBlack):
T86663 5.3: pool 3038; depool 3005,amssq55

https://gerrit.wikimedia.org/r/203316

Change 203316 merged by BBlack:
T86663 5.3: pool 3038; depool 3005,amssq55

https://gerrit.wikimedia.org/r/203316

Change 203319 had a related patch set uploaded (by BBlack):
T86663 5.3: switch cp3005 role

https://gerrit.wikimedia.org/r/203319

Change 203322 had a related patch set uploaded (by BBlack):
T86663 5.3: repool cp3005

https://gerrit.wikimedia.org/r/203322

Change 203324 had a related patch set uploaded (by BBlack):
T86663 5.4: pool 3039; depool 3006,amssq56

https://gerrit.wikimedia.org/r/203324

Change 203325 had a related patch set uploaded (by BBlack):
T86663 5.4: switch cp3006 role

https://gerrit.wikimedia.org/r/203325

Change 203326 had a related patch set uploaded (by BBlack):
T86663 5.4: repool cp3006

https://gerrit.wikimedia.org/r/203326

Change 203327 had a related patch set uploaded (by BBlack):
T86663 5.7: pool 3048; depool 3009,amssq57

https://gerrit.wikimedia.org/r/203327

Change 203328 had a related patch set uploaded (by BBlack):
T86663 5.7: switch cp3009 role

https://gerrit.wikimedia.org/r/203328

Change 203329 had a related patch set uploaded (by BBlack):
T86663 5.7: repool cp3009

https://gerrit.wikimedia.org/r/203329

Change 203330 had a related patch set uploaded (by BBlack):
T86663 5.8: pool 3049; depool 3010,amssq58

https://gerrit.wikimedia.org/r/203330

Change 203331 had a related patch set uploaded (by BBlack):
T86663 5.8: switch cp3010 role

https://gerrit.wikimedia.org/r/203331

Change 203332 had a related patch set uploaded (by BBlack):
T86663 5.8: repool cp3010

https://gerrit.wikimedia.org/r/203332

Change 203385 had a related patch set uploaded (by BBlack):
T86663: depool amssq56-58

https://gerrit.wikimedia.org/r/203385

Change 203324 merged by BBlack:
T86663 5.4: pool 3039; depool 3006

https://gerrit.wikimedia.org/r/203324

Change 203327 merged by BBlack:
T86663 5.7: pool 3048; depool 3009

https://gerrit.wikimedia.org/r/203327

Change 203330 merged by BBlack:
T86663 5.8: pool 3049; depool 3010

https://gerrit.wikimedia.org/r/203330