We must decommission more of the older appservers for:
- Free rack space in eqiad
- Decommission hardware that is more than 5 years old
- Test a number of cores/memory equal to what we have in codfw
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Invalid | None | T125673 Switch over from Eqiad to Codfw as primary datacentre | |||
| Resolved | jcrespo | T124670 Figure out and document the datacenter switchover process | |||
| Resolved | Krinkle | T124671 Switchover of the application servers to codfw | |||
| Resolved | None | T131748 Refresh the appservers puppet code/configs | |||
| Resolved | Joe | T126242 Reduce the number of appservers we're using in eqiad | |||
| Resolved | • Cmjohnson | T129060 mw1026-69 are shut down and should be physically decommissioned | |||
| Resolved | • Cmjohnson | T133770 mw1070-89 and mw1121-30 are shut down and should be physically decommissioned |
Event Timeline
Comment Actions
I depooled mw1025-1050 for now setting all of them to 'inactive'.
I'll wait tomorrow to merge the patches to make that definitive.
Comment Actions
Change 275374 had a related patch set uploaded (by Giuseppe Lavagetto):
appservers: decommission permanently mw1026-69
Comment Actions
Change 275374 merged by Giuseppe Lavagetto:
appservers: decommission permanently mw1026-69
Comment Actions
I have removed every reference to mw1026-1069 from puppet and conftool, and shut down the machines. I'' also opening a separated ticket for decommissioning
Comment Actions
Change 275383 had a related patch set uploaded (by Giuseppe Lavagetto):
scap: remove decommissioned appservers from the scap dsh group
Comment Actions
Change 275383 merged by Giuseppe Lavagetto:
scap: remove decommissioned appservers from the scap dsh group
Comment Actions
I think we can reduce the pool size further, but it's already smaller than the current pool in codfw
Comment Actions
Change 275756 had a related patch set uploaded (by Giuseppe Lavagetto):
Remove decommissioned appservers
Comment Actions
Change 275756 abandoned by Giuseppe Lavagetto:
Remove decommissioned appservers
Reason:
Already done in I040c2e27b750ea1906b989b2380a10bbd23f7906
Comment Actions
I will adjust the weights in various clusters, and start removing more servers today, up to the point where I don't feel comfortable removing more.
I want to have all the mw* clusters to an average utilization of around 20% at least.
Comment Actions
Mentioned in SAL [2016-04-01T07:00:27Z] <_joe_> depooling mw1070-89 from the appserver cluster. T126242
Comment Actions
I am waiting until we switch back mediawiki from codfw before I definitively decommission the last batch of appservers I removed.
Comment Actions
Change 285604 had a related patch set uploaded (by Giuseppe Lavagetto):
mediawiki: remove decommissioned appservers
Comment Actions
Change 285605 had a related patch set uploaded (by Giuseppe Lavagetto):
dhcp: remove entries for decommissioned appservers
Comment Actions
Mentioned in SAL [2016-04-27T08:40:50Z] <_joe_> stopping puppet on mw10[7-8][0-9] and mw112[1-9]/mw1130 for T126242
Comment Actions
Change 285604 merged by Giuseppe Lavagetto:
mediawiki: remove decommissioned appservers
Comment Actions
Mentioned in SAL [2016-04-27T09:56:30Z] <_joe_> clean puppet certs and facts on mw10[7-8][0-9] and mw112[1-9]/mw1130 for T126242
Comment Actions
Mentioned in SAL [2016-04-27T10:01:34Z] <_joe_> shutting down mw10[7-8][0-9] and mw112[1-9]/mw1130 for T126242
Comment Actions
I think I can close this task as resolved, the subtasks aren't real blockers, more of "related tickets"
Comment Actions
Change 285605 merged by Cmjohnson:
dhcp: remove entries for decommissioned appservers