Page MenuHomePhabricator

Replace stat1001
Closed, ResolvedPublic13 Story Points

Details

Related Gerrit Patches:
operations/puppet : productionRemove stat1001 from site.pp
operations/puppet : productionstat1001 is now a spare and can be reclaimed
operations/puppet : productionFix thorium typo
operations/puppet : productionSymlink limn-public-data so it is accessible at datasets.wikimedia.org
operations/puppet : productionRemove a few more references to stat1001
operations/puppet : productions/stat1001/throrium
operations/puppet : productionInstall zip for statistics::web role (on thorium)
operations/puppet : productionAdd thorium to list of statistics_services in hiera
analytics/pivot/deploy : masterAdd thorium to pivot scap targets
operations/puppet : productionEnsure geowiki base path exists
operations/puppet : productionPuppetize thorium as stat1001 replacement

Event Timeline

Nuria created this task.Oct 28 2016, 5:54 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 28 2016, 5:54 PM
Nuria added a comment.Oct 28 2016, 5:56 PM

From our notes:

Replaces OOW R510. stat1001 is just a webhost for large datasets, similar to dataset1001, but more dedicated for analytics cluster generated data. We may be able to consolodate data here onto dataset1* boxes, not sure. This possibly could even be hosted in codfw, if really really necessary.

Now it also runs pivot.

Nuria moved this task from Incoming to Backlog (Later) on the Analytics board.Oct 31 2016, 3:30 PM
Nuria moved this task from Backlog (Later) to Operational Excellence Future on the Analytics board.
Nuria added a comment.EditedNov 3 2016, 4:00 PM

What is the overall objective?

Replace outdated hardware

Add better resiliency for static domains.

  1. Order new box (Q2)
  1. Move everything (Q2)
  1. Move out everything that can be on a large vm so websites have better resiliency (Q3) pivot analytics.wikimedia.org
Nuria added a comment.Nov 3 2016, 4:02 PM

Tasks to copy everything:

  • rsync
  • apply puppet
  • put box in varnish
  • announcement
  • switch
Nuria set the point value for this task to 13.Nov 3 2016, 4:02 PM
Nuria edited projects, added Analytics-Kanban; removed Analytics.
elukey added a subscriber: elukey.Nov 3 2016, 4:18 PM
Ottomata claimed this task.Nov 29 2016, 5:15 PM

Change 324234 had a related patch set uploaded (by Ottomata):
Puppetize thorium as stat1001 replacement

https://gerrit.wikimedia.org/r/324234

Change 324234 merged by Ottomata:
Puppetize thorium as stat1001 replacement

https://gerrit.wikimedia.org/r/324234

Change 324235 had a related patch set uploaded (by Ottomata):
Ensure geowiki base path exists

https://gerrit.wikimedia.org/r/324235

Change 324236 had a related patch set uploaded (by Ottomata):
Add thorium to pivot scap targets

https://gerrit.wikimedia.org/r/324236

Change 324235 merged by Ottomata:
Ensure geowiki base path exists

https://gerrit.wikimedia.org/r/324235

Change 324236 merged by Ottomata:
Add thorium to pivot scap targets

https://gerrit.wikimedia.org/r/324236

Change 324237 had a related patch set uploaded (by Ottomata):
Add thorium to list of statistics_services in hiera

https://gerrit.wikimedia.org/r/324237

Change 324237 merged by Ottomata:
Add thorium to list of statistics_services in hiera

https://gerrit.wikimedia.org/r/324237

elukey added a comment.Dec 1 2016, 1:57 PM

thorium has been reimaged by Rob, re-deployed pivot (this time it works) and rsynced stats.w.o from stat1001.

Change 325314 had a related patch set uploaded (by Ottomata):
Install zip for statistics::web role (on thorium)

https://gerrit.wikimedia.org/r/325314

Change 325314 merged by Ottomata:
Install zip for statistics::web role (on thorium)

https://gerrit.wikimedia.org/r/325314

Change 326490 had a related patch set uploaded (by Ottomata):
s/stat1001/throrium

https://gerrit.wikimedia.org/r/326490

Announcement out. I plan do do this 15:00 UTC tomorrow Dec 13th.

Change 326490 merged by Ottomata:
s/stat1001/throrium

https://gerrit.wikimedia.org/r/326490

Change 326988 had a related patch set uploaded (by Ottomata):
Remove a few more references to stat1001

https://gerrit.wikimedia.org/r/326988

Change 326988 merged by Ottomata:
Remove a few more references to stat1001

https://gerrit.wikimedia.org/r/326988

Next up is to decom stat1001. I'll wait a week or so to verify that thorium is totally good.

Change 326994 had a related patch set uploaded (by Ottomata):
Symlink limn-public-data so it is accessible at datasets.wikimedia.org

https://gerrit.wikimedia.org/r/326994

Change 326994 merged by Ottomata:
Symlink limn-public-data so it is accessible at datasets.wikimedia.org

https://gerrit.wikimedia.org/r/326994

Change 327000 had a related patch set uploaded (by Ottomata):
Fix thorium typo

https://gerrit.wikimedia.org/r/327000

Change 327000 merged by Ottomata:
Fix thorium typo

https://gerrit.wikimedia.org/r/327000

Change 329243 had a related patch set uploaded (by Ottomata):
stat1001 is now a spare and can be reclaimed

https://gerrit.wikimedia.org/r/329243

Change 329243 merged by Ottomata:
stat1001 is now a spare and can be reclaimed

https://gerrit.wikimedia.org/r/329243

Mentioned in SAL (#wikimedia-operations) [2016-12-26T21:16:28Z] <ottomata> disabled active checks of stat1001 services T149438

Change 329247 had a related patch set uploaded (by Ottomata):
Remove stat1001 from site.pp

https://gerrit.wikimedia.org/r/329247

Change 329247 merged by Ottomata:
Remove stat1001 from site.pp

https://gerrit.wikimedia.org/r/329247

Nuria closed this task as Resolved.Jan 18 2017, 6:19 PM