Page MenuHomePhabricator

Labs: puppetize stripe_cache_size tweaks on labstores
Closed, ResolvedPublic

Description

This helped alleviate the pressure greatly on the 2015-04-13 labstore outage and needs to be made permanent.

Details

Related Gerrit Patches:
operations/puppet : productionLabs: move set-stripe-cache to cron
operations/puppet : productionLabs: set_strip_cache for labstores

Event Timeline

coren created this task.Apr 14 2015, 5:03 PM
coren claimed this task.
coren raised the priority of this task from to Unbreak Now!.
coren updated the task description. (Show Details)
coren added a subscriber: coren.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 14 2015, 5:03 PM
coren moved this task from Backlog to Doing on the Labs-Q4-Sprint-3 board.Apr 17 2015, 6:35 PM
coren added a comment.Apr 17 2015, 8:29 PM

This isn't a real sysctl, and only becomes available once the devices are assembled. Given that setting the size is an idempotent operation, the easiest thing to do is to just do it every puppet run (otherwise we need to complicate things by making an event-base udev response to the appearance of a disk for no immediately obvious benefit).

Change 204932 had a related patch set uploaded (by coren):
Labs: set_strip_cache for labstores

https://gerrit.wikimedia.org/r/204932

Change 204932 merged by coren:
Labs: set_strip_cache for labstores

https://gerrit.wikimedia.org/r/204932

coren closed this task as Resolved.Apr 17 2015, 9:20 PM
coren removed a project: Patch-For-Review.
coren set Security to None.

Deployed.

coren moved this task from Doing to Done on the Labs-Q4-Sprint-3 board.Apr 20 2015, 5:00 PM
coren reopened this task as Open.Jun 8 2015, 2:07 PM

Turns out puppet doesn't reliably run the script; this needs to be changed to cron.

coren moved this task from To Do to Doing on the Labs-Sprint-101 board.Jun 8 2015, 5:31 PM

Change 216976 had a related patch set uploaded (by coren):
Labs: move set-stripe-cache to cron

https://gerrit.wikimedia.org/r/216976

coren closed this task as Resolved.Jun 16 2015, 5:12 PM
coren moved this task from Code Review / Blocked to Done on the Labs-Sprint-102 board.