Page MenuHomePhabricator

Prepare EL queue sizes for backfilling {oryx} [3 pts]
Closed, ResolvedPublic

Description

On Tuesday 2015-12-15, there will be a 2-hour window for EL database maintenance, during which EL mysql consumers will be stopped. When restarting, the consumers will automatically backfill the events buffered in kafka.

With the current queue sizes (5000, 1000), 4 consumers backfilling buffered events from Kafka, may fill up RAM and cause out-of-memory errors. The queue sizes need to be smaller to not fill up RAM and permit backfilling without problems.

Event Timeline

mforns created this task.Dec 14 2015, 12:03 PM
mforns claimed this task.
mforns raised the priority of this task from to Needs Triage.
mforns updated the task description. (Show Details)
mforns added a project: Analytics-Kanban.
mforns added a subscriber: mforns.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptDec 14 2015, 12:03 PM

Change 258982 had a related patch set uploaded (by Mforns):
Reset low mysql consumer queue sizes

https://gerrit.wikimedia.org/r/258982

Change 258982 merged by Mforns:
Reset low mysql consumer queue sizes

https://gerrit.wikimedia.org/r/258982

mforns renamed this task from Prepare EL queue sizes for backfilling {oryx} to Prepare EL queue sizes for backfilling {oryx} [3 pts].Dec 14 2015, 10:23 PM
mforns set Security to None.
Nuria closed this task as Resolved.Dec 23 2015, 7:55 PM