On Tuesday 2015-12-15, there will be a 2-hour window for EL database maintenance, during which EL mysql consumers will be stopped. When restarting, the consumers will automatically backfill the events buffered in kafka.
With the current queue sizes (5000, 1000), 4 consumers backfilling buffered events from Kafka, may fill up RAM and cause out-of-memory errors. The queue sizes need to be smaller to not fill up RAM and permit backfilling without problems.
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Reset low mysql consumer queue sizes | eventlogging | master | +2 -2 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • Nuria | T120187 Pending maintenance on the eventlogging databases (db1046, db1047, dbstore1002, other dbstores) | |||
| Resolved | mforns | T121120 2 hour outage to update mysql on EL slaves {oryx} [3 pts] | |||
| Resolved | mforns | T121384 Prepare EL queue sizes for backfilling {oryx} [3 pts] |
Change 258982 had a related patch set uploaded (by Mforns):
Reset low mysql consumer queue sizes