Page MenuHomePhabricator

Inserts events by scid {oryx}
Closed, ResolvedPublic

Description

Due to performance problems in the EL consumer, this change tries to optimize batch EveneLogging inserts grouping them by scid.

There's a gerrit changeset already pushed for this:
https://gerrit.wikimedia.org/r/#/c/205759/

Event Timeline

mforns created this task.Apr 22 2015, 3:40 PM
mforns raised the priority of this task from to Needs Triage.
mforns updated the task description. (Show Details)
mforns added a subscriber: mforns.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 22 2015, 3:40 PM
mforns claimed this task.Apr 22 2015, 3:40 PM
mforns set Security to None.
mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.

Change 205759 had a related patch set uploaded (by Mforns):
Insert events by scid

https://gerrit.wikimedia.org/r/205759

Found a bug in the code, the itertools.groupby step is not working as expected...
Working on it to fix it right now.

kevinator renamed this task from Inserts events by scid to Inserts events by scid {oryx}.Apr 23 2015, 3:40 PM

The bug is fixed an pushed to gerrit.
However, I'll still write a script to generate lots of event requests, to be able to test "real" thoughput in beta-labs.
When I've tested that, will open the change for review.

kevinator triaged this task as High priority.Apr 24 2015, 3:34 PM

Change 205759 merged by Milimetric:
Insert events by scid

https://gerrit.wikimedia.org/r/205759

Milimetric closed this task as Resolved.Apr 29 2015, 3:36 PM
Milimetric moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Milimetric added a subscriber: Milimetric.