Page MenuHomePhabricator
Create Task
Maniphest T96872

Inserts events by scid {oryx}
Closed, ResolvedPublic
Actions

Description

Due to performance problems in the EL consumer, this change tries to optimize batch EveneLogging inserts grouping them by scid.

There's a gerrit changeset already pushed for this:
https://gerrit.wikimedia.org/r/#/c/205759/

Details

SubjectRepoBranchLines +/-
Insert events by scidmediawiki/extensions/EventLoggingmaster+271 -45
Customize query in gerrit

Related Objects
Search...

Event Timeline

mforns created this task.Apr 22 2015, 3:40 PM
mforns raised the priority of this task from to Needs Triage.
mforns updated the task description. (Show Details)
mforns added projects: Analytics-Kanban, MediaWiki-extensions-EventLogging.
mforns subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 22 2015, 3:40 PM
mforns claimed this task.Apr 22 2015, 3:40 PM
mforns set Security to None.
mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.
gerritbot subscribed.Apr 22 2015, 6:48 PM

Change 205759 had a related patch set uploaded (by Mforns):
Insert events by scid

https://gerrit.wikimedia.org/r/205759

gerritbot added a project: Patch-For-Review.Apr 22 2015, 6:48 PM
mforns added a comment.Apr 22 2015, 10:43 PM

Found a bug in the code, the itertools.groupby step is not working as expected...
Working on it to fix it right now.

mforns moved this task from In Progress to In Code Review on the Analytics-Kanban board.Apr 23 2015, 2:26 PM
• kevinator renamed this task from Inserts events by scid to Inserts events by scid {oryx}.Apr 23 2015, 3:40 PM
• kevinator added a subtask: T96082: Troubleshoot EventLogging missing chunks of data {oryx}.
mforns moved this task from In Code Review to In Progress on the Analytics-Kanban board.Apr 23 2015, 3:59 PM
mforns added a comment.Apr 23 2015, 4:48 PM

The bug is fixed an pushed to gerrit.
However, I'll still write a script to generate lots of event requests, to be able to test "real" thoughput in beta-labs.
When I've tested that, will open the change for review.

• kevinator triaged this task as High priority.Apr 24 2015, 3:34 PM
mforns moved this task from In Progress to In Code Review on the Analytics-Kanban board.Apr 27 2015, 8:11 PM
gerritbot added a comment.Apr 28 2015, 5:31 PM

Change 205759 merged by Milimetric:
Insert events by scid

https://gerrit.wikimedia.org/r/205759

mforns mentioned this in rMEXT44cf0feec11e: Updated mediawiki/extensions Project: mediawiki/extensions/EventLogging….Apr 28 2015, 5:31 PM
mforns mentioned this in rEEVLbe1e0555da81: Insert events by scid.
Milimetric closed this task as Resolved.Apr 29 2015, 3:36 PM
Milimetric moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Milimetric subscribed.
• kevinator moved this task from Ready to Deploy to Done on the Analytics-Kanban board.May 2 2015, 1:15 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL