Page MenuHomePhabricator

Wikitext events need to be sampled {lion}
Closed, ResolvedPublic1 Estimated Story Points

Description

The inflow of Wikitext events into eventlogging is to high for the system to the point that yesterday vanadium stopped processing events due to the VE inflow. Intake of events is 200 events per second about 150 of those events (per second) are invalid.

This can be fixing by sampling events, which is done for pretty much any incoming schema. Since VE needs all events for a session the sampling should happen around token, via bucketing. For example, if session token is between 1 and 10 we can send events it such if session token is <= 5, that would be a sampling of 50%.

Event Timeline

Nuria assigned this task to Milimetric.
Nuria raised the priority of this task from to Needs Triage.
Nuria updated the task description. (Show Details)
kevinator renamed this task from VE events need to be sampled to VE events need to be sampled {lion}.Mar 19 2015, 2:27 PM
kevinator triaged this task as High priority.
kevinator renamed this task from VE events need to be sampled {lion} to Wikitext events need to be sampled {lion}.Mar 27 2015, 11:02 PM
kevinator updated the task description. (Show Details)

Change 199132 had a related patch set uploaded (by Alex Monk):
Collect only 25% of WikiEditor events

https://gerrit.wikimedia.org/r/199132

Change 204156 had a related patch set uploaded (by Jforrester):
Collect only 25% of WikiEditor events

https://gerrit.wikimedia.org/r/204156

Change 204157 had a related patch set uploaded (by Jforrester):
Collect only 25% of WikiEditor events

https://gerrit.wikimedia.org/r/204157

Change 199132 merged by jenkins-bot:
Collect only 25% of WikiEditor events

https://gerrit.wikimedia.org/r/199132

Change 204157 merged by jenkins-bot:
Collect only 25% of WikiEditor events

https://gerrit.wikimedia.org/r/204157

Change 204156 merged by jenkins-bot:
Collect only 25% of WikiEditor events

https://gerrit.wikimedia.org/r/204156