Page MenuHomePhabricator

[Spike, 8hrs] Explain JS timer artifacts observed in EventLogging data
Closed, DeclinedPublicSpike

Description

Split off from T182314: Analyze results of enwiki and dewiki page previews a/b test:

When analyzing the data from the Popups schema about the duration of user interactions with the page previews feature (from the begin of a mouse hover over a link to the user action occurring), we encountered strange patterns such as this:

Frequency of actions per type over time (previews enabled, enwiki, Oct 23-Nov 12 2017) 1000-1200 ms.png (555×733 px, 54 KB)

(from T182314#3954765, see there for data source)

See the discussion at T182314 for what we found out about this so far, e.g. the detailed explanations about timers by @phuedx in T182314#3956099 and subsequent observations by @Jhernandez, @Gilles and others. While in that investigation we already figured out a lot of things about how the underlying timers work and reached a level of confidence that these anomalies should not materially affect the main takeaways for that A/B test, we have not yet found a satisfying explanation for them.
Since this kind of thing is likely to occur in similar web instrumentations in the future, there has been a sense that it warrants further investigation. In particular we would want to rule out the possibility that it is a "data smell" indicative of deeper problems with this method of measuring interaction times, rather than just a limitation of its accuracy.

Outcomes

A summary is added to the description of this task that satisfies Tilman enough to resolve this.

Event Timeline

Jdlrobson renamed this task from Explain JS timer artifacts observed in EventLogging data to [Spike, 8hrs] Explain JS timer artifacts observed in EventLogging data.Feb 25 2019, 9:05 PM
Jdlrobson added a project: Spike.
Jdlrobson updated the task description. (Show Details)
kzimmerman added subscribers: ovasileva, kzimmerman.

@ovasileva @phuedx I read through the explanations in the prior thread, and I would not expect any client-side event logs to be perfectly accurate down to the millisecond level. I recommend closing this as Declined; please do reopen & comment if you think this warrants further investigation.

Restricted Application changed the subtype of this task from "Task" to "Spike". · View Herald TranscriptJun 19 2019, 10:20 PM