As a data analyst, I want to understand link interaction without and with HoverCards so that I can help inform decisions regarding tuning and rollout of HoverCards.
Acceptance criteria:
- At start of task, schemas reviewed between @Tbayer, @dr0ptp4kt, and implementing engineer. Schedule a meeting at a mutually available time. (@dr0ptp4kt met with @Tbayer & @jhobs; @dr0ptp4kt also met with @bmansurov)
We're updating https://meta.wikimedia.org/wiki/Schema:Popups to capture all relevant EventLogging data for this task.
Some specs:
- Sampling in for event logging should be done on session-wide funnel (mw.user.sessionid()).
[?x] Specify sampling rate on a per-wiki basis. Use the default sampling rate otherwise.
- Only sample in to log when UA is sendBeacon capable. Do not log when UA is not sendBeacon capable.
- State machine is demarcated for determining session-, page-wide funnel and per-link-dwell subfunnels. A few gotchas: beware double clicks and clearly determine behavior for back and forward button navigation with respect to creation of page tokens.
[ x] Only fire link related events for links which would be eligible for a preview
Add these items to the event logging funnel:
- Page loaded event w/ HoverCards disposition
- Initial dwell 250ms or longer, but abandoned (and basic reason attached). (Non-abandoned is captured already.)
When logging events, in addition to existing event logging, do these things:
- Record user edit count bucket and whether logged in / anonymous user (like https://meta.wikimedia.org/wiki/Schema:QuickSurveysResponses)
- Hover bucket count (this value should also always be incremented even if the user isn't EL sampled; model using the approach from language switching event logging - 0, 1-4, 5-20, 21+) - increment only upon preview actually shown (we'll also need to update https://wikimediafoundation.org/wiki/Cookie_statement#3._What_types_of_cookies_does_Wikimedia_use.3F and https://wikimediafoundation.org/wiki/Privacy_policy/FAQ#Can_you_give_me_some_examples_of_types_of_cookies_and_how_you_use_local_storage.3F)
- Session funnel token (mw.user.sessionid()) and mechanism to ensure funnel analysis across multiple pages (but not past browser restarts).
- Page loaded token
- Link interaction token
- Destination namespaceid or equivalent
- Source namespaceid or equivalent
- Source Pageid
- Amount of time between hover and the time it actually rendered (determined exclusively client side)
- If errored out or timed out instead of rendering
- What was clicked: destination|settings cog
- HoverCards disabled by user events. The instrumentation ought to be setup such that it's possible to reconstruct how dwell delay / and time between initiation of dwell and actual render play into hover length and disablement of the feature.
- Timing of clickthroughs (or error; granted sometimes errors means EL won't even work, but it's conceivable there's a service outage for the specific endpoint)
- Versioning (which Hovercards UX?) with flexibility for A/B/C tests
- API endpoint: mwapi or restbase (signoff note: presently only mwapi is available, and is thus hardcoded; support for RestBase would need to add the appropriate support)
- Evaluate the difficulty of excluding mobile touch UAs (because they don't have hover capabilities) from sending events, or alternatively estimate the error introduced by including them (signoff note: mobile devices in particular turned out to be relatively small portion of desktop domain traffic)
See also T88166 (task from 2015 with discussions that led to the present version of the schema)