Page MenuHomePhabricator

Record and report metrics for audio and video playback
Open, LowPublic

Description

Currently we have limited information about media playback; we have some stats but I don't know how reliable they are in terms of showing which files/resolutions/formats are used, whether playback completes, whether it takes seeks into account etc.

It would also be nice to report some playback quality metrics when available (dropped frame count etc). ogv.js and Firefox provide some metrics in theory.

Added by TheDJ based on past conversations on this topic:
At an industry level, you'd probably want at least the metrics Play Rate and Video Engagement Rate.

Play rate:
percentage of page visitors who were presented the video, clicked play and started watching

Video engagement rate:
The percentage of a video that a viewer watches your content. Calculate engagement rate by dividing the total time played by the
total plays multiplied by the media's length.

To go with that, you want some quantifying information like: You likely also will want to know: audio or video, title, wiki etc.

To generate information like video engagement rate, you likely have to be listening to events of the video element / by making a plugin to the videojs player. You could implement a plugin similar to this for instance. https://github.com/apivideo/api.video-videojs-analytics/blob/main/index.ts

For privacy reasons as this is reading info, which generally isn't public, the information probably has to be sampled and be send with jitter and noise in the process (Definitely consult the Wikimedia privacy and analytics team about this and what they would be comfortable with).

This information will then have to be collected and submitted to an api (which one? EventLogging API? )
Possibly it requires a separate intake stream ?

This would give you analytics for our own webviews... Unfortunately we have MORE views. The apps would need separate support as they directly make use of the videos. So for a complete picture you would have to add separate metric plugins to each of those platforms as well.

Event Timeline

brooke raised the priority of this task from to Needs Triage.
brooke updated the task description. (Show Details)
brooke subscribed.

Should also track usage of inline vs embedded iframe mode

See also T88775

BTW I guess that by "low-level stats", you mean the mediacounts table?

TheDJ added subscribers: Krenair, TheDJ.

@Krenair, you've done eventlogging for wikieditor at some point right ? Would you care to explain this to me, after I get back from my holidays, so that I could set it up for TMH ?

@Krenair, you've done eventlogging for wikieditor at some point right ? Would you care to explain this to me, after I get back from my holidays, so that I could set it up for TMH ?

Yeah. I can (try to) answer any questions you have.

What do we want to measure ?

1: How often are players that are inserted into pages actually interacted with.
2: How often do people play/pause and actually finish watching a video ?
3: How often do people encounter issues with playback (tried to play but did not succeed ?)
4: How often do people use feature: X

  • Fullscreen
  • Share
  • Manually switch play back engine
  • Enable or disable subtitles
  • Manually switch resolution

5: What is the performance of the player setup ?

Properties

  • browser (version)
  • OS (version)
  • registered or IP
  • embedded vs file page (category pages?) vs. iframe
  • mobile vs non-mobile
  • playback resolution used
  • video tech used (html5, ogvjs, vlc)
  • video player used (embed player, videojs)

Events

  • Init player
  • Player ready
  • Playback
    • play
    • pause
    • ended
  • UI interactions
    • menu button
    • full screen button
    • subtitle button
  • Destroy player

@brion let us know whenever you want to talk about doing this!

@fdans & @DarTar yes let's schedule a brief talk to get started? I'm on pacific time, mostly flexible time other than Wednesday afternoons. Send me a calendar invite!

Restricted Application edited projects, added Analytics; removed Analytics-Radar. · View Herald TranscriptJun 10 2020, 6:33 AM
Restricted Application edited projects, added Analytics; removed Analytics-Radar. · View Herald TranscriptJun 10 2020, 6:36 AM