Page MenuHomePhabricator

Video track annotations for Wikimedia
Open, LowestPublicFeature

Description

Feature summary (what you would like to be able to do and where):
Allow people to add timed annotations to videos. Similar to the old youtube annotations.

  • make it possible to have clickable objects within videos
  • link towards specific Wikipedia topics or other resources
  • display inline image/html/note/speech bubbles on top of the video

Use case(s) (list the steps that you performed to discover that problem, and describe the actual underlying problem which you want to solve. Do not describe only a solution):

  • We are working on some pedagogical videos, and it would be great if subtitles could have links/wikilinks to articles, so the experience is richer.

Benefits (why should this be implemented?):
This will help authors create richer experiences for users of the content.

Requirements

  • Settle on an annotation json? format for authoring annotations. Each annotation needs
    • a relative position within the frame (webvtt ?)
    • a time code (webvtt)
    • a json format for the content of the annotation
    • annotation types would be at least: 'clickable area' (with link), 'text element'
    • additional types could be 'image' (Commons only), 'svg drawing?', 'note', 'speech bubble'
  • Create a file/wikipage format (webVTT inspired) to store annotations
  • Deliver the annotations as a track, using a custom track type through the API for consumption by players
  • Implement a videojs annotation library to be loaded on demand to:
    • enable/disable annotations
    • render annotations at the appropriate time on top of the video

Event Timeline

TheDJ triaged this task as Lowest priority.Feb 15 2022, 9:18 PM

Inspiration can maybe be drawn from another 'simple' annotation format, which is chapters/cues. https://player.support.brightcove.com/general/implementing-cue-points-programmatically.html

Thenks @TheDJ for creating this. Actually, I wasn't thinking on adding annotations (but this may work also), but on adding links to subtitles. Would it be impossible to add also the <a> html tag as we have<b> and <i>?

Would it be impossible to add also the <a> html tag as we have<b> and <i>?

No, because that is not a feature of HTML5 compliant subtitles which browsers can natively render. https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API#cue_payload_text_tags

HTML5 has explicitly excluded such functionality from subtitles, because you can roll your own annotations if you really want to.

So, as I understand, the subtitles TimedText tracks are rendered by the browser, and not by Kaltura. Mmmm... this could be something that would make videos on wiki special. Let's think.

Kaltura

VideoJS, Kaltura is getting removed completely.

We have captions within videowiki that have wikilinks that work

https://videowiki.wmcloud.org/en/videowiki/Video:HIV-AIDS?wikiSource=https://mdwiki.org

However when we save these videos to Commons the links are scrubbed

https://commons.wikimedia.org/wiki/TimedText:En.Video-Kivu_Ebola_epidemic.webm.en.srt

We had discussed imbedding the videowiki player within the mediawiki itself but developing that as an option is a ways down the road.

Yes, this is why I was asking, because we have been looking closely to the videowiki software (a Basque TTS is pending from the Indian side), and links are added there, but those are not visible at all on Wiki. And this is a feature that would make our platform unique.

So Wiki Project Med Foundation has taken over VideoWiki development. Yes getting Basque working is on our to do list. We are applying for some funding to pay for this work along with a bunch of other stuff so fingers crossed... https://meta.wikimedia.org/wiki/Grants:Project/Rapid/WPM:VideoWiki

I think this should be, somehow, inside T100106