Page MenuHomePhabricator

Wrap thumbnail in a picture element and add a hook to register additional source elements
Open, Needs TriagePublicFeature

Description

Feature summary :

  • Add an optional MediaWiki config to allow thumbnail <img> element to be wrapped in a <picture> element
  • Add a hook that allows extensions to register additional <source> element to the <picture> element

Original:

<figure typeof="mw:File/Thumb">
    <a href="/File:Thumbnail.jpg" class="mw-file-description">
        <img alt="" src="https://example.wiki/thumb/e/e2/Thumbnail.jpg/300px-Thumbnail.jpg" decoding="async"
            loading="lazy" width="300" height="169"
            srcset="https://example.wiki/thumb/e/e2/Thumbnail.jpg/450px-Thumbnail.jpg 1.5x, https://example.wiki/thumb/e/e2/Thumbnail.jpg/600px-Thumbnail.jpg 2x"
            data-file-width="1024" data-file-height="576">
    </a>
    <figcaption>Thumbnail caption</figcaption>
</figure>

Proposed:

<figure typeof="mw:File/Thumb">
    <a href="/File:Thumbnail.jpg" class="mw-file-description">
        <picture>
            <!-- Added by hook -->
            <source
                srcset="https://example.wiki/thumb/avif/e/e2/Thumbnail.jpg/300px-Thumbnail.avif, https://example.wiki/thumb/avif/e/e2/Thumbnail.jpg/450px-Thumbnail.avif 1.5x, https://example.wiki/thumb/avif/e/e2/Thumbnail.jpg/600px-Thumbnail.avif 2x"
                type="image/avif">
            <source
                srcset="https://example.wiki/thumb/webp/e/e2/Thumbnail.jpg/300px-Thumbnail.webp, https://example.wiki/thumb/webp/e/e2/Thumbnail.jpg/450px-Thumbnail.webp 1.5x, https://example.wiki/thumb/webp/e/e2/Thumbnail.jpg/600px-Thumbnail.webp 2x"
                type="image/webp">

            <!-- Original img element -->
            <img alt="" src="https://example.wiki/thumb/e/e2/Thumbnail.jpg/300px-Thumbnail.jpg" decoding="async"
                loading="lazy" width="300" height="169"
                srcset="https://example.wiki/thumb/e/e2/Thumbnail.jpg/450px-Thumbnail.jpg 1.5x, https://example.wiki/thumb/e/e2/Thumbnail.jpg/600px-Thumbnail.jpg 2x"
                data-file-width="1024" data-file-height="576">
        </picture>
    </a>
    <figcaption>Thumbnail caption</figcaption>
</figure>

Use case(s) :

  • Allow wikis and extension to serve different images based on the client's browser and device with only HTML, without needing additional CDN or Javascript
  • Allow browser native fallback support when using new image formats such as AVIF and WebP

Benefits:

  • Allow extensions to use the hook to add additional sources
    • Adding more optimized thumbnail formats such as Extension:WebP (T282453, T257652)
    • Adding responsive images based on client's viewport
    • Adding different URL for the thumbnail (e.g. using an external CDN for image processing)
  • Use of semantic HTML5 element (T25932)
  • Does not affect WMF wikis if it is disabled by default through a config flag

A proof of concept is available through Extension:PictureHtmlSupport. You can see it in action with Extension:WebP on the Star Citizen Wiki.

Event Timeline

An additional benefit, in contrast to what WMF does to serve WebP instead of PNG/JPG on the CDN layer, is that files will be served with the correct file extension. Currently on WMF, some thumbnails are served as WebP with .png extension, confusing users when downloading those thumbnails to their devices for later viewing/editing or uploading them somewhere else.

At the very least, this will require changing the spec of https://www.mediawiki.org/wiki/Specs/HTML/2.8.0#Images to allow <picture>.

It might also impact some of the styling, and we'd have to consider the impact on VisualEditor, and also on MobileFrontend [and its lazy loading plugin] I guess (with and without the flag). Preferably, I'd rather not have a permanent flag (just one to gate introduction of <picture>), as it further complicates all the different scenarios that skinning has to support, which is already problematic in general.

It shouldn't affect any styling since the <picture> element do not contain any styles. We tested with VE on the wiki that is linked above, no regression has been found with it. As for MobileFrontend, it would require some additional testing.

As for the config flag, it might not be permanent but it should be in the initial patch. It is a core change in the DOM structure so it might break in some edge cases (and there's a lot of them in the MW ecosystem). Having a config flag would allow us to catch those issues, while allowing users to fallback to the safer and proven behavior.

I would suggest leaving discussion on whether it should be a default behavior to after we implement it with a config flag. As that would involve a lot of stakeholders and it would need to be tested on WMF production, which takes a substantial amount of time.