Page MenuHomePhabricator

SPIKE Image Browsing: Determine how relevant images get included
Closed, ResolvedPublic3 Estimated Story PointsSpike

Description

Regardless of the UI design we eventually settle on, the Image Browsing prototype will need to be able to show "relevant images" to the user when it gets triggered.

"Relevant" images could include:

  • All images from the current article page
    • excluding small things like icons, flags, etc (what's the cut-off size?)
  • Images from equivalent pages on other-language wikis
  • Images from other related articles on the same wiki
  • Images from Commons (or Wikidata) that meet certain criteria (this may be out of scope for the MVP)

The MVP prototype will likely not exhaust the possibilities here.

Another high-level goal (aside from relevance) is that we want to prioritize community curation. If the community has already spent time and effort selecting images for a given article, we want to prioritize that content. If we do explore pulling images from places like Commons, what kinds of "signals" can we rely on for relevance?

Finally, while it is probably out of scope for the MVP, we may need to consider what an editorial workflow would look like for this feature – if we decided to scale out an image browsing prototype, we'd probably want to give editors a way to directly include/exclude content here (overriding whatever is automatically provided if desired).

Acceptance Criteria
    • RG engineers should discuss what kind of a procedure we want to rely on here at a high level before we build it in code.
  • Document the decisions made in terms of how to do this in an appropriate venue (wiki page, markdown ADR file in codebase, etc)

Actual implementation work should take place in a follow-up task; the goal here is to come up with a plan.

Event Timeline

egardner triaged this task as High priority.Jul 8 2025, 5:44 PM
ovasileva renamed this task from Image Browsing: Determine how relevant images get included to SPIKE Image Browsing: Determine how relevant images get included.Jul 22 2025, 4:30 PM
ovasileva added a project: Spike.
ovasileva set the point value for this task to 3.Jul 22 2025, 4:35 PM
ovasileva moved this task from Incoming/Inbox to Ready on the Reader Growth Team board.

I think there's benefit to using an "output HTML" model here -- on the pure JS experiment implementation we can do that all client-side extracting marked up images, and if we ever move it to a server-side service we can extract from rendered text using DOM processing on the PHP side.

Recommend double-checking MMV for how it detects which images to show in its slideshow view. There may be logic in there already to exclude navigational icons.

I'm gonna "be bold" and assign this to myself since I already gave myself a todo list above :) Will report back after diving into MMV this afternoon.

relevant logic in MMV lives in mmv.bootstrap.js:

  • selectors in processThumbs to find pingable images
  • detail extraction in processThumb, processParsoidThumb
  • file extension allow-list via isValidExtension
  • selectors in isAllowedThumb to check for exclusions

I'll see if I can adapt a minimal version of this we can plug in to our experiment easily.

Change #1171744 had a related patch set uploaded (by Bvibber; author: Bvibber):

[mediawiki/extensions/ReaderExperiments@master] WIP initial sketch of thumbnail extractor from live DOM

https://gerrit.wikimedia.org/r/1171744

HSwan-WMF lowered the priority of this task from High to Medium.Jul 23 2025, 4:42 PM
HSwan-WMF moved this task from Ready to Sprint 2 on the Reader Growth Team board.

Ok pretty happy with updated revision on https://gerrit.wikimedia.org/r/1171744

This handles lazy-load images as well and uses mw.util.parseImageUrl to extract the actual bucketed widths from the thumbnail URLs were using, as well as the file's name.

Should be possible to rebase on this fairly straightforwardly.

image.png (1,936×420 px, 157 KB)

Change #1171744 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] Thumbnail extractor from live DOM

https://gerrit.wikimedia.org/r/1171744

We have a basic approach working to include images from the current page in a manner similar to how MMV works – this code lives in the thumbExtractor.js file: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/ReaderExperiments/+/refs/heads/master/resources/ext.readerExperiments.imageBrowsing/thumbExtractor.js. The data structure generated by this thumbnail extraction process will likely serve as the foundation for a lot of other things that we do (and thus we have declared a formal interface for it using a Typescript definition).

I filed T400823 to capture the follow-up work of "how to get images from beyond the immediate article". So I think this can be signed off (maybe we should postpone any theoretical ADR until T400823 is also complete, so that we can describe the full story about how images are included).