Page MenuHomePhabricator

Improve handling of image limits by fixing the UX on pages where these limits are hit
Open, Needs TriagePublic

Description

First reported on mw:Parsoid/Feedback.

Compare Parsoid vs legacy.

As part of the task, let us revisit the following:

  • should we change the limits to something a bit higher?
  • can we ensure that the top N images are rendered vs. some N images on the page?
  • Per @stjn, is there a better UX for flagging these limits to the user (instead of incorrectly rendering them as red links)?

Plan of action

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
Isabelle Hurbain-Palatinmediawiki/vendormaster+246 -3 K
OSlegermediawiki/services/parsoidmaster+63 -51
OSlegermediawiki/coremaster+16 -9
OSlegermediawiki/coremaster+11 -3
Subramanya Sastrymediawiki/vendormaster+3 K -3 K
OSlegermediawiki/services/parsoidmaster+1 -1
OSlegermediawiki/services/parsoidmaster+68 -5
OSlegeroperations/mediawiki-configmaster+1 -1
OSlegermediawiki/coremaster+43 -0
Arlolramediawiki/vendormaster+78 -164
Arlolramediawiki/vendorwmf/1.47.0-wmf.11+80 -53
OSlegermediawiki/vendormaster+80 -53
OSlegermediawiki/services/parsoidmaster+15 -3
OSlegermediawiki/coremaster+6 -0
OSlegermediawiki/coremaster+6 -0
Arlolramediawiki/services/parsoidmaster+1 -1
Arlolraoperations/mediawiki-configmaster+3 -0
Show related patches Customize query in gerrit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

@stjn, I am responding to the user-notice tag, how should I word this for Tech/News?

Something like:

On Parsoid wikis, there is a new limit to the number of images displayed on a page. Initially it was 1000, now it is 1250. The reason is that Parsoid fails to render pages with too much images [ask the team for clarification?]. You can provide feedback in [[phab:T430854]].

While we work on adding a tracking category, a quarry can be useful, select il_from, count(*) as counts from imagelinks group by il_from having counts > 1250 order by counts desc; yields 236 rows on enwiki

https://quarry.wmcloud.org/query/107136

Thanks for the SQL query. I have forked it for dewiki and extended it so that the page title gets displayed directly:
https://quarry.wmcloud.org/query/107150

So this means we currently have 134 pages on dewiki where the limit gets hit.

Looks like the page in enwiki's main namespace with the highest number of unique images is https://en.wikipedia.org/wiki/Armorial_of_Russia with 2888. (https://quarry.wmcloud.org/query/107149). Pages in the user and project namespaces are of much lesser concern.

This does kind of beg the question, though - is the limit in Parsoid the number of unique images, or the total including duplicates? Database queries can't help with the latter.

This does kind of beg the question, though - is the limit in Parsoid the number of unique images, or the total including duplicates? Database queries can't help with the latter.

Unique images (well, keyed to the title, dimension, format, and other media options that affect the thumbnail rendering)

See the test case in https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/1275427/14/tests/parser/media.txt where the repeated file is only counted once [[File:Foobar.jpg| one ]] ... [[File:Foobar.jpg| four ]]

Something like:

On Parsoid wikis, there is a new limit to the number of images displayed on a page. Initially it was 1000, now it is 1250. The reason is that Parsoid fails to render pages with too much images [ask the team for clarification?]. You can provide feedback in [[phab:T430854]].

@STei-WMF: "On Parsoid-enabled wikis, Parsoid renders a maximum of 1250 images on a page. See [[phab:T430854]] for more details and to provide feedback." I think this may be simple enough and interested folks can come here for more details and discussion.

Is there a reason why only the first 1,186 files linked from https://en.wikipedia.org/w/index.php?title=Armorial_of_Russia are actually displayed (only those up to File:Coat of Arms of Susuman (Magadan oblast).png)?

1,250 images are displayed which is the current limit (it's not the first 1,250 since 3 icons in the lead are not displayed). Maybe you counted the number of "File:" in the source but some galleries like https://en.wikipedia.org/wiki/Armorial_of_Russia#Oblasts use syntax without "File:".

Maybe you counted the number of "File:" in the source but some galleries like https://en.wikipedia.org/wiki/Armorial_of_Russia#Oblasts use syntax without "File:".

That's what I was doing, yes. Thanks for the clarification!

Thanks!

Something like:

On Parsoid wikis, there is a new limit to the number of images displayed on a page. Initially it was 1000, now it is 1250. The reason is that Parsoid fails to render pages with too much images [ask the team for clarification?]. You can provide feedback in [[phab:T430854]].

@STei-WMF: "On Parsoid-enabled wikis, Parsoid renders a maximum of 1250 images on a page. See [[phab:T430854]] for more details and to provide feedback." I think this may be simple enough and interested folks can come here for more details and discussion.

@STei-WMF: Please remove unneeded full quotes, as they make tickets harder to read. Thanks!

Change #1310107 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/core@master] Add new tracking category "media-limit-reached"

https://gerrit.wikimedia.org/r/1310107

Change #1310109 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/services/parsoid@master] Add a tracking category for pages where Parsoid reaches the configured media limit.

https://gerrit.wikimedia.org/r/1310109

Change #1310126 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/core@master] Add new tracking category "media-limit-reached"

https://gerrit.wikimedia.org/r/1310126

Change #1310126 abandoned by OSleger:

[mediawiki/core@master] Add new tracking category "media-limit-reached"

Reason:

Duplicate

https://gerrit.wikimedia.org/r/1310126

Change #1310107 merged by jenkins-bot:

[mediawiki/core@master] Add new tracking category "media-limit-reached"

https://gerrit.wikimedia.org/r/1310107

Change #1310144 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a15

https://gerrit.wikimedia.org/r/1310144

@ssastry @stjn

Last week I didn't publish this because I had some questions last minute. The entry feels a bit contradictory:

On Parsoid wikis, there is a new limit to the number of images displayed on a page. Initially it was 1000, now it is 1250. The reason is that Parsoid fails to render pages with too much images [ask the team for clarification?]. You can provide feedback in [[phab:T430854]].

If the change is that it is now possible for more images to be displayed on parsoid wikis –– up to 1250 (previously less – 1000), but then the reason for the change is that pages fail when there are too many images, that seems conflicting. 1250 is more than 1000 right?

Can you help with some clarification please?

This should’ve just been published in ssastry’s version, IMO. No, the change is that Parsoid wikis can no longer display more than 1250 images on one page. This change was just unannounced by the WMF even though it happened already. It is regrettable that it ended up unannounced in Tech News this week, but I guess this allows to mention the new tracking category as well.

@stjn this is how I have finalised it. I have added it to the tech news edition draft, if you have any suggestions for edits, please do it directly on the tech news draft:

To improve performance and the user experience on Parsoid-enabled wikis, Parsoid now renders a maximum of 1,250 images per page. A new tracking category, "media-limit-reached", has also been added to identify pages where this limit is reached, making it easier to find content whose media output may have been restricted during rendering. See [[phab:T430854]] for more information and to provide feedback.

@STei-WMF: I had proposed this previously "On Parsoid-enabled wikis, Parsoid renders a maximum of 1250 images on a page. See [[phab:T430854]] for more details and to provide feedback." The tracking category isn't available yet and needs a bit more work, so let us hold on on that announcement. It is possible it will be available next week -- we will know by end of week, in which case we can tweak the announcement then. I'll go in and edit your tech news draft, but this is FYI.

(UPDATE: I decided to keep the tracking category announcement in your draft, but, tweaked it to say it will be soon made available.)

Change #1310109 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Add a tracking category for pages where Parsoid reaches the configured media limit.

https://gerrit.wikimedia.org/r/1310109

PSA: enwiki now has {{Template:Image rendering issues from Parsoid limit}} to help label articles over the limit, courtesy of FlammablePizza

Change #1310144 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a15

https://gerrit.wikimedia.org/r/1310144

Change #1312554 had a related patch set uploaded (by Arlolra; author: OSleger):

[mediawiki/vendor@wmf/1.47.0-wmf.11] Bump wikimedia/parsoid to 0.24.0-a15

https://gerrit.wikimedia.org/r/1312554

Change #1312554 merged by jenkins-bot:

[mediawiki/vendor@wmf/1.47.0-wmf.11] Bump wikimedia/parsoid to 0.24.0-a15

https://gerrit.wikimedia.org/r/1312554

Mentioned in SAL (#wikimedia-operations) [2026-07-20T20:17:16Z] <arlolra@deploy2003> Started scap sync-world: Backport for [[gerrit:1312547|Enable PRV on enwiki talk namespace, template namespace]], [[gerrit:1312554|Bump wikimedia/parsoid to 0.24.0-a15 (T221028 T226289 T353697 T430854 T431771)]], [[gerrit:1312556|Bump wikimedia/parsoid to 0.24.0-a15 (T431771)]]

Mentioned in SAL (#wikimedia-operations) [2026-07-20T20:20:59Z] <arlolra@deploy2003> arlolra, cscott: Backport for [[gerrit:1312547|Enable PRV on enwiki talk namespace, template namespace]], [[gerrit:1312554|Bump wikimedia/parsoid to 0.24.0-a15 (T221028 T226289 T353697 T430854 T431771)]], [[gerrit:1312556|Bump wikimedia/parsoid to 0.24.0-a15 (T431771)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-07-20T20:32:24Z] <arlolra@deploy2003> Finished scap sync-world: Backport for [[gerrit:1312547|Enable PRV on enwiki talk namespace, template namespace]], [[gerrit:1312554|Bump wikimedia/parsoid to 0.24.0-a15 (T221028 T226289 T353697 T430854 T431771)]], [[gerrit:1312556|Bump wikimedia/parsoid to 0.24.0-a15 (T431771)]] (duration: 15m 07s)

Change #1314959 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a16

https://gerrit.wikimedia.org/r/1314959

Change #1314959 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a16

https://gerrit.wikimedia.org/r/1314959

Change #1315966 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/services/parsoid@master] AddMediaInfo: distinguish known files after media limit

https://gerrit.wikimedia.org/r/1315966

Change #1315947 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/core@master] [Parser.php] Add media-limit-reached tracking category

https://gerrit.wikimedia.org/r/1315947

My article https://pl.wikipedia.org/wiki/Lista_dzie%C5%82_malarskich_Stanis%C5%82awa_Ignacego_Witkiewicza which catalogues ~2680 artworks by a Polish artists just got hit by "apierror-imagelimitexceeded". Article was up and doing fine for last 4 years. Is there a way to fix it without breaking it down into "part 1" and "part 2", which breaks ability to use table column sorting for different groupings. Also very little info is displayed on what is the issue: no error category and clicking red link links to image upload wizard, which is not the issue.

How do we find which other articles are affected?

@Jarekt This query lists only articles on the English Wikipedia that exceed the limit: https://quarry.wmcloud.org/query/107835. You can fork it and change "enwiki_p" in the box at the top to "plwiki_p".

@Ahecht, thanks. Apparently my article is the only one on Polish Wikipedia with 2204 images.

Change #1319500 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/services/parsoid@master] SiteConfig: Rise parsoid media limit to 5000

https://gerrit.wikimedia.org/r/1319500

Change #1319521 had a related patch set uploaded (by OSleger; author: OSleger):

[operations/mediawiki-config@master] Bump Parsoid image limit to 5000

https://gerrit.wikimedia.org/r/1319521

Change #1319530 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/core@master] DataAccess: Use findFile for single-file lookups

https://gerrit.wikimedia.org/r/1319530

Change #1319500 merged by jenkins-bot:

[mediawiki/services/parsoid@master] SiteConfig: Increase parsoid media limit to 5000

https://gerrit.wikimedia.org/r/1319500

Change #1315966 merged by jenkins-bot:

[mediawiki/services/parsoid@master] AddMediaInfo: After media limit is hit, don't treat rest of images as missing

https://gerrit.wikimedia.org/r/1315966

Change #1319521 merged by jenkins-bot:

[operations/mediawiki-config@master] Increase Parsoid image limit to 5000

https://gerrit.wikimedia.org/r/1319521

Mentioned in SAL (#wikimedia-operations) [2026-07-30T20:03:55Z] <arlolra@deploy1003> Started scap sync-world: Backport for [[gerrit:1319521|Increase Parsoid image limit to 5000 (T430854)]]

Mentioned in SAL (#wikimedia-operations) [2026-07-30T20:05:44Z] <arlolra@deploy1003> osleger, arlolra: Backport for [[gerrit:1319521|Increase Parsoid image limit to 5000 (T430854)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-07-30T20:13:15Z] <arlolra@deploy1003> Finished scap sync-world: Backport for [[gerrit:1319521|Increase Parsoid image limit to 5000 (T430854)]] (duration: 09m 20s)

Change #1320184 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/core@master] DataAccess: Pass File to BadFileLookup::isBadFile

https://gerrit.wikimedia.org/r/1320184

Change #1320187 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a18

https://gerrit.wikimedia.org/r/1320187

Emoji/Table page on Wikimedia Commons is #1 with 10k files

https://quarry.wmcloud.org/query/107882

At the moment page is displacing just fine, however if at some point it no longer does I can work with commons community on splitting such pages, especially is all show u in some automatic "problem" category

Change #1320187 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a18

https://gerrit.wikimedia.org/r/1320187

Change #1319530 merged by jenkins-bot:

[mediawiki/core@master] DataAccess: Use findFile for single-file lookups

https://gerrit.wikimedia.org/r/1319530

Change #1320184 merged by jenkins-bot:

[mediawiki/core@master] DataAccess: Pass File to BadFileLookup::isBadFile

https://gerrit.wikimedia.org/r/1320184

Change #1327153 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/services/parsoid@master] Update media limit counting and rendering order

https://gerrit.wikimedia.org/r/1327153

Change #1327153 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/services/parsoid@master] Update media limit counting and rendering order

https://gerrit.wikimedia.org/r/1327153

Follow up on the counting semantics discussed earlier: we previously described the media limit as counting unique images, keyed on the title plus the options that affect thumbnail rendering (dimensions, format, etc.). That matched the test case in media.txt, where repeated [[File:Foobar.jpg| one ]] … [[File:Foobar.jpg| four ]] counted once.

This change moves the counting from AddMediaInfo to WikilinkHandler, so the limit now applies in rendering (textual) order, and the semantics change:

  • Every rendered media link counts toward the limit, including repeats of the same file. The Foobar.jpg example above now counts four times.
  • An image with a thumbnail override counts once, not once for the override and once for the file.
  • Images in galleries or other extensions no longer get priority over images in regular article content. Everything is counted in the order it appears.
  • Category:Pages_with_broken_file_links is still added for missing files after the limit has been reached, so the tracking category stays accurate on pages that exceed it.

On the rationale: the limit was originally designed around the number of file info requests, since we assumed that was the bottleneck. Later analysis showed the real problem was that galleries weren't batching their file info requests. With that understood, the unique-request count matters less as a limiting factor, and counting in rendering order is simpler and easier to predict.

Change #1327153 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Update media limit counting and rendering order

https://gerrit.wikimedia.org/r/1327153

Change #1343538 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a24

https://gerrit.wikimedia.org/r/1343538

Change #1343538 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a24

https://gerrit.wikimedia.org/r/1343538