Page MenuHomePhabricator

Make hard limit of 5000 query results into a config variable
Open, MediumPublic

Description

  • https://en.wikipedia.org/w/index.php?title=Special:Contributions/Beyond_My_Ken&offset=&limit=5000&target=Beyond+My+Ken
  • https://en.wikipedia.org/w/index.php?title=Barack_Obama&offset=&limit=5000&action=history
  • https://en.wikipedia.org/w/index.php?title=Special:WantedPages&limit=5000&offset=0

In each case 5000 is the maximum accepted value of limit, and any higher value returns 5000. This includes the user preference setting Number of edits to show in recent changes, page histories, and in logs, by default (actually on enwiki the latter is now capped at 1000, hmm. On my home wiki with MediaWiki 1.27.4, I can increase it to 9223372036854780000, although such input is still interpreted as 5000).

IIRC this situation has existed since at least 2004, with no discussion of (broad-based) improvements to hardware and network administration tools in the meantime.

The use case for longer lists would be something like an internal corporate knowledge management system. Page/revision tables might become large, but few queries would run at any given time. End users would furthermore be accustomed to waiting, say, 30-300 seconds for a page to be served (from their experience with bloatware desktop apps and commercial cloud portals). Every so often, a huge set of pages would need review: legal discovery, regulatory audit, technical debt remediation.

Obviously such a limit would never be raised from its default on WMF projects, where volume always seems to increase to meet capacity :) and be accompanied by a sulphur-and-brimstone comment in DefaultSettings.php about the risk of DoS attacks and such.

Event Timeline

Xeriphas1994 renamed this task from ard limit of 5000 query results into a config variable to Make hard limit of 5000 query results into a config variable.Feb 19 2018, 10:38 PM
Xeriphas1994 updated the task description. (Show Details)

How to get those "query results"? Steps to reproduce welcome - see https://mediawiki.org/wiki/How_to_report_a_bug . Thanks!

Sorry, I thought that was obvious from the tags. You get a cupcake for not simply closing it as malformed.

Sorry, I thought that was obvious from the tags. You get a cupcake for not simply closing it as malformed.

@Xeriphas1994: If everybody managed to always set the right tags things indeed were "obvious". But as that's not always the case we ask you for clear steps at the top of the task creation form, to know that this is not an API request etc. Your second sentence is uncalled for and I ask you to avoid this in the future. Thanks.

Uh... that was meant sincerely. I have had submissions declined instantly in other organizations for skipping a single sub-bullet in the instructions. Point taken however; it won't happen again.

@Xeriphas1994: I am sorry. I interpreted it obviously in the wrong way.

dmattern triaged this task as Medium priority.

Change 414019 had a related patch set uploaded (by Dmattern; owner: Dmattern):
[mediawiki/core@master] Add configuration option for max query limit

https://gerrit.wikimedia.org/r/414019

Note that while having this setting for MW in general might be considered, it's highly unlikely that we will ever raise this value on WMF wikis.

My understanding is that the interest of the original reporter is not about wikis hosted by the Wikimedia Foundation (WMF) but about their MediaWiki installation(s).

Is the 500 limit on special:contribs even meant to be permenent or just temporary stop gap?

Aklapper added a subscriber: dmattern.

Removing task assignee due to inactivity, as this open task has been assigned to the same person for more than two years (see the emails sent to the task assignee on Oct27 and Nov23). Please assign this task to yourself again if you still realistically [plan to] work on this task - it would be welcome.
(See https://www.mediawiki.org/wiki/Bug_management/Assignee_cleanup for tips how to best manage your individual work in Phabricator.)