Page MenuHomePhabricator

Make hard limit of 5000 query results into a config variable
Open, NormalPublic

Description

  • https://en.wikipedia.org/w/index.php?title=Special:Contributions/Beyond_My_Ken&offset=&limit=5000&target=Beyond+My+Ken
  • https://en.wikipedia.org/w/index.php?title=Barack_Obama&offset=&limit=5000&action=history
  • https://en.wikipedia.org/w/index.php?title=Special:WantedPages&limit=5000&offset=0

In each case 5000 is the maximum accepted value of limit, and any higher value returns 5000. This includes the user preference setting Number of edits to show in recent changes, page histories, and in logs, by default (actually on enwiki the latter is now capped at 1000, hmm. On my home wiki with MediaWiki 1.27.4, I can increase it to 9223372036854780000, although such input is still interpreted as 5000).

IIRC this situation has existed since at least 2004, with no discussion of (broad-based) improvements to hardware and network administration tools in the meantime.

The use case for longer lists would be something like an internal corporate knowledge management system. Page/revision tables might become large, but few queries would run at any given time. End users would furthermore be accustomed to waiting, say, 30-300 seconds for a page to be served (from their experience with bloatware desktop apps and commercial cloud portals). Every so often, a huge set of pages would need review: legal discovery, regulatory audit, technical debt remediation.

Obviously such a limit would never be raised from its default on WMF projects, where volume always seems to increase to meet capacity :) and be accompanied by a sulphur-and-brimstone comment in DefaultSettings.php about the risk of DoS attacks and such.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 19 2018, 10:26 PM
Xeriphas1994 renamed this task from ard limit of 5000 query results into a config variable to Make hard limit of 5000 query results into a config variable.Feb 19 2018, 10:38 PM
Xeriphas1994 updated the task description. (Show Details)

How to get those "query results"? Steps to reproduce welcome - see https://mediawiki.org/wiki/How_to_report_a_bug . Thanks!

Xeriphas1994 updated the task description. (Show Details)Feb 20 2018, 8:30 AM

Sorry, I thought that was obvious from the tags. You get a cupcake for not simply closing it as malformed.

Sorry, I thought that was obvious from the tags. You get a cupcake for not simply closing it as malformed.

@Xeriphas1994: If everybody managed to always set the right tags things indeed were "obvious". But as that's not always the case we ask you for clear steps at the top of the task creation form, to know that this is not an API request etc. Your second sentence is uncalled for and I ask you to avoid this in the future. Thanks.

Uh... that was meant sincerely. I have had submissions declined instantly in other organizations for skipping a single sub-bullet in the instructions. Point taken however; it won't happen again.

@Xeriphas1994: I am sorry. I interpreted it obviously in the wrong way.

dmattern claimed this task.Feb 24 2018, 3:13 AM
dmattern triaged this task as Normal priority.

Change 414019 had a related patch set uploaded (by Dmattern; owner: Dmattern):
[mediawiki/core@master] Add configuration option for max query limit

https://gerrit.wikimedia.org/r/414019

MaxSem added a subscriber: MaxSem.Feb 24 2018, 3:19 AM

Note that while having this setting for MW in general might be considered, it's highly unlikely that we will ever raise this value on WMF wikis.

My understanding is that the interest of the original reporter is not about wikis hosted by the Wikimedia Foundation (WMF) but about their MediaWiki installation(s).