API and UI limits are inconsistent
Open, LowPublic

Description

Author: matthew.britton

Description:
Many things (for example user contributions, page history, user list) have a limit of 5000 items when accessed through the UI, by anyone, but only 500 when accessed through the API by a 'normal user' (by default, non-bot, non-sysop). This is inconsistent, and if anything is backwards since UI requests entail more work server-side. Moving (semi-)automated queries from UI to API, something that is presumably desired, can thus result in up to 10 times as many requests needing to be made, surely more than negating all benefit to both ends brought about by using the API in the first place.

If 5000 is too much, should the UI limit be reduced, if not, should the API limit be raised? If the worry is that the API lends itself more easily to rapid queries, perhaps some sort of rate limit should be introduced for expensive queries?


Version: unspecified
Severity: enhancement

Details

Reference
bz16781
bzimport raised the priority of this task from to Low.
bzimport set Reference to bz16781.
bzimport added a subscriber: Unknown Object (MLST).
bzimport created this task.Dec 24 2008, 1:19 PM

(In reply to comment #0)

If 5000 is too much, should the UI limit be reduced, if not, should the API
limit be raised? If the worry is that the API lends itself more easily to rapid
queries, perhaps some sort of rate limit should be introduced for expensive
queries?

I can't really comment on what's too high and what isn't, someone with more knowledge in the performance field should do that. Something that should be noted is that the 500/5000 limit in the API applies to each query separately, and since there are 34 query types in core (extensions like FlaggedRevs and GlobalBlocking provide even more), even a non-sysop user could get up to 34*500=17,000 results in one request.

matthew.britton wrote:

(In reply to comment #1)

I can't really comment on what's too high and what isn't, someone with more
knowledge in the performance field should do that. Something that should be
noted is that the 500/5000 limit in the API applies to each query separately,
and since there are 34 query types in core (extensions like FlaggedRevs and
GlobalBlocking provide even more), even a non-sysop user could get up to
34*500=17,000 results in one request.

True, I hadn't thought of that. I suppose that's a valid argument for keeping the limit down.

In practise, of course, there aren't any situations where such a request would provide results of any usefulness :)

(In reply to comment #2)

In practise, of course, there aren't any situations where such a request would
provide results of any usefulness :)

While combining all 34 queries is obviously over the top and only really useful for testing or DoS purposes, combinations of 2 or 3 queries are quite common, and even combinations of 10 queries can be useful (all prop= modules combined, for instance).

matthew.britton wrote:

(In reply to comment #3)

While combining all 34 queries is obviously over the top and only really useful
for testing or DoS purposes, combinations of 2 or 3 queries are quite common,
and even combinations of 10 queries can be useful (all prop= modules combined,
for instance).

Yes, I've used a few such queries myself. Perhaps a 'maximum number of queries in one request' limit if DoS is a potential problem?

(In reply to comment #4)

Yes, I've used a few such queries myself. Perhaps a 'maximum number of queries
in one request' limit if DoS is a potential problem?

Maybe, although I'm kind of leery of breaking backwards compatibility. I'll do this only if DoS is a real concern voiced by a developer with authority on the subject.

matthew.britton wrote:

Maybe, although I'm kind of leery of breaking backwards compatibility. I'll do
this only if DoS is a real concern voiced by a developer with authority on the
subject.

OK, just thinking of possibilities :)

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 16 2015, 3:41 PM
Reedy added a project: MediaWiki-API.
Reedy set Security to None.
Reedy moved this task from Unsorted to Non-Code on the MediaWiki-API board.
Catrope removed a subscriber: Catrope.Jan 13 2016, 9:28 PM