Page MenuHomePhabricator

Increase limits for list=search to match Special:Search
Closed, ResolvedPublic

Description

[12:28:21] <Dispenser> Why is &list=search limited to 50 (default: 10) while the website is limited to 500 (default: 20)

Any reason we can't make the API search match Special:Search?

Something like:

			'limit' => array(
				ApiBase::PARAM_DFLT => 20,
				ApiBase::PARAM_TYPE => 'limit',
				ApiBase::PARAM_MIN => 1,
				ApiBase::PARAM_MAX => ApiBase::LIMIT_SML2,
				ApiBase::PARAM_MAX2 => ApiBase::LIMIT_SML2
			),

Opened a task so we can have the discussion, rather than it just going through gerrit

Event Timeline

Reedy raised the priority of this task from to Medium.
Reedy updated the task description. (Show Details)
Reedy added subscribers: Reedy, Dispenser.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Reedy set Security to None.

I note that the default limit of 10 is standard in the API and I don't see much reason to change that to 20 for this one query, people who want 20 can easily specify that.

As for the maximum of 50 without apihighlimits and 500 with, I see these limits were explicitly lowered by @Catrope in r55615 "per rainman". Perhaps he can let us know what rainman said about it that prompted the change?

I note that the default limit of 10 is standard in the API and I don't see much reason to change that to 20 for this one query, people who want 20 can easily specify that.

As for the maximum of 50 without apihighlimits and 500 with, I see these limits were explicitly lowered by @Catrope in r55615 "per rainman". Perhaps he can let us know what rainman said about it that prompted the change?

I'd probably agree with not changing the default; not much gain.

I wonder if Roan would remember... But I wouldn't be surprised if it was mainly due to the inadequate cluster we had back then, and it was to essentially "rate limit" api users somewhat.

Hopefully the current search team can advise is there's any reason we can't make them match the user UI

Deskana lowered the priority of this task from Medium to Lowest.Dec 29 2015, 10:28 PM
Deskana added a subscriber: Deskana.

Hopefully the current search team can advise is there's any reason we can't make them match the user UI

In general I'd agree that the limits between the API and UI should be consistent. In the absence of any API consumer that really needs this, though, this is a low priority issue to fix.

Dabfix was going to use the API search, but it was faster and easier to make 1x request for 5,000 results (1 second) vs 100x requests for 50 results each (10+ seconds, 100 ms ping).

Change 273111 had a related patch set uploaded (by Anomie):
ApiQuerySearch: Match limits to those in Special:Search

https://gerrit.wikimedia.org/r/273111

Since Roan unsubscribed himself, it doesn't seem like he's interested in answering. Let's just fix this, or at least force a decision by submitting a patch.

@Deskana, please add appropriate Discovery-team people to code review the patch from a "would things blow up if normal users can do srlimit=500 and bots can do srlimit=5000?" standpoint.

@Deskana, please add appropriate Discovery-team people to code review the patch from a "would things blow up if normal users can do srlimit=500 and bots can do srlimit=5000?" standpoint.

Done. Thanks!

actually Special:Search allows anyone, even anonymous users, to fetch 5000 results: https://en.wikipedia.org/w/index.php?title=Special%3ASearch&profile=default&search=the&fulltext=Search&limit=5000

In our logs pretty much no-one uses it. As long as feature usage is low it shouldn't cause any problems.

Change 273111 merged by jenkins-bot:
ApiQuerySearch: Match limits to those in Special:Search

https://gerrit.wikimedia.org/r/273111

Anomie claimed this task.