Page MenuHomePhabricator

Allow limiting the number of results in queries using the MediaWiki API
Closed, ResolvedPublic

Description

This query, for example, returns more than 10 results:

SELECT * WHERE {
  SERVICE wikibase:mwapi {
      bd:serviceParam wikibase:api "Search" .
      bd:serviceParam wikibase:endpoint "www.wikidata.org" .
      bd:serviceParam mwapi:srsearch "test" .
      bd:serviceParam mwapi:srlimit 10 .
      ?title wikibase:apiOutput mwapi:title .
  }
} limit 50

I was expecting that srlimit would limit the results to the number given, since I only wanted the top 10 results.

It seems that, regardless of the value of srlimit, it fetches all pages of the results (presumably fetching srlimit each time).

I think it would make sense to only fetch all of the results when srlimit isn't used. If someone wants all the results, fetching them in smaller batches seems inefficient (and likely to cause timeouts). If they don't want all of the results, it's wasting time requesting information which isn't going to be used.

Event Timeline

Nikki created this task.Jun 15 2018, 9:57 PM
Restricted Application added projects: Wikidata, Discovery. · View Herald TranscriptJun 15 2018, 9:57 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

The problem here is that the API has no idea that "srlimit" is somehow special. Different APIs have different names for limit parameter. But I guess we could have some special parameter like bd:serviceParam wikibase:limit to specify overall limits.

Smalyshev triaged this task as Normal priority.Jun 19 2018, 1:08 AM
Smalyshev moved this task from Backlog to Doing on the User-Smalyshev board.

Change 440992 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[wikidata/query/rdf@master] Add parameter to limit results from API

https://gerrit.wikimedia.org/r/440992

Change 440992 merged by jenkins-bot:
[wikidata/query/rdf@master] Add parameter to limit results from API

https://gerrit.wikimedia.org/r/440992

Smalyshev removed a project: User-Smalyshev.
Smalyshev closed this task as Resolved.Jun 25 2018, 6:33 PM
Vvjjkkii renamed this task from Allow limiting the number of results in queries using the MediaWiki API to ctaaaaaaaa.Jul 1 2018, 1:03 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed Smalyshev as the assignee of this task.
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
JJMC89 renamed this task from ctaaaaaaaa to Allow limiting the number of results in queries using the MediaWiki API.Jul 1 2018, 1:35 AM
JJMC89 closed this task as Resolved.
JJMC89 assigned this task to Smalyshev.
JJMC89 lowered the priority of this task from High to Normal.
JJMC89 updated the task description. (Show Details)
JJMC89 added a subscriber: Aklapper.