Page MenuHomePhabricator

meta=tokens API does not use continuation when response gets truncated
Closed, ResolvedPublicBUG REPORT

Description

As an Action API user, I expect that whenever the response is truncated, I can follow continuation to get the remaining data with additional requests.

List of steps to reproduce (step by step, including full links if applicable):

  • Craft an API request that, without meta=tokens, just barely fits within the $wgAPIMaxResultSize.
  • Add meta=tokens, optionally with a type.

This is very unlikely to happen by accident, but it can be done manually: example URL

What happens?:

  • There is a warning that the result was truncated because it would otherwise be larger than the configured limit (12 MiB on Wikimedia wikis, “temporary” since at least 2012). This is expected.
  • The tokens are not included in the response, because ApiQuery::execute() processes prop modules, then list modules, then meta modules; we used prop modules to “fill up” the response, so by the time the API reaches the tokens module, it won’t fit anymore. This is fine (though I could also imagine the API processing meta modules first).
  • The response has no continue member. This is not fine: there is no way to get tokens based on this response, you don’t have anywhere to go from it.

What should have happened instead?:
There should be a continue member in the response, which when added to a subsequent request should result in a response without the data that was already returned, and with tokens in it. meta=siteinfo does it right, for example: initial URL (returns "continue": { "siprop": "general", "continue": "-||revisions" }), continued URL.

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc.:
1.39.0-wmf.14 (d160b61)

Event Timeline

Change 802893 had a related patch set uploaded (by Lucas Werkmeister; author: Lucas Werkmeister):

[mediawiki/core@master] Use continuation when meta=tokens doesn’t fit in response

https://gerrit.wikimedia.org/r/802893

Change 802893 merged by jenkins-bot:

[mediawiki/core@master] Use continuation when meta=tokens doesn’t fit in response

https://gerrit.wikimedia.org/r/802893

Is this worth backporting to previous releases? (Though perhaps we should first wait until it rolls out with the train next week, to check that it works in production.)

Works as expected in production (API sandbox URL).

Change 804695 had a related patch set uploaded (by Lucas Werkmeister; author: Lucas Werkmeister):

[mediawiki/core@REL1_35] Use continuation when meta=tokens doesn’t fit in response

https://gerrit.wikimedia.org/r/804695

Change 804696 had a related patch set uploaded (by Lucas Werkmeister; author: Lucas Werkmeister):

[mediawiki/core@REL1_37] Use continuation when meta=tokens doesn’t fit in response

https://gerrit.wikimedia.org/r/804696

Change 804706 had a related patch set uploaded (by Lucas Werkmeister; author: Lucas Werkmeister):

[mediawiki/core@REL1_38] Use continuation when meta=tokens doesn’t fit in response

https://gerrit.wikimedia.org/r/804706

Is this worth backporting to previous releases?

I’ve uploaded some backports now (for still-supported branches), since that’s what was done in T301041. I won’t mind if they’re not merged, though.

Change 804695 abandoned by Lucas Werkmeister:

[mediawiki/core@REL1_35] Use continuation when meta=tokens doesn’t fit in response

Reason:

doesn’t look like this will be merged, I guess 1.39 LTS is good enough

https://gerrit.wikimedia.org/r/804695

Change 804696 abandoned by Lucas Werkmeister:

[mediawiki/core@REL1_37] Use continuation when meta=tokens doesn’t fit in response

Reason:

doesn’t look like this will be merged, I guess 1.39 LTS is good enough

https://gerrit.wikimedia.org/r/804696

Change 804706 abandoned by Lucas Werkmeister:

[mediawiki/core@REL1_38] Use continuation when meta=tokens doesn’t fit in response

Reason:

doesn’t look like this will be merged, I guess 1.39 LTS is good enough

https://gerrit.wikimedia.org/r/804706