API's "Reverse continue" ability
Open, LowPublic

Description

"Continue" provides a good way to "page" results in the UI - the "next" button contains all "continue" values, and the handler appends them to the original query. But, there is no simple generic way to "reverse continue". I propose that continuable queries add an extra output block continuerev - that include all parameters needed to get the "previous page" of results. That might mean including opposite "dir" parameter, or the fact that it is now going in reverse.

If the user chooses to reverse the continuation direction on the result set #5 by appending continuerev instead of continue block to the original query, they would get previous page (result set #4), but the meaning of continue and continuerev should not switch meaning (otherwise the <previous> and <next> UI buttons would switch their meaning, or the user would have to use an extra parameter to keep track of the last usage direction. Using the continuerev block again should return the result set #3.

Yurik created this task.Mar 31 2015, 9:29 PM
Yurik updated the task description. (Show Details)
Yurik raised the priority of this task from to Needs Triage.
Yurik added a project: MediaWiki-API.
Yurik added a subscriber: Yurik.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 31 2015, 9:29 PM
Anomie triaged this task as Low priority.Mar 31 2015, 9:32 PM
Anomie set Security to None.
Anomie added a subscriber: Anomie.Mar 31 2015, 9:34 PM

I note this would likely require significant additional complexity in nearly all API query modules, so a real use case (beyond "someone might want a 'prev' button") would be helpful.

Yurik added a comment.EditedMar 31 2015, 9:57 PM

The use case would be any kind of paging user interface with lists. For example, if we were to replace Special:AllPages with a AJAXy one, we would need this functionality. Same goes for any other similar page - recent changes, or Gather's lists, etc. The alternative would be for each UI page to know how each list works, its ordering, and how to manually construct reverse continuation.

So nothing yet that's not hypothetical?

Yurik added a comment.Apr 1 2015, 1:46 PM

Gather team would use it if we had access to it - I think I will implement it for the list=lists & list=listpages, but revcontinue would not work properly with the rest of continuation system. I am sure apps & mobile web would love to use it to reduce reloading for all list pages. Its a matter of availability - if it exists, dynamic list paging will be implemented. If it does not exist, there will either be hacks, or we will simply continue using static reloads for each list page, making interface clunky.

Anomie added a comment.Apr 2 2015, 6:59 PM

Ok, so someone *would* actually use it right away. That makes it more interesting.

First step would probably be for someone (i.e. you) to audit all the existing query modules in core and extensions that support continuation at all and determine how their queries might support reverse-continuation:

  • If they're continuing with "offset=X", it's easy to supply X-$limit for reverse.
  • Many others are probably doing an equivalent of WHERE fields >= continue ORDER BY fields ASC for continuation; for reverse continuation they could use the first value fetched from the query and WHERE fields < continue ORDER BY fields DESC.
    • If forward and reverse continuation are using separate parameters rather that one 'continue' parameter with a direction indicator, we'd need to make specifying both an error. I'd probably prefer one 'continue' parameter with a direction indicator, though.
    • After a reverse-continuation, the forward-continuation would be the reverse-continuation that was supplied in the current query while reverse-continuation would be the last row actually output (but we'd still need to fetch $limit+1 to see if we've reached the start).
    • Special consideration might be needed for modules with 'dir' parameters if they do more than adjust ">=" vs "<=" in the continuation and the direction for ORDER BY.
    • You'd also need to consider that you'll probably be wanting to do an array_reverse() over arrays in the output, after fully populating them. This might turn out to be better done in some cases by adding a new metadata item to have ApiResult::transformForTypes() do it, rather than trying to track down all the prop subarrays on the various page nodes. 'assoc'-type arrays in the result should be fine, since they don't guarantee ordering.
  • Another option would be to just run the query in reverse with appropriate LIMIT and OFFSET to find the first item from the previous "page". That'd be rather inefficient though, since it requires a second database query.
  • Other cases, particularly ones where reversing is just not possible?

At the same time, you could develop a general outline as to how ApiResult would need to change to support reverse-continuation, both from the standpoint of addContinueParam() and from the standpoint of actually processing the data.

With reference to this question, I'm writing a mobile app that uses paging for going forward - this works very well. It would be great to have a reverse paging mechanism in place so that I don't have to work around this within the app itself.