API: Purpose of "batchcomplete" property and implementation
Open, LowestPublic

Description

Implemented in https://gerrit.wikimedia.org/r/#/c/152359/

Intended purpose or use is not clear.

Provided release note:

Simplified continuation will return a "batchcomplete" property in the result
when a batch of pages is complete.

Also, it seems to be added to responses for action=watch:

{
  "batchcomplete":"",
  "watch":[
    {"title":"Main Page","unwatched":"","message":"<p>The page \"<strong class=\"selflink\">Main Page</strong>\" has been removed from <a href=\"/wiki/Special:Watchlist\" title=Special:Watchlist>your watchlist</a>.\n</p>"}
  ]
}

Are clients expected to do something with this property?

If a client is using continuation, should they do something other than passing back the given continue value until exhausted?

I assume it doesn't let them influence the continuation, but one use case I can imagine is the user not being interested in part of the result and they might top following the continuation chain after batchcomplete and thus not bother starting the generator. Is that the (only) intended use?

Krinkle created this task.Dec 19 2014, 8:16 AM
Krinkle updated the task description. (Show Details)
Krinkle raised the priority of this task from to Needs Triage.
Krinkle assigned this task to Anomie.
Krinkle added a project: MediaWiki-API.
Krinkle changed Security from none to None.
Krinkle added a subscriber: Krinkle.
jayvdb added a subscriber: jayvdb.Dec 19 2014, 11:23 AM

The purpose of the property is to let clients know when a batch of pages is complete. It's mainly useful along with generators, but for consistency it works the same way any time continuation is a possibility. Note that action=watch can use generators to supply the list of pages to be watched, and while it isn't likely to ever run into an incomplete batch the consistency is probably a good idea.

For example, a query with generator=allpages and prop=links|linkshere might return 10 pages from the generator on the first query, but only return the links for the first three and half the links for the fourth, and return backlinks for the first five (but "first" sorted by namespace+title rather than by page_id). The next continuation could return more links for the fourth without finishing (if that page has lots of links) and backlinks for two more, and so on. Eventually all the links and all the backlinks for all of the 10 pages would be output, at which point "batchcomplete" is set to indicate that it's safe for the client to process these 10 pages without worrying that a future continuation will return additional data for any these pages. The continuation from the response with "batchcomplete" will begin returning data for the next set of 10 pages from the generator.

The client doesn't have to pay any attention to "batchcomplete" if it wants to slurp in the entire result set, but if the result set is going to be very large this may use a prohibitive amount of memory or other resources.

With the "raw" continuation where the client has to know when to ignore the continuation data returned for the generator module, this sort of batching was also expected to be handled on the client side. The new simplified continuation takes away the need for clients to know about the generator complexity, but that also means that clients have to be told about the batching.

It's not clear what the desired outcome is from this task. Is this just an instance of T2001?

Aklapper triaged this task as Lowest priority.Dec 29 2014, 10:43 PM
Anomie removed Anomie as the assignee of this task.Jan 16 2015, 7:25 PM

I'm going to tag this as Documentation and goodfirstbug and put it up for grabs. All it needs is someone expanding the text on mw:API:Query#Continuing queries to explain this.

Anomie moved this task from Unsorted to Non-Code on the MediaWiki-API board.Feb 19 2015, 6:18 PM

I'm going to tag this as Documentation and goodfirstbug and put it up for grabs. All it needs is someone expanding the text on mw:API:Query#Continuing queries to explain this.

I added a sentence there mentioning it, but it could probably use improvement by someone who has a better idea how much detail to copy from T84977#936343.