Page MenuHomePhabricator

Decide on pagination and links spec for Core REST API
Open, HighPublic

Description

The parent task proposes to include properties in the API response with links to next/previous page. Given that pagination is a very standard thing to do and will be needed elsewhere, we need to either come up with a standard way of embedding links to related resources and pagination, or (even better) use an existing standard.

http://samplacette.com/five-json-rest-api-link-formats-compared/ gives a fairly solid overview of the solutions and draft RFCs that exist out there.

Event Timeline

@Pchelolo I'm open to having this conversation but not to having it block shipping the history endpoint.

Are you OK with keeping "older"/"newer" in the definition for the release, and we can talk about a generic paging mechanism for the future? We're using a semantic versioning strategy, so the history endpoint will keep these properties in backwards-compatible future versions, but we can add other paging properties at a minor cost.

Sound good?

I'm open to having this conversation but not to having it block shipping the history endpoint.

That's why I've opened a separate ticket. I will be expanding in this here

I've seen using the link relations names as properties in JSON, which I think is what we did with Activity Streams 2.0 collections. So:

  • "first" : first segment in the collection
  • "next": link to next segment in the collection
  • "prev": link to previous segment in the collection
  • "last": link to last segment in the collection

As I mentioned to Petr, for collections sorted in reverse chronological order, this can be confusing for users. The "first" segment will include the last-added items, and the "last" page will include the oldest items, including the first item. The "next" link shows you items that were created previously, and the "prev" link shows you items that were created next after those. People mess this up all the time.

Adding missing MediaWiki-REST-API code project tag as Core Platform Team Initiatives (MW REST API in PHP) team tag is archived and its parent Platform Engineering team does not exist anymore

Checking in a few years later - could this task be reconsidered/reprioritized? We Connection-Team have quite a few endpoints where we didn't implement pagination due to the lack of a standard. And I imagine the same will be true for others; especially as the REST API sees broader adoption, which for the time being I imagine is partly hindered by the lack of essential features such as pagination.

Indeed! Thank you for raising this. Our work APP work for next year is focused on introducing standardization across Wikimedia API experiences, so this request is timely. Specifically, we want to look at things like versioning, pagination, error messages, etc. The goal is that developers would have a consistent web API experience, regardless of which set of services or endpoints they're using. We also hope that setting those standards with easy and repeatable adoption patterns will also make things a bit easier for API publishers like you 😁

I see the suggestions that were made in the earlier comment -- are those aligned with your latest thinking and preference for how pagination would look? If so, we will certainly take it into consideration when setting up the standard. Additionally, it sounds like you haven't actually implemented it anywhere yet -- is that correct?

Our work APP work for next year is focused on introducing standardization across Wikimedia API experiences, so this request is timely.

Thank you, happy to hear that!

I see the suggestions that were made in the earlier comment -- are those aligned with your latest thinking and preference for how pagination would look? If so, we will certainly take it into consideration when setting up the standard.

I don't have any preferences :) I do agree with T234470#5548026 that certain words might be confusing depending on the result ordering, but I don't have great proposals either.

Additionally, it sounds like you haven't actually implemented it anywhere yet -- is that correct?

We tried to implement as little pagination as possible, and only if we really needed it. For the CampaignEvents extension, we have two endpoints that accept a last_*_id for pagination, which really is the bare minimum; the number of results is not configurable either. I haven't audited other extensions or core for pagination-related parameters. The only other example that I'm aware of (but there might be more) is the page history endpoint discussed above, which uses bespoke older_than and newer_than parameters, and no way to change the limit.

Marking as high priority, with tentative goal to revisit in Q3.

This is a high priority task beacuse without this T387233 will not happen. We are waiting for T387233. T388348 task also affected by this task.