rest.php
Open, HighPublic
Actions

Assigned To

Authored By

	daniel
	May 7 2024, 3:59 PM

Description

Currently, we are exposing REST APIs for each wiki from two pathes:

/api/rest_v1/ for APIs handled by standalone services (restbase and otherwise).
/w/rest.php/ for APIs handled by MediaWiki (core and extensions)

We should consolidate the two, so clients do not have to be aware how an API is implemented, and we are free to change how it is implemented without breaking clients. Also, shared prefix for all REST APIsmakes configuration of the edge caches more straight forward.

Proposal

This can be achived by introducing a rewrite rule into the Apache config that maps /api/ to /w/rest.php exactly in the same way we map /wiki/ to /w/index.php.

In the ATS layer, we need a rule that will send traffic for /api/ to MW app servers in the API cluster. However, traffic for the
/api/rest_v1/ prefix still needs to be sent to RESTbase / REST Gateway.

Once the routing rules are in place, the $wgRestPath setting can be changed to /api/, analogous to $wgArticlePath. But that's outside of the scoep iof this ticket, and needs to be announced widely.

Impact

APIs that are currently exposed under /w/rest.php will also become availabe under /api/, for example:

/w/rest.php/v1/page/{title}/html will become available under /api/v1/page/{title}/html.
/w/rest.php/oauth2/authorize will become available under /api/oauth2/authorize.
/w/rest.php/ipinfo/v0/revision/{id} will become available under /api/ipinfo/v0/revision/{id}.

Note that we will have /api/rest_v1/ prefix side by side with core's /api/v1/ prefix, which is slightly confusing. The idea is to move APIs out of both generic prefixes over time:

With the sunsetting of RESTbase, we can move APIs out of the /api/rest_v1/ prefix to dedicated prefixes, e.g. /api/citation.v1/ instead of /api/rest_v1/data/citation/.
With the introducion of modules into the MediaWiki REST framework, we can move APIs out of the /api/v1/ prefix to dedicated prefixes, e.g. /api/content.v1/page/ instead of /api/v1/page/.

Future

If we have a need to route different paths under /api/ to different services (beyond rest_v1) , we can route all requests for the /api/ prefix through an API gateway. This would also provide unified intrumentation for all REST endpoints.

Aproval

This idea has been floated and discussed for a while. This task should be used to discuss any remaining issuesd, and document consensus.

SRE agrees (per meeting with @MShilova_WMF, @akosiaris, @Vgutierrez, and @FJoseph-WMF on May 22, see notes)
MW PMs agree (approved by @Bmueller)

References

Related Objects
Search...

		Status	Subtype	Assigned	Task
		In Progress		daniel	T366835 REST: API modularization and versioning (tracking)
		Open		daniel	T364400 map the /api/ prefix to /w/rest.php

Event Timeline

daniel created this task.May 7 2024, 3:59 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 7 2024, 3:59 PM

daniel updated the task description. (Show Details)May 7 2024, 4:00 PM

hnowlan subscribed.May 7 2024, 4:15 PM

Jdforrester-WMF awarded a token.May 7 2024, 4:26 PM

Jdforrester-WMF subscribed.May 7 2024, 4:29 PM

Scott_French subscribed.May 8 2024, 1:12 AM

This seems like a pretty reasonable idea to me. Could we implement this remapping at the ATS layer rather than the Apache one, in a manner that would mean that when we need to cache we only need to store each effective URL once?

are /api/v1/page/{title}/html, /api/oauth2/authorize and /api/ipinfo/v0/revision/{id} all new API paths or existing ones?

Vgutierrez moved this task from Backlog to Radar/Not for service by Traffic on the Traffic board.May 8 2024, 9:39 AM

In T364400#9779996, @hnowlan wrote:

Could we implement this remapping at the ATS layer rather than the Apache one, in a manner that would mean that when we need to cache we only need to store each effective URL once?

It would be preferable to not do so. The caching gains would be minimal, but more importantly: we hope to minimize the details of the application layer that are spread into the cache configuration (there will always be necessary cases, but the more we avoid it, the easier things are in the future).

In T364400#9779996, @hnowlan wrote:

This seems like a pretty reasonable idea to me. Could we implement this remapping at the ATS layer rather than the Apache one, in a manner that would mean that when we need to cache we only need to store each effective URL once?

I hope we can stop serving from the old prefix in the not-too-far-away future. We could turn it into a 308.

In any case, from the MW perspective, I don't care where the mapping is. I just recvalled Giuseppe saying that he'd prefer to have it in Apache. And as far as I know, that's where the mapping for index.php is. It's probably a good idea to keep them side by side.

are /api/v1/page/{title}/html, /api/oauth2/authorize and /api/ipinfo/v0/revision/{id} all new API paths or existing ones?

The endpoints all exist, but currently under /w/rest.php, not under /api/.

daniel added a subscriber: akosiaris.May 13 2024, 1:31 PM

daniel claimed this task.May 16 2024, 2:56 PM

daniel moved this task from Incoming (Needs Triage) to Needs Further Discussion on the MW-Interfaces-Team board.

daniel updated the task description. (Show Details)May 22 2024, 4:23 PM

daniel added subscribers: FJoseph-WMF, MShilova_WMF, Vgutierrez.

Krinkle subscribed.May 22 2024, 5:39 PM

FJoseph-WMF triaged this task as High priority.May 23 2024, 3:56 PM

daniel updated the task description. (Show Details)Thu, May 30, 6:09 PM

daniel added a parent task: T366835: REST: API modularization and versioning (tracking).Thu, Jun 6, 6:59 PM

@daniel thank you for al the prep work! This is good to go :-)

In T364400#9780622, @BBlack wrote:

In T364400#9779996, @hnowlan wrote:

Could we implement this remapping at the ATS layer rather than the Apache one, in a manner that would mean that when we need to cache we only need to store each effective URL once?

It would be preferable to not do so. The caching gains would be minimal, but more importantly: we hope to minimize the details of the application layer that are spread into the cache configuration (there will always be necessary cases, but the more we avoid it, the easier things are in the future).

Even more to @BBlack's comment, I would just have apache funnel anything under /api it receives to an endpoint in mediawiki, and do routing there.

In T364400#9926454, @Joe wrote:

Even more to @BBlack's comment, I would just have apache funnel anything under /api it receives to an endpoint in mediawiki, and do routing there.

For everything, including /api/rest_v1? I was thinking that we should keep routing /api/rest_v1 directly to ApiGateway/RestGateway. Doing that routing in MediaWiki will require some prep work, and would likely come with a latency panelty for the PCS endpoints.

But everything under /api outside of /api/rest_v1 should indeed just go to /w/rest.php.

map the /api/ prefix to /w/rest.phpOpen, HighPublicActions