Page MenuHomePhabricator

API Developer supports different request media types
Closed, ResolvedPublic

Description

"As an API Developer, I want to support different request body content types for my endpoints, so that I can handle requests from legacy clients or clients that use a network standard."

Although for new code in the Core REST API we'll almost always be using JSON, some REST APIs implemented in extensions might need other request body media types. Parsoid/PHP, for example, needs to accept JSON, form-urlencoded, and multipart-form data.

We need a mechanism for an extension to declare that an endpoint accepts non-JSON media types. The default should be JSON only; if media types are declared, they have to be exhaustive (i.e. if a list is provided, and JSON isn't on the list, don't accept JSON).

The router should check the media type of an incoming request against the acceptable media types for the endpoint. It should return a 415 error if the media type isn't supported.

It should parse the request body if the media type is known; otherwise it should pass the body directly to the REST Handler. Known media types should be JSON, form-urlencoded, and multipart-form to start. We might support other media types in the future.

Original description by @Arlolra below.


It seems to assume JSON,

$body = json_decode( $request->getBody()->getContents(), true ) ?? [];

https://github.com/wikimedia/parsoid/blob/master/extension/src/Rest/Handler/ParsoidHandler.php#L155

On the JS side, we have,

// application/json
// application/x-www-form-urlencoded
// multipart/form-data

https://github.com/wikimedia/parsoid/blob/master/lib/api/ParsoidService.js#L116-L141

There are a few mocha tests for this which are failing,
https://github.com/wikimedia/parsoid/blob/master/tests/mocha/api.js#L74-L113

RESTBase currently sends everything to Parsoid in JSON, so it hasn't been an issue. But currently, if I attempt to use VE without a RESTBase in the middle (with a setting like),

$wgVirtualRestConfig['modules']['parsoid'] = array(
	'url' => 'http://localhost/rest.php',
	'domain' => 'localhost',
);

It will render a page just fine but saving is broken since it tries to POST in an unsupported encoding.

Not sure that setup is something we want to support going forward though?

Details

Related Gerrit Patches:
mediawiki/services/parsoid : masterSupport for other content encodings in the REST API

Event Timeline

Arlolra created this task.Oct 30 2019, 5:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 30 2019, 5:22 PM
Arlolra renamed this task from Support for other content encodings the REST API to Support for other content encodings in the REST API.Oct 30 2019, 5:23 PM

@eprodromou This is a potential blocker for enabling use of VisualEditor with Parsoid/PHP on private wikis that don't use RESTBase. So, officewiki, wikitech, and other private wikis. Is this something your team can look at in the coming week?

@eprodromou This is a potential blocker for enabling use of VisualEditor with Parsoid/PHP on private wikis that don't use RESTBase. So, officewiki, wikitech, and other private wikis. Is this something your team can look at in the coming week?

I didn't even know this was a goal. Honestly, I think this is an AMAZING possibility. I think this would be something we could start next week, and could be complete for a two-week sprint. Does that fit in for your timeframe?

If this works... it would be amazing. What a huge leap forward for the usability of MW!

@eprodromou This is a potential blocker for enabling use of VisualEditor with Parsoid/PHP on private wikis that don't use RESTBase. So, officewiki, wikitech, and other private wikis. Is this something your team can look at in the coming week?

I didn't even know this was a goal.

Sorry yes. We didn't discover this till we tried out VE locally without RESTBase

Honestly, I think this is an AMAZING possibility. I think this would be something we could start next week, and could be complete for a two-week sprint. Does that fit in for your timeframe?

Yes, we can just leave the private wikis on Parsoid/JS till then.

eprodromou added a comment.EditedNov 6 2019, 4:58 PM

(deleted dupe)

eprodromou renamed this task from Support for other content encodings in the REST API to API Developer supports different request media types.Nov 6 2019, 8:02 PM
eprodromou updated the task description. (Show Details)

I changed the description to fit in with our user story format. Hope that's not too disruptive.

eprodromou triaged this task as High priority.Nov 6 2019, 8:37 PM

Should we consider deprecating sending non-json input?

Should we consider deprecating sending non-json input?

That is a MediaWiki-as-a-product decision. I don't have a strong opinion. If that makes sense for MediaWiki as a product, we should get clients to move over to JSON input as part of the Parsoid-PHP switchover.

Should we consider deprecating sending non-json input?

So, product-wise, I think not. We're going to be handling binary data for file uploads, so while I think we'll be predominately JSON-only, we should support non-JSON input data for selected endpoints.

I believe that only supporting non-JSON input when it's explicitly declared for the endpoint should be sufficient.

The core REST API doesn't have a policy on the content type of the request body. ParsoidHandler is the thing assuming JSON input, core does not do content type validation. The idea was that handlers would be responsible checking the content-type and responding with a 415 if necessary. Maybe some of the logic could be factored out into a helper provided by core, but I don't think the Router should be validating the request content type before it executes the handler. We talked about that in T221177 and concluded that it opens a big can of worms.

You say that the three formats you want are application/json, application/x-www-form-urlencoded, and multipart/form-data. For the latter two formats, these are parsed by PHP prior to request execution, and we provide access to the parsed array via the $_POST wrapper RequestInterface::getPostParams(). So all you really need is something like:

function getParsedBody() {
    $request = $this->getRequest();
    $contentType = $request->getHeader( 'Content-Type' )[0] ?? '';
    switch ( $contentType ) {
        case 'application/x-www-form-urlencoded':
        case 'multipart/form-data':
            return $request->getPostParams();
        case 'application/json':
            return json_decode( $request->getBody()->getContents(), true );
        default:
            throw new HttpException( 'Unsupported Media Type', 415 );
    }
}

Please let me know if you need any more help with this.

Change 551240 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Support for other content encodings in the REST API

https://gerrit.wikimedia.org/r/551240

Change 551240 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Support for other content encodings in the REST API

https://gerrit.wikimedia.org/r/551240

ssastry closed this task as Resolved.Fri, Nov 15, 8:24 PM
ssastry claimed this task.

Thanks @eprodromou and @tstarling for the attention and pointers.