Page MenuHomePhabricator

Error contacting the Parsoid/RESTBase server (HTTP 500) when saving page with accented character in title
Closed, InvalidPublic

Description

MediaWiki 1.35.0
Windows Server 2012
IIS
php 7.4.11
MySQL 5.7

Upon creating a page with accented character, I get a 500 error:

20201208MW_error.png (842×1 px, 103 KB)

Error in debug file:

[exception] [c11a41a1549cb534ecde26d4] /rest.php/10.21.37.104/v3/transform/html/to/wikitext/Titre_ᠣaract鳥   LogicException from line 344 of E:\Websites\mediawiki-1.35.0\extensions\VisualEditor\includes\VEParsoid\src\Rest\Handler\ParsoidHandler.php: Title not found!
#0 E:\Websites\mediawiki-1.35.0\extensions\VisualEditor\includes\VEParsoid\src\Rest\Handler\TransformHandler.php(136): VEParsoid\Rest\Handler\ParsoidHandler->createPageConfig(NULL, integer, NULL)
#1 E:\Websites\mediawiki-1.35.0\includes\Rest\Router.php(365): VEParsoid\Rest\Handler\TransformHandler->execute()
#2 E:\Websites\mediawiki-1.35.0\includes\Rest\Router.php(320): MediaWiki\Rest\Router->executeHandler(VEParsoid\Rest\Handler\TransformHandler)
#3 E:\Websites\mediawiki-1.35.0\includes\Rest\EntryPoint.php(133): MediaWiki\Rest\Router->execute(MediaWiki\Rest\RequestFromGlobals)
#4 E:\Websites\mediawiki-1.35.0\includes\Rest\EntryPoint.php(100): MediaWiki\Rest\EntryPoint->execute()
#5 E:\Websites\mediawiki-1.35.0\rest.php(31): MediaWiki\Rest\EntryPoint::main()
#6 {main}

Event Timeline

The following code seems to be fixing it.

E:\Websites\mediawiki-1.35.0>git diff extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php
warning: LF will be replaced by CRLF in extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php.
The file will have its original line endings in your working directory
diff --git a/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php b/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php
index 649a9aa3..e706aaa7 100644
--- a/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php
+++ b/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php
@@ -338,6 +338,8 @@ abstract class ParsoidHandler extends Handler {
                string $title, ?int $revision, string $wikitextOverride = null,
                string $pagelanguageOverride = null
        ): PageConfig {
+               $title = utf8_encode($title);
                $title = $title ? Title::newFromText( $title ) : Title::newMainPage();
                if ( !$title ) {
                        // TODO use proper validation

MediaWiki uses UTF-8 everywhere throughout. It looks like something else in Windows or IIS is converting the URL path to some legacy Windows' encoding, and then MediaWiki interprets those bytes as UTF-8.

I will look into it. But note I did not run into any issues with VisualEditor when running MW 1.27.3 or MW 1.33

I have checked my php settings (default-language is utf-8) and the globalization for IIS and everything seems to be fine.
The input on the POST line below is UTF-8.

[http] POST: http://10.21.37.104/rest.php/10.21.37.104/v3/transform/html/to/wikitext/Titre_%C3%A0_petits_caract%C3%A8res
[objectcache] MainWANObjectCache using store EmptyBagOStuff

But, further in the debug log, in the line below, it is ANSI

Start request POST /rest.php/10.21.37.104/v3/transform/html/to/wikitext/Titre_ᠰetits_caract鳥s
IP: 10.21.37.104
HTTP HEADERS:
CONTENT-TYPE: multipart/form-data; boundary=------------------------03db13d545ad6bfe
CONTENT-LENGTH: 461
API-USER-AGENT: VisualEditor-MediaWiki/1.35.0
USER-AGENT: VisualEditor-MediaWiki/1.35.0
HOST: 10.21.37.104
ACCEPT-LANGUAGE: en
ACCEPT: text/html; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/HTML/2.0.0"
(end headers)

It would help if someone can point to where the conversion takes place.

Another test: created a page with accents in the title without using VisualEditor, it works fine and the page title in the DB is utf-8 encoded.

Likely answer: bug in FastCGI module implementation (https://support.microsoft.com/en-us/help/2277918/fix-a-php-application-that-depends-on-the-request-uri-server-variable)
In any event, adding the registry key

reg add HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\w3svc\Parameters /v FastCGIUtf8ServerVariables /t REG_MULTI_SZ /d REQUEST_URI\0PATH_INFO

fixed the issue (without having to download the hotfix which means it was probably included in the later versions of the module). Now why is it that it only affects VE, that I am not sure off...

Now why is it that it only affects VE, that I am not sure off...

Hmm, now that I think about it, I realize that MediaWiki is able to recognize "fallback" encoding in URL parameters, depending on the site content language (for example, Windows-1252 for English, as defined by $fallback8bitEncoding = 'windows-1252'; in MessagesEn.php).

This is why you can access the page "Résumé" at the URL https://en.wikipedia.org/w/index.php?title=R%E9sum%E9, in addition to the "correct" URL https://en.wikipedia.org/w/index.php?title=R%C3%A9sum%C3%A9 (although, curiously, https://en.wikipedia.org/wiki/R%E9sum%E9 does not work).

That might be why the rest of your site worked fine? Perhaps the API code doesn't try to detect the fallback encoding, and perhaps it should (but it might also have been left off intentionally, as this kind of magical detection could also prove problematic in an API).

That might be why the rest of your site worked fine? Perhaps the API code doesn't try to detect the fallback encoding, and perhaps it should (but it might also have been left off intentionally, as this kind of magical detection could also prove problematic in an API).

Maybe the API is using a server environment variable (e.g. REQUEST_URI) to derive its target for the rest service and this is why it happens only for the API.

That might be why the rest of your site worked fine? Perhaps the API code doesn't try to detect the fallback encoding, and perhaps it should (but it might also have been left off intentionally, as this kind of magical detection could also prove problematic in an API).

Maybe the API is using a server environment variable (e.g. REQUEST_URI) to derive its target for the rest service and this is why it happens only for the API.

hi @ti_infotrad – would it be accurate for us to think this issue is resolved?

hi @ti_infotrad – would it be accurate for us to think this issue is resolved?

Yes

Per above, it was a configuration issue, not a VisualEditor bug.

I documented this on https://www.mediawiki.org/wiki/Extension:VisualEditor#Troubleshooting (https://www.mediawiki.org/w/index.php?title=Extension:VisualEditor&diff=4285920&oldid=4285674). @ti_infotrad Please have a look and correct/improve that if you can.

Per above, it was a configuration issue, not a VisualEditor bug.

I documented this on https://www.mediawiki.org/wiki/Extension:VisualEditor#Troubleshooting (https://www.mediawiki.org/w/index.php?title=Extension:VisualEditor&diff=4285920&oldid=4285674). @ti_infotrad Please have a look and correct/improve that if you can.

Looks good to me.

The following code seems to be fixing it.

E:\Websites\mediawiki-1.35.0>git diff extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php
warning: LF will be replaced by CRLF in extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php.
The file will have its original line endings in your working directory
diff --git a/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php b/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php
index 649a9aa3..e706aaa7 100644
--- a/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php
+++ b/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php
@@ -338,6 +338,8 @@ abstract class ParsoidHandler extends Handler {
                string $title, ?int $revision, string $wikitextOverride = null,
                string $pagelanguageOverride = null
        ): PageConfig {
+               $title = utf8_encode($title);
                $title = $title ? Title::newFromText( $title ) : Title::newMainPage();
                if ( !$title ) {
                        // TODO use proper validation

This fixed it for me!

Thank you very much.