Page MenuHomePhabricator

Parsoid in MW 1.35 is not compatible with PHP 7.2.0 (uses array_key_first())
Closed, ResolvedPublic

Description

Setup

  • MediaWiki 1.35.0-rc.2 (0f1a2a2) 18:43, 21 August 2020
  • PHP 7.2.24-0ubuntu0.18.04.6 (apache2handler)
  • MariaDB 10.1.44-MariaDB-0ubuntu0.18.04.1
  • VisualEditor 0.1.2 (cc95db5) 18:39, 21 August 2020
  • wikimedia/parsoid 0.12.0-a23

Configuration

$wgVirtualRestConfig['modules']['parsoid'] = [
        'url' => 'http://sandbox.semantic-mediawiki.org:8142',
        'domain' => '99071'
        ];
$wgVirtualRestConfig['modules']['parsoid']['forwardCookies'] = true;


wfLoadExtension( 'VisualEditor' );
$wgVisualEditorAvailableNamespaces = [
        'File' => false,
        'Category' => false,
        'Extra' => true,
        'Locked' => true
        ];
$wgVisualEditorEnableWikitext = true;
$wgDefaultUserOptions['visualeditor-newwikitext'] = 1;

Issue
With the configuration above VisualEditor works better than ever, i.e. it is faster and I can even switch between visual editing and source editing without problems. However once I comment out the $wgVirtualRestConfig settings, VisualEditor fails completely:

Error contacting the Parsoid/RESTBase server (HTTP 500)

From the communication about VisualEditor for Mw 1.35 I picked up, I may be wrong here, that $wgVirtualRestConfig settings are no longer required, since nodejs/parsoid is no longer required.

Basically the question is: Am I doing something wrong or is this expected?

Event Timeline

Kghbln created this task.Aug 22 2020, 9:56 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 22 2020, 9:56 AM

From the communication about VisualEditor for Mw 1.35 I picked up,

@Kghbln: Could you please link to such communication? https://www.mediawiki.org/wiki/Manual:$wgVirtualRestConfig states that it is "still in use".

Kghbln added a comment.EditedAug 22 2020, 10:23 AM

@Aklapper I got it from here: T259693 and mw.o instuctions. Moreover this is not about $wgVirtualRestConfig being still in use or not. I am just confused that I still have to connect VisualEditor to parsoid/nodejs which is what I understand I am currently doing with my configuration.

Ah, thanks! Was wondering if you maybe had T248343 in mind now that I just stumbled upon that ticket. :)

Aklapper renamed this task from VisualeEditor on MW 1.35 - clarification needed to VisualEditor on MW 1.35 - clarification needed: Still have to connect VE to parsoid/nodejs or not?.Aug 22 2020, 10:28 AM

Ah, thanks! Was wondering if you maybe had T248343 in mind now that I just stumbled upon that ticket. :)

No worries. This task however also claims that in an ideal world it should be possible by just invoking VisualEditor. I understand that I am passed this bug and its fix since I am using yesterdays VisualEditor code.

Hmm, let's wait for the VE experts.

Reedy updated the task description. (Show Details)
matmarex added a subscriber: matmarex.

$wgVirtualRestConfig should not be required, but it's still used to connect your wiki to JS Parsoid (or to PHP Parsoid running on a different server).

With the configuration you posted, it looks like your wiki is still using JS Parsoid, and by removing $wgVirtualRestConfig you would change it to use bundled PHP Parsoid, which apparently fails with the HTTP 500 error.

I've made some guesses to find the URL where PHP Parsoid would serve pages, and where the error can be seen: https://sandbox.semantic-mediawiki.org/w/rest.php/sandbox.semantic-mediawiki.org:8142/v3/page/html/Berlin

[28323e5608b719b94c777b15] /w/rest.php/sandbox.semantic-mediawiki.org:8142/v3/page/html/Berlin InvalidArgumentException from line 71 of /var/www/html/sbxsmw/w/vendor/guzzlehttp/psr7/src/Uri.php: Unable to parse URI: /w/rest.php/sandbox.semantic-mediawiki.org:8142/v3/page/html/Berlin

Backtrace:

#0 /var/www/html/sbxsmw/w/includes/Rest/RequestFromGlobals.php(36): GuzzleHttp\Psr7\Uri->__construct(string)
#1 /var/www/html/sbxsmw/w/includes/Rest/Router.php(266): MediaWiki\Rest\RequestFromGlobals->getUri()
#2 /var/www/html/sbxsmw/w/includes/Rest/EntryPoint.php(133): MediaWiki\Rest\Router->execute(MediaWiki\Rest\RequestFromGlobals)
#3 /var/www/html/sbxsmw/w/includes/Rest/EntryPoint.php(100): MediaWiki\Rest\EntryPoint->execute()
#4 /var/www/html/sbxsmw/w/rest.php(31): MediaWiki\Rest\EntryPoint::main()
#5 {main}

I don't know why it's failing. Maybe it doesn't like the :?

@Kghbln Can you try if backporting this patch solves the issue? https://gerrit.wikimedia.org/r/c/mediawiki/core/+/610034

Sorry for another question. May I ask if wfLoadExtension( 'Parsoid' ); is required or not?

Sorry for another question. May I ask if wfLoadExtension( 'Parsoid' ); is required or not?

It's not required, unless you wish to use a development version of Parsoid instead of the bundled one (https://www.mediawiki.org/wiki/Parsoid/PHP#Developer_setup).

Sorry for another question. May I ask if wfLoadExtension( 'Parsoid' ); is required or not?

It's not required, unless you wish to use a development version of Parsoid instead of the bundled one (https://www.mediawiki.org/wiki/Parsoid/PHP#Developer_setup).

Thank you for answering. But I am experiencing omitting wfLoadExtension( 'Parsoid', "$IP/vendor/wikimedia/parsoid/extension.json" ); breaks my local wiki instance's Visual editor and make Parsoid disappear in the Installed extensions list on [[Special:Version]].

Should I create a new bug report?

Yes, please.

@matmarex Thank you for looking at this.

With the configuration you posted, it looks like your wiki is still using JS Parsoid, and by removing $wgVirtualRestConfig you would change it to use bundled PHP Parsoid, which apparently fails with the HTTP 500 error.

Affirmative.

Can you try if backporting this patch solves the issue? https://gerrit.wikimedia.org/r/c/mediawiki/core/+/610034

Now cherry-picked. Unfortunately, it does not help the cause. I still get error 500.

@Kghbln Would it be possible for you to cherry-pick that patch on sandbox.semantic-mediawiki.org (or some other public site where you can reproduce the problem), so that I can experiment with it?

(I'd expect to see some different error message on https://sandbox.semantic-mediawiki.org/w/rest.php/sandbox.semantic-mediawiki.org:8142/v3/page/html/Berlin after it's cherry-picked.)

Kghbln added a comment.EditedAug 25 2020, 9:05 PM

@matmarex I cherry-picked on this very wiki. Since it was not working I enabled the settings to $wgVirtualRestConfig again. Now commented out.

Thanks! I think I solved the riddle:

First, I was actually looking at the wrong page. https://sandbox.semantic-mediawiki.org/w/rest.php/sandbox.semantic-mediawiki.org:8142/v3/page/html/Berlin is still failing with the same exception, which is very surprising to me, that might be a separate issue?

I think it should be failing with a message like this one: https://sandbox.semantic-mediawiki.org/w/rest.php/blahblah/v3/page/html/Berlin

[a0a3f6ed207549b7734b9548] /w/rest.php/blahblah/v3/page/html/Berlin Wikimedia\ParamValidator\ValidationException from line 159 of /var/www/html/sbxsmw/w/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php: Validation of `domain` failed: invalid-domain {"expected":"sandbox.semantic-mediawiki.org","actual":"blahblah"}

(this is expected)

So, that indicates that we should see the article parsed by Parsoid at this URL: https://sandbox.semantic-mediawiki.org/w/rest.php/sandbox.semantic-mediawiki.org/v3/page/html/Berlin

And, that has a new and interesting exception:

[e9389edac5ba1acf95228462] /w/rest.php/sandbox.semantic-mediawiki.org/v3/page/html/Berlin/46375 Error from line 99 of /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Wt2Html/PP/Handlers/DisplaySpace.php: Call to undefined function Wikimedia\Parsoid\Wt2Html\PP\Handlers\array_key_first()

Backtrace:

#0 [internal function]: Wikimedia\Parsoid\Wt2Html\PP\Handlers\DisplaySpace::leftHandler(DOMText, Wikimedia\Parsoid\Config\Env, array, boolean, NULL)
#1 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Utils/DOMTraverser.php(78): call_user_func(array, DOMText, Wikimedia\Parsoid\Config\Env, array, boolean, NULL)
#2 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Utils/DOMTraverser.php(147): Wikimedia\Parsoid\Utils\DOMTraverser->callHandlers(DOMText, Wikimedia\Parsoid\Config\Env, array, boolean, NULL)
#3 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Utils/DOMTraverser.php(159): Wikimedia\Parsoid\Utils\DOMTraverser->traverse(Wikimedia\Parsoid\Config\Env, DOMText, array, boolean, NULL)
#4 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Utils/DOMTraverser.php(159): Wikimedia\Parsoid\Utils\DOMTraverser->traverse(Wikimedia\Parsoid\Config\Env, DOMElement, array, boolean, NULL)
#5 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Utils/DOMTraverser.php(159): Wikimedia\Parsoid\Utils\DOMTraverser->traverse(Wikimedia\Parsoid\Config\Env, DOMElement, array, boolean, NULL)
#6 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(147): Wikimedia\Parsoid\Utils\DOMTraverser->traverse(Wikimedia\Parsoid\Config\Env, DOMElement, array, boolean, NULL)
#7 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(847): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->Wikimedia\Parsoid\Wt2Html\{closure}(DOMElement, array, boolean)
#8 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(896): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->doPostProcess(DOMDocument)
#9 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(913): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->process(DOMDocument)
#10 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(152): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->processChunkily(string, array)
#11 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(202): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parseChunkily(string, array)
#12 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipelineFactory.php(299): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parseToplevelDoc(string, array)
#13 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Core/WikitextContentModelHandler.php(78): Wikimedia\Parsoid\Wt2Html\ParserPipelineFactory->parse(string)
#14 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Parsoid.php(152): Wikimedia\Parsoid\Core\WikitextContentModelHandler->toDOM(Wikimedia\Parsoid\Config\Env)
#15 /var/www/html/sbxsmw/w/vendor/wikimedia/parsoid/src/Parsoid.php(184): Wikimedia\Parsoid\Parsoid->parseWikitext(VEParsoid\Config\PageConfig, array)
#16 /var/www/html/sbxsmw/w/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/ParsoidHandler.php(540): Wikimedia\Parsoid\Parsoid->wikitext2html(VEParsoid\Config\PageConfig, array, NULL)
#17 /var/www/html/sbxsmw/w/extensions/VisualEditor/includes/VEParsoid/src/Rest/Handler/PageHandler.php(110): VEParsoid\Rest\Handler\ParsoidHandler->wt2html(VEParsoid\Config\PageConfig, array)
#18 /var/www/html/sbxsmw/w/includes/Rest/Router.php(365): VEParsoid\Rest\Handler\PageHandler->execute()
#19 /var/www/html/sbxsmw/w/includes/Rest/Router.php(320): MediaWiki\Rest\Router->executeHandler(VEParsoid\Rest\Handler\PageHandler)
#20 /var/www/html/sbxsmw/w/includes/Rest/EntryPoint.php(133): MediaWiki\Rest\Router->execute(MediaWiki\Rest\RequestFromGlobals)
#21 /var/www/html/sbxsmw/w/includes/Rest/EntryPoint.php(100): MediaWiki\Rest\EntryPoint->execute()
#22 /var/www/html/sbxsmw/w/rest.php(31): MediaWiki\Rest\EntryPoint::main()
#23 {main}

The function array_key_first() was introduced in PHP 7.3.0. Your wiki is running PHP 7.2.24. I think that explains your problems.

MediaWiki 1.35.0 is supposed to only require 7.2.22. Therefore, I think Parsoid should not be using this function.

matmarex renamed this task from VisualEditor on MW 1.35 - clarification needed: Still have to connect VE to parsoid/nodejs or not? to Parsoid in MW 1.35 is not compatible with PHP 7.2.22 (uses array_key_first()).Aug 25 2020, 9:25 PM
matmarex edited projects, added Parsoid; removed VisualEditor.

Thanks! I think I solved the riddle:

Really cool. Thank you so much for digging into this. This is very much appreciated.

As you know there are discussions at T257879 to release MW 1.35 without officially supporting PHP 7.2. This one here may be a turning point in this discussion since it presents hard facts for dropping support of PHP 7.2 in case it does not get changed.

which is very surprising to me, that might be a separate issue?

Will do. Thanks for the note.

This should be a problem for Wikimedia wikis as well, since we run PHP 7.2.31 (https://en.wikipedia.org/wiki/Special:Version).

However, apparently we have a polyfill loaded from the Composer module 'symfony/polyfill-php73'. Neither Parsoid nor MediaWiki specifies it as a dependency, rather it's used by some other packages. I don't understand why it's even available in production, since it's only in packages-dev.

(I see now that it was added in https://gerrit.wikimedia.org/r/c/mediawiki/vendor/+/542282 but that offers no explanation.)

(I see now that it was added in https://gerrit.wikimedia.org/r/c/mediawiki/vendor/+/542282 but that offers no explanation.)

composer.lock tells all

Basically, the symfony packages need it. The symfony packages are needed by things like psysh

Change 622442 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/services/parsoid@master] Avoid array_key_first() for compatibility with PHP 7.2

https://gerrit.wikimedia.org/r/622442

Reedy renamed this task from Parsoid in MW 1.35 is not compatible with PHP 7.2.22 (uses array_key_first()) to Parsoid in MW 1.35 is not compatible with PHP 7.2.0 (uses array_key_first()).Aug 25 2020, 11:22 PM

Change 622224 had a related patch set uploaded (by C. Scott Ananian; owner: Bartosz Dziewoński):
[mediawiki/services/parsoid@REL1_35] Avoid array_key_first() for compatibility with PHP 7.2

https://gerrit.wikimedia.org/r/622224

Change 622224 merged by C. Scott Ananian:
[mediawiki/services/parsoid@REL1_35] Avoid array_key_first() for compatibility with PHP 7.2

https://gerrit.wikimedia.org/r/622224

Change 622442 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Avoid array_key_first() for compatibility with PHP 7.2

https://gerrit.wikimedia.org/r/622442

cscott closed this task as Resolved.Aug 26 2020, 1:24 PM
cscott claimed this task.
cscott added a subscriber: cscott.

Patched merged into master and to the REL1_35 branch.

Thanks at all for working on this. I'll sit tight for the next parsoid tag. As promised it created task T261344 for the second issue detected while working on this.

Change 625641 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a8

https://gerrit.wikimedia.org/r/625641

Change 625641 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a8

https://gerrit.wikimedia.org/r/625641