Maniphest T319218

Scribunto test failures on php8.1
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	Bawolff
	Oct 3 2022, 5:15 PM

Description

When running the tests locally:

bawolff@bawolff:/var/www/html/w$ php8 tests/phpunit/phpunit.php extensions/Scribunto/tests/phpunit/
Using PHP 8.1.2
PHPUnit 8.5.30 #StandWithUkraine

S.......S....................................................   61 / 1192 (  5%)
..........S...SS...SS........................................  122 / 1192 ( 10%)
..S............................................S.............  183 / 1192 ( 15%)
.E................................................SSS........  244 / 1192 ( 20%)
..S...........................S..............................  305 / 1192 ( 25%)
.............................................................  366 / 1192 ( 30%)
.S...........................................................  427 / 1192 ( 35%)
........S....................................................  488 / 1192 ( 40%)
.............................................................  549 / 1192 ( 46%)
.............................................................  610 / 1192 ( 51%)
.............................................................  671 / 1192 ( 56%)
..............................................S.S............  732 / 1192 ( 61%)
.............................................................  793 / 1192 ( 66%)
.............................................................  854 / 1192 ( 71%)
.............................................SS..............  915 / 1192 ( 76%)
.............................................................  976 / 1192 ( 81%)
............................................................. 1037 / 1192 ( 86%)
............................................SSSSSSSSSSSSSSSSS 1098 / 1192 ( 92%)
SSSSSSSSSSSSSSSSSSSSSSS...................................... 1159 / 1192 ( 97%)
.................................                             1192 / 1192 (100%)

Time: 52.61 seconds, Memory: 96.50 MB

There was 1 error:

1) LuaStandalone: MediaWiki\Extension\Scribunto\Tests\Engines\LuaCommon\LuaCommonTest::testNonUtf8Errors
preg_match_all(): Passing null to parameter #2 ($subject) of type string is deprecated

/var/www/html/w/includes/MagicWordArray.php:308
/var/www/html/w/includes/parser/Parser.php:4105
/var/www/html/w/includes/parser/Parser.php:1625
/var/www/html/w/includes/parser/Parser.php:713
/var/www/html/w/includes/language/MessageCache.php:1374
/var/www/html/w/includes/language/Message.php:1427
/var/www/html/w/includes/language/Message.php:1004
/var/www/html/w/includes/language/Message.php:1042
/var/www/html/w/extensions/Scribunto/includes/Engines/LuaCommon/LuaError.php:86
/var/www/html/w/extensions/Scribunto/tests/phpunit/Engines/LuaCommon/LuaCommonTest.php:809
/var/www/html/w/tests/phpunit/MediaWikiIntegrationTestCase.php:516

So far i haven't figured out what is going on here yet. Somehow the message is null, which should not happen.

Details

Subject	Repo	Branch	Lines +/-
Make sure that lua stack trace is valid UTF-8.	mediawiki/extensions/Scribunto	REL1_38	+18 -1
Make sure that lua stack trace is valid UTF-8.	mediawiki/extensions/Scribunto	REL1_39	+18 -1
Make sure that lua stack trace is valid UTF-8.	mediawiki/extensions/Scribunto	master	+18 -1

Customize query in gerrit

Related Objects

Mentioned In: T321234: Exception: "Malformed UTF-8 characters" in Parser\MagicWordArray (via WikitextContentHandler)
Mentioned Here: T266129: "Invariant failed: Bad UTF-8 (full string verification)" due to content in database (from Pasoid PegTokenizer)
T321234: Exception: "Malformed UTF-8 characters" in Parser\MagicWordArray (via WikitextContentHandler)

Event Timeline

Bawolff created this task.Oct 3 2022, 5:15 PM

Restricted Application added subscribers: Base, Aklapper. · View Herald TranscriptOct 3 2022, 5:15 PM

Reedy added a project: MW-1.39-release.Oct 3 2022, 5:56 PM

I think this might be more of a MW parser bug...

		# Now match and remove the rest of them
		$mwa = $this->magicWordFactory->getDoubleUnderscoreArray();
		$this->mDoubleUnderscores = $mwa->matchAndRemove( $text );

Trying to do replacements on $text of null...

Probably a few other cases in internalParse where it's pointlessly running over null....

The part that confuses me - is how is parse() being called on null. It seems like a real message is being used.

I think what's happening is that the stack trace contains invalid UTF-8, which is causing preg_replace to error and return null, which then gets passed through everything

So i feel like this is a real test failure, and MW core should be throwing exceptions instead of silently turning pages in null. I made https://gerrit.wikimedia.org/r/c/mediawiki/core/+/838174 for that part.

Change 838226 had a related patch set uploaded (by Brian Wolff; author: Brian Wolff):

[mediawiki/extensions/Scribunto@master] Make sure that lua stack trace is valid UTF-8.

https://gerrit.wikimedia.org/r/838226

gerritbot added a project: Patch-For-Review.Oct 4 2022, 5:44 PM

Reedy moved this task from Blocker to Not a blocker on the MW-1.39-release board.Oct 4 2022, 11:17 PM

Reedy moved this task from Backlog to Bundled extensions/skins on the PHP 8.1 support board.Oct 5 2022, 6:40 PM

Change 838226 merged by jenkins-bot:

[mediawiki/extensions/Scribunto@master] Make sure that lua stack trace is valid UTF-8.

https://gerrit.wikimedia.org/r/838226

Legoktm closed this task as Resolved.Oct 6 2022, 4:44 AM

Legoktm assigned this task to Bawolff.

Change 839042 had a related patch set uploaded (by Legoktm; author: Brian Wolff):

[mediawiki/extensions/Scribunto@REL1_39] Make sure that lua stack trace is valid UTF-8.

https://gerrit.wikimedia.org/r/839042

Change 839042 merged by jenkins-bot:

[mediawiki/extensions/Scribunto@REL1_39] Make sure that lua stack trace is valid UTF-8.

https://gerrit.wikimedia.org/r/839042

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.5; 2022-10-10).Oct 6 2022, 5:00 AM

Change 839012 had a related patch set uploaded (by Brian Wolff; author: Brian Wolff):

[mediawiki/extensions/Scribunto@REL1_38] Make sure that lua stack trace is valid UTF-8.

https://gerrit.wikimedia.org/r/839012

Change 839012 merged by jenkins-bot: