Page MenuHomePhabricator

ParserIntegrationTest for Scribunto failing in Wikibase CI
Closed, ResolvedPublic

Description

This is blocking all Wikibase patches from being merged.
Looks like it might have something to do with Scribunto.
Not done any investigation yet.

From https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-hhvm-docker/40538/console

16:35:16 1) ParserIntegrationTest::testParse with data set "luaParserTests.txt: Scribunto: expandTemplate with headers" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running Scribunto: expandTemplate with headers {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [gitinfo] [info] Computed cacheFile=/workspace/src/gitinfo.json for /workspace/src {"private":false}
16:35:16 [gitinfo] [info] Cache incomplete for /workspace/src {"private":false}
16:35:16 [MessageCache] [info] MessageCache::load: Loading en... local cache is empty, global cache is expired/volatile, loading from database {"private":false}
16:35:16 [Wikibase] [debug] {method}: setting {settingName} was given as a closure, resolve it to {logValue} {"method":"Wikibase\\SettingsArray::getSetting","settingName":"conceptBaseUri","logValue":"'http:\/\/example.org\/entity\/'"}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "luaParserTests.txt: Scribunto: expandTemplate with headers": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16  '<h2><span class="mw-headline" id="foo">foo</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/index.php?title=Parser_test&amp;action=edit&amp;section=1" title="Edit section: foo">edit</a><span class="mw-editsection-bracket">]</span></span></h2>
16:35:16 -<h2><span class="mw-headline" id="bar">bar</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/index.php?title=Template:Scribunto_template_with_headers&amp;action=edit&amp;section=T-1" title="Template:Scribunto template with headers">edit</a><span class="mw-editsection-bracket">]</span></span></h2>
16:35:16 -'
16:35:16 +<h2><span class="mw-headline" id="bar">bar</span><span class="mw-editsection"><span class="mw-editsection-bracket">[</span><a href="/index.php?title=Template:Scribunto_template_with_headers&amp;action=edit&amp;section=T-1" title="Template:Scribunto template with headers">edit</a><span class="mw-editsection-bracket">]</span></span></h2>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 2) ParserIntegrationTest::testParse with data set "luaParserTests.txt: Scribunto: Strip markers in CSS" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running Scribunto: Strip markers in CSS {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [gitinfo] [info] Computed cacheFile=/workspace/src/gitinfo.json for /workspace/src {"private":false}
16:35:16 [gitinfo] [info] Cache incomplete for /workspace/src {"private":false}
16:35:16 [MessageCache] [info] MessageCache::load: Loading en... local cache is empty, global cache is expired/volatile, loading from database {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "luaParserTests.txt: Scribunto: Strip markers in CSS": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16 -'<div style="color:#ff0000"></div>
16:35:16 -'
16:35:16 +'<div style="color:#ff0000"></div>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 3) ParserIntegrationTest::testParse with data set "parserTests.txt: Non-existent language" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running Non-existent language {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [MessageCache] [info] MessageCache::load: Loading en... local cache is empty, global cache is expired/volatile, loading from database {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: Non-existent language": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16 -'<div class="mw-highlight mw-content-ltr" dir="ltr"><pre>foobar</pre></div>
16:35:16 -'
16:35:16 +'<div class="mw-highlight mw-content-ltr" dir="ltr"><pre>foobar</pre></div>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 4) ParserIntegrationTest::testParse with data set "parserTests.txt: No language specified" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running No language specified {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [MessageCache] [info] MessageCache::load: Loading en... local cache is empty, global cache is expired/volatile, loading from database {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: No language specified": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16 -'<div class="mw-highlight mw-content-ltr" dir="ltr"><pre>foo</pre></div>
16:35:16 -'
16:35:16 +'<div class="mw-highlight mw-content-ltr" dir="ltr"><pre>foo</pre></div>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 5) ParserIntegrationTest::testParse with data set "parserTests.txt: No language specified (no wellformed xml)" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running No language specified (no wellformed xml) {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [MessageCache] [info] MessageCache::load: Loading en... local cache is empty, global cache is expired/volatile, loading from database {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: No language specified (no wellformed xml)": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16 -'<div class="mw-highlight mw-content-ltr" dir="ltr"><pre>bar</pre></div>
16:35:16 -'
16:35:16 +'<div class="mw-highlight mw-content-ltr" dir="ltr"><pre>bar</pre></div>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 6) ParserIntegrationTest::testParse with data set "parserTests.txt: XSS is escaped" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running XSS is escaped {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [MessageCache] [info] MessageCache::load: Loading en... local cache is empty, global cache is expired/volatile, loading from database {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: XSS is escaped": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16  &lt;/script>&lt;script>alert('XSS');&lt;/script>
16:35:16 -%253cscript%253ealert(document.cookie)%253c/script%253e</pre></div>
16:35:16 -'
16:35:16 +%253cscript%253ealert(document.cookie)%253c/script%253e</pre></div>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 7) ParserIntegrationTest::testParse with data set "parserTests.txt: Default behaviour (inner is pre)" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running Default behaviour (inner is pre) {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: Default behaviour (inner is pre)": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16  '<div class="mw-highlight mw-content-ltr" dir="ltr"><pre><span></span><span class="kd">var</span> <span class="nx">a</span><span class="p">;</span>
16:35:16 -</pre></div>
16:35:16 -'
16:35:16 +</pre></div>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 8) ParserIntegrationTest::testParse with data set "parserTests.txt: Multiline <source/> in lists" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running Multiline <source/> in lists {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [MessageCache] [info] MessageCache::load: Loading en... local cache is empty, global cache is expired/volatile, loading from database {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: Multiline <source/> in lists": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16  '<ul><li><div class="mw-highlight mw-content-ltr" dir="ltr"><pre>a
16:35:16  b</pre></div></li></ul>
16:35:16  <ul><li>foo <div class="mw-highlight mw-content-ltr" dir="ltr"><pre>a
16:35:16 -b</pre></div></li></ul>
16:35:16 -'
16:35:16 +b</pre></div></li></ul>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 9) ParserIntegrationTest::testParse with data set "parserTests.txt: Custom attributes" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running Custom attributes {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: Custom attributes": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16  '<div id="foo" class="bar mw-highlight mw-content-rtl" dir="rtl" style="font-size: larger;"><pre><span></span><span class="kd">var</span> <span class="nx">a</span><span class="p">;</span>
16:35:16 -</pre></div>
16:35:16 -'
16:35:16 +</pre></div>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 10) ParserIntegrationTest::testParse with data set "parserTests.txt: No code" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running No code {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: No code": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16 -'<div class="mw-highlight mw-content-ltr" dir="ltr"></div>
16:35:16 -'
16:35:16 +'<div class="mw-highlight mw-content-ltr" dir="ltr"></div>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 11) ParserIntegrationTest::testParse with data set "parserTests.txt: Just whitespace" ('[details omitted]')
16:35:16 === Logs generated by test case
16:35:16 [wfDebug] [debug] ParserTestRunner::runTest: running Just whitespace {"private":false}
16:35:16 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
16:35:16 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: Just whitespace": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
16:35:16 ===
16:35:16 Failed asserting that two strings are equal.
16:35:16 --- Expected
16:35:16 +++ Actual
16:35:16 @@ @@
16:35:16 -'<div class="mw-highlight mw-content-ltr" dir="ltr"></div>
16:35:16 -'
16:35:16 +'<div class="mw-highlight mw-content-ltr" dir="ltr"></div>'
16:35:16 
16:35:16 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
16:35:16 /workspace/src/maintenance/doMaintenance.php:94
16:35:16 
16:35:16 FAILURES!
16:35:16 Tests: 4372, Assertions: 14883, Failures: 11, Skipped: 38.

Event Timeline

Addshore triaged this task as Unbreak Now! priority.Mar 19 2019, 4:38 PM
Addshore created this task.
Restricted Application added subscribers: Liuxinyu970226, TerraCodes, Aklapper. · View Herald TranscriptMar 19 2019, 4:38 PM

This has shown up in Wikibase CI (but nowhere else) because Wikibase CI is one of the few places that actually runs CI with Scribunto loaded.

See: T200976, T125050, T204252

Addshore renamed this task from ParserIntegrationTest failing in Wikibase CI to ParserIntegrationTest for Scribunto failing in Wikibase CI.Mar 19 2019, 4:49 PM

Without looking to hard this smells like https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/497522/

I'll propose a revert and check if it fixes Wikibase CI

Change 497553 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/core@master] Revert "Sync up with Parsoid parserTests.txt"

https://gerrit.wikimedia.org/r/497553

Restricted Application added a project: User-Addshore. · View Herald TranscriptMar 19 2019, 4:54 PM
cscott added a subscriber: cscott.Mar 19 2019, 5:14 PM

Yeah, we should just fix the scributo tests, not revert patches. This was a set of 9 or so dependent patches to merge, unrolling would be quite a chore, and the scribunto fix should just be to remove some trailing newlines from the parser tests....

Change 497564 had a related patch set uploaded (by Jforrester; owner: Arlolra):
[mediawiki/extensions/Scribunto@master] Update tests to match parser changes

https://gerrit.wikimedia.org/r/497564

@cscott, that sounds great! :)

Change 497553 abandoned by Addshore:
Revert "Sync up with Parsoid parserTests.txt"

https://gerrit.wikimedia.org/r/497553

Anomie added a subscriber: Anomie.

Looks like two of the failures are in Scribunto, but the other 9 are in SyntaxHighlight.

Change 497571 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/extensions/SyntaxHighlight_GeSHi@master] Update tests to match parser changes

https://gerrit.wikimedia.org/r/497571

Change 497571 merged by jenkins-bot:
[mediawiki/extensions/SyntaxHighlight_GeSHi@master] Update tests to match parser changes

https://gerrit.wikimedia.org/r/497571

Change 497564 merged by jenkins-bot:
[mediawiki/extensions/Scribunto@master] Update tests to match parser changes

https://gerrit.wikimedia.org/r/497564

Arlolra closed this task as Resolved.Mar 19 2019, 6:58 PM

Since https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/497320 merged, I'm going to assume this is done.

@Addshore sorry for the pain :(

We are seeing a similar (but not quite the same) error here. Can it be related?

https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/GrowthExperiments/+/497955/
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-hhvm-docker/40899/console

01:34:35 1) ParserIntegrationTest::testParse with data set "parserTests.txt: <graph>" ('[details omitted]')
01:34:35 === Logs generated by test case
01:34:35 [wfDebug] [debug] ParserTestRunner::runTest: running <graph> {"private":false}
01:34:35 [wfDebug] [debug] Parser: using preprocessor: Preprocessor_Hash {"private":false}
01:34:35 [MessageCache] [info] MessageCache::load: Loading en... local cache is empty, global cache is expired/volatile, loading from database {"private":false}
01:34:35 [PHPUnitCommand] [info] FAILURE in ParserIntegrationTest::testParse with data set "parserTests.txt: <graph>": [PHPUnit_Framework_ExpectationFailedException] Failed asserting that two strings are equal. {"private":false}
01:34:35 ===
01:34:35 Failed asserting that two strings are equal.
01:34:35 --- Expected
01:34:35 +++ Actual
01:34:35 @@ @@
01:34:35  '<div class="mw-graph mw-graph-always" data-graph-id="cb66159a7f0070f4faaef2f353cb67c0d178d678"></div>
01:34:35 -<div class="mw-graph mw-graph-always" style="min-width:100px;min-height:200px" data-graph-id="35d4ad35da17b39e7e4464fd7af741dadda01a91"></div>
01:34:35 -'
01:34:35 +<div class="mw-graph mw-graph-always" style="min-width:100px;min-height:200px" data-graph-id="35d4ad35da17b39e7e4464fd7af741dadda01a91"></div>'
01:34:35 
01:34:35 /workspace/src/tests/phpunit/suites/ParserIntegrationTest.php:53
kostajh reopened this task as Open.Mar 21 2019, 2:01 PM

Yep, most likely related. @Arlolra will probably write you a patch once he wakes up/gets online this am, but if you're impatient just removing the newlines from the indicated places in parserTests.txt will get you going again.

Change 498137 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/extensions/Graph@master] Update tests to match parser changes

https://gerrit.wikimedia.org/r/498137

Change 498137 merged by jenkins-bot:
[mediawiki/extensions/Graph@master] Update tests to match parser changes

https://gerrit.wikimedia.org/r/498137

Arlolra closed this task as Resolved.Mar 21 2019, 4:38 PM
mmodell changed the subtype of this task from "Task" to "Production Error".Wed, Aug 28, 11:07 PM