Page MenuHomePhabricator

MediaWiki\Session\SessionTest::testSecretsRoundTripping makes mediawiki/core CI fail
Closed, ResolvedPublic

Description

10:46:59 There was 1 failure:
10:46:59 
10:46:59 1) MediaWiki\Session\SessionTest::testSecretsRoundTripping with data set #4 (true)
10:46:59 Failed asserting that 'cqO3CO075sqXbsDkowNX/e4RijSDqLVPPK9hVPZR8XY=.iFfh0r0+CDp8uTgJIIte2w==.2hXE5Q==' is not equal to true.
10:46:59 
10:46:59 /workspace/src/tests/phpunit/includes/session/SessionTest.php:111
10:46:59 /workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:516
10:46:59 /workspace/src/tests/phpunit/phpunit.php:101
10:46:59 /workspace/src/tests/phpunit/phpunit.php:153

Event Timeline

Zabe triaged this task as Unbreak Now! priority.Sep 14 2022, 10:48 AM

Blocks all core merges.

Interestingly, I couldn’t reproduce this after just pulling MediaWiki core, but the error started to appear after I updated composer dependencies:

luwe@hostname /var/www/html/wiki1 (master $ u=) $ git fetch
remote: Counting objects: 3596, done
remote: Finding sources: 100% (1128/1128)
remote: Getting sizes: 100% (290/290)
remote: Compressing objects: 100% (2707564/2707564)
remote: Total 1128 (delta 800), reused 916 (delta 686)
Receiving objects: 100% (1128/1128), 1.80 MiB | 3.33 MiB/s, done.
Resolving deltas: 100% (800/800), completed with 404 local objects.
From https://gerrit.wikimedia.org/r/mediawiki/core
   378bcecc7c..fde2ac1ac9  master              -> gerrit/master
   d5735fd334..d9f3ff2e21  REL1_35             -> gerrit/REL1_35
   f77d68a6b8..09f071cc5c  REL1_37             -> gerrit/REL1_37
   51ef6d6e82..ffce2b1364  REL1_38             -> gerrit/REL1_38
   38f4774ce9..c43fbe6e4c  REL1_39             -> gerrit/REL1_39
   de4b2bb986..dbcabe05a8  fundraising/REL1_35 -> gerrit/fundraising/REL1_35
   5597455b6f..3a4dc82e47  wmf/1.39.0-wmf.27   -> gerrit/wmf/1.39.0-wmf.27
   92abb13ed6..1934220d8d  wmf/1.39.0-wmf.28   -> gerrit/wmf/1.39.0-wmf.28
 * [new branch]            wmf/1.40.0-wmf.1    -> gerrit/wmf/1.40.0-wmf.1
luwe@hostname /var/www/html/wiki1 (master $ u-87) $ git rebase
Successfully rebased and updated refs/heads/master.
luwe@hostname /var/www/html/wiki1 (master $ u=) $ run-tests tests/phpunit/includes/session/SessionTest.php 
Using PHP 8.1.2
PHPUnit 8.5.28 #StandWithUkraine

..........                                                        10 / 10 (100%)

Time: 1.47 seconds, Memory: 78.50 MB

OK (10 tests, 29 assertions)
luwe@hostname /var/www/html/wiki1 (master $ u=) $ composer update
> ComposerHookHandler::onPreUpdate
Loading composer repositories with package information
Info from https://repo.packagist.org: #StandWithUkraine
Updating dependencies
Lock file operations: 0 installs, 4 updates, 0 removals
  - Upgrading league/uri (6.7.1 => 6.8.0)
  - Upgrading sebastian/comparator (3.0.3 => 3.0.5)
  - Upgrading sebastian/exporter (3.1.4 => 3.1.5)
  - Upgrading wikimedia/parsoid (v0.16.0-a20 => v0.17.0-a1)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 0 installs, 4 updates, 0 removals
  - Downloading sebastian/exporter (3.1.5)
  - Downloading sebastian/comparator (3.0.5)
  - Downloading league/uri (6.8.0)
  - Downloading wikimedia/parsoid (v0.17.0-a1)
  - Upgrading sebastian/exporter (3.1.4 => 3.1.5): Extracting archive
  - Upgrading sebastian/comparator (3.0.3 => 3.0.5): Extracting archive
  - Upgrading league/uri (6.7.1 => 6.8.0): Extracting archive
  - Upgrading wikimedia/parsoid (v0.16.0-a20 => v0.17.0-a1): Extracting archive
Package phpunit/php-token-stream is abandoned, you should avoid using it. No replacement was suggested.
Generating optimized autoload files
59 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
> ComposerVendorHtaccessCreator::onEvent
luwe@hostname /var/www/html/wiki1 (master $ u=) $ run-tests tests/phpunit/includes/session/SessionTest.php 
Using PHP 8.1.2
PHPUnit 8.5.28 #StandWithUkraine

......F...                                                        10 / 10 (100%)

Time: 1.43 seconds, Memory: 78.50 MB

There was 1 failure:

1) MediaWiki\Session\SessionTest::testSecretsRoundTripping with data set #4 (true)
Failed asserting that 'TgkoNGKwwmQLCXFVOJXTr8vv364u/4Bq1Ur5+5cy/Q8=.k9Kg41gBJlsrtu6o6FBRxQ==.R9grDA==' is not equal to true.

/var/www/html/wiki1/tests/phpunit/includes/session/SessionTest.php:111
/var/www/html/wiki1/tests/phpunit/MediaWikiIntegrationTestCase.php:516
=== Logs generated by test case
[objectcache] [debug] MainWANObjectCache using store {class} {"class":"EmptyBagOStuff"}
===

FAILURES!
Tests: 10, Assertions: 28, Failures: 1.

So it’s probably one of the upgraded dependencies? Unless the test is flaky (but it’s failed a few more times since then).

composer require sebastian/comparator==3.0.3 sebastian/exporter==3.1.4 fixes the test locally.

Apparently the new comparator version considers any truthy value to be equal to true, which in this test we don’t want. (But I don’t think we want to use assertSame() either, because one of the $data values is an object.) Probably due to https://github.com/sebastianbergmann/comparator/pull/99.

Change 832269 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832269

Change 831984 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@wmf/1.40.0-wmf.1] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/831984

Apparently the new comparator version considers any truthy value to be equal to true, […] Probably due to https://github.com/sebastianbergmann/comparator/pull/99.

Yeah, specifically it does this if the expected value is a boolean, then the loosely-compared to is first attempted to be interpreted as a boolean as well, akin to what would happen natively e.g. in a conditional statement where values are also effectively casted to booleans.

If the assert parameters are reversed, with the string being the known value, then it still passes as before. This whole type-based comparison is a bit weird to me, and departs from what the native == operator does, but I can see how it has benefits for the loose comparison case. It seems very opionated and something I'd definitely not expect in a minor release, and certainly not such that PHPUnit exposes it directly to end-users without as much as a patch release.

Change 832269 merged by jenkins-bot:

[mediawiki/core@master] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832269

Change 831984 merged by jenkins-bot:

[mediawiki/core@wmf/1.40.0-wmf.1] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/831984

Change 832315 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@REL1_39] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832315

Change 832315 merged by Jforrester:

[mediawiki/core@REL1_39] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832315

Change 832555 had a related patch set uploaded (by Brian Wolff; author: Krinkle):

[mediawiki/core@REL1_38] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832555

Change 832555 merged by Brian Wolff:

[mediawiki/core@REL1_38] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832555

Change 832558 had a related patch set uploaded (by Zabe; author: Krinkle):

[mediawiki/core@REL1_37] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832558

Change 832559 had a related patch set uploaded (by Zabe; author: Krinkle):

[mediawiki/core@REL1_35] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832559

Change 832558 merged by jenkins-bot:

[mediawiki/core@REL1_37] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832558

Change 832559 merged by jenkins-bot:

[mediawiki/core@REL1_35] session: Fix broken SessionTest case due to PHPUnit dependency change

https://gerrit.wikimedia.org/r/832559