PHP 8.2 CI for OATHAuth on 1.42 fails due to missing pimple/container for Flow...
Closed, DuplicatePublic
Actions

Assigned To

None

Authored By

	Reedy
	May 15 2024, 12:07 PM

Project Tags

Referenced Files

None

Subscribers

Aklapper

Jdforrester-WMF

Lucas_Werkmeister_WMDE

Reedy

Tgr

Description

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/OATHAuth/+/1031468
https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php82-noselenium/665/console

13:04:15 [18.9MiB/4.44s]   [merge-plugin] Loading extensions/Flow/composer.json...
13:04:15 [18.9MiB/4.44s]   [merge-plugin] Adding pimple/pimple
...
13:04:17 [22.6MiB/6.48s]   [merge-plugin] Loading extensions/Flow/composer.json...
13:04:17 [22.6MiB/6.48s]   [merge-plugin] Adding pimple/pimple
...
13:04:25 [72bb5e9fd0f4e11ee83e69ce] [no req]   Error: Class "Pimple\Container" not found
13:04:25 Backtrace:
13:04:25 from /workspace/src/extensions/Flow/includes/Container.php(5)
13:04:25 #0 /workspace/src/includes/AutoLoader.php(223): require()
13:04:25 #1 /workspace/src/extensions/Flow/maintenance/FlowSetUserIp.php(41): AutoLoader::autoload()
13:04:25 #2 /workspace/src/maintenance/includes/LoggedUpdateMaintenance.php(49): Flow\Maintenance\FlowSetUserIp->doDBUpdates()
13:04:25 #3 /workspace/src/maintenance/update.php(201): LoggedUpdateMaintenance->execute()
13:04:25 #4 /workspace/src/maintenance/includes/MaintenanceRunner.php(698): UpdateMediaWiki->execute()
13:04:25 #5 /workspace/src/maintenance/doMaintenance.php(100): MediaWiki\Maintenance\MaintenanceRunner->run()
13:04:25 #6 /workspace/src/maintenance/update.php(302): require_once(string)
13:04:25 #7 {main}

Details

Related Changes in Gerrit:

	Subject	Author	Repo	Branch	Lines +/-
	zuul: Remove WebAuthn from OATHAuth on REL1_XX	Reedy	integration/config	master	+7 -0

Customize query in gerrit

Related Objects

Mentioned In: T365254: Build extension dependancy tree tool for zuul dependancies
T245027: Enable downloading recovery codes for two-factor (TOTP)
Mentioned Here: T363639: web-auth/webauthn-lib must be upgraded to 4+ for PHP 8.2+ support
T362412: Wikibase CI jobs wikibase-repo and wikibase-client broken on REL1_42
T364273: Wikibase breaks schema update with "Class 'Onoi\MessageReporter\NullMessageReporter' not found"

Event Timeline

Reedy created this task.May 15 2024, 12:07 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 15 2024, 12:07 PM

Reedy added a project: StructuredDiscussions.May 15 2024, 12:08 PM

Restricted Application added a project: Growth-Team. · View Herald TranscriptMay 15 2024, 12:08 PM

Reedy renamed this task from CI for OATHAuth on 1.42 fails due to missing pimple/container... to CI for OATHAuth on 1.42 fails due to missing pimple/container for Flow....May 15 2024, 12:10 PM

Reedy added a project: Continuous-Integration-Config.

Reedy mentioned this in T245027: Enable downloading recovery codes for two-factor (TOTP).May 15 2024, 12:15 PM

13:04:15 [18.9MiB/4.44s]   [merge-plugin] Loading extensions/Flow/composer.json...
13:04:15 [18.9MiB/4.44s]   [merge-plugin] Adding pimple/pimple
...
13:04:17 [22.6MiB/6.48s]   [merge-plugin] Loading extensions/Flow/composer.json...
13:04:17 [22.6MiB/6.48s]   [merge-plugin] Adding pimple/pimple

Reedy added a project: PHP 8.2 support.May 15 2024, 1:07 PM

Reedy updated the task description. (Show Details)

Not a PHP 8.2 bug (almost certainly) but fails in PHP82 jobs (on gate), but not on the tests run normally, which run on PHP81...

Reedy renamed this task from CI for OATHAuth on 1.42 fails due to missing pimple/container for Flow... to PHP 8.2 CI for OATHAuth on 1.42 fails due to missing pimple/container for Flow....May 15 2024, 1:11 PM

Reedy updated the task description. (Show Details)

PHP 8.2 seems to be working in master branch gate-and-submit (e.g. in this change), FWIW.

As far as I can tell, container.php, includes/Container.php and includes/ServiceWiring.php (which seem like the files that would be most relevant to pimple, IIUC) haven’t been touched since REL1_42 was branched, so I have no idea where this comes from :/

hashar removed a project: Continuous-Integration-Config.May 17 2024, 8:11 AM

Hm, on another check, it failed in both php81 and php82 in the same way.

I wonder if this could be related to T364273: Wikibase breaks schema update with "Class 'Onoi\MessageReporter\NullMessageReporter' not found", where @Tgr also saw autoloader errors during the update / install step… (but Flow has had load_composer_autoloader since 2016, so the solution we tried there doesn’t apply here.)

Well, this is curious. In this Gerrit change, I hacked around Flow enough that it made it through the install step without crashing due to missing Pimple… only for it to fail with very similar errors during the actual unit tests:

There were 6 errors:

1) Flow\Tests\Data\ManagerGroupTest::testClearOnlyCallsRequestedManagers
Error: Class "Pimple\Container" not found

/workspace/src/extensions/Flow/includes/Container.php:5
/workspace/src/includes/AutoLoader.php:223
/workspace/src/extensions/Flow/tests/phpunit/unit/Data/ManagerGroupTest.php:16
/workspace/src/extensions/Flow/tests/phpunit/unit/Data/ManagerGroupTest.php:33
/workspace/src/tests/phpunit/MediaWikiUnitTestCase.php:127

2) Flow\Tests\Data\ManagerGroupTest::testClearCallsNoManagersWhenUnused
Error: Class "Pimple\Container" not found

/workspace/src/extensions/Flow/includes/Container.php:5
/workspace/src/includes/AutoLoader.php:223
/workspace/src/extensions/Flow/tests/phpunit/unit/Data/ManagerGroupTest.php:16
/workspace/src/extensions/Flow/tests/phpunit/unit/Data/ManagerGroupTest.php:44
/workspace/src/tests/phpunit/MediaWikiUnitTestCase.php:127

3) Flow\Tests\Data\ManagerGroupTest::testCachePurgeCallsAppropriateManager
Error: Class "Pimple\Container" not found

/workspace/src/extensions/Flow/includes/Container.php:5
/workspace/src/includes/AutoLoader.php:223
/workspace/src/extensions/Flow/tests/phpunit/unit/Data/ManagerGroupTest.php:16
/workspace/src/extensions/Flow/tests/phpunit/unit/Data/ManagerGroupTest.php:56
/workspace/src/tests/phpunit/MediaWikiUnitTestCase.php:127

4) WikimediaEvents\Tests\Unit\WikimediaEventsCountryCodeLookupTest::testGetCountryCode with data set "Country code comes from GeoIP2 Reader class" ('DE', 'DE', '127.0.0.1')
PHPUnit\Framework\MockObject\UnknownTypeException: Class or interface "GeoIp2\Database\Reader" does not exist

/workspace/src/extensions/WikimediaEvents/tests/phpunit/unit/WikimediaEventsCountryCodeLookupTest.php:50
/workspace/src/tests/phpunit/MediaWikiUnitTestCase.php:127

5) WikimediaEvents\Tests\Unit\WikimediaEventsCountryCodeLookupTest::testGetCountryCode with data set "Invalid IP" (null, null, 'foo')
PHPUnit\Framework\MockObject\UnknownTypeException: Class or interface "GeoIp2\Database\Reader" does not exist

/workspace/src/extensions/WikimediaEvents/tests/phpunit/unit/WikimediaEventsCountryCodeLookupTest.php:50
/workspace/src/tests/phpunit/MediaWikiUnitTestCase.php:127

6) WikimediaEvents\Tests\Unit\WikimediaEventsCountryCodeLookupTest::testGetCountryCode with data set "Return unknown instead of null if isoCode is null on country record" (null, null, '127.0.0.1')
PHPUnit\Framework\MockObject\UnknownTypeException: Class or interface "GeoIp2\Database\Reader" does not exist

/workspace/src/extensions/WikimediaEvents/tests/phpunit/unit/WikimediaEventsCountryCodeLookupTest.php:50
/workspace/src/tests/phpunit/MediaWikiUnitTestCase.php:127

Notice that the last three errors aren’t about Pimple, but about some other class (GeoIp2\Database\Reader). So this isn’t just limited to the installer step, as I thought, and it isn’t limited to Pimple either (that part is less surprising, I suppose). Somehow Flow’s composer dependencies are just generally missing?

Oh, now I think I’ve seen this before.

“composer test in /workspace/src/extensions/Flow” installs Pimple and other Flow dependencies
“Revert to git clean -xqdf in /workspace/src/extensions/Flow” wipes the installed dependencies again
“Install composer dev-requires for vendor.git” installs some dependencies again, but not including Pimple and other Flow-specific dependencies (because Flow isn’t bundled and so it’s not included in the REL1_* branches of mediawiki/vendor.git)

Now if only I could remember where this happened before, and what the solution was last time…

Ah, it was in T362412: Wikibase CI jobs wikibase-repo and wikibase-client broken on REL1_42 (specifically T362412#9710244), and we didn’t fully solve that task yet :/

I suppose one solution / workaround would be to disable the quibble-vendor-* builds on release branches for Flow and related extensions, and only keep the quibble-composer-* builds?

In T364986#9808090, @Lucas_Werkmeister_WMDE wrote:

Oh, now I think I’ve seen this before.

“composer test in /workspace/src/extensions/Flow” installs Pimple and other Flow dependencies

“Revert to git clean -xqdf in /workspace/src/extensions/Flow” wipes the installed dependencies again

“Install composer dev-requires for vendor.git” installs some dependencies again, but not including Pimple and other Flow-specific dependencies (because Flow isn’t bundled and so it’s not included in the REL1_* branches of mediawiki/vendor.git)

Now if only I could remember where this happened before, and what the solution was last time…

Where do you see that? E.g. https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php81-noselenium/4695/console doesn't seem to match: Pimple doesn't seem to get installed in the first place (at least there's no mention of it in the composer output, in either pass), and the git clean command isn't run in the Flow directory.

(It's not clear to me why Flow even gets installed in a test for the OAuth extension, but maybe there's some crazy dependency chain between the two.)

Reedy mentioned this in T365254: Build extension dependancy tree tool for zuul dependancies.May 17 2024, 2:21 PM

In T364986#9808841, @Tgr wrote:

Where do you see that? E.g. https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php81-noselenium/4695/console doesn't seem to match: Pimple doesn't seem to get installed in the first place (at least there's no mention of it in the composer output, in either pass), and the git clean command isn't run in the Flow directory.

I saw that in the Flow builds on this WIP change, e.g. this build. In OATHAuth builds, the situation is a bit different (the first step installs OATHAuth’s dependencies, not Flow’s, so it’s missing pimple; the second step cleans OATHAuth, not Flow), but I think the ultimate cause of the issue is the same (mediawiki/vendor.git doesn’t include Flow’s dependencies).

(It's not clear to me why Flow even gets installed in a test for the OAuth extension, but maybe there's some crazy dependency chain between the two.)

OATHAuth → CheckUser → CentralAuth → AbuseFilter → Echo → MobileFrontend → VisualEditor → DiscussionTools → Thanks → Flow, I think. (Maybe there’s a shorter path too, I kind of lost track.)

Hm, but my theory doesn’t explain the original failure that Reedy saw, which is in a composer build, not a vendor build. So maybe there’s also an issue with the autoloader during the installer, even when pimple was installed via composer (and not cleaned)?

I think this task is a straight duplicate of T363639. Let's merge?

I guess so… I hadn’t seen this part in Reedy’s CI output before:

[25.1MiB/6.94s] Dependency resolution completed in 0.002 seconds
[25.2MiB/6.94s] Your requirements could not be resolved to an installable set of packages.
[25.2MiB/6.95s] 
  Problem 1
    - symfony/process[v4.0.0, v4.0.1, v4.0.2, v4.0.3, v4.0.4, v4.0.5, v4.0.6, v4.0.7, v4.0.8, v4.0.9, v4.0.10, v4.0.11, v4.0.12, v4.0.13, v4.0.14, v4.0.15, v4.1.0, v4.1.1, v4.1.2, v4.1.3, v4.1.4, v4.1.5, v4.1.6, v4.1.7, v4.1.8, v4.1.9, v4.1.10, v4.1.11, v4.1.12, v4.2.0, v4.2.1, v4.2.2, v4.2.3, v4.2.4, v4.2.5, v4.2.6, v4.2.7, v4.2.8, v4.2.9, v4.2.10, v4.2.11, v4.2.12, v4.3.0, v4.3.1, v4.3.2, v4.3.3, v4.3.4, v4.3.5, v4.3.6, v4.3.7, v4.3.8, v4.3.9, v4.3.10, v4.3.11, v4.4.0, v4.4.1, v4.4.2, v4.4.3, v4.4.4, v4.4.5, v4.4.6, v4.4.7, v4.4.8, v4.4.9, v4.4.10] require php ^7.1.3 -> your php version (8.2.18) does not satisfy that requirement.
    - symfony/process[v5.0.0, v5.0.1, v5.0.2, v5.0.3, v5.0.4, v5.0.5, v5.0.6, v5.0.7, v5.0.8] require php ^7.2.5 -> your php version (8.2.18) does not satisfy that requirement.
    - Root composer.json requires web-auth/webauthn-lib ~3.3.12 -> satisfiable by web-auth/webauthn-lib[v3.3.12].
    - web-auth/webauthn-lib v3.3.12 requires symfony/process ^3.0|^4.0|^5.0 -> satisfiable by symfony/process[v3.0.0, v3.0.1, v3.0.2, v3.0.3, v3.0.4, v3.0.5, v3.0.6, v3.0.7, v3.0.8, v3.0.9, v3.1.0, v3.1.1, v3.1.2, v3.1.3, v3.1.4, v3.1.5, v3.1.6, v3.1.7, v3.1.8, v3.1.9, v3.1.10, v3.2.0, v3.2.1, v3.2.2, v3.2.3, v3.2.4, v3.2.5, v3.2.6, v3.2.7, v3.2.8, v3.2.9, v3.2.10, v3.2.11, v3.2.12, v3.2.13, v3.2.14, v3.3.0, v3.3.1, v3.3.2, v3.3.3, v3.3.4, v3.3.5, v3.3.6, v3.3.7, v3.3.8, v3.3.9, v3.3.10, v3.3.11, v3.3.12, v3.3.13, v3.3.14, v3.3.15, v3.3.16, v3.3.17, v3.3.18, v3.4.0, v3.4.1, v3.4.2, v3.4.3, v3.4.4, v3.4.5, v3.4.6, v3.4.7, v3.4.8, v3.4.9, v3.4.10, v3.4.11, v3.4.12, v3.4.13, v3.4.14, v3.4.15, v3.4.16, v3.4.17, v3.4.18, v3.4.19, v3.4.20, v3.4.21, v3.4.22, v3.4.23, v3.4.24, v3.4.25, v3.4.26, v3.4.27, v3.4.28, v3.4.29, v3.4.30, v3.4.31, v3.4.32, v3.4.33, v3.4.34, v3.4.35, v3.4.36, v3.4.37, v3.4.38, v3.4.39, v3.4.40, v3.4.41, v3.4.42, v3.4.43, v3.4.44, v3.4.45, v3.4.46, v3.4.47, v4.0.0, v4.0.1, v4.0.2, v4.0.3, v4.0.4, v4.0.5, v4.0.6, v4.0.7, v4.0.8, v4.0.9, v4.0.10, v4.0.11, v4.0.12, v4.0.13, v4.0.14, v4.0.15, v4.1.0, v4.1.1, v4.1.2, v4.1.3, v4.1.4, v4.1.5, v4.1.6, v4.1.7, v4.1.8, v4.1.9, v4.1.10, v4.1.11, v4.1.12, v4.2.0, v4.2.1, v4.2.2, v4.2.3, v4.2.4, v4.2.5, v4.2.6, v4.2.7, v4.2.8, v4.2.9, v4.2.10, v4.2.11, v4.2.12, v4.3.0, v4.3.1, v4.3.2, v4.3.3, v4.3.4, v4.3.5, v4.3.6, v4.3.7, v4.3.8, v4.3.9, v4.3.10, v4.3.11, v4.4.0, v4.4.1, v4.4.2, v4.4.3, v4.4.4, v4.4.5, v4.4.6, v4.4.7, v4.4.8, v4.4.9, v4.4.10, v4.4.11, v4.4.12, v4.4.13, v4.4.14, v4.4.15, v4.4.16, v4.4.17, v4.4.18, v4.4.19, v4.4.20, v4.4.22, v4.4.25, v4.4.26, v4.4.27, v4.4.30, v4.4.34, v4.4.35, v4.4.36, v4.4.37, v4.4.40, v4.4.41, v4.4.44, v5.0.0, v5.0.1, v5.0.2, v5.0.3, v5.0.4, v5.0.5, v5.0.6, v5.0.7, v5.0.8, v5.0.9, v5.0.10, v5.0.11, v5.1.0, v5.1.1, v5.1.2, v5.1.3, v5.1.4, v5.1.5, v5.1.6, v5.1.7, v5.1.8, v5.1.9, v5.1.10, v5.1.11, v5.2.0, v5.2.1, v5.2.2, v5.2.3, v5.2.4, v5.2.7, v5.2.10, v5.2.11, v5.2.12, v5.3.0, v5.3.2, v5.3.4, v5.3.7, v5.3.11, v5.3.12, v5.3.13, v5.3.14, v5.4.0, v5.4.2, v5.4.3, v5.4.5, v5.4.7, v5.4.8, v5.4.11, v5.4.19, v5.4.21, v5.4.22, v5.4.23, v5.4.24, v5.4.26, v5.4.28, v5.4.34, v5.4.35, v5.4.36, v5.4.39].
    - symfony/console v7.0.7 conflicts with symfony/process v5.4.39.
    - symfony/console v7.0.7 conflicts with symfony/process v5.3.2.
    - symfony/console v7.0.7 conflicts with symfony/process v5.0.11.
    - symfony/console v7.0.7 conflicts with symfony/process v4.4.44.
    - symfony/console v7.0.7 conflicts with symfony/process v4.4.26.
    - symfony/console v7.0.7 conflicts with symfony/process v3.4.47.
    - symfony/console v7.0.7 conflicts with symfony/process v3.3.6.
    - psy/psysh v0.12.3 requires symfony/console ^7.0 || ^6.0 || ^5.0 || ^4.0 || ^3.4 -> satisfiable by symfony/console[v7.0.7].
    - psy/psysh is locked to version v0.12.3 and an update of this package was not requested.

[23.3MiB/6.95s] 
Update to apply merge settings failed, reverting ./composer.lock to its original content.

And just for the record here, James also pointed out that all the tests I did (see above) were basically pointless, because:

"Check php" is only intended to be useful on master branches; it runs the gate-but-not-in-test PHP jobs, which in this case are the -vendor- rather than -composer- variants, and so not a good check of whether this will work.

Jdforrester-WMF closed this task as a duplicate of T363639: web-auth/webauthn-lib must be upgraded to 4+ for PHP 8.2+ support.May 21 2024, 3:31 PM

In T364986#9812728, @Jdforrester-WMF wrote:

I think this task is a straight duplicate of T363639. Let's merge?

I'm guessing that's not actually the right task... As that's webauthn lib upgrades

This is obviously still broken on 1.43 too

Oh, I see now. Sigh.

Change #1085943 had a related patch set uploaded (by Reedy; author: Reedy):

[integration/config@master] zuul: Remove WebAuthn from OATHAuth on REL1_XX

https://gerrit.wikimedia.org/r/1085943

gerritbot added a project: Patch-For-Review.Nov 2 2024, 12:57 PM

Change #1085943 merged by jenkins-bot: