Migrate Parser and extension tests away from deprecated PHPUnit TestSuite subclassing
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	Daimona
	Sep 1 2023, 10:38 PM

Description

We currently use "dynamic" test suites for parser tests, as well as extension/skin tests (including unit tests after r937559). On a theoretical level, this is not ideal because the list of tests to execute cannot be determined immediately, but is generated dynamically after PHPUnit has finished its setup. In practice, these dynamic test suites make it impossible to use paratest (T50217), which is unable to split the tests across multiple processes. On top of that extending TestSuite is no longer possible in PHPUnit 10 (T328919).

Therefore, we need to find another way to generate these test lists dynamically.

Details

	Subject	Repo	Branch	Lines +/-
	[POC] Introduce a CLI script to generate PHPUnit config	mediawiki/core	master	+114 -82

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T225730 Reduce runtime of MW shared gate Jenkins jobs to 5 min
Open	pmiazga	T50217 Speed up MediaWiki PHPUnit build by running integration tests in parallel
Stalled	None	T255792 Quibble runs core:unit tests twice!
Open	None	T328919 Upgrade to PHPUnit 10
Open	None	T345481 Migrate Parser and extension tests away from deprecated PHPUnit TestSuite subclassing

Event Timeline

Daimona created this task.Sep 1 2023, 10:38 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 1 2023, 10:38 PM

Daimona mentioned this in T328919: Upgrade to PHPUnit 10.Sep 1 2023, 10:39 PM

Daimona added parent tasks: T50217: Speed up MediaWiki PHPUnit build by running integration tests in parallel, T328919: Upgrade to PHPUnit 10.

Novem_Linguae subscribed.Sep 1 2023, 10:44 PM

Daimona mentioned this in T347407: SonarCloud not picking up unit test coverage on master and patch branches of extensions and core.Oct 23 2023, 8:15 PM

Krinkle renamed this task from Stop extending PHPUnit's TestSuite class to Migrate Parser tests away from deprecated PHPUnit TestSuite subclassing.Dec 14 2023, 10:59 PM

Krinkle moved this task from Inbox to PHPUnit on the MediaWiki-Core-Tests board.

Krinkle added projects: MediaWiki-Parser, Content-Transform-Team.

MSantos edited projects, added Content-Transform-Team-WIP, Essential-Work; removed Content-Transform-Team.Dec 21 2023, 3:09 PM

I've updated the task title to clarify that, unfortunately, it's not only parser tests that subclass TestSuite, but also extension/skin tests...

Daimona mentioned this in T358097: composer phpunit hangs forever on WSL when running inside docker.Feb 21 2024, 12:48 PM

Change 1005636 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/core@master] [POC] Introduce a CLI script to generate PHPUnit config

https://gerrit.wikimedia.org/r/1005636

gerritbot added a project: Patch-For-Review.Feb 22 2024, 12:26 AM

My current proposal is to introduce a new CLI script that loads LocalSettings, and then uses the list of loaded extensions to generate a local phpunit.xml file, which can then be used to run tests normally. I've implemented this in the above proof of concept. CC'ing @Krinkle, @pmiazga, @daniel since we discussed this earlier today, in case you'd like to take a look and provide feedback. If not, apologies and feel free to unsubscribe :)

I also realized that there are more dynamic suites in extensions. These also need to be migrated. If we choose the approach above (script to autogenerate PHPUnit config), we may allow extensions to hook into the script and add/change suites; however, this adds complexity. It would be better to use other solutions in extensions, if possible.

Daimona mentioned this in T358394: Stop using PHPUnit's TestSuite in Scribunto.Feb 23 2024, 9:55 PM

kostajh subscribed.Feb 26 2024, 8:28 AM

Daimona mentioned this in T50217: Speed up MediaWiki PHPUnit build by running integration tests in parallel.Feb 26 2024, 12:53 PM

pwangai subscribed.Feb 27 2024, 7:49 PM

Daimona added a project: User-Daimona.Mar 3 2024, 5:18 PM

Daimona moved this task from Backlog to Doing on the User-Daimona board.