The ResourcesTest.php class is used for making sure registered resources are sensible.# Background
Test platform main goal this quarter is to reduce the developer feedback time for the core CI below 10 minutes [[ https://phabricator.wikimedia.org/T420590 | T420590 ]].
We are almost there, We now have all the jobs running below <10 minutes except `quibble-with-gated-extensions-vendor-mysql-php83`.
# Problem
In `quibble-with-gated-extensions-vendor-mysql-php83` job, we have noticed that the ResourcesTest class takes
the most time to run by far, compared to other classes. Checking the [[ https://phpunit-results-cache.toolforge.org/results/quibble-with-gated-extensions-vendor-mysql-php83/37301 | cache results ]] of a sample run of the gated extensions job, the top 3 slowest classes are:
ResourcesTest ~ 420 sec
ParserTest_Cite_1_citeParserTests_Test ~98 sec
ParserIntegrationTest ~50 sec
When the PHPUnit test splitter distributes the classes to the available split groups, ResourcesTest gets assigned
to one split group. In this [[ https://integration.wikimedia.org/ci/job/quibble-with-gated-extensions-vendor-mysql-php83/37301/ | sample case ]] case, the database section timings are like this:
split_group_5 505.297s
This class has the longest runningsplit_group_1 215.208s
time in both quibble-with-gated-extensions-vendor-mysql-php83 job and quibble-for-mediawiki-core-vendor-mysql-php83 jobs. split_group_0 211.558s
In core's PHPUnit test run the class takes ~ [[ https://phpunit-results-cache.toolforge.org/results/quibble-vendor-mysql-php83 | 40 - 90 ]] seconds while in the gated extensions the job takes ~ [[ https://phpunit-results-cache.toolforge.org/results/quibble-with-gated-extensions-vendor-mysql-php83 | 290 - 375 ]] seconds as shown is the sample screenshot. split_group_2 200.622s
{F78227926} split_group_3 193.049s
split_group_4 141.554s
split_group_6 17.441s
split_group_7 1.395s
If we can optimize this test,The ResourcesTest class is assigned to split group 5, together with 6 other classes. the gated extension's job run time will easily go belowThe below timings indicate how long the sub 10 minute mark.classes took:
Acceptance Criteria ResourcesTest 470.568s
ApiStructureTest 20.383s
ContentHandlerFunctionalTest 4.149s
PerformanceBudgetTest 2.336s
ApiPrefixUniquenessTest 0.231s
DatabaseIntegrationTest 0.046s
DumpableObjectsTest 0.032s
[x] Investigate a way we can optimize this class furtherBecause the split groups run in parallel, the time taken by the database PHPUnit stage is decided by the slowest split group.
[x] In case of successThe job has to wait for all split groups to finish before it can move on. This means the group that gets ResourcesTest is
disproportionally slower than the other groups. Even though the other split groups are done, the job still has to wait for
the ResourcesTest group to finish.
# Exploration
We have tried a few ideas to make this class run faster, but with our limited knowledge of the PHPunit tests and ResourceLoader, create a patch and upload to gerrit.we
[] PHPUnit gated-extension job consistently run below the 10 minute mark.have not been able to land a solution that does not introduce a maintenance burden to allow continuity. Examples of what we
have tried: ([[ https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1279380 | 1279380 ]], [[ https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1284007 | 1284007 ]], [[ https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1296081 | 1296081 ]])
# Solution
If `ResourcesTest` can run faster, it would help us meet the goal we need. We are requesting input from anyone familiar
with `ResourcesTest` and the ResourceLoader module. We want to reduce ResourcesTest runtime while keeping its guarantees.