Page MenuHomePhabricator
Create Task
Maniphest T255792

Quibble runs core:unit tests twice!
Open, Stalled, HighPublic
Actions

Description

During a full run of Quibble, I noticed PHPUnit alerts twice about the slow test Argon2PasswordTest:testPartialConfig. Turns out it run twice, once as part of the core:unit suite and another time as part of the default phpunit.php suite. The commands Quibble runs are:

core:unit
phpunit --colors=always --testsuite=core:unit,extensions:unit,skins:unit '--exclude-group' 'Broken,ParserFuzz,Stub'

The suite is defined as:

/phpunit.xml.dist
<phpunit>
    <testsuites>
        <testsuite name="core:unit">
            <directory>tests/phpunit/unit</directory>
        </testsuite>
</phpunit>

The second command is:

1
php tests/phpunit/phpunit.php --exclude-group Broken,ParserFuzz,Stub,Database,Standalone --log-junit /workspace/log/junit-dbless.xml

Which I guess crawls everything and thus also discover tests under tests/phpunit/unit. I guess we should exclude them?

Related Objects
Search...

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusSubtypeAssignedTask
· · ·
StalledNoneT255792 Quibble runs core:unit tests twice!
OpenNoneT328919 Upgrade to PHPUnit 10
· · ·

Event Timeline

hashar created this task.Jun 18 2020, 4:40 PM
Jdforrester-WMF subscribed.Jun 18 2020, 5:35 PM

Oops.

Jdforrester-WMF triaged this task as High priority.Jun 18 2020, 5:35 PM
hashar added a comment.Jun 19 2020, 8:34 AM

It is not the end of the world though core:unit takes just a few seconds to execute ;-]

kostajh added a comment.Nov 24 2021, 2:08 PM

Is this still a problem, or a problem worth doing something about?

kostajh moved this task from Backlog to Watching on the User-kostajh board.Nov 24 2021, 2:08 PM
Daimona subscribed.Sep 5 2023, 1:24 AM
In T255792#7526593, @kostajh wrote:

Is this still a problem, or a problem worth doing something about?

This is still a problem. As for the second question: I don't think it's worth doing anything about this now, because PHPUnit 9 does not provide a CLI options to exclude a certain testsuite, and any other way we could come up with to exclude unit tests would be either a huge hack or a huge change to be implemented in all tests. OTOH, PHPUnit 10 does provide that option! It's --exclude-testsuite, see https://github.com/sebastianbergmann/phpunit/pull/4657. Once we upgrade, it should be really simple to add this option to the integration suite invocation. The only thing that makes this non-trivial is, I think, the fact that we'll probably have to write some BC logic so that the option is only passed if the repo uses PHPUnit >= 10.

Daimona added a subtask: T328919: Upgrade to PHPUnit 10.Sep 5 2023, 1:24 AM
Novem_Linguae subscribed.Sep 7 2023, 10:38 AM

BC logic

BC = beta cluster?

kostajh added a comment.Sep 7 2023, 10:47 AM
In T255792#9148823, @Novem_Linguae wrote:

BC logic

BC = beta cluster?

I beileve it's backwards-compatibility, in this context.

Daimona added a comment.Sep 7 2023, 11:44 AM
In T255792#9148847, @kostajh wrote:
In T255792#9148823, @Novem_Linguae wrote:

BC logic

BC = beta cluster?

I beileve it's backwards-compatibility, in this context.

Yes, sorry, it is backwards compatibility indeed.

Daimona changed the task status from Open to Stalled.Sep 7 2023, 4:48 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL