Page MenuHomePhabricator

Quibble doesn't run phpunit Database tests if selenium fails
Closed, DeclinedPublic

Description

It would be great if either by default or with configuration quibble could be made to continue if it fails on a earlier suite rather than exiting.

For example: https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2766/consoleFull has a selenium failure and so the phpunit @Database tests are never run.

Event Timeline

That is due to 541c07bf88c8092e72a544bffeed9c1e65c891be and is somehow intentional.

For mediawiki/core the Database PHPUnit tests are quite slow and thus I delay them after the Selenium tests (which are relatively faster).

For extensions/skins, before the commit above, there was no distinction and we ran all PHPUnit tests in a single run. The commit aligned extensions/skins with mediawiki/core. I think I did that when several extensions are tested together which causes the Database PHPUnit group to be slow as well. So the intent is to fail faster.

This is intentional for now. The PHPUnit marked with @group Database are usually way slower than the Selenium tests and the intent is to fail fast. That being said we can revisit later.

Note: you can skp Selenium tests with: quibble --skip selenium