Page MenuHomePhabricator

ContentTranslation phpunit run very slow due to inclusion of Scribunto and Wikibase
Closed, ResolvedPublic

Description

I'm not sure why this extension is loading these two seemingly unrelated extensions.

Whatever the case, this is slowing down the mwext-testextension-zend job from 1-2 minutes to 9-12 minutes. And sometimes causing it to timeout due to database and filesystem contention (because limited resources in Labs, which is its own problem that speaks to the isolation of the unit tests and possibly utility of something like php vfsStream that could make it faster).

Event Timeline

Krinkle raised the priority of this task from to Needs Triage.
Krinkle updated the task description. (Show Details)
Krinkle added a project: ContentTranslation.
Krinkle subscribed.
Amire80 triaged this task as Medium priority.Dec 16 2015, 12:46 PM
Amire80 set Security to None.

I'm not sure why this extension is loading these two seemingly unrelated extensions.

Wikibase is loaded because ContentTranslation automatically links each page that it creates to the page from which it was translated by adding a Wikidata sitelink, and it uses the Wikibase JS library for that. See modules/publish/ext.cx.wikibase.link.js under the ContentTranslation repo. Jenkins tests were failing until we added it as a dependency.

And I guess that Scribunto is loaded by Wikibase; AFAIK, ContentTranslation doesn't use Scribunto directly.

Adding @hoo, who is maintaining this Wikibase library, and @KartikMistry, who takes care of Jenkins issues that affect the Language team.

We do inject extensions as dependencies, and they all end up injecting their tests in the UnitTestsList hook so we end up running everything.

The Jenkins jobs invokes cd tests/phpunit && php phpunit.php --testsuite extensions.

Which calls tests/phpunit/suites/ExtensionsTestSuite.php which invokes the hook`UnitTestsList`. We also have the structure tests that look at the RL / Less / autoloader and are populated by all extensions, they are fast enough that we should not bother.

Maybe we can introduce a new suite that tests a single suite or figure out a way to filter out unwanted extensions in ExtensionsTestSuite.php.

Same work would need to happen for the parsertestsuite.php file.

Including those tests is also valuable since extensions might break each other. I'd recommend focussing in this task on making either ContentTranslation not hard-depend on these, or making the Wikibase/Scribunto tests faster.

At least for Scribunto that should be doable now with Quibble 0.0.41. There is a new Standalone PHPUnit group that let us skip running those PhpUnit tests when they come from a dependency (T225068)