Page MenuHomePhabricator

Have unit tests of all wmf deployed extensions pass when installed together, in both PHP-Zend and HHVM (tracking)
Open, NormalPublic

Description

When installing all the MediaWiki extensions currently deployed on the Wikimedia cluster with a dummy LocalSettings.php, we end up with a lot of failures, bad interactions between extensions or core tests broken via hooks. Some extensions even end up throwing fatal errors.

To help migrate to HHVM, we would need to be able to run the unit tests of core and of all the extensions and have them pass.

Ori has worked on this a bit for HHVM and already reported a fair amount of bugs.

Antoine has been working on an experimental Jenkins Job that run all wmf extensions tests.

Lets join the effort, fill bug and track them there. Then reach out to the other developers and get everything fixed.


Version: wmf-deployment
Severity: normal

Details

Reference
bz67216

Related Objects

StatusAssignedTask
Resolvedhashar
OpenNone
ResolvedNone
DeclinedNone
ResolvedNone
ResolvedNone
ResolvedNone
ResolvedNone
ResolvedNone
DeclinedNone
InvalidNone
ResolvedNone
OpenNone
InvalidTpt
ResolvedNone
ResolvedNone
ResolvedNone
OpenNone
ResolvedNone
ResolvedNone
OpenNone
OpenNone
ResolvedNone
OpenNone
ResolvedEBernhardson
Invaliddemon
ResolvedNone
OpenNone
DeclinedNone
ResolvedNone
OpenNone
ResolvedNone
Resolvedhashar
ResolvedReedy
OpenNone
ResolvedNone
ResolvedNone
ResolvedJdforrester-WMF
ResolvedNikerabbit
ResolvedNone
OpenNone
Resolvedhashar
Resolvedhashar
Resolvedhashar
ResolvedLegoktm
Resolvedhashar
Resolvedhashar
DeclinedNone
DeclinedNone
ResolvedNone
Resolvedhashar
Resolvedhashar
ResolvedJoe
ResolvedJoe
Resolvedhashar
Resolvedhashar
ResolvedEBernhardson
ResolvedQChris
ResolvedLegoktm
Resolvedhashar
Duplicate Mattflaschen-WMF
InvalidRyasmeen
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
DuplicateNone
Resolvedbd808
DuplicateNone
DeclinedNone
Declinedhashar
Resolvedhashar
DeclinedNone
Declinedchasemp
DeclinedNone
Resolvedcoren
Invalidhashar
Resolvedhashar
Declinedhashar
ResolvedCmjohnson
ResolvedCmjohnson
Resolvedhashar
ResolvedKrinkle
ResolvedKrinkle
ResolvedKrinkle
Resolvedhashar
ResolvedKrinkle
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
ResolvedAndrew
ResolvedKrinkle
Resolvedhashar
ResolvedKrinkle
ResolvedPaladox
ResolvedLegoktm
ResolvedLegoktm
ResolvedLegoktm
ResolvedLegoktm
ResolvedPaladox
ResolvedLegoktm
ResolvedLegoktm
Resolvedhashar
Resolvedhashar
Resolved yuvipanda
OpenNone

Event Timeline

bzimport raised the priority of this task from to High.
bzimport set Reference to bz67216.
bzimport added a subscriber: Unknown Object (MLST).
hashar created this task.Jun 27 2014, 9:35 PM

Add bugs Ori and I filled.

Relabel per discussion.

A few more extension that can pass the installer / fail tests.

Thanks a bunch for working on this. Those who shouted for help on wikitech-l and have been unable for years to run tests on their own environment / with their own configuration will be enthusiastic.

I am merely feeling bugs and we will attempt to fix as many possible. But we could use some help :-) Please spread the word to wikitech-l! New bugs would be specially appreciated.

The aim is to have core + all wmf extensions to pass the PHPUnit test suite.

ori added a comment.Jul 3 2014, 6:59 AM

This is awesome :) Thanks for pushing this, Antoine.

hashar added a comment.Jul 7 2014, 8:57 PM

Status as of today.

I have commented out the following extensions for various reasons mentioned in the commit message https://git.wikimedia.org/commit/integration%2Fjenkins-job-builder-config/50d2cda7ffca7655889a48b25fccd3ce15977ff6

#mediawiki/extensions/CentralAuth
#mediawiki/extensions/Math
#mediawiki/extensions/OAI
#mediawiki/extensions/Oversight
#mediawiki/extensions/RandomRootPage
#mediawiki/extensions/SemanticForms
#mediawiki/extensions/SemanticMediaWiki
#mediawiki/extensions/SemanticResultFormats
#mediawiki/extensions/Translate
#mediawiki/extensions/Validator
#mediawiki/extensions/ZeroRatedMobileAccess
#mediawiki/extensions/timeline

There are bug filled for most of them.

The complete list from the Jenkins configuration:

https://git.wikimedia.org/blob/integration%2Fjenkins-job-builder-config/50d2cda7ffca7655889a48b25fccd3ce15977ff6/mediawiki-extensions.yaml#L147

A test report can be find at: https://integration.wikimedia.org/ci/job/mediawiki-core-extensions-integration/68/testReport/

Almost all of the failures are parser tests which hardcode expected HTML output. But since some extensions register hooks that further adjust the HTML output, that does not play well. Bug 67302 - ""MultimediaViewer breaks Cite extension parser tests"" has some more details.

Created attachment 16044
php5 phpunit.php

I run php5 phpunit.php after enabling all gerrit extension having unit tests and almost all MediaWiki-Vagrant roles (http://paste.debian.net/111811/ minus roles globalcssjs math mathsearch cirrussearch) and there's quite some output to process...

Attached:

(In reply to Nemo from comment #8)

Created attachment 16044 [details]
php5 phpunit.php

I run php5 phpunit.php after enabling all gerrit extension having unit tests
and almost all MediaWiki-Vagrant roles (http://paste.debian.net/111811/
minus roles globalcssjs math mathsearch cirrussearch) and there's quite some
output to process...

I am going to have a Jenkins job triggered for all extensions. Gotta polish up some details but I have a working base to make it happen soon ™

Attached:

The extensions are slowly being added to the shared jobs mediawiki-extensions-hhvm and mediawiki-extensions-zend

An example is https://gerrit.wikimedia.org/r/#/c/185167/ which adds:

  • AbuseFilter
  • CheckUser
  • ConfirmEdit
  • EventLogging
  • SpamBlacklist

Moreover, the hhvm flavor of the job is available on ALL extensions in the experimental pipeline. If one want to verify whether its extension is a good candidate for inclusion, simply comment in Gerrit 'check experimental'. The extension will be injected in the list of cloned extensions and the tests will be run for the resulting set.

See also my recent announcement on wikitech-l.

hashar removed hashar as the assignee of this task.

Not actively working on it. We surely need more extensions to be added in.

I still have unreviewed patches from my latest testing sprint almost an year ago... once those are all closed I'll reclone everything and repeat.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 31 2015, 11:29 PM
hashar lowered the priority of this task from High to Normal.Oct 6 2015, 11:27 AM
hashar set Security to None.
Krinkle removed a subscriber: Krinkle.Feb 23 2016, 4:49 PM