Change Details

WARNING: THIS IS A DRAFT TASK MEANT TO BECOME A SAGA OF EPICS Patch RelatedArticles Zuul set : EXT_NAME=RelatedArticles Zuul add ['BetaFeatures', 'Cards', 'MobileFrontend'] + ['Echo', 'VisualEditor', 'ZeroBanner', 'MobileApp'] + whatever else 15 extensions cd MW_INSTALL_PATH npm run selenium # Run tests solely for the extension that sent the patch wdio --spec extensions/${EXT_NAME}/tests/selenium/**/*.js # MediaWiki sub set of tests: wdio --spec tests/selenium/smoke/**/*.js # Smokes from other extensions wdio --spec extensions/${EXT_NAME}/tests/selenium/smoke/**/*.js ---- That came up on a patch for Echo https://gerrit.wikimedia.org/r/#/c/313221/ which added a test requiring Flow behavior. I commented about how we could inject Flow as a dependency but that means all Flow tests will be run as well. The same split has to be done for PHPUnit as well ---- From a similar documentation back in December 14th 2016 related to PHPUnit extensions. https://docs.google.com/drawings/d/1ebVAFqM80Me7HCiLV-JdAkNCh3unLCGsuNqRcgiGtEI

WARNING: THIS IS A DRAFT TASK MEANT TO BECOME A SAGA OF EPICS ---- From a conversation with Zeljko related to webdriver.io testing Given a patch to the the mediawiki extension RelatedArticles. Zuul injects a parameter `EXT_NAME=RelatedArticles` Zuul then adds direct extensions dependencies: ['BetaFeatures', 'Cards', 'MobileFrontend'] which each bring more to it (eg: MobileFrontend also depends on ['Echo', 'VisualEditor', 'ZeroBanner', 'MobileApp'] The Webdriver.io jenkins job then: ``` cd MW_INSTALL_PATH npm run selenium ``` That ends up running every available tests because the wdio config has a glob pattern `extensions/*/tests/selenium/**/*.js`. We would need to use subsets for example: Run tests solely for the extension that sent the patch wdio --spec extensions/${EXT_NAME}/tests/selenium/**/*.js MediaWiki sub set of tests: wdio --spec tests/selenium/smoke/**/*.js Smokes from other extensions wdio --spec extensions/${EXT_NAME}/tests/selenium/smoke/**/*.js ---- That came up also on a patch for Echo https://gerrit.wikimedia.org/r/#/c/313221/ which added a test requiring Flow behavior. I commented about how we could inject Flow as a dependency but that means all Flow tests will be run as well. The same split has to be done for PHPUnit as well ---- Yurik and Hashar had a discussion on Dec. 14th 2016 about JSonConfig also running all the Scribunto tests, which is a recurring issue affecting multiple extensions (eg: T126670). IRC logs starting at 09:00 https://wm-bot.wmflabs.org/logs/%23wikimedia-releng/20161214.txt A lame drawing showing how the PHPUnit tests could be split: https://docs.google.com/drawings/d/1ebVAFqM80Me7HCiLV-JdAkNCh3unLCGsuNqRcgiGtEI

WARNING: THIS IS A DRAFT TASK MEANT TO BECOME A SAGA OF EPICS Patch RelatedArticles---- Zuul set : EXT_NAME=RelatedArticlesFrom a conversation with Zeljko related to webdriver.io testing Zuul add ['BetaFeatures', 'Cards', 'MobileFrontend'] + ['Echo', 'VisualEditor', 'ZeroBanner',Given a patch to the the mediawiki extension RelatedArticles. 'MobileApp'] + whatever elseZuul injects a parameter `EXT_NAME=RelatedArticles` 15 extensionsZuul then adds direct extensions dependencies: ['BetaFeatures', 'Cards', 'MobileFrontend'] which each bring more to it (eg: MobileFrontend also depends on ['Echo', 'VisualEditor', 'ZeroBanner', 'MobileApp'] The Webdriver.io jenkins job then: ``` cd MW_INSTALL_PATH npm run selenium ``` That ends up running every available tests because the wdio config has a glob pattern `extensions/*/tests/selenium/**/*.js`. # We would need to use subsets for example: Run tests solely for the extension that sent the patch wdio --spec extensions/${EXT_NAME}/tests/selenium/**/*.js MediaWiki sub set of tests: # MediaWiki sub set of tests: wdio --spec tests/selenium/smoke/**/*.js Smokes from other extensions # Smokes from other extensions wdio --spec extensions/${EXT_NAME}/tests/selenium/smoke/**/*.js ---- That came up also on a patch for Echo https://gerrit.wikimedia.org/r/#/c/313221/ which added a test requiring Flow behavior. I commented about how we could inject Flow as a dependency but that means all Flow tests will be run as well. The same split has to be done for PHPUnit as well ---- From a similar documentation back in December 14th 2016 related to PHPUnit extensions.Yurik and Hashar had a discussion on Dec. 14th 2016 about JSonConfig also running all the Scribunto tests, which is a recurring issue affecting multiple extensions (eg: T126670). IRC logs starting at 09:00 https://wm-bot.wmflabs.org/logs/%23wikimedia-releng/20161214.txt A lame drawing showing how the PHPUnit tests could be split: https://docs.google.com/drawings/d/1ebVAFqM80Me7HCiLV-JdAkNCh3unLCGsuNqRcgiGtEI