I have renamed the bridge Jenkins job to client. Amended the configuration to run extensions/Wikibase/build/jenkins/mw-apply-wb-settings.sh with either -r client or -r repo.

selenium-test:repo

https://integration.wikimedia.org/ci/job/wikibase-selenium-repo-docker/5/console

Spec Files: 5 passed, 4 failed, 9 total

termbox ones are still failing with element (".wb-ui-all-entered-languages-expandable__switch") still not existing after 10000ms

selenium-test:bridge

https://integration.wikimedia.org/ci/job/wikibase-selenium-client-docker/5/console

Spec Files: 0 passed, 7 failed, 7 total

The specs now fail with:

Error: badvalue: Unrecognized value for parameter "action": wbeditentity.

The top of LocalSettings.php has:

<?php
// mw-apply-wb-settings.php:
$wgEnableWikibaseRepo = false;
$wgEnableWikibaseClient = true;
// $wgWikimediaJenkinsCI is usually set by Jenkins/Quibble
$wgWikimediaJenkinsCI = true;
$wmgUseWikibaseRepo = false;
$wmgUseWikibaseClient = true;
?>

Change 676107 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: Provide 'client' and 'repo' Wikibase selenium jobs

https://gerrit.wikimedia.org/r/676107

I have a suspicion that we would not want to split out client/data-bridge tests.
data-bridge integrated directly with MediaWiki and should be gated, the browser tests cover this integration.

termbox could be a better candidate to split out into a separate job that would not run as part of gated jobs as an initial step

I would also propose that we put this firmly in the hands of extensions rather than in CI configuration and make some generic commands for use by wmf ci such as ci-selenium-test-gate and ci-selenium-test that we could then chop and change what runs as we see fit without needing to change things centrally.
This could perhaps then be used by other extensions too in the same generic way?

Something relevant to other complexities highlighted in this task would be T97012: Wikibase defaults should be usable out of the box

In T287582#7368381, @Addshore wrote:

I have a suspicion that we would not want to split out client/data-bridge tests.
data-bridge integrated directly with MediaWiki and should be gated, the browser tests cover this integration.

We could have the job running npm run selenium-test:bridge to be triggered solely from mediawiki/core and Wikibase this way we keep the gating between the two projects but we no more run those tests for every single jobs or repository that include Wikibase as a dependency (wmf-quibble-selenium-php72-docker or Math which depends on Wikibase).

Then it fails with Error: badvalue: Unrecognized value for parameter "action": wbeditentity., maybe because it depends on some other extensions which would have to trigger that job as well.

termbox could be a better candidate to split out into a separate job that would not run as part of gated jobs as an initial step

If my digging is correct the chain is:

May we get a new npm run selenium-test:termbox and split that one to a standalone job as a first step? If those tests are no more run by changes affecting non Wikibase that will already by small win though they only take ~ 40 seconds to run.

I would also propose that we put this firmly in the hands of extensions rather than in CI configuration and make some generic commands for use by wmf ci such as ci-selenium-test-gate and ci-selenium-test that we could then chop and change what runs as we see fit without needing to change things centrally.
This could perhaps then be used by other extensions too in the same generic way?

Currently CI crawls all the repository looking for a selenium-test npm script and blindly run all of them serially. If I understand your intent we would need a second entry point that would be invoked solely for the repository that triggered the patch? We have a similar system for PHPUnit so that run tests of a repository marked @group Standalone solely for a change that targets that repository.

I got the repo variant passing by adding a few extensions as dependencies:

The client one still fails with the same issue though, maybe it needs some more dependencies.

I have found the issue!

I have created the Selenium jobs after the wikibase-client-docker and wikibase-repo-docker phpunit jobs which invoke the extensions/Wikibase/build/jenkins/mw-apply-wb-settings.sh with -r client or -r repo respectively. For Selenium testing we need the repo to be setup I have thus made the jobs to always pass -r repo.

For the dependencies Cite does not seem to be needed.

https://gerrit.wikimedia.org/r/676107 creates bridge and repo Selenium jobs which has MobileFrontend, MinervaNeue and UniversalLanguageSelector as dependencies. The jobs are triggered from all those repositories as well as from mediawiki/core and mediawiki/vendor. I have done it in a similar way as the gated jobs (wmf-quibble-*). That should give a guarantee it will never break.

Then I guess we can remove or rename selenium-test from the Wikibase package.json in order for Quibble to no more run it for other extensions. That would speed up the wmf-quibble-selenium-* job.

When trying the jobs yesterday they failed again. The reason is that the extension dependencies are not injected by Zuul since the change is in progress and not deployed. I have been testing them by manually triggering them on Jenkins, I have added a temp change https://gerrit.wikimedia.org/r/c/integration/config/+/790984 to inject EXT_DEPENDENCIES=mediawiki/skins/MinervaNeue\nmediawiki/extensions/MobileFrontend\nmediawiki/extensions/UniversalLanguageSelector.

Looking at the steps durations for the jobs:

wikibase-selenium-bridge-docker

• 55 seconds to finish clone, dependencies and MediaWiki installation
• 1m24 for npm ci
• 2m08 for wdio

wikibase-selenium-repo-docker

• 52 seconds to finish clone, dependencies and MediaWiki installation
• 1m23 for npm ci
• 1m20 for wdio

In the wmf-quibble-selenium-php72-docker job that would shred roughly 5 minutes and 30 seconds of build time if I get it right. Given that jobs takes 12 to 15 minutes to run, that would be a noticeable improvement.

Then I guess from Wikibase repository we can rename the selenium-test target which will prevent Quibble from discovering them and thus stop running them in the CI jobs:

"scripts": {
        "selenium-test": "npm-run-all --continue-on-error selenium-test:*",
        "selenium-test:repo": "wdio repo/tests/selenium/wdio.conf.js",
        "selenium-test:bridge": "npm --prefix client/data-bridge run selenium-test",
}

The data bridge have been disabled back in August 2023 and are no more running (tracked by T354841).

Change #1112783 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/extensions/Wikibase@master] build: hide Selenium tests from CI

https://gerrit.wikimedia.org/r/1112783

Change #1112784 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: ensure npm script exists in wikibase-selenium job

https://gerrit.wikimedia.org/r/1112784

Change #1112783 abandoned by Hashar:

[mediawiki/extensions/Wikibase@master] build: hide Selenium tests from CI

Reason:

I guess I would revisit it later

https://gerrit.wikimedia.org/r/1112783

Change #676107 abandoned by Hashar:

[integration/config@master] Use standalone jobs for Wikibase Selenium tests

Reason:

I guess I would revisit it later

https://gerrit.wikimedia.org/r/676107

Change #1112784 abandoned by Hashar:

[integration/config@master] jjb: ensure npm script exists in wikibase-selenium job

Reason:

I guess I would revisit it later

https://gerrit.wikimedia.org/r/1112784

Change #676107 restored by Hashar:

[integration/config@master] Use standalone jobs for Wikibase Selenium tests

https://gerrit.wikimedia.org/r/676107

I have rebased the Jenkins job configuration definition and triggered a run of the resulting wikibase-selenium job with the following parameters:

It invokes the following commands:

extensions/Wikibase/build/jenkins/mw-apply-wb-settings.sh -r repo
cp --verbose $WORKSPACE/src/LocalSettings.php $LOG_DIR
php maintenance/update.php --quick
npm --prefix extensions/Wikibase ci
npm --prefix extensions/Wikibase run --if-present @selenium-test

And eventually that failed for whatever reason https://integration.wikimedia.org/ci/job/wikibase-selenium/12/console

When I look at the MediaWiki error log, the page fails due to a lack of MediaWiki\Hook\AbortEmailNotificationHook.

2026-01-29 19:35:55 4552a3b7c26f wikidb: [25c4484c8a4df8877b54a789] /index.php?title=Special%3AUserLogin   Error: Interface "MediaWiki\Hook\AbortEmailNotificationHook" not found
#0 /workspace/src/includes/AutoLoader.php(172): require_once()
#1 /workspace/src/vendor/wikimedia/object-factory/src/ObjectFactory.php(240): AutoLoader::autoload(string)
#2 /workspace/src/vendor/wikimedia/object-factory/src/ObjectFactory.php(149): Wikimedia\ObjectFactory\ObjectFactory::getObjectFromSpec(array, array)
#3 /workspace/src/includes/HookContainer/HookContainer.php(232): Wikimedia\ObjectFactory\ObjectFactory->createObject(array)
#4 /workspace/src/includes/HookContainer/HookContainer.php(289): MediaWiki\HookContainer\HookContainer->makeExtensionHandlerCallback(string, array, array)
#5 /workspace/src/includes/HookContainer/HookContainer.php(412): MediaWiki\HookContainer\HookContainer->normalizeHandler(string, array, array)
#6 /workspace/src/includes/HookContainer/HookContainer.php(128): MediaWiki\HookContainer\HookContainer->getHandlers(string, array)
#7 /workspace/src/includes/HookContainer/HookRunner.php(1022): MediaWiki\HookContainer\HookContainer->run(string, array, array)
#8 /workspace/src/includes/Output/OutputPage.php(3300): MediaWiki\HookContainer\HookRunner->onBeforePageDisplay(MediaWiki\Output\OutputPage, MediaWiki\Skins\Vector\SkinVector22)
#9 /workspace/src/includes/Actions/ActionEntryPoint.php(161): MediaWiki\Output\OutputPage->output(bool)
#10 /workspace/src/includes/MediaWikiEntryPoint.php(180): MediaWiki\Actions\ActionEntryPoint->execute()
#11 /workspace/src/index.php(44): MediaWiki\MediaWikiEntryPoint->run()
#12 {main}

That was removed from core on January 8th by https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1170627 . I forgot to rebase https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/1112783/ :]

Change #1112783 restored by Hashar:

[mediawiki/extensions/Wikibase@master] build: hide Selenium tests from CI

https://gerrit.wikimedia.org/r/1112783

I rebased Wikibase - 1112783 and it passed and the commands took 421.447 s. By moving that out to a standalone job, that will make the Selenium that faster for any project that has Wikibase as a dependency.

I'll check with WMDE people to sync up on this and get the change reviewed, merged and backported to the supported branches.

Change #1239331 had a related patch set uploaded (by WMDE-leszek; author: WMDE-leszek):

[integration/config@master] Demonstrate including Wikibase selenium tests in WikibaseLexeme CI

https://gerrit.wikimedia.org/r/1239331

Thanks @hashar for looking into this. Having look at the patch WMDE people would like to confirm our understanding of the influence of the change. Mind confirming or correcting the below observations?

1. Before the changes proposed, Quibble runs Wikibase selenium tests as they are invoked through the npm selenium-test script. This applies to Wikibase but also to mediawiki (mediawiki/core) and other extensions if they pull Wikibase's code through Jenkins job's dependency.
2. After the changes, in particular the Wikibase change Wikibase's selenium-test will no longer be picked up by Quibble, hence Wikibase selenium tests won't be run.
3. After the changes, In order to run Wikibase selenium tests one needs to explicitly include them in the CI config, by referring to a wikibase-selenium job (through a wikibase-selenium-gate job template). With your changes this will only be happening for changes in Wikibase itself and in mediawiki/core and mediawiki/vendor repositories.
4. After the changes, If one wanted to include Wikibase selenium tests in the CI build of another repository (e.g. another MW extension), they'd need to add wikibase-seleniun-gate to CI definition for that repository, like I have demonstrated in a dummy Wikibase Lexeme repository CI config change.

@WMDE-leszek sorry I had a very busy week! Thank you for you comment summarizing your understanding of this change. I am adding some further precisions below:

3. After the changes, In order to run Wikibase selenium tests one needs to explicitly include them in the CI config, by referring to a wikibase-selenium job (through a wikibase-selenium-gate job template).

*Correct*, that is similar to the quibble-with-gated-extensions which have a fixed set of repositories injected and have all those repositories triggering that job.

This change is thus a more specific case of quibble-with-gated-extensions with a subset affecting the Wikibase Selenium tests.

3. With your changes this will only be happening for changes in Wikibase itself and in mediawiki/core and mediawiki/vendor repositories.

And for the other dependencies that are required to run the Wikibase Selenium tests:

• mediawiki/skins/MinervaNeue
• mediawiki/extensions/MobileFrontend
• mediawiki/extensions/UniversalLanguageSelector

Those are injected via zuul/parameter_functions.py rather than zuul/dependencies.yaml. The three additional repositories also have the wikibase-selenium-gate job template applied in zuul/layout.yaml.

There is a Python test in integration/config to ensure both lists are in sync, it asserts the list of dependencies that are injected and the repositories having wikibase-selenium-gate template are equals.

4. After the changes, If one wanted to include Wikibase selenium tests in the CI build of another repository (e.g. another MW extension), they'd need to add wikibase-selenium-gate to CI definition for that repository, like I have demonstrated in a dummy Wikibase Lexeme repository CI config change.

The WikibaseLexeme will also need to be added to the list of extensions the job clones. Else if one sends a patch to another of the repository, for example mediawiki/skins/MinervaNeue then the job would not have WikibaseLexeme and maybe that change could cause one of its test to fail. Thus it should also be added to zuul/parameter_functions.py.

Your WIP change is currently in merge conflict because the parent is a merge conflict, but if the CI tests for integration/config ran they would complain about WikibaseLexeme not being in the zuul/parameter_functions.py.

Edit: I rebased my change and your WIP:

The test suite failed as expected and with the following message:

'mediawiki/extensions/WikibaseLexeme' : Zuul projects triggering Wikibase Selenium jobs (first set) and dependency list in zuul/parameter_functions.py (2nd set) must be equals.
In Zuul layout: apply the template wikibase-selenium-gate
In Zuul parameter function add the missing repo

:]

For the later actions, we'd want to use the same system for GrowthExperiments and move its Selenium tests outside of the quibble-with-gated-extensions for pretty much the same reason: its Selenium tests take a while and there is no point in running for every single extensions participating in that job. Instead, as for Wikibase, we can have a separate job that only triggers for the subset of extensions required for GrowthExperiments Selenium tests to pass.

On a longer term, I am considering phasing out the large quibble-with-gated-extensions job. That is really overkill and running to many test, and instead create smaller jobs that run in parallel and are triggered from less repo. That would lower the overall load on CI and make the CI feedback faster since less useless tests will run.

We have some metrics somewhere to gauge how long those CI jobs / changes take to merge, so we should see a noticeable improvement ™ 🎉

@WMDE-leszek @zeljkofilipin and I would like to deploy the change move the Wikibase Selenium tests to a standalone next week, let me know if you or your teams need a bit more time to process my wall of texts. Once we have deployed the change and if that is a success, we will follow up with GrowthExperiments and possibly generalize it to other extensions and/or PHPUnit tests as well.

Thanks for this additional information @hashar. Please proceed with the change

Mentioned in SAL (#wikimedia-releng) [2026-02-26T13:40:51Z] <hashar> Deployed Jenkins job https://integration.wikimedia.org/ci/job/wikibase-selenium/ # T287582

Change #676107 merged by jenkins-bot:

[integration/config@master] Use standalone jobs for Wikibase Selenium tests

https://gerrit.wikimedia.org/r/676107

Change #1244658 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Add wikibase-selenium job to Wikibase

https://gerrit.wikimedia.org/r/1244658

Change #1244658 merged by Hashar:

[integration/config@master] Add wikibase-selenium job to Wikibase

https://gerrit.wikimedia.org/r/1244658

Change #1244663 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Zuul: inject Wikibase to wikibase-selenium job

https://gerrit.wikimedia.org/r/1244663

Change #1244664 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Zuul: fix wikibase-selenium jobs matching in parameter function

https://gerrit.wikimedia.org/r/1244664

Change #1244663 merged by jenkins-bot:

[integration/config@master] Zuul: inject Wikibase to wikibase-selenium job

https://gerrit.wikimedia.org/r/1244663

Change #1244664 merged by jenkins-bot:

[integration/config@master] Zuul: fix wikibase-selenium jobs matching in parameter function

https://gerrit.wikimedia.org/r/1244664

Change #1244671 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/extensions/UniversalLanguageSelector@master] Verify ULS is gated with Wikibase via wikibase-selenium

https://gerrit.wikimedia.org/r/1244671

Change #1112783 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] build: hide Selenium tests from CI

https://gerrit.wikimedia.org/r/1112783

I have deployed the CI Configuration change that moves the Wikibase Selenium tests to a standalone job. I think it is working from the preliminary checks I have done. I'll revisit tomorrow to ensure everything is fine and if so claim it a success.

Change #1244671 abandoned by Hashar:

[mediawiki/extensions/UniversalLanguageSelector@master] Verify ULS is gated with Wikibase via wikibase-selenium

Reason:

This was a test change to validate T287582

https://gerrit.wikimedia.org/r/1244671

I think this made an impact on merge times.

Here's the rolling median of merge times (between starting gate-and-submit jobs to merge) for non-l10n changes to mainline branches for gated extensions between 2026-02-22 and today.

Fantastic results.