Page MenuHomePhabricator

Re-activate selenium tests
Closed, ResolvedPublic

Description

In T196960: Quibble should have a way for extensions to opt out of core selenium browser tests Wikispeech is listed as one of the extensions failing the browser tests. As a result we are running the extension-quibble-noselenium CI-job (see config).

With the OOUI patch merged I think we might be able to switch to the clean extension-quibble job.

Event Timeline

The main headache is that I don't know of an easy way of testing if we are currently passing should we switch.

pinging @hashar directly here instead as lead for CI.

Is it possible to do a test run of the extension-quibble CI-job for the repo prior to having a patch merged into integration-config (and thus affecting all patches for that repo)?

I looked into doing a local update and test but the instructions on mw:Continuous_integration/Jenkins_job_builder suggested only users in the ciadmin LDAP group can actually send those tests on to Jenkins.

The CI jobs run Quibble and are configured to skip running selenium tests with --skip=selenium. The documentation is at https://doc.wikimedia.org/quibble/

I ran them locally:

ZUUL_PROJECT=mediawiki/extensions/Wikispeech quibble \
   --packages-source composer \
   --git-cache "$HOME/projects" \
   --db mysql --run selenium

And they pass just fine.

Is it possible to do a test run of the extension-quibble CI-job for the repo prior to having a patch merged into integration-config (and thus affecting all patches for that repo)?

For that use case, we usually add the candidate job to the experimental pipeline. That only reacts when someone comments in Gerrit check experimental, ie it is on demand.

Then, given the run passes locally I guess we can just change the CI configuration to enable Selenium tests ;]

I looked into doing a local update and test but the instructions on mw:Continuous_integration/Jenkins_job_builder suggested only users in the ciadmin LDAP group can actually send those tests on to Jenkins.

Indeed, the creation of jobs is restricted to the CI administrators as well as the workflow configuration (Zuul). Ideally developers would have a more free form way to define what is being running, the workaround is that CI is in the end rather dumb, it just invokes well known commands in the repositories which developers can tweak to run almost whatever they want ( https://www.mediawiki.org/wiki/Continuous_integration/Entry_points ).

Change 607234 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Wikispeech: enable Selenium tests

https://gerrit.wikimedia.org/r/607234

Change 607234 merged by jenkins-bot:
[integration/config@master] Wikispeech: enable Selenium tests

https://gerrit.wikimedia.org/r/607234

Change 607241 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/Wikispeech@master] Test change for CI selenium job

https://gerrit.wikimedia.org/r/607241

I went bold and did the config change. The selenium tests seem to have passed on a dummy change targeting master: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikispeech/+/607241/

That ran the Selenium tests from mediawiki/core and Vector.

You can add some for Wikispeech by following the tutorial at https://www.mediawiki.org/wiki/Selenium/How-to/Create_the_first_test_in_a_repository . Given your repository has a package.json defining a selenium-test script (as described on the wiki) , Quibble will detects it and runs it :]

Lokal_Profil moved this task from 📥 Backlog to ☑️ Done on the User-LokalProfil board.

The CI jobs run Quibble and are configured to skip running selenium tests with --skip=selenium. The documentation is at https://doc.wikimedia.org/quibble/

I ran them locally:

ZUUL_PROJECT=mediawiki/extensions/Wikispeech quibble \
   --packages-source composer \
   --git-cache "$HOME/projects" \
   --db mysql --run selenium

And they pass just fine.

Is it possible to do a test run of the extension-quibble CI-job for the repo prior to having a patch merged into integration-config (and thus affecting all patches for that repo)?

For that use case, we usually add the candidate job to the experimental pipeline. That only reacts when someone comments in Gerrit check experimental, ie it is on demand.

Many thanks for the explanation and for re-enabling the tests.

Change 607241 abandoned by Hashar:
Test change for CI selenium job

Reason:
The selenium tests passed just fine.

https://gerrit.wikimedia.org/r/607241