Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T420590 Decrease feedback loop for core tests to less than 10 minutes | |||
| Resolved | Peter | T421476 Speed up quibble-with-gated-extensions-selenium-php83 | |||
| Resolved | Peter | T419567 Splitting GrowthExperiments from gated extensions | |||
| Resolved | zeljkofilipin | T419574 Create a separate CI job for GrowthExperiments cypress tests | |||
| Open | None | T422786 Run @selenium-test npm script in *future* release branches |
Event Timeline
The already cut release branches still have selenium-test and have the selenium tests running. A more recent example is https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/1269414 made for REL1_45 it does run the browser test since the npm script is still selenium-test.
The problem is for future release branches: they will be based on current master which has @selenium-test and that will hide from CI.
| Branch | Project tested | Running? | How |
|---|---|---|---|
| master | GrowthExperiments | YES | exlplicitly via quibble-with-GrowthExperiments new job |
| master | gated extensions | No | quibble-with-gated-extensions does not see @selenium-test |
| wmf/* | GrowthExperiments | YES | same as master (see above) |
| wmf/* | gated extensions | No | same as master (see above) |
| current REL* | GrowthExperiments | Yes | Via Quibble job, branches still have selenium-test |
| current REL* | gated extensions | No | No gate for REL |
| Future REL | GrowthExperiments | NO | Would have the renamed @selenium-test** |
| Future REL | gated extensions | No | No gate for REL |
When cutting the future release branches we could rename @selenium-test to selenium-test but that we will certainly forget about it and AFAIK we do not have a runbook/automatization to deal with that for us.
It is thus less of an emergency than we thought this morning.
Potentially we could have Quibble to npm run @selenium-test for the project under test when the branch is a release branch.