Audit the failures of https://releases-jenkins.wikimedia.org/job/MediaWiki%20branch%20and%20publish%20WMF%20single-version%20image/ looking for commonly failing tests. If we can find tests that failing repeatedly they should be reported on T291254: Find flaky Selenium tests and attempts made to get help in fixing, removing, or replacing those tests.
See also:
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| selenium: Skip flaky tests | mediawiki/extensions/VisualEditor | master | +4 -2 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Lferreira | T215217 deployment-prep (beta cluster): Code stewardship request | |||
| Open | Goal | None | T369112 Pretrain (née Group -1) QTE validation environment | ||
| Resolved | Goal | dancy | T398868 [FY25-26 WE6.1.1] Move image build to deployment server and update for backports | ||
| Resolved | dduvall | T400761 Audit "MediaWiki branch and publish WMF single-version image" job failures and recruit help fixing flaky tests |
Change #1178622 had a related patch set uploaded (by Dduvall; author: Dduvall):
[mediawiki/extensions/VisualEditor@master] selenium: Remove flaky tests
Change #1178622 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] selenium: Skip flaky tests
Let's call this one {{done}}. Disabling those 2 flaky ContentEditable tests (T401573: Flaky VisualEditor Selenium test: Content Editable should insert a table) brought us down to only 2 build failures in the past month. The global flaky test failure rate also fell to an average of 2.5% from a prior benchmark of an average of 9.4% as measured by the flaky-tests dashboard.