Skip or fix the Minerva selenium test editor_wikitext_nosave.js browser test
Closed, ResolvedPublic
Actions

Description

The editor_wikitext_nosave.js selenium-daily-beta-Minerva test has been failing since 2/7/22.

https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-daily-beta-Minerva/1123/console

Details

Subject	Repo	Branch	Lines +/-
selenium: selenium-daily runs only tests that pass on the Beta Cluster	mediawiki/skins/MinervaNeue	master	+10 -10
[QA] Skip wikitext editor test on beta cluster	mediawiki/skins/MinervaNeue	master	+2 -2
[Tests] Check for VisualEditor in editor overlay integration test	mediawiki/skins/MinervaNeue	master	+1 -1

Customize query in gerrit

Related Objects

Mentioned In: T282246: `npm ci` fails when using Fresh in MinervaNeue

Event Timeline

cjming created this task.Feb 11 2022, 5:04 PM

Restricted Application added subscribers: Masumrezarock100, Aklapper. · View Herald TranscriptFeb 11 2022, 5:04 PM

Adding to sprint board as this test has been failing several weeks now, and as a result is losing us protection from regressions as it's teaching us to ignore these errors even if they are correct.

The test is either failing because

something is broken

or 2) Problem with testing infrastructure

If the reason is that it's broken, there's potential for unbreak now, as it would suggest that editors cannot see the editor overlay or there is a problem in the log in workflow.

I would suggest the following:

a web engineer analyzes what is going on here
If we determines the test is flaking, disables the test immediately. if a bug fix it.
If we disable the test check in with @Edtadros and @ovasileva about how important this test is.
Schedule time to fix the test if important. if not, throw this task back into the backlog.

Jdlrobson renamed this task from Minerva selenium test editor_wikitext_nosave.js failing to Skip or fix the Minerva selenium test editor_wikitext_nosave.js browser test.Feb 24 2022, 12:00 AM

Jdlrobson moved this task from Needs Analysis to Upcoming on the Web-Team-Backlog (Kanbanana-FY-2021-22) board.

Edtadros claimed this task.Feb 25 2022, 2:00 AM

Edtadros moved this task from Upcoming to Doing on the Web-Team-Backlog (Kanbanana-FY-2021-22) board.Feb 28 2022, 6:24 PM

Change 771075 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):

[mediawiki/skins/MinervaNeue@master] Check for VisualEditor

https://gerrit.wikimedia.org/r/771075

gerritbot added a project: Patch-For-Review.Mar 15 2022, 10:41 PM

I'm pretty sure this is failing because the VisuaLEditor is loading by default. We can try the above patch and if that doesn't work, skip it.

Change 771075 merged by jenkins-bot:

[mediawiki/skins/MinervaNeue@master] [Tests] Check for VisualEditor in editor overlay integration test

https://gerrit.wikimedia.org/r/771075

ReleaseTaggerBot added a project: MW-1.39-notes (1.39.0-wmf.1; 2022-03-21-early).Mar 16 2022, 6:00 PM

Maintenance_bot removed a project: Patch-For-Review.Mar 16 2022, 6:10 PM

Change 771479 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):

[mediawiki/skins/MinervaNeue@master] Disable wikitext editor test on beta cluster

https://gerrit.wikimedia.org/r/771479

gerritbot added a project: Patch-For-Review.Mar 16 2022, 11:23 PM

Change 771692 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/skins/MinervaNeue@master] WIP selenium: Do not run tests that fail in the daily job

https://gerrit.wikimedia.org/r/771692

zeljkofilipin claimed this task.Mar 18 2022, 9:57 AM

zeljkofilipin added a project: User-zeljkofilipin.

zeljkofilipin moved this task from Inbox to Test engineering on the Quality-and-Test-Engineering-Team board.

zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.

zeljkofilipin mentioned this in T282246: `npm ci` fails when using Fresh in MinervaNeue.Mar 18 2022, 1:42 PM

zeljkofilipin reassigned this task from zeljkofilipin to Edtadros.Mar 18 2022, 2:27 PM

zeljkofilipin subscribed.

This is fixed now. The issue was that somehow, the beta cluster test user had switched to "show VisualEditor" preference. A manual intervention has fixed that behaviour.

I'm not sure how this happened - it could be that VisualEditor changed a default, or somebody logged in manually to this account and used the editor.

I'm happy we don't have to skip this.

Jdlrobson removed Edtadros as the assignee of this task.Mar 21 2022, 3:45 PM

Jdlrobson moved this task from Code Review to Ready for Signoff on the Web-Team-Backlog (Kanbanana-FY-2021-22) board.

Change 771479 abandoned by Jdlrobson:

[mediawiki/skins/MinervaNeue@master] [QA] Skip wikitext editor test on beta cluster

Reason:

No longer needed

https://gerrit.wikimedia.org/r/771479

Jdlrobson moved this task from Backlog to Browser tests on the web-team-revisit-later board.Mar 22 2022, 8:47 PM

Change 771692 abandoned by Zfilipin:

[mediawiki/skins/MinervaNeue@master] selenium: selenium-daily runs only tests that pass on the Beta Cluster