Please don't delete them, a ton of work went into them. I've done many rounds of making them less flaky. They still are, and I don't have time to fix them because this isn't part of our quarterly goals which we're trying to wrap up.

Also, a week is insufficient as a way to measure that a job is unsalvageable. For all we know, they're failing because of a real bug, it's just that nobody has had the bandwidth to look at it for a while. You work on browser tests every day, so a week might make sense to you, but for a team like multimedia it wasn't a weekly focus, it's something that happened when we found time.

Ok, looks like somebody cares about browser tests, so we will not delete any jobs. :) Feel fee to reach out to Release-Engineering-Team team if you need help. Also, I have proposed Workshop: Fix broken browsertests/Selenium Jenkins jobs (T94299) for this year's hackathons.

I've had a look at the failures and so far it looks like a lot of random timeouts. It could just be beta slowness (this is suggested by the few actual under-the-hood requests timing out), as the test we have running against mediawiki.org is very stable in comparison:

https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/

I'll do the stupid approach, which is to increase the timeout values for assertions timing out, we'll see if there's any improvement.

Given that MediaViewer is now in a stable state and not seeing active development, switching all MediaViewer browser tests to targeting mediawiki.org might be something to consider as well.

Change 202024 had a related patch set uploaded (by Gilles):
Increase values for frequent beta browser tests timeouts

https://gerrit.wikimedia.org/r/202024

Change 202024 merged by jenkins-bot:
Increase values for frequent beta browser tests timeouts

https://gerrit.wikimedia.org/r/202024

If the test framework supports that option, repeating failed tests and only reporting in case of two successive failures could help to filter out random errors resulting from beta being unstable.

Mass-removing the Multimedia tag from MediaViewer tasks, as this is now being worked on by the Reading department, not Editing's Multimedia team.

@Gilles do you need the mmw.performance browser test still? It doesn't seem to quite work and I'm wondering if it's safe to delete?

It depends on why it fails. It's a safeguard against media viewer's performance becoming worse than the file page's. If it's started failing a lot, it might not be a sign that the test has a problem, it could be that it's doing its job and telling you that media viewer has experienced a performance regression.

The error log clearly shows that it's a syntax problem, probably a change in the framework or cucumber due to an update:

https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/819/console

https://github.com/wikimedia/mediawiki-extensions-MultimediaViewer/blob/master/tests/browser/features/step_definitions/mmv_performance_steps.rb#L27

That line was touched a month ago by @zeljkofilipin, apparently related to a change to the framework:

https://github.com/wikimedia/mediawiki-extensions-MultimediaViewer/commit/2c3ac55a673ec5aba0080bb73789f2ea1110bd93

I'd strongly advise against deleting any test anyway. Especially in this sort of situation where it amounts to tearing down a house because you've found a leaking pipe. Whatever amount of work/difficulty encountered trying to fix the test is negligible compared to the amount of work it took to write it.

@zeljkofilipin would you be willing to fix this since I am not too familiar with the inner workings of the framework?

Is anyone monitoring the failing Firefox, IE8 and IE9 builds? I'd be tempted to kill them and just go with the passing https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-chrome-sauce/

And destroying man-weeks of work out of laziness to actually address a minor upgrade issue.

Can you please truly assume ownership of that code and maintain it, instead of giving up before even trying? Every time I see something Media Viewer-related you guys are looking for someone else to fix things for you, or you just give up without even trying to understand the code. That's fine for the first couple of months of ownership transfer, but now you own it.

It's even disrespectful of the time I spent on Dec 24 figuring out what was wrong. That stuff looks like a trivial upgrade change, but I'm not going to fix it for you.

I'd get it if you tried and then hit a wall, but this isn't even trying. Please realize that this didn't take 5 minutes to write and that you will be very glad if it works once you start modifying Media Viewer and aren't completely sure that you didn't break something. Which is something to expect when you're stumbling around a codebase you newly own. The tests should be the first thing you care about fixing. You'd be shooting yourselves in the foot in the long run deleting tests now, particularly given the simple reason why they've started breaking.

In T94157#1909192, @Jdlrobson wrote:

@zeljkofilipin would you be willing to fix this since I am not too familiar with the inner workings of the framework?

@Jdlrobson: I am willing to help you fix it, but I do not think it is a good idea to just fix it myself. If you learn how to fix the tests, you will know how to it yourself the next time. :) Let me know if you would like to pair on it.

Change 273234 had a related patch set uploaded (by Zfilipin):
WIP Fix browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce

https://gerrit.wikimedia.org/r/273234