In T229032#5366126, @Krinkle wrote:

Affects AbuseFilter commits

• https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/AbuseFilter/+/525598/
• https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-hhvm-docker/31783/console

Affects mediawiki/core commits:

• https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/525589/
• https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-hhvm-docker/20209/console

(And also all other patch sets in the last 20 minutes for mediawiki/core.)

Change 525611 had a related patch set uploaded (by Niedzielski; owner: Stephen Niedzielski):
[mediawiki/skins/MinervaNeue@master] [test] remove "Page diff Added and removed content" test

https://gerrit.wikimedia.org/r/525611

Change 525618 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Disable diff test due to issue with RunJobs

https://gerrit.wikimedia.org/r/525618

It looks like this is getting stuck in the Runjobs step:

10:31:24 RunJobs through requests to the front page (run 1).
10:31:24 RunJobs detected 54 more queued job(s).
10:31:24 RunJobs through requests to the front page (run 2).
10:31:24 RunJobs detected 54 more queued job(s).
10:31:24 RunJobs through requests to the front page (run 3).
10:31:24 RunJobs detected 54 more queued job(s).
10:31:24 RunJobs through requests to the front page (run 4).
10:31:25 RunJobs detected 54 more queued job(s).
10:31:25 RunJobs through requests to the front page (run 5).
10:31:25 RunJobs detected 54 more queued job(s).
10:31:25 RunJobs through requests to the front page (run 6).
10:31:25 RunJobs detected 54 more queued job(s).
10:31:25 RunJobs through requests to the front page (run 7).
10:31:25 RunJobs detected 54 more queued job(s).
10:31:25 RunJobs through requests to the front page (run 8).
10:31:25 RunJobs detected 54 more queued job(s).
10:31:25 RunJobs through requests to the front page (run 9).
10:31:25 RunJobs detected 54 more queued job(s).
10:31:25 RunJobs through requests to the front page (run 10).
10:31:25 RunJobs detected 54 more queued job(s).
10:31:25 RunJobs stopping requests to the front page due to reached limit.

	browser.call( () => RunJobs.run() );

As a result this call never happens:

	ArticlePage.open( pageTitle );

which breaks the rest of the test.

I don't know about the internals of RunJobs but that seems to be the problem here. We can disable this for the time being, but this issue is likely to resurface elsewhere and further put off people from writing Selenium tests unless addressed, so I recommend keeping this open and debugging some more.

Who maintains wdio-mediawiki/RunJobs ?

Change 525611 abandoned by Niedzielski:
[test] remove "Page diff Added and removed content" test

https://gerrit.wikimedia.org/r/525611

wdio-mediawiki/RunJobs is a wrapper around refreshing the wiki's main page (introduced by WMDE for rollback tests). It hasn't changed recently. The bug is more likely to do with the way actual jobs are run in MW core and/or deferred updates queued by the feature under test.

What part of the mobile diff depends on an async job having completed?

Change 525618 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Disable diff test due to issue with RunJobs

https://gerrit.wikimedia.org/r/525618

Hi, what is the action you'd like us to take here?

Just FYI, @zeljkofilipin is on vacation until Aug 9th.

Change 525814 had a related patch set uploaded (by Krinkle; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@wmf/1.34.0-wmf.15] Disable diff test due to issue with RunJobs

https://gerrit.wikimedia.org/r/525814

Change 525814 merged by Krinkle:
[mediawiki/skins/MinervaNeue@wmf/1.34.0-wmf.15] Disable diff test due to issue with RunJobs

https://gerrit.wikimedia.org/r/525814

Is this resolved by Krinkle's last merged patch? If not, could someone please answer T229031#5367511 ("what is the action you'd like RelEng to take here")? TIA

The test got disabled. We don't know how to fix it. So the bug is now about restoring that test.

We can't work out why this is not working so we could use some help from a selenium expert. Weve hit many issues with tests passing locally but failing in the ci builds since porting to Node.js and they have put my team off writing browser tests so I'm keen to work with RelEng to work out the root cause and make these more reliable.

In T229031#5366700, @Jdlrobson wrote:

Annoyingly when a test fails we only get a screenshot not a video

As far as I remember, the framework does not record videos in before and after hooks, only in the tests themselves. If there's a failure in before hook, wdio will automatically take a screenshot, but there will be no video.