Change 506453 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/MobileFrontend@master] Revert "💚 Run user_page test on non-existent user page to fix QA build"

https://gerrit.wikimedia.org/r/506453

I don't think this is specifically a problem with the MobileFrontend test so adding Release engineering in case something went wron

Change 506453 abandoned by Reedy:
Revert "💚 Run user_page test on non-existent user page to fix QA build"

https://gerrit.wikimedia.org/r/506453

https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/506421/ is still failing on wmf-quibble-core-vendor-mysql-php72-docker https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-php72-docker/2138/console

Why it would pass on hhvm but not php 7.2...

https://gerrit.wikimedia.org/r/505683 is failing too on the zend wmf-quibble jobs, but not on hhvm

https://gerrit.wikimedia.org/r/505648 too...

https://gerrit.wikimedia.org/r/c/mediawiki/core/+/487057 too, blocked that merge.

Change 506541 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/MobileFrontend@master] tests: Disable flaky "user_page" spec

https://gerrit.wikimedia.org/r/506541

In T221860#5137777, @Jdlrobson wrote:

I'm not able to replicate this and Jenkins is still verifying patches like https://gerrit.wikimedia.org/r/502820 and https://gerrit.wikimedia.org/r/504221 so I don't think this is blocking ALL merges so dropping to high.

It is a race condition, ergo a flaky test. It's clearly failing on some commits. Disable first, investigate later.

Change 506544 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/MobileFrontend@master] Disable "Check user page is editable" test

https://gerrit.wikimedia.org/r/506544

Change 506544 abandoned by Reedy:
Disable "Check user page is editable" test

https://gerrit.wikimedia.org/r/506544

Change 506541 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] tests: Disable flaky "user_page" spec

https://gerrit.wikimedia.org/r/506541

@Jdlrobson This is just a hunch, but from a quick glance over those steps I couldn't find any waitFor-instruction on the element it is asserting visibility of. Perhaps it's just the page hasn't finished loading yet, which could explain the race condition.

Thank you for the quick ''fix'' :]

This will need to be fixed in wdio-mediawiki. After debugging this, the problem whenever it fails is:

Unhandled rejection Error: badtoken: Invalid CSRF token.
19:06:08     at rawRequest.then (/workspace/src/node_modules/mwbot/src/index.js:257:31)
19:06:08     at tryCatcher (/workspace/src/node_modules/bluebird/js/release/util.js:16:23)
19:06:08     at Promise._settlePromiseFromHandler (/workspace/src/node_modules/bluebird/js/release/promise.js:512:31)
19:06:08     at Promise._settlePromise (/workspace/src/node_modules/bluebird/js/release/promise.js:569:18)
19:06:08     at Promise._settlePromise0 (/workspace/src/node_modules/bluebird/js/release/promise.js:614:10)
19:06:08     at Promise._settlePromises (/workspace/src/node_modules/bluebird/js/release/promise.js:694:18)
19:06:08     at _drainQueueStep (/workspace/src/node_modules/bluebird/js/release/async.js:138:12)
19:06:08     at _drainQueue (/workspace/src/node_modules/bluebird/js/release/async.js:131:9)
19:06:08     at Async._drainQueues (/workspace/src/node_modules/bluebird/js/release/async.js:147:5)
19:06:08     at Immediate.Async.drainQueues (/workspace/src/node_modules/bluebird/js/release/async.js:17:14)
19:06:08     at runCallback (timers.js:672:20)
19:06:08     at tryOnImmediate (timers.js:645:5)
19:06:08     at processImmediate [as _immediateCallback] (timers.js:617:5)

The test does:

browser.deleteCookie();
		Api.createAccount( username, password );
		RunJobs.run();
		UserLoginPage.login( username, password );

however when Api.createAccount( username, password ); fails (with the bad token error) the rest of the test fails.

I believe this is an issue in the mwbot library and the platform as the test makes use of Api.createAccount( username, password );
Both these issues seem related: https://github.com/Fannon/mwbot/issues/7 https://github.com/Fannon/mwbot/issues/9

FWIW I am seeing lots of token issues as I write new tests and I'm not sure mwbot may be the best choice for the Selenium tests as it seems to be geared more towards editing rather than how we use it here. Have we considered using packageFiles and using mediawiki.Api inside mediawiki core for doing this kind of thing?

Change 506583 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] ✅ Restore the user page test

https://gerrit.wikimedia.org/r/506583

Several runs of the above (with catch for mwbot problem) are all passing and in theory this should be no different when run in core (unless core is running a different set of tests?)

I'm not sure how to check against the problem we were seeing before but I'd like this test to be running again at least against the beta cluster!

This will be talked about tomorrow in a meeting with @zeljkofilipin

Change 506583 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] ✅ Restore the user page test

https://gerrit.wikimedia.org/r/506583

Change 508609 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] Revert "Disable test blocking merges on GrowthExperiments repo"

https://gerrit.wikimedia.org/r/508609

@Reedy is this task (as described in title and description) resolved? This seems to no longer block merges.

The link in the task description is broken. If the task is not resolved, could you please update the description with links to gerrit patches?

In T221860#5172257, @zeljkofilipin wrote:

@Reedy is this task (as described in title and description) resolved? This seems to no longer block merges.

The link in the task description is broken. If the task is not resolved, could you please update the description with links to gerrit patches?

In T221860#5156795, @Jdlrobson wrote:

@zfilipin there seems to be a serious issue with Node.js CI infrastructure given the second revert we've seen here on what is quite a straightforward test.

I am reluctant for us to write any new tests until we have this problem nailed down.

@Jdlrobson I'm confused by this task. Core merges failed because of MobileFrontend tests? That's what I've understood, but it makes no sense, since as far as I know core does not run MobileFrontend tests.

It would make sense if an extension that depends on MobileFrontend was blocked on CI because it ran MobileFrontend tests and they failed. I've looked at Gerrit patches mentioned in the task, but all Jenkins jobs I've looked at were already deleted.

Which patch(es) introduced the problem? Is there a way to reproduce the problem locally?

@Reedy thanks, but I'm still confused. Please see T221860#5172285.

@zeljkofilipin MobileFrontend is one of the extensions part of the wmf-gatedextensions list (zuul config).

This means that commits to core, and commits to any of these repos all run a similar Jenkins job (Quibble) where MediaWiki core + all these extensions are installed, and then we run PHPUnit, QUnit and Selenium WDIO tests.

This has been the case for a few years and the general idea is this enforces that tests of these repos must be passing at all times, and not be broken by patches in other repos. E.g. we do not want it to be possible to merge a commit in mediawiki/core that passes core tests, and then find out an hour later that now MobileFrontend tests are no longer passing with the latest master.

What changed recently is that MobileFrontend now has WDIO tests (in addition to PHPUnit+QUnit).

@Krinkle Thanks, now it makes sense. I thought that only tests from dependencies are run (so core would only run core tests). I didn't know we run tests for all those extensions and skins even for patches in core.

Change 508609 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Fix and re-enable user_page wdio specs

https://gerrit.wikimedia.org/r/508609

Tests are green, looks like the issue is resolved.