Page MenuHomePhabricator

AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage on wmf-quibble PHP jobs
Closed, ResolvedPublic5 Estimated Story PointsPRODUCTION ERROR

Description

Seen on a core patch

https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-php72-docker/2134/console

15:22:42 1) User:<username> Check user page is editable:
15:22:42 false === true
15:22:42 running chrome
15:22:42 AssertionError: false === true
15:22:42     at thereShouldBeALinkToCreateMyUserPage (/workspace/src/extensions/MobileFrontend/tests/selenium/features/step_definitions/user_page_steps.js:16:9)
15:22:42     at Context.it (/workspace/src/extensions/MobileFrontend/tests/selenium/specs/user_page.js:27:3)
15:22:42     at Promise.F (/workspace/src/node_modules/core-js/library/modules/_export.js:36:28)
15:22:42

Event Timeline

Reedy triaged this task as Unbreak Now! priority.Apr 25 2019, 3:03 PM

It's blocking other merges

Change 506453 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/MobileFrontend@master] Revert "💚 Run user_page test on non-existent user page to fix QA build"

https://gerrit.wikimedia.org/r/506453

Jdlrobson lowered the priority of this task from Unbreak Now! to High.Apr 25 2019, 3:30 PM
Jdlrobson added a subscriber: Jdlrobson.

I'm not able to replicate this and Jenkins is still verifying patches like https://gerrit.wikimedia.org/r/502820 and https://gerrit.wikimedia.org/r/504221 so I don't think this is blocking ALL merges so dropping to high.

I don't think this is specifically a problem with the MobileFrontend test so adding Release engineering in case something went wron

Change 506453 abandoned by Reedy:
Revert "💚 Run user_page test on non-existent user page to fix QA build"

https://gerrit.wikimedia.org/r/506453

https://gerrit.wikimedia.org/r/505683 is failing too on the zend wmf-quibble jobs, but not on hhvm

Reedy renamed this task from AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage to AssertionError: false === true at thereShouldBeALinkToCreateMyUserPage on wmf-quibble PHP jobs.Apr 25 2019, 4:42 PM
Reedy removed a project: Patch-For-Review.

Change 506541 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/MobileFrontend@master] tests: Disable flaky "user_page" spec

https://gerrit.wikimedia.org/r/506541

I'm not able to replicate this and Jenkins is still verifying patches like https://gerrit.wikimedia.org/r/502820 and https://gerrit.wikimedia.org/r/504221 so I don't think this is blocking ALL merges so dropping to high.

It is a race condition, ergo a flaky test. It's clearly failing on some commits. Disable first, investigate later.

Change 506544 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/MobileFrontend@master] Disable "Check user page is editable" test

https://gerrit.wikimedia.org/r/506544

Change 506544 abandoned by Reedy:
Disable "Check user page is editable" test

https://gerrit.wikimedia.org/r/506544

Change 506541 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] tests: Disable flaky "user_page" spec

https://gerrit.wikimedia.org/r/506541

@Jdlrobson This is just a hunch, but from a quick glance over those steps I couldn't find any waitFor-instruction on the element it is asserting visibility of. Perhaps it's just the page hasn't finished loading yet, which could explain the race condition.

hashar added a subscriber: hashar.

Thank you for the quick ''fix'' :]

This will need to be fixed in wdio-mediawiki. After debugging this, the problem whenever it fails is:

Unhandled rejection Error: badtoken: Invalid CSRF token.
19:06:08     at rawRequest.then (/workspace/src/node_modules/mwbot/src/index.js:257:31)
19:06:08     at tryCatcher (/workspace/src/node_modules/bluebird/js/release/util.js:16:23)
19:06:08     at Promise._settlePromiseFromHandler (/workspace/src/node_modules/bluebird/js/release/promise.js:512:31)
19:06:08     at Promise._settlePromise (/workspace/src/node_modules/bluebird/js/release/promise.js:569:18)
19:06:08     at Promise._settlePromise0 (/workspace/src/node_modules/bluebird/js/release/promise.js:614:10)
19:06:08     at Promise._settlePromises (/workspace/src/node_modules/bluebird/js/release/promise.js:694:18)
19:06:08     at _drainQueueStep (/workspace/src/node_modules/bluebird/js/release/async.js:138:12)
19:06:08     at _drainQueue (/workspace/src/node_modules/bluebird/js/release/async.js:131:9)
19:06:08     at Async._drainQueues (/workspace/src/node_modules/bluebird/js/release/async.js:147:5)
19:06:08     at Immediate.Async.drainQueues (/workspace/src/node_modules/bluebird/js/release/async.js:17:14)
19:06:08     at runCallback (timers.js:672:20)
19:06:08     at tryOnImmediate (timers.js:645:5)
19:06:08     at processImmediate [as _immediateCallback] (timers.js:617:5)

The test does:

browser.deleteCookie();
		Api.createAccount( username, password );
		RunJobs.run();
		UserLoginPage.login( username, password );

however when Api.createAccount( username, password ); fails (with the bad token error) the rest of the test fails.

I believe this is an issue in the mwbot library and the platform as the test makes use of Api.createAccount( username, password );
Both these issues seem related: https://github.com/Fannon/mwbot/issues/7 https://github.com/Fannon/mwbot/issues/9

FWIW I am seeing lots of token issues as I write new tests and I'm not sure mwbot may be the best choice for the Selenium tests as it seems to be geared more towards editing rather than how we use it here. Have we considered using packageFiles and using mediawiki.Api inside mediawiki core for doing this kind of thing?

Change 506583 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] ✅ Restore the user page test

https://gerrit.wikimedia.org/r/506583

Several runs of the above (with catch for mwbot problem) are all passing and in theory this should be no different when run in core (unless core is running a different set of tests?)

I'm not sure how to check against the problem we were seeing before but I'd like this test to be running again at least against the beta cluster!

ovasileva set the point value for this task to 5.Apr 30 2019, 4:21 PM

This will be talked about tomorrow in a meeting with @zeljkofilipin

Change 506583 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] ✅ Restore the user page test

https://gerrit.wikimedia.org/r/506583

@zfilipin there seems to be a serious issue with Node.js CI infrastructure given the second revert we've seen here on what is quite a straightforward test.

I am reluctant for us to write any new tests until we have this problem nailed down.

Change 508609 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] Revert "Disable test blocking merges on GrowthExperiments repo"

https://gerrit.wikimedia.org/r/508609

@Reedy is this task (as described in title and description) resolved? This seems to no longer block merges.

The link in the task description is broken. If the task is not resolved, could you please update the description with links to gerrit patches?

@Reedy is this task (as described in title and description) resolved? This seems to no longer block merges.

The link in the task description is broken. If the task is not resolved, could you please update the description with links to gerrit patches?

@zfilipin there seems to be a serious issue with Node.js CI infrastructure given the second revert we've seen here on what is quite a straightforward test.

I am reluctant for us to write any new tests until we have this problem nailed down.

@Jdlrobson I'm confused by this task. Core merges failed because of MobileFrontend tests? That's what I've understood, but it makes no sense, since as far as I know core does not run MobileFrontend tests.

It would make sense if an extension that depends on MobileFrontend was blocked on CI because it ran MobileFrontend tests and they failed. I've looked at Gerrit patches mentioned in the task, but all Jenkins jobs I've looked at were already deleted.

Which patch(es) introduced the problem? Is there a way to reproduce the problem locally?

@zeljkofilipin MobileFrontend is one of the extensions part of the wmf-gatedextensions list (zuul config).

This means that commits to core, and commits to any of these repos all run a similar Jenkins job (Quibble) where MediaWiki core + all these extensions are installed, and then we run PHPUnit, QUnit and Selenium WDIO tests.

This has been the case for a few years and the general idea is this enforces that tests of these repos must be passing at all times, and not be broken by patches in other repos. E.g. we do not want it to be possible to merge a commit in mediawiki/core that passes core tests, and then find out an hour later that now MobileFrontend tests are no longer passing with the latest master.

What changed recently is that MobileFrontend now has WDIO tests (in addition to PHPUnit+QUnit).

@Krinkle Thanks, now it makes sense. I thought that only tests from dependencies are run (so core would only run core tests). I didn't know we run tests for all those extensions and skins even for patches in core.

Change 508609 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Fix and re-enable user_page wdio specs

https://gerrit.wikimedia.org/r/508609

Tests are green, looks like the issue is resolved.

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:07 PM