Page MenuHomePhabricator

Login step failing for certain tests in MobileFrontend browser tests - blocking merges
Closed, ResolvedPublic

Description

Run bundle exec cucumber features/editor_wikitext_nosave.feature locally
For some reason this test fails locally. It seems the login step doesn't work and then the anonymous editor workflow is unexpectedly entered.

Event Timeline

Jdlrobson raised the priority of this task from to High.
Jdlrobson updated the task description. (Show Details)
Jdlrobson moved this task to To Do on the Reading-Web-Sprint-54-28-Days-Later board.
Jdlrobson added a subscriber: Jdlrobson.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 19 2015, 3:57 PM
Jdlrobson set Security to None.Aug 19 2015, 3:59 PM
Jdlrobson added subscribers: dduvall, Esanders.
Esanders raised the priority of this task from High to Unbreak Now!.Aug 19 2015, 10:45 PM
phuedx claimed this task.Aug 20 2015, 2:10 PM
phuedx moved this task from To Do to Doing on the Reading-Web-Sprint-54-28-Days-Later board.

I'm seeing these tests (features/editor_wikitest_nosave.feature) pass locally using PhantomJS and Firefox.

All integration tests but the "Search for partial text" test pass for me locally.

You may need to set MEDIAWIKI_ENVIRONMENT=integration

phuedx removed phuedx as the assignee of this task.Aug 20 2015, 2:48 PM
phuedx moved this task from Doing to To Do on the Reading-Web-Sprint-54-28-Days-Later board.
phuedx claimed this task.Aug 20 2015, 4:44 PM
phuedx moved this task from To Do to Doing on the Reading-Web-Sprint-54-28-Days-Later board.

@Jdlrobson: Thanks for the hint. Guess which browser the integration environment uses/which browser I didn't run the tests with in T109593#1557091… Chrome.

Jdlrobson moved this task from Doing to Code Review on the Reading-Web-Sprint-54-28-Days-Later board.
Jdlrobson moved this task from Code Review to Doing on the Reading-Web-Sprint-54-28-Days-Later board.
Jdlrobson added a subscriber: phuedx.

Change 232827 had a related patch set uploaded (by Jdlrobson):
Give login step time to complete and success

https://gerrit.wikimedia.org/r/232827

For transparency I have no idea why this is happening. If anyone can shed light on this I would be most appreciative. I still suspect something in the jjb job but @dduvall seems pretty certain that's not possible.

4:53 PM <jdlrobson> in reference to "marxarelli> one of a million moving browser test parts" (in that they were all passing and now suddenly they are breaking and stopping merges and we don't know why)
4:54 PM <jdlrobson> i assume someone committed something somewhere and it had a knock on effect here...
4:54 PM <•marxarelli> possibly
4:54 PM <jdlrobson> we need qa for our qa heh
4:54 PM <•marxarelli> could have been in mw core, any of the MF dependencies, or in MF itself
4:55 PM <•marxarelli> but it wasn't mw-selenium or the jjb job, i can tell ya that
4:55 PM <•marxarelli> your version of mw-selenium is actually versioned and the dependency is clearly locked :)
4:57 PM <jdlrobson> so any theories on what's happened?
4:57 PM <jdlrobson> did we merge something which didn't run the test job properly and borked everything?
4:57 PM <•marxarelli> fwict, mediawiki doesn't have a dependency system
4:57 PM <jdlrobson> it doesn't
4:57 PM <jdlrobson> i just don't get it.. :-/
4:58 PM → SPF|Cloud joined (uid11755@wikipedia/Southparkfan)
4:58 PM <jdlrobson> ok i think i gotta fix for this login issue
4:59 PM <•marxarelli> i mean, you could try git bisect and running the scenario, just to rule out changes in MF
5:01 PM <jdlrobson> yeh it's just super weird

Only possible theory is all our page loads got micro-seconds slower and caused this....

Change 232827 merged by jenkins-bot:
QA: Assert login step has completely successfully before continuing

https://gerrit.wikimedia.org/r/232827

Looks like these are still flaking.
https://integration.wikimedia.org/ci/job/mwext-mw-selenium/442/console

Can someone drop everything and look at this asap? I suspect what is happening is for whatever reason steps are not waiting until the previous page has finished loading. A few steps such as "check this completed" should suffice.

@dduvall ping - this is getting quite serious, so any help your team can give would be much appreciated, we may have to consider turning off the job temporarily and reverting back to Barry.

bmansurov moved this task from To Do to Doing on the Reading-Web-Sprint-54-28-Days-Later board.

Change 232891 had a related patch set uploaded (by Bmansurov):
Work around failing browser tests

https://gerrit.wikimedia.org/r/232891

There are some style comments from @dduvall that you might want to take a look at.

So it now seems to be flaking occasionally without @bmansurov's patch. Looking closely at a few failures the video suggests the test should be passing so I think it's just a case of increasing a timeout somewhere...

I'm not seeing any problems right now so let's keep an eye on this and not sign off till we know for sure all is good again

Increasing timeout on https://gerrit.wikimedia.org/r/232956 seems to work... and should reduce the flakiness can someone merge?

Change 232891 abandoned by Bmansurov:
Work around failing browser tests

Reason:
see jdlrobson's patch

https://gerrit.wikimedia.org/r/232891

Jhernandez closed this task as Resolved.Aug 25 2015, 9:58 AM
Jhernandez added a subscriber: Jhernandez.

Closing since it seems to be done. Reopen if it is still an issue.