Run bundle exec cucumber features/editor_wikitext_nosave.feature locally
For some reason this test fails locally. It seems the login step doesn't work and then the anonymous editor workflow is unexpectedly entered.
Description
Details
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Duplicate | None | T109794 Chrome window resizes without height in @integration tests - toggling test fails | |||
Resolved | • bmansurov | T109593 Login step failing for certain tests in MobileFrontend browser tests - blocking merges |
Event Timeline
I'm seeing these tests (features/editor_wikitest_nosave.feature) pass locally using PhantomJS and Firefox.
@Jdlrobson: Thanks for the hint. Guess which browser the integration environment uses/which browser I didn't run the tests with in T109593#1557091… Chrome.
Change 232827 had a related patch set uploaded (by Jdlrobson):
Give login step time to complete and success
For transparency I have no idea why this is happening. If anyone can shed light on this I would be most appreciative. I still suspect something in the jjb job but @dduvall seems pretty certain that's not possible.
4:53 PM <jdlrobson> in reference to "marxarelli> one of a million moving browser test parts" (in that they were all passing and now suddenly they are breaking and stopping merges and we don't know why)
4:54 PM <jdlrobson> i assume someone committed something somewhere and it had a knock on effect here...
4:54 PM <•marxarelli> possibly
4:54 PM <jdlrobson> we need qa for our qa heh
4:54 PM <•marxarelli> could have been in mw core, any of the MF dependencies, or in MF itself
4:55 PM <•marxarelli> but it wasn't mw-selenium or the jjb job, i can tell ya that
4:55 PM <•marxarelli> your version of mw-selenium is actually versioned and the dependency is clearly locked :)
4:57 PM <jdlrobson> so any theories on what's happened?
4:57 PM <jdlrobson> did we merge something which didn't run the test job properly and borked everything?
4:57 PM <•marxarelli> fwict, mediawiki doesn't have a dependency system
4:57 PM <jdlrobson> it doesn't
4:57 PM <jdlrobson> i just don't get it.. :-/
4:58 PM → SPF|Cloud joined (uid11755@wikipedia/Southparkfan)
4:58 PM <jdlrobson> ok i think i gotta fix for this login issue
4:59 PM <•marxarelli> i mean, you could try git bisect and running the scenario, just to rule out changes in MF
5:01 PM <jdlrobson> yeh it's just super weird
Only possible theory is all our page loads got micro-seconds slower and caused this....
Change 232827 merged by jenkins-bot:
QA: Assert login step has completely successfully before continuing
Looks like these are still flaking.
https://integration.wikimedia.org/ci/job/mwext-mw-selenium/442/console
Can someone drop everything and look at this asap? I suspect what is happening is for whatever reason steps are not waiting until the previous page has finished loading. A few steps such as "check this completed" should suffice.
@dduvall ping - this is getting quite serious, so any help your team can give would be much appreciated, we may have to consider turning off the job temporarily and reverting back to Barry.
Change 232891 had a related patch set uploaded (by Bmansurov):
Work around failing browser tests
So it now seems to be flaking occasionally without @bmansurov's patch. Looking closely at a few failures the video suggests the test should be passing so I think it's just a case of increasing a timeout somewhere...
Am not sure what to make of this:
https://integration.wikimedia.org/ci/job/mwext-mw-selenium/464/artifact/log/Page%20diff%3A%20Added%20and%20removed%20content.mp4
https://integration.wikimedia.org/ci/job/mwext-mw-selenium/464/console
The element is clearly visible but it fails to assert that... @dduvall weird hey?
I'm not seeing any problems right now so let's keep an eye on this and not sign off till we know for sure all is good again
Increasing timeout on https://gerrit.wikimedia.org/r/232956 seems to work... and should reduce the flakiness can someone merge?
Change 232891 abandoned by Bmansurov:
Work around failing browser tests
Reason:
see jdlrobson's patch