Browser tests no longer work on Chrome
Closed, ResolvedPublic5 Estimated Story Points
Actions

Assigned To

Authored By

	• Gilles
	Jan 14 2015, 3:57 PM

Description

Following the recent additions to the browser tests:

https://integration.wikimedia.org/ci/view/BrowserTests/view/-Dashboard/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce/

This is probably caused by Chrome being picky with z-index

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Fix browser tests in Chrome	mediawiki/extensions/UploadWizard	master	+72 -104

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• Gilles	T91651 Significantly increase UploadWizard test coverage
		Resolved		• Gilles	T86770 Browser tests no longer work on Chrome

Event Timeline

• Gilles created this task.Jan 14 2015, 3:57 PM

• Gilles assigned this task to MarkTraceur.

• Gilles raised the priority of this task from to High.

• Gilles updated the task description. (Show Details)

• Gilles added projects: UploadWizard, Multimedia.

• Gilles subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 14 2015, 3:57 PM

• Gilles renamed this task from UW browser tests no longer work on Chrome to Browser tests no longer work on Chrome.Jan 14 2015, 3:58 PM

• Gilles set Security to None.

• Gilles moved this task from Untriaged to Next up on the Multimedia board.Jan 14 2015, 4:36 PM

@Cmcmahon has noted that this is an issue everyone is seeing. We can definitely fix it by putting "sleep 1" in front of every click action, but that's not exactly the best way.

I'm going to do a bisect on a few commits to see what I can find.

Hm, I am now of the opinion that this isn't a problem in UploadWizard, because the tests fail in Chrome all the way back to October...

MarkTraceur moved this task from Next up to Prototyping on the Multimedia board.Feb 4 2015, 5:53 PM

• Gilles added a project: Multimedia-Sprint-2015-02-18.Feb 18 2015, 10:11 AM

It's certainly a matter of waiting for the right things, or short of that being possible, adding sleeps.

• Gilles removed MarkTraceur as the assignee of this task.Feb 18 2015, 10:21 AM

• Gilles added a subscriber: MarkTraceur.

• Gilles claimed this task.Feb 19 2015, 1:33 PM

Change 191645 had a related patch set uploaded (by Gilles):
Fix browser tests in Chrome

https://gerrit.wikimedia.org/r/191645

Patch-For-Review

Aklapper added a project: Patch-For-Review.Feb 20 2015, 12:32 PM

• Gilles moved this task from Backlog to Needs code review on the Multimedia-Sprint-2015-02-18 board.Feb 23 2015, 10:35 AM

• Gilles moved this task from Needs code review to Backlog on the Multimedia-Sprint-2015-02-18 board.Feb 23 2015, 5:28 PM

• Gilles edited a custom field.

• Gilles moved this task from Backlog to Reviewed, needs improvements on the Multimedia-Sprint-2015-02-18 board.Feb 23 2015, 6:03 PM

• Gilles moved this task from Reviewed, needs improvements to Needs code review on the Multimedia-Sprint-2015-02-18 board.Feb 23 2015, 7:00 PM

• Gilles added a project: Multimedia-Sprint-2015-02-25.Feb 25 2015, 10:57 AM

• Gilles moved this task from Backlog to Needs code review on the Multimedia-Sprint-2015-02-25 board.

Change 191645 merged by jenkins-bot:
Fix browser tests in Chrome

https://gerrit.wikimedia.org/r/191645

MarkTraceur mentioned this in rEUWIa8dd8d2a1dca: Fix browser tests in Chrome.Feb 25 2015, 8:25 PM

• Gilles moved this task from Needs code review to Ready for testing on the Multimedia-Sprint-2015-02-25 board.Feb 25 2015, 8:26 PM

• Gilles removed a project: Multimedia-Sprint-2015-02-18.Feb 25 2015, 8:54 PM

The first run on the integration machine has a bunch of failures: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce/500/#showFailuresLink moving this back to development and bumping the points count.

• Gilles edited a custom field.Feb 26 2015, 4:26 PM

• Gilles moved this task from Ready for testing to Backlog on the Multimedia-Sprint-2015-02-25 board.

I guess the differences are probably due to this running on linux while my machine is OS X. Running the test on saucelabs should answer that question.

The good news is that it all seems to be the same failure being repeated.

Indeed, the issue is reproducible on saucelabs with linux chrome.

The test is doing what it's supposed to, I looked for hacks but couldn't find anything that works. It seems to be a limitation of the linux chrome driver in relationship to the upload button hack done in UploadWizard. I think that the sanest thing to do is to test UploadWizard using OS X/Windows Chrome, since we know that at least the OS X driver doesn't suffer from that issue.

I'll go ahead and change the configuration on the integration server.

Turns out that it's just a Chrome version issue. 28 fails, 31 (chosen randomly) works fine. Bumping the version to 40 on the integration server solves the issue:

https://integration.wikimedia.org/ci/view/BrowserTests/view/-Dashboard/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce/509/console

• Gilles closed this task as Resolved.Mar 2 2015, 11:00 AM

• Gilles moved this task from Backlog to Ready for testing on the Multimedia-Sprint-2015-02-25 board.

• Gilles added a parent task: T91651: Significantly increase UploadWizard test coverage.Mar 5 2015, 4:40 PM

Diffusion mentioned this in rMEXT2152aaa879af: Updated mediawiki/extensions Project: mediawiki/extensions/UploadWizard….Mar 11 2015, 6:27 AM