Page MenuHomePhabricator
Create Task
Maniphest T94153

Fix Flow scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily
Closed, ResolvedPublic
Actions

Description

All jobs:

https://integration.wikimedia.org/ci/view/BrowserTests/view/Flow/

Jobs failing for the last month:

https://integration.wikimedia.org/ci/view/BrowserTests/view/Flow/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce/
https://integration.wikimedia.org/ci/view/BrowserTests/view/Flow/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/
https://integration.wikimedia.org/ci/view/BrowserTests/view/Flow/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce/
https://integration.wikimedia.org/ci/view/BrowserTests/view/Flow/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/
https://integration.wikimedia.org/ci/view/BrowserTests/view/Flow/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-10-sauce/

Anybody cares about the jobs? Do you need help fixing them?

The plan is to delete the jobs if there is no reply in a week or so.

Details

SubjectRepoBranchLines +/-
Followup 9eaa667: Use @skip instead so it's more obvious and greppablemediawiki/extensions/Flowmaster+4 -0
Disable scenarios that fail at en.wikipedia.beta.wmflabs.org from running dailymediawiki/extensions/Flowmaster+0 -2
Disable scenarios that fail at en.wikipedia.beta.wmflabs.org from running dailymediawiki/extensions/Flowmaster+7 -12
Deleted failing Flow browsertests Jenkins jobintegration/configmaster+0 -7
Move tooltip to the rightmediawiki/extensions/Flowmaster+2 -2
Browser tests: fix javascript detection on loadmediawiki/extensions/Flowmaster+8 -5
Fix browser testsmediawiki/extensions/Flowmaster+15 -8
Make browser tests work with or without visual editormediawiki/extensions/Flowmaster+24 -4
Hover on the post action menu instead of clicking on itmediawiki/extensions/Flowmaster+12 -12
Notify Collaboration team of failing browser testsintegration/configmaster+7 -4
Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
• DannyH added a comment.Apr 22 2015, 7:01 PM

Elena will check this over.

• DannyH moved this task from Ready for next sprint to Current workboard on the Collaboration-Team-Triage board.Apr 22 2015, 7:01 PM
• DannyH edited projects, added Collaboration-Team-Sprint-A-2015-05-06; removed Collaboration-Team-Triage.Apr 22 2015, 7:08 PM
Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptApr 22 2015, 7:08 PM
SBisson moved this task from Sprint A - ends May 6 to In Development on the Collaboration-Team-Sprint-A-2015-05-06 board.Apr 22 2015, 7:39 PM
• DannyH renamed this task from Fix failed Flow browsertests Jenkins jobs to A5. Fix failed Flow browsertests Jenkins jobs.Apr 22 2015, 8:04 PM
• DannyH moved this task from Current workboard to Untriaged on the Collaboration-Team-Triage board.
• DannyH moved this task from Untriaged to Current workboard on the Collaboration-Team-Triage board.Apr 22 2015, 8:07 PM
hashar updated the task description. (Show Details)Apr 24 2015, 10:03 AM

Rephrasing the task details. The Echo and Flow jobs are listed at https://integration.wikimedia.org/ci/view/BrowserTests/view/Echo+Flow/

SBisson added a subtask: T92871: Parsoid patches don't update Beta Cluster automatically -- only deploy repo patches seem to update that code.Apr 24 2015, 1:09 PM
• Mattflaschen-WMF added a comment.Apr 25 2015, 4:36 AM

They are now failing with:

"timed out after 5 seconds, waiting for {:name=>"topiclist_content", :tag_name=>"textarea"} to become present (Watir::Wait::TimeoutError)"

• Mattflaschen-WMF added a comment.Apr 25 2015, 4:37 AM

This is after Beta was fixed: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/651/testReport/

gerritbot added a comment.Apr 27 2015, 8:11 PM

Change 206880 had a related patch set uploaded (by Catrope):
Hover on the post action menu instead of clicking on it

https://gerrit.wikimedia.org/r/206880

gerritbot added a comment.Apr 27 2015, 8:55 PM

Change 206880 merged by jenkins-bot:
Hover on the post action menu instead of clicking on it

https://gerrit.wikimedia.org/r/206880

Catrope mentioned this in rEFLW748ec6157e63: Hover on the post action menu instead of clicking on it.Apr 27 2015, 8:55 PM
SBisson mentioned this in rMEXTaa4876181ab8: Updated mediawiki/extensions Project: mediawiki/extensions/Flow….Apr 27 2015, 8:55 PM
gerritbot added a comment.Apr 29 2015, 8:36 PM

Change 207527 had a related patch set uploaded (by Catrope):
Make browser tests work with or without visual editor

https://gerrit.wikimedia.org/r/207527

gerritbot added a comment.Apr 29 2015, 8:39 PM

Change 207527 merged by jenkins-bot:
Make browser tests work with or without visual editor

https://gerrit.wikimedia.org/r/207527

Diffusion mentioned this in rMEXT473c2b7e65e8: Updated mediawiki/extensions Project: mediawiki/extensions/Flow….Apr 29 2015, 8:39 PM
Catrope mentioned this in rEFLWfbada3851b34: Make browser tests work with or without visual editor.Apr 29 2015, 8:40 PM
gerritbot added a comment.May 5 2015, 5:00 PM

Change 209000 had a related patch set uploaded (by Sbisson):
Fix browser tests

https://gerrit.wikimedia.org/r/209000

gerritbot added a comment.May 5 2015, 5:37 PM

Change 209000 merged by jenkins-bot:
Fix browser tests

https://gerrit.wikimedia.org/r/209000

SBisson mentioned this in rMEXT8ee426044a3e: Updated mediawiki/extensions Project: mediawiki/extensions/Flow….May 5 2015, 5:37 PM
SBisson mentioned this in rEFLWd52e857a055f: Fix browser tests.
gerritbot added a comment.May 5 2015, 9:25 PM

Change 209127 had a related patch set uploaded (by Sbisson):
Fix javascript detection on load

https://gerrit.wikimedia.org/r/209127

gerritbot added a comment.May 5 2015, 10:10 PM

Change 209127 merged by jenkins-bot:
Browser tests: fix javascript detection on load

https://gerrit.wikimedia.org/r/209127

SBisson mentioned this in rMEXT0daece5da32d: Updated mediawiki/extensions Project: mediawiki/extensions/Flow….May 5 2015, 10:11 PM
Catrope mentioned this in rEFLW561fa6eb4832: Browser tests: fix javascript detection on load.May 5 2015, 10:11 PM
gerritbot added a comment.May 6 2015, 3:19 PM

Change 209239 had a related patch set uploaded (by Sbisson):
Move tooltip to the right

https://gerrit.wikimedia.org/r/209239

gerritbot added a comment.May 6 2015, 3:27 PM

Change 209239 merged by jenkins-bot:
Move tooltip to the right

https://gerrit.wikimedia.org/r/209239

SBisson mentioned this in rEFLW0af330bda239: Move tooltip to the right.May 6 2015, 3:29 PM
SBisson mentioned this in rMEXTa92c54d97b2c: Updated mediawiki/extensions Project: mediawiki/extensions/Flow….
SBisson moved this task from In Development to Done on the Collaboration-Team-Sprint-A-2015-05-06 board.May 6 2015, 8:18 PM
• DannyH closed this task as Resolved.May 6 2015, 8:31 PM
• DannyH moved this task from Current workboard to Resolved on the Collaboration-Team-Triage board.May 6 2015, 10:41 PM
zeljkofilipin moved this task from Next to Done on the Browser-Tests-Infrastructure board.Jun 11 2015, 3:16 PM
zeljkofilipin renamed this task from A5. Fix failed Flow browsertests Jenkins jobs to Fix or delete failing Flow browsertests Jenkins jobs.Sep 25 2015, 1:39 PM
zeljkofilipin reopened this task as Open.
zeljkofilipin removed SBisson as the assignee of this task.
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin removed projects: Collaboration-Team-Sprint-A-2015-05-06, Patch-For-Review.
zeljkofilipin edited a custom field.
zeljkofilipin moved this task from Done to Waiting on the Browser-Tests-Infrastructure board.
zeljkofilipin mentioned this in T94150: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily.
zeljkofilipin added a subscriber: SBisson.
zeljkofilipin updated the task description. (Show Details)Sep 25 2015, 1:47 PM
zeljkofilipin lowered the priority of this task from High to Medium.Sep 25 2015, 2:25 PM
zeljkofilipin moved this task from Waiting to Next on the Browser-Tests-Infrastructure board.Sep 30 2015, 1:25 PM
SBisson added a comment.Sep 30 2015, 6:37 PM

The IE job can be deleted since it's never going to pass unless we recreate the IE browser factory in mediawiki_selenium.

The other jobs are passing once in a while. I pay close attention to the browser tests. I run them locally every morning and even though they are pretty much stable locally, they still fail for the most random reasons on Jenkins/Saucelabs.

I would like to keep them and see them green more often than not. Any help or advice you can provide would be appreciated.

zeljkofilipin updated the task description. (Show Details)Sep 30 2015, 7:06 PM
zeljkofilipin moved this task from Next to In Progress on the Browser-Tests-Infrastructure board.Oct 1 2015, 9:25 AM
zeljkofilipin claimed this task.Oct 1 2015, 9:40 AM
zeljkofilipin added a comment.Oct 1 2015, 1:46 PM

@SBisson: I will delete IE job until T114362 is fixed.

gerritbot added a comment.Oct 1 2015, 1:52 PM

Change 242887 had a related patch set uploaded (by Zfilipin):
Deleted failing Flow browsertests Jenkins job

https://gerrit.wikimedia.org/r/242887

gerritbot added a project: Patch-For-Review.Oct 1 2015, 1:52 PM
zeljkofilipin added a subtask: T114368: undefined method `last_session_ids=' for MediawikiSelenium::BrowserFactory::Chrome:Class (NoMethodError).Oct 1 2015, 2:06 PM
zeljkofilipin added a comment.Oct 1 2015, 2:17 PM

All other jobs are failing because of T114368.

zeljkofilipin added a comment.Oct 1 2015, 5:02 PM

@SBisson: as a first step, I would recommend not running scenarios that fail in Jenkins. When the jobs are back to green, we can debug the scenarios that fail. I can create a patch if you agree.

SBisson added a comment.Oct 1 2015, 5:10 PM
In T94153#1693894, @zeljkofilipin wrote:

@SBisson: as a first step, I would recommend not running scenarios that fail in Jenkins. When the jobs are back to green, we can debug the scenarios that fail. I can create a patch if you agree.

I have done a lot of work on those tests. I run them locally every morning and I haven't seen a failure in weeks. But if you can reproduce and fix some of the instability we see in jenkins, please do!

One thing I have seen quite often in jenkins is:
Exception Caught: Could not find local user data for New user 72425089784816018079612099150495189889@metawiki (internal_api_error_Exception) (MediawikiApi::ApiError)
We create new users all the time in the opt-in tests and sometimes it fails like this. I've never seen it locally.

gerritbot added a comment.Oct 5 2015, 12:39 PM

Change 242887 merged by jenkins-bot:
Deleted failing Flow browsertests Jenkins job

https://gerrit.wikimedia.org/r/242887

hashar mentioned this in rCICF2635d8452b63: Deleted failing Flow browsertests Jenkins job.Oct 5 2015, 12:39 PM
zeljkofilipin moved this task from In Progress to Next on the Browser-Tests-Infrastructure board.Oct 14 2015, 1:31 PM
zeljkofilipin closed subtask T114368: undefined method `last_session_ids=' for MediawikiSelenium::BrowserFactory::Chrome:Class (NoMethodError) as Resolved.Oct 24 2015, 4:11 PM
hashar closed subtask T92871: Parsoid patches don't update Beta Cluster automatically -- only deploy repo patches seem to update that code as Resolved.Oct 27 2015, 2:33 PM
hashar unsubscribed.
SBisson closed this task as Resolved.Nov 3 2015, 5:41 PM

Flow browser tests are being monitored on a daily basis for regression and instability. Quick changes are being done right away while bigger issues are being logged and prioritized. They are still failing often (not always) but they are getting better.

zeljkofilipin reopened this task as Open.Nov 4 2015, 4:04 PM

Hm, as far as I can see, all Flow browser test jobs are still failing:

https://integration.wikimedia.org/ci/view/BrowserTests/view/Flow/

zeljkofilipin removed zeljkofilipin as the assignee of this task.Nov 5 2015, 11:44 AM
zeljkofilipin removed a project: Patch-For-Review.
zeljkofilipin moved this task from Next to Ruby on the Browser-Tests-Infrastructure board.
• DannyH unsubscribed.Nov 6 2015, 12:49 AM
zeljkofilipin edited projects, added Browser-Tests; removed Browser-Tests-Infrastructure.Nov 9 2015, 2:43 PM
gerritbot added a comment.Feb 18 2016, 4:03 PM

Change 271539 had a related patch set uploaded (by Zfilipin):
Deleting @login Cucumber tag

https://gerrit.wikimedia.org/r/271539

gerritbot added a project: Patch-For-Review.Feb 18 2016, 4:03 PM
gerritbot added a comment.Feb 22 2016, 3:27 PM

Change 272471 had a related patch set uploaded (by Zfilipin):
Disable scenarios that fail at en.wikipedia.beta.wmflabs.org from running daily

https://gerrit.wikimedia.org/r/272471

zeljkofilipin claimed this task.Feb 22 2016, 3:30 PM
zeljkofilipin renamed this task from Fix or delete failing Flow browsertests Jenkins jobs to Disable Flow scenarios that fail at en.wikipedia.beta.wmflabs.org from running daily.Feb 22 2016, 3:39 PM
gerritbot added a comment.Feb 25 2016, 5:04 AM

Change 272471 merged by jenkins-bot:
Disable scenarios that fail at en.wikipedia.beta.wmflabs.org from running daily

https://gerrit.wikimedia.org/r/272471

ReleaseTaggerBot added a project: MW-1.27-release (WMF-deploy-2016-03-01_(1.27.0-wmf.15)).Feb 25 2016, 6:00 AM
zeljkofilipin renamed this task from Disable Flow scenarios that fail at en.wikipedia.beta.wmflabs.org from running daily to Fix Flow scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily.Feb 25 2016, 3:15 PM
gerritbot added a comment.Mar 2 2016, 10:19 AM

Change 274355 had a related patch set uploaded (by Zfilipin):
Disable scenarios that fail at en.wikipedia.beta.wmflabs.org from running daily

https://gerrit.wikimedia.org/r/274355

zeljkofilipin removed zeljkofilipin as the assignee of this task.Mar 10 2016, 2:11 PM
gerritbot added a comment.Mar 12 2016, 2:10 AM

Change 274355 merged by jenkins-bot:
Disable scenarios that fail at en.wikipedia.beta.wmflabs.org from running daily

https://gerrit.wikimedia.org/r/274355

gerritbot added a comment.Mar 15 2016, 1:04 PM

Change 277261 had a related patch set uploaded (by Zfilipin):
Followup 9eaa667: Use @skip instead so it's more obvious and greppable

https://gerrit.wikimedia.org/r/277261

ReleaseTaggerBot added a project: MW-1.27-release (WMF-deploy-2016-03-22_(1.27.0-wmf.18)).Mar 15 2016, 6:02 PM
zeljkofilipin claimed this task.Mar 16 2016, 12:34 PM
zeljkofilipin removed zeljkofilipin as the assignee of this task.Mar 21 2016, 11:21 AM
zeljkofilipin closed this task as Resolved.Mar 29 2016, 9:43 AM
zeljkofilipin claimed this task.
gerritbot added a comment.Apr 29 2016, 7:59 PM

Change 277261 abandoned by Mattflaschen:
Followup 9eaa667: Use @skip instead so it's more obvious and greppable

Reason:
Seems this approach is not the best solution for the current failures.

https://gerrit.wikimedia.org/r/277261

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits