|integration/config : master||Additionally run Wikidata browsertests without saucelabs|
|Invalid||None||T108946 [Epic] Improve the development infrastructure|
|Resolved||zeljkofilipin||T94150 Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily|
|Duplicate||None||T105985 Run Wikidata browser tests on testwikidata via Jenkins|
|Resolved||• Jonas||T88541 [DO NOT USE] Wikidata Browsertests (tracking) [superseded by #Browser-Tests]|
|Resolved||Tobi_WMDE_SW||T101497 [Story] Run browsertests regularly on test.wikidata.org via Jenkins|
|Resolved||JanZerebecki||T101498 [Task] Create a user with necessary privilegs for browsertests on test.wikidata.org|
|Resolved||Tobi_WMDE_SW||T101499 [Task] Setup a Jenkins job to run Wikidata browsertests on test.wikidata.org|
|Resolved||Tobi_WMDE_SW||T92619 make Wikidata browsertests non-flaky (tracking)|
|Resolved||JanZerebecki||T116166 move wikidata browsertests to not use saucelabs|
- Mentioned In
- T117561: selenium fails to connect to firefox (headless not sauce)
- Mentioned Here
- T115500: browsertest failure reports don't show the failing tests saucelabs link, but a different one
T117418: Ensure ChromeDriver is installed for jobs that run Selenium tests
T88288: Upstream: Issue with Chrome driver with resizing window
Yes that seems to disable saucelabs. But now I get "unable to obtain stable firefox connection in 60 seconds" as a failure for the second scenario:
After looking at the results running without saucelabs:
I filed: T117418: Ensure ChromeDriver is installed for jobs that run Selenium tests
The jobs run in about half the time.
It seems the runs have more failures than with saucelabs.
Is this the right link to see the results?
So it's mostly timeouts?
This sadly made me laugh:
Scenario: Edit sitelink Unexpected modal dialog (text: A script on this page may be busy, or it may have stopped responding. You can stop the script now, open the script in the debugger, or let the script continue.
A timeout will be also be reported when a link can not be found when the instruction was wait for that link to appear. So it is not that simple. Most of them seem not to be "A script on this page may be busy, or it may have stopped responding.".
Now the old jobs are switched back to saucelabs again and new jobs are created that do not use saucelabs. There are many different problems at the same time, so it is unclear if saucelabs bug is even a problem. Once we manage to get one of the jobs non-flaky, we'll revisit if we run them on saucelabs.