The same Wikidata tests are failing for days now: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/282/
Failing Scenarios: cucumber tests/browser/features/aliases.feature:158 # Scenario: Save duplicated aliases cucumber tests/browser/features/sitelinks.feature:36 # Scenario: Type site id cucumber tests/browser/features/sitelinks.feature:183 # Scenario: Add multiple sitelinks cucumber tests/browser/features/sitelinks.feature:195 # Scenario: Remove multiple sitelinks cucumber tests/browser/features/sitelinks.feature:228 # Scenario: Add new sitelink to already referenced site cucumber tests/browser/features/special_set_label.feature:12 # Scenario: Special:SetLabel page has all required elements cucumber tests/browser/features/special_set_label.feature:31 # Scenario: Add a label cucumber tests/browser/features/special_set_label.feature:43 # Scenario: Edit an existing label cucumber tests/browser/features/special_set_label.feature:54 # Scenario: Edit using an invalid language fails cucumber tests/browser/features/special_set_label.feature:64 # Scenario: Edit using an invalid ID fails
Some are still printing these hilarious "Uh-oh. Could not find link to Sauce Labs job URL."
The Special:SetLabel tests look like they are actually broken, which may be the case because https://gerrit.wikimedia.org/r/233423 actually changed the HTML of this page. But I'm entirely unable to do anything about this. I just wasted hours trying to run any test. I tried to run the test locally the same way I always did, but all I get is an empty Firefox window that does nothing. I followed https://phabricator.wikimedia.org/diffusion/EWBA/browse/master/tests/browser/README.md and https://github.com/wikimedia/mediawiki-selenium step by step but all I get is an empty Firefox window that does nothing.
This is by far not the first time I'm running into such problems. The test framework appears so fragile, brittle and impossible to debug, I find this unacceptable.