Page MenuHomePhabricator

Fix failed Wikidata browsertests Jenkins job
Closed, DuplicatePublic

Event Timeline

The Wikidata-WikidataTests one got aborted due to a timeout. It has been raised from 3 to 4 hours with https://gerrit.wikimedia.org/r/#/c/199919/ .

The SmokeTests one passed yesterday but failed with:

Scenario: Edit an existing label
Could not create a new page.
It already exists. (failed-save) (MediawikiApi::ApiError)

So it seems we can keep both :)

Ok, first, please do not remove the Wikidata jobs, even if they are failing. There are several reasons:

  1. Due to the high number of scenarios we have, there are almost every time a few tests that fail due to timeout or because of a problem with beta. Only if a scenario fails more than once in a row, we are concerned and take a detailed look. We could discuss of splitting the over-3-hours job into several smaller ones, but this is another topic.
  2. sometimes when we add tests that actually fail because of a bug in the software. So this scenario will continue to fail until we fix the bug in the software (if this is in one of our components that have releases, it will only be fixed after we released a new version).
  3. We are not ignoring failures at all and we are actively working on improving our browsertests. But sometimes we just don't have the resources to look at broken things immediately, so it could occur from time to time that jobs will fail for some time because they are broken.

At the moment we have indeed some problems with falsely failing scenarios - all are somehow related to how our selectors (snaktype-, rank-, and badgeselector) work. We will probably not be able to look closely at those issues before Easter but we will definitely have a look once we have the resources again.

So, no reason to delete those jobs, they are extremely helpful also with some failing scenarios. If you don't want them to be reported to the QA-list or the releng-channel, we could remove these notifications. Our team will be notified by email in any case.

Ok, Wikidata team obviously cares about browser tests, we will not delete any jobs. :) Feel fee to reach out to Release-Engineering-Team team if you need help. Also, I have proposed Workshop: Fix broken browsertests/Selenium Jenkins jobs (T94299) for this year's hackathons.

zeljkofilipin renamed this task from Delete or fix failed Wikidata browsertests Jenkins job to Fix failed Wikidata browsertests Jenkins job.Mar 30 2015, 1:54 PM
zeljkofilipin set Security to None.