Page MenuHomePhabricator

Investigate causes of failures of daily ruby browser tests targetting beta
Closed, ResolvedPublic5 Story Points

Description

Investigate the different reasons for T208851: Rewrite Ruby browser tests as Javascsript browser tests, then create more subtasks to address them individually. (Hopefully, there shouldn’t be 80 different errors.)

Event Timeline

Michael claimed this task.Jan 2 2019, 2:07 PM
Restricted Application added a project: User-Michael. · View Herald TranscriptJan 2 2019, 2:07 PM

Currently, it seems like all test failures have the same error message:

Uh-oh. Could not find link to Sauce Labs job URL.

See, for example the job #443.

That is confusing to me - wasn't the idea to not use SauceLabs for these tests anymore, but to run them on Jenkins?

good catch. Please ignore this message, it is an artefact from the days when tests were run using sauce labs. The same message is shown also in passing tests, I believe.
Looking at the job 443 you used as an example, the first failing tests is https://integration.wikimedia.org/ci/view/Selenium/job/selenium-Wikibase-chrome/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/443/testReport/junit/(root)/Using%20time%20properties%20in%20statements/Check_UI_for_invalid_values__outline_example_____32_12_2015___/
In this case, the actual error message is:

timed out after 30 seconds (Watir::Wait::TimeoutError)

Based on the current job #451:

  • The save has failed. (failed-save) (MediawikiApi::ApiError)
    • about 70 failed tests
  • timed out after 15 seconds, Element was not visible in 15 seconds (Watir::Wait::TimeoutError)
    • about 8 failed tests
  • Assertation failures: (RSpec::Expectations::ExpectationNotMetError)
    • about 5 failed tests (in the geo related tests)
Lucas_Werkmeister_WMDE closed this task as Resolved.Jan 16 2019, 5:53 PM

I think we can resolve this – the investigation looks good to me, and we can pick up the resulting tasks as part of the sprint.