Page MenuHomePhabricator

Investigate causes of failures of daily ruby browser tests targetting beta
Closed, ResolvedPublic5 Estimated Story Points


Investigate the different reasons for T208851: Rewrite Ruby browser tests as Javascsript browser tests, then create more subtasks to address them individually. (Hopefully, there shouldn’t be 80 different errors.)

Event Timeline

Currently, it seems like all test failures have the same error message:

Uh-oh. Could not find link to Sauce Labs job URL.

See, for example the job #443.

That is confusing to me - wasn't the idea to not use SauceLabs for these tests anymore, but to run them on Jenkins?

good catch. Please ignore this message, it is an artefact from the days when tests were run using sauce labs. The same message is shown also in passing tests, I believe.
Looking at the job 443 you used as an example, the first failing tests is,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/443/testReport/junit/(root)/Using%20time%20properties%20in%20statements/Check_UI_for_invalid_values__outline_example_____32_12_2015___/
In this case, the actual error message is:

timed out after 30 seconds (Watir::Wait::TimeoutError)

Based on the current job #451:

  • The save has failed. (failed-save) (MediawikiApi::ApiError)
    • about 70 failed tests
  • timed out after 15 seconds, Element was not visible in 15 seconds (Watir::Wait::TimeoutError)
    • about 8 failed tests
  • Assertation failures: (RSpec::Expectations::ExpectationNotMetError)
    • about 5 failed tests (in the geo related tests)

I think we can resolve this – the investigation looks good to me, and we can pick up the resulting tasks as part of the sprint.