Page MenuHomePhabricator

[Bug] SSL failure doesn't cause browser test to be marked as failed
Open, LowestPublic

Description

Steps to reproduce

Expected results

  • The test should be marked as failed.

Environments observed

See the following:

[0]
21:25:59 [BrowserTests] $ /bin/bash -xe /tmp/jenkins4215198959961511203.sh
21:25:59 ++ ruby -e 'print RbConfig::CONFIG["ruby_version"]'
21:25:59 + RUBY_VERSION=2.1.0
21:25:59 + export GEM_HOME=/mnt/home/jenkins-deploy/.gem/2.1.0
21:25:59 + GEM_HOME=/mnt/home/jenkins-deploy/.gem/2.1.0
21:25:59 + bundle install --verbose
21:25:59 Using rake 10.5.0
21:25:59 0:  rake (10.5.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rake-10.5.0.gemspec
21:25:59 Using ast 2.3.0
21:25:59 0:  ast (2.3.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/ast-2.3.0.gemspec
21:25:59 Using builder 3.2.2
21:25:59 0:  builder (3.2.2) from /usr/share/rubygems-integration/all/specifications/builder-3.2.2.gemspec
21:25:59 Using ffi 1.9.10
21:25:59 0:  ffi (1.9.10) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/ffi-1.9.10.gemspec
21:25:59 Using childprocess 0.5.9
21:25:59 0:  childprocess (0.5.9) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/childprocess-0.5.9.gemspec
21:25:59 Using chunky_png 1.3.5
21:25:59 0:  chunky_png (1.3.5) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/chunky_png-1.3.5.gemspec
21:25:59 Using diff-lcs 1.2.5
21:25:59 0:  diff-lcs (1.2.5) from /usr/share/rubygems-integration/all/specifications/diff-lcs-1.2.5.gemspec
21:25:59 Using multi_json 1.12.1
21:25:59 0:  multi_json (1.12.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/multi_json-1.12.1.gemspec
21:25:59 Using gherkin 2.12.2
21:25:59 0:  gherkin (2.12.2) from /usr/share/rubygems-integration/2.1/specifications/gherkin-2.12.2.gemspec
21:25:59 Using multi_test 0.1.2
21:25:59 0:  multi_test (0.1.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/multi_test-0.1.2.gemspec
21:25:59 Using cucumber 1.3.20
21:25:59 0:  cucumber (1.3.20) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/cucumber-1.3.20.gemspec
21:25:59 Using i18n 0.7.0
21:25:59 0:  i18n (0.7.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/i18n-0.7.0.gemspec
21:25:59 Using faker 1.6.3
21:25:59 0:  faker (1.6.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/faker-1.6.3.gemspec
21:25:59 Using yml_reader 0.7
21:25:59 0:  yml_reader (0.7) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/yml_reader-0.7.gemspec
21:25:59 Using data_magic 0.22
21:25:59 0:  data_magic (0.22) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/data_magic-0.22.gemspec
21:25:59 Using unf_ext 0.0.7.2
21:25:59 0:  unf_ext (0.0.7.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/unf_ext-0.0.7.2.gemspec
21:25:59 Using unf 0.1.4
21:25:59 0:  unf (0.1.4) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/unf-0.1.4.gemspec
21:25:59 Using domain_name 0.5.20161129
21:25:59 0:  domain_name (0.5.20161129) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/domain_name-0.5.20161129.gemspec
21:25:59 Using multipart-post 2.0.0
21:25:59 0:  multipart-post (2.0.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/multipart-post-2.0.0.gemspec
21:25:59 Using faraday 0.10.0
21:25:59 0:  faraday (0.10.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/faraday-0.10.0.gemspec
21:25:59 Using http-cookie 1.0.3
21:25:59 0:  http-cookie (1.0.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/http-cookie-1.0.3.gemspec
21:25:59 Using faraday-cookie_jar 0.0.6
21:25:59 0:  faraday-cookie_jar (0.0.6) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/faraday-cookie_jar-0.0.6.gemspec
21:25:59 Using faraday_middleware 0.10.1
21:25:59 0:  faraday_middleware (0.10.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/faraday_middleware-0.10.1.gemspec
21:25:59 Using headless 2.3.1
21:25:59 0:  headless (2.3.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/headless-2.3.1.gemspec
21:25:59 Using json 1.8.3
21:25:59 0:  json (1.8.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/json-1.8.3.gemspec
21:25:59 Using mediawiki_api 0.7.0
21:25:59 0:  mediawiki_api (0.7.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/mediawiki_api-0.7.0.gemspec
21:25:59 Using page_navigation 0.9
21:25:59 0:  page_navigation (0.9) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/page_navigation-0.9.gemspec
21:25:59 Using rubyzip 1.2.1
21:25:59 0:  rubyzip (1.2.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rubyzip-1.2.1.gemspec
21:25:59 Using websocket 1.2.3
21:25:59 0:  websocket (1.2.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/websocket-1.2.3.gemspec
21:25:59 Using selenium-webdriver 2.53.0
21:25:59 0:  selenium-webdriver (2.53.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/selenium-webdriver-2.53.0.gemspec
21:25:59 Using watir-webdriver 0.9.1
21:25:59 0:  watir-webdriver (0.9.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/watir-webdriver-0.9.1.gemspec
21:25:59 Using page-object 1.1.0
21:25:59 0:  page-object (1.1.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/page-object-1.1.0.gemspec
21:25:59 Using mime-types 2.99.3
21:25:59 0:  mime-types (2.99.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/mime-types-2.99.3.gemspec
21:25:59 Using netrc 0.11.0
21:25:59 0:  netrc (0.11.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/netrc-0.11.0.gemspec
21:25:59 Using rest-client 1.8.0
21:25:59 0:  rest-client (1.8.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rest-client-1.8.0.gemspec
21:25:59 Using rspec-core 2.99.2
21:25:59 0:  rspec-core (2.99.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rspec-core-2.99.2.gemspec
21:25:59 Using rspec-expectations 2.99.2
21:25:59 0:  rspec-expectations (2.99.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rspec-expectations-2.99.2.gemspec
21:25:59 Using syntax 1.2.1
21:25:59 0:  syntax (1.2.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/syntax-1.2.1.gemspec
21:25:59 Using thor 0.19.4
21:25:59 0:  thor (0.19.4) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/thor-0.19.4.gemspec
21:25:59 Using mediawiki_selenium 1.7.3
21:25:59 0:  mediawiki_selenium (1.7.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/mediawiki_selenium-1.7.3.gemspec
21:25:59 Using parallel 1.12.0
21:25:59 0:  parallel (1.12.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/parallel-1.12.0.gemspec
21:25:59 Using parser 2.4.0.2
21:25:59 0:  parser (2.4.0.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/parser-2.4.0.2.gemspec
21:25:59 Using powerpack 0.1.1
21:25:59 0:  powerpack (0.1.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/powerpack-0.1.1.gemspec
21:25:59 Using rainbow 2.2.2
21:25:59 0:  rainbow (2.2.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rainbow-2.2.2.gemspec
21:25:59 Using ruby-progressbar 1.9.0
21:25:59 0:  ruby-progressbar (1.9.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/ruby-progressbar-1.9.0.gemspec
21:25:59 Using unicode-display_width 1.3.0
21:25:59 0:  unicode-display_width (1.3.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/unicode-display_width-1.3.0.gemspec
21:25:59 Using rubocop 0.51.0
21:25:59 0:  rubocop (0.51.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rubocop-0.51.0.gemspec
21:25:59 Using bundler 1.7.4
21:25:59 0:  bundler (1.7.4) from /usr/share/rubygems-integration/all/specifications/bundler-1.7.4.gemspec
21:25:59 Your bundle is complete!

Stack trace

[1]
21:27:10   @smoke @integration
21:27:10   Scenario: Closing editor (overlay button)               # tests/browser/features/editor_wikitext_nosave.feature:11
21:28:18     When I click the wikitext editor overlay close button # tests/browser/features/step_definitions/editor_steps.rb:15
21:28:18     Then I should not see the wikitext editor overlay     # tests/browser/features/step_definitions/editor_steps.rb:31
21:28:18       hostname "saucelabs.com" does not match the server certificate (OpenSSL::SSL::SSLError)
21:28:18       /usr/lib/ruby/2.1.0/openssl/ssl.rb:232:in `post_connection_check'
21:28:18       /usr/lib/ruby/2.1.0/net/http.rb:922:in `connect'
21:28:18       /usr/lib/ruby/2.1.0/net/http.rb:1447:in `begin_transport'
21:28:18       /usr/lib/ruby/2.1.0/net/http.rb:1404:in `transport_request'
21:28:18       /usr/lib/ruby/2.1.0/net/http.rb:1378:in `request'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/rest-client-1.8.0/lib/restclient/request.rb:270:in `net_http_do_request'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/rest-client-1.8.0/lib/restclient/request.rb:418:in `block in transmit'
21:28:18       /usr/lib/ruby/2.1.0/net/http.rb:853:in `start'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/rest-client-1.8.0/lib/restclient/request.rb:413:in `transmit'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/rest-client-1.8.0/lib/restclient/request.rb:176:in `execute'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/rest-client-1.8.0/lib/restclient/request.rb:41:in `execute'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/remote_browser_factory.rb:58:in `block in teardown'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/browser_factory/base.rb:174:in `each'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/browser_factory/base.rb:174:in `each'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/remote_browser_factory.rb:53:in `teardown'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/environment.rb:455:in `block in teardown'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/environment.rb:449:in `each'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/environment.rb:449:in `teardown'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/helpers/screenshot_helper.rb:37:in `teardown'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/cucumber/hooks.rb:69:in `block in <top (required)>'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_hook.rb:14:in `invoke'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:114:in `invoke'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:108:in `block in execute_after'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:107:in `reverse_each'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:107:in `execute_after'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:19:in `after'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:112:in `block in fire_hook'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:111:in `each'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:111:in `fire_hook'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:113:in `after'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:100:in `before_and_after'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:82:in `block in with_hooks'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:120:in `call'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:120:in `block (3 levels) in around'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:9:in `block in around'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:97:in `call'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:97:in `execute_around'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:8:in `around'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:119:in `block (2 levels) in around'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:123:in `call'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:123:in `around'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:94:in `around'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:81:in `with_hooks'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:13:in `execute'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:32:in `block in accept'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:79:in `with_visitor'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:31:in `accept'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:58:in `block in visit_feature_element'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:57:in `visit_feature_element'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:38:in `block in accept'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:37:in `each'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:37:in `accept'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:27:in `block in visit_feature'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:26:in `visit_feature'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:28:in `block in accept'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:27:in `accept'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:20:in `visit_features'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:49:in `run!'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/cli/main.rb:47:in `execute!'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/bin/cucumber:13:in `<top (required)>'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `load'
21:28:18       /mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `<main>'

Event Timeline

phuedx created this task.Jan 17 2019, 2:39 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 17 2019, 2:39 PM
phuedx updated the task description. (Show Details)Jan 17 2019, 2:40 PM
phuedx updated the task description. (Show Details)
hashar added a subscriber: hashar.
hostname "saucelabs.com" does not match the server certificate (OpenSSL::SSL::SSLError)

From the stacktrace that comes from:

lib/mediawiki_selenium/remote_browser_factory.rb
module MediawikiSelenium

    # Submits status and Jenkins build info to Sauce Labs.
    #
    def teardown(env, status)
      artifacts = super

      each do |browser|
        sid = browser.driver.session_id
        username = env.lookup(:sauce_ondemand_username)
        key = env.lookup(:sauce_ondemand_access_key)

        RestClient::Request.execute(
          method: :put,
          url: "https://saucelabs.com/rest/v1/#{username}/jobs/#{sid}",
          user: username,
          password: key,
          headers: { content_type: 'application/json' },
          payload: {
            public: true,
            passed: status == :passed,
            build: env.lookup(:build_number, default: nil)
          }.to_json
        )

        artifacts["http://saucelabs.com/jobs/#{sid}"] = 'text/url'
      end

      artifacts
    end

The test already passed, I am assuming that a fault in the teardown function does not cause the test to then be flagged as failing. The end result is that the Saucelabs build report would lack the job status and the build artifacts.

Doing a grep in the console logs for SSLError in all builds of selenium-* jobs gives me:

selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/lastFailedBuild/log
selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/793/log
selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/805/log
selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/796/log
selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/761/log
selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/800/log
selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/804/log
selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/794/log
selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/lastUnsuccessfulBuild/log
selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/802/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/lastFailedBuild/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/793/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/805/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/796/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/761/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/800/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/804/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/794/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/lastUnsuccessfulBuild/log
selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/802/log

Somehow there is something wrong with MinervaNeue? Maybe using a later version of mediawiki-selenium gem would better hide the problem (if not resolve it).

zeljkofilipin triaged this task as Lowest priority.Jan 17 2019, 3:14 PM
zeljkofilipin added a subscriber: zeljkofilipin.

Ruby framework was deprecated in October 2017, Blog Post: Selenium Ruby framework deprecated. I'm probably one of the most familiar with the framework, but I don't have the time to debug this. 😢