From Firefox 47 Selenium tests are broken. More information:
Description
Details
Subject | Repo | Branch | Lines +/- | |
---|---|---|---|---|
WIP Marionette | mediawiki/selenium | master | +4 -1 |
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Duplicate | None | T138095 Cucumber tests won't run locally on firefox 47 | |||
Resolved | zeljkofilipin | T137561 Firefox v47 breaks mediawiki_selenium | |||
Declined | None | T137540 Update mediawiki_selenium to use Marionette |
Event Timeline
Updating mediawiki_selenium to support Mozilla Marionette was a priority due to Firefox 47.0 breaking webdriver. 47.0.1 fix it though:
Marionette support is definitely needed down the road, but it is not much of a priority right now.
Firefox 48.0.2
mediawiki_selenium 1.7.2 + 310286
watir 6.0.0.beta4
selenium-webdriver 3.0.0.beta3.1
Marionette/geckodriver
$ bundle exec cucumber tests/browser/features/create_account.feature:10 /usr/local/lib/ruby/gems/2.3.0/gems/page-object-1.2.0/lib/page-object.rb:13:in `require': `require "watir-webdriver"` is deprecated. Please, use `require "watir"`. /usr/local/lib/ruby/gems/2.3.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/atoms.rb:4: warning: already initialized constant Watir::Atoms::ATOMS /usr/local/lib/ruby/gems/2.3.0/gems/watir-6.0.0.beta4/lib/watir/atoms.rb:4: warning: previous definition of ATOMS was here @chrome @firefox @vagrant Feature: Create account Scenario Outline: Go to Create account page # tests/browser/features/create_account.feature:4 Given I go to Create account page at <path> # tests/browser/features/step_definitions/create_account_steps.rb:1 Then form has Create account button # tests/browser/features/step_definitions/create_account_steps.rb:5 Examples: | path | | Special:CreateAccount | unknown option: {:profile=>#<Selenium::WebDriver::Firefox::Profile:0x007fff0dbaab80 @model=nil, @native_events=false, @secure_ssl=false, @untrusted_issuer=true, @load_no_focus_lib=false, @additional_prefs={"browser.startup.homepage_override.mstone"=>"ignore"}, @extensions={}>} (ArgumentError) /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:230:in `new' /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:230:in `new_browser' /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:186:in `new_browser_for' /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:140:in `browser_for' /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/environment.rb:179:in `browser' /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/cucumber/embed_browser_session.rb:13:in `browser' /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/page_factory.rb:25:in `on_page' tests/browser/features/create_account.feature:5:in `Given I go to Create account page at <path>' Failing Scenarios: cucumber tests/browser/features/create_account.feature:4 # Scenario: Go to Create account page 1 scenario (1 failed) 2 steps (1 failed, 1 skipped) 0m0.548s
Explodes in [[ https://phabricator.wikimedia.org/diffusion/MSEL/browse/master/lib/mediawiki_selenium/browser_factory/base.rb;c5034d361b34d9a61f138ea07121b487b3a841ce$230 | base.rb ]]:
def new_browser(options) Watir::Browser.new(browser_name, options) end
[3] pry(#<MediawikiSelenium::BrowserFactory::Firefox>)> browser_name => :firefox [4] pry(#<MediawikiSelenium::BrowserFactory::Firefox>)> options => {:http_client=>#<Selenium::WebDriver::Remote::Http::Default:0x007fe753246fc0 @timeout=nil>, :desired_capabilities=> #<Selenium::WebDriver::Remote::W3CCapabilities:0x007fe753246de0 @capabilities= {:browser_name=>"firefox", :browser_version=>nil, :platform_name=>nil, :platform_version=>:any, :accept_ssl_certs=>false, :page_load_strategy=>"normal", :proxy=>nil, :name=>"Create account: Go to Create account page: | Special:CreateAccount |"}>, :profile=> #<Selenium::WebDriver::Firefox::Profile:0x007fe753247150 @additional_prefs={"browser.startup.homepage_override.mstone"=>"ignore"}, @extensions={}, @load_no_focus_lib=false, @model=nil, @native_events=false, @secure_ssl=false, @untrusted_issuer=true>}
Firefox 47.0.1
mediawiki_selenium 1.7.2
watir-webdriver 0.9.3
selenium-webdriver 2.53.4
firefox driver
$ bundle exec cucumber tests/browser/features/create_account.feature:10 @chrome @firefox @vagrant Feature: Create account Scenario Outline: Go to Create account page # tests/browser/features/create_account.feature:4 Given I go to Create account page at <path> # tests/browser/features/step_definitions/create_account_steps.rb:1 Then form has Create account button # tests/browser/features/step_definitions/create_account_steps.rb:5 Examples: | path | | Special:CreateAccount | 1 scenario (1 passed) 2 steps (2 passed) 0m18.474s
[[ https://github.com/mozilla/geckodriver#firefox-capabilities | firefoxOptions ]] is the way to go, but is a feature of not yet released geckodriver 0.11.
Firefox 47.0.1
mediawiki_selenium 1.7.2
watir-webdriver 0.9.3
selenium-webdriver 2.53.4
firefox driver
$ bundle exec rake selenium /usr/local/Cellar/ruby/2.3.1/bin/ruby -S bundle exec cucumber tests/browser --tags @firefox ... 20 scenarios (20 passed) 103 steps (103 passed) 3m43.872s
Firefox 48.0.2
mediawiki_selenium 1.7.2 + 310286
watir 6.0.0.beta4
selenium-webdriver 3.0.0.beta3.1
Marionette/geckodriver
$ bundle exec rake selenium /usr/local/Cellar/ruby/2.3.1/bin/ruby -S bundle exec cucumber tests/browser --tags @firefox ... 20 scenarios (8 failed, 12 passed) 103 steps (8 failed, 9 skipped, 86 passed) 4m32.233s
I don't think Marionette/geckodriver is ready yet. Setting the profile has changed. I have disabled profiles for this test and Firefox runs, but still 40% of the current test suite (8/20 scenarios) fail. I will continue working on this when geckodriver 0.11 is released.
Unlikely to ever be resolved because of T139740: Port Selenium tests from Ruby to Node.js.