Page MenuHomePhabricator

Update mediawiki_selenium to use Marionette
Closed, DeclinedPublic

Event Timeline

zeljkofilipin lowered the priority of this task from High to Medium.Jun 14 2016, 2:19 PM

Updating mediawiki_selenium to support Mozilla Marionette was a priority due to Firefox 47.0 breaking webdriver. 47.0.1 fix it though:

Firefox 47.0.1 has been released on June 28th https://www.mozilla.org/en-US/firefox/47.0.1/releasenotes/

Fixed: Selenium WebDriver may cause Firefox to crash at startup

It is not yet in Ubuntu Trusty though.

Marionette support is definitely needed down the road, but it is not much of a priority right now.

Change 310286 had a related patch set uploaded (by Zfilipin):
WIP Marionette

https://gerrit.wikimedia.org/r/310286

Firefox 48.0.2
mediawiki_selenium 1.7.2 + 310286
watir 6.0.0.beta4
selenium-webdriver 3.0.0.beta3.1
Marionette/geckodriver

$ bundle exec cucumber tests/browser/features/create_account.feature:10
/usr/local/lib/ruby/gems/2.3.0/gems/page-object-1.2.0/lib/page-object.rb:13:in `require': `require "watir-webdriver"` is deprecated. Please, use `require "watir"`.
/usr/local/lib/ruby/gems/2.3.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/atoms.rb:4: warning: already initialized constant Watir::Atoms::ATOMS
/usr/local/lib/ruby/gems/2.3.0/gems/watir-6.0.0.beta4/lib/watir/atoms.rb:4: warning: previous definition of ATOMS was here
@chrome @firefox @vagrant
Feature: Create account

  Scenario Outline: Go to Create account page   # tests/browser/features/create_account.feature:4
    Given I go to Create account page at <path> # tests/browser/features/step_definitions/create_account_steps.rb:1
    Then form has Create account button         # tests/browser/features/step_definitions/create_account_steps.rb:5

    Examples: 
      | path                  |
      | Special:CreateAccount |
      unknown option: {:profile=>#<Selenium::WebDriver::Firefox::Profile:0x007fff0dbaab80 @model=nil, @native_events=false, @secure_ssl=false, @untrusted_issuer=true, @load_no_focus_lib=false, @additional_prefs={"browser.startup.homepage_override.mstone"=>"ignore"}, @extensions={}>} (ArgumentError)
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:230:in `new'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:230:in `new_browser'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:186:in `new_browser_for'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:140:in `browser_for'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/environment.rb:179:in `browser'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/cucumber/embed_browser_session.rb:13:in `browser'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/page_factory.rb:25:in `on_page'
      tests/browser/features/create_account.feature:5:in `Given I go to Create account page at <path>'

Failing Scenarios:
cucumber tests/browser/features/create_account.feature:4 # Scenario: Go to Create account page

1 scenario (1 failed)
2 steps (1 failed, 1 skipped)
0m0.548s

Explodes in [[ https://phabricator.wikimedia.org/diffusion/MSEL/browse/master/lib/mediawiki_selenium/browser_factory/base.rb;c5034d361b34d9a61f138ea07121b487b3a841ce$230 | base.rb ]]:

def new_browser(options)
  Watir::Browser.new(browser_name, options)
end
[3] pry(#<MediawikiSelenium::BrowserFactory::Firefox>)> browser_name
=> :firefox

[4] pry(#<MediawikiSelenium::BrowserFactory::Firefox>)> options
=> {:http_client=>#<Selenium::WebDriver::Remote::Http::Default:0x007fe753246fc0 @timeout=nil>,
 :desired_capabilities=>
  #<Selenium::WebDriver::Remote::W3CCapabilities:0x007fe753246de0
   @capabilities=
    {:browser_name=>"firefox",
     :browser_version=>nil,
     :platform_name=>nil,
     :platform_version=>:any,
     :accept_ssl_certs=>false,
     :page_load_strategy=>"normal",
     :proxy=>nil,
     :name=>"Create account: Go to Create account page: | Special:CreateAccount |"}>,
 :profile=>
  #<Selenium::WebDriver::Firefox::Profile:0x007fe753247150
   @additional_prefs={"browser.startup.homepage_override.mstone"=>"ignore"},
   @extensions={},
   @load_no_focus_lib=false,
   @model=nil,
   @native_events=false,
   @secure_ssl=false,
   @untrusted_issuer=true>}

Firefox 47.0.1
mediawiki_selenium 1.7.2
watir-webdriver 0.9.3
selenium-webdriver 2.53.4
firefox driver

$ bundle exec cucumber tests/browser/features/create_account.feature:10
@chrome @firefox @vagrant
Feature: Create account

  Scenario Outline: Go to Create account page   # tests/browser/features/create_account.feature:4
    Given I go to Create account page at <path> # tests/browser/features/step_definitions/create_account_steps.rb:1
    Then form has Create account button         # tests/browser/features/step_definitions/create_account_steps.rb:5

    Examples: 
      | path                  |
      | Special:CreateAccount |

1 scenario (1 passed)
2 steps (2 passed)
0m18.474s

[[ https://github.com/mozilla/geckodriver#firefox-capabilities | firefoxOptions ]] is the way to go, but is a feature of not yet released geckodriver 0.11.

Firefox 47.0.1
mediawiki_selenium 1.7.2
watir-webdriver 0.9.3
selenium-webdriver 2.53.4
firefox driver

$ bundle exec rake selenium
/usr/local/Cellar/ruby/2.3.1/bin/ruby -S bundle exec cucumber tests/browser --tags @firefox
...
20 scenarios (20 passed)
103 steps (103 passed)
3m43.872s

Firefox 48.0.2
mediawiki_selenium 1.7.2 + 310286
watir 6.0.0.beta4
selenium-webdriver 3.0.0.beta3.1
Marionette/geckodriver

$ bundle exec rake selenium
/usr/local/Cellar/ruby/2.3.1/bin/ruby -S bundle exec cucumber tests/browser --tags @firefox
...
20 scenarios (8 failed, 12 passed)
103 steps (8 failed, 9 skipped, 86 passed)
4m32.233s

I don't think Marionette/geckodriver is ready yet. Setting the profile has changed. I have disabled profiles for this test and Firefox runs, but still 40% of the current test suite (8/20 scenarios) fail. I will continue working on this when geckodriver 0.11 is released.

Change 310286 abandoned by Zfilipin:
WIP Marionette

Reason:
just a test

https://gerrit.wikimedia.org/r/310286

zeljkofilipin lowered the priority of this task from Medium to Low.

Unlikely to ever be resolved because of T139740: Port Selenium tests from Ruby to Node.js.