Page MenuHomePhabricator

Update mediawiki_selenium to use Marionette
Closed, DeclinedPublic

Details

Related Gerrit Patches:
mediawiki/selenium : masterWIP Marionette

Event Timeline

Restricted Application added subscribers: Zppix, Aklapper. ยท View Herald TranscriptJun 10 2016, 11:24 AM
zeljkofilipin triaged this task as High priority.Jun 11 2016, 8:06 AM
zeljkofilipin lowered the priority of this task from High to Medium.Jun 14 2016, 2:19 PM
zeljkofilipin removed zeljkofilipin as the assignee of this task.Jul 7 2016, 11:38 AM

Updating mediawiki_selenium to support Mozilla Marionette was a priority due to Firefox 47.0 breaking webdriver. 47.0.1 fix it though:

Firefox 47.0.1 has been released on June 28th https://www.mozilla.org/en-US/firefox/47.0.1/releasenotes/

Fixed: Selenium WebDriver may cause Firefox to crash at startup

It is not yet in Ubuntu Trusty though.

Marionette support is definitely needed down the road, but it is not much of a priority right now.

zeljkofilipin moved this task from Gems to In Progress on the Browser-Tests-Infrastructure board.

Change 310286 had a related patch set uploaded (by Zfilipin):
WIP Marionette

https://gerrit.wikimedia.org/r/310286

zeljkofilipin added a comment.EditedSep 20 2016, 12:21 PM

Firefox 48.0.2
mediawiki_selenium 1.7.2 + 310286
watir 6.0.0.beta4
selenium-webdriver 3.0.0.beta3.1
Marionette/geckodriver

$ bundle exec cucumber tests/browser/features/create_account.feature:10
/usr/local/lib/ruby/gems/2.3.0/gems/page-object-1.2.0/lib/page-object.rb:13:in `require': `require "watir-webdriver"` is deprecated. Please, use `require "watir"`.
/usr/local/lib/ruby/gems/2.3.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/atoms.rb:4: warning: already initialized constant Watir::Atoms::ATOMS
/usr/local/lib/ruby/gems/2.3.0/gems/watir-6.0.0.beta4/lib/watir/atoms.rb:4: warning: previous definition of ATOMS was here
@chrome @firefox @vagrant
Feature: Create account

  Scenario Outline: Go to Create account page   # tests/browser/features/create_account.feature:4
    Given I go to Create account page at <path> # tests/browser/features/step_definitions/create_account_steps.rb:1
    Then form has Create account button         # tests/browser/features/step_definitions/create_account_steps.rb:5

    Examples: 
      | path                  |
      | Special:CreateAccount |
      unknown option: {:profile=>#<Selenium::WebDriver::Firefox::Profile:0x007fff0dbaab80 @model=nil, @native_events=false, @secure_ssl=false, @untrusted_issuer=true, @load_no_focus_lib=false, @additional_prefs={"browser.startup.homepage_override.mstone"=>"ignore"}, @extensions={}>} (ArgumentError)
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:230:in `new'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:230:in `new_browser'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:186:in `new_browser_for'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/browser_factory/base.rb:140:in `browser_for'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/environment.rb:179:in `browser'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/cucumber/embed_browser_session.rb:13:in `browser'
      /Users/z/Documents/gerrit/mediawiki/selenium/lib/mediawiki_selenium/page_factory.rb:25:in `on_page'
      tests/browser/features/create_account.feature:5:in `Given I go to Create account page at <path>'

Failing Scenarios:
cucumber tests/browser/features/create_account.feature:4 # Scenario: Go to Create account page

1 scenario (1 failed)
2 steps (1 failed, 1 skipped)
0m0.548s
zeljkofilipin added a comment.EditedSep 20 2016, 2:49 PM

Explodes in base.rb:

def new_browser(options)
  Watir::Browser.new(browser_name, options)
end
[3] pry(#<MediawikiSelenium::BrowserFactory::Firefox>)> browser_name
=> :firefox

[4] pry(#<MediawikiSelenium::BrowserFactory::Firefox>)> options
=> {:http_client=>#<Selenium::WebDriver::Remote::Http::Default:0x007fe753246fc0 @timeout=nil>,
 :desired_capabilities=>
  #<Selenium::WebDriver::Remote::W3CCapabilities:0x007fe753246de0
   @capabilities=
    {:browser_name=>"firefox",
     :browser_version=>nil,
     :platform_name=>nil,
     :platform_version=>:any,
     :accept_ssl_certs=>false,
     :page_load_strategy=>"normal",
     :proxy=>nil,
     :name=>"Create account: Go to Create account page: | Special:CreateAccount |"}>,
 :profile=>
  #<Selenium::WebDriver::Firefox::Profile:0x007fe753247150
   @additional_prefs={"browser.startup.homepage_override.mstone"=>"ignore"},
   @extensions={},
   @load_no_focus_lib=false,
   @model=nil,
   @native_events=false,
   @secure_ssl=false,
   @untrusted_issuer=true>}

Firefox 47.0.1
mediawiki_selenium 1.7.2
watir-webdriver 0.9.3
selenium-webdriver 2.53.4
firefox driver

$ bundle exec cucumber tests/browser/features/create_account.feature:10
@chrome @firefox @vagrant
Feature: Create account

  Scenario Outline: Go to Create account page   # tests/browser/features/create_account.feature:4
    Given I go to Create account page at <path> # tests/browser/features/step_definitions/create_account_steps.rb:1
    Then form has Create account button         # tests/browser/features/step_definitions/create_account_steps.rb:5

    Examples: 
      | path                  |
      | Special:CreateAccount |

1 scenario (1 passed)
2 steps (2 passed)
0m18.474s

firefoxOptions is the way to go, but is a feature of not yet released geckodriver 0.11.

Firefox 47.0.1
mediawiki_selenium 1.7.2
watir-webdriver 0.9.3
selenium-webdriver 2.53.4
firefox driver

$ bundle exec rake selenium
/usr/local/Cellar/ruby/2.3.1/bin/ruby -S bundle exec cucumber tests/browser --tags @firefox
...
20 scenarios (20 passed)
103 steps (103 passed)
3m43.872s

Firefox 48.0.2
mediawiki_selenium 1.7.2 + 310286
watir 6.0.0.beta4
selenium-webdriver 3.0.0.beta3.1
Marionette/geckodriver

$ bundle exec rake selenium
/usr/local/Cellar/ruby/2.3.1/bin/ruby -S bundle exec cucumber tests/browser --tags @firefox
...
20 scenarios (8 failed, 12 passed)
103 steps (8 failed, 9 skipped, 86 passed)
4m32.233s

I don't think Marionette/geckodriver is ready yet. Setting the profile has changed. I have disabled profiles for this test and Firefox runs, but still 40% of the current test suite (8/20 scenarios) fail. I will continue working on this when geckodriver 0.11 is released.

zeljkofilipin removed zeljkofilipin as the assignee of this task.Sep 21 2016, 3:24 PM
zeljkofilipin moved this task from In Progress to Waiting on the Browser-Tests-Infrastructure board.

Change 310286 abandoned by Zfilipin:
WIP Marionette

Reason:
just a test

https://gerrit.wikimedia.org/r/310286

zeljkofilipin closed this task as Declined.May 26 2017, 4:30 PM
zeljkofilipin lowered the priority of this task from Medium to Low.

Unlikely to ever be resolved because of T139740: Port Selenium tests from Ruby to Node.js.