Ensure ChromeDriver is installed for jobs that run Selenium tests
Closed, ResolvedPublic

Description

Make the selenium driver for chrome available.

Unable to find the chromedriver executable. Please download the server from http://chromedriver.storage.googleapis.com/index.html and place it somewhere on your PATH. More info at https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver. (Selenium::WebDriver::Error::WebDriverError)
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/selenium-webdriver-2.48.0/lib/selenium/webdriver/chrome/service.rb:37:in `executable_path'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/selenium-webdriver-2.48.0/lib/selenium/webdriver/chrome/service.rb:50:in `default_service'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/selenium-webdriver-2.48.0/lib/selenium/webdriver/chrome/bridge.rb:33:in `initialize'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/selenium-webdriver-2.48.0/lib/selenium/webdriver/common/driver.rb:60:in `new'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/selenium-webdriver-2.48.0/lib/selenium/webdriver/common/driver.rb:60:in `for'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/selenium-webdriver-2.48.0/lib/selenium/webdriver.rb:86:in `for'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/watir-webdriver-0.9.0/lib/watir-webdriver/browser.rb:46:in `initialize'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/mediawiki_selenium-1.5.0/lib/mediawiki_selenium/browser_factory/base.rb:226:in `new'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/mediawiki_selenium-1.5.0/lib/mediawiki_selenium/browser_factory/base.rb:226:in `new_browser'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/mediawiki_selenium-1.5.0/lib/mediawiki_selenium/browser_factory/base.rb:186:in `new_browser_for'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/mediawiki_selenium-1.5.0/lib/mediawiki_selenium/browser_factory/base.rb:140:in `browser_for'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/mediawiki_selenium-1.5.0/lib/mediawiki_selenium/environment.rb:157:in `browser'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/mediawiki_selenium-1.5.0/lib/mediawiki_selenium/support/modules/headless_helper.rb:71:in `browser'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/mediawiki_selenium-1.5.0/lib/mediawiki_selenium/page_factory.rb:25:in `on_page'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/page-object-1.1.0/lib/page-object/page_factory.rb:53:in `visit_page'
/mnt/jenkins-workspace/workspace/browsertests-Wikidata-WikidataTests-linux-chrome-sauce/tests/browser/features/step_definitions/entity_steps.rb:42:in `block in <top (required)>'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_step_definition.rb:97:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/step_match.rb:25:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:60:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_world.rb:52:in `step'
/mnt/jenkins-workspace/workspace/browsertests-Wikidata-WikidataTests-linux-chrome-sauce/tests/browser/features/step_definitions/entity_steps.rb:23:in `block in <top (required)>'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_step_definition.rb:97:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/step_match.rb:25:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:60:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_world.rb:52:in `step'
/mnt/jenkins-workspace/workspace/browsertests-Wikidata-WikidataTests-linux-chrome-sauce/tests/browser/features/step_definitions/entity_steps.rb:18:in `block in <top (required)>'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_step_definition.rb:97:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/step_match.rb:25:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:60:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:38:in `accept'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:106:in `block in visit_step'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:105:in `visit_step'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:19:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `each'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `accept'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:100:in `block in visit_steps'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:99:in `visit_steps'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/background.rb:44:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/background.rb:54:in `block in with_visitor'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:79:in `with_visitor'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/background.rb:53:in `with_visitor'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/background.rb:41:in `accept'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:64:in `block in visit_background'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:63:in `visit_background'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:36:in `accept'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:27:in `block in visit_feature'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:26:in `visit_feature'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:28:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:27:in `accept'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:20:in `visit_features'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:49:in `run!'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/lib/cucumber/cli/main.rb:47:in `execute!'
/mnt/home/jenkins-deploy/.gem/2.0.0/gems/cucumber-1.3.20/bin/cucumber:13:in `<top (required)>'
/mnt/home/jenkins-deploy/.gem/2.0.0/bin/cucumber:23:in `load'
/mnt/home/jenkins-deploy/.gem/2.0.0/bin/cucumber:23:in `<main>'
features/aliases.feature:12:in `Given I am on an item page'
JanZerebecki updated the task description. (Show Details)
JanZerebecki raised the priority of this task from to Needs Triage.
JanZerebecki added a subscriber: JanZerebecki.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 2 2015, 2:12 PM

I was sure we already had this task, but I can not find it now. I think we already have Chromium available, but we need to let Selenium know where it is:

https://github.com/watir/watirbook/blob/master/manuscript/installation/ubuntu.md#chromium

This should be implemented in mediawiki_selenium gem.

zeljkofilipin triaged this task as Normal priority.Nov 2 2015, 2:55 PM
zeljkofilipin set Security to None.
zeljkofilipin moved this task from Ruby to Gems on the Browser-Tests-Infrastructure board.
zeljkofilipin added subscribers: hashar, dduvall.
Krinkle renamed this task from make the selenium driver for chrome available to Ensure chromedriver is installed (for Selenium).Nov 17 2016, 2:52 AM
Krinkle added a subscriber: Krinkle.

The CI slaves have it via puppet modules/contint/manifests/browsers.pp

Distro PackageBinary
Debianchromedriver/usr/lib/chromium/chromedriver
Ubuntuchromium-chromedriver/usr/lib/chromium-browser/chromedriver

On Nodepool:

Jessie
ii chromedriver 53.0.2785.143 amd64 web browser - WebDriver support
ii chromium 53.0.2785.143 amd64 web browser

@hashar Looks like we probably just need to ensure mw-set-env-mw-selenium.sh#L10 is applied to the new selenium jobs. See https://gerrit.wikimedia.org/r/#/c/320829/5.

Maybe move them to a more central place, or add it to those jobs?

# Selenium requires the chromedriver binary to be found in our PATH
for path in /usr/lib/chromium-browser /usr/lib/chromium; do
	if test -d "$path"; then
		export PATH="$PATH:$path"
	fi
done

Good catch Timo :] I was wondering this morning how the ruby based jobs manage to find chromedriver, we have the answer. The script is invoked for all the selenium based jobs:

mediawiki-selenium-integration
mediawiki-selenium-integration-jessie
mwext-mw-selenium
mwext-mw-selenium-composer
mwext-mw-selenium-composer-jessie
mwext-mw-selenium-jessie

Maybe we can have it loaded for all the npm jobs (not sure about all the impacts that could have).

zeljkofilipin moved this task from Gems to Next on the Browser-Tests-Infrastructure board.
zeljkofilipin moved this task from Backlog to Next on the User-zeljkofilipin board.
zeljkofilipin claimed this task.
zeljkofilipin moved this task from Next to In Progress on the Browser-Tests-Infrastructure board.

The chrome drivers are installed in Puppet via modules/contint/manifests/browsers.pp I have no idea why there are no links in /usr/bin, maybe we can get puppet to generate symbolic links in /usr/local/bin and done! :]

zeljkofilipin added a comment.EditedNov 29 2016, 2:35 PM

My puppet fu is not strong. I have found this on creating symlinks:

https://www.puppetcookbook.com/posts/creating-a-symlink.html

class symlinker {
  file { '/tmp/link-to-motd':
    ensure => 'link',
    target => '/etc/motd',
  }
}

I was looking at the operations/puppet repository for usage patterns and I could not find that symlinker is used anywhere, so I guess that is not the way to go. :|

Looks like this is the way to go:

https://docs.puppet.com/puppet/latest/reference/types/file.html

file { '/etc/inetd.conf':
  ensure => link,
  target => '/etc/inet/inetd.conf',
}
zeljkofilipin renamed this task from Ensure chromedriver is installed (for Selenium) to Ensure ChromeDriver is installed for jobs that run Selenium tests.Nov 29 2016, 3:29 PM

Change 324203 had a related patch set uploaded (by Zfilipin):
WIP Ensure ChromeDriver is installed for jobs that run Selenium tests

https://gerrit.wikimedia.org/r/324203

$ ssh integration-saltmaster.integration.eqiad.wmflabs 
hashar@integration-saltmaster:~$ sudo su -
root@integration-saltmaster:~# salt -v '*' cmd.run 'which chromedriver'
Executing job with jid 20161130103426631702
-------------------------------------------

integration-slave-precise-1012.integration.eqiad.wmflabs:
integration-slave-jessie-android.integration.eqiad.wmflabs:
integration-slave-jessie-1001.integration.eqiad.wmflabs:
    /usr/local/bin/chromedriver
castor.integration.eqiad.wmflabs:
integration-slave-jessie-1002.integration.eqiad.wmflabs:
    /usr/local/bin/chromedriver
integration-slave-docker-1000.integration.eqiad.wmflabs:
integration-puppetmaster01.integration.eqiad.wmflabs:
integration-slave-trusty-1004.integration.eqiad.wmflabs:
    /usr/local/bin/chromedriver
integration-slave-precise-1002.integration.eqiad.wmflabs:
integration-slave-trusty-1003.integration.eqiad.wmflabs:
    /usr/local/bin/chromedriver
integration-slave-trusty-1006.integration.eqiad.wmflabs:
    /usr/local/bin/chromedriver
integration-publisher.integration.eqiad.wmflabs:
buildlog.integration.eqiad.wmflabs:
integration-slave-trusty-1001.integration.eqiad.wmflabs:
    /usr/local/bin/chromedriver
integration-slave-precise-1011.integration.eqiad.wmflabs:
integration-slave-trusty-1011.integration.eqiad.wmflabs:
    /usr/local/bin/chromedriver
repository.integration.eqiad.wmflabs:
integration-saltmaster.integration.eqiad.wmflabs:
root@integration-saltmaster:~# 

Change 324203 merged by Alexandros Kosiaris:
ChromeDriver should be in PATH for jobs that run Selenium tests

https://gerrit.wikimedia.org/r/324203

zeljkofilipin closed this task as Resolved.Dec 7 2016, 2:09 PM

The Nodepool Jessie image had the patch included. I am refreshing the Trusty one.

Mentioned in SAL (#wikimedia-releng) [2016-12-07T15:04:42Z] <hashar> Image ci-trusty-wikimedia-1481122712 in wmflabs-eqiad is ready T117418