Blocked on getting geckodriver installed in CI.
Description
Description
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | zeljkofilipin | T182986 Selenium framework improvements | |||
| Resolved | zeljkofilipin | T190995 Someday/maybe Selenium framework improvements | |||
| Declined | None | T161697 Run Selenium tests using Firefox | |||
| Declined | None | T183163 Docker container with geckodriver |
Event Timeline
Comment Actions
Change 345312 had a related patch set uploaded (by Zfilipin):
[mediawiki/core@master] WIP Run WebdriverIO tests using Firefox
Comment Actions
Looks like this is now implemented: https://github.com/webdriverio/webdriverio/pull/1933#issuecomment-341566193
Comment Actions
Change 390233 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/core@master] WIP Upgrade WebdriverIO to 4.9.2
Comment Actions
According to webdriverio/pull/1933#issuecomment-342816260 Firefox will run better with webdriverio 4.9.
We are currently using version 4.6.2. Our tests (in Chrome, did not try Firefox yet) work fine up until version 4.7.1. Since version 4.8.0 our tests break because wdio tries to open http://127.0.0.1:8080/index.php... instead of http://127.0.0.1:8080/w/index.php... (please notice missing /w).
/w is configured in [[ https://phabricator.wikimedia.org/source/mediawiki/browse/master/tests/selenium/wdio.conf.js;858067fec39ffb88371fc936a40e4f88e0606ec4$123 | wdio.conf.js ]]
Comment Actions
Change 390268 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/core@master] WIP Run WebdriverIO tests using Firefox
Comment Actions
5 out of 6 core tests fail with Unknown locator strategy id.
$ npm run selenium > @ selenium /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki > killall -0 geckodriver 2>/dev/null || geckodriver & grunt webdriver:test; killall geckodriver 1510243567959 geckodriver INFO geckodriver 0.19.1 1510243568028 geckodriver INFO Listening on 127.0.0.1:4444 Running "webdriver:test" (webdriver) task 1510243569973 mozrunner::runner INFO Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-profile" "/var/folders/5l/f437v7x97x3975w3c1_cv1240000gn/T/rust_mozprofile.1kpVqOZRX9Nx" 1510243570180 addons.xpi WARN Error parsing extensions state: [Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [amIAddonManagerStartup.readStartupData]" nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)" location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: loadExtensionState :: line 1554" data: no] Stack trace: loadExtensionState()@resource://gre/modules/addons/XPIProvider.jsm:1554 < getInstallState()@resource://gre/modules/addons/XPIProvider.jsm:1589 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3109 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2188 < callProvider()@resource://gre/modules/AddonManager.jsm:269 < _startProvider()@resource://gre/modules/AddonManager.jsm:739 < startup()@resource://gre/modules/AddonManager.jsm:906 < startup()@resource://gre/modules/AddonManager.jsm:3090 < observe()@jar:file:///Applications/Firefox.app/Contents/Resources/omni.ja!/components/addonManager.js:65 1510243570516 Marionette INFO Enabled via --marionette 2017-11-09 17:06:11.336 plugin-container[4939:1331407] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0xa33b, name = 'com.apple.tsm.portname' See /usr/include/servers/bootstrap_defs.h for the error codes. 2017-11-09 17:06:11.337 plugin-container[4939:1331407] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0xa703, name = 'com.apple.CFPasteboardClient' See /usr/include/servers/bootstrap_defs.h for the error codes. 1510243571730 Marionette INFO Listening on port 53660 1510243571792 Marionette DEBUG Register listener.js for window 2147483649 1510243571841 Marionette DEBUG Received DOM event "beforeunload" for "about:blank" 1510243572220 Marionette DEBUG Received DOM event "pagehide" for "about:blank" 1510243572220 Marionette DEBUG Received DOM event "unload" for "about:blank" 1510243573298 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243574416 Marionette DEBUG Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243574444 Marionette DEBUG Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243574868 Marionette DEBUG Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243574868 Marionette DEBUG Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243576190 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit" 1510243576206 Marionette DEBUG Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit" [17:06:16] Saved screenshot: ERROR_firefox_2017-11-09T16-06-16.251Z.png Screenshot location: ./log/should-be-creatable.png 1510243578806 Marionette DEBUG Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit" 1510243579447 Marionette DEBUG Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit" 1510243579447 Marionette DEBUG Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit" 1510243579502 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit" 1510243579620 Marionette DEBUG Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit" [17:06:19] Saved screenshot: ERROR_firefox_2017-11-09T16-06-19.643Z.png Screenshot location: ./log/should-be-editable.png 1510243581891 Marionette DEBUG Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit" 1510243582281 Marionette DEBUG Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit" 1510243582281 Marionette DEBUG Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit" 1510243583111 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=0.8922329403749922&action=history" 1510243583118 Marionette DEBUG Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=0.8922329403749922&action=history" 1510243583312 Marionette INFO New connections will no longer be accepted ------------------------------------------------------------------ [firefox #0-0] Session ID: 43cc0488-b735-f540-a4c9-d144d5d59270 [firefox #0-0] Spec: /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/tests/selenium/specs/page.js [firefox #0-0] Running: firefox [firefox #0-0] [firefox #0-0] Page [firefox #0-0] [firefox #0-0] Page [firefox #0-0] 1) should be creatable [firefox #0-0] 2) should be editable [firefox #0-0] ✓ should have history [firefox #0-0] [firefox #0-0] [firefox #0-0] 1 passing (14s) [firefox #0-0] 2 failing [firefox #0-0] [firefox #0-0] 1) Page should be creatable: [firefox #0-0] Unknown locator strategy id [firefox #0-0] Error: The arguments passed to a command are either invalid or malformed. [firefox #0-0] at element("#wpTextbox1") - index.js:423:3 [firefox #0-0] [firefox #0-0] 2) Page should be editable: [firefox #0-0] Unknown locator strategy id [firefox #0-0] Error: The arguments passed to a command are either invalid or malformed. [firefox #0-0] at element("#wpTextbox1") - index.js:423:3 [firefox #0-0] 1510243584061 mozrunner::runner INFO Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-profile" "/var/folders/5l/f437v7x97x3975w3c1_cv1240000gn/T/rust_mozprofile.r4yEgFXVztHg" 1510243584269 addons.xpi WARN Error parsing extensions state: [Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [amIAddonManagerStartup.readStartupData]" nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)" location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: loadExtensionState :: line 1554" data: no] Stack trace: loadExtensionState()@resource://gre/modules/addons/XPIProvider.jsm:1554 < getInstallState()@resource://gre/modules/addons/XPIProvider.jsm:1589 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3109 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2188 < callProvider()@resource://gre/modules/AddonManager.jsm:269 < _startProvider()@resource://gre/modules/AddonManager.jsm:739 < startup()@resource://gre/modules/AddonManager.jsm:906 < startup()@resource://gre/modules/AddonManager.jsm:3090 < observe()@jar:file:///Applications/Firefox.app/Contents/Resources/omni.ja!/components/addonManager.js:65 1510243584619 Marionette INFO Enabled via --marionette 2017-11-09 17:06:25.469 plugin-container[4946:1332260] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0xa23b, name = 'com.apple.tsm.portname' See /usr/include/servers/bootstrap_defs.h for the error codes. 2017-11-09 17:06:25.470 plugin-container[4946:1332260] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0xa503, name = 'com.apple.CFPasteboardClient' See /usr/include/servers/bootstrap_defs.h for the error codes. 1510243585906 Marionette INFO Listening on port 53744 1510243585984 Marionette DEBUG Register listener.js for window 2147483649 1510243586025 Marionette DEBUG Received DOM event "beforeunload" for "about:blank" 1510243586421 Marionette DEBUG Received DOM event "pagehide" for "about:blank" 1510243586422 Marionette DEBUG Received DOM event "unload" for "about:blank" 1510243587572 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243588577 Marionette DEBUG Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243588608 Marionette DEBUG Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243588998 Marionette DEBUG Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243588998 Marionette DEBUG Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243589728 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount" 1510243589735 Marionette DEBUG Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount" [17:06:29] Saved screenshot: ERROR_firefox_2017-11-09T16-06-29.763Z.png Screenshot location: ./log/should-be-able-to-create-account.png 1510243591083 Marionette DEBUG Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount" 1510243591481 Marionette DEBUG Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount" 1510243591481 Marionette DEBUG Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount" 1510243591513 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243591599 Marionette DEBUG Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" [17:06:31] Saved screenshot: ERROR_firefox_2017-11-09T16-06-31.620Z.png Screenshot location: ./log/should-be-able-to-log-in.png 1510243592839 Marionette DEBUG Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243593338 Marionette DEBUG Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243593338 Marionette DEBUG Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243593364 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" 1510243593455 Marionette DEBUG Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin" [17:06:33] Saved screenshot: ERROR_firefox_2017-11-09T16-06-33.476Z.png Screenshot location: ./log/should-be-able-to-change-preferences.png 1510243593780 Marionette INFO New connections will no longer be accepted ------------------------------------------------------------------ [firefox #0-1] Session ID: 13c48aa7-9881-e14e-9aae-b145a07bdda3 [firefox #0-1] Spec: /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/tests/selenium/specs/user.js [firefox #0-1] Running: firefox [firefox #0-1] [firefox #0-1] User [firefox #0-1] [firefox #0-1] User [firefox #0-1] 1) should be able to create account [firefox #0-1] 2) should be able to log in [firefox #0-1] 3) should be able to change preferences [firefox #0-1] [firefox #0-1] [firefox #0-1] 3 failing (10s) [firefox #0-1] [firefox #0-1] 1) User should be able to create account: [firefox #0-1] Unknown locator strategy id [firefox #0-1] Error: The arguments passed to a command are either invalid or malformed. [firefox #0-1] at element("#wpName2") - index.js:423:3 [firefox #0-1] [firefox #0-1] 2) User should be able to log in: [firefox #0-1] Unknown locator strategy id [firefox #0-1] Error: The arguments passed to a command are either invalid or malformed. [firefox #0-1] at element("#wpName1") - index.js:423:3 [firefox #0-1] [firefox #0-1] 3) User should be able to change preferences: [firefox #0-1] Unknown locator strategy id [firefox #0-1] Error: The arguments passed to a command are either invalid or malformed. [firefox #0-1] at element("#wpName1") - index.js:423:3 [firefox #0-1] Warning: Task "webdriver:test" failed. Use --force to continue. Aborted due to warnings. sh: line 1: 4936 Terminated: 15 geckodriver
Comment Actions
Change 390268 abandoned by Zfilipin:
WIP Run WebdriverIO tests using Firefox
Reason:
just a test
Comment Actions
- geckodriver/issues/140 Location strategies by ID and by name #140
- webdriverio/issues/2392 Locate elements by id is not supported in Geckodriver #2392
Comment Actions
Declining to reflect that there is no effort to add Selenium tests for Firefox. A few years ago Firefox required the addition of geckodriver (similar to chrome driver) but it was unpackaged in Debian and required some build step using cargo/rust, that was deemed to be too much maintenance effort. There is also an extension that had to be added which required a Firefox developer edition in order to be able to install a 3rd party extension not signed by Mozilla.