Run Selenium tests using Firefox
Open, Stalled, LowPublic

Description

Blocked on getting geckodriver installed in CI.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 29 2017, 11:31 AM

Change 345312 had a related patch set uploaded (by Zfilipin):
[mediawiki/core@master] WIP Run WebdriverIO tests using Firefox

https://gerrit.wikimedia.org/r/345312

zeljkofilipin changed the task status from Open to Stalled.Apr 5 2017, 2:11 PM

Change 345312 abandoned by Zfilipin:
WIP Run WebdriverIO tests using Firefox

https://gerrit.wikimedia.org/r/345312

zeljkofilipin removed zeljkofilipin as the assignee of this task.Nov 8 2017, 2:36 PM

Change 390233 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/core@master] WIP Upgrade WebdriverIO to 4.9.2

https://gerrit.wikimedia.org/r/390233

According to webdriverio/pull/1933#issuecomment-342816260 Firefox will run better with webdriverio 4.9.

We are currently using version 4.6.2. Our tests (in Chrome, did not try Firefox yet) work fine up until version 4.7.1. Since version 4.8.0 our tests break because wdio tries to open http://127.0.0.1:8080/index.php... instead of http://127.0.0.1:8080/w/index.php... (please notice missing /w).

/w is configured in wdio.conf.js

zeljkofilipin changed the task status from Stalled to Open.Nov 9 2017, 3:59 PM

Change 390268 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/core@master] WIP Run WebdriverIO tests using Firefox

https://gerrit.wikimedia.org/r/390268

5 out of 6 core tests fail with Unknown locator strategy id.

$ npm run selenium

> @ selenium /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki
> killall -0 geckodriver 2>/dev/null || geckodriver & grunt webdriver:test; killall geckodriver

1510243567959	geckodriver	INFO	geckodriver 0.19.1
1510243568028	geckodriver	INFO	Listening on 127.0.0.1:4444
Running "webdriver:test" (webdriver) task
1510243569973	mozrunner::runner	INFO	Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-profile" "/var/folders/5l/f437v7x97x3975w3c1_cv1240000gn/T/rust_mozprofile.1kpVqOZRX9Nx"
1510243570180	addons.xpi	WARN	Error parsing extensions state: [Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [amIAddonManagerStartup.readStartupData]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: loadExtensionState :: line 1554"  data: no] Stack trace: loadExtensionState()@resource://gre/modules/addons/XPIProvider.jsm:1554 < getInstallState()@resource://gre/modules/addons/XPIProvider.jsm:1589 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3109 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2188 < callProvider()@resource://gre/modules/AddonManager.jsm:269 < _startProvider()@resource://gre/modules/AddonManager.jsm:739 < startup()@resource://gre/modules/AddonManager.jsm:906 < startup()@resource://gre/modules/AddonManager.jsm:3090 < observe()@jar:file:///Applications/Firefox.app/Contents/Resources/omni.ja!/components/addonManager.js:65
1510243570516	Marionette	INFO	Enabled via --marionette
2017-11-09 17:06:11.336 plugin-container[4939:1331407] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0xa33b, name = 'com.apple.tsm.portname'
See /usr/include/servers/bootstrap_defs.h for the error codes.
2017-11-09 17:06:11.337 plugin-container[4939:1331407] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0xa703, name = 'com.apple.CFPasteboardClient'
See /usr/include/servers/bootstrap_defs.h for the error codes.
1510243571730	Marionette	INFO	Listening on port 53660
1510243571792	Marionette	DEBUG	Register listener.js for window 2147483649
1510243571841	Marionette	DEBUG	Received DOM event "beforeunload" for "about:blank"
1510243572220	Marionette	DEBUG	Received DOM event "pagehide" for "about:blank"
1510243572220	Marionette	DEBUG	Received DOM event "unload" for "about:blank"
1510243573298	Marionette	DEBUG	Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243574416	Marionette	DEBUG	Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243574444	Marionette	DEBUG	Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243574868	Marionette	DEBUG	Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243574868	Marionette	DEBUG	Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243576190	Marionette	DEBUG	Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit"
1510243576206	Marionette	DEBUG	Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit"
[17:06:16]  	Saved screenshot: ERROR_firefox_2017-11-09T16-06-16.251Z.png

	Screenshot location: ./log/should-be-creatable.png 

1510243578806	Marionette	DEBUG	Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit"
1510243579447	Marionette	DEBUG	Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit"
1510243579447	Marionette	DEBUG	Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=0.28228449522170407&action=edit"
1510243579502	Marionette	DEBUG	Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit"
1510243579620	Marionette	DEBUG	Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit"
[17:06:19]  	Saved screenshot: ERROR_firefox_2017-11-09T16-06-19.643Z.png

	Screenshot location: ./log/should-be-editable.png 

1510243581891	Marionette	DEBUG	Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit"
1510243582281	Marionette	DEBUG	Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit"
1510243582281	Marionette	DEBUG	Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=0.1191405809278292&action=edit"
1510243583111	Marionette	DEBUG	Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=0.8922329403749922&action=history"
1510243583118	Marionette	DEBUG	Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=0.8922329403749922&action=history"
1510243583312	Marionette	INFO	New connections will no longer be accepted
------------------------------------------------------------------
[firefox #0-0] Session ID: 43cc0488-b735-f540-a4c9-d144d5d59270
[firefox #0-0] Spec: /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/tests/selenium/specs/page.js
[firefox #0-0] Running: firefox
[firefox #0-0]
[firefox #0-0]   Page
[firefox #0-0]
[firefox #0-0]   Page
[firefox #0-0]       1) should be creatable
[firefox #0-0]       2) should be editable
[firefox #0-0]       ✓ should have history
[firefox #0-0]
[firefox #0-0]
[firefox #0-0] 1 passing (14s)
[firefox #0-0] 2 failing
[firefox #0-0]
[firefox #0-0] 1) Page should be creatable:
[firefox #0-0] Unknown locator strategy id
[firefox #0-0] Error: The arguments passed to a command are either invalid or malformed.
[firefox #0-0]     at element("#wpTextbox1") - index.js:423:3
[firefox #0-0]
[firefox #0-0] 2) Page should be editable:
[firefox #0-0] Unknown locator strategy id
[firefox #0-0] Error: The arguments passed to a command are either invalid or malformed.
[firefox #0-0]     at element("#wpTextbox1") - index.js:423:3
[firefox #0-0]

1510243584061	mozrunner::runner	INFO	Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-profile" "/var/folders/5l/f437v7x97x3975w3c1_cv1240000gn/T/rust_mozprofile.r4yEgFXVztHg"
1510243584269	addons.xpi	WARN	Error parsing extensions state: [Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [amIAddonManagerStartup.readStartupData]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: loadExtensionState :: line 1554"  data: no] Stack trace: loadExtensionState()@resource://gre/modules/addons/XPIProvider.jsm:1554 < getInstallState()@resource://gre/modules/addons/XPIProvider.jsm:1589 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3109 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2188 < callProvider()@resource://gre/modules/AddonManager.jsm:269 < _startProvider()@resource://gre/modules/AddonManager.jsm:739 < startup()@resource://gre/modules/AddonManager.jsm:906 < startup()@resource://gre/modules/AddonManager.jsm:3090 < observe()@jar:file:///Applications/Firefox.app/Contents/Resources/omni.ja!/components/addonManager.js:65
1510243584619	Marionette	INFO	Enabled via --marionette
2017-11-09 17:06:25.469 plugin-container[4946:1332260] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0xa23b, name = 'com.apple.tsm.portname'
See /usr/include/servers/bootstrap_defs.h for the error codes.
2017-11-09 17:06:25.470 plugin-container[4946:1332260] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0xa503, name = 'com.apple.CFPasteboardClient'
See /usr/include/servers/bootstrap_defs.h for the error codes.
1510243585906	Marionette	INFO	Listening on port 53744
1510243585984	Marionette	DEBUG	Register listener.js for window 2147483649
1510243586025	Marionette	DEBUG	Received DOM event "beforeunload" for "about:blank"
1510243586421	Marionette	DEBUG	Received DOM event "pagehide" for "about:blank"
1510243586422	Marionette	DEBUG	Received DOM event "unload" for "about:blank"
1510243587572	Marionette	DEBUG	Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243588577	Marionette	DEBUG	Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243588608	Marionette	DEBUG	Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243588998	Marionette	DEBUG	Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243588998	Marionette	DEBUG	Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243589728	Marionette	DEBUG	Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount"
1510243589735	Marionette	DEBUG	Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount"
[17:06:29]  	Saved screenshot: ERROR_firefox_2017-11-09T16-06-29.763Z.png

	Screenshot location: ./log/should-be-able-to-create-account.png 

1510243591083	Marionette	DEBUG	Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount"
1510243591481	Marionette	DEBUG	Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount"
1510243591481	Marionette	DEBUG	Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=Special:CreateAccount"
1510243591513	Marionette	DEBUG	Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243591599	Marionette	DEBUG	Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
[17:06:31]  	Saved screenshot: ERROR_firefox_2017-11-09T16-06-31.620Z.png

	Screenshot location: ./log/should-be-able-to-log-in.png 

1510243592839	Marionette	DEBUG	Received DOM event "beforeunload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243593338	Marionette	DEBUG	Received DOM event "pagehide" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243593338	Marionette	DEBUG	Received DOM event "unload" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243593364	Marionette	DEBUG	Received DOM event "DOMContentLoaded" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
1510243593455	Marionette	DEBUG	Received DOM event "pageshow" for "http://127.0.0.1:8080/w/index.php?title=Special:UserLogin"
[17:06:33]  	Saved screenshot: ERROR_firefox_2017-11-09T16-06-33.476Z.png

	Screenshot location: ./log/should-be-able-to-change-preferences.png 

1510243593780	Marionette	INFO	New connections will no longer be accepted
------------------------------------------------------------------
[firefox #0-1] Session ID: 13c48aa7-9881-e14e-9aae-b145a07bdda3
[firefox #0-1] Spec: /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/tests/selenium/specs/user.js
[firefox #0-1] Running: firefox
[firefox #0-1]
[firefox #0-1]   User
[firefox #0-1]
[firefox #0-1]   User
[firefox #0-1]       1) should be able to create account
[firefox #0-1]       2) should be able to log in
[firefox #0-1]       3) should be able to change preferences
[firefox #0-1]
[firefox #0-1]
[firefox #0-1] 3 failing (10s)
[firefox #0-1]
[firefox #0-1] 1) User should be able to create account:
[firefox #0-1] Unknown locator strategy id
[firefox #0-1] Error: The arguments passed to a command are either invalid or malformed.
[firefox #0-1]     at element("#wpName2") - index.js:423:3
[firefox #0-1]
[firefox #0-1] 2) User should be able to log in:
[firefox #0-1] Unknown locator strategy id
[firefox #0-1] Error: The arguments passed to a command are either invalid or malformed.
[firefox #0-1]     at element("#wpName1") - index.js:423:3
[firefox #0-1]
[firefox #0-1] 3) User should be able to change preferences:
[firefox #0-1] Unknown locator strategy id
[firefox #0-1] Error: The arguments passed to a command are either invalid or malformed.
[firefox #0-1]     at element("#wpName1") - index.js:423:3
[firefox #0-1]

Warning: Task "webdriver:test" failed. Use --force to continue.

Aborted due to warnings.
sh: line 1:  4936 Terminated: 15          geckodriver

Change 390268 abandoned by Zfilipin:
WIP Run WebdriverIO tests using Firefox

Reason:
just a test

https://gerrit.wikimedia.org/r/390268

Change 390233 abandoned by Zfilipin:
WIP Upgrade WebdriverIO to 4.9.2

https://gerrit.wikimedia.org/r/390233

zeljkofilipin renamed this task from Run WebdriverIO tests using Firefox to Run Selenium tests using Firefox.May 29 2018, 5:21 PM