Page MenuHomePhabricator

browsertests under phantomjs have failures on Jenkins
Closed, DeclinedPublic


I have setup up browsertests in Wikimedia Jenkins setup (bug 54388). The first run console using phantomjs browser dies miserably with some "Unable to pick a platform for the provided browser (RuntimeError)" errors.

Full output:

48 scenarios (25 failed, 23 skipped)

Version: wmf-deployment
Severity: normal



Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:12 AM
bzimport set Reference to bz54489.
bzimport added a subscriber: Unknown Object (MLST).

Blocks bug 53697 "trigger browser tests from Gerrit" since it would be nice to have tests passing :-]

As far as I understand it, console says phantomjs is not installed. For example:

03:47:27 Unable to find phantomjs executable. (Selenium::WebDriver::Error::WebDriverError)

I forgot to install phantomjs during my crazy night.

Next issue:

unable to connect to phantomjs @
after 20 seconds (Selenium::WebDriver::Error::WebDriverError

That is supposed to be the port of the phantomjs Selenium driver

vendor/ruby/1.9.1/gems/selenium-webdriver-2.35.0 \

... /lib/selenium/webdriver/phantomjs/service.rb

12: DEFAULT_PORT = 8910

The timeout is caused by {{bug|45868}}, that is, instances cant not access the public IP address. One has to rewrite outgoing packets using iptables rules as a workaround:

iptables -t nat -I OUTPUT --dest -j DNAT --to-dest
iptables -t nat -I OUTPUT --dest -j DNAT --to-dest
iptables -t nat -I OUTPUT --dest -j DNAT --to-dest
iptables -t nat -I OUTPUT --dest -j DNAT --to-dest
iptables -t nat -I OUTPUT --dest -j DNAT --to-dest

No more timeout:

We know have some issues such as:

timed out after 5 seconds, waiting for {:text=>"Yes", :tag_name=>"span"} to become present (Watir::Wait::TimeoutError)

Error Message => 'Can't find variable: mw'

Željko told me it is because the pages do not look the same under phantomJS, thus the tests would need to be fixed :-)

Following a pair session with Željko, the tests can be flagged with a tag which we can then exclude when running tests. For ULS, I have introduced the tag @specific-settings which let us skip any tests that are not going to pass on a fresh wiki installation: was for ULS. This bug is for qa/browsertest.

The idea was that whenever a change is proposed (or merged?) for qa/browsertests, we would have run the whole test suites using phantomJS on the WMF Jenkins.

Lot of tests being migrated to extensions I don't think it make sense anymore. We might want to repurpose qa/browsertest as smoke tests targeting production which would be quite useful for monitoring.

So either wont fix or resolve later on :-D

In that case, resolved wontfix. We can open new bugs for specific repositories if needed.