Page MenuHomePhabricator

Figure how to get WebDriver to spoof browser geolocation preferences
Closed, DeclinedPublic

Description

Author: jhall

Description:
The Mobile team (specifically Jon Robson) would like to add a browser test that can spoof a specific geolocation that we know will returns result on the "Nearby" page (https://en.wikipedia.org/wiki/Special:Nearby)

An initial patch for such a test is in Gerrit: https://gerrit.wikimedia.org/r/#/c/92018/

At least for Firefox, the necessary "about:permissions" site-specific setting can be saved in a custom profile, and it is possible for WebDriver to load an existing profile on demand via simple steps like this:

require 'watir-webdriver'
driver = Watir::Browser.new :firefox, :profile => 'Test for WMF mobile'
driver.goto('http://en.wikipedia.org/wiki/Special:Nearby')

This works fine, but to make the test usable in multiple environments, profiles need to be portable. Unfortunately, the site-specific setting of interest (Boolean "geo") is stored in a SQLite database (permissions.sqlite), which does not seem to make that profile setting very portable.

We need to find a way to make this setting portable, so that a browser test for the "Nearby" page can successfully set geolocation settings AND run in any environment, including Vagrant VMs and Jenkins (CloudBees).


Version: unspecified
Severity: normal
Whiteboard: zfilipin
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=58720

Details

Reference
bz57333

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 2:29 AM
bzimport set Reference to bz57333.

http://code.google.com/p/selenium/wiki/TipsAndTricks
has code to set the profile, and suggests there are Selenium/WebDriver functions to set profile features.

Can WebDriver drive Firefox internal chrome? If so, it could load about:permissions, scroll to the wmflabs.org domain, and set Shared Location to Allow.

jhall wrote:

Based on a discussion during the QA summary meeting this morning, I'm going to spend some time looking into the process of having the test create the permissions.sqlite file as a setup step, and populate the moz_hosts table with the desired data.

The table schema itself is:

CREATE TABLE moz_hosts (
id INTEGER PRIMARY KEY,
host TEXT,
type TEXT,
permission INTEGER,
expireType INTEGER,
expireTime INTEGER,
appId INTEGER,
isInBrowserElement INTEGER)

…and the relevant row(s) of interest would have values such as these:

INSERT INTO moz_hosts(id, host, type, permission, expireType, expireTime, appId, isInBrowserElement) VALUES("12","en.wikipedia.org","geo","1","0","0","0","0")

jhall wrote:

Script to create the database and default data for a new Firefox profile

Attached script will create the relevant SQLite database and default row data, not including the row(s) we want to insert for the wiki-specific 'geo' settings.

Attached:

jhall wrote:

Additional inserts to implement 'geo' setting for our primary test environments:

INSERT INTO moz_hosts(id, host, type, permission, expireType, expireTime, appId, isInBrowserElement) VALUES("16","en.wikipedia.org","geo","1","0","0","0","0");

INSERT INTO moz_hosts(id, host, type, permission, expireType, expireTime, appId, isInBrowserElement) VALUES("17","en.wikipedia.beta.wmflabs.org","geo","1","0","0","0","0");

INSERT INTO moz_hosts(id, host, type, permission, expireType, expireTime, appId, isInBrowserElement) VALUES("18","commons.wikimedia.beta.wmflabs.org","geo","1","0","0","0","0");

INSERT INTO moz_hosts(id, host, type, permission, expireType, expireTime, appId, isInBrowserElement) VALUES("19","test2.wikipedia.org","geo","1","0","0","0","0");

jhall wrote:

This workflow was successful on my MacBook:

  1. Use SQL commands indicated above to create a canned "permissions.sqlite" file.
  1. Create new "empty" Firefox profile (without opening the application): ./firefox -CreateProfile "NearbyTestProfile /Users/jeffreyhall/Documents/NearbyTestProfile"
  1. Copy the canned "permissions.sqlite" file into that newly-created profile directory: cp permissions.sqlite ../NearbyTestProfile/
  1. Start Firefox with the new profile: ./firefox -profile "/Users/jeffreyhall/Documents/NearbyTestProfile"
  1. Navigate to the Nearby page: http://en.wikipedia.org/wiki/Special:Nearby

Page loads with no "share location" prompt - hurray!

jhall wrote:

Standalone script for the basic workflow

Attached Ruby script works for setting up a profile and visiting a couple of wikis for which the 'geo' setting has been made in a canned 'permissions.sqlite' file.

Next step is to further abstract values in the script so that they can (theoretically) work in any environment.

Attached:

jhall wrote:

Working solution is to drop a copy of the "canned" permissions.sqlite file into the ../test/browser/features/support directory of the mediawiki-extensions-MobileFrontend repo, and then inject these lines into env.rb for the same repo:

permissions_file = "./features/support/permissions.sqlite"

if RUBY_PLATFORM == 'x86_64-darwin12.5.0'
  firefox_executable = "/Applications/Firefox.app/Contents/MacOS/firefox"
elsif RUBY_PLATFORM == 'x86_64-linux'
  firefox_executable = '/usr/bin/firefox'
else
  puts 'Cannot identify local OS, so cannot locate Firefox executable!'
end
profile_path = ("/tmp/NearbyProfile/")
unless File.exist?(profile_path)
  system("mkdir " + profile_path)
end
system(firefox_executable + " -CreateProfile 'NearbyProfile " + profile_path + "'")
system("cp " + permissions_file + " " + profile_path)
browser = Watir::Browser.new :firefox, :profile => 'NearbyProfile'

This isn't elegant, but it is working!

jhall wrote:

Paired with Jon Robson to look into the fix today, and we've amended the patch with the code noted above as well as some related changes.

A few things left to do before we can really merge this into master:

  1. en.wikipedia.beta.wmflabs.org and test2.wikipedia.org have very limited geodata content, so the test will need to spoof a location. An example page that does have geodata:

http://en.wikipedia.beta.wmflabs.org/wiki/Page_with_geodata

  1. Even with geodata in place in the test environments, BZ 57450 is a blocking bug (geodata doesn't work at all for en.wikipedia.beta.wmflabs.org).
  1. We should add separate test pathways for desktop vs. mobile:

    Desktop: http://en.wikipedia.org/wiki/Special:Nearby Mobile: http://en.wikipedia.org/wiki/Special:Nearby?useformat=mobile
  1. We should add tags indicating that the Nearby test will only work for the Firefox browser and for test environments already defined in permissions.sqlite (see comment #4 above).

We should contact Firefox/Chromium people and ask them how they test geolocation.

I did not have the time to work on this yet. Jeff, did you make any progress?

jhall wrote:

Blocked on this issue because of the BZ 57450 issue:

https://bugzilla.wikimedia.org/show_bug.cgi?id=57450

...but we discussed that issue during the RelEng/QA team meeting this morning, and Antoine has grabbed BZ 57450 and will into it.

jhall wrote:

Per Max Semenik's fix to BZ 57450, "Nearby" page in beta labs no longer throws an exception, although I'm not getting any relevant hits even where relevant content exists in beta labs that ought to produce matches for this request:

http://en.wikipedia.beta.wmflabs.org/wiki/Special:Nearby

…with these params:

action = query
colimit = max
format = json
generator = geosearch
ggscoord = 37.786688999999996|-122.39947719999998
ggslimit = 50
ggsnamespace = 0
ggsradius = 10000
pilimit = 50
pithumbsize = 180
prop = pageimages|coordinates

In any case, we're no longer getting a stack trace, so that's progress.

Hi Jeff
In theory you just need to fill the wiki with some data.

You can use this wikitext to add a coordinate at a given location.

I think the data will take some time to show up however on Nearby. Not sure if there is a quicker way - Max may know.

(In reply to comment #15)

{{Coord|37.786971|N|122.399677|W}}

Provided that the wiki has this template.

I think the data will take some time to show up however on Nearby. Not sure
if
there is a quicker way - Max may know.

Solr index is updated every 30 minutes. In principle, on betalabs we can use job queue for updates which should be faster yet still not intantaneously. However, this would create a discrepancy with production.

jhall wrote:

Thanks Jon and Max - this does seem to be working better now in beta labs. Will try to wrap up work on this new browser test tomorrow.

jhall wrote:

Proposed test is in Gerrit for review:

https://gerrit.wikimedia.org/r/#/c/101129/

If reviewers are OK with this test, I can implement a similar "non-mobile" test for the Nearby page in the qa/browsertests repo.

Change 101129 had a related patch set uploaded by Jdlrobson:
New browser test for "Nearby" page

https://gerrit.wikimedia.org/r/101129

Change 101129 merged by jenkins-bot:
QA: New browser test for "Nearby" page

https://gerrit.wikimedia.org/r/101129

Fix got reverted due to bug 59174 sadly

Interesting blog post with code examples in C#

Faking Geolocation in Selenium WebDriver with Firefox

http://watirmelon.com/2014/09/18/faking-geolocation-in-selenium-webdriver-with-firefox/

It looks to me like Mozilla changed how this is done in Firefox. As I recall, when Jeff and I were trying to make this work,

set ‘geo.prompt.testing’ to true
set ‘geo.prompt.testing.allow’ to true

were not accessible setting from about:config, and that permission had to be already set in Firefox's profile at the time the browser instance was created.

Pretty neat, though, I might take a stab at this.

Maybe the setting is new: "This is only supported in very recent versions of Firefox (I tested version 31)."

greg removed a subscriber: Cmcmahon.
Jdlrobson claimed this task.

Doesn't look like this will happen..