Note: MobileFrontend had a few additional setup steps that including setting up languages, installing a license and a few other things. It would be great to have vagrant roles for stock data around this - we should pair with @bd808 together and see if we can resolve T62116 to help with this.

@Jdlrobson: What are the "few other things"? Could we make a list?

They are all documented in tests/browser/README

For posterity: https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/blob/723c7edb2c88e0e5ed736b33d419b30e594c0b50/tests/browser/README.mediawiki

In T104842#1434463, @phuedx wrote:

For posterity: https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/blob/723c7edb2c88e0e5ed736b33d419b30e594c0b50/tests/browser/README.mediawiki

It should be possible to create a Puppet role that automates all of this. The interwiki link edit and the user with a non-zero edit count may be the trickiest bits. I'm actually working on some interwiki stuff for T93375 that may end up making that part easier.

I have a install script going at https://github.com/rmoen/Barry-the-Browser-Test-Bot-Installer

This creates a user on the system and configures a bot to run as the user. Tis possible there are some issues. Though, barrybot.py seems to be running when executing the generated shell script as the user.

This is very primitive at the moment and has room for improvement. But for purposes of this task, it seems to take care of the bulk of the bot installation.

This is great Rob! Nice unix foo :)

As a suggestion, given that MEDIAWIKI_LOAD_URL etc should be the same for all labs instance - could we automate these rather than prompt the user? If we can automatically create the user Mr Selenium with password test this would also be one less step. Not sure if we can do that with the API though without the wrath of the captcha...

@Jdlrobson hmmm well captcha isn't enabled by default and i found an easy way to create an account from the api. I'm going to add that in

Sweet! I'm in progress of setting up a new instance.

A few improvements as I've encountered them so far:

• barrybot/run.sh should only have one call to barrybot.py in the body of the script.
  • the prompt should ask you for a tag for tests to run

Note:
New automation step needed:
export MEDIAWIKI_ENVIRONMENT=barry

Not quite seamless yet...

cd ~/barrybot && ./run.sh

>> 
>> bundler: command not found: cucumber
>> Install missing gem executables with `bundle install`
>> 
>> Reviewing commit 241c485f2398637a78303af71c6fba366ed4977b with (is good = False)..
>> Posting to Gerrit...
>> 
>> 
>> Your user account isn't allowed to install to the system Rubygems.
>> You can cancel this installation and run:
>> 
>>     bundle install --path vendor/bundle
>> 
>> to install the gems into ./vendor/bundle/, or you can enter your password
>> and install the bundled gems to Rubygems using sudo.
>>

Extra steps needed from me to be able to run this:
(as user jdlrobson)

sudo apt-get install ruby-dev && sudo gem install bundler
cd /vagrant/mediawiki/extensions/MobileFrontend
bundle install
sudo su barrythebrowsertestbot
cd ~/barrybot && ./run.sh

(something is going wrong with setting up the username as evidenced by P1039)

Issues with the script

• Remove: --noupdates 1 from run.sh - we want to run updates
• I set dependencies as empty string but it adds the thing it's testing as a dependency. If no dependencies are given don't add the argument [1]
• Add --paste 1 to make use of the paste api

Issues with running script:

• Tests failing for freshly set up MobileFrontend (see P1039)
• Freezes on posting to Gerrit with paste set (see P1039)
• Permission denied (publickey) when I run ssh -p 29418 gerrit.wikimedia.org gerrit review --code-review 0 --message "'Barry says good job. Keep it up.'" 241c485f2398637a78303af71c6fba366ed4977b as my user. I can verify /home/barrythebrowsertestbot/.ssh/id_rsa.pub matches what's in Gerrit (RSA host key for IP address '208.80.154.81' not in list of known hosts. Permission denied (publickey).)~ [edit - this was my problem by username was wrong edited ~/.ssh/known_hosts

[1]:

--test /vagrant/mediawiki/extensions/MobileFrontend/ --dependencies /vagrant/mediawiki/extensions/MobileFrontend

When I run sudo ./install.sh I get the following error:

Add your new public key to gerrit at:  https://gerrit.wikimedia.org/r/#/settings/ssh-keys


cat: .pub: No such file or directory
Please enter a project name (example: Gather)
MobileFrontend
Please enter the name of extensions this project depends on (example: MobileFrontend)

Please enter a test tag (optional. example: smoke)

./install.sh: 167: ./install.sh: cannot create /home/bmansurov/barrybot/run.sh: Directory nonexistent
chmod: cannot access ‘/home/bmansurov/barrybot/run.sh’: No such file or directory
chown: invalid group: ‘bmansurov:bmansurov’
Just created /home/bmansurov/barrybot/run.sh, please make any modifications needed.

I've created issues on the github https://github.com/rmoen/Barry-the-Browser-Test-Bot-Installer/issues that need to get solved to make this smoother.

There is another issue with languages not working and thus the language tests failing - http://mf-browser-tests.wmflabs.org/wiki/Selenium_language_test_page

I've made Barry run MobileFrontend tests separately against http://mf-browser-tests.wmflabs.org/wiki/Selenium_language_test_page now.

http://gather-browser-tests.wmflabs.org/ is only used for Gather tests.

Barry is now running from mf-browser-tests for MobileFrontend and gather-browser-tests for Gather. Once all the issues I have opened are closed let's try this again from scratch. Do we maintain any other extensions with browser tests?

@Jdlrobson

• Made extension dependency optional
• Now running bundle install in project directory
• Added --paste and removed --noupdates flags

Now looking into the configured user issue. (WrongPass)

@bmansurov Will look into that. There are some edge cases with running the script again on the same user account. Removing the account should work for now.

Making progress. Some tests still failing. Still working on it.

@Jdlrobson, any ideas?

Also I believe pasting is throwing an error. Probably a good thing until the tests pass.

Failing Scenarios:
cucumber features/language.feature:12 # Scenario: Closing language overlay (overlay button)
cucumber features/ui_links.feature:8 # Scenario: Check existence of important UI components on the main page
cucumber features/ui_links.feature:18 # Scenario: Check existence of important UI components on other pages.

19 scenarios (3 failed, 1 skipped, 15 passed)
127 steps (3 failed, 15 skipped, 109 passed)
1m48.276s
1 warning due to missing wiki extensions

Pasting commit 27459afc68ab1719e7e7db06b2f4c1ac8753a813 with (is good = False)..
Traceback (most recent call last):
  File "./barrybot.py", line 229, in <module>
    args.nobundleinstall
  File "./barrybot.py", line 197, in watch
    output = get_paste_url(output)
  File "./barrybot.py", line 172, in get_paste_url
    output, error = run_shell_command(['arc', 'paste'], ['echo', text])
  File "./barrybot.py", line 33, in run_shell_command
    process = subprocess.Popen(args, stdin=pre_pipe.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

I ran through this one more time and I'm pretty confident this is in a good place now.
Someone when signing this off should setup a bot. You can reuse a new account I've setup called frankiebot (irc/ e-mail me for the password) to setup an instance for testing Gather browser tests. Hopefully your experience will be seamless.

Something went wrong with frankiebot, so I'm gonna have a go at setting this up again...