Page MenuHomePhabricator

Set up a staging environment for development
Closed, ResolvedPublic3 Estimated Story Points

Description

  • A development environment is set up with our extensions and dependencies. (MobileFrontend, Gather and deps (CirrusSearch, PageImages, TextExtracts)
  • How it was set up, and where it is, is publicly documented on-wiki.
  • Development env auto-updates from the development branch continuously and stays in sync with the top of the HEAD of development.

  • We have an accesible staging environment (Labs-Vagrant?)
  • Can access & use MobileFrontend on staging.
  • Can access & use Gather on staging.
  • Environment provisioning is automated and can be re-used on a clean vm.
  • Set-up and provisioning process is publicly documented on-wiki.
  • Can access and verify latest code from HEAD of dev automatically (or in a short amount of time).
  • Engineers have access to staging for troubleshooting.

Event Timeline

Jhernandez claimed this task.
Jhernandez raised the priority of this task from to High.
Jhernandez updated the task description. (Show Details)
Jhernandez updated the task description. (Show Details)
Jhernandez set Security to None.
Jhernandez added subscribers: Jhernandez, Aklapper, phuedx.
Jhernandez removed Jhernandez as the assignee of this task.Jul 2 2015, 11:44 AM
phuedx updated the task description. (Show Details)Jul 6 2015, 4:46 PM
KLans_WMF edited a custom field.Jul 6 2015, 4:52 PM
KLans_WMF moved this task from Needs Analysis to To Do on the Reading-Web-Sprint-51-YOLO board.
phuedx claimed this task.Jul 7 2015, 7:35 PM
phuedx moved this task from To Do to Doing on the Reading-Web-Sprint-51-YOLO board.
phuedx updated the task description. (Show Details)Jul 9 2015, 10:11 AM

MobileFrontend browser smoke tests pass against http://reading-web-staging.wmflabs.org as do the Gather browser tests.

phuedx updated the task description. (Show Details)Jul 9 2015, 11:23 AM

I've added dev branches to the MobileFrontend and Gather repositories and checked 'em out on the server. The vagrant user currently runs labs-vagrant git-update every 10 minutes.

phuedx added a comment.EditedJul 9 2015, 2:46 PM

Here are the notes of what I did to get this far:

$ labs-vagrant provision
$ labs-vagrant enable-role mobilefrontend gather geodata confirmedit
$ labs-vagrant enable-role visualeditor cite echo
$ labs-vagrant provision

$ cd /vagrant/mediawiki/extensions/MobileFrontend
$ git checkout -b dev -t origin/dev

$ cd /vagrant/mediawiki/extensions/Gather
$ git checkout -b dev -t origin/dev

$ sudo su vagrant
$ crontab -e # Add */10 * * * * labs-vagrant git-update

Followed the instructions at https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/tree/5d20be9f5083ce14c1cf1a769f30d39fa0e41c58/tests/browser.

Change 224048 had a related patch set uploaded (by Phuedx):
Update browser test environment documentation

https://gerrit.wikimedia.org/r/224048

phuedx updated the task description. (Show Details)Jul 10 2015, 11:44 AM
phuedx moved this task from Doing to Code Review on the Reading-Web-Sprint-51-YOLO board.

Moving to Code Review as there is a small patch to review.

Change 224048 merged by jenkins-bot:
Update browser test environment documentation

https://gerrit.wikimedia.org/r/224048

Hey @phuedx, docs look great 👏👍.

How exactly would I join that project? What steps do I need to follow? Does anybody know? cc/ @bmansurov @Jdlrobson @rmoen

Going to leave this in sign off until we have confirmation that we can all access the instances.

@Jhernandez go to that page, log in using your Gerrit credentials and edit the page... I think... although it seems to have had no effect for me. I cannot run ssh reading-web-staging.eqiad.wmflabs and I'm not sure what joining a project does.
This stuff always confuses me...

The documentation on https://wikitech.wikimedia.org/wiki/Help:Getting_Started#Join_Projects is lacking. *cough* @yuvipanda

@Jdlrobson I've added myself to https://wikitech.wikimedia.org/wiki/Nova_Resource:Reading-web-staging but I don't show up on the page (I do on the wikitext source :s).

I can't ssh into it either Permission denied (publickey).

I've updated the docs a bit.

Looks like @phuedx or @bd808 will have to add us to reading-web-staging project

bd808 added a comment.Jul 21 2015, 2:06 AM

Looks like @phuedx or @bd808 will have to add us to reading-web-staging project

Successfully added Jdlrobson to projectadmin.
Successfully added Jhernandez to projectadmin.
Successfully added Robmoen to projectadmin.

Thanks! @joakino can we sign off now?

@bmansurov @rmoen @Jhernandez please confirm you can ssh in with

ssh reading-web-staging-2.eqiad.wmflabs

I cannot:
Connection closed by UNKNOWN

I've been able to log in, had clear the known_hosts fingerprint (seems to have changed in the last months), and add my user name to the domain to ssh into and to the entry on my .ssh/config about *.eqiad.wmflabs

.ssh/config (only the *.eqiad.wmflabs part)

Host *.eqiad.wmflabs
    ProxyCommand ssh -a -W %h:%p jhernandez@bastion1.eqiad.wmflabs

And ssh'd doing: ssh jhernandez@reading-web-staging-2.eqiad.wmflabs

Not sure why I'm having to specify the username twice, but if I don't do it, it won't let me log in.


@bmansurov that is a very weird error message. Do you have your .ssh/config configured properly? There is some help here https://wikitech.wikimedia.org/wiki/Help:Access Let's troubleshoot this before the sprint ends.

bd808 added a comment.Jul 29 2015, 3:51 PM

@bmansurov that is a very weird error message. Do you have your .ssh/config configured properly? There is some help here https://wikitech.wikimedia.org/wiki/Help:Access Let's troubleshoot this before the sprint ends.

My settings look something like this:

1# WMF Labs
2Host gerrit gerrit.wikimedia.org
3 Hostname gerrit.wikimedia.org
4 Port 29418
5 User <YOUR GERRIT SHELL NAME HERE (eg bd808)>
6 IdentityFile <PATH TO YOUR GERRIT SSH KEY HERE (eg ~/.ssh/gerrit_rsa)>
7
8Host bastion.wmflabs.org
9 ProxyCommand none
10 ControlMaster no
11
12Host *.wmflabs *.wmflabs.org
13 User <YOUR LABS SHELL NAME HERE (eg bd808)>
14 IdentityFile <PATH TO YOUR LABS SSH KEY HERE (eg ~/.ssh/wmflabs_rsa)>
15 IdentitiesOnly yes
16
17Host *.wmflabs
18 ProxyCommand ssh -a -W %h:%p bastion.wmflabs.org
19
20
21# WMF Prod
22Host bast1001.wikimedia.org
23 ProxyCommand none
24 ControlMaster no
25
26Host *.eqiad.wmnet *.codfw.wmnet *.wikimedia.org !gerrit.wikimedia.org
27 User <YOUR PROD SHELL NAME HERE (eg bd808)>
28 IdentityFile <PATH TO YOUR PROD SSH KEY HERE (eg ~/.ssh/cluster_rsa)>
29 IdentitiesOnly yes
30 ProxyCommand ssh -a -W %h:%p bast1001.wikimedia.org
31
32
33# Use strong ciphers when connecting to WMF servers
34Host *.wmnet *.wmflabs *.wikimedia.org !gerrit.wikimedia.org
35 Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
36 KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
37
38# Global defaults
39Host *
40 ForwardAgent no
41 ForwardX11 no
42 Protocol 2,1
43 ControlMaster auto
44 ControlPath ~/.ssh/sockets/%C
45 KeepAlive yes
46 ServerAliveInterval 60
47 PreferredAuthentications publickey

Looks like @phuedx or @bd808 will have to add us to reading-web-staging project

Successfully added Jdlrobson to projectadmin.
Successfully added Jhernandez to projectadmin.
Successfully added Robmoen to projectadmin.

I don't see my name above.

bd808 added a comment.Jul 29 2015, 7:14 PM

I don't see my name above.

Fixed

Thanks. I can ssh to reading-web-staging-2.eqiad.wmflabs now.

Jhernandez closed this task as Resolved.Jul 30 2015, 5:07 PM
Jhernandez updated the task description. (Show Details)
Jhernandez moved this task from Code Review to Done on the Reading-Web-Sprint-52-Zoolander board.