Page MenuHomePhabricator

Set up a staging environment for development
Closed, ResolvedPublic3 Estimated Story Points

Description

  • A development environment is set up with our extensions and dependencies. (MobileFrontend, Gather and deps (CirrusSearch, PageImages, TextExtracts)
  • How it was set up, and where it is, is publicly documented on-wiki.
  • Development env auto-updates from the development branch continuously and stays in sync with the top of the HEAD of development.

  • We have an accesible staging environment (Labs-Vagrant?)
  • Can access & use MobileFrontend on staging.
  • Can access & use Gather on staging.
  • Environment provisioning is automated and can be re-used on a clean vm.
  • Set-up and provisioning process is publicly documented on-wiki.
  • Can access and verify latest code from HEAD of dev automatically (or in a short amount of time).
  • Engineers have access to staging for troubleshooting.

Event Timeline

Jhernandez claimed this task.
Jhernandez raised the priority of this task from to High.
Jhernandez updated the task description. (Show Details)
Jhernandez updated the task description. (Show Details)
Jhernandez set Security to None.
Jhernandez added subscribers: Jhernandez, Aklapper, phuedx.

I've added dev branches to the MobileFrontend and Gather repositories and checked 'em out on the server. The vagrant user currently runs labs-vagrant git-update every 10 minutes.

Here are the notes of what I did to get this far:

$ labs-vagrant provision
$ labs-vagrant enable-role mobilefrontend gather geodata confirmedit
$ labs-vagrant enable-role visualeditor cite echo
$ labs-vagrant provision

$ cd /vagrant/mediawiki/extensions/MobileFrontend
$ git checkout -b dev -t origin/dev

$ cd /vagrant/mediawiki/extensions/Gather
$ git checkout -b dev -t origin/dev

$ sudo su vagrant
$ crontab -e # Add */10 * * * * labs-vagrant git-update

Followed the instructions at https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/tree/5d20be9f5083ce14c1cf1a769f30d39fa0e41c58/tests/browser.

Change 224048 had a related patch set uploaded (by Phuedx):
Update browser test environment documentation

https://gerrit.wikimedia.org/r/224048

phuedx moved this task from Doing to Code Review on the Reading-Web-Sprint-51-YOLO board.

Moving to Code Review as there is a small patch to review.

Change 224048 merged by jenkins-bot:
Update browser test environment documentation

https://gerrit.wikimedia.org/r/224048

Hey @phuedx, docs look great 👏👍.

How exactly would I join that project? What steps do I need to follow? Does anybody know? cc/ @bmansurov @Jdlrobson @rmoen

Going to leave this in sign off until we have confirmation that we can all access the instances.

@Jhernandez go to that page, log in using your Gerrit credentials and edit the page... I think... although it seems to have had no effect for me. I cannot run ssh reading-web-staging.eqiad.wmflabs and I'm not sure what joining a project does.
This stuff always confuses me...

The documentation on https://wikitech.wikimedia.org/wiki/Help:Getting_Started#Join_Projects is lacking. *cough* @yuvipanda

@Jdlrobson I've added myself to https://wikitech.wikimedia.org/wiki/Nova_Resource:Reading-web-staging but I don't show up on the page (I do on the wikitext source :s).

I can't ssh into it either Permission denied (publickey).

Looks like @phuedx or @bd808 will have to add us to reading-web-staging project

Looks like @phuedx or @bd808 will have to add us to reading-web-staging project

Successfully added Jdlrobson to projectadmin.
Successfully added Jhernandez to projectadmin.
Successfully added Robmoen to projectadmin.

Thanks! @joakino can we sign off now?

@bmansurov @rmoen @Jhernandez please confirm you can ssh in with

ssh reading-web-staging-2.eqiad.wmflabs

I cannot:
Connection closed by UNKNOWN

I've been able to log in, had clear the known_hosts fingerprint (seems to have changed in the last months), and add my user name to the domain to ssh into and to the entry on my .ssh/config about *.eqiad.wmflabs

.ssh/config (only the *.eqiad.wmflabs part)

Host *.eqiad.wmflabs
    ProxyCommand ssh -a -W %h:%p jhernandez@bastion1.eqiad.wmflabs

And ssh'd doing: ssh jhernandez@reading-web-staging-2.eqiad.wmflabs

Not sure why I'm having to specify the username twice, but if I don't do it, it won't let me log in.


@bmansurov that is a very weird error message. Do you have your .ssh/config configured properly? There is some help here https://wikitech.wikimedia.org/wiki/Help:Access Let's troubleshoot this before the sprint ends.

@bmansurov that is a very weird error message. Do you have your .ssh/config configured properly? There is some help here https://wikitech.wikimedia.org/wiki/Help:Access Let's troubleshoot this before the sprint ends.

My settings look something like this:

1# WMF Labs
2Host gerrit gerrit.wikimedia.org
3 Hostname gerrit.wikimedia.org
4 Port 29418
5 User <YOUR GERRIT SHELL NAME HERE (eg bd808)>
6 IdentityFile <PATH TO YOUR GERRIT SSH KEY HERE (eg ~/.ssh/gerrit_rsa)>
7
8Host bastion.wmflabs.org
9 ProxyCommand none
10 ControlMaster no
11
12Host *.wmflabs *.wmflabs.org
13 User <YOUR LABS SHELL NAME HERE (eg bd808)>
14 IdentityFile <PATH TO YOUR LABS SSH KEY HERE (eg ~/.ssh/wmflabs_rsa)>
15 IdentitiesOnly yes
16
17Host *.wmflabs
18 ProxyCommand ssh -a -W %h:%p bastion.wmflabs.org
19
20
21# WMF Prod
22Host bast1001.wikimedia.org
23 ProxyCommand none
24 ControlMaster no
25
26Host *.eqiad.wmnet *.codfw.wmnet *.wikimedia.org !gerrit.wikimedia.org
27 User <YOUR PROD SHELL NAME HERE (eg bd808)>
28 IdentityFile <PATH TO YOUR PROD SSH KEY HERE (eg ~/.ssh/cluster_rsa)>
29 IdentitiesOnly yes
30 ProxyCommand ssh -a -W %h:%p bast1001.wikimedia.org
31
32
33# Use strong ciphers when connecting to WMF servers
34Host *.wmnet *.wmflabs *.wikimedia.org !gerrit.wikimedia.org
35 Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
36 KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
37
38# Global defaults
39Host *
40 ForwardAgent no
41 ForwardX11 no
42 Protocol 2,1
43 ControlMaster auto
44 ControlPath ~/.ssh/sockets/%C
45 KeepAlive yes
46 ServerAliveInterval 60
47 PreferredAuthentications publickey

Looks like @phuedx or @bd808 will have to add us to reading-web-staging project

Successfully added Jdlrobson to projectadmin.
Successfully added Jhernandez to projectadmin.
Successfully added Robmoen to projectadmin.

I don't see my name above.

I don't see my name above.

Fixed

Thanks. I can ssh to reading-web-staging-2.eqiad.wmflabs now.

Jhernandez updated the task description. (Show Details)
Jhernandez moved this task from Code Review to Done on the Reading-Web-Sprint-52-Zoolander board.