Page MenuHomePhabricator
Create Task
Maniphest T104558

Set up a staging environment for development
Closed, ResolvedPublic3 Estimated Story Points
Actions

Description

  • A development environment is set up with our extensions and dependencies. (MobileFrontend, Gather and deps (CirrusSearch, PageImages, TextExtracts)
  • How it was set up, and where it is, is publicly documented on-wiki.
  • Development env auto-updates from the development branch continuously and stays in sync with the top of the HEAD of development.
  • We have an accesible staging environment (Labs-Vagrant?)
  • Can access & use MobileFrontend on staging.
  • Can access & use Gather on staging.
  • Environment provisioning is automated and can be re-used on a clean vm.
  • Set-up and provisioning process is publicly documented on-wiki.
  • Can access and verify latest code from HEAD of dev automatically (or in a short amount of time).
  • Engineers have access to staging for troubleshooting.

Details

SubjectRepoBranchLines +/-
Update browser test environment documentationmediawiki/extensions/MobileFrontendmaster+1 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

• Jhernandez created this task.Jul 2 2015, 11:22 AM
• Jhernandez claimed this task.
• Jhernandez raised the priority of this task from to High.
• Jhernandez updated the task description. (Show Details)
• Jhernandez added a project: Reading-Web-Sprint-51-YOLO.
• Jhernandez updated the task description. (Show Details)
• Jhernandez set Security to None.
• Jhernandez added subscribers: • Jhernandez, Aklapper, phuedx.
• Jhernandez updated the task description. (Show Details)Jul 2 2015, 11:29 AM
• Jhernandez mentioned this in T104559: Setting up staging is reproducible by engineers.
• Jhernandez added a parent task: T104559: Setting up staging is reproducible by engineers.
• Jhernandez removed • Jhernandez as the assignee of this task.Jul 2 2015, 11:44 AM
• Jhernandez mentioned this in T104562: Document how to set up the testing environment and run browser tests per-patch..Jul 2 2015, 11:58 AM
phuedx updated the task description. (Show Details)Jul 6 2015, 4:46 PM
• KLans_WMF edited a custom field.Jul 6 2015, 4:52 PM
• KLans_WMF moved this task from Needs Analysis to To Do on the Reading-Web-Sprint-51-YOLO board.
phuedx mentioned this in T104994: Staging environment for Reading Web.Jul 7 2015, 1:37 PM
phuedx added a subtask: T104994: Staging environment for Reading Web.
bd808 closed subtask T104994: Staging environment for Reading Web as Resolved.Jul 7 2015, 6:07 PM
phuedx claimed this task.Jul 7 2015, 7:35 PM
phuedx moved this task from To Do to Doing on the Reading-Web-Sprint-51-YOLO board.
phuedx updated the task description. (Show Details)Jul 9 2015, 10:11 AM
phuedx added a comment.Jul 9 2015, 11:11 AM

MobileFrontend browser smoke tests pass against http://reading-web-staging.wmflabs.org as do the Gather browser tests.

phuedx updated the task description. (Show Details)Jul 9 2015, 11:23 AM
phuedx added subscribers: • bmansurov, • rmoen, Jdlrobson.Jul 9 2015, 11:26 AM

@bmansurov @Jhernandez @Jdlrobson @rmoen: Could y'all join the reading-web-staging project please.

phuedx added a comment.Jul 9 2015, 11:33 AM

I've added dev branches to the MobileFrontend and Gather repositories and checked 'em out on the server. The vagrant user currently runs labs-vagrant git-update every 10 minutes.

phuedx added a comment.EditedJul 9 2015, 2:46 PM

Here are the notes of what I did to get this far:

$ labs-vagrant provision
$ labs-vagrant enable-role mobilefrontend gather geodata confirmedit
$ labs-vagrant enable-role visualeditor cite echo
$ labs-vagrant provision

$ cd /vagrant/mediawiki/extensions/MobileFrontend
$ git checkout -b dev -t origin/dev

$ cd /vagrant/mediawiki/extensions/Gather
$ git checkout -b dev -t origin/dev

$ sudo su vagrant
$ crontab -e # Add */10 * * * * labs-vagrant git-update

Followed the instructions at https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/tree/5d20be9f5083ce14c1cf1a769f30d39fa0e41c58/tests/browser.

gerritbot subscribed.Jul 10 2015, 10:54 AM

Change 224048 had a related patch set uploaded (by Phuedx):
Update browser test environment documentation

https://gerrit.wikimedia.org/r/224048

gerritbot added a project: Patch-For-Review.Jul 10 2015, 10:54 AM
phuedx added a comment.Jul 10 2015, 11:44 AM

Initial draft of the setup process here: https://www.mediawiki.org/wiki/Reading/Web/Setting_up_a_staging_environment

phuedx updated the task description. (Show Details)Jul 10 2015, 11:44 AM
phuedx moved this task from Doing to Code Review on the Reading-Web-Sprint-51-YOLO board.

Moving to Code Review as there is a small patch to review.

phuedx mentioned this in T104556: Merge to development branch on MobileFrontend by default.Jul 10 2015, 2:42 PM
gerritbot added a comment.Jul 13 2015, 1:56 PM

Change 224048 merged by jenkins-bot:
Update browser test environment documentation

https://gerrit.wikimedia.org/r/224048

Diffusion mentioned this in rMEXTf1ab259a90d6: Updated mediawiki/extensions Project: mediawiki/extensions/MobileFrontend….Jul 13 2015, 1:56 PM
phuedx mentioned this in rEMFR95b14c1ce0d6: Update browser test environment documentation.Jul 13 2015, 1:56 PM
• Jhernandez added a comment.Jul 13 2015, 1:56 PM

Hey @phuedx, docs look great 👏👍.

In T104558#1440788, @phuedx wrote:

@bmansurov @Jhernandez @Jdlrobson @rmoen: Could y'all join the reading-web-staging project please.

How exactly would I join that project? What steps do I need to follow? Does anybody know? cc/ @bmansurov @Jdlrobson @rmoen

• Jhernandez moved this task from Code Review to Ready for Signoff on the Reading-Web-Sprint-51-YOLO board.Jul 13 2015, 1:57 PM

Going to leave this in sign off until we have confirmation that we can all access the instances.

• Forrestbot added a project: WMF-deploy-2015-07-14_(1.26wmf14).Jul 13 2015, 2:00 PM
Jdlrobson added a subscriber: yuvipanda.Jul 16 2015, 9:06 AM

@Jhernandez go to that page, log in using your Gerrit credentials and edit the page... I think... although it seems to have had no effect for me. I cannot run ssh reading-web-staging.eqiad.wmflabs and I'm not sure what joining a project does.
This stuff always confuses me...

The documentation on https://wikitech.wikimedia.org/wiki/Help:Getting_Started#Join_Projects is lacking. *cough* @yuvipanda

• Jhernandez added a comment.Jul 19 2015, 2:46 PM

@Jdlrobson I've added myself to https://wikitech.wikimedia.org/wiki/Nova_Resource:Reading-web-staging but I don't show up on the page (I do on the wikitext source :s).

I can't ssh into it either Permission denied (publickey).

yuvipanda added a comment.Jul 19 2015, 4:27 PM

I've updated the docs a bit.

• KLans_WMF edited projects, added Reading-Web-Sprint-52-Zoolander; removed Reading-Web-Sprint-51-YOLO.Jul 20 2015, 4:05 PM
• KLans_WMF moved this task from Needs Analysis to Ready for Signoff on the Reading-Web-Sprint-52-Zoolander board.
Jdlrobson added a subscriber: bd808.Jul 20 2015, 8:22 PM

Looks like @phuedx or @bd808 will have to add us to reading-web-staging project

bd808 added a comment.Jul 21 2015, 2:06 AM
In T104558#1465795, @Jdlrobson wrote:

Looks like @phuedx or @bd808 will have to add us to reading-web-staging project

Successfully added Jdlrobson to projectadmin.
Successfully added Jhernandez to projectadmin.
Successfully added Robmoen to projectadmin.
Jdlrobson added a comment.Jul 21 2015, 11:39 PM

Thanks! @joakino can we sign off now?

Jdlrobson added a comment.Jul 22 2015, 9:51 PM

@bmansurov @rmoen @Jhernandez please confirm you can ssh in with

ssh reading-web-staging-2.eqiad.wmflabs

• bmansurov added a comment.Jul 22 2015, 9:54 PM

I cannot:
Connection closed by UNKNOWN

• Jhernandez added a comment.Jul 29 2015, 10:34 AM

I've been able to log in, had clear the known_hosts fingerprint (seems to have changed in the last months), and add my user name to the domain to ssh into and to the entry on my .ssh/config about *.eqiad.wmflabs

.ssh/config (only the *.eqiad.wmflabs part)

Host *.eqiad.wmflabs
    ProxyCommand ssh -a -W %h:%p jhernandez@bastion1.eqiad.wmflabs

And ssh'd doing: ssh jhernandez@reading-web-staging-2.eqiad.wmflabs

Not sure why I'm having to specify the username twice, but if I don't do it, it won't let me log in.

@bmansurov that is a very weird error message. Do you have your .ssh/config configured properly? There is some help here https://wikitech.wikimedia.org/wiki/Help:Access Let's troubleshoot this before the sprint ends.

• Jhernandez moved this task from Ready for Signoff to -1 (Needs More Work) on the Reading-Web-Sprint-52-Zoolander board.Jul 29 2015, 10:34 AM
bd808 added a comment.Jul 29 2015, 3:51 PM
In T104558#1491116, @Jhernandez wrote:

@bmansurov that is a very weird error message. Do you have your .ssh/config configured properly? There is some help here https://wikitech.wikimedia.org/wiki/Help:Access Let's troubleshoot this before the sprint ends.

My settings look something like this:

P1154 ~/.ssh/config magic for WMF hosts
1# WMF Labs
2Host gerrit gerrit.wikimedia.org
3 Hostname gerrit.wikimedia.org
4 Port 29418
5 User <YOUR GERRIT SHELL NAME HERE (eg bd808)>
6 IdentityFile <PATH TO YOUR GERRIT SSH KEY HERE (eg ~/.ssh/gerrit_rsa)>
7
8Host bastion.wmflabs.org
9 ProxyCommand none
10 ControlMaster no
11
12Host *.wmflabs *.wmflabs.org
13 User <YOUR LABS SHELL NAME HERE (eg bd808)>
14 IdentityFile <PATH TO YOUR LABS SSH KEY HERE (eg ~/.ssh/wmflabs_rsa)>
15 IdentitiesOnly yes
16
17Host *.wmflabs
18 ProxyCommand ssh -a -W %h:%p bastion.wmflabs.org
19
20
21# WMF Prod
22Host bast1001.wikimedia.org
23 ProxyCommand none
24 ControlMaster no
25
26Host *.eqiad.wmnet *.codfw.wmnet *.wikimedia.org !gerrit.wikimedia.org
27 User <YOUR PROD SHELL NAME HERE (eg bd808)>
28 IdentityFile <PATH TO YOUR PROD SSH KEY HERE (eg ~/.ssh/cluster_rsa)>
29 IdentitiesOnly yes
30 ProxyCommand ssh -a -W %h:%p bast1001.wikimedia.org
31
32
33# Use strong ciphers when connecting to WMF servers
34Host *.wmnet *.wmflabs *.wikimedia.org !gerrit.wikimedia.org
35 Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
36 KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
37
38# Global defaults
39Host *
40 ForwardAgent no
41 ForwardX11 no
42 Protocol 2,1
43 ControlMaster auto
44 ControlPath ~/.ssh/sockets/%C
45 KeepAlive yes
46 ServerAliveInterval 60
47 PreferredAuthentications publickey

• bmansurov added a comment.Jul 29 2015, 5:09 PM
In T104558#1466191, @bd808 wrote:
In T104558#1465795, @Jdlrobson wrote:

Looks like @phuedx or @bd808 will have to add us to reading-web-staging project

Successfully added Jdlrobson to projectadmin.
Successfully added Jhernandez to projectadmin.
Successfully added Robmoen to projectadmin.

I don't see my name above.

bd808 added a comment.Jul 29 2015, 7:14 PM
In T104558#1492180, @bmansurov wrote:
In T104558#1466191, @bd808 wrote:

I don't see my name above.

Fixed

• bmansurov added a comment.Jul 29 2015, 7:21 PM

Thanks. I can ssh to reading-web-staging-2.eqiad.wmflabs now.

phuedx moved this task from -1 (Needs More Work) to Code Review on the Reading-Web-Sprint-52-Zoolander board.Jul 30 2015, 10:45 AM
• Jhernandez closed this task as Resolved.Jul 30 2015, 5:07 PM
• Jhernandez updated the task description. (Show Details)
• Jhernandez moved this task from Code Review to Done on the Reading-Web-Sprint-52-Zoolander board.
• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:49 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL