Page MenuHomePhabricator

Setup staging-tin as deployment host
Closed, DeclinedPublic

Description

For deployment fun and profit

Details

Related Gerrit Patches:
operations/puppet : productionmediawiki: Ensure that /etc/php5/apache dir exists
operations/mediawiki-config : masterAdd submodules to master checkoutMediaWiki
operations/puppet : productionscap: Clone mediawiki-config on all scap masters

Event Timeline

demon created this task.Feb 3 2015, 4:18 PM
demon raised the priority of this task from to Needs Triage.
demon updated the task description. (Show Details)
demon added a project: Staging.
demon added a subscriber: demon.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 3 2015, 4:18 PM

Let's look at the roles on tin:

include standard
include role::deployment::deployment_servers::production
include mediawiki
include role::scap::master
include mysql
include role::labsdb::manager
include ssh::hostkeys-collect
include role::releases::upload
include admin

Of this, quite a few aren't needed on staging. Obvious ones being admin, standard, labsdb::manager, releases::upload.

That leaves:

  1. role::deployment::deployment_servers::production
  2. ssh::hostkeys-collect
  3. mediawiki
  4. role::scap::master
  5. mysql

We'll investigate and see what all needs to happen :)

Alright, so 'mysql' is just a mysql client role, and that's fine.

mediawiki role requires scap, which is role::scap::master. However, scap itself requires the deployment_server, which we need to first unify labs / prod for before going there.

So next order of business is to unify role::deployment:deployment_servers. It is a silly name, we should maybe call it 'role::deployment::server' maybe?

So I now have a WIP https://gerrit.wikimedia.org/r/#/c/195340/ where I am basically hacking away at things until this kind of works. I'll split that up into separate patches later on.

Change 195340 had a related patch set uploaded (by Greg Grossmeier):
deployment: Combine labs/prod deployment server roles

https://gerrit.wikimedia.org/r/195340

Deep, deep down in a rabbit hole...

So I got it to try to install scap via trebuchet, and that kept failing. According to https://wikitech.wikimedia.org/wiki/Trebuchet it looks like first deploy has to be manual (EUGH?), so I did that and scap is now installed....

Now to get scap to *work*, so that the mediawiki role can be installed :)

Manual steps I've had to do so far:

  1. git deploy start / sync on scap
  2. Clone /srv/mediawiki-staging to be mediawiki-config
  3. Clone /srv/mediawiki-staging/php-master to be mediawiki itself
  4. Create /home/l10nupdate
  5. Create /srv/mediawiki-staging/private/PrivateSettings.php
greg added a subscriber: greg.Mar 10 2015, 3:20 PM

According to https://wikitech.wikimedia.org/wiki/Trebuchet it looks like first deploy has to be manual (EUGH?), so I did that and scap is now installed....

That is dirty dirty dirty dirty dirty dirty. No sir I don't like it.

@greg me neither. the plan is for me to get this patch in, then destroy staging-tin, and rebuild it. Repeat until 0 manual steps remain.

Alright, looks like I'll have to merge https://gerrit.wikimedia.org/r/#/q/status:open+project:operations/puppet+branch:production+topic:ssh-userkey,n,z to get this done without introducing hacks. I'll do that tomorrow.

mmodell moved this task from Backlog to In Progress on the Staging board.Mar 12 2015, 4:57 PM

Change 196773 had a related patch set uploaded (by Yuvipanda):
mediawiki: Ensure that /etc/php5/apache dir exists

https://gerrit.wikimedia.org/r/196773

Change 197355 had a related patch set uploaded (by Yuvipanda):
scap: Clone mediawiki-config on all scap masters

https://gerrit.wikimedia.org/r/197355

Change 197355 merged by Yuvipanda:
scap: Clone mediawiki-config on all scap masters

https://gerrit.wikimedia.org/r/197355

P409 is the 'bootstrap' script for git-deploy, should be run manually once on the deployment server (tin)

Note that ^ is a total hack :D should be fixed in git-deploy itself... at some point... by someone...

greg triaged this task as Medium priority.Mar 20 2015, 6:50 PM
greg set Security to None.

Change 204080 had a related patch set uploaded (by Thcipriani):
Add submodules to master checkoutMediaWiki

https://gerrit.wikimedia.org/r/204080

This all has to be puppetized/whatevered, but here are the steps, roughly, to a working mediawiki (sans database, guessing that will come from addWiki.php, still to be investigated). This does depend on a handful of patches that need to be merged.

  1. /srv/mediawiki-staging/multiversion/checkoutMediaWiki master php-master
  2. /srv/mediawiki-staging/multiversion/refreshWikiversionsCDB
  3. cp /srv/mediawiki-staging/private/PrivateSettings.php.example /srv/mediawiki-staging/private/PrivateSettings.php
  4. mwscript rebuildLocalisationCache.php --wiki=aawiki --outdir=/tmp/scap_l10n_$(date +%s)
  5. sudo -u l10nupdate cp -R /tmp/scap_l10n_1429139458/l10n_cache-*.cdb /srv/mediawiki-staging/php-master/cache/l10n/
  6. cd /srv/mediawiki-staging/wmf-config
  7. mwscript mergeMessageFileList.php --wiki=aawiki --list-file=wmf-config/extension-list --output=/tmp/ExtensionMessages-master.php
  8. sudo cp -R /tmp/ExtensionMessages-master.php /srv/mediawiki-staging/wmf-config
  9. sync-common

Change 204080 merged by jenkins-bot:
Add submodules to master checkoutMediaWiki

https://gerrit.wikimedia.org/r/204080

thcipriani moved this task from In Progress to Backlog on the Staging board.May 27 2015, 3:05 PM
yuvipanda removed yuvipanda as the assignee of this task.Jun 7 2015, 4:52 PM

Change 196773 abandoned by Yuvipanda:
mediawiki: Ensure that /etc/php5/apache dir exists

https://gerrit.wikimedia.org/r/196773

mmodell closed this task as Declined.Feb 3 2016, 5:34 PM
mmodell claimed this task.
mmodell added a subscriber: mmodell.

staging project is no longer