Page MenuHomePhabricator

Setup staging-tin as deployment host
Closed, DeclinedPublic

Description

For deployment fun and profit

Event Timeline

demon raised the priority of this task from to Needs Triage.
demon updated the task description. (Show Details)
demon added a project: Staging.
demon subscribed.

Let's look at the roles on tin:

include standard
include role::deployment::deployment_servers::production
include mediawiki
include role::scap::master
include mysql
include role::labsdb::manager
include ssh::hostkeys-collect
include role::releases::upload
include admin

Of this, quite a few aren't needed on staging. Obvious ones being admin, standard, labsdb::manager, releases::upload.

That leaves:

  1. role::deployment::deployment_servers::production
  2. ssh::hostkeys-collect
  3. mediawiki
  4. role::scap::master
  5. mysql

We'll investigate and see what all needs to happen :)

Alright, so 'mysql' is just a mysql client role, and that's fine.

mediawiki role requires scap, which is role::scap::master. However, scap itself requires the deployment_server, which we need to first unify labs / prod for before going there.

So next order of business is to unify role::deployment:deployment_servers. It is a silly name, we should maybe call it 'role::deployment::server' maybe?

So I now have a WIP https://gerrit.wikimedia.org/r/#/c/195340/ where I am basically hacking away at things until this kind of works. I'll split that up into separate patches later on.

Change 195340 had a related patch set uploaded (by Greg Grossmeier):
deployment: Combine labs/prod deployment server roles

https://gerrit.wikimedia.org/r/195340

Deep, deep down in a rabbit hole...

So I got it to try to install scap via trebuchet, and that kept failing. According to https://wikitech.wikimedia.org/wiki/Trebuchet it looks like first deploy has to be manual (EUGH?), so I did that and scap is now installed....

Now to get scap to *work*, so that the mediawiki role can be installed :)

Manual steps I've had to do so far:

  1. git deploy start / sync on scap
  2. Clone /srv/mediawiki-staging to be mediawiki-config
  3. Clone /srv/mediawiki-staging/php-master to be mediawiki itself
  4. Create /home/l10nupdate
  5. Create /srv/mediawiki-staging/private/PrivateSettings.php

According to https://wikitech.wikimedia.org/wiki/Trebuchet it looks like first deploy has to be manual (EUGH?), so I did that and scap is now installed....

That is dirty dirty dirty dirty dirty dirty. No sir I don't like it.

@greg me neither. the plan is for me to get this patch in, then destroy staging-tin, and rebuild it. Repeat until 0 manual steps remain.

Alright, looks like I'll have to merge https://gerrit.wikimedia.org/r/#/q/status:open+project:operations/puppet+branch:production+topic:ssh-userkey,n,z to get this done without introducing hacks. I'll do that tomorrow.

Change 196773 had a related patch set uploaded (by Yuvipanda):
mediawiki: Ensure that /etc/php5/apache dir exists

https://gerrit.wikimedia.org/r/196773

Change 197355 had a related patch set uploaded (by Yuvipanda):
scap: Clone mediawiki-config on all scap masters

https://gerrit.wikimedia.org/r/197355

Change 197355 merged by Yuvipanda:
scap: Clone mediawiki-config on all scap masters

https://gerrit.wikimedia.org/r/197355

P409 is the 'bootstrap' script for git-deploy, should be run manually once on the deployment server (tin)

Note that ^ is a total hack :D should be fixed in git-deploy itself... at some point... by someone...

greg triaged this task as Medium priority.Mar 20 2015, 6:50 PM
greg set Security to None.

Change 204080 had a related patch set uploaded (by Thcipriani):
Add submodules to master checkoutMediaWiki

https://gerrit.wikimedia.org/r/204080

This all has to be puppetized/whatevered, but here are the steps, roughly, to a working mediawiki (sans database, guessing that will come from addWiki.php, still to be investigated). This does depend on a handful of patches that need to be merged.

  1. /srv/mediawiki-staging/multiversion/checkoutMediaWiki master php-master
  2. /srv/mediawiki-staging/multiversion/refreshWikiversionsCDB
  3. cp /srv/mediawiki-staging/private/PrivateSettings.php.example /srv/mediawiki-staging/private/PrivateSettings.php
  4. mwscript rebuildLocalisationCache.php --wiki=aawiki --outdir=/tmp/scap_l10n_$(date +%s)
  5. sudo -u l10nupdate cp -R /tmp/scap_l10n_1429139458/l10n_cache-*.cdb /srv/mediawiki-staging/php-master/cache/l10n/
  6. cd /srv/mediawiki-staging/wmf-config
  7. mwscript mergeMessageFileList.php --wiki=aawiki --list-file=wmf-config/extension-list --output=/tmp/ExtensionMessages-master.php
  8. sudo cp -R /tmp/ExtensionMessages-master.php /srv/mediawiki-staging/wmf-config
  9. sync-common

Change 204080 merged by jenkins-bot:
Add submodules to master checkoutMediaWiki

https://gerrit.wikimedia.org/r/204080

Change 196773 abandoned by Yuvipanda:
mediawiki: Ensure that /etc/php5/apache dir exists

https://gerrit.wikimedia.org/r/196773

mmodell claimed this task.
mmodell subscribed.

staging project is no longer