Page MenuHomePhabricator

Use scap to deploy on apaches
Closed, ResolvedPublic

Description

Scap is the deployment tool used for production cluster deploys and one of the goals of beta is to track production as closely as is reasonable (bug 51494). Using scap to push MediaWiki updates across the beta cluster will provide a good test of changes to the scap codebase. It may also improve the speed of the wikis in beta slightly by allowing them to use local storage rather than NFS for l10n cache and the MediaWiki php code.

There was a previous effort for this documented in bug 37078 which was WONTFIXED a year ago. At that time git-deploy (nee trebuchet) was slated as the replacement. Although git-deploy was configured (see bug 43339) at some point this was abandoned and shared disk became the method of deployment again.


Version: unspecified
Severity: normal

Details

Reference
bz63746

Event Timeline

bzimport raised the priority of this task from to Normal.Nov 22 2014, 3:11 AM
bzimport set Reference to bz63746.
bd808 created this task.Apr 9 2014, 8:51 PM

I have actually been working towards this since the start of the eqiad labs migration for deployment-prep. Thus far we (Antoine, Ryan and myself) have established a project local puppet master and a dedicated salt server that remove many previous roadblocks to making sweeping changes to the beta environment.

I have recently made puppet changes to configure a dedicated user (mwdeploy) to have ssh access across the beta cluster. I will continue to build on that with more puppet-managed configuration that will replicate the master-slave rsync servers used in production and associated configuration needed in scap itself.

The end goal here will be to have scap triggered automatically by jenkins each time that the current shared code checkout is updated. This will push the latest code and configuration changes staged into beta by jenkins out to the apaches.

Change 123674 had a related patch set uploaded by BryanDavis:
[WIP] Configure scap master and clients in beta

https://gerrit.wikimedia.org/r/123674

Change 125359 had a related patch set uploaded by BryanDavis:
Fix IRCSocketHandler to work from Jenkins

https://gerrit.wikimedia.org/r/125359

An experimental Jenkins job (https://integration.wikimedia.org/ci/job/beta-scap-eqiad/) has been created to run a scap after each beta-code-update-eqiad job finishes. This will probably need some tweaks but we are getting pretty close to a working setup.

Change 125359 merged by jenkins-bot:
Fix IRCSocketHandler to work from Jenkins

https://gerrit.wikimedia.org/r/125359

Change 125515 had a related patch set uploaded by BryanDavis:
Make logging destinations configurable

https://gerrit.wikimedia.org/r/125515

deployment-{apache0[12],videoscaler01,jobrunner01} are now using the copy of MediaWiki that scap manages in /srv/common-local.

Change 125515 merged by jenkins-bot:
Make logging destinations configurable

https://gerrit.wikimedia.org/r/125515

Change 123674 merged by Ori.livneh:
Configure scap master and clients in beta

https://gerrit.wikimedia.org/r/123674

Change 127399 had a related patch set uploaded by BryanDavis:
Move beta scap source directory off of NFS

https://gerrit.wikimedia.org/r/127399

Change 127399 merged by Ori.livneh:
Move beta scap source directory off of NFS

https://gerrit.wikimedia.org/r/127399