Page MenuHomePhabricator

WIP: Use git to sync proxies
Changes PlannedPublic

Authored by mmodell on Mar 27 2017, 3:57 PM.

Details

Reviewers
dduvall
demon
thcipriani
Group Reviewers
Release-Engineering-Team
Patch without arc
git checkout -b D605 && curl -L https://phabricator.wikimedia.org/D605?download=true | git apply
Summary

This will use git to sync /srv/deployment/mediawiki/deploy on the
deployment master (currently a symlink to the git repo created in
/srv/mediawiki) to /srv/deployment/mediawiki/deploy on the targets.

TODO

  1. Ensure that puppet creates an /srv/deployment/mediawiki that is owned by mwdeploy on the targets so that the symlink swapping can happen therein.
  2. Ensure that it is not problematic for proxies that the mediawiki install is in /srv/deployment/mediawiki/mediawiki this either re-point apaches/rsync servers to this directory or (short-term) make /srv/mediawiki a symlink to /srv/deployment/mediawiki/deploy.
  3. This means 3 checkouts of mediawiki on all hosts. There are filespace ramifications to this.
  4. Find scripts that hard-code /srv/mediawiki and update/target them for update.
  5. Ensure the deploy_dir on proxies is set to /srv/deployment/mediawiki/deploy so that cdbs are rebuilt. I believe we can use the environments feature of the scap.cfg to detect whether or not we are in a "proxy" environment and change the config for the environment. More testing is needed there.

One possibility for rollout would be to sync to all proxies, depool
proxies, point /srv/mediawiki -> /srv/deployment/mediawiki/deploy,
repool proxies.

At the end of this transition period we will have:

  1. A .git repo on the deployment host at /srv/deployment/mediawiki/deploy that represents all deployed files
  2. An instantanous deployment to all proxy nodes (via symlink swapping)
Test Plan

Need to test this alongside puppet patches in some staging environment
since this changes a lot about how mw deployments are done. I've done
preliminary testing to ensure that everything works as expected;
however, moving around directories and apache configs is going to be the
primary work here. I expect that this will span mw-config + puppet +
scap and a rollout plan is needed.

Revert Plan

I think I would recommend that this live along side the current setup
initially and a revert of a puppet patch + some manual intervention
during rollout should be all that is needed.

Event Timeline

thcipriani created this revision.Mar 27 2017, 3:57 PM
Restricted Application added a reviewer: Release-Engineering-Team. · View Herald TranscriptMar 27 2017, 3:57 PM
Restricted Application added a project: Release-Engineering-Team. · View Herald Transcript
mmodell edited edge metadata.

This looks pretty good, I'll look over the code more thoroughly today / this weekend.

mmodell accepted this revision as: mmodell.Apr 8 2017, 2:15 AM

Can it really be this easy?

This revision is now accepted and ready to land.Apr 8 2017, 2:15 AM
mmodell commandeered this revision.Oct 2 2017, 5:57 PM
mmodell edited reviewers, added: thcipriani; removed: mmodell.

Gonna try to land this but I need to wrap it in a feature flag.

This revision now requires review to proceed.Oct 2 2017, 5:57 PM
mmodell planned changes to this revision.Feb 22 2018, 12:23 AM