Page MenuHomePhabricator

Move integration/docroot.git deployment to scap
Closed, ResolvedPublic

Description

integration/docroot.git is used as the Apache DocumentRoot for https://doc.wikimedia.org/ and https://integration.wikimedia.org/ . Both websites used to be on the same host which also happened to run CI we just had a Jenkins job to just git pull.

With time we have moved doc.wikimedia.org to a standalone host doc1001.eqiad.wmnet and we plan to add a second host. The deployment is done there by doing a ssh to the machine and issuing a git pull.

We also have two CI masters (one is a spare one though). One is automagically by the legacy job but the other is stall.

The system is too complicated, targets 3 hosts (and later 4) and has two different deployment methods.

We should migrate to use Scap instead.

Related Objects

Event Timeline

Change 607041 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Stop autodeployment of integration/docroot

https://gerrit.wikimedia.org/r/607041

Change 607053 had a related patch set uploaded (by Hashar; owner: Hashar):
[labs/private@master] Add fake ssh key pair for integration/docroot deployment

https://gerrit.wikimedia.org/r/607053

Change 607053 merged by Hashar:
[labs/private@master] Add fake ssh key pair for integration/docroot deployment

https://gerrit.wikimedia.org/r/607053

Change 607055 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/docroot@master] scap configuration

https://gerrit.wikimedia.org/r/607055

Change 607056 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] scap configuration for integration/docroot

https://gerrit.wikimedia.org/r/607056

hashar triaged this task as Medium priority.Jun 22 2020, 4:18 PM

Change 607076 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] ci: switch integration.wikimedia.org to scap DocumentRoot

https://gerrit.wikimedia.org/r/607076

Change 607055 merged by jenkins-bot:
[integration/docroot@master] scap configuration

https://gerrit.wikimedia.org/r/607055

Change 607056 merged by Jcrespo:
[operations/puppet@production] scap configuration for integration/docroot

https://gerrit.wikimedia.org/r/607056

And of course my patch failed to apply on the deployment server:

[15:50:57] <jynus> Error: Execution of '/usr/bin/git -c core.sharedRepository=group clone --recurse-submodules https://gerrit.wikimedia.org/r/mediawiki/services/integration/docroot.git /srv/deployment/integration/docroot' returned 128

Change 610066 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] Fix scap config for integration/docroot

https://gerrit.wikimedia.org/r/610066

Change 610066 merged by Jcrespo:
[operations/puppet@production] Fix scap config for integration/docroot

https://gerrit.wikimedia.org/r/610066

Change 610068 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] Revert "Fix scap config for integration/docroot"

https://gerrit.wikimedia.org/r/610068

Change 610068 abandoned by Hashar:
[operations/puppet@production] Revert "Fix scap config for integration/docroot"

Reason:
That worked on a second run

https://gerrit.wikimedia.org/r/610068

After much troubles with the scap configuration and the keyholder, Jynus and I managed to get it working:

$ scap deploy --no-log-message
14:57:16 Started deploy [integration/docroot@708d3eb]
14:57:16 Deploying Rev: HEAD = 708d3eba6bf056e8bfb9ff516f8ee93108880cab
14:57:16 Started deploy [integration/docroot@708d3eb]: (no justification provided)
14:57:16 
== DEFAULT ==
:* doc1001.eqiad.wmnet
:* contint2001.wikimedia.org
:* contint1001.wikimedia.org
integration/docroot: fetch stage(s): 100% (ok: 3; fail: 0; left: 0)             
integration/docroot: config_deploy stage(s): 100% (ok: 3; fail: 0; left: 0)     
integration/docroot: promote stage(s): 100% (ok: 3; fail: 0; left: 0)           
14:57:20 
== DEFAULT ==
:* doc1001.eqiad.wmnet
:* contint2001.wikimedia.org
:* contint1001.wikimedia.org
integration/docroot: finalize stage(s): 100% (ok: 3; fail: 0; left: 0)          
14:57:21 Finished deploy [integration/docroot@708d3eb]: (no justification provided) (duration: 00m 05s)
14:57:21 Finished deploy [integration/docroot@708d3eb] (duration: 00m 05s)

Change 607041 merged by jenkins-bot:
[integration/config@master] Stop autodeployment of integration/docroot

https://gerrit.wikimedia.org/r/607041

We have scap all setup and can deploy on all hosts.

doc.wikimedia.org is not using the deployment yet, that needs some refactoring in the integration/docroot PHP app. Tracked in parent task T149924