Page MenuHomePhabricator
Create Task
Maniphest T256005

Move integration/docroot.git deployment to scap
Closed, ResolvedPublic
Actions

Description

integration/docroot.git is used as the Apache DocumentRoot for https://doc.wikimedia.org/ and https://integration.wikimedia.org/ . Both websites used to be on the same host which also happened to run CI we just had a Jenkins job to just git pull.

With time we have moved doc.wikimedia.org to a standalone host doc1001.eqiad.wmnet and we plan to add a second host. The deployment is done there by doing a ssh to the machine and issuing a git pull.

We also have two CI masters (one is a spare one though). One is automagically by the legacy job but the other is stall.

The system is too complicated, targets 3 hosts (and later 4) and has two different deployment methods.

We should migrate to use Scap instead.

Details

SubjectRepoBranchLines +/-
Stop autodeployment of integration/docrootintegration/configmaster+0 -22
Revert "Fix scap config for integration/docroot"operations/puppetproduction+1 -31
Fix scap config for integration/docrootoperations/puppetproduction+2 -1
scap configuration for integration/docrootoperations/puppetproduction+30 -1
scap configurationintegration/docrootmaster+5 -0
Add fake ssh key pair for integration/docroot deploymentlabs/privatemaster+4 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

hashar created this task.Jun 22 2020, 2:43 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 22 2020, 2:43 PM
gerritbot added a comment.Jun 22 2020, 2:43 PM

Change 607041 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Stop autodeployment of integration/docroot

https://gerrit.wikimedia.org/r/607041

gerritbot added a project: Patch-For-Review.Jun 22 2020, 2:43 PM
RhinosF1 updated the task description. (Show Details)Jun 22 2020, 2:46 PM
Restricted Application added a subscriber: RhinosF1. · View Herald TranscriptJun 22 2020, 2:46 PM
gerritbot mentioned this in T149924: Clear /srv/.git on contint1001; move integration.wikimedia.org docroot to new location.Jun 22 2020, 2:46 PM
gerritbot mentioned this in T224591: Migrate contint* hosts to Buster.
gerritbot added a comment.Jun 22 2020, 3:07 PM

Change 607053 had a related patch set uploaded (by Hashar; owner: Hashar):
[labs/private@master] Add fake ssh key pair for integration/docroot deployment

https://gerrit.wikimedia.org/r/607053

gerritbot added a comment.Jun 22 2020, 3:07 PM

Change 607053 merged by Hashar:
[labs/private@master] Add fake ssh key pair for integration/docroot deployment

https://gerrit.wikimedia.org/r/607053

hashar mentioned this in rLPRIea407abb71ca: Add fake ssh key pair for integration/docroot deployment.Jun 22 2020, 3:13 PM
gerritbot added a comment.Jun 22 2020, 3:15 PM

Change 607055 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/docroot@master] scap configuration

https://gerrit.wikimedia.org/r/607055

gerritbot added a comment.Jun 22 2020, 3:16 PM

Change 607056 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] scap configuration for integration/docroot

https://gerrit.wikimedia.org/r/607056

hashar claimed this task.Jun 22 2020, 3:17 PM
hashar added a comment.Jun 22 2020, 4:18 PM

Pending review :)

integration/docroot scap.cfghttps://gerrit.wikimedia.org/r/#/c/integration/docroot/+/607055/
scap::target in puppethttps://gerrit.wikimedia.org/r/#/c/operations/puppet/+/607056/
hashar triaged this task as Medium priority.Jun 22 2020, 4:18 PM
gerritbot added a comment.Jun 22 2020, 4:40 PM

Change 607076 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] ci: switch integration.wikimedia.org to scap DocumentRoot

https://gerrit.wikimedia.org/r/607076

gerritbot added a comment.Jun 22 2020, 5:22 PM

Change 607055 merged by jenkins-bot:
[integration/docroot@master] scap configuration

https://gerrit.wikimedia.org/r/607055

hashar mentioned this in T256138: Create ssh keypair for integration/docroot deployment with scap.Jun 23 2020, 3:32 PM
hashar added a subtask: T256138: Create ssh keypair for integration/docroot deployment with scap.
hashar moved this task from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)) board.Jun 24 2020, 2:06 PM
Jdforrester-WMF moved this task from Untriaged to In-progress on the Continuous-Integration-Infrastructure board.Jun 25 2020, 5:02 PM
hashar closed subtask T256138: Create ssh keypair for integration/docroot deployment with scap as Resolved.Jun 29 2020, 3:10 PM
gerritbot added a comment.Jul 7 2020, 1:47 PM

Change 607056 merged by Jcrespo:
[operations/puppet@production] scap configuration for integration/docroot

https://gerrit.wikimedia.org/r/607056

hashar added a comment.Jul 7 2020, 2:01 PM

And of course my patch failed to apply on the deployment server:

[15:50:57] <jynus> Error: Execution of '/usr/bin/git -c core.sharedRepository=group clone --recurse-submodules https://gerrit.wikimedia.org/r/mediawiki/services/integration/docroot.git /srv/deployment/integration/docroot' returned 128
gerritbot added a comment.Jul 7 2020, 2:05 PM

Change 610066 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] Fix scap config for integration/docroot

https://gerrit.wikimedia.org/r/610066

gerritbot added a comment.Jul 7 2020, 2:07 PM

Change 610066 merged by Jcrespo:
[operations/puppet@production] Fix scap config for integration/docroot

https://gerrit.wikimedia.org/r/610066

gerritbot added a comment.Jul 7 2020, 2:13 PM

Change 610068 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] Revert "Fix scap config for integration/docroot"

https://gerrit.wikimedia.org/r/610068

hashar mentioned this in T257316: scap::target failed to clone a new deployment repository.Jul 7 2020, 2:19 PM
hashar mentioned this in T257317: scap deploy --init on deployment server fails on first puppet run.
hashar added subtasks: T257317: scap deploy --init on deployment server fails on first puppet run, T257316: scap::target failed to clone a new deployment repository.Jul 7 2020, 2:21 PM
gerritbot added a comment.Jul 7 2020, 2:22 PM

Change 610068 abandoned by Hashar:
[operations/puppet@production] Revert "Fix scap config for integration/docroot"

Reason:
That worked on a second run

https://gerrit.wikimedia.org/r/610068

hashar mentioned this in T257319: scap deploy --init fails on the spare deployment server due to a global scap lock being held.Jul 7 2020, 2:31 PM
hashar added a comment.Jul 7 2020, 2:59 PM

After much troubles with the scap configuration and the keyholder, Jynus and I managed to get it working:

$ scap deploy --no-log-message
14:57:16 Started deploy [integration/docroot@708d3eb]
14:57:16 Deploying Rev: HEAD = 708d3eba6bf056e8bfb9ff516f8ee93108880cab
14:57:16 Started deploy [integration/docroot@708d3eb]: (no justification provided)
14:57:16 
== DEFAULT ==
:* doc1001.eqiad.wmnet
:* contint2001.wikimedia.org
:* contint1001.wikimedia.org
integration/docroot: fetch stage(s): 100% (ok: 3; fail: 0; left: 0)             
integration/docroot: config_deploy stage(s): 100% (ok: 3; fail: 0; left: 0)     
integration/docroot: promote stage(s): 100% (ok: 3; fail: 0; left: 0)           
14:57:20 
== DEFAULT ==
:* doc1001.eqiad.wmnet
:* contint2001.wikimedia.org
:* contint1001.wikimedia.org
integration/docroot: finalize stage(s): 100% (ok: 3; fail: 0; left: 0)          
14:57:21 Finished deploy [integration/docroot@708d3eb]: (no justification provided) (duration: 00m 05s)
14:57:21 Finished deploy [integration/docroot@708d3eb] (duration: 00m 05s)
hashar mentioned this in T257329: After arming a new key in keyholder, the identity file path does not show up.Jul 7 2020, 3:11 PM
hashar added a subtask: T257329: After arming a new key in keyholder, the identity file path does not show up.
hashar added a subtask: T257319: scap deploy --init fails on the spare deployment server due to a global scap lock being held.Jul 7 2020, 3:53 PM
hashar closed subtask T257319: scap deploy --init fails on the spare deployment server due to a global scap lock being held as Resolved.Jul 7 2020, 3:55 PM
hashar closed subtask T257316: scap::target failed to clone a new deployment repository as Resolved.
hashar mentioned this in T257413: scap configuration in puppet defaults to forge the git repo name with 'mediawiki/services/'.Jul 8 2020, 9:13 AM
hashar added a subtask: T257413: scap configuration in puppet defaults to forge the git repo name with 'mediawiki/services/'.
• jcrespo closed subtask T257329: After arming a new key in keyholder, the identity file path does not show up as Resolved.Jul 8 2020, 2:24 PM
thcipriani edited projects, added Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)); removed Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)).Jul 8 2020, 5:34 PM
hashar moved this task from INBOX to New Work on the Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)) board.Jul 27 2020, 1:36 PM
gerritbot added a comment.Aug 26 2020, 4:50 PM

Change 607041 merged by jenkins-bot:
[integration/config@master] Stop autodeployment of integration/docroot

https://gerrit.wikimedia.org/r/607041

hashar closed this task as Resolved.Sep 2 2020, 5:13 PM

We have scap all setup and can deploy on all hosts.

doc.wikimedia.org is not using the deployment yet, that needs some refactoring in the integration/docroot PHP app. Tracked in parent task T149924

hashar closed subtask T257413: scap configuration in puppet defaults to forge the git repo name with 'mediawiki/services/' as Resolved.Dec 8 2020, 2:07 PM
Izno removed a subtask: T257317: scap deploy --init on deployment server fails on first puppet run.Jun 1 2021, 11:13 PM
hashar added a subtask: T257317: scap deploy --init on deployment server fails on first puppet run.Jun 7 2021, 1:54 PM
hashar added a subscriber: Izno.Jun 7 2021, 1:57 PM

@Izno please do not mess with task relationships. Solving this task had a bunch of defects in the way we add a new repository to our deployment system (scap), most got addressed and the task that you have removed is related but still need to be addressed. It is intentionally kept as a sub task so that I can track all the issues we had (or in this case still have). Thank you.

Jdforrester-WMF removed a subtask: T257317: scap deploy --init on deployment server fails on first puppet run.Nov 30 2021, 8:21 PM
hashar added a subtask: T257317: scap deploy --init on deployment server fails on first puppet run.Apr 20 2022, 1:44 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits