14:23 < bd808|dep> mutante: Trying to get deploy sync scap from tin with
result of "0/230 minions completed fetch"
Version: wmf-deployment
Severity: blocker
14:23 < bd808|dep> mutante: Trying to get deploy sync scap from tin with
result of "0/230 minions completed fetch"
Version: wmf-deployment
Severity: blocker
Setting it to Immediate importance.
Opsen: please triage/assign to whoever will debug/diagnose.
14:39 < bd808> mutante: I would say that May 7 22:49 was the last time
`git deploy sync` was used on the scap repo
tin:/srv/deployment/scap/scap (git master)
bd808$ git deploy sync
Repo: scap/scap
Tag: scap/scap-sync-20140520-181227
0/230 minions completed fetch
Continue? ([d]etailed/[C]oncise report,[y]es,[n]o,[r]etry): r
Continue? ([d]etailed/[C]oncise report,[y]es,[n]o,[r]etry): r
Continue? ([d]etailed/[C]oncise report,[y]es,[n]o,[r]etry): d
Repo: scap/scap
Tag: scap/scap-sync-20140520-181227
0/230 minions completed fetch
Details:
mw1068.eqiad.wmnet:
fetch status: 10 [started: 0 mins ago, last-return: 0 mins ago]
[..snip..]
Continue? ([d]etailed/[C]oncise report,[y]es,[n]o,[r]etry): r
Continue? ([d]etailed/[C]oncise report,[y]es,[n]o,[r]etry): c
Repo: scap/scap
Tag: scap/scap-sync-20140520-181227
0/230 minions completed fetch
Continue? ([d]etailed/[C]oncise report,[y]es,[n]o,[r]etry): n
Not continuing to checkout phase. A deployment is still underway, please finish,
sync, or abort.
tin:/srv/deployment/scap/scap (git master)
bd808$ git deploy abort
Deployment aborted.
$ ls -ld /srv/deployment/scap/scap
drwxrws--- 6 trebuchet wikidev 4096 May 1 20:07 /srv/deployment/scap/scap/
Directory permissions are borked. The apache user needs to be able to read the directory in order to serve it to the minions. Should be "drwxrwsr-x".
14:59 < Ryan_Lane> chmod o+rx /srv/deployment/scap/scap/
14:59 < mutante> already done
(after test of git-deploy of scap)
15:01 < bd808> greg-g: \o/
Directory permissions were probably broken initially because in the current setup tin is acting as both the deploy server and a minion for the scap deploy. When the minion gets the fetch/checkout commands from salt it runs as root. I sort of wondered if this would cause problems and I think now I have my answer.
We can either try to rearrange the way that classes are defined in operations/puppet.git to eliminate this problem or figure out how to make trebuchet aware of the edge case and avoid updating the deployment server if it is also in the minions list.