Page MenuHomePhabricator

scap3 + git-fat results in git status with permissions errors
Open, Needs TriagePublic

Description

On stat1007.eqiad.wmnet in /srv/deployment/wikimedia/discovery/analytics:

ebernhardson@stat1007:/srv/deployment/wikimedia/discovery/analytics$ git status
Traceback (most recent call last):
  File "/usr/bin/git-fat", line 698, in <module>
    fat.cmd_filter_clean()
  File "/usr/bin/git-fat", line 330, in cmd_filter_clean
    self.filter_clean(sys.stdin, sys.stdout)
  File "/usr/bin/git-fat", line 283, in filter_clean
    fd, tmpname = tempfile.mkstemp(dir=self.objdir)
  File "/usr/lib/python2.7/tempfile.py", line 314, in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags)
  File "/usr/lib/python2.7/tempfile.py", line 244, in _mkstemp_inner
    fd = _os.open(file, flags, 0600)
OSError: [Errno 13] Permission denied: '.git/fat/objects/tmpW8BFEl'
error: external filter 'git-fat filter-clean' failed 1
error: external filter 'git-fat filter-clean' failed

[ strip repeated errors ]

HEAD detached at 67293be
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   artifacts/kafka-1.3.5-py2.py3-none-any.whl
        modified:   artifacts/pip-18.0-py2.py3-none-any.whl
        modified:   artifacts/wheel-0.31.1-py2.py3-none-any.whl

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        oozie/transfer_to_es/venv.zip

no changes added to commit (use "git add" and/or "git commit -a")

This interferes with another script that deploys the repository to hdfs for oozie. That script checks git status to see if the repo is dirty and it always comes up dirty since it can't check the artifacts against the git fat objects.

The problem seems to be git-fat trying to create a temporary file inside .git/fat/objects and being rejected. Only deploy-service has write access to .git/fat/objects (which seems completely reasonable).

Event Timeline

Some random background on what's happening here:

git status triggers the git clean filter when the st_time of a file on disk is the same or older than the st_time for that file stored in its index. When the git clean filter is triggered, git fat filter-clean is called with the content of the file to "clean" written to stdin of that process.

The first thing git-fat does is to determine if the file you are attempting to clean is "hanging" (that is, is it a git-fat stub, or is it a real file). Git's smudge and clean filters (unfortunately) work by sending the contents of a file to the stdin of your filter, and writing whatever shows up on stdout of the filter. On clean, git-fat accepts a file on stdin, writes that file to a temporary location, checks to see if it's the magic 74bytes and looks like a git-fat stub and if not, dumps it to stdout (https://github.com/jedbrown/git-fat/blob/master/git-fat#L239-L246) which is what you're seeing happen here.