Page MenuHomePhabricator

Install git-lfs client (at least on scap targets & masters)
Closed, ResolvedPublic

Description

There's debian packages for git-lfs, but they're binary packages so we'll probably want/need to build it ourselves (hopefully not hard, they did the work).

If it's not too hard, we should just install it everywhere we install git, just for consistency. But at the very least it should be installed on all scap-related hosts (masters & targets)

Event Timeline

demon created this task.Nov 15 2017, 7:30 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 15 2017, 7:30 PM

@thcipriani Noticed that it appears it's been added to buster. So should just be a backport?

Halfak added a subscriber: Halfak.Dec 1 2017, 9:17 PM

We'll need this in labs too.

Ottomata triaged this task as Normal priority.Jan 16 2018, 8:09 PM
mmodell moved this task from Needs triage to Debt on the Scap board.Feb 1 2018, 12:22 AM
Halfak awarded a token.Feb 1 2018, 2:52 PM

@akosiaris: What would it take to get the git-lfs package back-ported to stretch? It's written in go, however, I am unsure if it will work with the version of go that's available in stretch.

@akosiaris: What would it take to get the git-lfs package back-ported to stretch? It's written in go, however, I am unsure if it will work with the version of go that's available in stretch.

A direct backporting and rebuilding is impossible as it turns out. There just way too many dependencies that are present in buster and not in stretch. That being said, it's a go app so it compiles to not really dynamically linked executable (aside from libpthread and libc) so the same package provided for buster works on stretch as well (and I did a quick test of git lfs install and git lfs track to verify).

I 'll try and have it uploaded to stretch-wikimedia by the end of the day.

I 've just uploaded git-lfs to stretch-wikimedia/main.

@akosiaris would it be easy to backport that to Jessie too?

@akosiaris Awesome, thank you!

Change 417226 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] scap::target: Install git-lfs

https://gerrit.wikimedia.org/r/417226

Change 417226 merged by Alexandros Kosiaris:
[operations/puppet@production] scap::target: Install git-lfs

https://gerrit.wikimedia.org/r/417226

The scap targets that would benefit from this (namely ores* boxes) now have git-lfs installed. @mmodell do we also need this on the scap masters ? I am not fully clear about the workflow that is going to be used here and where the git lfs related files are going to be fetched from.

mmodell added a comment.EditedMar 12 2018, 6:02 PM

@akosiaris: I think it's needed on masters, at least to enable deployers to issue git-lfs commands. I'm unsure if scap itself does git-lfs commands on the master, I believe it's only on targets.

@akosiaris: I think it's needed on masters, at least to enable deployers to issue git-lfs commands. I'm unsure if scap itself does git-lfs commands on the master, I believe it's only on targets.

Why do the deployers issue git-lfs commands on the masters? Is it required per the process ? or just beneficial ? or nothing of the above ? Should we jot down the workflow (and even get it working in beta or some other environment) before we start trying it in production ?

@akosiaris: tl;dr I can't think of any reason that we must have git-lfs on masters, I've only got vague hand-wavy notions about it being convenient.

I don't know if the workflow requires it, I was really just imagining that it'd be convenient for debugging. Now that you mention it, I agree, it would be a good idea to figure out the workflow and get it written down somewhere. Trying it on beta is probably a good idea as well.

I've double-checked and it appears that we are only calling git-lfs/git-fat from one place in deploy-local, so git-lfs definitely isn't required (by scap) to be installed on masters.

Maybe @demon or @awight have a better idea than I do about the workflow. I'm only just getting familiar with git-lfs myself and I have never used git-fat (which is essentially the same workflow, afaik)..

demon added a comment.Mar 14 2018, 4:29 PM

I can't think of any compelling reason why it would be *required* on the masters...but I could see it as being useful if you want to actually have your large file contents on the master if you're debugging/curious about something...?

I can't think of any compelling reason why it would be *required* on the masters...but I could see it as being useful if you want to actually have your large file contents on the master if you're debugging/curious about something...?

This, exactly ^

Change 420409 had a related patch set uploaded (by Awight; owner: Awight):
[operations/puppet@production] Install git-lfs on scap source and target

https://gerrit.wikimedia.org/r/420409

The catch seems to be that tin and deployment-tin are running jessie, so the git-lfs package isn't easily available.

tin will be replaced very soon

tin has been replaced by deploy1001 which is running stretch

[deploy1001:~] $ apt-cache show git-lfs
Package: git-lfs
Version: 2.3.4-1

(not installed yet though)

Change 420409 merged by Alexandros Kosiaris:
[operations/puppet@production] Install git-lfs on scap source and target

https://gerrit.wikimedia.org/r/420409

Confirmed that git-lfs is installed on deploy1001, removing blocking task dependency.

awight closed this task as Resolved.Jun 12 2018, 3:00 PM
awight claimed this task.