I rewrote all of the commits. If you have an existing clone, throw it away and clone a new one. I think more is needed (branches need to be deleted)

https://github.com/wikimedia/WikimediaUI-Style-Guide/pull/249 Contributing.md update.

In the course we've also needed to force push gerrit clone again as history diverged.
And lfs config for design/style-guide repo needed to be added https://gerrit.wikimedia.org/r/#/c/All-Projects/+/543173/ – thanks @mmodell

In the course, the force-push privs for the Gerrit repo had to be removed temporarily in order to update the repo.

@Ladsgroup git-lfs is not installed on the prod servers and the puppet git:::clone class also does not support changing the command yet. So this breaks cloning on the prod servers.

Change 547778 had a related patch set uploaded (by 20after4; owner: 20after4):
[operations/puppet@production] Add git::lfs on design/style-guide targets

https://gerrit.wikimedia.org/r/547778

Related T232494

It would be very good to get clarification if Git LFS is going to be supported long-term or not. @20after4 @akosiaris

@Volker_E: I agree, some clarity would be good.

Technically:

• scap nominally supports git-lfs but we have very little experience with it in production.
• Gerrit supports git-lfs but it's not the greatest implementation.

Beyond that, as far as institutional support, I really don't know where we stand.

cc: @thcipriani

I'd love to hear what @akosiaris and @thcipriani think about it.

Also I want to voice my huge appreciation to @Dzahn for all the help so far on this. Daniel has gone above and beyond all reasonable expectations, worked through lunch, answered tons of questions, and generally been an awesome SRE.

In T235013#5627732, @mmodell wrote:

@Volker_E: I agree, some clarity would be good.

Technically:

• scap nominally supports git-lfs but we have very little experience with it in production.
• Gerrit supports git-lfs but it's not the greatest implementation.

Beyond that, as far as institutional support, I really don't know where we stand.

cc: @thcipriani

In T235013#5627747, @mmodell wrote:

I'd love to hear what @akosiaris and @thcipriani think about it.

I don't know if anyone is using gerrit and scap's "support" for git-lfs or not in production(?). ORES uses something for large files, but I've never looked into what exactly they do; maybe @Ladsgroup or @akosiaris could speak to that?

git-lfs is not well supported has been my impression. We support git-fat pretty well in scap — not sure if that could be made to work in this case.

Also I want to voice my huge appreciation to @Dzahn for all the help so far on this. Daniel has gone above and beyond all reasonable expectations, worked through lunch, answered tons of questions, and generally been an awesome SRE.

+1 <3 @Dzahn :)

Same here, both @mmodell and @Dzahn have provided valuable input and hands-on help extensively.

Design Team's goals and requirements have been to provide history to the whole Style Guide including binary design files used by collaborators and volunteers in and outside the Foundation. And to make the contribution hurdle to the repository as low as possible. Reason for git lfs came out of a recommendation to avoid blowing up the Git history due to the known (performance & bandwidth) limitations of Git with large binary files.

Some of the confusion is my fault, I've been saying that git-lfs works, and every indication I've seen has lead me to believe that it does. There are, howver, some unknowns:

• Nobody in Release-Engineering-Team is quite sure that ores ever used git-lfs, for real in production or if it was just a failed experiment.
• scap's support for lfs was committed by @chad with relatively little testing
• I'm not 100% sure that the target machines (vega and bromine) can talk to gerrit directly. There may be a firewall in the way. For git-lfs to work, the scap deploy-local process on the targets needs to call git-lfs which will make an https request back to gerrit to download large files.

As stated by @thcipriani, git-fat is rather well tested and doesn't have so many unknowns. Since that is the more well worn path, it's probably safer to use that instead of git-lfs, at least for now. There are instructions for setting it up on wikitech: https://wikitech.wikimedia.org/wiki/Archiva#Setting_up_git-fat_for_your_project and it's not much different from git-lfs. The two have a similar design and essentially equivalent functionality.

I will modify the git-lfs page on wikitech to indicate that it's currently unsupported (e.g. use at your own risk) and that we have no immediate plans to invest in developing it further.

Note: This does not preclude working on it in the future, it just isn't on the roadmap for this quarter and would need to be planned / prioritized appropriately if we decide to do it.

Cons for git-lfs: it doesn't support ssh, only http(s), although apparently there's an authentication shim "git-lfs-authenticate". Pros for git-fat: it uses rsync for delivery of the large files, which can mean significant bandwidth and time savings.

Do we have a logical place to put files for git-fat, separated nicely by repo name? Is this a leap we want to make generally for our repos?

In T235013#5580957, @Dzahn wrote:

@Ladsgroup git-lfs is not installed on the prod servers and the puppet git:::clone class also does not support changing the command yet. So this breaks cloning on the prod servers.

That's not entirely correct. git-lfs is installed everywhere that scap::target is used, that is 598/1371 hosts. But that's tangential and more a byproduct of how we config scap and less a conscious choice.

ORES does use indeed git-lfs but it's the only thing that I know that uses it.

I'm not 100% sure that the target machines (vega and bromine) can talk to gerrit directly. There may be a firewall in the way. For git-lfs to work, the scap deploy-local process on the targets needs to call git-lfs which will make an https request back to gerrit to download large files.

They can.

Do we have a logical place to put files for git-fat, separated nicely by repo name? Is this a leap we want to make generally for our repos?

This has been historically connected to archiva, so that jar files are deployed without needing to have them in the git repo itself. And as far as I know it can be separated by repo name.

Do note that github.com does limit git lfs bandwith[1] to 1G and used space to 1G. It's surprisingly easy to reach them. Do note that this might cause issues to contributors even if we manage to avoid them (e.g. using gerrit)

I will modify the git-lfs page on wikitech to indicate that it's currently unsupported (e.g. use at your own risk) and that we have no immediate plans to invest in developing it further.

I think this line alone responds to the question of whether we should be investing in git-lfs more or not. Unless some other team can show up and take over the work of supporting git-lfs.

[1] https://help.github.com/en/github/managing-large-files/about-storage-and-bandwidth-usage

Change 547778 abandoned by 20after4:
Add git::lfs on design/style-guide targets

Reason:
unneeded

https://gerrit.wikimedia.org/r/547778

Just wanted to add something i recently noticed while upgrading the Gerrit replica. 44% of the entire Gerrit git data size was "design" due to the large files.

But when i checked now again i see it has been fixed and we are back to a much more reasonable 3.5%

The size of /srv/gerrit/git (all of Gerrit git data) is now down to just 38G again.

So thanks to who pruned the history there or something similar.

In T235013#5641942, @Dzahn wrote:

But when i checked now again i see it has been fixed and we are back to a much more reasonable 3.5%

The size of /srv/gerrit/git (all of Gerrit git data) is now down to just 38G again.

Interesting.

Much appreciation to @akosiaris and @ArielGlenn for further clarity on this! You've cleared up a lot of confusion on this issue.

In T235013#5641952, @mmodell wrote:

Interesting.

Actually 1001 and 2001 are different. I noticed it when upgrading 2001.

[gerrit1001:/] $ sudo du -hs /srv/gerrit/git
38G	/srv/gerrit/git
[gerrit1001:/] $ sudo du -hs /srv/gerrit/git/design
1.3G	/srv/gerrit/git/design

[gerrit2001:~] $ sudo du -hs /srv/gerrit/git
67G	/srv/gerrit/git
[gerrit2001:~] $ sudo du -hs /srv/gerrit/git/design
30G	/srv/gerrit/git/design

[gerrit1001:~] $ du -hs /srv/gerrit/git/design/**
13M	/srv/gerrit/git/design/landing-page.git
39M	/srv/gerrit/git/design/strategy.git
1.3G	/srv/gerrit/git/design/style-guide.git

[gerrit2001:/srv/gerrit/git] $ du -hs design/**
13M	design/landing-page.git
39M	design/strategy.git
30G	design/style-guide.git

So i'm not entirly sure how gerrit2001 managed to grow like this. I wonder if we should delete on gerrit2001 and just replicate a fresh copy to see.

Most of that 30GB is in packfiles. Those packfiles all seem to contain the same objects:

thcipriani@gerrit2001:~$ du -chs /srv/gerrit/git/design/style-guide.git/objects/pack/* | sort -rh | head -4
30G     total
220M    /srv/gerrit/git/design/style-guide.git/objects/pack/pack-dc13f6820e29fa1245d9ecbb804edb77e63224ee.pack
220M    /srv/gerrit/git/design/style-guide.git/objects/pack/pack-69e918208071f2dac83e1ab9d3adc9d85f239a5c.pack
220M    /srv/gerrit/git/design/style-guide.git/objects/pack/pack-687d80a14d5590ade85bc101eb042671d187bf64.pack
thcipriani@gerrit2001:~$ git -C /srv/gerrit/git/design/style-guide.git verify-pack -v /srv/gerrit/git/design/style-guide.git/objects/pack/pack-dc13f6820e29fa1245d9ecbb804edb77e63224ee.idx | grep -v chain | sort -k3nr | head -n2
198f171c78e5b4f6bc37db2311cdb2435c80937f blob   39633398 29773531 13361086
4461a416b7039735425b8ac366a49f650ad6f3fc blob   39235142 38823785 187428478
thcipriani@gerrit2001:~$ git -C /srv/gerrit/git/design/style-guide.git verify-pack -v /srv/gerrit/git/design/style-guide.git/objects/pack/pack-69e918208071f2dac83e1ab9d3adc9d85f239a5c.idx | grep -v chain | sort -k3nr | head -n2
198f171c78e5b4f6bc37db2311cdb2435c80937f blob   39633398 29773531 13361827
4461a416b7039735425b8ac366a49f650ad6f3fc blob   39235142 38823785 187423835
thcipriani@gerrit2001:~$ git -C /srv/gerrit/git/design/style-guide.git verify-pack -v /srv/gerrit/git/design/style-guide.git/objects/pack/pack-687d80a14d5590ade85bc101eb042671d187bf64.idx | grep -v chain | sort -k3nr | head -n2
198f171c78e5b4f6bc37db2311cdb2435c80937f blob   39633398 29773531 13361066
4461a416b7039735425b8ac366a49f650ad6f3fc blob   39235142 38823785 187419655

^ there are just a whole bunch of pack files with similar (large) contents. My guess is that these are all generated by fetches by git's smart-protocol and will be gc'd eventually.

I'm not sure about git fat, the original repo is on github and using git fat in github doesn't look super easy to me.

In T235013#5640726, @akosiaris wrote:

Do note that github.com does limit git lfs bandwith[1] to 1G and used space to 1G. It's surprisingly easy to reach them. Do note that this might cause issues to contributors even if we manage to avoid them (e.g. using gerrit)

[1] https://help.github.com/en/github/managing-large-files/about-storage-and-bandwidth-usage

We have a coupon that makes everything in github.com/wikimedia free forever, we already pass that limit for ores so I "bought" a base-level package for git lfs for $5/month which turned to $0.0 because of the coupon but it required me to put a billing point to it, so now if you go to settings of wikimedia org in github, you will see my paypal there (please don't remove the coupon then). I asked WMF people to fix this for years and nothing happened. I'm going to remove my billing if nothing happens and then git lfs for github.com will be broken.

With Figma on the brink of being our main source of truth for Design Style Guide image resources and components, I want to get a clear statement how much above is still an issue for RelEng?
Considering after switching fully over to web-based Figma for large binary resources to archive https://github.com/wikimedia/WikimediaUI-Style-Guide/ and start a new repo https://github.com/wikimedia/Wikimedia-Design-Style-Guide/ with no close to non large binary files in it anymore…

@thcipriani @Dzahn @mmodell ?

In T235013#7113897, @Volker_E wrote:

With Figma on the brink of being our main source of truth for Design Style Guide image resources and components, I want to get a clear statement how much above is still an issue for RelEng?

The issue of lots of storage use for design? It looks to be an OK size to me.

4.0K    /srv/gerrit/git/design/style-guide.git/branches
4.0K    /srv/gerrit/git/design/style-guide.git/config
4.0K    /srv/gerrit/git/design/style-guide.git/HEAD
4.0K    /srv/gerrit/git/design/style-guide.git/hooks
52K     /srv/gerrit/git/design/style-guide.git/logs
444M    /srv/gerrit/git/design/style-guide.git/objects
4.0K    /srv/gerrit/git/design/style-guide.git/packed-refs
20K     /srv/gerrit/git/design/style-guide.git/refs
444M    total

Considering after switching fully over to web-based Figma for large binary resources to archive https://github.com/wikimedia/WikimediaUI-Style-Guide/ and start a new repo https://github.com/wikimedia/Wikimedia-Design-Style-Guide/ with no close to non large binary files in it anymore…

That'd be great. For clarity we don't delete repos, we archive them.

@thcipriani @Dzahn @mmodell ?

(Adding @hashar as acting Gerrit czar)

@thcipriani I mentioned “archive”. ;)

There is a similar terrible process for some ORES repositories which has/had two major limitations:

• GitHub default LFS storage space is very limited, that forced someone to add some credits using their debit card/paypal to get some free extra spaces provided by GitHub. Maybe cause we are a non profit. That is fragile and can be dropped at anytime.
• The replication between Github and Gerrit does fails somehow from time to time and is a messy recovery as far as I can understand

If I had the power to make the call against the will of developers and energy to allocate, I would just slice out those GitHub repository and migrate them entirely to Gerrit where we can properly support a git LFS service. That is for the ranting part.

For the design style guide, my understanding is the project ultimately serves https://design.wikimedia.org/style-guide/ which is deployed to scap. But since the canonical repository is on GitHub https://github.com/wikimedia/WikimediaUI-Style-Guide/ it get pushed to our Gerrit to workaround the limitation that we don't deploy from GitHub. I don't even want to know how the replication is done, but one sure thing there is zero audit trail about it since the project has only two unrelated changes in Gerrit https://gerrit.wikimedia.org/r/q/project:design/style-guide but I am ranting. If we get LFS activated on Github and port it to Gerrit, we would have the same issue we have with the ORES repositories and I would rather NOT reproduce that setup.

Since the usage of LFS is to offload git from large files I went to investigate. I have git clone --mirror the Github repository which gives me all branches / pull requests etc. It is 441 MBytes right now which is not that much.

I went to copy paste some long command to list files larges than 1 MB:

(git rev-list --objects --all \
| git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \
| sed -n 's/^blob //p' \
| sort --numeric-sort --key=2 \
| cut -c 1-12,41- \
| $(command -v gnumfmt || echo numfmt) --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest

Result:

85498584436f   40MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
253b1f4577d8   38MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
8684d64874e0   38MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
58c51c8e61de   38MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
c52280b4c765   38MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
db07d013e73f   38MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
198f171c78e5   38MiB img/visual-style/illustrations-using.svg
0cd0c36bd3ad   37MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
bd7c85fd6956   37MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
4461a416b703   37MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
330748ebb634   32MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
002cd151f406   32MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
02e36a31349f   13MiB img/visual-style/illustrations-using--mobile.svg
45588d333227  7,1MiB resources/Wikimedia_Design_Style_Guide-imagery.sketch
cce6609c76ab  7,1MiB resources/WikimediaUI-Style-Guide-Imagery.sketch
9453a6af4ba6  7,0MiB resources/WikimediaUI-Style-Guide-Imagery.sketch
8f41ce523b5a  7,0MiB resources/WikimediaUI-Style-Guide-Imagery.sketch
0b9830fc53b4  5,9MiB resources/WikimediaUI.sketch
5c9f3463642d  5,8MiB resources/WikimediaUI.sketch
69056d09772d  4,9MiB img/patterns/bruegel.jpg
8671cf532ea4  4,6MiB img/screenshots/WikiCite.png
2a67538c9bbb  4,6MiB resources/WikimediaUI-Style-Guide-Imagery.sketch
9da2024e3370  4,2MiB img/Grantmaking_Lunch,_Wikimania_2013.jpg
e0844690904c  3,7MiB resources/WikimediaUI-components_applications_and_explanations.sketch
0379298cab7a  3,6MiB img/Wikimania_2016_-_group_photo_02.jpg
932ba5f471ad  3,3MiB resources/WikimediaUI-components_overview.png
f928ab8cbd47  3,3MiB resources/WikimediaUI-components_overview.png
bad9d604003e  3,2MiB img/visual-style/illustrations-header.svg
8cb9771a2786  3,2MiB img/visual-style/illustrations-header--start.svg
c67c71b67191  3,1MiB resources/WikimediaUI_illustrations-building-block.sketch
c8f5499d888f  3,1MiB resources/WikimediaUI-Style-Gudie-Illustrations.sketch
f22f5643f420  3,0MiB resources/WikimediaUI_building-block-illustrations.sketch
1a9301587eb7  3,0MiB resources/WikimediaUI-Style-Gudie-Illustrations.sketch
77ec7b127409  2,8MiB resources/WikimediaUI-Style-Guide-Imagery.sketch
995513ffa195  2,6MiB resources/WikimediaUI-components_overview.png
79975ce0d01e  2,6MiB resources/WikimediaUI-components_overview.png
488b23896e73  2,6MiB resources/WikimediaUI-components_overview.png
d78e13ba607b  2,6MiB resources/WikimediaUI-components_overview.png
0b0731828c46  2,5MiB resources/WikimediaUI-components_overview.png
2c748810fe60  2,5MiB resources/WikimediaUI-components_overview.png
3f712ae48d75  2,5MiB resources/WikimediaUI-components_overview.png
a9152316fc6b  2,5MiB resources/WikimediaUI-components_overview.png
c9808bf73ea7  2,5MiB resources/WikimediaUI-components_overview.png
6dea4e6e115e  2,5MiB img/design-principles/Wikimania_2016_-_group_photo_03.jpg
6b67bf2551f9  2,5MiB resources/WikimediaUI_illustrations-building-blocks.zip
f713be586b7a  2,4MiB resources/WikimediaUI-components_overview.png
509937446035  2,4MiB resources/WikimediaUI Overview.png
b6499c0cef4b  2,1MiB img/patterns/bruegel 2.jpg
05b3021688ba  2,0MiB resources/WikimediaUI.sketch
b02a7428e39e  2,0MiB resources/WikimediaUI.sketch
683e09ac65d0  2,0MiB resources/WikimediaUI.sketch
b2556d2992e4  2,0MiB img/patterns/sirma-krusteva-3172.jpg
7fd067f2c004  1,9MiB img/patterns/jan-mellstrom-242087.jpg
c1197c788b8f  1,7MiB resources/WikimediaUI-components_overview.sketch
a5a904029b89  1,7MiB resources/WikimediaUI-components_overview.sketch
e0cb05e4b9a3  1,7MiB resources/WikimediaUI-components_overview.sketch
654641939914  1,7MiB resources/WikimediaUI-components_overview.sketch
f13950169484  1,7MiB resources/WikimediaUI-components_overview.sketch
d8604d4c70fa  1,7MiB resources/WikimediaUI-components_overview.sketch
dbf694889457  1,7MiB resources/WikimediaUI-components_overview.sketch
c8831fcbd701  1,7MiB resources/WikimediaUI-components_overview.sketch
c3d27c42c408  1,7MiB resources/WikimediaUI-components_overview.sketch
3c34331d6049  1,7MiB resources/WikimediaUI-components_overview.sketch
a4e1e2ec4572  1,7MiB resources/WikimediaUI-components_overview.sketch
d7d7ebd2be32  1,6MiB resources/WikimediaUI-components_overview.sketch
29e0407233fc  1,6MiB resources/WikimediaUI-components_overview.sketch
dcd8995aced5  1,6MiB resources/WikimediaUI-components_overview.sketch
5d444a65e702  1,6MiB resources/WikimediaUI-components_overview.sketch
7586f29942cc  1,5MiB resources/WikimediaUI-icons.ai
5d130634da9c  1,5MiB img/patterns/peter-feghali-354275.jpg
297879a6ba5d  1,5MiB resources/WikimediaUI-icons.ai
20d0bf098e92  1,5MiB resources/WikimediaUI-icons.ai
2a3adead4b39  1,5MiB resources/WikimediaUI-icons.ai
9b858422d5dd  1,5MiB resources/WikimediaUI-icons.ai
b031445a1f84  1,5MiB resources/WikimediaUI-icons.ai
f4f37ea99fbd  1,5MiB img/patterns/scott-webb-256482.jpg
314fa484eb15  1,5MiB resources/WikimediaUI-icons.ai
620f2f176920  1,5MiB img/patterns/samuel-zeller-483391.jpg
1f2fedfca1da  1,5MiB resources/WikimediaUI-icons.ai
13816d3bb776  1,5MiB resources/WikimediaUI-icons.ai
54b2efa38732  1,5MiB resources/WikimediaUI-icons.ai
3dc910a77579  1,4MiB resources/WikimediaUI-components_overview.sketch
cf32f4ebbaf4  1,4MiB resources/WikimediaUI-components_overview.sketch
6d3e79b6f2cf  1,4MiB resources/WikimediaUI-components_overview.sketch
a10b85fa2266  1,4MiB resources/WikimediaUI-components_overview.sketch
af1865f2b447  1,4MiB resources/WikimediaUI-components_overview.sketch
4b8008ff7fb3  1,4MiB resources/WikimediaUI-components_overview.sketch
142cf6c5c97f  1,4MiB resources/WikimediaUI-components_overview.sketch
16a1a3d002f1  1,4MiB resources/WikimediaUI-components_overview.sketch
414429b5201d  1,4MiB resources/WikimediaUI-components_overview.sketch
d4ebeeff29fd  1,4MiB resources/WikimediaUI-components_overview.sketch
642b74cb1413  1,4MiB resources/WikimediaUI-components_overview.sketch
97fc93533b1d  1,4MiB resources/WikimediaUI-components_overview.sketch
3b9708d2d1d2  1,4MiB resources/WikimediaUI-components_overview.sketch
9a0c2840792c  1,4MiB resources/WikimediaUI-components_overview.sketch
810077f95661  1,4MiB resources/WikimediaUI-components_overview.sketch
b33852541790  1,4MiB resources/WikimediaUI-components_overview.sketch
e1c04b6a0988  1,4MiB resources/WikimediaUI-components_overview.sketch
8ebe031dff21  1,4MiB resources/WikimediaUI-components_overview.sketch
4986ada6225b  1,4MiB resources/WikimediaUI-components_overview.sketch
ecb35057b2c8  1,4MiB resources/WikimediaUI-components_overview.sketch
a5fffa5a5ae3  1,4MiB resources/WikimediaUI-components_overview.sketch
bcf1dc4c1b8b  1,4MiB resources/WikimediaUI-components_overview.sketch
aa3709824661  1,4MiB resources/WikimediaUI-components_overview.sketch
269be291ae77  1,4MiB resources/WikimediaUI-components_overview.sketch
7439173582f6  1,4MiB resources/WikimediaUI-components_overview.sketch
49b75abfcedb  1,4MiB resources/WikimediaUI-components_overview.sketch
279c051a195f  1,4MiB resources/WikimediaUI-components_overview.sketch
a62438a52aeb  1,4MiB resources/WikimediaUI-components_overview.sketch
b3bea2cb4288  1,4MiB resources/WikimediaUI-components_overview.sketch
a818c1d4f7ab  1,4MiB resources/WikimediaUI-components_overview.sketch
b7cbf5258bb7  1,4MiB resources/WikimediaUI-components_overview.sketch
3953a66cd7de  1,4MiB resources/WikimediaUI-components_overview.sketch
7160f8e19d7e  1,4MiB resources/WikimediaUI-components_overview.sketch
ed74c1103819  1,3MiB img/visual-style/typography-grid.png
1d3af08f09cf  1,3MiB resources/WikimediaUI-components_overview.sketch
c4444f79686c  1,3MiB resources/WikimediaUI-components_overview.sketch
88d3ae73d39d  1,3MiB resources/WikimediaUI-components_overview.sketch
110dd31f57b1  1,3MiB resources/WikimediaUI-components_overview.sketch
638c70ade853  1,3MiB resources/WikimediaUI-components_overview.sketch
aa7f208c9c65  1,3MiB img/patterns/ryan-searle-397209.jpg
9e25da1276c4  1,3MiB package-lock.json
a00279336c27  1,3MiB img/visual-style/typography-readability.png
556a7773bd3b  1,2MiB img/patterns/ricardo-gomez-angel-180819.jpg
e49421b0be52  1,2MiB img/patterns/peter-feghali-354275.jpg
b9bb27e995c4  1,2MiB resources/WikimediaUI-components_overview.png
761a20285d9a  1,1MiB img/patterns/ludde-lorentz-47866.jpg
0f1d6468c731  1,1MiB img/patterns/samuel-zeller-483391.jpg
6c70a298463e  1,1MiB resources/WikimediaUI-components_overview.png
928aa6ffc3a6  1,1MiB img/patterns/liana-mikah-343240.jpg
6e0ac6c2c210  1,1MiB resources/WikimediaUI-components_overview.sketch
c00b86a49aeb  1,1MiB resources/WikimediaUI-components_overview.sketch
2051818bbede  1,1MiB resources/WikimediaUI-components_overview.sketch
85ef03cc5934  1,1MiB resources/WikimediaUI-components_overview.sketch
4ddcf023c93a  1,1MiB resources/WikimediaUI-components_overview.sketch

So surely those big .sketch files should not be in stored in git which is not really suitable for storing large files since they pill up quickly. They are zipfiles containing some binary pictures that are probably reused as is from an iteration to another and could potentially be stored only once, but since the pictures are wrapped in a zip, git store the whole data again. It is not ideal :]

Conclusion

Given:

• the aim is to migrate the design guide to Figma which would not rely on binary files
• it is already envisioned to start a new repository to host those materials
• LFS from Github to Gerrit is a pain
• the large .sketch files are barely edited and thus the repository does not grow that much,
• the repository is 440MB right now which is manageable
• End users can fetch just the latest commit and skip downloading all the history and its binary blobs by using: git clone --depth=1

I recommend to:

• NOT use git lfs and stick with the status quo until we archive it
• When moving to Figma create a new git repository and avoid storing binary files
• migrate from GitHub to Gerrit/Phabricator

And if we end up willing to support offloading large file, it will have to be done on Gerrit, not on Github.

So I think we can decline this task?

@hashar perfectly said whatever I wanted to say. +1 to every word. Thanks! I had so much trouble getting ores work in production.

+1 to @hashar These large binary files should not be in git. They need some other storage solution (or as was said, reuse a single copy of them where possible by avoiding the zips). Plus the way it's setup now is kind of circumventing that we don't deploy from github and it has caused considerable effort in the past already.

Great summary @hashar, thanks!
With this I'd like to propose to archive the current https://github.com/wikimedia/WikimediaUI-Style-Guide/ repo including the large binary files.
And to open https://github.com/wikimedia/Wikimedia-Design-Style-Guide/ without those.

With the change over to Figma for basically all large binary files apart from web optimized images in T286676 there's an opening to finally resolve this in an excellent way IMO.