[Epic] Use LFS for large ORES files
Closed, ResolvedPublic

Description

  • Models
  • Datasets
  • Reference data such as embeddings

We need to plan these migrations, they probably involve rewriting repository history among other nastiness.

awight created this task.Jun 13 2018, 11:01 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 13 2018, 11:01 AM
Vvjjkkii renamed this task from [Epic] Use LFS for large ORES files to f4aaaaaaaa.Jul 1 2018, 1:04 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot renamed this task from f4aaaaaaaa to [Epic] Use LFS for large ORES files.
CommunityTechBot added a subscriber: Aklapper.
Ladsgroup added a subscriber: Ladsgroup.

Already on it.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptAug 20 2018, 1:19 PM

I put this back to active because I want to turn wheels to git lfs as well, since it's growing really quickly and it's already 220MB since it's binary files and git grows horribly with files it can't delta.

Change 455629 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[All-Projects@refs/meta/config] Enable LFS for research/ores/wheels

https://gerrit.wikimedia.org/r/455629

Change 455629 merged by 20after4:
[All-Projects@refs/meta/config] Enable LFS for research/ores/wheels

https://gerrit.wikimedia.org/r/455629

Now I rewrote whole history of wheels, that was the last bit. Now I'm going to deploy it.

Change 472230 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/services/ores/deploy@master] Bump wheels to HEAD

https://gerrit.wikimedia.org/r/472230

Change 472230 merged by Ladsgroup:
[mediawiki/services/ores/deploy@master] Bump wheels to HEAD

https://gerrit.wikimedia.org/r/472230

Stashbot added a subscriber: Stashbot.

Mentioned in SAL (#wikimedia-operations) [2018-11-07T21:03:28Z] <ladsgroup@deploy1001> Started deploy [ores/deploy@25dfa4f]: T191842 T197096

Mentioned in SAL (#wikimedia-operations) [2018-11-07T21:20:52Z] <ladsgroup@deploy1001> Finished deploy [ores/deploy@25dfa4f]: T191842 T197096 (duration: 17m 24s)

Ladsgroup closed this task as Resolved.Mon, Dec 17, 12:39 PM