Currently, we point to git-ssh, which is overkill since we only read from these repos, and furthermore our team doesn't work on submodules in place, but in isolation.
We should change to use submodule URLs like https://phabricator.wikimedia.org/source/draftquality.git
| Subject | Author | Repo | Branch | Lines +/- | |
|---|---|---|---|---|---|
| Switch to using HTTPS for submodules | Awight | mediawiki/services/ores/deploy | master | +5 -5 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Halfak | T171619 [Epic] ORES should use a git large file plugin for storing serialized binaries | |||
| Resolved | • demon | T171758 Support git-lfs files in gerrit | |||
| Declined | None | T179009 ORES deployment submodules should point to phabricator HTTPS repos. |
Change 386407 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/services/ores/deploy@master] Switch to using HTTPS for submodules
Change 386407 merged by Ladsgroup:
[mediawiki/services/ores/deploy@master] Switch to using HTTPS for submodules
We used to point to the https in phabricator explodes when you try to pull from our giant repos.
@demon: fixing phabricator is going to be difficult. Git was never meant to be so huge, and phabricator's git-http functionality is badly broken for huge repos in ways which are complex (at best) to address.
Now I'm wondering if Phabricator has the same bug with it's own git-lfs support (file size limit due to large HTTP transactions buffering in the web server and/or PHP)
This may have been a bad decision. I get a lot of timeouts trying to clone our submodules onto deployment-tin and locally.
Yep, this became an issue again. I can't make fresh checkouts of the ores-deploy repo because of timeouts:
Cloning into '/home/awight/ores-prod-deploy/submodules/articlequality'... error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504 Gateway Time-out fatal: The remote end hung up unexpectedly fatal: clone of 'https://phabricator.wikimedia.org/source/wikiclass.git' into submodule path '/home/awight/ores-prod-deploy/submodules/articlequality' failed Failed to clone 'submodules/articlequality'. Retry scheduled Cloning into '/home/awight/ores-prod-deploy/submodules/draftquality'... Cloning into '/home/awight/ores-prod-deploy/submodules/editquality'... error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504 Gateway Time-out fatal: The remote end hung up unexpectedly fatal: clone of 'https://phabricator.wikimedia.org/source/editquality.git' into submodule path '/home/awight/ores-prod-deploy/submodules/editquality' failed
Just noticed that we've already rolled this back for git-lfs work, we're now using gerrit again. Since I created the task originally, and my motivation was murky, this might be fine.
@demon: At least for now, since the swift backend I've been working on for phabricator's git-lfs storage still isn't finished.