Page MenuHomePhabricator

Recursive clone over ssh uses https for submodules
Closed, DeclinedPublic

Description

<~/pywikibot>git clone --recursive ssh://multichill@gerrit.wikimedia.org:29418/pywikibot/core.git
Cloning into 'core'...
Warning: Permanently added the RSA host key for IP address '[208.80.154.81]:29418' to the list of known hosts.
Enter passphrase for key 'key':
remote: Counting objects: 10842, done
remote: Finding sources: 100% (10842/10842)
remote: Getting sizes: 100% (1495/1495)
remote: Compressing objects: 100% (2292733/2292733)
remote: Total 10842 (delta 7190), reused 10548 (delta 7165)
Receiving objects: 100% (10842/10842), 23.42 MiB | 326 KiB/s, done.
Resolving deltas: 100% (7337/7337), done.
Submodule 'externals/httplib2' (https://gerrit.wikimedia.org/r/p/pywikibot/externals/httplib2.git) registered for path 'externals/httplib2'
Submodule 'i18n' (https://gerrit.wikimedia.org/r/p/pywikibot/i18n.git) registered for path 'scripts/i18n'
Cloning into 'externals/httplib2'...
error: SSL certificate problem: self signed certificate in certificate chain while accessing https://gerrit.wikimedia.org/r/p/pywikibot/externals/httplib2.git/info/refs?service=git-upload-pack
fatal: HTTP request failed
Clone of 'https://gerrit.wikimedia.org/r/p/pywikibot/externals/httplib2.git' into submodule path 'externals/httplib2' failed

I'm cloning over ssh. Why are the submodules being loaded over https?


Version: core-(2.0)
Severity: normal

Details

Reference
bz55654

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:30 AM
bzimport set Reference to bz55654.
bzimport added a subscriber: Unknown Object (????).

Basically, there are two options:

  1. an absolute externals path, which always overrules the initial clone url -- this can be a problem for people who cannot clone over https.
  2. a relative externals path, which breaks for people cloning from e.g. github, where the urls are not /pywikibot/externals/httplib2 but pywikibot-externals-httplib2

Would it help if we switch to git:// instead of https://?

While it cloned the submodules over https, the clone recursive worked for me, so I have updated the bug summary to reflect this.

The "error: SSL certificate problem: self signed certificate in certificate chain while accessing ..." doesnt appear in my output, so something has radically changed since October 2013, which is probably masking this bug. Either the certificate has been changed, or this problem only existed in an earlier git version.

$ git clone --recursive ssh://jayvdb@gerrit.wikimedia.org:29418/pywikibot/core.git
Cloning into 'core'...
remote: Counting objects: 2792, done
remote: Finding sources: 100% (2373/2373)
remote: Getting sizes: 100% (710/710)
remote: Compressing objects: 100% (5777484/5777484)
remote: Total 16923 (delta 1184), reused 16381 (delta 1136)
Receiving objects: 100% (16923/16923), 27.47 MiB | 1.12 MiB/s, done.
Resolving deltas: 100% (11100/11100), done.
Checking connectivity... done.
Submodule 'externals/httplib2' (https://gerrit.wikimedia.org/r/p/pywikibot/externals/httplib2.git) registered for path 'externals/httplib2'
Submodule 'i18n' (https://gerrit.wikimedia.org/r/p/pywikibot/i18n.git) registered for path 'scripts/i18n'
Cloning into 'externals/httplib2'...
remote: Counting objects: 1035, done
remote: Finding sources: 100% (202/202)
remote: Getting sizes: 100% (29/29)
remote: Compressing objects: 100% (306895/306895)
remote: Total 1902 (delta 34), reused 1887 (delta 32)
Receiving objects: 100% (1902/1902), 3.88 MiB | 233.00 KiB/s, done.
Resolving deltas: 100% (946/946), done.
Checking connectivity... done.
Submodule path 'externals/httplib2': checked out '5dc145682eb726bd54eaf7e9f940844ca234db71'
Cloning into 'scripts/i18n'...
remote: Counting objects: 192, done
remote: Finding sources: 100% (171/171)
remote: Getting sizes: 100% (52/52)
remote: Compressing objects: 100% (1291342/1291342)
remote: Total 2802 (delta 113), reused 2759 (delta 110)
Receiving objects: 100% (2802/2802), 12.82 MiB | 664.00 KiB/s, done.
Resolving deltas: 100% (2216/2216), done.
Checking connectivity... done.
Submodule path 'scripts/i18n': checked out 'ab431d98501bc229c933ee931ce0d7f2c1e2d46c'

Im not sure, but this might be solvable with git subtree instead of git submodule.

After doing a bit more reading, I am pretty confident that git subtree will solve this. Now need to check what happens when switching from submodules to subtrees does to svn checkouts.

After doing a bit more reading, I am pretty confident that git subtree will solve this. Now need to check what happens when switching from submodules to subtrees does to svn checkouts.

I tried this with scripts/i18n, and fnfortunately TortoiseSVN doesnt appear to handle the external switching to be a normal directory in the repository. it complains there is a conflict, and then doesnt update scripts/i18n to use the files now located in the repository.

Xqt changed the task status from Open to Stalled.May 11 2018, 5:56 AM
Xqt subscribed.

Is this still valid with pwb3? I haven't this problem:

git.exe clone --progress --recursive -v "ssh://xqt@gerrit.wikimedia.org:29418/pywikibot/core" "C:\pwb\git\core-test"

Cloning into 'C:\pwb\git\core-test'...
remote: Counting objects: 870, done
remote: Finding sources: 100% (96/96)
Receiving objects: 100% (46206/46206), 27.12 MiB | 237.00 KiB/s, done.
Resolving deltas: 100% (35848/35848), done.
remote: Getting sizes: 100% (51/51)
remote: Compressing objects: 100% (463858/463858)
remote: Total 46206 (delta 46), reused 46153 (delta 19)
Checking connectivity... done.
Submodule 'i18n' (https://gerrit.wikimedia.org/r/pywikibot/i18n) registered for path 'scripts/i18n'
Cloning into 'scripts/i18n'...
Submodule path 'scripts/i18n': checked out '5a4de72f67c4f13f290be2484b16c5e3f9a31c49'

Success (216061 ms @ 11.05.2018 07:48:45)

I cloned Pywikibot using $ git clone --recursive ssh://multichill@gerrit.wikimedia.org:29418/pywikibot/core.git pywikibot

:~/pywikibot$ cat .gitmodules
[submodule "i18n"]
path = scripts/i18n
url = https://gerrit.wikimedia.org/r/pywikibot/i18n
branch = master
ignore = all

So yes, still an issue

That is expected as you carnt use ssh in there since only you would be able to clone it.

The solution is doing relative submodule paths (eg: ../foo). Otherwise, of course, it will load the submodule from the protocol specific (git doesn't know better...this isn't a Gerrit issue really).

(nb: git:// is a non-option. It won't do what you're hoping here, and we don't have it enabled anyway)

Tbh though, HTTPS > SSH anyway :)

Aklapper changed the task status from Stalled to Open.May 19 2020, 10:43 AM

Nothing stalled in this task as questoin has been answered, hence setting status to open