Slow connection to gerrit via ssh
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Osnard
	Jun 27 2025, 11:52 AM

Description

Since a couple of weeks, I experience very slow downloads when cloning a repo from gerrit.wikimedia.org via SSH. I was wondering if my user account or IP have been throttled on purpose. In the past, I had some automated scripts running on a number of those repos, which might have caused too much traffic. If that is the case, I'd like to apologize. I have switched to the github mirror repos for most of those scripts in the meantime. Unfortunately, there are some things I need to work with the gerrit repos (e.g. tagging).

Can you please help me to understand if I did something wrong?

Event Timeline

Osnard created this task.Jun 27 2025, 11:52 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 27 2025, 11:52 AM

Peachey88 added a project: Gerrit.Jun 27 2025, 12:39 PM

Aklapper added a project: Infrastructure-Foundations.Jun 27 2025, 6:29 PM

I don't think we throttle bandwidth, but we do have systems limiting concurrent connections which would end up blocking for some minutes. I don't think it would affect you though and if that is the case you would end up not being able to reach Gerrit at all.

I have looked at the Gerrit ssh log, andI found a few of your fetches that took more than 10 seconds:

zgrep -i rvogel.*git-upload-pack /var/log/gerrit/sshd_log.2025-06-2?.gz|cut -d]  -f3-|cut -d. -f2-|column -t|grep -P '\d{5}ms'
/mediawiki/extensions/BlueSpiceBookshelf.git                 3ms  98084ms   '3ms  0ms  0ms  3ms   3ms   97662ms   97668ms   5    5   15718  9000587'   0  git/2.34.1  256ms  260ms  21873536
/mediawiki/skins/BlueSpiceDiscovery.git                      2ms  104809ms  '2ms  1ms  0ms  2ms   1ms   104399ms  104403ms  4    5   14387  8489849'   0  git/2.34.1  250ms  240ms  20960728
/mediawiki/extensions/BlueSpiceExtendedSearch.git            3ms  34377ms   '4ms  1ms  1ms  4ms   55ms  33907ms   33968ms   7    35  16809  4230506'   0  git/2.34.1  160ms  150ms  17522960
/mediawiki/extensions/BlueSpiceFoundation.git                2ms  327094ms  '4ms  0ms  0ms  3ms   1ms   326544ms  326548ms  1    0   54212  18776175'  0  git/2.34.1  534ms  530ms  34839096
/mediawiki/extensions/CognitiveProcessDesigner.git           3ms  36965ms   '1ms  1ms  1ms  2ms   30ms  36528ms   36562ms   9    12  6297   4905229'   0  git/2.34.1  172ms  160ms  14687088
/mediawiki/extensions/Workflows.git                          3ms  38676ms   '2ms  1ms  0ms  1ms   0ms   38255ms   38257ms   3    4   9860   3928785'   0  git/2.34.1  121ms  120ms  10857336

The fields are documented at https://gerrit.wikimedia.org/r/Documentation/logs.html#_sshd_log . The ones between single quotes are details specific to git-upload-pack. The slowest one took 326548ms (326 seconds, 5 minutes) to transfer 18776175 bytes (17.90MBytes). That is 56.2 KBytes/seconds.

You can try doing fetches over https to see whether it improves thing. Meanwhile I am marking this resolved, I don't think the slow transfer is due to Gerrit.

Thank you very much! Yes, "56.2 KBytes/seconds" looks like the speed I faced recently. Today I didn't experience this anymore. At least I now know I didn't cause this by my automation. Again, thank you.

Slow connection to gerrit via sshClosed, ResolvedPublicActions

Description

Event Timeline

Slow connection to gerrit via ssh
Closed, ResolvedPublic
Actions