Page MenuHomePhabricator

The quibble-with-gated-extensions-vendor-mysql-php83 CI job for WikibaseMediaInfo failed due to a transient network error during the Zuul clone phase. No tests were executed.
Open, Needs TriagePublicBUG REPORT

Description

The failure occurred when cloning mediawiki/extensions/Babel from Gerrit:

ERROR:zuul.Repo:Unable to initialize repo for https://gerrit.wikimedia.org/r/mediawiki/extensions/Babel
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)

cmdline: git clone -v -- https://gerrit.wikimedia.org/r/mediawiki/extensions/Babel /workspace/src/extensions/Babel
stderr: 'Cloning into '/workspace/src/extensions/Babel'...

POST git-upload-pack (142 bytes)
POST git-upload-pack (gzip 5645 to 2779 bytes)
error: RPC failed; curl 16 Error in the HTTP2 framing layer
fatal: error reading section header 'shallow-info'
'

This appears to be an infrastructure issue between the CI agents and Gerrit, possibly related to HTTP/2 negotiation or a connectivity problem specific to repos cloned without a local cache (Babel is cloned directly from upstream rather than from the git cache).

Event Timeline

hashar subscribed.

Thank you for your report, I did see it last week but did not jump immediately on it.

It might be an aftermath of having moved Gerrit behind our CDN (T411895), connections about to time out can be reused and terminated abruptly which I guess would show that kind of issue. That got solved for the CDNApache Gerrit path (final tweak to be done with T417998), there is still the rare issue between Apache GerritGerrit Jetty web server which is T246763).

I have looked at the Apache2 error logs on Gerrit and I don't see an error at that time. So maybe that was on the CDN side.

I went to grep for Error in the HTTP2 framing layer in the Jenkins build logs:

$ find /srv/jenkins/builds/  -type f -name log -exec grep -l 'Error in the HTTP2 framing layer' {} \+
/srv/jenkins/builds/mwext-phan-php83/26145/log
/srv/jenkins/builds/quibble-with-gated-extensions-vendor-mysql-php83/16071/log
/srv/jenkins/builds/quibble-vendor-mysql-php83-selenium/26331/log

Two other builds had the same issue:

-rw-rw-r-- 1 jenkins jenkins 15349 Mar  1 00:00 /srv/jenkins/builds/mwext-phan-php83/26145/log
-rw-rw-r-- 1 jenkins jenkins 48052 Mar  5 00:01 /srv/jenkins/builds/quibble-vendor-mysql-php83-selenium/26331/log
00:00:36   stderr: 'Cloning into '/workspace/src/extensions/GoogleNewsSitemap'...
00:00:36 POST git-upload-pack (142 bytes)
00:00:36 POST git-upload-pack (gzip 1795 to 836 bytes)
00:00:36 error: RPC failed; curl 16 Error in the HTTP2 framing layer
00:00:36 fatal: error reading section header 'shallow-info'
# I believe that is for mediawiki/core
00:00:42 git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
00:00:42   cmdline: git fetch --force --tags -v -- origin
00:00:42   stderr: 'error: RPC failed; curl 16 Error in the HTTP2 framing layer
00:00:42 fatal: error reading section header 'acknowledgments''

I'll try to find whether something can be found in the CDN logs.

Looking at the Apache2 access log for the GoogleNewsSitemap issue on March 1sth. I have searched for requests having:

  • The user-agent is git
  • time around 2026-03-01T00:00:35
  • requests for GoogleNewsSitemap

I can't see any errors and all transactions looks okish. So something is cutting out somewhere along the path and that is frankly a mystery.

I am keeping that in mind though.