Some MediaWiki translation exports are failing in commit-msg hook
Closed, ResolvedPublic1 Estimated Story Points
Actions

Assigned To

Authored By

	Nikerabbit
	Tue, Sep 15, 8:32 AM

Project Tags

Referenced Files

None

Subscribers

Aklapper

Nikerabbit

Description

Some MediaWiki translation exports fail while creating commits with:

.git/hooks/commit-msg: 2: Syntax error: newline unexpected
error: cannot rebase: Your index contains uncommitted changes.
error: Please commit or stash them.

Investigation found that some .git/hooks/commit-msg files contain a Wikimedia 429 "Too many requests" HTML response instead of the Gerrit commit message hook.

clupdate-git-repo currently downloads the Gerrit hook when it is missing. With many MediaWiki repositories this can result in rate limiting (because we are not setting an User-Agent), and a failed HTTP response can be left behind as an executable hook. Since the file then exists, subsequent runs do not repair it.

Expected behavior

MediaWiki Gerrit export repositories have a valid Gerrit commit-msg hook without requiring repeated downloads from Gerrit.
Existing broken hooks can be repaired/replaced automatically.
Repository update/export/commit operations remain safe to rerun.
Translation exports can create commits with the expected Gerrit Change-Id and do not fail because of an invalid commit-msg hook.

Implementation notes

The proposed change in change 1341700 provides the Gerrit commit message hook locally and adjusts the symlink handling in clupdate-git-repo.

Dependencies / prerequisites

No known dependencies.

No configuration, schema, or documentation changes are required.

QA plan

The main regression risk is breaking repository setup or replacing an intentional hook incorrectly.

Before deployment:

Run the relevant repository update/setup logic against a test/temporary checkout and verify that the resulting commit-msg hook is valid and executable. (Done while developing)
Verify that running the setup again is idempotent.
Verify that an existing invalid/broken hook is repaired as intended.

After deploying to translatewiki.net:

Update one repository known to have the broken HTML hook, such as AdvancedSearch.
Verify that the hook has been repaired.
Run the export/commit flow for the repository and verify that a commit can be created with a Gerrit Change-Id.
Re-run the previously failing MediaWiki export flow and verify that there are no remaining failures caused by invalid commit-msg hooks.

There is no separate staging environment for the translatewiki.net export checkouts, so the final integration verification is performed on translatewiki.net after deployment.

Details

Related Changes in Gerrit:

	Subject	Author	Repo	Branch	Lines +/-
	clupdate-git-repo: Add Gerrit commit message hook and update symlink handling	Nikerabbit	translatewiki	master	+121 -6

Customize query in gerrit

Event Timeline

Nikerabbit created this task.Tue, Sep 15, 8:32 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptTue, Sep 15, 8:32 AM

Initial exploration:

clupdate-git-repo does:

if [[ ! -e .git/hooks/commit-msg ]];
then
    mkdir -p .git/hooks
    curl -Lo .git/hooks/commit-msg https://gerrit.wikimedia.org/r/tools/hooks/commit-msg
    chmod +x .git/hooks/commit-msg
fi

Inspecting a failing repo:

twn:/scratch/exports/mediawiki-extensions/AdvancedSearch/.git/hooks$ cat commit-msg
<!DOCTYPE html>
<html lang="en">
<meta charset="utf-8">
<title>Wikimedia Error</title>
<style>
* { margin: 0; padding: 0; }
body { background: #fff; font: 15px/1.6 sans-serif; color: #333; }
.content { margin: 7% auto 0; padding: 2em 1em 1em; max-width: 640px; display: flex; flex-direction: row; flex-wrap: wrap; }
.footer { clear: both; margin-top: 14%; border-top: 1px solid #e5e5e5; background: #f9f9f9; padding: 2em 0; font-size: 0.8em; text-align: center; }
img { margin: 0 2em 2em 0; }
a img { border: 0; }
h1 { margin-top: 1em; font-size: 1.2em; }
.content-text { flex: 1; }
p { margin: 0.7em 0 1em 0; }
a { color: #0645ad; text-decoration: none; }
a:hover { text-decoration: underline; }
code { font-family: sans-serif; }
summary { font-weight: bold; cursor: pointer; }
details[open] { background: #970302; color: #dfdedd; }
.text-muted { color: #777; }
@media (prefers-color-scheme: dark) {
  a { color: #9e9eff; }
  body { background: transparent; color: #ddd; }
  .footer { border-top: 1px solid #444; background: #060606; }
  #logo { filter: invert(1) hue-rotate(180deg); }
  .text-muted { color: #888; }
}
</style>
<meta name="color-scheme" content="light dark">
<div class="content" role="main">
<a href="https://www.wikimedia.org"><img id="logo" src="https://www.wikimedia.org/static/images/wmf-logo.png" srcset="https://www.wikimedia.org/static/images/wmf-logo-2x.png 2x" alt="Wikimedia" width="135" height="101">
</a>
<div class="content-text">
<h1>Error</h1>

<p>Too many requests</p>
</div>
</div>
<div class="footer"><p>If you report this error to the Wikimedia System Administrators, please include the details below.</p><p class="text-muted"><code>Request served via cp3069 cp3069, Varnish XID 493982701<br>Upstream caches: cp3069 int<br>Error: 429, Too many requests at Wed, 09 Sep 2026 12:01:08 GMT<br><details><summary>Sensitive client information</summary>IP address: XXX</details></code></p>
</div>
</html>

Vs. expected:

twn:/scratch/exports/mediawiki-extensions$ cat AbuseFilter/.git/hooks/commit-msg
#!/bin/sh
# From Gerrit Code Review 3.10.6
#
# Part of Gerrit Code Review (https://www.gerritcodereview.com/)
#
# Copyright (C) 2009 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -u

# avoid [[ which is not POSIX sh.
if test "$#" != 1 ; then
  echo "$0 requires an argument."
  exit 1
fi

if test ! -f "$1" ; then
  echo "file does not exist: $1"
  exit 1
fi

# Do not create a change id if requested
case "$(git config --get gerrit.createChangeId)" in
  false)
    exit 0
    ;;
  always)
    ;;
  *)
    # Do not create a change id for squash/fixup commits.
    if head -n1 "$1" | LC_ALL=C grep -q '^[a-z][a-z]*! '; then
      exit 0
    fi
    ;;
esac


if git rev-parse --verify HEAD >/dev/null 2>&1; then
  refhash="$(git rev-parse HEAD)"
else
  refhash="$(git hash-object -t tree /dev/null)"
fi

random=$({ git var GIT_COMMITTER_IDENT ; echo "$refhash" ; cat "$1"; } | git hash-object --stdin)
dest="$1.tmp.${random}"

trap 'rm -f "$dest" "$dest-2"' EXIT

if ! cat "$1" | sed -e '/>8/q' | git stripspace --strip-comments > "${dest}" ; then
   echo "cannot strip comments from $1"
   exit 1
fi

if test ! -s "${dest}" ; then
  echo "file is empty: $1"
  exit 1
fi

reviewurl="$(git config --get gerrit.reviewUrl)"
if test -n "${reviewurl}" ; then
  token="Link"
  value="${reviewurl%/}/id/I$random"
  pattern=".*/id/I[0-9a-f]\{40\}"
else
  token="Change-Id"
  value="I$random"
  pattern=".*"
fi

if git interpret-trailers --parse < "$1" | grep -q "^$token: $pattern$" ; then
  exit 0
fi

# There must be a Signed-off-by trailer for the code below to work. Insert a
# sentinel at the end to make sure there is one.
# Avoid the --in-place option which only appeared in Git 2.8
if ! git interpret-trailers \
         --trailer "Signed-off-by: SENTINEL" < "$1" > "$dest-2" ; then
  echo "cannot insert Signed-off-by sentinel line in $1"
  exit 1
fi

# Make sure the trailer appears before any Signed-off-by trailers by inserting
# it as if it was a Signed-off-by trailer and then use sed to remove the
# Signed-off-by prefix and the Signed-off-by sentinel line.
# Avoid the --in-place option which only appeared in Git 2.8
# Avoid the --where option which only appeared in Git 2.15
if ! git -c trailer.where=before interpret-trailers \
         --trailer "Signed-off-by: $token: $value" < "$dest-2" |
     sed -e "s/^Signed-off-by: \($token: \)/\1/" \
         -e "/^Signed-off-by: SENTINEL/d" > "$dest" ; then
  echo "cannot insert $token line in $1"
  exit 1
fi

if ! mv "${dest}" "$1" ; then
  echo "cannot mv ${dest} to $1"
  exit 1
fi

Change #1341700 had a related patch set uploaded (by Nikerabbit; author: Nikerabbit):

[translatewiki@master] clupdate-git-repo: Add Gerrit commit message hook and update symlink handling

https://gerrit.wikimedia.org/r/1341700

gerritbot added a project: Patch-For-Review.Tue, Sep 15, 8:54 AM

Nikerabbit updated the task description. (Show Details)Tue, Sep 15, 9:02 AM

Nikerabbit set the point value for this task to 1.

Change #1341700 merged by jenkins-bot:

[translatewiki@master] clupdate-git-repo: Add Gerrit commit message hook and update symlink handling

https://gerrit.wikimedia.org/r/1341700

Maintenance_bot removed a project: Patch-For-Review.Tue, Sep 15, 9:31 AM

Nikerabbit triaged this task as High priority.Tue, Sep 15, 9:59 AM

Nikerabbit moved this task from Ready for dev to Needs QA on the LPL sprints (S3 2026 September) board.

twn:/scratch/exports$ ls -lah mediawiki-extensions/AdvancedSearch/.git/hooks/commit-msg
-rwxr-xr-x 1 l10n-bot users 2.0K Sep  9 14:01 mediawiki-extensions/AdvancedSearch/.git/hooks/commit-msg*
twn:/scratch/exports$ sudo -u l10n-bot repomulti update mediawiki-extensions
[output]
twn:/scratch/exports$ ls -lah mediawiki-extensions/AdvancedSearch/.git/hooks/commit-msg
lrwxrwxrwx 1 l10n-bot users 56 Sep 15 12:02 mediawiki-extensions/AdvancedSearch/.git/hooks/commit-msg -> /home/betawiki/config/bin/data/gerrit-commit-msg-hook.sh*

I ran autoexport-mediawiki manually to check that everything works (and it does).

Some MediaWiki translation exports are failing in commit-msg hookClosed, ResolvedPublic1 Estimated Story PointsActions