Page MenuHomePhabricator

Missing wmf/1.34.0-wmf.16 branch on rEOAT
Closed, ResolvedPublic

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 4 2019, 11:37 AM
MarcoAurelio renamed this task from Missing 1.34.0/wmf.16 branch on rEOAT to Missing wmf/1.34.0-wmf.16 branch on rEOAT.Aug 4 2019, 11:39 AM
MarcoAurelio updated the task description. (Show Details)
MarcoAurelio added a comment.EditedAug 4 2019, 11:44 AM

Okay I've found that the repository is configured to observe from https://gerrit-slave.wikimedia.org/r/mediawiki/extensions/OATHAuth. I'll change the URI to clone from the actual gerrit repo.

Hmm. This was changed today by @thcipriani (https://phabricator.wikimedia.org/diffusion/EOAT/uri/view/3583/#5790) via Conduit. Any reason?

Mentioned in SAL (#wikimedia-releng) [2019-08-04T12:11:51Z] <hauskatze> gerrit: ssh -p 29418 gerrit.wikimedia.org replication start mediawiki/extensions/OATHAuth --wait # T229756

Mentioned in SAL (#wikimedia-releng) [2019-08-04T12:11:51Z] <hauskatze> gerrit: ssh -p 29418 gerrit.wikimedia.org replication start mediawiki/extensions/OATHAuth --wait # T229756

It looks this fixed it. The wmf.16 branch is now on rEOAT.

Leaving the task open so we can detect if more extensions are affected. A random check on other Diffusion repos did returned that they had the wmf.16 branch there but obviously I can't check all WMF-deployed extensions by hand :)

hrm. I wrote a little shell script that checks all wmf-deployed extensons:

check-gerrit-branch.sh
#!/usr/bin/env bash

set -euo pipefail

# Path to local copy of mediawiki/tools/release
TOOLS_REPO_PATH="$1"
BASE_PATH="https://gerrit-slave.wikimedia.org/r/mediawiki"
BRANCH='wmf/1.34.0-wmf.16'

for ext in $(jq -r '.extensions | .[]' "$TOOLS_REPO_PATH"/make-wmf-branch/config.json); do
    printf 'CHECKING %s....' "$ext"
    if ! git ls-remote --heads "${BASE_PATH}/${ext}" | grep -q "$BRANCH"; then
        printf '\n\t[ERRROR] no branch "%s" found for "%s"!\n' "$BRANCH" "$ext"
        exit 1
    fi
    printf 'Found!\n'
done

And there are a number or repos that don't have that branch. All commits are there, but the branches aren't.

Repos that were missing wmf.16 branches are things like mediawiki/extensions/Newsletter where the head of the branch hasn't moved since wmf.15; i.e.:

https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Newsletter/+/wmf/1.34.0-wmf.15 == https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Newsletter/+/wmf/1.34.0-wmf.16

It's strange that github would have the tags and gerrit2001 doesn't. Looking at the docs the push refspec is supposed to be refs/*:refs/*:

Defaults to refs/*:refs/* (push all refs) if not specified,
or +refs/*:refs/* (force push all refs) if not specified and
gerrit.defaultForceUpdate is true.

Our current settings are:

hieradata/role/common/gerrit.yaml
gerrit::jetty::replication:
    github:
        url: 'git@github.com:wikimedia/${name}'
        authGroup: 'mediawiki-replication'
        remoteNameStyle: 'dash'
        mirror: true
        push:
            - '+refs/heads/*:refs/heads/*'
            - '+refs/tags/*:refs/tags/*'
        createMissingRepositories: false
        maxRetries: 50
    slaves:
        url: 'gerrit2@gerrit2001.wikimedia.org:/srv/gerrit/git/${name}.git'
        mirror: true
        replicateProjectDeletions: true
        replicateHiddenProjects: true

Change 528181 had a related patch set uploaded (by Thcipriani; owner: Thcipriani):
[operations/puppet@production] gerrit: replication replicateOnStartup

https://gerrit.wikimedia.org/r/528181

thcipriani triaged this task as Normal priority.Aug 5 2019, 4:24 PM
thcipriani moved this task from INBOX to Doing on the Release-Engineering-Team-TODO (201908) board.

Change 528181 merged by Dzahn:
[operations/puppet@production] gerrit: replication replicateOnStartup

https://gerrit.wikimedia.org/r/528181

thcipriani closed this task as Resolved.Aug 13 2019, 1:09 PM
thcipriani claimed this task.

hrm. I wrote a little shell script that checks all wmf-deployed extensons:

check-gerrit-branch.sh
#!/usr/bin/env bash
set -euo pipefail
# Path to local copy of mediawiki/tools/release
TOOLS_REPO_PATH="$1"
BASE_PATH="https://gerrit-slave.wikimedia.org/r/mediawiki"
BRANCH='wmf/1.34.0-wmf.16'
for ext in $(jq -r '.extensions | .[]' "$TOOLS_REPO_PATH"/make-wmf-branch/config.json); do
    printf 'CHECKING %s....' "$ext"
    if ! git ls-remote --heads "${BASE_PATH}/${ext}" | grep -q "$BRANCH"; then
        printf '\n\t[ERRROR] no branch "%s" found for "%s"!\n' "$BRANCH" "$ext"
        exit 1
    fi
    printf 'Found!\n'
done

Re-ran this for BRANCH=wmf/1.34.0-wmf.17 and all branches were found. I think the addition of replicateOnStartup + the existing replication events seems to have solved this. Thanks for noticing this @MarcoAurelio !