Page MenuHomePhabricator

Clean up Diffusion's repo mirroring HTTP 403 failures in /var/log/phd/daemons.log on phab1004
Open, LowPublic

Description

...because the Projects-Cleanup template has not always been followed over the last years so lots of noise in there, potentially hiding other issues. Found because I was originally looking for pointers in T274559#9695403.

  1. Confirm repo does not exist anymore, browser style (https://gerrit.wikimedia.org/g/path/to/deleted/repo.git) and local (git clone ssh://gerrit.wikimedia.org:29418/path/to/deleted/repo.git).
  2. Then corresponding sudo /srv/phab/phabricator/bin/remove destroy rKILLKILLKILL for whatever it's called in Diffusion. Ignore some DB errors about edges.
  3. FYI SELECT CONCAT("https://phabricator.wikimedia.org/diffusion/", r.id, "/manage/uris/") AS repoURI, r.name, u.uri FROM phabricator_repository.repository_uri u INNER JOIN phabricator_repository.repository r ON r.phid = u.repositoryPHID WHERE u.uri LIKE "%deleted/reponame%"; can sometimes also come handy.

Randomly mirroring repos around between four different systems based on idealistic incentives or questionable beliefs may have downsides; T359549: Drop our mirroring of code to Diffusion and empty the repos may have a point.

Event Timeline

I think I destroyed all mirrored Diffusion repos throwing a HTTP 403 in the logs except for rNRDK (no idea what that one is doing). Those are/were all Gerrit mirrors.

There are also GitLab mirrors with errors with a misleading could not read Username for 'https://gitlab.wikimedia.org': No such device or address in the logs, e.g. R2394 trying to pull/mirror from non-existing https://gitlab.wikimedia.org/toolforge-repos/revi-kowikibots.git (random example).

aklapper@phab1004:~$ grep -r "Error while updating the" /var/log/phd/daemons.log.1 | grep "repository. {>} (CommandException) Command failed with error #255!" | cut -c 84- | sort -u
Error while updating the "rEHPR" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "rNRDK" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "rWRES" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-animanga-db-matcher" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-api-docs" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-axel" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-botutor" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-contentcontributor" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-discord-irc" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-milhistbot-bd808-test" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-phabricator-bug-status" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-revi-kowikibots" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-vipbot" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-wikishootme" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-xtools-legacy" repository. {>} (CommandException) Command failed with error #255!
Error while updating the "tool-ytcleaner" repository. {>} (CommandException) Command failed with error #255!

Rest are tool repos still empty / not created yet in Gitlab.