Bigger picture: T330347: A tool for displaying the canonical location of all Wikimedia repositories and any known mirrors
Docs: https://we.phorge.it/book/phorge/article/diffusion_uris/#reference-i-o-types
URI stats (note that the number of repos is lower, and that this query ignores the active/disabled state of the repo itself):
mysql:phstats@m3-slave.eqiad.wmnet [phabricator_maniphest]> SELECT COUNT(u.id), u.ioType FROM phabricator_repository.repository_uri u WHERE u.isDisabled = 0 GROUP BY u.ioType; +-------------+-----------+ | COUNT(u.id) | ioType | +-------------+-----------+ | 1970 | default | | 74 | mirror | | 366 | none | | 2846 | observe | | 18026 | read | | 8 | readwrite | +-------------+-----------+
Per SELECT CONCAT("https://phabricator.wikimedia.org/diffusion/", r.id, "/manage/uris/"), r.name, u.uri FROM phabricator_repository.repository_uri u INNER JOIN phabricator_repository.repository r ON r.phid = u.repositoryPHID WHERE u.ioType = "mirror" AND u.isDisabled = 0 AND r.details NOT LIKE "%\"importing\":false%" ORDER BY u.uri; we mirror (up to) 74 repos from Diffusion to other places:
- 3 times to Gerrit (1913, 1334, 2341) by using K18,
- 4 times to Gerrit via ssh://phab@gerrit.wikimedia.org:29418 (2668, 1914, 2047, 1912) by using K19,
1 time to personal GitLab (see below)4 times to personal GitHub (2311, 2310, 2761, 2579) (see below),- the rest is all to GitHub under toolforge or wikimedia.
- Some of these GitHub repos might be read-only so trying to mirror should fail (e.g. malu in 1876, likely more).
- Some of these GitHub repos have been deleted so trying to mirror should fail (e.g. 1972, likely more)?
- Some repos like 1876 are "disabled" already in Diffusion so does Diffusion still try to push a mirror out then? Do I need to fix the SQL queries above to also take that in consideration by adding AND r.details NOT LIKE "%\"importing\":false%" ? This reduces the mirror number from 74 down to 30: 29 being mirrored to github.com/wikimedia/* and 1 being https://phabricator.wikimedia.org/diffusion/PHES/manage/uris/
- https://phabricator.wikimedia.org/diffusion/PHES/manage/uris/ states it mirrors to Gerrit but that repo is marked as inactive, where to find that info in the Phabricator database?
Incomplete list of some action items:
- T349610: Three repositories mirrored by Diffusion from GitLab into both "Toolforge" and "Wikimedia" on GitHub
- We try to mirror to Mbch331's personal GitLab from https://phabricator.wikimedia.org/source/tool-ddescriptions/manage/uris/ (credentials in K24) with a target which does not exist anymore according to https://gitlab.com/users/mbch331/projects - Should this URI be deleted? - Yes, disabled https://phabricator.wikimedia.org/source/tool-ddescriptions/uri/view/19560/ per T347577#9206606
- We try mirror to Urbanecm's personal GitHub in 2310 and 2311. Both use destroyed K25, so I'd say these URIs should be deleted or disabled? - Yes, Martin disabled mirroring to Github for 2310 and 2311
- Among the mirrors to personal GitHub, disable mirroring for 2579 (owned by @Dibya) to Github because no credentials configured thus broken anyway
- Changed Github mirror of https://phabricator.wikimedia.org/source/malu/manage/uris/ from Mirror to No I/O (as GitHub repo is archived)
- T348325: Delete GitLab repo toolforge-repos/povoconta and its Diffusion mirror (code hosted in GitHub instead)
- Disable readwrite repos - see T347577#9277852 below