...as that was last updated two years ago.
https://phabricator.wikimedia.org/T146135#3176718 (basically a copy as a SparQL query) might also be outdated.
Description
Description
Related Objects
Related Objects
- Mentioned In
- T146135: Exclude upstream repositories in the default view on wikimedia.biterg.io (by setting up "Projects" once Bestiary is available?)
T179329: Investigate listing the "Onboarding New Developers" KPIs on a custom dashboard
T184420: Explain difference in number of repositories when trying to manually exclude imported third party repositories - Mentioned Here
- T146135: Exclude upstream repositories in the default view on wikimedia.biterg.io (by setting up "Projects" once Bestiary is available?)
T103292: Check whether it is true that we have lost 40% of (Git) code contributors in the past 12 months
Event Timeline
Comment Actions
Well, https://github.com/Bitergia/mediawiki-repositories/ is not used anymore in our stack so the summary is slightly misleading, but the situation is:
- https://wikimedia.biterg.io/app/kibana#/dashboard/Gerrit lists 1769 repositories.
- ssh -l username -p 29418 gerrit.wikimedia.org gerrit ls-projects | wc -l lists 1902 repositories (was 1305 in T103292#1404495 32 months ago)
- https://wikimedia.biterg.io/app/kibana#/dashboard/Git lists 1142 repositories.
(The topic reminds me of bits and pieces again in T103292#1693361.)
Comment Actions
Trying to gather understanding by running diff -pu gerrit-repo-list-from-ssh-T187711.txt gerrit-repo-list-from-bitergia-T187711.txt:
- output that shows a +: historical repositories which saw activity and hence are indexed by Bitergia, but those repositories do not exist anymore nowadays hence ls-projects does not list them, like https://gerrit.wikimedia.org/r/#/admin/projects/?filter=wikimedia%252Ffundraising%252Ftwig
- output that shows a -: existing repositories in G[err]it not indexed by Bitergia because there has not been any review activity in Gerrit, like https://gerrit.wikimedia.org/r/#/q/project:search/query_string_plus_plus_plus
Comment Actions
Using grep -r "organization" wikimedia-affiliations.json | sort | uniq -c to create the query author_org_name:"Canonical" OR author_org_name:"Deviantart" OR author_org_name:"Etsy" OR author_org_name:"Facebook" OR author_org_name:"Google" OR author_org_name:"HP" OR author_org_name:"IBM" OR author_org_name:"Intel" OR author_org_name:"Phacility" OR author_org_name:"RedHat" OR author_org_name:"Suse" (affilations I added intentionally to our database to be able to find potential pulled 3rd party repos), the Gerrit stats look as empty as I had expected and hoped.
Which leaves us with the (less important) Git stats: Applying the same query we get 22 repositories to take a closer look at. So I think we can throw away https://phabricator.wikimedia.org/T146135#3176718 mostly nowadays.
On a related note, I am puzzled why there is a Github repository in there: https://github.com/wikimedia/jquery.ime
Comment Actions
Well. For Git in https://wikimedia.biterg.io/goto/32ce4404dcba6ba93efb2aee2c9dd752 , picking a random repo from the bottom, like repo_name:"https://gerrit.wikimedia.org/r/operations/debs/kafka", you of course still get 171 authors and just checking like in the previous comment via author_org_name does not help as most contributors to that upstream project are under Independent per definition (no company email address used). We have three Wikimedia contributors though in that upstream project.
So nothing new, this boils down to the old "cannot identify repositories pulled from upstream" problem.
I'll ignore Git for the rest of this comment, as we concentrate on Gerrit in our metrics anyway.
I took the one year old Gerrit query from T146135#3176718 and inverted it to only show those Gerrit repos excluded in T146135 to see its contributors and if that still makes sense and went for the last 6 months to avoid a visualization timeout of 30s. The listed author names don't show pollution either.
I updated https://www.mediawiki.org/w/index.php?title=New_Developers%2FQuarterly%2FHow_to_Create_a_Report&type=revision&diff=2747355&oldid=2718239 accordingly.