Page MenuHomePhabricator

Basic metrics about contributors exercising +2/-2 permissions in Gerrit
Closed, ResolvedPublic

Description

The Engineering Community team wants to measure the involvement of non-WMF code contributors looking at how many users with +2 permissions we have in Gerrit, and which is their affiliation.

Tables

  • Full list of contributors who have exercised +2 OR -2 in Gerrit (name, primary affiliation, number of changes resolved) in the past month.
  • Same as above for the past 12 months.
  • Same as above for all time.

Graphs

  • Month by month, total number of contributors who have exercised +2 OR -2 in the past three months / in the past 12 months / all time.
    • X: months
    • Y = number of contributors
    • Three lines for current month, last 12 months, all times
    • Tooltip shows the data for each month
  • Month by month, percentage of contributors who have exercised +2 OR -2 in the past three months, by primary affiliation.
    • X: months
    • Y: percentage of each affiliation
    • Percent stack chart showing the distribution of the 100%
    • Tooltip shows the data for each month

Currently available: https://github.com/mzmcbride/gerrit-reports/blob/master/reports/code-review-activity.py / https://www.mediawiki.org/wiki/Gerrit/Reports/Code_review_activity (for almost-all time, can be filtered with an option "--wildcards 201501*" in the tar line).

Details

Reference
bz57038

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 2:38 AM
bzimport added a project: wikimedia.biterg.io.
bzimport set Reference to bz57038.

I do not think there is a way to easily find out who has +2 on
projects.

  • totally unfeasible approach below ---

If you absolutely have to find out, you can do by hand by going
through the projects settings (and parent projects recursively),
resolve the ACLs by hand, and resolve the groups with +2
permission to contributors. But that's a huge amount of work.
And note that while resolving the groups, gerrit is hiding groups that
you have no permission to see. So you might miss groups, if you
haven't got sufficient permissions.
And note that the ls-members (even with --recursive) does not expand
ldap groups.

Isn't https://gerrit.wikimedia.org/r/#/admin/groups/11,members the (glibal) list of folks with +2 in Gerrit, or am I wrong?

Acs added a comment.Nov 14 2013, 12:06 PM

Maybe a first approach could be to list the people that have used the +2 rights. With the actual data gathered from gerrit I think it should be too difficult.

Let me try:

mysql -u root acs_gerrit_mediawiki_2428

mysql> select count(distinct(changed_by)) from changes where new_value='2';
+-----------------------------+

count(distinct(changed_by))

+-----------------------------+

149

+-----------------------------+

149 people have used the +2 rights. We have join with upeople_companies table to search for its affiliations.

mysql> select count(distinct(changes.id)) as total, changed_by, people.name, company_id, companies.name from changes, people_upeople, people, acs_cvsanaly_mediawiki_2428.upeople_companies, acs_cvsanaly_mediawiki_2428.companies where people.id = people_upeople.people_id and changes.changed_by=people_upeople.people_id and upeople_companies.upeople_id = people_upeople.upeople_id and upeople_companies.company_id = companies.id and new_value='2' group by changed_by order by total desc limit 20;
+-------+------------+---------------------+------------+-----------------------+

totalchanged_bynamecompany_idname

+-------+------------+---------------------+------------+-----------------------+

385432jenkins-bot172Unknown
2814229L10n-bot3051translatewiki.net
476515Reedy3053Wikimedia Foundation
286269Siebrand3053Wikimedia Foundation
2530101Mark Bergsma3053Wikimedia Foundation
243927Hashar3053Wikimedia Foundation
215925Catrope3053Wikimedia Foundation
186961Ryan Lane3053Wikimedia Foundation
186835Aaron Schulz3053Wikimedia Foundation
1862120Jeroen De Dauw3052Wikimedia Deutschland
156770GWicke3053Wikimedia Foundation
146910Ori.livneh3053Wikimedia Foundation
1448156Tobias Gritschacher3052Wikimedia Deutschland
136020MaxSem3053Wikimedia Foundation
1356299Ottomata3053Wikimedia Foundation
1239353Dzahn3053Wikimedia Foundation
123554Tim Starling3053Wikimedia Foundation
12019Nikerabbit3053Wikimedia Foundation
104912Krinkle3053Wikimedia Foundation
1048167awjrichards3053Wikimedia Foundation

+-------+------------+---------------------+------------+-----------------------+

With this query you have the top20 +2 reviewers with the org.

If we want to exclude Wikimedia:

mysql> select count(distinct(changes.id)) as total, changed_by, people.name, company_id, companies.name from changes, people_upeople, people, acs_cvsanaly_mediawiki_2428.upeople_companies, acs_cvsanaly_mediawiki_2428.companies where people.id = people_upeople.people_id and changes.changed_by=people_upeople.people_id and upeople_companies.upeople_id = people_upeople.upeople_id and upeople_companies.company_id = companies.id and new_value='2' and companies.name not like '%Wikimedia%' group by changed_by order by total desc limit 20;
+-------+------------+------------------+------------+-------------------+

totalchanged_bynamecompany_idname

+-------+------------+------------------+------------+-------------------+

385432jenkins-bot172Unknown
2814229L10n-bot3051translatewiki.net
51536IAlex172Unknown
468170Mwjames3054Independent
46694Santhosh172Unknown
29331Hoo man3054Independent
264405Xqt3054Independent
2337Matmarex3054Independent
19983Raimond Spekking3054Independent
19666Addshore3054Independent
19128Alex Monk3054Independent
183242Yaron Koren3055WikiWorks
178171Pgehres3054Independent
14826Parent54463054Independent
14650Legoktm3054Independent
117132Tpt3054Independent
109293Merlijn van Deen3054Independent
10891Petr Onderka172Unknown
10622TheDJ3054Independent
93109Ladsgroup172Unknown

+-------+------------+------------------+------------+-------------------+
20 rows in set (0.41 sec)

We can create querterly reports also from this core SQL query.

Something like that Quim?

(In reply to comment #2)

Isn't https://gerrit.wikimedia.org/r/#/admin/groups/11,members the (glibal)
list of folks with +2 in Gerrit [...]

For projects underneath mediawiki/* group #11 (mediawiki) should have
+2. But they are not the only one, and they do not have +2 on all
branches (see refs/heads/wmf/* section of [1]). There mediawiki only
has +1 and for example wmf-deployment and ldap/ops have +2.

And looking also outside of mediawiki/* ... All-Projects (the root of
the hierarchy [2]) comes with +2 for “Project Owner”. “Project Owner” is
(typically) different for each repository. And some projects again
override this.

That's only the tip of the iceberg :-/

[1] https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki,access
[2] https://gerrit.wikimedia.org/r/#/admin/projects/All-Projects,access

Alvaro, I think your approach is very good. *Exercising* +2 rights is what matters about *having* +2 rights.

Useful data that we will use *tomorrow* at the Enginnering Community team quarterly review, if available:

(The scope is July - September 2013. All the better if we can have results of the previous three quarters in order to have a one-year view)

  • Total amount of users that exercised +2 rights.
  • Full list of users with affiliation.

This will be also useful to identify more "unknown". For instance:

jenkins-bot: shouldn't be counted
IAlex: independent
Santosh: WMF
Petr Onderka: independent (GSoC student, yay!)
Ladsgroup: independent

We might still fall in situation where two email addresses belong to a same person, or Contributor X was independent a year ago and now works at the WMF, but exposing this data will help us fine tuning user data.

Getting active +2 contributors right is a good goal per se. Setting priority accordingly. It is ok to let other tasks aside in order to get this data asap. Thank you very much for the flexibility!

Sorry for the bugmailspam.

Acs added a comment.Nov 15 2013, 1:18 PM

Quim, i need to review the companies mapping because there are some probs in the data. Once I debug it, I will provide the data you need. I am sorry not to have the data available for today!

Acs added a comment.Nov 15 2013, 2:32 PM

Ok Quim, finally I have found time for gathering the results.

From Jul to Sep 2013:

  • 34 people with +2 actions not from WMF
  • The complete list below

mysql> select count(distinct(changes.id)) as total, changed_by, people_upeople.upeople_id, people.name, company_id, companies.name from changes, people_upeople, people, acs_cvsanaly_mediawiki_2428.upeople_companies, acs_cvsanaly_mediawiki_2428.companies where people.id = people_upeople.people_id and changes.changed_by=people_upeople.people_id and upeople_companies.upeople_id = people_upeople.upeople_id and upeople_companies.company_id = companies.id and new_value='2' and changed_on > '2013-06' and changed_on < '2013-10' and companies.name <>'Wikimedia Deutschland' and companies.name<>'Wikimedia Foundation'group by changed_by order by total desc;
+-------+------------+------------+------------------+------------+-------------------+

totalchanged_byupeople_idnamecompany_idname

+-------+------------+------------+------------------+------------+-------------------+

169512164jenkins-bot172Unknown
797229171L10n-bot3051translatewiki.net
161170293Mwjames3054Independent
1377256Matmarex3054Independent
12566791Addshore3054Independent
124405151Xqt3054Independent
11950161Legoktm3054Independent
10891719Petr Onderka172Unknown
9331288Hoo man577cox
8026389Parent54463054Independent
78109169Ladsgroup172Unknown
78293128Merlijn van Deen3054Independent
5994529Santhosh172Unknown
3938997DrTrigon172Unknown
392258TheDJ3054Independent
3513261Tpt3054Independent
32122783Physikerwelt172Unknown
3183341Raimond Spekking3054Independent
2921417Brian Wolff3054Independent
26242554Yaron Koren3055WikiWorks
20159537KartikMistry172Unknown
1928261Alex Monk3054Independent
14232747Foxtrott172Unknown
1436363IAlex172Unknown
10365196Stefan.petrea3054Independent
649770Wikinaut172Unknown
3330866Christian1303wikia
381551Nischayn223054Independent
24241200Kelson2169btinternet
2140785Inez1303wikia
2408115Russell Blau98googlemail
152351Liangent3054Independent
1199345SPQRobin3054Independent
123598QChris3054Independent

+-------+------------+------------+------------------+------------+-------------------+
34 rows in set, 2 warnings (0.63 sec)

I hope it is useful for you!

(In reply to comment #8)

5994529Santhosh172Unknown

might work for WMF if it's https://wikimediafoundation.org/wiki/User:Sthottingal

20159537KartikMistry172Unknown

works for WMF: https://wikimediafoundation.org/wiki/User:KMistry_%28WMF%29

123598QChris3054Independent

works for WMF as contractor so depends on how you interpret it; http://lists.wikimedia.org/pipermail/wikitech-l/2012-February/058294.html

2921417Brian Wolff3054
381551Nischayn223054

FYI, have worked for WMF in the past

Ok, thank you very much! With this I have enough for today. Enjoy your weekend! This task is also useful to resolve the Gerrit review queue key performance indicator: https://www.mediawiki.org/wiki/Community_metrics#Gerrit_review_queue

I think the best approach for the unclear cases is to ask them to fill The Google Form where thy can provide details such as the affiliation.

The "cox", "btinternet", and "googlemail" entries can be reassigned as "Independent".

Also, Wikimedia Germany seems to be missing here, since they have +2 developers but they are not WMF.

Acs added a comment.Nov 20 2013, 3:22 AM

With this SQL query you can get the information for all quarters:

SELECT COUNT(DISTINCT(changes.id)) AS total, QUARTER(changed_on) as quarter, YEAR(changed_on) year, changed_by,

people_upeople.upeople_id, people.name, company_id, companies.name

FROM changes, people_upeople, people, acs_cvsanaly_mediawiki_2428.upeople_companies, acs_cvsanaly_mediawiki_2428.companies
WHERE people.id = people_upeople.people_id AND changes.changed_by=people_upeople.people_id AND upeople_companies.upeople_id = people_upeople.upeople_id
AND upeople_companies.company_id = companies.id
AND new_value='2'
AND companies.name <>'Wikimedia Deutschland' and companies.name<>'Wikimedia Foundation'
GROUP BY QUARTER(changed_on), YEAR(changed_on), changed_by ORDER BY year, quarter, total DESC;

+-------+---------+------+------------+------------+---------------------+------------+-------------------+

totalquarteryearchanged_byupeople_idnamecompany_idname

+-------+---------+------+------------+------------+---------------------+------------+-------------------+

138381201329171L10n-bot3051translatewiki.net
6346120132164jenkins-bot172Unknown
88612013385343Pyoungmeister172Unknown
8011201354236Tim Starling172Unknown
75912013156505Tobias Gritschacher172Unknown
667120136340Kaldari172Unknown

....

132013330866Christian172Unknown
9011420132164jenkins-bot172Unknown
32264201329171L10n-bot3051translatewiki.net
23942013299187Ottomata172Unknown
17642013405151Xqt3054Independent
135420132092MaxSem172Unknown
12342013241216Faidon Liambotis172Unknown
115420136510Mwalker172Unknown
108420137256Matmarex3054Independent
9542013389Jdlrobson172Unknown

...
185 rows in set (0.48 sec)

As you said it is a pretty good query to debug companies mapping!

Acs added a comment.Nov 20 2013, 3:37 AM

(In reply to comment #8)

Ok Quim, finally I have found time for gathering the results.
From Jul to Sep 2013:

  • 34 people with +2 actions not from WMF
  • The complete list below

mysql> select count(distinct(changes.id)) as total, changed_by,
people_upeople.upeople_id, people.name, company_id, companies.name from
changes, people_upeople, people,
acs_cvsanaly_mediawiki_2428.upeople_companies,
acs_cvsanaly_mediawiki_2428.companies where people.id =
people_upeople.people_id and changes.changed_by=people_upeople.people_id and
upeople_companies.upeople_id = people_upeople.upeople_id and
upeople_companies.company_id = companies.id and new_value='2' and changed_on

'2013-06' and changed_on < '2013-10' and companies.name <>'Wikimedia
Deutschland' and companies.name<>'Wikimedia Foundation'group by changed_by
order by total desc;

Guys, this query was not correct because "changed_on > '2013-06' and changed_on < '2013-10'". This should be "changed_on > '2013-06-31' and changed_on < '2013-10-01'".

It is better for quarter analysis to use:

"QUARTER(changed_on) = 3 and YEAR(changed_on) = '2013'"

Acs added a comment.Nov 20 2013, 3:39 AM

(In reply to comment #12)

(In reply to comment #8)

Ok Quim, finally I have found time for gathering the results.
From Jul to Sep 2013:

  • 34 people with +2 actions not from WMF

With the correct query this number is:

  • 49 people with +2 actions not from WMF in 2013 Q3.

I am sorry about the noise!

  • The complete list below

mysql> select count(distinct(changes.id)) as total, changed_by,
people_upeople.upeople_id, people.name, company_id, companies.name from
changes, people_upeople, people,
acs_cvsanaly_mediawiki_2428.upeople_companies,
acs_cvsanaly_mediawiki_2428.companies where people.id =
people_upeople.people_id and changes.changed_by=people_upeople.people_id and
upeople_companies.upeople_id = people_upeople.upeople_id and
upeople_companies.company_id = companies.id and new_value='2' and changed_on

'2013-06' and changed_on < '2013-10' and companies.name <>'Wikimedia
Deutschland' and companies.name<>'Wikimedia Foundation'group by changed_by
order by total desc;

Guys, this query was not correct because "changed_on > '2013-06' and
changed_on
< '2013-10'". This should be "changed_on > '2013-06-31' and changed_on <
'2013-10-01'".
It is better for quarter analysis to use:
"QUARTER(changed_on) = 3 and YEAR(changed_on) = '2013'"

Note that self-merges must not be counted as +2 actions, see bug 53485 comment 35 and 39 for references.

Ok, we need to find a way to get this data in http://korma.wmflabs.org/browser/gerrit_review_queue.html

What about a table like "Reviews merged"at http://korma.wmflabs.org/browser/who_contributes_code.html but counting the amount of people exercising +2 every month, by organization?

Let's change this to low until we really know what we need. It is unclear now. Ideas welcome.

Maybe a way to approach this task is to think what do we want to report in the ECT quarterly reviews.

Qgil raised the priority of this task from Low to Normal.Jan 8 2015, 11:21 AM

What about this:

  • How many users have exercised +2 OR -2 in a given month.
  • Percentage of users that have exercised +2 OR -2 by affiliation in a given month.
Krenair added a subscriber: Krenair.Jan 8 2015, 1:22 PM

This entire idea is flawed because it does not account for multiple affiliations (e.g. Independent and WMF/WMDE).

The list of people is short, and we can look at the details when we have it. Potentially flawed data is still better than no data, imho.

As for "Percentage of users that have exercised +2 OR -2 by affiliation in a given month", I think that no data is better. What's the point?

To see whether there is any trend over time, mainly WMF vs the rest. I think a community with 50% of non-WMF users actively using their +2 is more desirable and healthier than a community where basically all commits go through WMF employees.

But hey, if I'm the only one finding this data point interesting, no problem. What really matter is to see the trend of users exercising their +2 powers.

In T59038#968641, @Qgil wrote:

To see whether there is any trend over time, mainly WMF vs the rest. I think a community with 50% of non-WMF users actively using their +2 is more desirable and healthier than a community where basically all commits go through WMF employees.

As I thought. :) I agree this is interesting, but perhaps more easily seen by measuring how many +2 come from users who qualified themselves as "Independent" first (this delegates "independency" considerations to each user). This way we don't need to worry about all affiliations, nor about precise numbers given to each affiliation.

demon removed a subscriber: demon.Jan 12 2015, 12:05 PM

It's not that we have dozens of types of affiliation. In order to understand "Independent" we need to see the magnitude of "Unknown", because otherwise we would assume that !independent == professionals, which is probably not true. Once you have Independent and Unknown, it is not difficult to identify WMDE and WMF. Is there any other affiliation likely to have +2 members?

Once we have these numbers and a list of names, then we can discuss details like people with more than one affiliation if needed.

Nemo_bis added a comment.EditedJan 13 2015, 8:11 AM

That's not quite a detail. The WMF people doing code review for non-WMF people are typically those who were active in MediaWiki before being in WMF, or who consider themselves independent/MediaWiki first, or who have other clients/projects.

If you don't collect this information, considering it a "detail", you have good chances to miss the only important driving factor and possible way for improvement we have.

ori added a subscriber: ori.Jan 13 2015, 8:13 AM

That's not quite a detail. The WMF people doing code review for non-WMF people are typically those who were active in MediaWiki before being in WMF, or who consider themselves independent/MediaWiki first, or who have other clients/projects.

Less dramatic: "The people who do the most code review are those that have been around the longest and have the most experience."

Nemo, I don't understand your arguments in this task. First you say that it is not worth analyzing data by affiliation. Then you make a statement about WMF employees reviewing changes contributed by independents versus those who don't, which (true or not) it shows that you care about affiliations indeed.

In order to have data to reply to complex questions like "Are WMF developers prioritizing WMF patches / discriminating independent patches?" (which seems to be your underlying assumption), we need some building blocks first. I will keep focusing on these basic building blocks before speculating about the complex questions.

Qgil renamed this task from Metrics about contributors with +2 permissions in Gerrit to Basic metrics about contributors exercising +2/-2 permissions in Gerrit.Jan 13 2015, 9:26 AM
Qgil updated the task description. (Show Details)
Qgil set Security to None.

I have edited the description proposing the data we want to gather in this task. Suggestions for improvement are welcome.

Let's start with this:

  • A table with the full list of contributors who have exercised +2 OR -2 in Gerrit (name, primary affiliation, number of changes resolved) in the past month.
  • A graph showing total number of contributors who have exercised +2 OR -2 each month.
    • X: months
    • Y = number of contributors
    • Tooltip shows the data for each month
Qgil reassigned this task from Acs to Dicortazar.Feb 5 2015, 10:49 AM
Nemo_bis updated the task description. (Show Details)Feb 5 2015, 2:11 PM
Dicortazar added a comment.EditedFeb 18 2015, 5:32 PM

Hi,

I've been working a bit on this. Let's have a look at first results just to be sure that we're working on the right direction and check that the dataset makes sense :).

First, this is the analysis of active core reviewers. An active code reviewer is defined as those people that can exercise a +2 or -2 in the review process.

The attached chart shows the evolution of the different people that exercised that review in 1,206 gerrit projects available in the database in 2013, 2014 and 2015 in a monthly basis.

In addition, the following list shows the list of people that exercised that action and that number of actions. In this table I'm missing affiliations just to check. I have also removed the Translation bot. I have also added only those with at least 30 different +2/-2 actions.

  • People exercising a +2/-2 action in Gerrit between 2015-01-18 and 2015-02-18.
legoktm233
jforrester102
Catrope92
hoo87
Bartosz Dziewoński81
Krinkle78
Sam Reed77
jayvdb74
Adrian Lang66
Ed S65
Yaron Koren63
Trevor Parscal51
Christopher Johnson50
Matthew Flaschen50
thiemowmde50
Jon Robson45
bmansurov44
Bryan Davis40
ori40
hashar39
Niklas Laxström39
Umherirrender36
ssastry36
Santhosh Thottingal35
Phuedx34
Moritz Schubotz33
MaxSem31
Gilles Dubuc30
Anomie30
Fabian Neundorf30
mlitn30

[...]

These results were retrieved using a python script, not sure if I should upload that here as an example of use.

Thank you, this is going in the right direction.

About the graphic, I wonder why there is such spike on April 2013. The rest looks good.

About the table, for this +2/-2 metric I think it is worth listing absolutely everybody, with affiliation.

You can use Paste to share simple scripts, and then you can include those pastes here with this format: {P123}

Qgil moved this task from Ready To Go to Doing on the ECT-February-2015 board.Feb 19 2015, 1:43 PM

I have also removed the Translation bot.

Why did you need to remove l10n-bot? It's not needed, if you exclude self-merges from counts (which you really must).

Oh well, it's not a big deal. I just removed what seemed to me bots.

In the following iteration I'll keep that info in the list and double check the self-merges.

Thanks for the pointers!

Hi, new numbers and list of developers.

First, I've been working on removing self-merges. These are defined as reviews that are merged by the same developer that submitted the changeset. There may be cases where in the review process third parties upload new patchsets. Those third party developers were not traced in this analysis.

As mentioned by @Nemo_bis, removing the self-merges removed the bots. Numbers are a bit lower than previous chart.

In addition the list of developers that at some point during the last month added a +2 or -2 together with their affiliation.

useraffiliationCore Review Event (+2/-2)
legoktmIndependent214
jforresterUnknown118
CatropeWikimedia Foundation111
hooUnknown80
Ed SUnknown74
Bartosz DziewońskiIndependent68
Adrian LangUnknown58
Trevor ParscalWikimedia Foundation53
jayvdbUnknown45
Sam ReedWikimedia Foundation43
Matthew FlaschenWikimedia Foundation42
Niklas LaxströmWikimedia Foundation42
PhuedxUnknown39
ssastryUnknown38
KrinkleWikimedia Foundation35
Bryan DavisWikimedia Foundation35
zfilipinUnknown34
UmherirrenderUnknown33
thiemowmdeWikimedia Deutschland33
Santhosh ThottingalWikimedia Foundation33
bmansurovUnknown31
MaxSemWikimedia Foundation29
mlitnWikimedia Foundation28
ArlolraWikimedia Foundation27
krenairUnknown26
siebrandUnknown25
AnomieWikimedia Foundation25
ebernhardsonUnknown25
jsahleenUnknown24
Fabian NeundorfUnknown23
Jon RobsonWikimedia Foundation22
ChadWikimedia Foundation21
Gilles DubucWikimedia Foundation21
Adam Roses WightWikimedia Foundation19
oriUnknown19
isarraUnknown19
amire80Wikimedia Foundation18
florianschmidtwelzowUnknown18
hasharUnknown17
jeroendedauwWikimedia Deutschland17
rmoenUnknown17
MarkTraceurWikimedia Foundation15
dbrantUnknown14
Aaron SchulzWikimedia Foundation14
JanZerebeckiUnknown13
xqtIndependent13
mhurdUnknown13
bsitzmannUnknown13
audeUnknown13
Henning SnaterUnknown13
dr0ptp4ktUnknown13
kaldariWikimedia Foundation11
addshoreIndependent11
danielUnknown11
TTOUnknown9
jhernandezUnknown9
marcoilUnknown9
KartikMistryUnknown8
EjeggUnknown8
Chris McMahonWikimedia Foundation8
Andrew GarrettWikimedia Foundation8
MooeypooUnknown7
Brion VibberWikimedia Foundation7
Robert VogelUnknown7
Gabriel WickeWikimedia Foundation6
Gergo TiszaUnknown6
Andrew GreenWikimedia Foundation5
Jack PhoenixUnknown5
yurikUnknown5
Merlijn van DeenIndependent5
Tim StarlingWikimedia Foundation4
Dan DuvallUnknown4
MpaaUnknown4
milimetricUnknown4
ryasmeenUnknown4
yuvipandaUnknown4
hartmanUnknown3
Moritz SchubotzUnknown3
Alexandros KosiarisWikimedia Foundation3
tptIndependent3
JackmcbarnUnknown3
gnosygnuUnknown3
ladsgroupUnknown2
csteippUnknown2
Tyler Anthony RomeoIndependent2
wirthUnknown2
tamaraslosarekUnknown2
Mukunda ModellUnknown2
nuriaUnknown2
wctaiwanUnknown2
tonythomas01Unknown2
Yaron KorenWikiWorks2
Katie HornWikimedia Foundation2
Raimond SpekkingIndependent2
dzahnUnknown2
AndrewUnknown2
DeskanaUnknown2
jonas.keutelUnknown1
PleaseStandUnknown1
MalvineousUnknown1
cicaleseUnknown1
ottoUnknown1
mwjamesUnknown1
prtksxnaUnknown1
PigpenUnknown1
ManybubblesWikimedia Foundation1
ricordisamoaUnknown1
Marcel Ruiz FornsUnknown1
pginerUnknown1
tosfosUnknown1
JoelKPUnknown1
cscottUnknown1
Felipe SchenoneUnknown1
soeren.oldagUnknown1
MobrovacUnknown1
HypergroveUnknown1
mahUnknown1
russblauUnknown1
rushUnknown1
Marielle VolzUnknown1
pcoombeUnknown1
o0lilu0o1980Unknown1
TweichartUnknown1
Giuseppe LavagettoUnknown1
markUnknown1
filippoUnknown1
FoxtrottUnknown1
MZMcBrideUnknown1

The work described in T59038#1017504 is covered by this graphic and this table. Thank you! Now we need to decide where to publish it.

There may be cases where in the review process third parties upload new patchsets. Those third party developers were not traced in this analysis.

Good point. This needs to be explained somewhere in korma when we publish this metric.

As the following step, the defined chart and table will be part of the dashboard in the code review overview at http://korma.wmflabs.org/browser/scr.html.

A first prototype will be provided with information from this task (T59038: Basic metrics about contributors exercising +2/-2 permissions in Gerrit) and T86152: Active Gerrit users on a monthly basis.

Please, have a look at the first version.

In order to keep in line with upstream, the first approach is to add the list of top ten in the scr overview page. This first step can be seen in the attached image.

If we want to keep this version, we're missing the following information:

  • the rest of the list up to 900 participants and some dozens of active core reviewers.
  • we're missing affiliations.

Thus, in order to add the missing information. We're considering the option of adding them to the contributors page. If you check the left hidden menu, there's no current version of scr-contributors.html, but this exists for instance for scm (scm-contributors.html). So, the idea would be to add the full list of contributors (participants or core reviewers) there.

With this second step, it would be difficult to add the affiliation (let's say that this is not in general supported in the dashboard, except in some special cases for Wikimedia). But, I'd say that this is doable if this is necessary.

Well, there is a disconnect between what we had agreed in February and what we are being offered now. We don't need to discuss the information provided again, right? I understand the constraints you have keeping an upstream version, but in this case we are not really interested in who is in the top 10, we are interested in knowing whether the total amount of users exercising their +2/-2 rights increases or decreases, and what is the trend of activity of paid developers vs volunteers.

If we need to have a Wikimedia-only page for this, so be it. We will assume the cost of production and maintenance.

There is indeed a disconnect, but as I commented, we were trying to keep inline with upstream.

Metrics are clear and what you expect is what I provided in previous comments as a table and as an evolutionary chart.

Regarding to the panel, we'll produce it with exactly that information. Probably adding both metrics (participants and active core reviewers) in the same page would be enough.

Dicortazar added a comment.EditedMar 26 2015, 2:50 PM

Hi again!

To follow up with the discussion, please find attached the aforementioned contributors page (please, ignore the dataset you can see):

As commented in the previous comment with the first image, that would be the information added to the scr overview panel. I'd say that it's interesting to have such data there in any case.

And then, the attached image would contain the specific information you asked for (without affiliations, some comments about this at the end of this comment).

So, summarizing: the addition of participants and active core reviewers would be added into two main panels:

  • Code review -> Overview, that would be similar to the first attached image.
  • Code review -> Contributors, that woulud be similar to the image attached in this comment.

Next steps:

QChris removed a subscriber: QChris.Mar 26 2015, 3:31 PM

Alright, thank you!

About

  • OK to the graph.
  • OK to the two tables.

BUT I think there is a problem with the labels used, because they are difficult to understand if you just land in that page, even if you are a code review expert.

  • "Participants" could be renamed to "Code review users"
  • "Active core reviewers" could be renamed to "Code review committers"
  • "Events" is not bad, but maybe "Actions" is clearer?
  • "Merged" is inaccurate if we are counting -2 there as well. "Resolutions"?

About

  • I think it is better to reuse the same graph with both data combined, because it allows to compare them better.
  • OK to the tables, waiting for affiliations.

All this considering the same changes proposed for labels.

I agree with your labels-renaming suggestions. Let's use them

And with respect to the chart, not an issue to merge the two charts into one. This would be consistent with the overview panel.

Thanks for the comments!.

This task is blocked till we get updated dataset from the migration of Korma to SortingHat and upstream.

Dicortazar moved this task from Backlog to Doing on the ECT-April-2015 board.Apr 22 2015, 11:41 AM

This panel is ready to go (http://korma.wmflabs.org/browser/scr-contributors.html).

Please have a look at it.

Affiliations were added and bots removed as final steps

Dicortazar added a comment.EditedApr 22 2015, 3:02 PM

@Qgil, we've (@Lcanasdiaz and me among others) discussed about the renames a bit more.

  • "Participants" makes sense to be renamed as "Code review users"
  • We disagree with the use of "Code review committers". The use of "committers" may be misunderstood with the usual "committer" found in Git. In addition to this, the term "Active core reviewers" is used by other projects such as OpenStack. We'd use "Active core reviewers".
    • Active because we only count those that are active (we do not know anything about those that have not voted a +2/-2).
    • Core because only core people can vote a +2/-2.
    • And reviewer because they are reviewers and they are not committing anything to the source code afaik.
  • It's ok to use "Actions" instead of "Events".
  • Regarding to Merged, you were right that this is wrong. We should use another term. We're not sure about "Resolutions" and we prefer to rename this as "votes". So probably adding "+2/-2 Votes" is enough with this respect and helps to avoid misunderstanding with other review systems.

Core is generally meant to mean MediaWiki core and the MediaWiki core group in gerrit, who can do CR almost everywhere: see https://www.mediawiki.org/w/index.php?title=Developers

Technically, people who do +2 *are* the committers in git sense: they commit the merge (at some point jenkins-bot was the committer).

Qgil added a comment.EditedApr 23 2015, 12:33 PM

These graphs are interesting.

As Nemo say, in our context a "core reviewer" is a reviewer of MediaWiki core. Not the end of the world, but if we can find a better term. They are git committers indeed, a role different than the "authors" shown in http://korma.wmflabs.org/browser/scm.html

"+2/-2 votes" works in our case with Gerrit now, but other systems don't have the same concept. Anyway, good for now.

We are not going to keep this task blocked for the labels, though. Please implement the labels agreed and close the task. We can continue the discussion about labels here if needed.

The labels were changed according to the agreement.

  • Code review users instead of participants
  • Code review committers instead of active core reviewers
  • Actions instead of events
  • +2/-2 votes instead of merged

Thanks @Lcanasdiaz!

Qgil closed this task as Resolved.Apr 30 2015, 9:21 AM

Thank you! This is done.