RFC @Milimetric and @MGerlach

Ok, proposal looks solid. I like the detail and am excited we get to work with someone of your background. One higher level thought does come to mind, and I'd love @MGerlach's opinion as well. The data that I know is available to compute this type of metric is the user's gender, voluntarily filled in by each user. I have lost track of recent work on understanding that data, and I think @Isaac has a much fresher perspective, but here are my assumptions phrased as questions:

• sparse: not a lot of people fill out this preference?
• relatively constant: doesn't change very much from year to year?
• potentially biased towards folks who feel comfortable self-identifying on a platform like ours?

If the answer to these is more "yes" than "no", then maybe this work could compute a yearly number instead of a monthly metric. We could publish this in a separate section of Wikistats on a yearly basis, with other metrics that we don't think change much. And if we get better data and the number starts moving more quickly, we can add it as a metric. To be clear, I think this would still be very valuable to surface and we've been thinking of such a yearly report for other overall totals.

If the answer is more "no" than "yes", let's make sure to take into account how the data is better now, as we compute the metric.

Thanks, and looking forward to this!

Thanks @Milimetric you for you input.

The Gini coefficient (and other inequality metrics) has been used by us (see WikiChron) and many other studies to measure concentration of work production (edits) within a particular wiki / communities. This can be seen as a proxy, for instance, of how horizontal a community is. Nonetheless, online communities (and also CBPP) commonly follow the 90-9-1 rule (see article in Wikipedia). so inequality is, in all the cases, expected.

I'd like to refer here to my talk in Wikimedia Hackathon 2018, where I explain better how we used these metrics to analyze productivity and concentration of work in Fandom (Wikia) wikis, as well as to the respective published article in OpenSym.

I hadn't thought before to use it as a way to measure gender gaps. My opinion is that further discussion is needed to deepen in how to use inequality metrics for this (you think to compare edits made by man vs woman, number of users of each, number of top editors,..?). To the best of my knowledge, I don't know of any work which has done along these lines for wikis previously. Also, I share the concern that gender typification on the mediawiki ecosystem is not very well spread and reliable, although I am aware of some gender research which has been successfully done for Wikipedia.

Also, I'd like to ask @awight for his input on this.

Finally, I want to highlight that I could only commit part-time, given the current situation and my possible future situation in summer.

In T248964#6030168, @Quasipodo wrote:

Finally, I want to highlight that I could only commit part-time, given the current situation and my possible future situation in summer.

About the time commitment, around 30+ hours per week are required as per the program guidelines: https://developers.google.com/open-source/gsoc/faq#how_much_time_does_gsoc_participation_take. @Quasipodo Will it be possible for you to commit that many hours?

I like the proposal and think tracking inequality metrics over time as described above can be very informative about health of a community.
Regarding the discussion on gender, my understanding is that the current proposal only looks at inequality of editing across all its users regardless of gender. I do think that looking at gender-inequality specifically would be super interesting. If possible, it would be great to include but I suspect this would be a project in itself. This discussion came up previously in the Analytics/Research office hours, so I am putting some pointers to relevant work here:

• Gender on editors from surveys, such as the Community Insights Survey
• Lam et al. 2011 [1] explored gender imbalance in edits based on user preferences, which is stored in the user_properties-table

[1] Lam, S. (tony) K., Uduwage, A., Dong, Z., Sen, S., Musicant, D. R., Terveen, L., & Riedl, J. (2011). WP:clubhouse? an exploration of Wikipedia’s gender imbalance. Proceedings of the 7th International Symposium on Wikis and Open Collaboration, 1–10. https://doi.org/10.1145/2038558.2038560

The data that I know is available to compute this type of metric is the user's gender, voluntarily filled in by each user. I have lost track of recent work on understanding that data, and I think @Isaac has a much fresher perspective, but here are my assumptions phrased as questions:

• sparse: not a lot of people fill out this preference?
• relatively constant: doesn't change very much from year to year?
• potentially biased towards folks who feel comfortable self-identifying on a platform like ours?

@Milimetric yeah see some basic analyses I did here: https://meta.wikimedia.org/wiki/Research:Surveys_on_the_gender_of_editors/Report#User_Preferences

Quick takeaways would be:

• Yes, it is sparse, especially among new editors: ~2% of new users have filled it out vs. 25% or greater of long-time editors
  • You can account for some of this by joining in edit counts and balancing the results so they aren't dominated by long-term editors
• I didn't look at how quickly it changes but I did look at how it varies depending on how you define active editors.
• We didn't see any evidence that men or women were more likely to set the user preferences gender field but we only looked at Arabic, English, Norwegian Wikipedia, so this behavior might occur in other contexts.
• The main issue was that most editing communities are quite small combined with low response rates for setting gender, so plausibly if you have an editathon where they encourage new users to add their gender properties, this could significantly move the numbers for that language community. So I just don't consider the data robust unless new users are nudged to set the property when they create their accounts (but then I have a bunch of suggestions for what else would have to happen too).

It sounds interesting for me as well to try to measure the gender gap using inequality metrics. I can see it as a follow-up task once some inequality metrics have been implemented for the general edits.

Also other interesting variants can be explored like comparing Gini on males editors vs Gini on female editors on different wikis (see if there's more equality in one gender than another or if they behave similarly), I don't know of any previous research doing this kind of analysis: inequality of work by gender in collaborative online communities, and can be of broad interest. All of this given the premise that we have an statistical representative "population" of editors with labeled gender.

We might be going too much off-topic now and maybe another phabricator task / subtask should be added. Just for the record of it, I'd just wanna add that there was a specific report in 2018 about gender equity in Wikipedia based on surveys and there are some wiki pages specifically dedicated to this topic and with a good compilation of related references both in Wikipedia and in meta Wikimedia.

@Quasipodo perhaps you already know, but I came across the wikidata knowledge imbalance dashboard which calculates the gini-coefficient for different wikidata items and does a good job of putting the obtained numbers into context and explaining what they mean. Maybe a useful reference and inspiration for how to present metrics.

@MGerlach Wow! very visual and interactive way to explore the gini coefficient. I really like it!

Hi everyone! After checking with Google about the time requirements for the program, I learned that we would not be able to accept this proposal as part of GSoC as it does not meet the requirement of +30 hrs time commitment per week. Of course, @Quasipodo can work on the project outside GSoC.

Good to know. In that case, I'm also happy to keep an eye on this task and help as I can, and we can be less worried about deadlines.

(Maybe, should we move all the non-GSoC discussion to the other phabricator task?)

In T248964#6061882, @MGerlach wrote:

@Quasipodo perhaps you already know, but I came across the wikidata knowledge imbalance dashboard which calculates the gini-coefficient for different wikidata items and does a good job of putting the obtained numbers into context and explaining what they mean. Maybe a useful reference and inspiration for how to present metrics.

Anyway, and just for the record of it, I think something similar to what they do in that webapp could be done for articles within categories in Wikipedia, or to compare different wikipedia languages, etc. etc. The range of options can span quite widely and I guess it mostly depends on what the foundation is more interested in inspect, visualize and analyse.

@Quasipodo Thanks for your application. The GSoC projects have been announced, and as per T248964#6063593 we couldn't select this project. I've updated the task title and removed the GSoC tag. :)

@Quasipodo sorry to hear that the project was not selected.
I am still happy to keep an eye on this task and help out if I can if you decide to pick up any of this in any case.

Likewise, @Quasipodo.

Thank you @Milimetric and @MGerlach, I appreciate that. Let's see how everything evolves, I'm currently deeply involved in finishing up my master's thesis.