An experiment: T113695: Clean the code review queue of analytics/wikistats. Developer-Advocacy could be in charge of filing a task like this one for the repository leading the classification of repositories with oldest open changesets.

EDIT: on a second thought, maybe not just the pole position, but the 10 slowest repositories. Different repos frequently affect different developers, and there is no reason not to ping #2 and #3 while #1 is being addressed.

There is a proposed WikiDev16 session about improving code review, especially for volunteers: T114419: Event on "Make code review not suck". You are welcome to comment there if interested.

It seems as though looking at queue size focuses our attention on "time to merge", whereas a more important metric for a lot of new contributors would be "time to first response". While both are obviously important, it seems that the most occurrences for long-term success of a contributor are:

• Getting prompt, friendly feedback
• Constructive coaching through the process of first merge

Any metrics we define in this area should measure artifacts of that healthy behavior. Obsessing about the size of the review queue encourages reviewers to dispose of bad code quickly. "Dispose" could mean "reject", or it could be "blindly merge"; either outcome is probably awful for new contributors and/or our codebase.

@Qgil and @Aklapper, do you all have metrics that emphasize the healthy behavior described?

Do reviewers have a way to know who is a new contributor? Finding a way to alerting people about that and/or to list recent patches by new contributors would be a good start.

Wikipedia invested a lot of effort into engaging and retaining newcomers; we could look at what work there and consider what would be the equivalent for new developers. E.g. how would mentors or a Teahouse look for MediaWiki?

(That said, while thinking about new developers is important, I don't think it is sufficient to fix our code review problem. Experienced developers are both hit worse (because they submit more complex patches which less people are able to review) and the productivity loss by failing to review their code is larger.)

Quoting myself from the WikiDev task:

It would be nice to run a survey asking volunteers (or everyone) about their experience as reviewers and reviewees, so that we have something more than opinions by the summit.

Do reviewers have a way to know who is a new contributor?

I use https://www.mediawiki.org/wiki/Gerrit/Reports/Open_changesets_by_newbie_owner

In T101686#1881699, @Nemo_bis wrote:

I use https://www.mediawiki.org/wiki/Gerrit/Reports/Open_changesets_by_newbie_owner

I wasn't aware of that, thanks!
Although if we focus on initial response times, some sort of "recent changesets by newbie owner" would be more helpful.

In T101686#1882793, @Tgr wrote:

In T101686#1881699, @Nemo_bis wrote:

I use https://www.mediawiki.org/wiki/Gerrit/Reports/Open_changesets_by_newbie_owner

I wasn't aware of that, thanks!
Although if we focus on initial response times, some sort of "recent changesets by newbie owner" would be more helpful.

Interesting! Do we have a mechanism to assign "newbie owners" for newbies? "Newbie owner" is funny wording, since I'm going to guess that many people might take just as much/more offense to implying that they are "owned" as they would being called a "newbie". In the name of bikeshedding the naming: "new Wikimedia developer" and "mentor"?

Although if we focus on initial response times, some sort of "recent changesets by newbie owner" would be more helpful.

I use the history/watchlist for that.

Robla, I don't understand what you are asking. "Owner" is simply the name gerrit uses (as opposed to author and committer). If you're asking about the procedure to act on the list, it's very simple: watchlist the page, check new stuff and review or add reviewers as needed. I've been doing this regularly since the report existed.

In T101686#1882814, @RobLa-WMF wrote:

Interesting! Do we have a mechanism to assign "newbie owners" for newbies? "Newbie owner" is funny wording, since I'm going to guess that many people might take just as much/more offense to implying that they are "owned" as they would being called a "newbie". In the name of bikeshedding the naming: "new Wikimedia developer" and "mentor"?

I think "newbie owner" means that newbies 'own' changesets, not that newbies are 'owned' by someone else. But a mentoring program would be useful indeed :)

Are we actively looking for and training potential future reviewers?

Are we actively looking for and training potential future reviewers?

AFAIK training happens in gerrit. As for actively looking for potential +2'ers, I do, using http://koti.kapsi.fi/~federico/crstats/

Preliminary results at https://www.mediawiki.org/wiki/User:AKlapper_%28WMF%29/Code_Review , to be fine-tuned in the next days but this task is pretty much done.

Closing after T114419 has happened.

The success criteria of this quarterly goal is:

Identify potential actions for WMF Engineering to address the code review queue of changesets submitted by volunteers (to propose and discuss with Eng teams, Team-Practices and volunteer maintainers in Q1/2016)

We need the status report for our imminent quarterly review.

I still have to go through the outcome of T114419 to finish this task (and same as for T114419: Define action items as subtasks if needed - likely to be the same ones).
Hence adding DevRel-January-2016 here.

After a post- T114419 conversation with @Bawolff I'm trying to lay out the next steps. @Bawolff: Please correct me if anything is wrong.

We should start off with:

• T129067: Document a structured, standardized approach for reviewing code contributions
• T129068: Improve code contribution guidelines for patch authors

Potential further steps (can also be done in parallel):

• T115852: Fix unclear maintenance responsibilities for some parts of MediaWiki core repository
• T128372: Document use of Owners in Phabricator and advertise it
• T128371: Set up Code Review office hours
• T128370: Update ownership list on [[mw:Developers/Maintainers]]

I've also dropped an email to TPG about the patch review culture of Wikimedia teams.