Gerrit Cleanup Day on Wednesday September 23, 2015
A full day committed to reviewing proposed code changes in Gerrit.
Result
Partially completed
All WMF developer teams using Gerrit participated, although with different degrees of engagement.
The queue of changesets without any review was reduced by 18% (total) and 25% (last 3 months).
BUT 752 changesets were still unreviewed (was 910), 314 from the past 3 months (was 406).
- We committed to the goal of 100% without a calculator; we are still happy about the 18%-24%.
- Lessons learned are on their way, but first impression is that the experience was positive and it helps sensibilizing our teams and our technical community about improving our code review practices.
Situation and aim
Health check survey results/FY2014-15 Q3 indicates an overall notable trend on "Persistent concerns about code review".
There are more than 1200 changesets waiting for review and the age of open changesets is increasing.
Cleaning up Gerrit alone will not solve this, but will help raising the awareness of this problem and help evaluating approaches to improve our code review practices.
Overall aim: Reduce queue length and median age of unreviewed changesets with a focus on volunteer contributions.
Involved parties
All WMF Engineering and Technology teams join the Day. @Aklapper from the #Engineering-Community Team is the main organizer of the Day. @MBinder_WMF helps with related Team-Practices. Release-Engineering-Team is available in #wikimedia-releng in case of problems with Jenkins/CI.
Your steps to follow
- Join the #wikimedia-dev channel on Freenode IRC for communication and assistance.
- Query Gerrit for unreviewed changesets in your code area, submitted in the last three months. Prioritize those changesets authored by volunteers and review them.
- See section below for example queries per team/area! Corrections welcome!
- Query Gerrit for any further unreviewed changesets in your code area. Prioritize those changesets authored by volunteers and review them.
- Example query for unreviewed MediaWiki core changesets, sorted by last update (replace the project name in Gerrit's search field by your project)
- Once your area is "clean", or if your area has less patches to review:
- Help reviewing unreviewed changesets in other areas (e.g. MediaWiki core patches without review in areas nobody might feel responsible for) if you feel comfortable with the programming language. Or MediaWiki extensions!
- Review/Update further older patches by anybody:
- Examine CR-1 changesets waiting for author's fixes or to be abandoned (Example query for MediaWiki Core)
- Examine CR+1 changesets waiting to be merged or re-evaluated (Example query for MediaWiki Core)
- Check the list of 100 oldest open Gerrit changesets without code review
- Ask on the #mediawiki-dev IRC channel which team needs help
- General:
- Mark WIP changesets by adding a "[WIP]" prefix
- Work-In-Progress changesets: Try to exclude patchsets with "[WIP]" in title.
- Merging and Quality: The goal is not to merge as many changesets as possible: We are not after reducing the quality of our codebase but after cleaning our backlog and give feedback to contributors. When merging changesets, mergers are supposed to test on Beta Cluster afterwards. Also, changesets without tests in areas where it is possible to write tests should not be merged.
- Dead code repositories: List unmaintained/deprecated/inactive code repositories in the Etherpad so these repositories can be marked afterwards (cf. T102920), to improve the correctness of our statistics.
Goals
- All open changesets submitted by new volunteers in the last three months have at least one review.
- (Extended goal) All patches submitted by volunteers (non-WMF/non-WMDE/etc) have at least one review.
- (Extended goal) Review/Update further older patches by anybody and examine CR-1 and CR+1 changesets.
Gerrit queries per team/area
Your corrections and additions of repositories to the queries are welcome! Please edit!
Gerrit queries for unreviewed changesets per team/area, submitted in the last three months:
Measurement of success / Evaluation
- 👎 All WMF developer teams join the Day.
- 👎 All patches contributed by volunteers and all patches from the past 3 months have at least one review.
- Compare number of patchsets in queue before and after. (T110947)
- Compare median age of patchsets in queue before and after. (T110947)
- Total number of changesets waiting for a reviewer action and Unknown affiliation: 628 (was 751 ~ -17%)
- Total number of changesets waiting for a reviewer action and Independent affiliation: 124 (was 159 ~ -23,1%)
- Total number of changesets waiting for a reviewer action, Unknown affiliation and open in the last three months: 261 (was 335 ~ -22%)
- Total number of changesets waiting for a reviewer action, Independent affiliation and open in the last three months: 53 (was 71 ~ -25,4%
(Data gathered at 00:00 UTC / 17:00 PST hence might be cut-off at the end)
See T113378: Lessons learned from the first Gerrit Cleanup Day for more information / discussion.
Afterwards / Related links
- Organize retrospective to evaluate the event, identify lessons learned in T113378, and decide if this experience was useful and what to do next.
- Nice-to-have: Identify repositories / teams championing code review, and the black holes (by having a log of all actions that day; and doing analysis after that day.)
- Mark inactive code repositories listed in the Etherpad as such, see T102920 for the process
- Relation with WMF Call to Action: STRENGTHEN: improve technology & execution / T98351: WMF to improve its process for community input and allocate dedicated technical resources to community requests.
- This task is part of ECT's T101100.
- For potential next steps, see proposals in T101099: Developer Relations Roadmap / T78768: Agree on and implement actions to prioritize code review of patches submitted by volunteers (T78768#1013963 was the origin of this idea) / T101686: Goal: Define potential actions to reduce code review queues and waiting times
- Future ideas: Reduce the queue of open changesets submitted by volunteers with +1 or no reviews.
- Future ideas: Reduce the median age of last upload in open changesets submitted by volunteers.