This is a followup to {T88531} and is to be incorporated in {T115844}.
== Regarding the Cleanup Day itself ==
=== Communication before the Day ===
* ☑ Weekday: Chose Wednesday to not end up merging changes before [[ https://www.mediawiki.org/wiki/MediaWiki_1.27/Roadmap | branching / deployment train ]]
* Timing: End of quarter isn't great as teams are busy anyway. Also note that some teams do have high-priority time-sensitive tasks.
* Agreeing on a Cleanup date and getting commitment seems to be easier in a WMF Engineering management meeting instead of sending out emails
** TODO: Add a banner on Gerrit as [[ https://gerrit-review.googlesource.com/Documentation/config-themes.html#_html_header_footer | a theme with `GerritSiteHeader.html` ]]
** ☑ Edited Phabricator default dashboard to display announcement on homepage
** TODO: [[ https://meta.wikimedia.org/wiki/CentralNotice | Request a banner ]] on mediawiki.org
** TODO: [[ https://phabricator.wikimedia.org/calendar/event/create/ | Add an event to Phabricator calendar ]]
** ☑ Sent early email to [[ https://lists.wikimedia.org/pipermail/wikitech-l/2015-August/082968.html | wikitech-l ]], pywikibot, ops, [[ https://lists.wikimedia.org/pipermail/wikidata/2015-September/007009.html | wikidata ]] mailing lists
** ☑ Sent email to internal WMF Engineering list
** ☑ Added entry to internal WMF Engineering calendar
** ☑ A week before, contacted teams (mailing lists and team leaders if existing) whether they are ready or not help (e.g. [[ https://lists.wikimedia.org/pipermail/pywikibot/2015-September/009342.html | Pywikibot ]], [[ https://lists.wikimedia.org/pipermail/wikidata/2015-September/007163.html | Wikidata ]])
*** TODO: Some partially rather unused lists (e.g. performance@) were contacted as no better/clearer ones were found or existed
*** TODO: Must send earlier than one week: Ended up in some moderation queues without getting timely clearance; plus hard to find out names of some mailing list (e.g. fr-tech)
** ☑ Two days before, sent [[ https://lists.wikimedia.org/pipermail/wikitech-l/2015-September/083321.html | reminder ]] to wikitech-l
** TODO: More widely announce, e.g. tech-ambassadors@? Where is our extended target audience?
=== Social aspects ===
* Different levels of team preparation and enthusiasm. Examples: [[ https://www.mediawiki.org/wiki/Discovery_plans_for_gerrit_cleanup_day_2015 | Discovery team planning page ]]; Yuvi's internal ops@ email listing potential reviewers for patches; [[ https://phabricator.wikimedia.org/T113028 | RelEng preparations ]].
* Different understandings of expectation of joining among teams or individuals ("is it really mandatory to join?")
** "I did some reviews, but basically our team considers itself too busy to participate. More accurately: When it was discussed, it was left up to each developer to decide themselves whether they want to participate. I wish we would have done more."
* Basically no cross-team interaction seen in #wikimedia-dev (or it happened in other places?), everybody sticking to their areas.
** TODO: List of main contacts (Name, IRC nick, IRC channel) to also include timezone info?
** TODO: No formal buddy system was created - wikipage with section "I'm looking for a review-buddy"? Europeans (early timezone) with only few changesets: How can they help others and "find a buddy" in their timezone? How to improve involving volunteers, also those who realize on IRC that this is going on right now?
** TODO: Praise: Stats to highlight the most active/effective developers / repositories / teams during the Day. More efficient communication structure welcome. No clear numbers on participation of WMF teams and who exactly was (not) involved. Asking "How is the Cleanup Day going? How can we help?" in many IRC channels is not the most efficient way, checking with each main contact (timezones!) is neither; following the Gerrit notifications is cumbersome.
== General Code Review aspects / issues ==
* Created initial Gerrit queries per team/area in Phabricator task for the Cleanup Day; requested help for improvement from each team. Mixed results: Some teams came up with better queries or fixed their list of projects, no feedback from other teams. TODO: Improve queries (and responsibilities).
** Some teams mentioned that initial queries included deprecated projects
**DONE: Allow marking inactive repositories as such: T102920
* Potential "not our responsibility / NIH" attitude of individuals though repository is a related area
** unclear responsibility for cross-areas patches, e.g. touching i18n parts (L10N eng?) of extensions (extension maintainer?) like [[ https://gerrit.wikimedia.org/r/#/c/229078/ | 229078 ]]?
** Unclear responsibility for parts of MediaWiki core so people might avoid looking at its patches and rather stick to their "easier to find" areas?
** "Is there a list of //deployed but not really owned// repos that people could prefer looking at patches in?". Regarding MW Core: "I was hoping to find something less intimidating to point some newer teammates at"
* TODO: Stats to highlight the most active/effective developers / repositories / teams in korma.wmflabs.org.
* TODO: Have a policy to review contributions within x days?: {T113707}
* {T114419} (WikiDev16 talk)
* Technical restrictions in Gerrit:
** [[ https://phabricator.wikimedia.org/T63463 | `file:` queries in gerrit not enabled ]], hence not possible to search specific paths, e.g. of the MediaWiki core codebase
** When someone reviews a patch in Gerrit (e.g. CR-2) and then removes themself from the list of reviewers, that CR-2 is removed too. Numerous examples exist: [[ https://gerrit.wikimedia.org/r/#/c/190717/ | 190717 ]], [[ https://gerrit.wikimedia.org/r/#/c/120811/ | 120811 ]], [[ https://gerrit.wikimedia.org/r/#/c/175394/ | 175394 ]], [[ https://gerrit.wikimedia.org/r/#/c/175192/ | 175192 ]], [[ https://gerrit.wikimedia.org/r/#/c/171562/ | 171562 ]], etc.
* Misc:
** TODO: Document to also mark "controversial" changes as CR-2 ([[ https://gerrit.wikimedia.org/r/#/c/231316 | example ]])?
** TODO: Document counterproductive actions: Mass comments like "-1 needs manual rebase" are not helpful ("The -1 discourages other reviewers, without actually checking the substance of the patch. And Gerrit's 'Can't merge' indicator is often broken")
** TODO: Define a cut-off date for old tasks to abandon them (can be restored)?: {T113706}
*** As @Reedy wrote, "half tempted to abandon stuff pre 2014 that's untouched, that's sitting CR-1/V-1. 29 patches in mw core alone." - have some discussion/policy for that? See past discussions: http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/60150/ , http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/65931/ , http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/76459/ , http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/76640/
** TODO: Prepare some review comment template(s) for Gerrit, like "Hi $Username, sorry that your proposed code changes have not received a timely review. Insert reason for CR-1, e.g. manual rebase. Would you fancy updating your patch? See https://www.mediawiki.org/wiki/Gerrit/Tutorial#Amending_a_change_.28your_own_or_someone_else.27s.29 for more information."
* Technical threats to validity of numbers:
** Stats on Korma need improvement on user affiliation data: unreliable "Independent/Unknown" data after fixing T100189 (more people without affiliation than actually the case due to non-official email addresses used) but we did not go through the list of accounts afterwards. TODO: {T112527}
** {T108507}
== Evaluation ==
* ☑ We [[ https://phabricator.wikimedia.org/T110947 | gathered Korma data to compare ]] one day before and after Cleanup Day (though at 00:00 UTC / 17:00 PST hence might be cut-off at the end):
> * Total number of changesets waiting for a reviewer action and Unknown affiliation: 628 (was 751 ~ -17%)
> * Total number of changesets waiting for a reviewer action and Independent affiliation: 124 (was 159 ~ -23,1%)
>
> * Total number of changesets waiting for a reviewer action, Unknown affiliation and open in the last three months: 261 (was 335 ~ -22%)
> * Total number of changesets waiting for a reviewer action, Independent affiliation and open in the last three monthts: 53 (was 71 ~ -25,4%
Additional statistics show a 10 % decrease in open patches:
> * ~200 changesets closed, half of which by abandoning. https://www.mediawiki.org/w/index.php?title=Gerrit/Reports/Changesets_by_status&diff=prev&oldid=1897960
> * ~100 changesets closed the following day, also half of which by abandoning (in good part thanks to Hashar&Siebrand cleaning up core and i18n respectively). https://www.mediawiki.org/w/index.php?title=Gerrit/Reports/Changesets_by_status&diff=next&oldid=1897960
Regarding the measurements of success, we have failed:
* 👎 All WMF developer teams join the Day.
* 👎 All patches contributed by volunteers and all patches from the past 3 months have at least one review.