Page MenuHomePhabricator

Developer Wishlist 2017 postmortem
Closed, ResolvedPublic

Description

According to Wikipedia, a project post-mortem is a process, usually performed at the conclusion of a project, to determine and analyze elements of the project that were successful or unsuccessful.

Let's collect feedback on the Developer Wishlist survey process and figure out what wen well and what could be done better.

Event Timeline

Tgr created this task.Feb 15 2017, 2:14 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 15 2017, 2:14 AM
Tgr added a comment.EditedFeb 28 2017, 9:48 PM

My thoughts:

  • With 76 proposals, 144 editors and 952 votes cast, I would call this a reasonably successful engagement project. (Of course the real success or failure will be whether the get the wishes done, but still.)
  • From the initial brainstorming to the conclusion of the vote it took about five weeks, and maybe half a dozen volunteers. I'd estimate the total time spent to be something like 40-60 man-hours (maybe 15-20 of which was staff time), a large part of which is not needed next time (e.g. adapting the AddMe gadget). That means the process is sustainable, even without WMF support.
  • In my impression, the weakest point was soliciting and curating the tasks:
  • We do not have clear communications practices (what channels are OK to use for this, when does it start getting spammy). As a consequence, the communications were sometimes ad hoc (e.g. a "voting closes in 24 hours" announcement was sent to many lists, but not wikitech-l).
  • We considered making at least the templates translatable but did not have enough time.
  • The vote button worked well (apart from an unconfirmed bug report). The endorsement button not so much: few people used it, and even fewer used it in a way that the placeholder message asked for (feedback from people who ran into the issue personally; possibly team/group endorsement). I was hoping to use endorsements to keep a balance between popularity and impact (ie. see how badly an issue affects developers); that did not work out.
  • Phabricator vs. MediaWiki was something we debated quite a bit during the course of the planning. The vote had to be on MediaWiki (or at least not on Phabricator, which does not have any useful support); having the descriptions on Phabricator meant we had to move them to the wiki and convert to wikitext, which had to be automated, which limited the options (e.g. we could not ask "add the description for the wishlist as a comment" because it would have been too hard to locate). Many people were reluctant to rewrite the original description (or just did not care enough) so we ended up with descriptions that were short and uninformative or not really focused on explaining to a wide audience what the task is even about.
  • We had some vague plans about collecting demographic information but that fell through due to lack of time, so it's hard to tell what groups the wishlist did/did not reach (although some information could be gleaned from user accounts). A survey with questions like how experienced you are with MediaWiki / where did you hear about the wishlist would be really useful next time.
Tgr added a comment.EditedFeb 28 2017, 9:55 PM

The CommTech wishlist published an outreach report which might be interesting to compare.

Tgr added a comment.Feb 28 2017, 10:05 PM

Overall, I think the main questions that should be revisited next time:

  • how can we ensure wide particiation? Or at least find a way to measure whether we are getting it?
  • how can we solicit better proposal descriptions?
  • how can we find out more about the impact of a proposal? (how to tell the difference between "first time I thought about this but it sounds like a good idea" and "the lack of this made my life hell" votes)
Qgil added a subscriber: Qgil.Mar 1 2017, 5:00 PM
Qgil moved this task from To triage to Team radar on the Developer-Advocacy board.
Tgr added a subscriber: MusikAnimal.Mar 1 2017, 7:26 PM

Might be worth documenting what tools were used for the wishlist:

Tgr added a comment.Mar 1 2017, 7:47 PM

(Filed T159365 and T159367 for making the tools reusable.)

Tgr closed this task as Resolved.Mar 1 2017, 7:48 PM
Tgr claimed this task.

I think this is done, unless someone else wants to add their perspective (which would be great! feel free to reopen).