Write postmortem for the portal A/B test problems
Closed, ResolvedPublic1 Story Points


It would be good to write up a postmortem of the problems that happened with the recent A/B test on the Wikipedia portal, so that:

  1. we have a clear, overall description of the problems in a single public place we can point to (e.g. for the quarterly review)
  2. we're clear on exactly what caused the problem
  3. we have documented steps to prevent the same problem from occurring again

Much of this is actually already done, but it's in varying different places, so part of this task will simply involve pulling in that text.

Examples of similar incident documentation are here: https://wikitech.wikimedia.org/wiki/Incident_documentation

I don't know whether to suggest that this goes on wikitech or on mediawiki. Wikitech fits better, but mediawiki is where most of our stuff is. Fortunately, it's super easy to move if someone complains, so let's just go with putting it on mediawiki.org.

Deskana created this task.Dec 21 2015, 8:24 PM
Deskana updated the task description. (Show Details)
Deskana raised the priority of this task from to High.
Deskana added a subscriber: Deskana.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 21 2015, 8:24 PM
Deskana updated the task description. (Show Details)Dec 21 2015, 8:26 PM
Deskana set Security to None.
Deskana moved this task from Needs triage to On Sprint Board on the Discovery board.
Deskana updated the task description. (Show Details)Dec 21 2015, 9:08 PM
Ironholds edited a custom field.Dec 21 2015, 9:55 PM
Ironholds moved this task from Backlog to Needs review on the Discovery-Analysis (Current work) board.
Ironholds claimed this task.
Deskana closed this task as Resolved.Dec 22 2015, 9:11 PM

This was the output (not actually on mediawiki.org, but it's fine in an email): https://lists.wikimedia.org/pipermail/discovery/2015-December/000621.html