I'm imagining everything from "Define the A/B test" through "schedule deployment of the config change to enable the test" through "Sanity-check data after 24 hours" through "Publish final report". Importantly it would also have things like "Create phab tickets for upcoming tasks X, Y, and Z". There are probably a dozen more steps in there. Presumably this should be on wiki, and would serve as a nice timeline-based checklist for what we need to do.
Description
Description
Event Timeline
Comment Actions
Good idea. Some of this documentation will help reduce bus factor, other bits of it will just be more mechanical.
Comment Actions
It could also help other departments as they ramp up their testing. And of course we could learn from them as well.