@ifried quick question: you mentioned before that you would want to instrument to track adoption and health outcomes, more of a "before and after" than a true experiment. Is that still the case, or will the team be running an experiment? The reason I ask is that I expect sample size to be quite limiting given the criteria. Happy to talk through this!

@JVanderhoop-WMF: Hey there! Yes, we're interested in click-through-rates and health outcomes, rather than running an A/B test. We have a few reasons why, but the main reason that we don't want to do an A/B test for this work, most likely, is that we just don't think we would learn very much from it (for this particular set of features). We are already pretty confident that any control group would *not* be able to find the events we promote. So, it would be extra set-up for minimal learning benefits.

As for the sample size, yes, our experiment A will likely have a very small sample size. However, we're now considering launching an experiment B, which could have a much larger sample size (since it could conceivably target any newcomer or junior editor who makes a reasonably substantial edit to an article on any wiki). You can see some of our ideas for experiment B in this ticket. The sample size of experiment B could be so large that we may want to limit to certain pilot wikis. We're still finalizing exactly what will be experiment B (it may change), but the fundamental idea is that we reach out to a lot of junior/newcomer editors if they make edits in an article that is merely within the same topical area of the event. There are only a few topics within LiftWing, so the likelihood of promoting an event to them is much higher than if there is a worklist match.

Ok great - thanks! In that case, your team will want to follow steps for measuring product health

With this, you won't be running an experiment, but you will be using Test Kitchen. (we help with both!)

Yes, @JVanderhoop-WMF, thanks for the clarification and link to documentation. Not an experiment, but running Test Kitchen. Much appreciated!

@cmelo, can you add some technical details to the task for how we set this up? Thank you!