I believe we've covered this in our documentation on measuring product health-- please reopen if you believe it needs more.

Making it impossible to use the schemas for instrumentation outside of wikis where the Test Kitchen SDKs are not available.

Thank you for tagging this task with good first task for Wikimedia newcomers!

We now have validation that prevents creating instruments or experiments with names longer than 63 characters.

@Milimetric do you think we could close this in favour of the work that you and @tchin and others are doing wrt automated traffic?

Michael added: There exists https://alerts.wikimedia.org where you could add an alert with team=growth, which then also would be mirrored to slack for us. An existing alternative would be that we see analytics event validation data in Logstash.

Email or slack will work -- which is easiest to implement?

Maybe we should have a spike to look into overlap %s across all experiments in the past 90 days (or whenever we started populating other_assigned)?

I've asked a few of our partner groups in Slack:

Shared in slack

I've written a draft here if you could review!

Thank you for tagging this task with good first task for Wikimedia newcomers!

@cjming Any thoughts as to why I don't see the content on https://wikitech.wikimedia.org/wiki/Test_Kitchen/Conduct_an_experiment#Experiment_design:_non-cache_splitting_experiments?

We could use this to our advantage to nudge experiment owners (mainly product managers) towards that single-key-goal-metric practice.

Ok great - thanks! In that case, your team will want to follow steps for measuring product health

@ifried quick question: you mentioned before that you would want to instrument to track adoption and health outcomes, more of a "before and after" than a true experiment. Is that still the case, or will the team be running an experiment? The reason I ask is that I expect sample size to be quite limiting given the criteria. Happy to talk through this!

We have deployed this morning that new release I mentioned above. I would say the weird behaviour you described shouldn't be there. I have been testing a bit and I'm unable to replicate that issue right now on production. In fact, now Test Kitchen UI doesn't change the planned start date when the experiment is activated. Apart from other related things, we were fixing that behaviour with this new release

As we move away from TKUI for experiments, this is quite low priority.

Parent task was declined

Thank you for tagging this task with good first task for Wikimedia newcomers!

Closing -- this is low priority and an unused field is ok.

With the magnitude of instruments we have now, it's very easy to turn them all off individually, and we don't have different user permissions to separate actions. Given the capacity constraints and how long this has been listed at low priority, I'm closing this task to better reflect what the team is likely to be able to deliver.

We're moving away from TKUI, using it only for instruments moving forward, so I'm going to decline this. It still exists should we need to reopen, but our calculus has changed.

We're officially moving to GrowthBook, and SDS2.3 work will ensure that we can adopt phases. Closing this.

@Sfaci something to consider for SDS2.3 - Configure GrowthBook

As we switch from TKUI to GrowthBook for Experiments, I'll mark this as "not doing". TKUI will remain for instruments, but there will be many fewer entries to navigate.

Suggest hosting with our Decision Records

I've updated the task description to reflect discussion amongst the team.

Do we have a working list of what the platform needs? We will need valid config T422528: [GrowthBook Integration][SPIKE] Define a valid config, and ideally we want to enforce this in the GBUI and force privacy reminders around obligations and risk levels. (e.g. we implemented a way to show when configuration leads to medium- and high-risk data collection activities in TKUI. How might we do the same for GB?)

Since this is listed after TKUI removal T421814: [GrowthBook Integration] Remove stitching logic and transition to GrowthBook-only service, removing the AC to distinguish TKUI vs GrowthBook