Page MenuHomePhabricator

[MILESTONE] Run experiment to evaluate impact of changing Tone Check model's confidence threshold
Open, Needs TriagePublic

Description

WARNING: prioritizing work on this task cannot happen until T387918 is completed. Reason: this analysis will confirm the extent to which the findings from T394463 prove to be statistically significant.

This ticket holds the work of running a follow-on experiment to T387918 wherein we will learn how different Tone Check model confidence thresholds impact editing behavior.

Background

The idea/need for this experiment emerged in response to the following:

  • In T394463 we learned that Tone Check appears impactful and targeted
  • In T394463#11282441, we decided not to explore this adjustment in the A/B test that's ongoing