Page MenuHomePhabricator

[Impact Analysis] Evaluate Impact of mobile Topic Subscriptions + Usability Improvements
Closed, DeclinedPublic

Description

T298055 will introduce the Topic Subscriptions and the set of Usability Improvements to mobile talk pages at an initial set of wikis.

This task represents the work with conducting an impact analysis to evaluate what impact introducing Topic Subscriptions and Usability Improvements to mobile talk pages led to:

  1. People being more likely to publish disruptive edits using mobile talk pages
  2. People being less likely to publish new talk page comments or discussions using mobile talk pagse
  3. People being less likely to recognize talk pages as spaces to communicate with others

Decision to be made

This impact analysis will help us make the following decision: Do any adjustment need to be made to Topic Subscriptions and the set of mobile Usability Improvements before an A/B test can be run of the set of mobile DiscussionTools features (T298062)?

Hypotheses

IDHypothesisMetric(s) for evaluation

Decision Matrix

IDScenarioPlan of action

Analysis Scope + Timing

  • The data analyzed as part of this ticket should be limited to the two weeks before and after T298222 is resolved.
  • This analysis should happen >= 4 weeks after T298058 is resolved

Related Objects

Event Timeline

Per the conversation @MNeisler and I had offline today, we're going decline work on this task and instead evaluate the impact the set of mobile talk page Usability Improvements are having through an A/B test.

The work to run and analyze the results of this "A/B test" will happen in T298062.

We made the decision to forego this analysis because:

  1. We do not have enough time to complete all of the analyses we have planned and were therefore looking for analyses to de-scope and
  2. We do not expect to see changes in the metrics this task was "asking" us to report on that would be of the magnitude that we could even detect them through an impact analysis