We have three analyses planned to evaluate the impact of the suite interventions we will be deploying to mobile web talk pages:
- T298058: [Impact Analysis] Evaluate Impact of Mobile Reply and New Discussion Tools
- T298065: [Impact Analysis] Evaluate Impact of mobile Topic Subscriptions + Usability Improvements
- T298062: [A/B Test] Run an A/B test to evaluate impact of mobile DiscussionTools
This task involves the work with defining what – if any – new instrumentation will need to be added to mobile web talk pages (both the MobileFrontend talk page overlay as well as the Read as wiki page view) in order to complete the analyses listed above.
Approaches
Potential approaches for instrumenting the mobile talk overlay and read as wiki page views in a way that enables us to compare how people use each.
- ❌Approach #1: Use EditAttempt to track how people use/engage with both talk page views
- This approach is not feasible. Reason: it is too complex and inconsistent with the purpose of EditAttempt to add page view data to it.
- Approach #2: Use MobileWebUIAction and EditAttemptStep schemas
- Considerations
- There are inherent disparities between the UX of the mobile frontend overlay and the read as wiki page view
- Considerations
- Approach #3: Join EditAttemptStep, Talk_Page_Edit, and MobileWebUIActions
Open question(s)
- 1. What – if any – new instrumentation will need to be added to mobile web talk pages in order to complete the analyses listed below?
- T298058
- T298065
- T298062
Instrumentation Spec
Done
- Answers to all ===Open question(s) are documented in this ticket
- Instrumentation spec is created that documents all of the events we will need in order to report on the metrics defined in the Talk Pages Project/Mobile/Measurement Plan