Page MenuHomePhabricator

Quick Analysis: impact of adding Thanks to more pages
Closed, ResolvedPublic


User story & summary:

As the Growth team PM, I want to understand the impact T51541 and T90404, because that will ensure I make better, data-driven prioritization decisions (especially in relation to the Positive Reinforcement project)

As a Wikimedian, I'm curious about the impact of features like Thanks, because I care about the long-term health of the wikis.

Background & research:

This task is important because understanding the impact will help us make more informed decisions in the future.

This task covers just the initial quick impact analysis of:


This research should help support or reject following hypothesis:

  • Adding Thanks to more pages on Wikipedia will increase the percentage of Edits that get thanked.
Acceptance Criteria:

Quick analysis with a short summary of methodology and results posted in this task.

Note: since we didn't set up A/B testing for these changes we can only look at before and after data to measure the impact. Both Thanks tasks (T51541 and T90404) were on the 2023-01-02 release train, so should have been added to Group 2 wikis on 2023-01-05.

Event Timeline

KStoller-WMF added subscribers: NRodriguez, Jdlrobson.

@Jdlrobson & @NRodriguez - pinging you in case you are interested in following this task.

Adding this to the Product Analytics kanban board, keeping the High priority. The plan is to start this work on March 20 to get an initial perspective on the usage of Thanks as it relates to edits made in the most recent months, then go on from there based on what our needs are.

This comment was removed by Aklapper.

This analysis has been completed. We created a slide deck presenting the findings and decided to upload that to Commons for wider accessibility: The notebooks used for data gathering and visualization are in this GitHub repository

Our quick analysis of edits being thanked does not find a clear signal that the wider availability of the feature resulted in more edits being thanked. Patterns of usage are different between wikis, both in the sense of what proportion of edits are thanked as well as whether we see differences in January and February compared to December. There are also differences when it comes to thanks given by editor tenure.