Quick Analysis: impact of adding Thanks to more pages
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	KStoller-WMF
	Fri, Mar 17, 7:54 PM

Description

User story & summary:

As the Growth team PM, I want to understand the impact T51541 and T90404, because that will ensure I make better, data-driven prioritization decisions (especially in relation to the Positive Reinforcement project)

As a Wikimedian, I'm curious about the impact of features like Thanks, because I care about the long-term health of the wikis.

Background & research:

This task is important because understanding the impact will help us make more informed decisions in the future.

This task covers just the initial quick impact analysis of:

Hypothesis:

This research should help support or reject following hypothesis:

Adding Thanks to more pages on Wikipedia will increase the percentage of Edits that get thanked.

Acceptance Criteria:

Quick analysis with a short summary of methodology and results posted in this task.

Note: since we didn't set up A/B testing for these changes we can only look at before and after data to measure the impact. Both Thanks tasks (T51541 and T90404) were on the 2023-01-02 release train, so should have been added to Group 2 wikis on 2023-01-05.

Related Objects

Mentioned In: T330379: Set up Herald rule to automatically answer some common newcomer questions
Mentioned Here: T51541: Add "thanks" links to Special:Watchlist and Special:Contributions
T90404: Add Thank link on Recent Changes pages

Event Timeline

KStoller-WMF created this task.Fri, Mar 17, 7:54 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFri, Mar 17, 7:55 PM

KStoller-WMF updated the task description. (Show Details)Fri, Mar 17, 8:01 PM

@Jdlrobson & @NRodriguez - pinging you in case you are interested in following this task.

KStoller-WMF triaged this task as High priority.Fri, Mar 17, 8:04 PM

KStoller-WMF moved this task from Incoming to Ready for Development on the Growth-Team (Current Sprint) board.

Adding this to the Product Analytics kanban board, keeping the High priority. The plan is to start this work on March 20 to get an initial perspective on the usage of Thanks as it relates to edits made in the most recent months, then go on from there based on what our needs are.

Restricted Application added a comment. · View Herald TranscriptSat, Mar 18, 12:27 AM

This comment was removed by Aklapper.

nettrom_WMF mentioned this in T330379: Set up Herald rule to automatically answer some common newcomer questions.Sat, Mar 18, 12:52 AM

KStoller-WMF moved this task from Ready for Development to Top Product Priorities on the Growth-Team (Current Sprint) board.Mon, Mar 20, 2:46 AM

nettrom_WMF moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.Mon, Mar 20, 5:15 PM

nettrom_WMF moved this task from Top Product Priorities to In Progress on the Growth-Team (Current Sprint) board.

This analysis has been completed. We created a slide deck presenting the findings and decided to upload that to Commons for wider accessibility: https://commons.wikimedia.org/wiki/File:T332441_-_Thanks_Usage_-_Introduction.pdf The notebooks used for data gathering and visualization are in this GitHub repository

Our quick analysis of edits being thanked does not find a clear signal that the wider availability of the feature resulted in more edits being thanked. Patterns of usage are different between wikis, both in the sense of what proportion of edits are thanked as well as whether we see differences in January and February compared to December. There are also differences when it comes to thanks given by editor tenure.

Quick Analysis: impact of adding Thanks to more pagesClosed, ResolvedPublicActions