Page MenuHomePhabricator

[SPIKE] What is the makeup of people who have adjusted the state of the Discussion tools beta feature?
Closed, ResolvedPublic

Description

In T291307#7567905, we assumed the majority of people who have explicitly adjusted the state of the Discussion tools beta feature within Special:Preferences#mw-prefsection-betafeatures are Senior Contributors.

This task involves the work with verifying this assumption so that we can decide whether to continue moving forward with excluding people from the A/B test who have explicitly adjusted the state of the Discussion tools beta feature within Special:Preferences#mw-prefsection-betafeatures.

More context in T291307#7567905.

Open question(s)

Of the people who have explicitly adjusted the state of the Discussion tools beta feature within Special:Preferences#mw-prefsection-betafeatures, what percentage of these people are Junior Contributors and what percentage of these people are Senior Contributors?

  • "Explicitly adjusted" in this context means that someone will have visited Special:Preferences#mw-prefsection-betafeatures > checked or unchecked the checkbox that appears next to the Discussion tools beta feature, and clicked Save.
  • Timeframe: we are interested in people who have adjusted the state of the Discussion tools beta feature at any point after 18 February 2021 (the date the New Discussion Tool was first introduced i).

Done

  • Answers to all ===Open question(s) are answered and documented in this ticket

i. https://www.mediawiki.org/wiki/Talk_pages_project/New_discussion#18_February_2021

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

@ppelberg Initial findings indicate that of the talk page contributors that have explicitly adjusted the DiscussionTools Beta Preference since deployed, 51% of these contributors are Junior Contributors (defined as making under cumulative 100 edits). See summary below.

Result summary:

  • There are a total of 290,842 users (not limited to just contributors) that have explicitly adjusted the discussion tools beta feature across all wikis.
  • Of all talk page contributors[1], around 17.6% of these contributors have explicitly adjusted the DiscussionTools Beta Preference. Here is the breakdown of those contributors by experience level

Talk Page Contributors[1] that have explicitly adjusted the DiscussionTools Beta Preference

Edit Count Group# of ContributorsPercent of Contributors
under 1003262451.59%
100-500680010.75%
over 5003262437.66%

Data Details and Caveats:

  • Data on the state of the Discussion Tools Beta preferences obtained via:

Data via:

query='''
SELECT COUNT(*) 
FROM user_properties 
WHERE up_property="discussiontools-betaenable"
'''
  • Assumes that the discussiontools-betaenable property has no default value, so if it's present at all the user has manually toggled it.
  • I joined the user_properties data with edit history logged in the latest snapshot of mediawiki_history to determine the proportion of these users that are senior and junior contributors. For this analysis I specifically, looked at non-bot editor of talk pages.

Timeframe: we are interested in people who have adjusted the state of the Discussion tools beta feature at any point after 18 February 2021 (the date the New Discussion Tool was first introduced

The user_properties table only provides a current snapshot of the state of non-default user preferences and we do not have data in this table on the specific time the preference was changed or updated.

Notebook


[1] Defined as a contributor that had published at least 1 non-reverted edit to a talk page since 18 Feb 2021 (deployment of the New Discussion Tool. Bots excluded.