Page MenuHomePhabricator

[SPIKE] What percentage of talk page edits are made with DiscussionTools?
Open, MediumPublic

Description

To understand how widely the Reply and New Discussion Tools are being used, we would like to use this task to learn what percentage of all talk page edits do people use DiscussionTools to make.

Timing

We would like to do this analysis ~3 months after the Reply Tool and New Discussion Tools are made available as on-by-default features on desktop and mobile.

Requirements

Note: the requirements below likely can be met with data we can already access/visualize within Superset. E.g. https://superset.wikimedia.org/r/838.

A chart showing the percentage of all talk page edits made within an arbitrary timeframe each editing interfaces is "responsible" for that can be filtered by user experience level (read: edit count), project, platform, and revert status.

Event Timeline

I spent a little bit of time on this since I was curious myself.

I couldn't do it in Superset, since you apparently need to write queries for the "edits_hourly" schema in a weird SQL dialect and I can't even figure out what it's called, much less how to work it. So I just wrote some queries in Quarry, operating on the public database replicas rather than analytics data (although the analytics data would make it easier to query across all wikis, my query is just for any individual wiki).

Also, I'm checking what percentage of comments are made with DiscussionTools, which is not exactly the same thing as talk page edits, but I think it's what you really meant. (You can have talk page edits that are not comments and are probably not relevant here; and you can have comments on non-talk pages. The query relies on the 'discussiontools-added-comment' tag to identify them.)

Here is the query: https://quarry.wmcloud.org/query/59985

Results for a few wikis (this covers October 2021):

  • frwiki: 2946/41891 = 7% (opt-out)
  • plwiki: 1439/9514 = 15% (opt-out)
  • hewiki: 4747/21125 = 22% (opt-out)
  • commonswiki: 613/47427 = 1.3% (opt-in)
  • enwiki: 12398/313344 = 4% (opt-in)
  • ruwiki: 322/43208 = 0.7% (opt-in)

[you can fork the query and change the database at the top to see results for other wikis]

The results also show how many comments are added using other tools (at least, the tools that add tags of their own) – for example:

  • enwiki: 56426 (18%) using Twinkle and 15068 (4.8%) using Huggle (mostly antivandalism stuff)
  • commonswiki: 3931 (8%) using VisualFileChange (mostly notifications about deletion discussions)
  • ruwiki: 7161 (22%) using Convenient Discussions

(Also, there's some noise caused by tags added by abuse filters, so all these exact numbers may actually be a little off. It would take some manual work for each wiki to filter them out.)

Updated results for a few wikis (this covers July 2022):

frwiki: 9658/36936 = 26% (opt-out)
plwiki: 1868/8247 = 23% (opt-out)
hewiki: 6750/20488 = 33% (opt-out)
commonswiki: 5341/38720 = 14% (opt-out)
enwiki: 67243/324454 = 21% (opt-out)
ruwiki: 1108/42511 = 2.6% (opt-in)

Updated results (December 2022):

frwiki: 9052/36062 = 25% (opt-out)
plwiki: 2987/11802 = 25% (opt-out)
hewiki: 4896/20554 = 24% (opt-out)
commonswiki: 6278/46380 = 14% (opt-out)
enwiki: 72854/322597 = 23% (opt-out)
ruwiki: 8025/45392 = 18% (opt-out)