Page MenuHomePhabricator

[Analytics] [Request] Baseline data for Item deletions
Closed, ResolvedPublic

Description

WMDE Analytics Request

This task was generated using the WMDE Analytics request form. Please use the task templates linked on our project page to create tasks for the team. Thank you!

Why (Context & Decision)

What problem are you trying to solve, and what decision will this analysis inform? Briefly explain the organisational or strategic context, why this matters now, and what action will be taken based on the outcome.

We need to better understand trends in the creation of non-notable Items to support admins and patrollers more. This is important for our KR work with users with extended rights and the work on combating promotional editing (editing to ensure edits are featuring highly for SEO / availability for Gen AI - which is unsustainable).

Additional context for this task: it relates to the wider Data Governance support WMDE wants to provide. Specifically supporting notability guidelines. https://docs.google.com/document/d/1XksiQ_8UUbo9g4vJfiHCpC07LzOwzfjSAB3IPy9jdcg/edit?tab=t.0#heading=h.cwk2ngiagshf && https://docs.google.com/spreadsheets/d/1_SWgeoRQAbJQCiQ-IuYeZH26u7nbVCzz8WVgbr3Qfds/edit?gid=0#gid=0

What (Scope & Output)

Describe the specific question(s), metrics, segments, or deliverables (e.g., dashboard, deep dive, experiment analysis), including any relevant definitions or constraints.

Questions:
For a given month...

  • How many new Items were created in that time period?
  • How many new Items were created by new accounts/accounts with few edits/... (Do we have a good standard criteria here?) in that time period?
  • How many of the new Items created that month have already been deleted since then?
  • How many of the new Items created by new accounts/.../... have already been deleted since then?
  • Output: dashboard with a graph of the above metrics over time with absolute and relative numbers

Example output as table:

monthItems created (total)Items created by new accountsItems deletedItems created by new accounts deletedPercentage of Items that were deletedPercentage of Items created by new accounts that were deleted
Jan 202610050252550100

Note: Item creations by IP should probably be treated as if they were coming from a registered account for this.
Note (i): New accounts we will define as <50 edits. We discussed also to potentially break this edit count down further / create edit groupings.
Note (ii): We ideally want a lookback / data showing 2025-2026. Please be aware though that the Google AI summaries were ramped up March 2025, so we might (if possible) also extend back to 2024...

By When (Timing & Priority)

Provide a clear deadline, any key milestones (e.g. launch, leadership review), and note if timing is flexible or fixed.

ASAP but no later than 2026-07-15


Information below this point is filled out by the analyst.

Sub Tasks

A breakdown of the steps to complete this task.

  • Ticket refinement / check with requester.
  • Table / source data check: wmf.mediawiki_history - any others?
  • Query prepared.
  • Query run... Will not be a DAG, so check best way to build the table / or just run in Superset?
  • Dashboard / output prepared.
  • Discussion with Lydia about the output / data.

Estimation

Estimate:
Actual:

Data

The tables that will be referenced in this task and the sample sizes from them that will be used.

  • wmf.mediawiki_history

Event Timeline

Hey @Lydia_Pintscher - have a few clarifiying questions as I've started to look at the data today!

  1. Just want to be 110% sure this is only for items, so just Q IDs?
  2. Do we need to account for whether the user is classified as temporary or not? Was having a look to check my user count numbers, and found this so wanted to check!
  3. Do we also need to ensure that we exclude bots? Again in the above I've seen this be done in previous work, so just would like to check.

Hey @Lydia_Pintscher - have a few clarifiying questions as I've started to look at the data today!

  1. Just want to be 110% sure this is only for items, so just Q IDs?

Yes correct.

  1. Do we need to account for whether the user is classified as temporary or not? Was having a look to check my user count numbers, and found this so wanted to check!

Hmm that would actually be useful. Yeah.

  1. Do we also need to ensure that we exclude bots? Again in the above I've seen this be done in previous work, so just would like to check.

For this I'd not exclude them.

Hey @Lydia_Pintscher - just sharing with you here the outcome of my testing (last 2 months). I don't have a baseline idea of item creation numbers (couldn't find any dashboards on Superset), so just wanted to check with you the levels make sense? https://docs.google.com/spreadsheets/d/1vYQUKj0Crl27S5Jc8uLDf1PiL6n-t13-VXGszMpfUkk/edit?gid=0#gid=0

Hey @Lydia_Pintscher - just sharing with you here the outcome of my testing (last 2 months). I don't have a baseline idea of item creation numbers (couldn't find any dashboards on Superset), so just wanted to check with you the levels make sense? https://docs.google.com/spreadsheets/d/1vYQUKj0Crl27S5Jc8uLDf1PiL6n-t13-VXGszMpfUkk/edit?gid=0#gid=0

Looked over it and left a comment.

Potentially also interesting for cross-checking: https://www.wikidata.org/wiki/User:Bovlb/unconfirmed_users_with_10_creations

Hi @Lydia_Pintscher - we have finalised the dashboard for this work. Please see it here in Superset.

I hope everything is clear, but do let me know if anything needs updating!

@catherine.kelsey.wmde I get this error when opening the superset link:

image.png (835×868 px, 52 KB)

Any idea what's wrong?

Hi @Lydia_Pintscher - mega apologies! Andrew just told me I had not published it (sorry is a new tool for me, so didn't see this step). Here is the link again. So hoepfully should be available now!

Perfect! Thank you.
Do we know if the picture materially changes if we lower the threshold for what's considered a new account to say 20 edits?

Hey @Lydia_Pintscher - thanks for taking a look :) - we could re-run / add with this condition if you like?

Hey @Lydia_Pintscher - this is updated now. Have added an additional view on all charts to show "new accounts" (defined as <20 edits, and then "new accounts" defined as between 20-50 edits. Hope this is helpful!

Quite interestingly (likely as expected) - the accounts with <20 edits are seeing a large number of deletions, and a slightly higher proportion of deletions than accounts with 20-50 edits

Thank you!
Published on-wiki in the notability policy RfC.

Small feedback from the editors: The colors of the graphs are not distinct enough to be easily readable. We should take that into account for future graphs.

Thanks @Lydia_Pintscher - that's useful feedback, I've updated the colour scheme! It's not fully customisable as I would like, but hopefully is a bit clearer now :)