Analyse Nuke usage across Wikimedia projects
Closed, ResolvedPublicSpike
Actions

Description

Nuke can be used on both registered and unregistered users. Its behaviour for unregistered users may need to change. To help us make a decision we want to better understand how often it is used on unregistered users.

Our qualitative research so far has focused primarily on English Wikipedia editors, where unregistered Nuking is not common as they can't create main namespace pages. We'd like to know if there are other communities where we should target our research.

Questions

What percentage of Nuke actions are currently taken on unregistered vs registered users?
Are there any Wikimedia projects with a particularly high rate of unregistered Nuke deletions?

Related Objects
Search...

Status	Subtype	Assigned	Task
			Restricted Task
Resolved		kostajh	T294511 2021 Security Team wikireplicas audit
Declined		None	T284948 Raw IPs of logged-out users disclosed in wiki-replicas
In Progress		Niharika	T324492 Temporary accounts - MVP
Open		None	T326816 [Epic] Update features for temporary accounts
Open		None	T339244 Update Moderator Tools-owned products that may be affected by IP Masking
Resolved		kostajh	T326935 Prepare Nuke extension for IP Masking
Resolved	Spike	Samwalton9-WMF	T340631 Does Nuke need substantial feature changes to retain current functionality with IP Masking?
Resolved	Spike	KCVelaga_WMF	T341564 Analyse Nuke usage across Wikimedia projects

Event Timeline

Samwalton9-WMF created this task.Jul 11 2023, 10:34 AM

Samwalton9-WMF updated the task description. (Show Details)

Samwalton9-WMF added a project: Moderator-Tools-Team (Kanban).Jul 13 2023, 2:16 PM

KCVelaga_WMF claimed this task.Jul 13 2023, 2:21 PM

KCVelaga_WMF added a project: Product-Analytics (Kanban).

KCVelaga_WMF triaged this task as Medium priority.Jul 14 2023, 10:35 AM

KCVelaga_WMF changed the task status from Open to In Progress.Jul 17 2023, 7:12 AM

KCVelaga_WMF moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.

KCVelaga_WMF moved this task from Ready to In Progress on the Moderator-Tools-Team (Kanban) board.Jul 17 2023, 1:57 PM

@Samwalton9 Sorry, this took a bit longer than excepted. Although I gathered all-time logs, given the scope of the task i.e. to analyse nuke usage to inform changes required due to IP masking, only recent records (last 3 years) were considered for the analysis, which included ~240000+ nuke actions across wikis. My reasoning is that if there has been no usage of the feature at all during the last three years, it won't be helpful to identify wikis to talk to, and also 3 years is a reasonable amount of time to understand the distributions.

Here are the findings:

Across wikis, a considerable amount of nuke actions (~33%) are taken on pages created by unregistered users (note: this includes pages in both content and non-content namespaces).

Among the bigger wikis (according the overall size rank from wiki comparison data), here are the top 10 wikis having more than 25% of nuke actions against unregistered users (with at least 300 actions in total during the last three years)

database code	IP	user	total	IP_percentage	user_percentage	overall size rank
jawiki	5763	9107	14870	38.76	61.24	3
dewiki	679	782	1461	46.48	53.52	4
ruwiki	925	2469	3394	27.25	72.75	6
itwiki	1060	1664	2724	38.91	61.09	9
wikidatawiki	9261	17325	26586	34.83	65.17	11
nlwiki	361	162	523	69.02	30.98	16
trwiki	793	706	1499	52.9	47.1	17
ukwiki	410	515	925	44.32	55.68	18
enwiktionary	1996	2144	4140	48.21	51.79	19
kowiki	7580	2545	10125	74.86	25.14	20

Overall, the following are the top 20 wikis with the highest percentage of nuke actions against unregistered users (with least 300 actions in total during the last three years)

database code	IP	user	total	IP_percentage	user_percentage	overall size rank
thwikiquote	419	0	419	100.0	0.0	442
tnwiki	471	2	473	99.58	0.42	420
gotwiki	447	4	451	99.11	0.89	580
stwiki	668	7	675	98.96	1.04	474
satwiki	1113	25	1138	97.8	2.2	348
sowiki	617	52	669	92.23	7.77	169
fiwiktionary	931	97	1028	90.56	9.44	120
ltwiki	335	38	373	89.81	10.19	50
jawikiversity	531	67	598	88.8	11.2	541
jawikibooks	2663	362	3025	88.03	11.97	95
scowiki	313	50	363	86.23	13.77	135
aswiki	515	104	619	83.2	16.8	121
huwiki	1446	394	1840	78.59	21.41	26
jawiktionary	3202	882	4084	78.4	21.6	85
bgwiki	358	101	459	78.0	22.0	38
enwikivoyage	432	130	562	76.87	23.13	55
fiwiki	730	221	951	76.76	23.24	28
kowiki	7580	2545	10125	74.86	25.14	20
jawikisource	530	209	739	71.72	28.28	177
simplewiki	3758	1513	5271	71.3	28.7	31

I don't have very direct answers to the questions mentioned in the task description, but I believe this data should be helpful for you to answer them.

If you'd like the data for all wikis, I have published the file on Github here
For the approach, the code and the analysis, the notebook is here

KCVelaga_WMF moved this task from Doing to Needs Review on the Product-Analytics (Kanban) board.Jul 24 2023, 12:15 PM

KCVelaga_WMF moved this task from Needs Review to [Deprecated] Done (previously: Needs sign-off) on the Product-Analytics (Kanban) board.Jul 24 2023, 12:19 PM

Thanks @KCVelaga_WMF, this is great.

It looks like the parent task is still worth considering, since usage of Nuke on IP addresses is relatively common.

Additionally, it seems that the following wikis would be good candidates for followup investigation, selecting the ones with a good mix of size and Nuke usage: