Overview
We are interested in categorizing different types of /reasons for deletions of uploaded media files (how: based on analysis of a sample of filed deletion requests). Once we understand the main reasons, and a rough proportion of deletion types, we can identify most problematic ones and prioritize improvements focused on minimizing their in-flow.
This is part of Design research on Commons. We would first do a programmatic analysis and then ask the design research for qualitative analysis on top.
USeful infformation about the baselines for uploads and some deletion request ratios can be found in comments here https://phabricator.wikimedia.org/T337466
Requirements
Step 1: Preliminary analysis
- Which data can we get about a deletion request? Before proceeding to the sampling and analyses, send an example with all data we can get to Sneha and Alexandra for review and discussion about which data to include in the analysis
Step 2: Analysis a sample
Retrieve a random sample of 1000 deletion requests over the last year and try to categorise based on the following parameters:
- Type of deletion request (speedy or regular)
- Time to resolve (less than 1 week, 1 week to 1 month, 1 month to 3 months, 3 months+, haven't been resolved)
- Reasons - see reasons in this write-up. Implementation note: Reasons for deletion requests should have tags, so can probably use those
Questions we want to answer:
Share/% of each deletion class
What are the reasons most commonly reported within in each class
Is there any correlation between e.g. time to close and specific reasons?
Step 3: We would like to ensure that the analysis is representative and not biased to the latest 1000 deletion requests. As such, we would like to run the same analysis for several historical samples to minimize bias.
Preliminary analysis
Here's a sample of 100 Commons pages that got deleted between 2022-05-01 and 2023-06-01 by non-bot users, CC @AUgolnikova-WMF, @Sneha.
The deletion event edit message (comment_text field) seems like a relevant piece of information that enables the analysis of coarse-grained deletion types/classes and fine-grained reasons.
Deleted pages dataset
- interval: 13 months
- start date: 2022-05-01
- end date: 2023-06-01
- total rows: 1.3 M (1,285,839)
- total deleted revisions: 1.3 M (1,278,527)
- total deleted pages (counted via page ID): 497 k (497,106)
- total deleted pages (counted via page title): 489 k (488,890)
- total distinct deletion edit messages: 154 k (154,017)
- data lake query:
- most frequent (> 1000 times) edit messages:
Deletion requests dataset
- input: deletion requests archive
- interval: 13 months
- start date: 2022-05-01
- end date: 2023-06-01
- total requests closed with a deletion: 68 k (68,071)
First sample analysis
- input: deletion requests dataset as above
- speedy deletion threshold: 7 days
- % of each deletion class:
- 38 % speedy (379)
- 62 % regular (621), of which:
- 62 % (384) 1 week to 1 month
- 23 % (141) 1 to 3 months
- 15 % (96) 3+ months
- most commonly reported reasons:
- the top speedy reasons seem related to the project scope, a very broad topic that encompasses more specific reasons
- the top regular reasons seem related to copyright violation, which can break down into more specific ones, typically freedom of panorama in this case
- correlation between time to close and reasons: TODO
Speedy deletion requests
Dataset at https://docs.google.com/spreadsheets/d/1aajH1XI4Gd5HPjOTDBV3j6hYmJGQIJsz3zaGUqAngew/edit?usp=sharing
Regular deletion requests
Dataset at https://docs.google.com/spreadsheets/d/1BT7oFNUHPFrgr65Wo6ZHrcYYqHTL49fkBm6plnIVhNw/edit?usp=sharing
Analysis scale up
- input: deletion requests dataset merged with deleted pages dataset
- total requests: 53 k (53,021)
- resolution time buckets:
- up to 1 week - 38 % (20,242)
- 1 week to 1 month - 37 % (19,777)
- 1 to 3 months - 15 % (7,936)
- 3+ months - 10 % (5,066)
- top 10 wikilinks shared by all buckets:
- COM:DW - derivative works
- COM:FOP - freedom of panorama
- COM:SCOPE - project scope
- COM:VRT - volunteer response team
- top 10 wikilinks unique to each buckets:
- COM:NOTHOST - Commons is not a free Web host
- none
- COM:TOO UK - United Kingdom's threshold of originality
- COM:PCP - precautionary principle
- top 10 words shared by all buckets:
- copyright
- uploader - typically related to either not own work or mistaken uploads
- top 10 words unique to each bucket:
- educational, logo, personal, quality, uploaded
- possible
- free, see
- author, de, initially, tagged
Up to 1 week
1 week to 1 month
1 to 3 months
3+ months
Top reasons taxonomy
- copyright violation
- derivative work
- freedom of panorama
- by country
- threshold of originality
- logo
- Google maps
- album cover
- screenshot
- poster
- banner
- book
- not own work
- non-free license
- inquiry to volunteer response team
- not suitable for work
- not educational
- nudity
- penis
- not a free Web host
- personal use
- unused file
- selfie
- low quality
- deletion requested by the uploader -
- mistake
- better version available
- duplicate
- down-scaled
- lower quality
Viable reasons frequency
We count how many wikilinks or full opening reason messages contain given keywords that are likely to trigger the above reasons.
Focus is on those that can be implemented as viable targets for automatic classifiers.
The table is sorted in descending order of full message percentages.
| reason | wikilink % | total | full message % | total | contains |
| freedom of panorama | 20 | 3,992 | 9 | 4,866 | fop or freedom of panorama |
| logo | 0.8 | 172 | 5 | 2507 | logo |
| screenshot | 0.09 | 18 | 1.8 | 975 | screenshot |
| duplicate | N.A. | N.A. | 1.7 | 918 | duplicate |
| album cover | ~0 | 1 | 1.6 | 841 | album |
| not suitable for work | 3 | 589 | 1.3 | 702 | penis or vulva or vagina or nudity |
| poster | 1 | 216 | 1 | 571 | poster |
| book | ~0 | 7 | 0.9 | 475 | book |
| banner | ~0 | 2 | 0.3 | 188 | banner |
For the sake of completeness, we also report the following reasons:
| reason | wikilink % | total | full message % | total | contains |
| derivative work | 3 | 697 | 2.5 | 1,324 | dw or derivative |
| not a free Web host | 1 | 264 | 1.4 | 738 | host |
| threshold of originality | 2 | 465 | 1.2 | 625 | too or threshold |
Deletion requests for multiple files
We run the analysis over an extended dataset that includes deletion requests for multiple files.
- input: deletion requests archive
- total requests closed with a deletion: 168k (167,555)
- total deleted files: 633k (633,451)
Top 10 opening reasons
Viable reasons frequency
- total wikilinks: 73k (73,167)
- total opening reason messages: 170k (170,448)
| reason | wikilink % | total | full message % | total | contains |
| freedom of panorama | 14 | 10,405 | 8.1 | 13,846 | fop or freedom of panorama |
| logo | 1 | 732 | 7.2 | 12,399 | logo |
| book | ~0 | 63 | 2.4 | 4,008 | book |
| screenshot | ~0 | 74 | 1.9 | 3,303 | screenshot |
| album cover | ~0 | 30 | 1.9 | 3,182 | album |
| duplicate | ~0 | 2 | 1.5 | 2,512 | duplicate |
| poster | 0.4 | 293 | 0.8 | 1,327 | poster |
| not suitable for work | 1.3 | 973 | 0.7 | 1,216 | penis or vulva or vagina or nudity |
| banner | ~0 | 9 | 0.2 | 332 | banner |
Conclusion
- The analysis is quite consistent with the previous dataset:
- freedom of panorama still ranks first, despite being a little less represented (-0.9%)
- logo still ranks second and gains +2%
- book now ranks third
- deletion requests for multiple files may be very large, e.g., this one accounts for 57k files