Page MenuHomePhabricator

[M] Extend logo detection metrics to tools besides Upload Wizard
Closed, ResolvedPublic

Description

T362211#9903480 reports a tiny amount of logo deletion requests out of all Upload Wizard's uploads (see UW_logo_drs_within_30_days versus UW_all columns).
This suggests that UW might not be the main source of logo uploads.
Therefore, we should take into account other upload tools, e.g., Visual Editor.
CC @cchen.

Tasks

  • retrieve edit tags that correspond to other upload tools. UW seems to have ct_tag_id=12
  • re-use relevant parts of this notebook to compute counts of other tools

Tools considered

nameEdit tag ID# edits
Upload Wizard1224,885,182
OAuth Uploader *376,406,377
improved old upload form192,697,046
Cross-wiki upload171,823,387
FileImporter4241,209,076
CropTool 1.5 (latest version)486930,297

* CommonsHelper + Flickr2Commons + Url2Commons + geograph2commons

See T368167#9939792 for more details.

Method

  • input: edit tag ID; deletion requests (DR) dataset as per this script
  • gather all deleted and non-deleted file uploads
    • with a DR created within 30 days
    • maybe with a deletion edit comment
    • that have a case-insensitive logo occurrence in the page title or DR title or DR opening reason or DR closing reason or deletion edit comment

Results

The tables below show the percentage of files with DRs and/or deletions due to logo reasons. The percentage is computed against total uploads.
We report a 6-month average and a random`2024-04` snapshot for the sake of comparison with the average.

NOTE: The percentage takes into account open and/or closed DRs, plus files deleted without a DR. See with_logo_dr and deleted_without_dr in detailed results at T368167#9956995 and T368167#9959868.

Average

From 2023-11 to 2024-04.

edit tagtotal uploadslogo %
Cross-wiki upload17k0.5
mobile edit80k0.29
FileImporter1.9k0.14
Upload Wizard360k0.13
improved old form39k0.05
CropTool 1.519k0.02

2024-04

edit tagtotal uploadslogo %
Cross-wiki upload17k0.26
mobile edit67k0.23
Upload Wizard399k0.08
FileImporter1.3k0.07
CropTool18k0.03
improved old form39k0.005

Detailed results

See T368167#9956995 and T368167#9959868.

Conclusion

  • Cross-wiki upload is the tool with the highest percentage of logo-related DRs/deletions
  • Mobile edit has way more uploads and has the 2nd highest percentage. According to https://commons.wikimedia.org/wiki/Special:Tags, this edit tag can also indicate edits from the (Android) app
  • compared to initial metrics, we report an increase in Upload Wizard's logo DR percentage, likely due to the expanded scope of the query, i.e., case-insensitive and more fields: for e.g., 2024-03, 0.082 in T368167#9956995 VS 0.025 in T362211#9903480 (cf. % logo DRs within 30 days column)

Event Timeline

mfossati renamed this task from Extend logo detection metrics to tools besides Upload Wizard to [M] Extend logo detection metrics to tools besides Upload Wizard.Jul 1 2024, 8:31 AM
mfossati changed the task status from Open to In Progress.
mfossati claimed this task.

This also means that we want to extend the DR metrics for logos to all uploads to Commons, not just the ones uploaded through Upload Wizard as we will be detecting all logos uploaded to Commons post upload. cc @cchen

Top 10 edit tags
nameID# editsupload tool?
uploadwizard1224,885,182yes
PAWS (all versions)500 & 625 & 116 & 61019,572,986 (12,628,752 + 3,759,957 + 1,830,448 + 1,353,829)?
mobile edit147,283,648?
OAuth Uploader *376,406,377yes
mobile web edit155,548,263?
Flickr2Commons469 & 3815,235,827 (3,378,242 + 1,857,585)yes
improved old upload form192,697,046yes
cross-wiki-upload171,823,387yes
CropTool (all versions)486 & 27 & 1241,519,127 (930,297 + 523,169 + 65,661)yes
FileImporter4241,209,076yes

* CommonsHelper + Flickr2Commons + Url2Commons + geograph2commons

NOTE: From a category query, the Android app has 418,151 images uploaded.
Sources

Results

Upload Wizard

yearmonthtotal_uploadswith_logo_drdeleted_without_drtotal
202406259
2024053936700.0132090329463764070.0378489597886554760.05105799273503188
2024043994370.036551446160470860.043811664918372610.08036311107884347
2024034067770.082354705403697840.0621962402004046440.1445509456041025
2024023532490.051521731130165970.080679633912622530.1322013650427885
2024013524670.123983238147117310.068942624415902760.19292586256302008
2023123107980.059845944954600740.078829336096113870.1386752810507146
2023113375880.0521345545457776930.053615649845373650.10575020439115135

OAuth Uploader

No data since July 2020.

Improved old upload form

yearmonthtotal_uploadswith_logo_drdeleted_without_drtotal
2024065
202405457510.00218574457388909540.0065572337216672860.008742978295556382
202404395140.00253074859543452960.00253074859543452960.005061497190869059
202403634010.012618097506348480.003154524376587120.0157726218829356
202402252990.0513854302541602440.110676311316652840.16206174157081307
202401330100.0121175401393517120.0242350802787034240.036352620418055134
202312298160.016769519720955190.0134156157767641540.030185135497719343
202311407890.0147098482433989570.012258206869499130.026968055112898084

Cross-wiki upload

yearmonthtotal_uploadswith_logo_drdeleted_without_drtotal
20240626
202405170860.064380194311131930.117054898747512570.1814350930586445
202404173700.092112838226827860.172711571675302240.26482440990213013
202403175210.18834541407453910.42805775926031620.6164031733348553
202402166170.210627670457964740.28886080519949450.49948847565745924
202401177490.197194208124401360.242267169981407380.43946137810580876
202312166000.228915662650602430.433734939759036150.6626506024096386
202311170600.25205158264947240.293083235638921470.5451348182883939

FileImporter

yearmonthtotal_uploadswith_logo_drdeleted_without_drtotal
20240516200.06172839506172839
20240413370.07479431563201197
20240319140.052246603970741906
20240221870.045724737082761770.13717421124828530.18289894833104708
20240122970.043535045711798
20231220470.048851978505129456
20231115870.37807183364839320.06301197227473220.4410838059231254

CropTool 1.5

yearmonthtotal_uploadswith_logo_drdeleted_without_drtotal
20240639
202405155760.00642013353877760650.0128402670775552130.019260400616332822
202404183720.027215327672545177
202403188630.0265069183056777860.0159041509834066670.04241106928908445
202402175140.0114194358798675330.0057097179399337670.0171291538198013
202401190290.0052551368963161490.0157654106889484470.021020547585264594
202312185130.0108032193593690930.0108032193593690930.021606438718738186
202311191440.005223568742164647

Other edit tags

PAWS

No deletions due to logo except a negligible 0.002 % on Feb 2024.

Mobile edit

yearmonthtotal_uploadswith_logo_drdeleted_without_drtotal
202406178
202405633290.0521090.1184290.170538
202404674120.1127400.1171900.229929
202403602490.1858950.1510400.336935
202402537030.1750370.2495210.424557
202401666790.1889650.1169780.305943
2023121159270.1164530.0750470.191500
2023111176600.1912290.0475950.238824

Pasting the conclusion here for convenience.

Conclusion

  • Cross-wiki upload is the tool with the highest percentage of logo-related DRs/deletions
  • Mobile edit has way more uploads and has the 2nd highest percentage. According to https://commons.wikimedia.org/wiki/Special:Tags, this edit tag can also indicate edits from the (Android) app
  • compared to initial metrics, we report an increase in Upload Wizard's logo DR percentage, likely due to the expanded scope of the query, i.e., case-insensitive and more fields: for e.g., 2024-03, 0.082 in T368167#9956995 VS 0.025 in T362211#9903480 (cf. % logo DRs within 30 days column)

Just a couple of extra data points on logos (from https://gitlab.wikimedia.org/cparle/notebooks/-/blob/main/logos.ipynb)

Uploaded in 2023

All12807507
Deleted364488
% deleted2.85%
In Category:Logos (or subcat, up to 3 levels), not deleted63807
Deleted, with "logo" in deletion reason8493
% of logos deleted11.75%
Has {{Own}} template, in Category:Logos (or subcat, up to 3 levels), not deleted12726
Has {{Own}} template, deleted with "logo" in deletion reason5316
% of ownwork logos deleted41.78%

... so:

  • if something is a logo it's ~4 times more likely to be deleted compared to the background rate
  • if something is a logo and ownwork it's almost 15 times more likely to be deleted compared to the background rate
  • however even an ownwork logo has a less that 50/50 likelihood of being deleted