Page MenuHomePhabricator

ApiUpload allows duplicates when ignorewarnings is set.
Open, Needs TriagePublic

Description

In ApiUpload, the ignorewarnings flag allows the same image to be uploaded under multiple names. That may be surprising, and is not documented on API:Upload. It may be useful to have more control over which warnings should be ignored.

Originally filed as:
Wikimedia Commons API allows to upload same file several times

Steps to Reproduce:

Actual Results:

  • Wikimedia Commons API allows to upload the same file several times

Expected Results:

  • Wikimedia Commons API forbids to upload the same file several times

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Reedy added a subscriber: Reedy.

replag is currently 27h:19m:36s on s4. At the time of file uploads it was 16h:45m:18s

Replag where/according to what?

https://commons.wikimedia.org/w/api.php?action=query&meta=siteinfo&siprop=dbrepllag&sishowalldb= is less than a second

Sure, to cloud it's pretty lagged as per https://replag.toolforge.org/, but that's irrelevant to uploading to commons

Replag where/according to what?
Sure, to cloud it's pretty lagged as per https://replag.toolforge.org/, but that's irrelevant to uploading to commons

Yes I was talking about this one, sorry for the confusion.

kevinbazira added subscribers: daniel, kevinbazira.

I consulted @daniel who confirmed that the WikiCommons API uses the MediaWiki API as this documentation suggests.

Then tried reproducing this bug in my local MediaWiki environment by taking the steps below;

    1. Added the following to my LocalSettings.php
        $wgEnableUploads = true;
        $wgGroupPermissions['user']['upload_by_url'] = true;
        $wgAllowCopyUploads = true;
        $wgCopyUploadsFromSpecialUpload = true;
    2. Loaded this APISandbox link, then auto-generated a token and made an API request.
        Step2 resulted in the image not being uploaded with a warning of a duplicate image.
    3. Repeated step2 with this APISandbox link.
        Step3 resulted in the image being uploaded successfully with a warning of a duplicate image.

@Don-vip, when you made the second API call, did you have the ignorewarnings flag set to 1?

Please confirm. Thanks!

@kevinbazira Yes I set this flag. My params are set as follows:

Map.of(
    "action", "upload",
    "comment", "#Spacemedia - Upload of " + url + " via [[:Commons:Spacemedia]]",
    "format", "json",
    "filename", Objects.requireNonNull(filename, "filename"),
    "ignorewarnings", "1",
    "text", Objects.requireNonNull(wikiCode, "wikiCode"),
    "token", token
)

@Don-vip in that case, everything is working as designed. Duplicate prevention is only advisory, if you ignore warnings, you can upload dupes.

I suppose this comes down to improving documentation. We could also think about more fine grained control over which warnings to ignore.

What prompted you to set the ignorewarnings flag?

daniel changed the subtype of this task from "Bug Report" to "Task".Sep 23 2020, 7:15 PM

Sending this back to the inbox for re-triage and appropriate routing.

daniel raised the priority of this task from Medium to Needs Triage.Sep 23 2020, 7:17 PM
daniel renamed this task from Wikimedia Commons API allows to upload same file several times to ApiUpload allows duplicates when ignorewarnings is set..Sep 23 2020, 7:19 PM
daniel updated the task description. (Show Details)

@Don-vip in that case, everything is working as designed. Duplicate prevention is only advisory, if you ignore warnings, you can upload dupes.

Why is that? I don't understand why a duplicate upload is considered a warning and not a blocking error. I can't see any valid reason to upload a perfect duplicate.

What prompted you to set the ignorewarnings flag?

I don't remember. Either I enabled it while reading the documentation to be sure my bot's uploads would always succeed, or I faced a particular warning I did have to bypass. I will try to disable this flag to see if I really need it.

@Don-vip I understand that this is confusing.

Moving the ticket to feature request review, since there are product level questions to be answered here. The ignorewarnings option is not very helpful, since there are some warnings which clients will generally want to ignore (e.g. a file with the same name having existed in the past), while there are other warnings that they may not want to ignore (such as a different file existing under the same name, or the same file existing under a different name).

Perhaps ignorewarnings should accept a list of warnings to ignore, rather than being a boolean.

For now, I have updated https://www.mediawiki.org/wiki/API:Upload to include basic information about warnings.