When the target wiki has the abuse filter enabled, use it
Open, NormalPublic2 Story Points


Files uploaded through the file importer should adhere to current standards. Therefore they should be able to pass the abuse filter, if the target wiki (such as Commons) has one

If appropriate, we should:
When saving the fileImporter specialPage, the imported file should pass the abuse filter.

What makes it difficult is that we need to enter the old revisions in order, i.e. the current file revision needs to be the last one.Old revisions should also trigger the abuse filter

Lea_WMDE created this task.Mar 15 2017, 4:21 PM
Andrew-WMDE removed Andrew-WMDE as the assignee of this task.Oct 30 2017, 4:12 PM
Andrew-WMDE added a subscriber: Andrew-WMDE.
Lea_WMDE renamed this task from Investigate and use the abuse filter for the FileImporter if appropriate to When the target wiki has the abuse filter enabled, use it.Nov 2 2017, 9:03 AM
Lea_WMDE updated the task description. (Show Details)
Tobi_WMDE_SW set the point value for this task to 21.Feb 13 2018, 4:02 PM
Tobi_WMDE_SW added a subscriber: Tobi_WMDE_SW.

We probably need to distinguish between images and text.

WMDE-Fisch claimed this task.

So when it comes to uploads, the AbuseFilter extension hooks itself into the UploadVerifyUpload hook, that allows stuff do be checkd on the file, text, edit and edit summary of an upload in the UploadBase class. These are the checks that IMO should be also performed for the imported file and text revisions. And as I see it they are not at the moment because the only place this hook is called is in UploadBase::performUpload. I will try to call that hook at corresponding places in our code.

The AbuseFilter has certain text it provides when it took action. - E.g. When I run the hook on our file uploads, and the filter triggers that blocks the upload, I get an error text from it that looks like that:

This action has been automatically identified as harmful, and therefore disallowed.
If you believe your action was constructive, please inform an administrator of what you were trying to do.
A brief description of the abuse rule which your action matched is: Upload Large Test

Upload Large Test is the name of my filter here ( set via the SpecialPage from AbuseFilter to configure filters on wiki. e.g. see https://commons.wikimedia.org/w/index.php?title=Special:AbuseFilter )


  • Do we want to expose that message to the user as it is?
  • Do we want to replace it with something genereic? ( e.g. some filter rules are in place that prevent your upload. )
  • Do we want to expose it and add some additional message?

@Lea_WMDE @Charlie_WMDE

Thanks @WMDE-Fisch I'm adding @Hanna_Petruschat_WMDE since she is the new UX lead for the FileImporter.
One question: When will this be triggered? The moment that the user hits the "import" button, or when they click the link?
And one reason why we might want to change the message is that we have multiple revisions that can be affected, whereas usually there is only one. Fisch, do we know which revision is affected specifically?

Sooo for the time being, I would trigger this the moment the user hits "Import". - At least for now and for the FileRevisions, because at that moment we have a similar state of the data like when the UploadVerifyUpload Hook is called in UploadBase and we also do other final checks there. - But I assume that could be changed. - We could also have data about the revsion affected, since we loop through them checking each separately.

Even if we might want to move the checks to the start we still would need at least one final check for the text in cases it got changed.

Change 417307 had a related patch set uploaded (by WMDE-Fisch; owner: WMDE-Fisch):
[mediawiki/extensions/FileImporter@master] [WIP] Include Hooks relevant for AbuseFilter


... and thinking about it, if you just display the error messages somewhere for now, I guess it would be best. We'll have another ticket about improving error messages anyways, and then this ticket won't be blocked.

Tobi_WMDE_SW changed the point value for this task from 21 to 2.Tue, Mar 20, 4:06 PM