Page MenuHomePhabricator

Uploads blocked by AbuseFilter should be reported with a different error in the API than 'verification-error'
Closed, ResolvedPublic

Description

Uploads blocked by AbuseFilter should be reported with a different error in the API than 'verification-error'. It's a wholly different kind of verification, and reusing the error code causes confusion in uploading tools.

Event Timeline

(This doesn't actually require T89302, I think.)

Change 295314 had a related patch set uploaded (by Bartosz Dziewoński):
Use custom error code 'abusefilter-forbidden' for edit and upload API responses

https://gerrit.wikimedia.org/r/295314

Change 295315 had a related patch set uploaded (by Bartosz Dziewoński):
Better handle ApiMessage errors from UploadVerifyFile hook

https://gerrit.wikimedia.org/r/295315

The API results changes are as follows:

NotesAPI queryFileResponse beforeResponse after
Upload to stash, bad file (AbuseFilter)action=upload&format=json&filename=asdfasdf.jpg&file=...&stash=1
1{
2 "error": {
3 "code": "verification-error",
4 "info": "This file did not pass file verification: This action has been automatically identified as harmful, and therefore disallowed.\nIf you believe your action was constructive, please inform an administrator of what you were trying to do.\nA brief description of the abuse rule which your action matched is: Frequently uploaded non-free files by SHA1",
5 "details": [
6 "abusefilter-disallowed",
7 "Frequently uploaded non-free files by SHA1",
8 7
9 ],
10 "*": "See http://localhost:3080/w/api.php for API usage"
11 }
12}
1{
2 "error": {
3 "code": "abusefilter-disallowed",
4 "info": "Hit AbuseFilter: Frequently uploaded non-free files by SHA1",
5 "details": [
6 "abusefilter-disallowed",
7 "Frequently uploaded non-free files by SHA1",
8 7
9 ],
10 "message": "abusefilter-disallowed",
11 "abusefilter": {
12 "id": 7,
13 "description": "Frequently uploaded non-free files by SHA1",
14 "actions": [
15 "disallow"
16 ]
17 },
18 "warning": "This action has been automatically identified as harmful, and therefore disallowed.\nIf you believe your action was constructive, please inform an administrator of what you were trying to do.\nA brief description of the abuse rule which your action matched is: Frequently uploaded non-free files by SHA1",
19 "*": "See http://localhost:3080/w/api.php for API usage"
20 }
21}
Upload, bad file (AbuseFilter)action=upload&format=json&filename=asdfasdf.jpg&file=...
1{
2 "error": {
3 "code": "verification-error",
4 "info": "This file did not pass file verification: This action has been automatically identified as harmful, and therefore disallowed.\nIf you believe your action was constructive, please inform an administrator of what you were trying to do.\nA brief description of the abuse rule which your action matched is: Frequently uploaded non-free files by SHA1",
5 "details": [
6 "abusefilter-disallowed",
7 "Frequently uploaded non-free files by SHA1",
8 7
9 ],
10 "*": "See http://localhost:3080/w/api.php for API usage"
11 }
12}
1{
2 "error": {
3 "code": "abusefilter-disallowed",
4 "info": "Hit AbuseFilter: Frequently uploaded non-free files by SHA1",
5 "details": [
6 "abusefilter-disallowed",
7 "Frequently uploaded non-free files by SHA1",
8 7
9 ],
10 "message": "abusefilter-disallowed",
11 "abusefilter": {
12 "id": 7,
13 "description": "Frequently uploaded non-free files by SHA1",
14 "actions": [
15 "disallow"
16 ]
17 },
18 "warning": "This action has been automatically identified as harmful, and therefore disallowed.\nIf you believe your action was constructive, please inform an administrator of what you were trying to do.\nA brief description of the abuse rule which your action matched is: Frequently uploaded non-free files by SHA1",
19 "*": "See http://localhost:3080/w/api.php for API usage"
20 }
21}
Upload, bad description (AbuseFilter)action=upload&format=json&filename=asdfasdf.jpg&comment=too+short&file=...any
1{
2 "error": {
3 "code": "internal-error",
4 "info": "Unknown error: \"abusefilter-disallowed\"",
5 "0": [
6 "abusefilter-disallowed",
7 "Very short upload page text",
8 8
9 ],
10 "filekey": "1440rfqz1wp0.7vlk2v.1.jpg",
11 "sessionkey": "1440rfqz1wp0.7vlk2v.1.jpg",
12 "invalidparameter": null,
13 "*": "See http://localhost:3080/w/api.php for API usage"
14 }
15}
1{
2 "error": {
3 "code": "abusefilter-disallowed",
4 "info": "Hit AbuseFilter: Very short upload page text",
5 "0": [
6 "abusefilter-disallowed",
7 "Very short upload page text",
8 8
9 ],
10 "filekey": "1443shqmck30.fbnkio.1.png",
11 "sessionkey": "1443shqmck30.fbnkio.1.png",
12 "invalidparameter": null,
13 "message": "abusefilter-disallowed",
14 "abusefilter": {
15 "id": 8,
16 "description": "Very short upload page text",
17 "actions": [
18 "disallow"
19 ]
20 },
21 "warning": "This action has been automatically identified as harmful, and therefore disallowed.\nIf you believe your action was constructive, please inform an administrator of what you were trying to do.\nA brief description of the abuse rule which your action matched is: Very short upload page text",
22 "*": "See http://localhost:3080/w/api.php for API usage"
23 }
24}
Edit, bad text (AbuseFilter)action=edit&format=json&title=A&text=test-disallow
1{
2 "edit": {
3 "code": "abusefilter-disallowed",
4 "info": "Hit AbuseFilter: Test filter disallow",
5 "warning": "This action has been automatically identified as harmful, and therefore disallowed.\nIf you believe your action was constructive, please inform an administrator of what you were trying to do.\nA brief description of the abuse rule which your action matched is: Test filter disallow",
6 "result": "Failure"
7 }
8}
1{
2 "edit": {
3 "code": "abusefilter-disallowed",
4 "message": "abusefilter-disallowed",
5 "abusefilter": {
6 "id": 3,
7 "description": "Test filter disallow",
8 "actions": [
9 "disallow"
10 ]
11 },
12 "info": "Hit AbuseFilter: Test filter disallow",
13 "warning": "This action has been automatically identified as harmful, and therefore disallowed.\nIf you believe your action was constructive, please inform an administrator of what you were trying to do.\nA brief description of the abuse rule which your action matched is: Test filter disallow",
14 "result": "Failure"
15 }
16}
Upload, bad file (zero-length)action=upload&format=json&filename=asdfasdf.jpg&file=...
1{
2 "error": {
3 "code": "empty-file",
4 "info": "The file you submitted was empty",
5 "*": "See http://localhost:3080/w/api.php for API usage"
6 }
7}
1{
2 "error": {
3 "code": "empty-file",
4 "info": "The file you submitted was empty",
5 "*": "See http://localhost:3080/w/api.php for API usage"
6 }
7}
Testing with broken chmod on images/ directory to cause the erroraction=upload&format=json&filename=asdfasdf.jpg&comment=long+enoughhhhh&file=...any
1{
2 "error": {
3 "code": "internal-error",
4 "info": "Unknown error: \"directorycreateerror\"",
5 "0": [
6 "directorycreateerror",
7 "mwstore://local-backend/local-public/a/ad"
8 ],
9 "stashfailed": "Stashing temporary file failed: UploadStashFileException Error storing file in '/tmp/phpLn0Uqt': Directory \"mwstore://local-backend/local-temp/4/4d\" is read-only.",
10 "invalidparameter": null,
11 "*": "See http://localhost:3080/w/api.php for API usage"
12 }
13}
1{
2 "error": {
3 "code": "internal-error",
4 "info": "Unknown error: \"directorycreateerror\"",
5 "0": [
6 "directorycreateerror",
7 "mwstore://local-backend/local-public/a/ad"
8 ],
9 "stashfailed": "Stashing temporary file failed: UploadStashFileException Error storing file in '/tmp/phpw6xCnd': Directory \"mwstore://local-backend/local-temp/6/64\" is read-only.",
10 "invalidparameter": null,
11 "*": "See http://localhost:3080/w/api.php for API usage"
12 }
13}

This is assuming you have the following filters defined:

{"row":{"af_id":"3","af_pattern":"new_wikitext contains 'test-disallow'","af_user":"1","af_user_text":"Matma Rex","af_timestamp":"20160407050213","af_enabled":"1","af_comments":"","af_public_comments":"Test filter disallow","af_hidden":"0","af_hit_count":"7","af_throttled":"1","af_deleted":"0","af_actions":"disallow","af_global":"0","af_group":"default"},"actions":{"disallow":{"action":"disallow","parameters":[""]}}}
{"row":{"af_id":"7","af_pattern":"(action == 'upload' | action == 'stashupload')\r\n\u0026 contains_any(file_sha1,\r\n\r\n/* Microsoft Windows sample images: */\r\n'4eee44b18576e84de7b163142b537d2fe6231845', /* Blue hills.jpg */\r\n'f5f8ad26819a471318d24631fa5055036712a87e', /* Chrysanthemum.jpg */\r\n'30420d1a9afb2bcb60335812569af4435a59ce17', /* Desert.jpg */\r\n'd997e1c37edc05ad87d03603e32ad495ee2cfce1', /* Hydrangeas.jpg */\r\n'3b15be84aff20b322a93c0b9aaa62e25ad33b4b4', /* Jellyfish.jpg */\r\n'9c3dcb1f9185a314ea25d51aed3b5881b32f420c', /* Koala.jpg */\r\n'1b4605b0e20ceccf91aa278d10e81fad64e24e27', /* Lighthouse.jpg */\r\n'df7be9dc4f467187783aca68c7ce98e4df2172d0', /* Penguins.jpg */\r\n'5bd2ef6030d665aa615147512a0fea3055930cc6', /* Sunset.jpg */\r\n'54c2f1a1eb6f12d681a5c7078421a5500cee02ad', /* Tulips.jpg */\r\n'fb662cbd45033e03f65e0f278f44f4206a3c4293', /* Water lilies.jpg */\r\n'ca6ec835bffff37e28896df424db5559012d48b6', /* Winter.jpg */\r\n\r\n)","af_user":"1","af_user_text":"Matma Rex","af_timestamp":"20160617173136","af_enabled":"1","af_comments":"https://commons.wikimedia.org/wiki/Special:AbuseFilter/31","af_public_comments":"Frequently uploaded non-free files by SHA1","af_hidden":"0","af_hit_count":"11","af_throttled":"0","af_deleted":"0","af_actions":"disallow","af_global":"0","af_group":"default"},"actions":{"disallow":{"action":"disallow","parameters":[""]}}}
{"row":{"af_id":"8","af_pattern":"action='upload' \u0026 new_size \u003C 10","af_user":"1","af_user_text":"Matma Rex","af_timestamp":"20160621210123","af_enabled":"1","af_comments":"","af_public_comments":"Very short upload page text","af_hidden":"0","af_hit_count":"14","af_throttled":"0","af_deleted":"0","af_actions":"warn,disallow","af_global":"0","af_group":"default"},"actions":{"disallow":{"action":"disallow","parameters":[""]},"warn":{"action":"warn","parameters":["abusefilter-warning-tooshort"]}}}

PS2 of https://gerrit.wikimedia.org/r/295314 changes the error code from 'abusefilter-forbidden' to 'abusefilter-disallowed'/'abusefilter-warning', to avoid breaking backwards compatibility and to provide better information.

I updated the test results above.

Change 295315 merged by jenkins-bot:
ApiUpload: Better handle ApiMessage errors from UploadVerifyFile hook

https://gerrit.wikimedia.org/r/295315

Change 295314 merged by jenkins-bot:
Use custom error code for all edit and upload API responses

https://gerrit.wikimedia.org/r/295314