Page MenuHomePhabricator
Create Task
Maniphest T232583

Adjust Content Translation to changes in AbuseFilter and SpamBlacklist systems
Closed, ResolvedPublic
Actions

Description

When saving an edit is prevented by the AbuseFilter or SpamBlacklist extensions, the error is currently reported as a successful API response with a 'failure' code in the body ( AbuseFilter: P8988, SpamBlacklist: P8990 ).

In the future, these will be reported as standard API errors (AbuseFilter: P8989, SpamBlacklist: P8991 ).
Clients that do handle AbuseFilter or SpamBlacklist failures specially will need to be updated to check for error codes 'abusefilter-warning', 'abusefilter-disallowed', and/or 'spamblacklist' and handle them as they do the current AbuseFilter and SpamBlacklist failures, if they want to preserve their current special handling.

Note that edit failures due to CAPTCHAs from ConfirmEdit are not being changed at this time. They will continue to be reported as before (ConfirmEdit: P9076 )

Details

SubjectRepoBranchLines +/-
Update Abuse Filter error handling codemediawiki/extensions/ContentTranslationmaster+50 -55
Remove obsolete abuse filter error handling in CX1mediawiki/extensions/ContentTranslationmaster+2 -9
Customize query in gerrit

Related Objects
Search...

Event Timeline

Pginer-WMF created this task.Sep 11 2019, 9:50 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 11 2019, 9:50 AM
Pginer-WMF triaged this task as Medium priority.Sep 11 2019, 9:50 AM
Pginer-WMF moved this task from Backlog to Priority backlog on the Language-Team (Language-2019-July-September) board.
Pginer-WMF updated the task description. (Show Details)
Pginer-WMF raised the priority of this task from Medium to High.Sep 11 2019, 11:57 AM

In order for the adjustments to catch the train before the Abuse filter changes take effect, the code should be ideally reviewed, merged and tested by September 17 (i.e., during the next 6 days). So, I'm increasing the priority to high due the time constraints.

gerritbot added a comment.Sep 11 2019, 3:32 PM

Change 535874 had a related patch set uploaded (by Petar.petkovic; owner: Petar.petkovic):
[mediawiki/extensions/ContentTranslation@master] Update Abuse Filter error handling code

https://gerrit.wikimedia.org/r/535874

gerritbot added a project: Patch-For-Review.Sep 11 2019, 3:32 PM
• Petar.petkovic claimed this task.Sep 11 2019, 3:32 PM
• Petar.petkovic moved this task from Priority backlog to In Review on the Language-Team (Language-2019-July-September) board.
gerritbot added a comment.Sep 11 2019, 3:54 PM

Change 535881 had a related patch set uploaded (by Petar.petkovic; owner: Petar.petkovic):
[mediawiki/extensions/ContentTranslation@master] Remove obsolete abuse filter error handling in CX1

https://gerrit.wikimedia.org/r/535881

Amorymeltzer subscribed.Sep 11 2019, 4:33 PM
gerritbot added a comment.Sep 12 2019, 5:31 AM

Change 535881 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] Remove obsolete abuse filter error handling in CX1

https://gerrit.wikimedia.org/r/535881

gerritbot added a comment.Sep 12 2019, 5:49 AM

Change 535874 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] Update Abuse Filter error handling code

https://gerrit.wikimedia.org/r/535874

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.23; 2019-09-17).Sep 12 2019, 6:00 AM
Maintenance_bot removed a project: Patch-For-Review.Sep 12 2019, 6:10 AM
Nikerabbit merged a task: T232683: Update ContentTranslation for changes in AbuseFilter and SpamBlacklist API error responses.Sep 12 2019, 7:39 AM
Nikerabbit added subscribers: matmarex, Liuxinyu970226.
• Petar.petkovic mentioned this in T232683: Update ContentTranslation for changes in AbuseFilter and SpamBlacklist API error responses.Sep 12 2019, 9:04 AM
• Petar.petkovic moved this task from In Review to Needs QA on the Language-Team (Language-2019-July-September) board.Sep 12 2019, 12:03 PM
• Jpita subscribed.Sep 12 2019, 4:29 PM

@Petar.petkovic any hints on how to test this?

matmarex added a parent task: T232682: Update WMF-deployed extensions for changes in AbuseFilter and SpamBlacklist API error responses.Sep 12 2019, 4:55 PM
• Petar.petkovic added a comment.Sep 13 2019, 9:26 AM
In T232583#5488235, @Jpita wrote:

@Petar.petkovic any hints on how to test this?

There were changes in how Abuse Filter sends data when publishing is prevented due to some filter being hit. Due to good communication, the changes were brought to our attention in time, before changes in Abuse Filter ended up in production. The changes are rolling out in 1.34.0-wmf.23.
We reacted and adapted our code to changes. That means, there is no environment (besides local) where you can see what would happen if we didn't adapt to changes.

What you can do is check that errors are reported correctly in Content Translation when Abuse Filter is hit and preventing publishing. You can use cx2-testing for this and check Abuse Filter rules created over there. Look for rules which have "Disallow". I used rule #4, which checks for YouTube links. Steps how I got abuse filter to get hit:

  1. Created new account to be sure user_age < 600 condition defined in Abuse Filter is satisfied
  2. Start translating any article with any language pair
  3. Add first paragraph and type "youtube.com" somewhere inside the paragraph
  4. Add another paragraph. Error in issue card system should be registered
  5. Now that we have error, publish button is disabled and we cannot attempt publishing, but due to T226280, if we add another section, button gets enabled again
  6. Try publishing and you should see:

af-cx2-testing.png (938×1 px, 70 KB)

Pginer-WMF moved this task from Needs Triage to Enhancements on the ContentTranslation board.Sep 13 2019, 10:48 AM
• Jpita added a comment.Sep 23 2019, 2:12 PM

Trying to translate Аскаров, Аскар Сайпулаевич from ru to en, I added the word BITCH which, according to abuse filter 225 should not be allowed and I was able to publish to my personal space https://en.wikipedia.org/wiki/User:Jpita23/Аскаров,_Аскар_Сайпулаевич

Nikerabbit subscribed.Sep 23 2019, 2:42 PM

User namespace is not among the namespaces (Main, User_talk) checked by that filter.

matmarex added a comment.Sep 23 2019, 4:31 PM

FYI for SpamBlacklist testing, I've always tried adding links to http://youtu.be/test (the entire domain is disallowed for some reason).

• Jpita closed this task as Resolved.Oct 8 2019, 2:34 PM
• Jpita moved this task from Needs QA to Done on the Language-Team (Language-2019-July-September) board.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL