Page MenuHomePhabricator
Create Task
Maniphest T269713

Run the MigrateAflFilter script for AbuseFilter
Closed, ResolvedPublic
Actions

Description

What: Dry-run (on a single wiki, for sanity), then run the migrateAflFilter .php script for AbuseFilter
When: Whenever this patch reaches production.
Where: All wikis - beta cluster first, then production (should be next week).
Who: Whoever wants to do that. (@Daimona can do the Beta Cluster part).
How:

  1. First of all, the script should be checked with migrateAflFilter.php --dry-run on a random wiki for sanity (no errors expected)
  2. If everything is fine, it can be run for all wikis (gradually), without the need to dry-run it first.

Command to run

mwscript extensions/AbuseFilter/maintenance/MigrateAflFilter.php --wiki=… --batch-size=1000
  • Beta Cluster
  • Closed wikis
  • Test wikis
  • Group0 wikis
  • Group1 wikis
  • Group2 wikis

Related Objects
Search...

Event Timeline

Daimona created this task.Dec 8 2020, 9:58 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 8 2020, 9:58 PM
Daimona mentioned this in T269712: Migrate afl_filter to afl_filter_id and afl_global.Dec 8 2020, 9:58 PM
Daimona added a parent task: T269712: Migrate afl_filter to afl_filter_id and afl_global.
Jdforrester-WMF updated the task description. (Show Details)Dec 9 2020, 3:41 PM
Jdforrester-WMF added subscribers: Urbanecm, Jdforrester-WMF.Jan 15 2021, 1:06 AM

OK, and it's now fully run (automatically) for Beta Cluster. Are we confident that there were no issues? If we're happy to proceed, does @Urbanecm want to run this AF update in prod, like they did the last one?

Jdforrester-WMF updated the task description. (Show Details)Jan 15 2021, 1:06 AM
Daimona added a comment.Jan 15 2021, 1:08 AM
In T269713#6750042, @Jdforrester-WMF wrote:

OK, and it's now fully run (automatically) for Beta Cluster. Are we confident that there were no issues? If we're happy to proceed, does @Urbanecm want to run this AF update in prod, like they did the last one?

I think we can be fairly confident. The config patch should be merged first (I forgot to follow-up 2 days ago), and if no issues arise, we can proceed as usual to see if we spot any error (which I think is quite unlikely at this point).

Urbanecm added a comment.Jan 15 2021, 7:49 PM
In T269713#6750042, @Jdforrester-WMF wrote:

OK, and it's now fully run (automatically) for Beta Cluster. Are we confident that there were no issues? If we're happy to proceed, does @Urbanecm want to run this AF update in prod, like they did the last one?

I can do that :).

Jdforrester-WMF assigned this task to Urbanecm.Jan 15 2021, 9:04 PM
Restricted Application added a project: User-Urbanecm. · View Herald TranscriptJan 15 2021, 9:04 PM
Jdforrester-WMF added a comment.Jan 15 2021, 9:04 PM

Thanks!

Daimona added a comment.Jan 15 2021, 10:08 PM
In T269713#6750045, @Daimona wrote:

The config patch should be merged first (I forgot to follow-up 2 days ago)

Actually, discard that. The stage is already WRITE_BOTH as that's the default in AbuseFilter. You can start running the script whenever you please.

Jdforrester-WMF added a comment.EditedJan 15 2021, 11:36 PM
In T269713#6752473, @Daimona wrote:
In T269713#6750045, @Daimona wrote:

The config patch should be merged first (I forgot to follow-up 2 days ago)

Actually, discard that. The stage is already WRITE_BOTH as that's the default in AbuseFilter. You can start running the script whenever you please.

No, for safety we intentionally were extremely conservative in enabling the new write mode in production, so that it didn't potentially break the train or whatever. (But then it changed and I forgot. Oops.)

Daimona added a comment.Jan 16 2021, 12:18 AM
In T269713#6752672, @Jdforrester-WMF wrote:
In T269713#6752473, @Daimona wrote:
In T269713#6750045, @Daimona wrote:

The config patch should be merged first (I forgot to follow-up 2 days ago)

Actually, discard that. The stage is already WRITE_BOTH as that's the default in AbuseFilter. You can start running the script whenever you please.

No, for safety we intentionally were extremely conservative in enabling the new write mode in production, so that it didn't potentially break the train or whatever. (But then it changed and I forgot. Oops.)

Ah right, it was set explicitly to OLD. Nevermind then.

Stashbot added a comment.Jan 19 2021, 3:15 PM

Mentioned in SAL (#wikimedia-operations) [2021-01-19T15:15:44Z] <Urbanecm> Run foreachwikiindblist closed extensions/AbuseFilter/maintenance/MigrateAflFilter.php (T269713)

Urbanecm updated the task description. (Show Details)Jan 19 2021, 3:18 PM
Stashbot added a comment.Jan 19 2021, 5:04 PM

Mentioned in SAL (#wikimedia-operations) [2021-01-19T17:04:40Z] <Urbanecm> mwscript extensions/AbuseFilter/maintenance/MigrateAflFilter.php --wiki=testwiki --batch-size=1000 # T269713

Stashbot added a comment.Jan 19 2021, 5:06 PM

Mentioned in SAL (#wikimedia-operations) [2021-01-19T17:06:20Z] <Urbanecm> mwscript extensions/AbuseFilter/maintenance/MigrateAflFilter.php --wiki=test2wiki --batch-size=1000 # T269713

Stashbot added a comment.Jan 19 2021, 5:08 PM

Mentioned in SAL (#wikimedia-operations) [2021-01-19T17:08:48Z] <Urbanecm> Run extensions/AbuseFilter/maintenance/MigrateAflFilter.php for all group0 wikis (T269713)

Stashbot added a comment.Jan 19 2021, 5:30 PM

Mentioned in SAL (#wikimedia-operations) [2021-01-19T17:30:09Z] <Urbanecm> Start of foreachwikiindblist group1 extensions/AbuseFilter/maintenance/MigrateAflFilter.php --batch-size=1000 (T269713)

Urbanecm triaged this task as High priority.Jan 19 2021, 5:40 PM
Urbanecm updated the task description. (Show Details)Jan 19 2021, 8:50 PM
Urbanecm updated the task description. (Show Details)
Stashbot added a comment.Jan 19 2021, 9:00 PM

Mentioned in SAL (#wikimedia-operations) [2021-01-19T21:00:36Z] <Urbanecm> Start of foreachwikiindblist group2 extensions/AbuseFilter/maintenance/MigrateAflFilter.php --batch-size=1000 (T269713)

Urbanecm closed this task as Resolved.Jan 21 2021, 8:33 PM

This is done.

Jdforrester-WMF updated the task description. (Show Details)Jan 21 2021, 9:07 PM
Daimona awarded a token.Jan 21 2021, 9:40 PM
Daimona mentioned this in T276351: Requesting access to restricted for Daimona.Mar 3 2021, 4:19 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits