See T225563: Deploy ORES filters for jawiki for discussion.
Japanese Wikipedia's goodfaith model is not useful enough for the deployment of recentchanges filters that would use it.
We should investigate why this is the case and see if we can make some improvements.
(While you're at it, the damaging model isn't great either: it's not able to identify damaging edits with 60% precision or higher.)
We need someone from Japanese Wikipedia to help out with this. @Keegan, do you have any suggestions for how we might find a Japanese Wikipedia who might be able to help us review where our ORES models are working/not?
Dropping a note that I'm looking into this, apologies for the delay. The week got away from me.
Thanks @Keegan and no worries. We'll be ready to go when you are. There's always other work to do in the meantime ^_^
As a first pass, I'd love to just have someone try to use what we have deployed and tell us what it tends to get wrong. The best way to do that is to use the filters in Special:RecentChanges to do patrolling work. If we get some interest, I would like to produce some specific sample data with ORES scores that people can label and give us notes on. From that I think it'll become apparent what types of things we're missing.
This page isn't very active. I think we need to ask some others to look at the post, or put a notice about the post up somewhere more effective. I'll look into it.
Thanks, @Keegan!
I wonder if there is a place that patrollers in jawiki gather and discuss issues of counter-vandalism. That seems like a good place to reach out.
As discussed with halfak on IRC, we're not aware of much inter-editor communication or coordination around counter-vandalism work at jawiki.
I've posted the message at the Wikipedia's "News" page.
@jeena took a list at a bunch of example edits that scored as likely to be badfaith and confirmed that most of them look good. I think the right next step here is to interrogate our labeled data to see if Japanese Wikipedians would confirm or refute the "badfaith" labeled edits.
The quickest way to gather examples of edits that were labeled badfaith is to access https://labels.wmflabs.org/campaigns/jawiki/15/?tasks and look for "badfaith": true.