Page MenuHomePhabricator

Why is jawiki's goodfaith model so bad?
Open, LowPublic

Description

See T225563: Deploy ORES filters for jawiki for discussion.

Japanese Wikipedia's goodfaith model is not useful enough for the deployment of recentchanges filters that would use it.

We should investigate why this is the case and see if we can make some improvements.

Event Timeline

Halfak created this task.Aug 21 2019, 7:23 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 21 2019, 7:23 PM
Halfak renamed this task from Why is jawiki to Why is jawiki's goodfaith model so bad?.Aug 21 2019, 7:24 PM
Halfak updated the task description. (Show Details)

(While you're at it, the damaging model isn't great either: it's not able to identify damaging edits with 60% precision or higher.)

Halfak triaged this task as Low priority.Sep 11 2019, 9:05 PM
Halfak moved this task from Untriaged to Maintenance/cleanup on the Scoring-platform-team board.
Halfak added a subscriber: Keegan.

We need someone from Japanese Wikipedia to help out with this. @Keegan, do you have any suggestions for how we might find a Japanese Wikipedia who might be able to help us review where our ORES models are working/not?

Keegan added a comment.EditedSep 13 2019, 7:19 PM

I'll see if I can help further next week, whether identifying a place or a person.

\o/ Thank you.

I'll see if I can help further next week, whether identifying a place or a person.

Dropping a note that I'm looking into this, apologies for the delay. The week got away from me.

Thanks @Keegan and no worries. We'll be ready to go when you are. There's always other work to do in the meantime ^_^

As a first pass, I'd love to just have someone try to use what we have deployed and tell us what it tends to get wrong. The best way to do that is to use the filters in Special:RecentChanges to do patrolling work. If we get some interest, I would like to produce some specific sample data with ORES scores that people can label and give us notes on. From that I think it'll become apparent what types of things we're missing.

Keegan added a comment.Oct 1 2019, 3:54 PM

This page isn't very active. I think we need to ask some others to look at the post, or put a notice about the post up somewhere more effective. I'll look into it.

Halfak added a comment.Oct 1 2019, 7:32 PM

Thanks, @Keegan!

I wonder if there is a place that patrollers in jawiki gather and discuss issues of counter-vandalism. That seems like a good place to reach out.

Keegan added a comment.Oct 3 2019, 8:39 PM

As discussed with halfak on IRC, we're not aware of much inter-editor communication or coordination around counter-vandalism work at jawiki.

I've posted the message at the Wikipedia's "News" page.

https://ja.wikipedia.org/wiki/Wikipedia:%E3%81%8A%E7%9F%A5%E3%82%89%E3%81%9B#Built-in_edit_filters_for_Japanese_need_testing

Rxy added a subscriber: Rxy.Oct 9 2019, 12:12 PM