Page MenuHomePhabricator

Investigate issues with vandalism detection on Water (Q283)
Closed, ResolvedPublic

Description

This card is done when a report is post to a wiki about the changes.

Event Timeline

Halfak created this task.Nov 16 2015, 3:17 PM
Halfak updated the task description. (Show Details)
Halfak raised the priority of this task from to Needs Triage.
Halfak moved this task to Backlog on the Scoring-platform-team (Current) board.
Halfak added a subscriber: Halfak.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 16 2015, 3:17 PM
Halfak assigned this task to Ladsgroup.Nov 16 2015, 3:17 PM
Halfak set Security to None.
Halfak added a comment.EditedNov 18 2015, 5:37 PM

From #wikimedia-ai:

[13:18:32] <halfak> Oh! I should check the model against water.
[13:18:34] <halfak> One sec.
[13:19:12] <halfak> Wikidata search is awful!
[13:20:22] <halfak> Amir1, looks like we're scoring edits to water with a bit less extreme scores.
[13:20:55] <Amir1> :))))
[13:21:22] <halfak> We're still scoring highly, but not at the 99-100% level.
[13:22:10] <halfak> Last 5 edits: 0.79, 0.94, 0.86, 0.81, 0.92
[13:22:25] <Amir1> that's better
[13:22:32] <Amir1> but we still need to work on them
[13:22:52] <halfak> Compared to 0.98, 1.00, 1.00, 0.98, 0.99

ToAruShiroiNeko triaged this task as High priority.Nov 20 2015, 6:21 PM

AUC = 0.8467 for Wikidata with user.age -- Still need to test against Water.

Just got a new dataset for training against from @Ladsgroup.

https://tools.wmflabs.org/dexbot/damaging_73k.tsv is list of 73K edits randomly sampled, balanced and ready to be sampled and fed to the training system. (the second row is damaging or not)

He7d3r added a subscriber: He7d3r.Nov 28 2015, 10:56 AM

How was that detected? Does "is damaging" means "was reverted" here?

Just got a new dataset for training against from @Ladsgroup.

https://tools.wmflabs.org/dexbot/damaging_73k.tsv is list of 73K edits randomly sampled, balanced and ready to be sampled and fed to the training system. (the second row is damaging or not)

Halfak updated the task description. (Show Details)Dec 4 2015, 6:36 PM
Ladsgroup moved this task from Backlog to Done on the wb_vandalism board.Dec 31 2015, 12:36 AM
Halfak closed this task as Resolved.Jan 21 2016, 3:42 PM