Retrain damaging/goodfaith models for huwiki
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Halfak
	Jul 15 2019, 4:38 PM

Description

Train a new model using the new labeling campaign for huwiki. https://github.com/wikimedia/editquality/pull/195/files

Related Objects
Search...

Status	Assigned	Task
Open	None	T223892 [EPIC] Support Hungarian Wikipedia editor retention project
Resolved	Bencemac	T210224 Revert FlaggedRevs changes on the Hungarian Wikipedia
Resolved	Tgr	T209224 Analyze effect of huwiki FlaggedRevs configuration change on problematic edits and new user retention
Resolved	• ACraze	T229848 ORES deployment, Early August 2019
Resolved	Tgr	T230031 Update ORES filter thresholds for huwiki
Resolved	• ACraze	T228078 Retrain damaging/goodfaith models for huwiki
Resolved	Halfak	T223882 Re-label huwiki damaging and badfaith edits
Open	None	T223899 Information about finished campaigns should be accessible in Wikilabels

Event Timeline

Halfak created this task.Jul 15 2019, 4:38 PM

Restricted Application added a project: artificial-intelligence. · View Herald TranscriptJul 15 2019, 4:38 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Halfak added a subtask: T223882: Re-label huwiki damaging and badfaith edits.Jul 15 2019, 4:39 PM

Halfak added a parent task: T209224: Analyze effect of huwiki FlaggedRevs configuration change on problematic edits and new user retention.

Halfak closed subtask T223882: Re-label huwiki damaging and badfaith edits as Resolved.Jul 16 2019, 7:37 PM

Tgr added a project: User-Tgr.Jul 19 2019, 5:35 PM

Tgr mentioned this in T223900: Create ORES dataset for huwiki edits in the last two years or so.Jul 20 2019, 8:48 AM

@Halfak the models have been retrained, let me know your thoughts about the tuning reports
https://github.com/wikimedia/editquality/pull/207

I just reviewed. Just a minor note on file naming. The fitness statistics look good to me. I think we're running into issues with the fragility of our AUC measures. If you look at the ROC AUC and the PR AUC of the target classes (true for damaging, false for badfaith) then it looks like we're getting a nice improvement here.

• ACraze moved this task from Review to Pending deployment on the Machine-Learning-Team (Active Tasks) board.Jul 25 2019, 12:31 AM

Halfak added a parent task: T229848: ORES deployment, Early August 2019.Aug 5 2019, 4:25 PM

Samat mentioned this in T209224: Analyze effect of huwiki FlaggedRevs configuration change on problematic edits and new user retention.Aug 5 2019, 11:21 PM

• ACraze moved this task from Pending deployment to Completed on the Machine-Learning-Team (Active Tasks) board.Aug 6 2019, 6:01 PM

Halfak added a parent task: T230031: Update ORES filter thresholds for huwiki.Aug 7 2019, 3:51 PM

Halfak closed this task as Resolved.Aug 7 2019, 3:53 PM

Samat awarded a token.Aug 7 2019, 5:47 PM