Page MenuHomePhabricator

[Epic] Build draft quality model (spam, vandalism, attack, or OK)
Closed, ResolvedPublic

Event Timeline

Halfak created this task.Oct 13 2016, 2:08 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 13 2016, 2:08 PM
Halfak updated the task description. (Show Details)Oct 13 2016, 2:56 PM
Halfak triaged this task as High priority.Oct 13 2016, 3:05 PM
Halfak moved this task from Untriaged to Ideas on the Scoring-platform-team board.
Halfak moved this task from Ideas to Research & analysis on the Scoring-platform-team board.
Halfak renamed this task from Build draft quality model (spam, vandalism, attack, or OK) to [Epic] Build draft quality model (spam, vandalism, attack, or OK).Jan 17 2017, 9:02 PM

FYI: https://github.com/wiki-ai/draftquality/blob/master/models/enwiki.draft_quality.gradient_boosting.model

ScikitLearnClassifier
 - type: GradientBoosting
 - params: init=null, scale=false, max_leaf_nodes=null, center=false, warm_start=false, presort="auto", random_state=null, subsample=1.0, max_features="log2", max_depth=7, balanced_sample_weight=false, min_samples_split=2, n_estimators=700, loss="deviance", balanced_sample=false, min_samples_leaf=1, verbose=0, learning_rate=0.01, min_weight_fraction_leaf=0.0
 - version: None
 - trained: 2017-01-17T21:19:14.173403

Table:
	             ~OK    ~attack    ~spam    ~vandalism
	---------  -----  ---------  -------  ------------
	OK         24848          5     1078           326
	attack        82        249      499          1229
	spam         616         14    15915          1154
	vandalism    654        209     1813          3827

Accuracy: 0.854
ROC-AUC:
	-----------  -----
	'OK'         0.983
	'attack'     0.93
	'spam'       0.97
	'vandalism'  0.923
	-----------  -----

F1:
	---------  -----
	vandalism  0.587
	attack     0.197
	OK         0.947
	spam       0.86
	---------  -----
Halfak claimed this task.Jan 19 2017, 3:37 PM