⚓ T148038 [Epic] Build draft quality model (spam, vandalism, attack, or OK)

Status	Assigned	Task
Resolved	Halfak	T148038 [Epic] Build draft quality model (spam, vandalism, attack, or OK)
Resolved	Halfak	T135644 Generate spam and vandalism new page creation dataset
Resolved	Halfak	T148580 Build feature set for draft quality model
Resolved	Halfak	T148581 Extract features for deleted page (draft quality model)
Resolved	Halfak	T150307 Create manually vetted dataset of spam/vandalism/attack pages
Resolved	Halfak	T151819 Analyze differentiation of FA, Spam, Vandalism, and Attack models/sentences.
Resolved	Halfak	T148037 Generate PCFG sentence models
Resolved	Halfak	T148034 Sentence bank for vandalism
Resolved	Halfak	T148035 Sentence bank for personal attacks
Resolved	Halfak	T148033 Sentence bank for Featured Articles
Resolved	Halfak	T148032 Sentence bank for spam
Resolved	Halfak	T148867 Implement sentences datascources

Halfak created this task.Oct 13 2016, 2:08 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 13 2016, 2:08 PM

Halfak added a subtask: T135644: Generate spam and vandalism new page creation dataset.Oct 13 2016, 2:11 PM

Halfak updated the task description. (Show Details)Oct 13 2016, 2:56 PM

Halfak triaged this task as High priority.Oct 13 2016, 3:05 PM

Halfak moved this task from Unsorted to Ideas on the Machine-Learning-Team board.

Halfak moved this task from Ideas to Research & analysis on the Machine-Learning-Team board.

Halfak added a project: Research-and-Data-2016-17-Q2.Oct 18 2016, 8:05 PM

Halfak created subtask T148580: Build feature set for draft quality model.

Halfak created subtask T148581: Extract features for deleted page (draft quality model).

Halfak edited projects, added Machine-Learning-Team (Active Tasks); removed Machine-Learning-Team.Nov 10 2016, 4:03 PM

Halfak added a subtask: T150307: Create manually vetted dataset of spam/vandalism/attack pages.

Halfak moved this task from Parked to Non-Epic on the Machine-Learning-Team (Active Tasks) board.Nov 10 2016, 5:28 PM

Halfak added a project: draftquality-modeling.Nov 28 2016, 9:36 PM

Halfak added a subtask: T151819: Analyze differentiation of FA, Spam, Vandalism, and Attack models/sentences. .Nov 28 2016, 9:45 PM

Halfak closed subtask T150307: Create manually vetted dataset of spam/vandalism/attack pages as Resolved.Nov 30 2016, 9:05 PM

Halfak renamed this task from Build draft quality model (spam, vandalism, attack, or OK) to [Epic] Build draft quality model (spam, vandalism, attack, or OK).Jan 17 2017, 9:02 PM

Halfak moved this task from Non-Epic to Completed on the Machine-Learning-Team (Active Tasks) board.Jan 17 2017, 10:12 PM

FYI: https://github.com/wiki-ai/draftquality/blob/master/models/enwiki.draft_quality.gradient_boosting.model

ScikitLearnClassifier
 - type: GradientBoosting
 - params: init=null, scale=false, max_leaf_nodes=null, center=false, warm_start=false, presort="auto", random_state=null, subsample=1.0, max_features="log2", max_depth=7, balanced_sample_weight=false, min_samples_split=2, n_estimators=700, loss="deviance", balanced_sample=false, min_samples_leaf=1, verbose=0, learning_rate=0.01, min_weight_fraction_leaf=0.0
 - version: None
 - trained: 2017-01-17T21:19:14.173403

Table:
	             ~OK    ~attack    ~spam    ~vandalism
	---------  -----  ---------  -------  ------------
	OK         24848          5     1078           326
	attack        82        249      499          1229
	spam         616         14    15915          1154
	vandalism    654        209     1813          3827

Accuracy: 0.854
ROC-AUC:
	-----------  -----
	'OK'         0.983
	'attack'     0.93
	'spam'       0.97
	'vandalism'  0.923
	-----------  -----

F1:
	---------  -----
	vandalism  0.587
	attack     0.197
	OK         0.947
	spam       0.86
	---------  -----

Halfak claimed this task.Jan 19 2017, 3:37 PM

Halfak closed this task as Resolved.Feb 7 2017, 8:31 PM

Halfak closed subtask T151819: Analyze differentiation of FA, Spam, Vandalism, and Attack models/sentences. as Resolved.

Halfak closed subtask T148581: Extract features for deleted page (draft quality model) as Resolved.

Halfak closed subtask T148580: Build feature set for draft quality model as Resolved.