Maniphest T133698

[Spike] Explore using PR-AUC to score when tuning
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Halfak
	Apr 26 2016, 3:16 PM

Tags

Referenced Files

None

Subscribers

Description

http://scikit-learn.org/stable/modules/model_evaluation.html

"average_precision" == PR-AUC

This score corresponds to the area under the precision-recall curve.

This task is complete when we have reviewed the predictions of new modeling strategies (that score better via PR-AUC) for *intuitive* fitness.

Event Timeline

Halfak created this task.Apr 26 2016, 3:16 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 26 2016, 3:16 PM

Halfak moved this task from Unsorted to New development on the Machine-Learning-Team board.May 2 2016, 4:53 PM

@Ladsgroup, this one should be easy to pick up and the results will likely be very interesting :)

I'm running some tests on this right now.

https://github.com/wiki-ai/editquality/pull/31/files

Ladsgroup edited projects, added Machine-Learning-Team (Active Tasks); removed Machine-Learning-Team.May 14 2016, 2:58 PM

Ladsgroup moved this task from Parked to Backlog on the Machine-Learning-Team (Active Tasks) board.

Ladsgroup moved this task from Backlog to Review on the Machine-Learning-Team (Active Tasks) board.May 16 2016, 8:40 AM

Halfak updated the task description. (Show Details)May 16 2016, 3:44 PM

Halfak assigned this task to Ladsgroup.May 16 2016, 3:47 PM

He7d3r subscribed.May 20 2016, 2:05 AM

https://meta.wikimedia.org/wiki/Research_talk:Revision_scoring_as_a_service/Work_log/2016-05-23

Ladsgroup closed this task as Resolved.Jun 1 2016, 9:32 AM

• Phabricator_maintenance added a project: User-Ladsgroup.Aug 12 2016, 8:09 PM