Page MenuHomePhabricator

Demonstrate collab threshold detection needs in new 'thresholds' system
Closed, ResolvedPublic

Description

In the ERI/RCFilters config, I'd like to be able to define the threshold for "maybe damaging" as:

max(
    min_threshold_with_min_precision( 0.15, "true" ),
    max_threshold_with_min_recall( 0.9, "true" )
)

Where min_threshold_with_min_precision( N, V ) means "the lowest threshold such that the precision for value V is >= N" and max_threshold_with_min_recall( N, V ) means "the highest threshold such that the recall for value V is >= N".

For "maybe bad faith" everything is flipped, so I would want:

min(
    max_threshold_with_min_precision( 0.15, "false" ),
    min_threshold_with_min_recall( 0.9, "false" )
)

This task is done when there's a document that explains how to do all of the above stuff using the new thresholds system.

Event Timeline

Can someone attach some tags to this task?

Halfak renamed this task from Allow threshold definitions based on minimum/maximum of a precision and a recall target to Demonstrate collab threshold detection needs in new 'thresholds' system.Aug 16 2017, 5:11 PM
Halfak triaged this task as Medium priority.
Halfak updated the task description. (Show Details)
Halfak moved this task from Unsorted to Blocked on community input on the Machine-Learning-Team board.
Halfak added a project: Documentation.

I promise we'll actually have some proper documentation at some point :D