Page MenuHomePhabricator

Demonstrate collab threshold detection needs in new 'thresholds' system
Closed, ResolvedPublic

Description

In the ERI/RCFilters config, I'd like to be able to define the threshold for "maybe damaging" as:

max(
    min_threshold_with_min_precision( 0.15, "true" ),
    max_threshold_with_min_recall( 0.9, "true" )
)

Where min_threshold_with_min_precision( N, V ) means "the lowest threshold such that the precision for value V is >= N" and max_threshold_with_min_recall( N, V ) means "the highest threshold such that the recall for value V is >= N".

For "maybe bad faith" everything is flipped, so I would want:

min(
    max_threshold_with_min_precision( 0.15, "false" ),
    min_threshold_with_min_recall( 0.9, "false" )
)

This task is done when there's a document that explains how to do all of the above stuff using the new thresholds system.

Event Timeline

Catrope created this task.Aug 10 2017, 4:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 10 2017, 4:22 PM
Reedy added a subscriber: Reedy.Aug 10 2017, 7:00 PM

Can someone attach some tags to this task?

Reedy removed a subscriber: Reedy.Aug 10 2017, 7:00 PM
Restricted Application added a project: Scoring-platform-team. · View Herald TranscriptAug 10 2017, 7:13 PM
Halfak renamed this task from Allow threshold definitions based on minimum/maximum of a precision and a recall target to Demonstrate collab threshold detection needs in new 'thresholds' system.Aug 16 2017, 5:11 PM
Halfak triaged this task as Medium priority.
Halfak updated the task description. (Show Details)
Halfak added a project: Documentation.

I promise we'll actually have some proper documentation at some point :D

Halfak closed this task as Resolved.Sep 22 2017, 9:17 PM