See https://phabricator.wikimedia.org/phame/post/view/68/more_better_model_information_and_threshold_optimizations/
This is a breaking change about how "thresholds" are reported. Currently you can test against ores-misc.wmflabs.org. Soon we'll deploy to ores-beta.wmflabs.org too.
The ORES extension currently resolves thresholds by calling the model_info API and interpreting its output (see [[https://github.com/wikimedia/mediawiki-extensions-ORES/blob/0a76c7dd0c1c008e436d91f307a7a98e4e4d6d26/includes/Stats.php#L77|Stats.php from line 77 onwards]]). Currently, the thresholds to look up are specified using the names from the old-style model_info output; see [[https://github.com/wikimedia/mediawiki-extensions-ORES/blob/0a76c7dd0c1c008e436d91f307a7a98e4e4d6d26/extension.json#L155|the default value of wgOresFiltersThresholds]] and how it's [[https://github.com/wikimedia/operations-mediawiki-config/blob/ce0313293133b0e5ca2c79d951d85bad31a055f1/wmf-config/InitialiseSettings.php#L18816|overridden in InitialiseSettings.php]] for specific wikis.
I think the best way to go about this would be to make the `wgOresFiltersThresholds` config format more structured, so that we have something like `'recall' => 0.9` instead of `'filter_rate_at_recall(min_recall=0.9)'`. That would give us enough information to generate requests against both the old and new APIs, so then we could either have a config var that tells us which to use, or code that tries one and falls back on the other (or in some other way discovers which version is used by the ORES API it's talking to).
In the future, I'd like for a threshold to be defined as the min or max of multiple lookups as explained in {T173019}. That can be done separately from this though, and is probably best off waiting until the new API is deployed and compatibility code for the old one is removed, but in designing the format of our config var we might want to keep that in mind.
Strawman config proposal:
```lang=json
"wgOresFiltersThresholds": {
"damaging": {
"likelygood": {"min": 0, "max": { "precision": 0.995 } },
"maybebad": { "min": { "recall": 0.9 }, "max": 1 },
....
},
"goodfaith": {
...
}
}
```
Then maybe for multi-lookup stuff we can express that as `"maybebad": { "min": [ { "recall": 0.9 }, { "precision": 0.15 } ], "max": 1" }`, or maybe `"maybebad": { "min": { "recall": 0.9, "precision": "0.15" } }`? (Whether the min or max of those should be used could be inferred: if specified for min, use the max; if specified for max, use the min. Always use the narrower of the two intervals.)