The SD team has done some analysis on the results from user testing of image recommendations, see https://phabricator.wikimedia.org/T272710#7119669

We think this gives us a pretty good feel for creating a confidence score for mediasearch results

First we need to do T283837, and then we suggest that the API returns a low, medium or high confidence_score for the mediasearch results based on the following:

• if the score returned from the call to mediasearch is <64 then` confidence_score` should be set to low
• if the score returned from the call to mediasearch is >=64 and <89 then` confidence_score` should be set to medium
• if the score returned from the call to mediasearch is >89 then` confidence_score` should be set to high

According to our tests this gives us a similar likelihood-of-an-image-being-good as the different confidence_score values from the IMA