Just a few hours before this breakage, we discussed the idea of writing a browser test that checks if the ORES filters are available on Special:RC on every production wiki that's supposed to have them. That would help with most historical cases, but not this one though; for that, @Etonkovidova proposes adding a browser test that checks that the "very likely good" filter shows more than 0 results.

As for unit tests, they did catch T182936: [wmf.12 -regression] ORES filters do not work ! But they were updated (to the wrong values) by the same commit.

Yeah, I've been thinking about how we can write proper tests for this. It's complex due to the nature of the hook, I have been thinking about writing tests that actually have to access database. One good case is the API hook handlers we have. i.e. We write more of integration test than unit test for the extension.

Change 400623 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/extensions/ORES@master] Integration tests for API

https://gerrit.wikimedia.org/r/400623

Change 400623 merged by jenkins-bot:
[mediawiki/extensions/ORES@master] Integration tests for API

https://gerrit.wikimedia.org/r/400623