I have the python code listed below, as well as 2 commits from rowiki for which I try to obtin revertrisk scores:
- 15891878 is very large and the multilingual model returns 504 from python and curl - presumably due to the size (model limitation)
- 15891604 is not very special, but fails from python (reproducible), although it works from curl.
- both pass for the language-agnostic model
Expected results: 15891604 should return a valid score using all methods and models. Ideally, the other commit should also return valid results if the model allows it.
Curl results:
$ curl https://api.wikimedia.org/service/lw/inference/v1/models/revertrisk-multilingual:predict -X POST -d '{"rev_id": 15891878, "lang": "ro"}' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 91 100 57 100 34 1 1 0:00:57 0:00:30 0:00:27 13{"httpCode":504,"httpReason":"upstream request timeout"} $ curl https://api.wikimedia.org/service/lw/inference/v1/models/revertrisk-language-agnostic:predict -X POST -d '{"rev_id": 15891878, "lang": "ro"}' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 242 100 208 100 34 361 59 --:--:-- --:--:-- --:--:-- 422{"model_name":"revertrisk-language-agnostic","model_version":"2","wiki_db":"rowiki","revision_id":15891878,"output":{"prediction":false,"probabilities":{"true":0.4380935728549957,"false":0.5619064271450043}}}
$ curl https://api.wikimedia.org/service/lw/inference/v1/models/revertrisk-language-agnostic:predict -X POST -d '{"rev_id": 15891604, "lang": "ro"}' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 241 100 207 100 34 464 76 --:--:-- --:--:-- --:--:-- 544{"model_name":"revertrisk-language-agnostic","model_version":"2","wiki_db":"rowiki","revision_id":15891604,"output":{"prediction":true,"probabilities":{"true":0.8956175446510315,"false":0.1043824553489685}}} $ curl https://api.wikimedia.org/service/lw/inference/v1/models/revertrisk-multilingual:predict -X POST -d '{"rev_id": 15891604, "lang": "ro"}' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 236 100 202 100 34 222 37 --:--:-- --:--:-- --:--:-- 259{"model_name":"revertrisk-multilingual","model_version":"4","wiki_db":"rowiki","revision_id":15891604,"output":{"prediction":true,"probabilities":{"true":0.6360696956584585,"false":0.3639303043415415}}}
Python code:
def get_result(self, lang: str, revid: int) -> (float, bool): score = None prediction = False headers = {'Content-Type': 'application/json', 'User-Agent': 'PatrocleBot (patroclebot@strainu.ro)' } data = {"lang": lang, "rev_id": revid} r = requests.post(url=self.url, headers=headers, data=json.dumps(data)) if r.status_code != 200: raise ValueError(f"Obtaining the {self.name} score for revision {revid} failed with code {r.status_code}") try: resp = r.json() score = resp["output"]["probabilities"]["true"] prediction = resp["output"]["prediction"] except Exception as e: raise ValueError(f"Obtaining the {self.name} score for revision {revid} failed with error {e}. URL was {url}") finally: r.close() return score, prediction