Whenever we query a model for revision scores, the response should include both the model name and a version number under that.
Halfak also floated the idea of perhaps a second API which provides metadata about the model.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Invalid | Halfak | T112319 Revscoring integration as a beta feature | |||
| Resolved | None | T120183 Improve mwext-ORES integration with RecentChanges | |||
| Resolved | None | T120923 Ask for consensus to enable and deploy ORES extension to production | |||
| Resolved | awight | T112856 Write prototype MediaWiki extension to surface ORES scores | |||
| Resolved | Halfak | T112995 Include model version in ORES response structure |
Event Timeline
Comment Actions
For the score response, something like:
{
"10004": {
"prediction": true,
"probability": {
"false": 0.26805633607044865,
"true": 0.7319436639295512
}
}
}Would become:
{
"10004": {
"version": "0.3.1"
"prediction": true,
"probability": {
"false": 0.26805633607044865,
"true": 0.7319436639295512
}
}
}This would mean that the version gets duplicated for every score that is returned. I don't think that bandwidth will be much of an issue, but what about other metadata? We probably don't want to add a ton if it here.
Comment Actions
Oh also. The model name is part of the request. When you request scores for multiple models, then the model name is part of the response.
E.g.: http://ores.wmflabs.org/scores/enwiki/?models=reverted|wp10&revids=422342
{
"422342": {
"reverted": {
"prediction": true,
"probability": {
"false": 0.23331789339036138,
"true": 0.7666821066096383
}
},
"wp10": {
"prediction": "Start",
"probability": {
"B": 0.09061797203000406,
"C": 0.01802754795982005,
"FA": 0.0001330671989354624,
"GA": 0.000124750499001996,
"Start": 0.5318895485569294,
"Stub": 0.35920711375530856
}
}
}
}Maybe we should only support one format and always return the model name as a key.
Comment Actions
I don't need any other metadata at the moment. It's probably wise to include an API version in the request, however.
Comment Actions
Please review https://github.com/wiki-ai/ores/pull/97
I opted for not including the info in every score. It seems like a periodic request for model info would be appropriate.