Page MenuHomePhabricator

Include model version in ORES response structure
Closed, ResolvedPublic

Description

Whenever we query a model for revision scores, the response should include both the model name and a version number under that.

Halfak also floated the idea of perhaps a second API which provides metadata about the model.

Event Timeline

awight created this task.Sep 17 2015, 10:55 PM
awight raised the priority of this task from to Normal.
awight updated the task description. (Show Details)
awight added subscribers: gerritbot, awight, Aklapper, Legoktm.
Halfak added a subscriber: Halfak.Sep 17 2015, 11:10 PM

For the score response, something like:

{
  "10004": {
    "prediction": true,
    "probability": {
      "false": 0.26805633607044865,
      "true": 0.7319436639295512
    }
  }
}

Would become:

{
  "10004": {
    "version": "0.3.1"
    "prediction": true,
    "probability": {
      "false": 0.26805633607044865,
      "true": 0.7319436639295512
    }
  }
}

This would mean that the version gets duplicated for every score that is returned. I don't think that bandwidth will be much of an issue, but what about other metadata? We probably don't want to add a ton if it here.

Oh also. The model name is part of the request. When you request scores for multiple models, then the model name is part of the response.

E.g.: http://ores.wmflabs.org/scores/enwiki/?models=reverted|wp10&revids=422342

{
  "422342": {
    "reverted": {
      "prediction": true,
      "probability": {
        "false": 0.23331789339036138,
        "true": 0.7666821066096383
      }
    },
    "wp10": {
      "prediction": "Start",
      "probability": {
        "B": 0.09061797203000406,
        "C": 0.01802754795982005,
        "FA": 0.0001330671989354624,
        "GA": 0.000124750499001996,
        "Start": 0.5318895485569294,
        "Stub": 0.35920711375530856
      }
    }
  }
}

Maybe we should only support one format and always return the model name as a key.

I don't need any other metadata at the moment. It's probably wise to include an API version in the request, however.

DarTar set Security to None.
Halfak claimed this task.Oct 2 2015, 5:41 PM
Halfak moved this task from Active to Backlog on the Scoring-platform-team (Current) board.
Halfak added a comment.EditedOct 2 2015, 9:45 PM

Please review https://github.com/wiki-ai/ores/pull/97

I opted for not including the info in every score. It seems like a periodic request for model info would be appropriate.

awight closed this task as Resolved.Oct 17 2015, 10:22 AM