Page MenuHomePhabricator

Move Revert-risk language agnostic model from staging to production
Closed, ResolvedPublic

Description

Background

The ML team has decided to limit the experimental namespace to ml-staging in order to prevent non-production-ready model servers from being deployed to Lift Wing (production). We'll come up with guidance on requirements for production-level models in T332711.

The revert-risk language-agnostic model was properly tested in T321594, but one thing is missing before it can be deployed to production/API gateway:

  • A model card

Once the model card is finished, to-dos for the ML team:

  • Create a revertrisk generic namespace to host the model and future models in the revert-risk family.
  • Deploy the model to production.
  • Enable the model in the API gateway.

After the above tasks are completed, T330148 should be unblocked.

Event Timeline

achou updated the task description. (Show Details)
achou added subscribers: diego, klausman.

The model has been moved to a new bucket:

aikochou@stat1005:~$ s3cmd -c /etc/s3cmd/cfg.d/ml-team.cfg ls s3://wmf-ml-models/revertrisk/language-agnostic/20221026144108/
2023-05-12 17:38       499465  s3://wmf-ml-models/revertrisk/language-agnostic/20221026144108/model.pkl

Change 920725 had a related patch set uploaded (by AikoChou; author: AikoChou):

[operations/deployment-charts@master] ml-services: change isvc name to revertrisk-language-agnostic

https://gerrit.wikimedia.org/r/920725

Change 920725 merged by Elukey:

[operations/deployment-charts@master] ml-services: change isvc name to revertrisk-language-agnostic

https://gerrit.wikimedia.org/r/920725

This model has been deployed to Lift Wing production. Note that the isvc/model has been renamed to revertrisk-language-agnostic

test the internal endpoint:

aikochou@deploy1002:~$ time curl "https://inference.svc.eqiad.wmnet:30443/v1/models/revertrisk-language-agnostic:predict" -d @input.json -H "Host: revertrisk-language-agnostic.revertrisk.wikimedia.org" --http1.1
{"lang":"en","rev_id":1096086751,"score":{"prediction":true,"probability":{"true":0.5144057273864746,"false":0.4855942726135254}}}
real	0m0.252s
user	0m0.007s
sys	0m0.007s

Next step is to config API gateway to enable the public endpoint.

Created https://api.wikimedia.org/wiki/API_reference/Service/Lift_Wing/Get_reverted_risk_language_agnostic_prediction, we need to add:

  1. The return object
  2. The model card (so far I found only the one for multilingual).

@diego Hiiiii! Do you have a model card that we can review? :)