Thanks, that is what I am proposing as well. @achou, how feasible do you think this is from your side? It would involve accepting a POST with all the features (https://gitlab.wikimedia.org/repos/research/knowledge_integrity/-/blob/main/knowledge_integrity/featureset.py?ref_type=heads) needed.

@kostajh @XiaoXiao-WMF thanks for tagging. Sorry I was unaware of the discussion here. The ML team is currently in the middle of quarterly planning. I will bring up the proposal during our planning and get back to you shortly!

I built a RRML image locally using the Pytorch 2.2.x base image from T360638.

We have deployed the new RRLA model server to production.

This task is complete. I've created T361881 to follow up on the above test results issue.

FYI @MunizaA :)

In T360423#9690484, @achou wrote:

The new RRLA model server featuring KI v.0.6 has been deployed to ML-staging. I used wrk to conduct load testing and compare the performance between the old and new versions. The results for the previous version are under P59447, and the results for the new version are under P59464. From these results, it's clear that the new KI version does not affect the performance metrics, such as average latency and RPS.

The new RRLA model server featuring KI v.0.6 has been deployed to ML-staging. I used wrk to conduct load testing and compare the performance between the old and new versions. The results for the previous version are under P59447, and the results for the new version are under P59464. From these results, it's clear that the new KI version does not affect the performance metrics, such as average latency and RPS.

I repost what I previously wrote here as the issue is more related to deployment.

This task is complete. Check out these examples:

This task is complete. Check out these examples of new error messages:

$ curl "https://inference-staging.svc.codfw.wmnet:30443/v1/models/revertrisk-language-agnostic:predict" -d '{"rev_id": 15925124, "lang": "ro"}' -H "Host: revertrisk-language-agnostic.revertrisk.wikimedia.org" --http1.1 -k |  jq '.'
{
  "detail": "Could not make prediction for revision 15925124 (ro). Reason: revision_missing"
}

@kevinbazira posed a question - how can end users switch between batch and non-batch requests?

@isarantopoulos do you remember the config values in locust.conf when you ran the revertrisk tests? I can't reproduce the result in revertrisk_stats.csv. I haven't deployed RRLA to staging yet, so it's the same model you tested.

It would be nice to wait for an additional patch (improving error messages) to be merged.

In T360177#9636947, @kevinbazira wrote:

Running into the error below which is caused by a missing events module. This module is used to generate and send a topic prediction event to EventGate. Turns out this module is in python/events.py and the model-server can't locate it because it is not running like a python module.

Traceback (most recent call last):
  File "/home/inference-services/outlink-topic-model/model-server/model.py", line 6, in <module>
    import events
ModuleNotFoundError: No module named 'events'
make[1]: *** [Makefile:97: run-server] Error 1
make[1]: Leaving directory '/home/inference-services'
make: *** [Makefile:76: articletopic-outlink] Error 2

The PR for pydantic v2 in kserve has been merged! We can use this commit https://github.com/kserve/kserve/commit/426fe21da0612ea6ef4a116b5114270313e02bbb to test the RRLA model-server :)

Knowledge Integrity v0.6.0 improved error representations by introducing an Error data class and different error codes for various situations when fetching MediaWiki API for revisions. (See https://gitlab.wikimedia.org/repos/research/knowledge_integrity/-/blob/main/knowledge_integrity/mediawiki.py?ref_type=heads#L14-26)

Following a discussion with Ilias, we will keep an eye on the progress of https://github.com/kserve/kserve/pull/3374. Once the PR is merged, we will use the pre-release version for testing.

The latest changes to requirements.txt still resulted in a failed docker image build. Therefore, the torch version conflict between the knowledge integrity and inference services repo was not the cause of the failure.