As we worked on T406179: Q2 FY2025-26 Goal: Host Wikidata Revert Risk model on LiftWing, we conducted locust load tests on the revertrisk-wikidata inference service staging endpoint. These tests ran for 120 seconds with 2 users, each sending requests at intervals between 1 and 5 seconds, using sample Wikidata revision IDs got from the Research team's expert_sample.csv.

The results showed an average response time of 583ms, with a 0% failure rate over 65 requests:

Based on the tip in this task's description, our understanding is that WME is going to further evaluate this service's scale and latency:

In T409388, @prabhat wrote:

Tip: you can reuse existing liftwing scale test under experiments repos

Once you have completed your load tests, please share the results with us so we can optimize where needed.

Let us know if you need any additional information or support from our side.

Hey @kevinbazira thank very much for running the loading tests for Revert-Risk wikidata.
I think we should change a little bit the configuration in order to simulate a more realistic scenario close to reality.
We also need to run heavier tests spawning more users in order to check our API's capacity and capability to handle maximum RPS.
I ran three different locust tests with heavier configuration, you can see the results in the following phab paste:

Ideas

• We experiment with more resources on isvc
• Configure AutoScaling
• We could probably use Kserve Batcher?
• Digg into the model's server logic and make it faster.

The revertrisk-wikidata inference service production endpoint uses similar scaling configs that other revertrisk inference-services use: https://github.com/wikimedia/operations-deployment-charts/blob/8412fc655d3b1e10b38cf0c954d910b820e93a05/helmfile.d/ml-services/revertrisk/values.yaml#L145-L150

IMO the prod endpoint should scale well unless results from the WME folks say otherwise.

Update

In T409388#11405117, @kevinbazira wrote:

The revertrisk-wikidata inference service production endpoint uses similar scaling configs that other revertrisk inference-services use: https://github.com/wikimedia/operations-deployment-charts/blob/8412fc655d3b1e10b38cf0c954d910b820e93a05/helmfile.d/ml-services/revertrisk/values.yaml#L145-L150

IMO the prod endpoint should scale well unless results from the WME folks say otherwise.

@Kevin you are right, the above locust tests in: https://phabricator.wikimedia.org/T409388#11400499 were targeting staging where autoscaling is not activated:

revertrisk-wikidata:
  predictor:
    image: "machinelearning-liftwing-inference-services-revertrisk-wikidata"
    image_version: "2025-11-17-105041-publish"
    custom_env:
      - name: MODEL_NAME
        value: "revertrisk-wikidata"
      - name: STORAGE_URI
        value: "s3://wmf-ml-models/revertrisk/wikidata/20251104121312/"
      - name: FORCE_HTTP
        value: "True"
    container:
      resources:
        limits:
          cpu: "2"
          memory: 4Gi
        requests:
          cpu: "2"
          memory: 4Gi

I rerun the tests using the configuration we are using on prod where the autoscaling is activated and we are using more resources:

revertrisk-wikidata:
  annotations:
    autoscaling.knative.dev/target: "3"
  predictor:
    config:
      minReplicas: 5
      maxReplicas: 15
    image: "machinelearning-liftwing-inference-services-revertrisk-wikidata"
    image_version: "2025-11-17-105041-publish"
    custom_env:
      - name: MODEL_NAME
        value: "revertrisk-wikidata"
      - name: STORAGE_URI
        value: "s3://wmf-ml-models/revertrisk/wikidata/20251104121312/"
      - name: FORCE_HTTP
        value: "True"
    container:
      resources:
        limits:
          cpu: "2"
          memory: 4Gi
        requests:
          cpu: "2"
          memory: 4Gi

Here are the results from the final loading tests using the prod configuration with autoscaling activated (above config).
Results:

@prabhat, has the WME team had a chance to run scale and latency tests on the revertrisk-wikidata inference service? Does this service meet your performance requirements?

If you run into issues or if the service does not meet your performance requirements, please let us know so we can further optimize it. Thanks!