Page MenuHomePhabricator

Upgrade articletopic-outlink model servers from debian bullseye to bookworm
Closed, ResolvedPublic

Description

Use the latest bookworm image and update articletopic-outlink server.

Operating_system_upgrade_policy

Event Timeline

Testing bookworm for articletopic-outlink model

Create a Dockerfile with bookworm base image at ml-testing machine

echo "FROM docker-registry.wikimedia.org/bookworm:20250720 
RUN apt-get update && apt-get install -y build-essential python3-pip python3-dev python3-setuptools python3-venv git g++ wget vim curl
" > Dockerfile
2. Build the image:
docker build -t outlink:bookworm .
docker images # you should be able to see the built image with this command
3. Run the container:
docker run -it --entrypoint=/bin/bash outlink:bookworm
cd home
4. Clone the inference-services repo
cd into cloned repo
5. Use make to run both predictor and transformer
5.1. In first terminal run predictor:
make articletopic-outlink-predictor
5.2. Open second terminal and run transformer:
docker exec -it f096224eec6b /bin/bash 
cd into repo you cloned in step 4
make articletopic-outlink-transformer
5.3. Open third terminal and make a request:
docker exec -it f096224eec6b /bin/bash
Hit the API
root@409ab8e9d771:/# curl localhost:8080/v1/models/outlink-topic-model:predict -i -X POST -d '{"page_title": "Douglas_Adams", "lang": "en"}'
HTTP/1.1 200 OK
date: Fri, 25 Jul 2025 11:53:13 GMT
server: uvicorn
content-length: 270
content-type: application/json

{"prediction":{"article":"https://en.wikipedia.org/wiki/Douglas_Adams","results":[{"topic":"Culture.Media.Media*","score":0.6859594583511353},{"topic":"Culture.Biography.Biography*","score":0.5544804334640503},{"topic":"Culture.Literature","score":0.5156299471855164}]}}
Server Side Logs

Predictor server:

root@409ab8e9d771:~/inference-services# make articletopic-outlink-predictor
...........
Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.
2025-07-25 11:51:04.078 62 kserve INFO [model_server.py:register_model():346] Registering model: outlink-topic-model
2025-07-25 11:51:04.079 62 kserve INFO [model_server.py:setup_event_loop():264] Setting max asyncio worker threads as 12
2025-07-25 11:51:04.106 62 kserve INFO [server.py:start():142] Starting uvicorn with 1 workers
2025-07-25 11:51:04.122 uvicorn.error INFO:     Started server process [62]
2025-07-25 11:51:04.122 uvicorn.error INFO:     Waiting for application startup.
2025-07-25 11:51:04.129 62 kserve INFO [server.py:start():68] Starting gRPC server with 4 workers
2025-07-25 11:51:04.129 62 kserve INFO [server.py:start():69] Starting gRPC server on [::]:8081
2025-07-25 11:51:04.130 uvicorn.error INFO:     Application startup complete.
2025-07-25 11:51:04.130 uvicorn.error INFO:     Uvicorn running on http://0.0.0.0:8181 (Press CTRL+C to quit)
2025-07-25 11:53:14.154 kserve.trace requestId: N.A., preprocess_ms: 0.020265579, explain_ms: 0, predict_ms: 2.904891968, postprocess_ms: 0.020980835
2025-07-25 11:53:14.155 uvicorn.access INFO:     127.0.0.1:37666 62 - "POST /v1/models/outlink-topic-model%3Apredict HTTP/1.1" 200 OK
2025-07-25 11:53:14.156 kserve.trace kserve.io.kserve.protocol.rest.v1_endpoints.predict: 0.0066738128662109375 ['http_status:200', 'http_method:POST', 'time:wall']
2025-07-25 11:53:14.156 kserve.trace kserve.io.kserve.protocol.rest.v1_endpoints.predict: 0.006104999999998917 ['http_status:200', 'http_method:POST', 'time:cpu']

Transformer server

root@409ab8e9d771:~/inference-services# make articletopic-outlink-transformer
. my_venv/bin/activate && \
my_venv/bin/python src/models/outlink_topic_model/transformer/transformer.py \
--predictor_host="localhost:8181" --model_name="outlink-topic-model"
2025-07-25 11:52:21.127 89 kserve INFO [model_server.py:register_model():346] Registering model: outlink-topic-model
2025-07-25 11:52:21.128 89 kserve INFO [model_server.py:setup_event_loop():264] Setting max asyncio worker threads as 12
2025-07-25 11:52:21.150 89 kserve INFO [server.py:start():142] Starting uvicorn with 1 workers
2025-07-25 11:52:21.174 uvicorn.error INFO:     Started server process [89]
2025-07-25 11:52:21.174 uvicorn.error INFO:     Waiting for application startup.
2025-07-25 11:52:21.181 89 kserve INFO [server.py:start():68] Starting gRPC server with 4 workers
2025-07-25 11:52:21.181 89 kserve INFO [server.py:start():69] Starting gRPC server on [::]:8081
2025-07-25 11:52:21.182 uvicorn.error INFO:     Application startup complete.
2025-07-25 11:52:21.182 uvicorn.error INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
INFO:root:Opening a new Asyncio session for mwapi.
INFO:httpx:HTTP Request: POST http://localhost:8181/v1/models/outlink-topic-model:predict "HTTP/1.1 200 OK"
2025-07-25 11:53:14.159 kserve.trace requestId: N.A., preprocess_ms: 268.442869186, explain_ms: 0, predict_ms: 96.072673798, postprocess_ms: 0.020980835
2025-07-25 11:53:14.161 uvicorn.access INFO:     127.0.0.1:58390 89 - "POST /v1/models/outlink-topic-model%3Apredict HTTP/1.1" 200 OK
2025-07-25 11:53:14.162 kserve.trace kserve.io.kserve.protocol.rest.v1_endpoints.predict: 0.3684825897216797 ['http_status:200', 'http_method:POST', 'time:wall']
2025-07-25 11:53:14.162 kserve.trace kserve.io.kserve.protocol.rest.v1_endpoints.predict: 0.1522129999999997 ['http_status:200', 'http_method:POST', 'time:cpu']

Change #1172620 had a related patch set uploaded (by Gkyziridis; author: Gkyziridis):

[machinelearning/liftwing/inference-services@main] articletopic-outlink-model: Update base image from bullseye to the latest bookworm image.

https://gerrit.wikimedia.org/r/1172620

Change #1172620 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] articletopic-outlink-model: Update base image from bullseye to the latest bookworm image.

https://gerrit.wikimedia.org/r/1172620

Change #1174410 had a related patch set uploaded (by Gkyziridis; author: Gkyziridis):

[operations/deployment-charts@master] ml-services: Deploy latest images for articletopic-outlink-model on staging.

https://gerrit.wikimedia.org/r/1174410

Change #1174410 merged by jenkins-bot:

[operations/deployment-charts@master] ml-services: Deploy latest images for articletopic-outlink-model on staging.

https://gerrit.wikimedia.org/r/1174410

Change #1175478 had a related patch set uploaded (by Gkyziridis; author: Gkyziridis):

[operations/deployment-charts@master] ml-services: Deploy latest images for articletopic-outlink-model on production.

https://gerrit.wikimedia.org/r/1175478

Change #1175478 merged by jenkins-bot:

[operations/deployment-charts@master] ml-services: Deploy latest images for articletopic-outlink-model on production.

https://gerrit.wikimedia.org/r/1175478