The LiftWing model-server repo has a Makefile that makes building and running of model-servers locally much easier.
In this task, we will update the existing Makefile to support building and running the articletopic-outlink model-server.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | achou | T352689 Add a script for running the Revert Risk model server locally | |||
| Resolved | kevinbazira | T360177 Support building and running of articletopic-outlink model-server via Makefile |
Event Timeline
Comment Actions
While building the articletopic-outlink model-server locally, the error below is thrown. We encountered a similar error in T357382#9536821 and resolved it by adding the wheel package to the requirements.txt before installing fasttext.
Collecting fasttext==0.9.2 (from -r outlink-topic-model/model-server/requirements.txt (line 30))
Using cached fasttext-0.9.2.tar.gz (68 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [25 lines of output]
/home/inference-services/my_venv/bin/python3: No module named pip
Traceback (most recent call last):
File "<string>", line 38, in __init__
ModuleNotFoundError: No module named 'pybind11'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/inference-services/my_venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/inference-services/my_venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/inference-services/my_venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-bqovm3xs/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-bqovm3xs/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-bqovm3xs/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 487, in run_setup
super().run_setup(setup_script=setup_script)
File "/tmp/pip-build-env-bqovm3xs/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 72, in <module>
File "<string>", line 41, in __init__
RuntimeError: pybind11 install failed.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.Comment Actions
Currently, the method that loads a model has a hardcoded model path. When we set the path through an environmental variable, the error below is thrown. To resolve this, we need to refactor the model server so that it can accept the model path through an environment variable, similar to how other model servers operate.
Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.
Traceback (most recent call last):
File "/home/inference-services/outlink-topic-model/model-server/model.py", line 104, in <module>
model = OutlinksTopicModel("outlink-topic-model")
File "/home/inference-services/outlink-topic-model/model-server/model.py", line 27, in __init__
self.load()
File "/home/inference-services/outlink-topic-model/model-server/model.py", line 45, in load
self.model = fasttext.load_model("/mnt/models/model.bin")
File "/home/inference-services/my_venv/lib/python3.9/site-packages/fasttext/FastText.py", line 441, in load_model
return _FastText(model_path=path)
File "/home/inference-services/my_venv/lib/python3.9/site-packages/fasttext/FastText.py", line 98, in __init__
self.f.loadModel(model_path)
ValueError: /mnt/models/model.bin cannot be opened for loading!
make[1]: *** [Makefile:97: run-server] Error 1
make[1]: Leaving directory '/home/inference-services'
make: *** [Makefile:76: articletopic-outlink] Error 2Comment Actions
Change rETDA10109799aa72 had a related patch set uploaded (by Kevin Bazira; author: Kevin Bazira):
[machinelearning/liftwing/inference-services@main] articletopic-outlink: load model path from environment variable
Comment Actions
Change rETDA10109799aa72 merged by jenkins-bot:
[machinelearning/liftwing/inference-services@main] articletopic-outlink: load model path from environment variable
Comment Actions
Running into the error below which is caused by a missing events module. This module is used to generate and send a topic prediction event to EventGate. Turns out this module is in python/events.py and the model-server can't locate it because it is not running like a python module.
Traceback (most recent call last):
File "/home/inference-services/outlink-topic-model/model-server/model.py", line 6, in <module>
import events
ModuleNotFoundError: No module named 'events'
make[1]: *** [Makefile:97: run-server] Error 1
make[1]: Leaving directory '/home/inference-services'
make: *** [Makefile:76: articletopic-outlink] Error 2Comment Actions
The problem could be resolved by changing the import events to import python.events, which is the same idea when we use functions in python.preprocess_utils in other model servers. In the makefile, we’ve added the inference-services repo to the PYTHONPATH, so the “python” module could be found.
However, I have a broader question about this task. It's unclear to me how to run the outlink-topic model server locally. This model server operates differently from others because it comprises a transformer and a model server that need to communicate with each other. I’m unsure if there's a simpler method than testing them using Docker containers (see https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/KServe#Example_3_-_Testing_outlink-topic-model_(two_containers)). The local configuration could be complex and I will be hesitant if it is worth investing time on it. Maybe I’m wrong, do you have any insights on this?
Comment Actions
Change 1012647 had a related patch set uploaded (by Kevin Bazira; author: Kevin Bazira):
[machinelearning/liftwing/inference-services@main] articletopic-outlink: run model server as python module
Comment Actions
@achou I figured out a way to run both the transformer and predictor locally in the same container. All we have to do is change the predictor's port to 8181 so that the transformer can use port 8080 and --predictor_host="localhost:8181". I am working on a patch that has a README file with instructions on how to run the articletopic-outlink model-server locally.
Comment Actions
Change 1012647 merged by jenkins-bot:
[machinelearning/liftwing/inference-services@main] articletopic-outlink: run model server as python module
Comment Actions
Change 1012663 had a related patch set uploaded (by Kevin Bazira; author: Kevin Bazira):
[machinelearning/liftwing/inference-services@main] Makefile: add support for articletopic-outlink
Comment Actions
Change 1012663 merged by Kevin Bazira:
[machinelearning/liftwing/inference-services@main] Makefile: add support for articletopic-outlink
Comment Actions
Support for building and running the articletopic-outlink model-server using the Makefile was added and it can be tested using:
# first terminal
$ make articletopic-outlink-predictor
# second terminal
$ make articletopic-outlink-transformer
# third terminal
$ curl localhost:8080/v1/models/outlink-topic-model:predict -i -X POST -d '{"page_title": "Douglas_Adams", "lang": "en"}'
$ MODEL_TYPE=articletopic make clean