During the upgrade of the revscoring model servers to Bullseye and Python 3.9 we had to upgrade their kserve dependency to 0.10 to use a more up-to-date version of numpy. The upgrade to kserve 0.10 is a lot more invasive than the other ones, since it involves dropping tornado in favor of fastapi.
The main problem in our codebase is that we raise tornado-specific exceptions when an error condition occurs, to return a specific HTTP response. With the drop of tornado we'll have to use something different, and at the moment for revscoring we opted for some nice built in exceptions: InvalidInput and InferenceError. Due to how fastapi works, it is possible to also return Json responses with a certain code, we'll see what's best when we work on this task.
Since we have common modules, like events.py, we decided to use RuntimeError exceptions in there, since it seemed a good compromise to make everything work (tornado and fastapi). Once we have everything on kserve 0.10 we should also update the common modules to more meaningful exceptions (if needed).
To recap:
- Upgrade nsfw to Kserve 0.10 - see T331416
- Upgrade revert risk to Kserve 0.10
- Upgrade outlink to Kserve 0.10
- Check common modules like events.py if they need better exceptions.