Page MenuHomePhabricator

Allow setting huggingfaceserver cmd args from deployment-charts
Open, Needs TriagePublic2 Estimated Story Points

Description

As an engineer I want to be able to define command line arguments directly on the deployment-charts repository, so that I can:

  • configure huggingfaceserver using all the available options
  • update deployments with a single patch in deployment-charts repo instead of changing them first in inference-services
  • be able to experiment quicker in ml-staging experimental namespace where we can directly change the isvc.

At the moment we cannot set any of the arguments available in the huggingfaceserver as the docker entrypoint is a fixed one defined in our blubber/docker image.

The solution to this is to

  • either update the entrypoint.sh script and expose all the available options as environment variables (when each of them are defined)
  • or ideally just override the docker entrypoint command in the kserve chart as creating a bash script with so many options seems more like a hack and would require us to manually update the arguments in the event of a kserve/huggingface upgrade.

Event Timeline

Change #1036297 had a related patch set uploaded (by Ilias Sarantopoulos; author: Ilias Sarantopoulos):

[operations/deployment-charts@master] ml-services: set command for hf image and remove nllb

https://gerrit.wikimedia.org/r/1036297

klausman set the point value for this task to 2.

Change #1036297 merged by jenkins-bot:

[operations/deployment-charts@master] ml-services: set command for hf image and remove nllb

https://gerrit.wikimedia.org/r/1036297